javascript - How can Selenium batch many isElementDisplayed calls?

Question

Selenium has a convenient mechanism for pulling dozens or hundreds of elements from the DOM fairly efficiently and in a single fast round-trip:

buttons = driver.find_elements_by_css_selector('button')

The result buttons can be a list of dozens or hundreds of elements without any problem.

But things slow to a crawl if you want to filter the elements using is_displayed() (or as Java calls it, isElementDisplayed) because every call to the method involves a round-trip back to the browser. Even on my fastest development machine each such call takes about 0.1s, which means that the following filter across 100 elements takes 10 seconds (!):

[b.is_displayed() for b in buttons]

The same problem happens if you want to look for a button with particular text, because each lookup of the .text attribute in fact invokes a property with its own round-trip back to the browser:

[('Subscribe' in b.text) for b in buttons]

This makes it difficult to write robust Selenium tests which pay attention to the two things that are really user-facing about the DOM: whether an element is visible, and what text it contains. While pivoting entirely away from visibility and text content — say, to unique IDs or combinations of classes or document location — would make our tests run faster, it would be creating an invisible link between our Selenium tests and the button it was looking for, instead of letting our tests keep their eye on the user experience and what users can see and read on the screen.

My question:

• Is there any way to either apply is_displayed() or a text test to elements over on the browser during the initial fetch of elements?

• Or else is there some way to do a batch is_displayed() call that asks about lots of elements instead of just one?

• Or is a 0.1s round-trip time from a Python test to Selenium under Firefox just completely unreasonable and it runs faster for everyone else and that's why obvious batch versions of these common operations do not exist?

I had thought that I could just pivot to running execute_script() from Python (Java name: executeScript()) and from inside my JavaScript code somehow execute Selenium's logic that lies behind the complicated idea of “is this element visible”. Unfortunately executeScript() does not seem to give scripts access to any of the util functions that Selenium itself finds essential, so to get access to an is-visible function we would have to pull either jQuery or a random Selenium code fragment into pages that otherwise lack them, interfering with the whole concept of a test: that it tests the page as-is without changing its JS profile merely in order to run our test.

Thanks for any ideas! I am rather surprised that Selenium expects callers to executeScript() to be rebuilding capability like is_displayed() that, as a glance at the code suggests, is a major feature of Selenium that looks like it took a lot of work to get correct and that one would want to access from all the code that possibly could, to avoid re-inventing such an important wheel.

Answer 1

From what I understand after looking into selenium python bindings source code, there is no option to send commands, like isElementDisplayed, in batches. execute() method, which is responsible for transmitting the command via the JSON Wire Protocol, handles a single command at a time only.

FYI, here is the underlying isDisplayed() algorithm from the w3c webdriver specification. And, I think, this is what chrome webdriver itself has implemented.

As a workaround and relying on this solution, we can execute javascript (not tested):

script = """
function isHidden(el) {
    var style = window.getComputedStyle(el);
    return (style.display === 'none')
}

var result = [];
for (var i = 0; i < arguments.length; i++) {
    result.push(isHidden(arguments[i]));
}
return result;
"""

driver.execute_script(script, *buttons)

Note that it is not more than a workaround since, strictly speaking, it is not even close to what webdriver really does to determine visibility.

Speaking about checking the text value of the button elements, one possible workaround, aside from taking a similar execute_script() approach, would be to use an XPath expression and verify the text:

buttons = driver.find_elements_by_xpath('//button[contains(., "some text")]')