Question

159

Selenium: Scroll to end of page in dynamically loading webpage

rated 0 times [ 160] [ 1] / answers: 1 / hits: 32353 / 6 Years ago, sun, february 18, 2018, 12:00:00

I have a webpage that keeps loading new items when scrolling down the page until every item is loaded.

I'm working with Selenium in Java, and need to scroll down to the bottom of the page in order to load everything.

I have tried several different options, like scrolling to an element of the bottom of the page:

WebElement copyrightAtEndOfPage = webDriver.findElement(By.xpath(//a[@href='/utils/copyright.html']));

((JavascriptExecutor) webDriver).executeScript(arguments[0].scrollIntoView();, copyrightAtEndOfPage);

This only scrolls down once though, and then the webpage keeps loading.

I also tried this approach, which also only scrolls down once, because it only takes the browser height into consideration.

Any help is highly appreciated.

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

jeanettee

Add To Favorites

Follow

Total Points: 209

Total Questions: 97

Total Answers: 98

Location: Papua New Guinea

Member since Thu, Jul 9, 2020

4 Years ago

answered 6 Years ago kristinsonjab · Accepted Answer

I will provide you code in Python for this. I think it's easy to translate to Java:

def scroll_down(self):

    A method for scrolling the page.



    # Get scroll height.

    last_height = self.driver.execute_script(return document.body.scrollHeight)



    while True:



        # Scroll down to the bottom.

        self.driver.execute_script(window.scrollTo(0, document.body.scrollHeight);)



        # Wait to load the page.

        time.sleep(2)



        # Calculate new scroll height and compare with last scroll height.

        new_height = self.driver.execute_script(return document.body.scrollHeight)



        if new_height == last_height:



            break



        last_height = new_height

Hope it helps you!