We run test suites using SeleniumLibrary against our server UI for automated configuration and testing of various functions there. These tests were taking a total of about 35 minutes to run but we noticed that there was a recent jump in the total time taken to around 1 hour and 8 mins - around double the time normally taken.
For reference, we are running our tests on Fedora 38/39 Linux distros and using robotframework-seleniumlibrary 6.2.0. I’ve updated to latest selenium 4.17.2 but this does not seem to have made a difference. Having done some testing, including running the same tests against Chrome 120 again, we can conclude that the issue has something to do with recently upgrading to Chrome 121.
It further appears that there’s a cumulative effect in the slowdown where if I run one of the test suites from later in the testing with very few tests running before it then it completes in, say, time X. If I run some more tests before that particular suite then the time for that particular suite gets larger with a delta Y (so total time is X+Y). The more tests that I run before that particular suite then the bigger the time delta Y gets. I’m not sure if this points to a memory issue or some other resource issue in Chrome 121?
To be honest, I would have expected to see some more comments/problems related to this issue in forums/stackoverflow/Chromium issues etc. but I’m not finding very much so far and so I am wondering if this is a broader issue that others are seeing with Chrome 121 or is this something peculiar to our setup. Perhaps not many people are yet testing with SeleniumLibrary and v121?
Any feedback from the community would be helpful. Do you see a slowdown in SeleniumLibrary tests with Chrome 121? Are your tests taking much longer to complete?
You might check out the Selenium Slack workspace as well and see if there is any conversations about it there. I haven’t run against v121 so no isnight from me.
I am not a developer of SeleniumLibrary, but what could be helpful in tracking down the cause is if you can the same test with Chrome 120 and then again Chrome 121, then compare the keyword duration times. Try to identify which keywords are taking longer with Chrome 121.
I sounds like this is not a SeleniumLibrary issue, but it might be a Selenium issue? or even a chromedriver issue? but knowing where it’s slower and showing the developers that they can reproduce the slower issue will be the first step in getting it resolved.
Chrome 122 beta seems to show performance similar to Chrome 120 again so hopefully just a transient issue that is limited to Chrome 121 only.
Interestingly I’ve found that 2 other smaller test suites that we run using Chrome 121 do not show any difference compared to Chrome 120. Perhaps because they are shorter and lighter on resources then they don’t trigger the CPU/memory over-utilisation issues that our longer suite does. I’m going to dig a little more to see if I can figure out why but seeing as this is likely to be gone again in a few weeks with Chrome 122, I’m not going to over-invest time into it.