Hang in the execution of test scripts

Robot Framework
1

Hello community,
I have been observing a hang in the test execution of my scripts when running a test suite which consists 700 test cases . The first 100-150 test cases got executed, but all of a sudden the execution halted and no traces of logs and reports were found.

Robot framework version -4.1
Extension - Robot Code robot framework support version 106.1
Visual studio code version - 1.95.3

2

It is hard to tell without having logs

  • try to see appium / selenium logs
  • i think that you can find console logs wether on your laptop or ci
  • run tests with
robot --loglevel TRACE -d output_directory path_to_your_tests
  • does upgrading versions of robot could be considered in your case ?
  • does it work before ? i believe that the issue is related to your environment , please provide your setup versions ( python , appium/selenium , dependencies versions like libraries etc …)
3

Hi,

Can you give any info on your test environment? Or what are you testing? Does this happen every time? Does it happen in the same test case or keyword every time?

700 test cases in one test suite sounds big count for me. Are they really in one file?
Would it be possible to split the test execution so that you run e.g. 100 test cases at one time? And then the next 100 test cases and so on… Does the test execution halt then?

This is just guessing but maybe your test execution consumes too much memory and crashes because of that?

Br,
Sami

4

Basically, I’m testing happy paths, denials and every corner test cases of my application. I wanted to check all the scenarios simultaneously under a single run. I am experiencing intermittent execution hangs, occurring approximately once every three runs. Also, I’m logging my output and reports to a separate file. Please find the libraries and dependencies below:
Python version - 3.8.3.
Package Version

certifi 2024.2.2
cffi 1.16.0
charset-normalizer 3.3.2
clr-loader 0.2.6
easyocr 1.7.1
filelock 3.14.0
fsspec 2024.5.0
idna 3.7
imageio 2.34.1
intel-openmp 2021.4.0
Jinja2 3.1.4
keyboard 0.13.5
lazy_loader 0.4
MarkupSafe 2.1.5
mkl 2021.4.0
MouseInfo 0.1.3
mpmath 1.3.0
networkx 3.2.1
ninja 1.11.1.1
numpy 1.26.4
opencv-python 4.9.0.80
opencv-python-headless 4.9.0.80
packaging 24.0
pillow 10.3.0
pip 22.0.4
PyAutoGUI 0.9.54
pyclipper 1.3.0.post5
pycparser 2.22
PyGetWindow 0.0.9
PyMsgBox 1.0.9
pyperclip 1.8.2
PyRect 0.2.0
PyScreeze 0.1.30
pytesseract 0.3.10
python-bidi 0.4.2
pythonnet 3.0.3
pytweening 1.2.0
PyYAML 6.0.1
requests 2.32.2
robotframework 4.1
robotframework-requests 0.9.7
scikit-image 0.22.0
scipy 1.13.0
setuptools 58.1.0
shapely 2.0.4
six 1.16.0
sympy 1.12
tbb 2021.12.0
tifffile 2024.5.10
torch 2.3.0
torchvision 0.18.0
typing_extensions 4.11.0
urllib3 2.2.1

5

If the hangs are more or less random it makes it even more harder to solve.
Does it happen in the same test case or keyword every time? Does it happen in the same time of a day?

What about your system? Does it have e.g. some automatic maintenance which forces reboots etc. every now and then?

Or maybe there is in a test case a loop which exit criteria is in some situations never fulfilled and it loops until the system runs out of memory? Make sure that in possible loops there are always a timeout or other exit method to avoid indefinite loops.

Br,
Sami

6

By any change are you opening subprocesses during your test execution and using stdout redirection ?

7

That is the thing that is haunting. It happens randomly at different test cases and different keywords .I have been using the timeouts after every test case and there are no infinite loops

8

Since you are running on windows and did not answer my question;

If use and pipe output from any commands and do not read the input fully - this happens. It’s windows feature. Hangs are random because it depends on how much stuff the execution generates.

1 Like