Robot tests fail when calling the test suite from jenkins pipeline

Tools Jenkins
1

Robot tests fail when calling the test suite from jenkins pipeline
and the test.robot contains the code like below

${processOutput}= Run Process ${exepath} --option=get timeout=${timeoutValue}

When i execute the robot manually from command prompt as robot test_suite.robot
I get the proper output in ${processOutput} .If i run from jenkins pipeline it does not give proper output . Any inputs?

2

Hi Shilpa,

Start with the simple stuff, paths

  • check the path to the robot file when you run manually (it should be in the log)
  • check the path to the robot file jenkins runs it (it should be in the log)
  • are they the same? note any difference
  • check on the jenkins runner if all the paths to any resource files exists and the resource file exists as well
  • check any files referenced in resource files as well, keep going down that rabbit hole till you’ve checked them all

if your still getting errors, show the error message so we can have a better chance to help.

Dave.

3

Hi Dave,

I dont see the difference in path when executing locally and from jenkinsfile.
17:07:12.383 INFO Starting process: C:\jenkins\workspace\PR-372\src\tests\robot/bin/xusbif.exe --version
Please find the logs below.

Ran manually :- Working logs

KEYWORD ${processOutput} = Process . Run Process C:\XCOMR\853504app\src\tests\robot/bin/xusbif.exe, --version
Documentation:
Runs a process and waits for it to complete.

Start / End / Elapsed: 20221222 14:28:29.405 / 20221222 14:28:29.590 / 00:00:00.185
14:28:29.405 INFO Starting process:
C:\XCOMR\853504app\src\tests\robot/bin/xusbif.exe --version
14:28:29.566 INFO Waiting for process to complete.
14:28:29.590 INFO Process completed.
14:28:29.590 INFO ${processOutput} = <result object with rc 0>

Ran from Jenkinsfile:

KEYWORD ${processOutput} = Process . Run Process C:\jenkins\workspace\853504app\src\tests\robot/bin/xusbif.exe, --version

Documentation: Runs a process and waits for it to complete.
Start / End / Elapsed: 20221222 17:07:12.383 / 20221222 17:07:12.390 / 00:00:00.007

17:07:12.383 INFO Starting process: C:\jenkins\workspace\853504app\src\tests\robot/bin/xusbif.exe --version

17:07:12.385 INFO Waiting for process to complete.

17:07:12.390 INFO Process completed.

17:07:12.390 INFO ${processOutput} = <result object with rc 2>

you can see the difference that from jenkins rc is 2

Jenkinsfile in is in below path

C:\XCOMR\853504app\jenkins_pipeline

and code in jenkinsfile is as below:
ir (“${env.WORKSPACE}\src\tests\robot”){
script {
sh(returnStatus:true, script:“python -m rflint --ignore LineTooLong test_suite.robot”)
robotRunResult = sh(returnStatus:true, script:“python -m robot.run test_suite.robot”)

4

Hi Shilpa,

I have good news and bad news for you:

  • Good news first, robot framework is behaving exactly the same locally and in your Jenkins pipeline
  • The bad news is it’s xusbif.exe that’s behaving differently in your Jenkins pipeline and so not knowing what this is the help I can give is going to be limited, but i’ll try my best to guide you in the right direction.

I gather from your output that you have a line similar to this in your test case:

${processOutput}= 	Run Process 	/bin/xusbif.exe 	--version

So the first thing I’ll suggest is you add some lines like this just after that one:

Log  	${processOutput.stdout}
Log  	${processOutput.stderr}

This will let you see the output from xusbif.exe and hopefully give you a clue to why you are getting a return code 2

It could be something simple like something missing from the path or a dll that xusbif.exe is dependant on not being installed on the Jenkins pipeline machine, or it could be something more complicated. But once you have the output from stdout and stderr you can show that information to the developers of xusbif.exe and they should be able to tell you what the problem is.

Dave.

5

Hi Dave , I have already printed the Log ${processOutput.stdout}
Log ${processOutput.stderr}

and i just get the empty string I mean no output for those. But do you know what does return code 2 refer to if its some specific error?

and /bin/xusbif.exe this executable if i place it anywhere on any machine i can it without any problem it does not have any dependencies.

6

Hi Shilpa,

No I have no idea what an return code of 2 means you will need to refer to the documentation or the developer of xusbif.exe.

The only standard return code is 0 meaning exited normally with no errors, any non zero return code is an error but it’s up to the developer to document the meaning of each non zero return code.

I would suggest you start by raising a defect/issue for xusbif.exe for returning a non zero return code with no output on stderr as this is a bad practice.

Dave.