Here is the situation, some time certain test cases have reboot functionality to verify then in that case how to handle robot testing to cover this test scenario or any tag that can inducted in our .robot configuration file along with test configuration may achieve this goal.
It will be helpful if some one can guild.
In advance thanks for your support.
20210705 12:34:15.161 - INFO - ±-- END TEST: REBOOT_Test (130301)
Hi Manmohan,
Let’s see if I understood, you have a test case where your robot test connects to a serial device and then one of the keywords causes the serial device to reboot and this is causing your test suite to stop?
I would suggest trying to put a “reconnect” keyword in between the keyword that causes the reboot and any other keywords after the reboot.
the “reconnect” keyword would look something like this I would guess:
*** keywords ***
reconnect serial device
Wait Until Keyword Succeeds 10 min 1 sec Port Should Be Closed <your port identifier>
Wait Until Keyword Succeeds 10 min 1 sec Port Should Be Open <your port identifier>
You may also need to use Open Port in place of Port Should Be Open, i’ve not used SerialLibrary, so i’m guessing here based on the documentation.
FYI Wait Until Keyword Succeeds comes from the BuiltIn library.
Hope this helps.
Dave.
Hi Dave,
Thanks for your input and really appreciated. Yes, you understood my problem but with little correction. I am not using any keywords for reboot instead a test script has been instructed to reboot the target. As I understand you are suggesting “reconnect” keyword to use in my code to resolve this issue. When reboot test script trigger, serial console was asking for password and not able to reconnect as per our serial robot configuration we didn’t ask for password but it failed due to this.
20210705 09:46:00.140 - INFO - ±-- END TEST: REBOOT_Test (130066)
Snnipet of serial.robot code:
cat serial.robot
*** Settings ***
Resource common.robot
Library SerialLibrary encoding=ascii
*** Variables ***
*** Keywords ***
Open Serial Port
[Documentation] Opens connection to the target serial port
Add Port ${TA_SERIAL}
... baudrate=${TA_SERIAL_BAUDRATE}
... bytesize=8
... parity=N
... stopbits=1
... timeout=${TA_SERIAL_TIMEOUT} # Dealing with the serial port at most for 60 seconds
Serial Login
[Documentation] Opens connection to the target serial port and logs in
… to a serial console
Open Serial Port
# Wait for login prompt and issue username
Write Data ${\n}
${read} = Read Until terminator=${TA_LOGIN_PROMPT}
Should Contain ${read} ${TA_LOGIN_PROMPT}
Log ${read}
Write Data ${TA_USERNAME}${\n}
${read} = Read Until terminator=${TA_PROMPT}
Should Contain ${read} ${TA_PROMPT}
Log ${read}
Serial Logout
[Documentation] Logs out from the serial console and closes the serial
… connection
# Exit from the console
Write Data exit${\n}
Sleep 1
Delete All Ports
Execute Command And Verify Return Code
[Documentation] Executes a command/script over serial console and
… returns the exit code
[Arguments] ${command}
Write Data ${command}${\n}
${read} = Read Until terminator=${TA_PROMPT}
Log ${read}
# Get return code
Write Data echo $?${\n}
${read} = Read Until # Skip echo line
${rc} = Read Until # Get return code
${rc} = Evaluate ${rc.strip()}
log ${rc}
${read} = Read Until terminator=${TA_PROMPT}
#Should Be Equal As Integers ${rc} 0
[Return] ${rc}Hi Manmohan,
In that case you can just do this:
*** keywords ***
reconnect serial device
Wait Until Keyword Succeeds 10 min 1 sec Port Should Be Closed <your port identifier>
Wait Until Keyword Succeeds 10 min 1 sec Serial Login
BTW when you put the content of your robot file, put ``` on the line before and the line after then the formatting won’t get messed up, same with log data.
Another way you could approach it is to put a longer timeout on Open Serial Port in Serial Login or use the keywords in Serial Login in reconnect serial device with a longer timeout, the importance of the longer timeout is simply to give the serial device time to finish it’s reboot (not sure how long that takes, but sometimes it might be more than 60sec?)
Anyway I think you have the idea,
Dave.
Hi Dave,
I tried to increase timeout but it didn’t work. Also as u suggested , added “reconnect serial device” but it didn’t trigger. May be something missing?
Also my test.robot configuration looks alike
*** Settings ***
Resource serial.robot
Suite Setup Serial Login
Suite Teardown Serial Logout
*** Variables ***
*** Keywords ***
*** Test Cases ***
# TEST ID
# [Documentation] Some documentation
# [Tags] functional sys tag1 tag2
#
# ${rc} = Execute Command And Verify Return Code <Test script file path on the target>
REBOOT_Test
[Documentation] REBOOT Test
[Tags] functional XXX linux system
${rc} = Execute Command And Verify Return Code
... ${TA_HOME}/REBOOT-test.sh
Should Be Equal As Integers ${rc} 0
Issue seems with “Execute Command And Verify Return Code”
Hi Manu,
I guess that would be expected, a reboot command probably never returns any errorcode it just disconnects your session.
Can you execute without waiting for a return code?
BTW which library provides the keyword Execute Command And Verify Return Code as that is not from the serial library as far as I can see. is that
Dave.
Hi Dave,
This is our piece of code that we do execute on every test execution.
Execute Command And Verify Return Code
[Documentation] Executes a command/script over serial console and
… returns the exit code
[Arguments] ${command}
Write Data ${command}${\n}
${read} = Read Until terminator=${TA_PROMPT}
Log ${read}
# Get return code
Write Data echo $?${\n}
${read} = Read Until # Skip echo line
${rc} = Read Until # Get return code
${rc} = Evaluate ${rc.strip()}
log ${rc}
${read} = Read Until terminator=${TA_PROMPT}
#Should Be Equal As Integers ${rc} 0
[Return] ${rc}
I was thinking to modify this call with some condition Like:
${prev-read}=${read}
# Get return code
IF Log ${read} Contain ${TA_LOGIN_PROMPT}
Log To Console Do Something.
Pass Execution Serial Login
ELSE
Log To Console Do Nothing.
END
Can you help in this regard how to use conditional check here.
Hi Manu,
As Execute Command And Verify Return Code is your own custom keyword, that approach looks like it could work, unfortunately we are getting into the territory of where what will and won’t work depends on the behaviour of the device you are testing, so you will need to just try things and see what works.
BTW i’ve not actually ever used the serial library, so my advice has really been just general in nature. When i’m creating keywords to deal with exceptions like this I often run them more than 3 times till I get a sequence that i’m confident will be reliable, so if you don’t get it to work first go, don’t get discouraged, just keep trying different variations till you get it working.
Dave.
Hi Dave,
Thanks again, It might be possible I have been figure out the issue to handle this situation but having robot framework related hurdle i.e. IF is a reserved keyword.
Error:
REBOOT_Test :: REBOOT Test | FAIL |
'If' is a reserved keyword.
Piece of code integrated to resolve reboot issue-
IF "${rc}" == "Password:"
Log To Console Do Something.
Pass Execution Serial Login
ELSE
Log To Console Do Nothing.
END
Don’t know what is wrong in this piece of code. plz have a look.
Hi Manu,
Try it like this:
IF "${rc}" == "Password:"
Log To Console Do Something.
Pass Execution Serial Login
ELSE
Log To Console Do Nothing.
END
I’m not 100% sure if the indentation is required in robot framework, but all the examples in the documentation shows the indents in place. robot framework is built in python and pyton requires the indents so i wouldn’t be surprised if that is where you are encountering the issue.
Also what version of robot framework are you using? this IF / ELSE / END syntax you are using is only available in robot framework version 4.x . if you are using 3.x version of robot framework you’ll need the old method using if keywords like Run Keyword If.
Dave.
Hi Dave,
I tried with all the possible ways even upgraded system with robot 4.x version too but the problem remains same. Even python version linked to 3.6 and also tried with your suggestions have same result i.e. “If is a reserved keyword”.
$ robot --version
Robot Framework 4.0.2 (Python 3.6.9 on linux)
But no luck.
Manu.
Hi Manu,
Interesting, so I checked my machine:
Robot Framework 4.0.1 (Python 3.9.1 on darwin)
And then went an looked at some working examples of IF statements from my robot files (see below), not sure if this will make a difference but try changing the double quotes to single quotes.
IF '${rc}' == 'Password:'
Log To Console Do Something.
Pass Execution Serial Login
ELSE
Log To Console Do Nothing.
END
Working examples of IF statements:
IF ${length} > 0
Click Button Cancel
ELSE
${votecount}= Evaluate ${votecount} + 1
END
IF '${mins}[0]' == '0'
${mins}= Set Variable ${mins}[1]
END
Dave.
Hi Dave,
Thanks for supporting and spending time for this issue, tried all ways but IF/ELSE construct is not working at all. I tried with Old method “Run Keyword if” where one sample test case gets executed successfully but another reboot test case gets fail may be in evaluating variable(sysntax error). Sharing piece of code and related log to check possibility of error. if possible plz have a sharp look n spare some time.
Here is the code snnipet:
Execute Command And Verify Return Code
[Documentation] Executes a command/script over serial console and
... returns the exit code
[Arguments] ${command}
Write Data ${command}${\n}
${read} = Read Until terminator=${TA_PROMPT}
Log ${read}
# Get return code
Write Data echo $?${\n}
${read} = Read Until # Skip echo line
${rc} = Read Until # Get return code
#${rc} = Evaluate ${rc.strip()}
Log ${rc}
Run Keyword if ${rc} == ${reboot} Serial Login ${rc}=0
#${rc} = Evaluate ${rc.strip()}
log ${rc}
${read} = Read Until terminator=${TA_PROMPT}
#Should Be Equal As Integers ${rc} 0
[Return] ${rc}
Working sample testcase log:
root@euto-v9-sadk:~#
20210709 15:11:59.458 - INFO - +--- END KW: BuiltIn.Log (1)
20210709 15:11:59.458 - INFO - +--- START KW: SerialLibrary.Write Data [ echo $?${\n} ]
20210709 15:11:59.459 - INFO - +--- END KW: SerialLibrary.Write Data (1)
20210709 15:11:59.459 - INFO - +--- START KW: SerialLibrary.Read Until [ ]
20210709 15:11:59.471 - INFO - ${read} = echo $?
20210709 15:11:59.471 - INFO - +--- END KW: SerialLibrary.Read Until (12)
20210709 15:11:59.472 - INFO - +--- START KW: SerialLibrary.Read Until [ ]
20210709 15:11:59.472 - INFO - ${rc} = 0
20210709 15:11:59.472 - INFO - +--- END KW: SerialLibrary.Read Until (0)
20210709 15:11:59.472 - INFO - +--- START KW: BuiltIn.Log [ ${rc} ]
20210709 15:11:59.473 - INFO - 0
20210709 15:11:59.473 - INFO - +--- END KW: BuiltIn.Log (1)
20210709 15:11:59.473 - INFO - +--- START KW: BuiltIn.Run Keyword If [ ${rc} | == | ${reboot} | Serial Login | ${rc}=0 ]
20210709 15:11:59.474 - INFO - +--- END KW: BuiltIn.Run Keyword If (1)
20210709 15:11:59.474 - INFO - +--- START KW: BuiltIn.Log [ ${rc} ]
20210709 15:11:59.475 - INFO - 0
20210709 15:11:59.475 - INFO - +--- END KW: BuiltIn.Log (1)
20210709 15:11:59.475 - INFO - +--- START KW: SerialLibrary.Read Until [ terminator=${TA_PROMPT} ]
20210709 15:11:59.503 - INFO - ${read} = root@euto-v9-sadk:~#
20210709 15:11:59.503 - INFO - +--- END KW: SerialLibrary.Read Until (28)
20210709 15:11:59.504 - INFO - ${rc} = 0
20210709 15:11:59.504 - INFO - +-- END KW: serial-mod.Execute Command And Verify Return Code (346)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
20210709 15:11:59.504 - INFO - +-- START KW: BuiltIn.Should Be Equal As Integers [ ${rc} | 0 ]
20210709 15:11:59.505 - DEBUG - Argument types are:
<type 'unicode'>
<type 'unicode'>
20210709 15:11:59.506 - INFO - +-- END KW: BuiltIn.Should Be Equal As Integers (1)
------------------------------------------------------------------------------
20210709 15:11:59.507 - INFO - +- END TEST: SAMPLE-TEST (350)
------------------------------------------------------------------------------
Failure reboot testcase log:
0 2 0:[0:0115982377] [00004460 ms] euto-v9-sadk login:
20210709 15:12:24.856 - INFO - +--- END KW: BuiltIn.Log (3)
20210709 15:12:24.857 - INFO - +--- START KW: SerialLibrary.Write Data [ echo $?${\n} ]
20210709 15:12:24.858 - INFO - +--- END KW: SerialLibrary.Write Data (1)
20210709 15:12:24.858 - INFO - +--- START KW: SerialLibrary.Read Until [ ]
20210709 15:12:24.875 - INFO - ${read} = echo $?
20210709 15:12:24.876 - INFO - +--- END KW: SerialLibrary.Read Until (18)
20210709 15:12:24.876 - INFO - +--- START KW: SerialLibrary.Read Until [ ]
20210709 15:12:39.923 - INFO - ${rc} = Password:
20210709 15:12:39.923 - INFO - +--- END KW: SerialLibrary.Read Until (15047)
20210709 15:12:39.924 - INFO - +--- START KW: BuiltIn.Log [ ${rc} ]
20210709 15:12:39.924 - INFO - Password:
20210709 15:12:39.925 - INFO - +--- END KW: BuiltIn.Log (1)
20210709 15:12:39.925 - INFO - +--- START KW: BuiltIn.Run Keyword If [ ${rc} | == | ${reboot} | Serial Login | ${rc}=0 ]
20210709 15:12:39.926 - FAIL - Evaluating expression 'Password: ' failed: SyntaxError: invalid syntax (<string>, line 1)
20210709 15:12:39.927 - DEBUG - Traceback (most recent call last):
None
20210709 15:12:39.927 - INFO - +--- END KW: BuiltIn.Run Keyword If (2)
20210709 15:12:39.928 - INFO - +-- END KW: serial-mod.Execute Command And Verify Return Code (40420)
------------------------------------------------------------------------------
20210709 15:12:39.930 - INFO - +- END TEST: REBOOT_Test (40422)
------------------------------------------------------------------------------
Don’t know in one test case it is passing and with another test showing syntax error. May be something missing?
Manu.
Hi Manu,
This line:
Run Keyword if ${rc} == ${reboot} Serial Login ${rc}=0
in the first example log ${rc} is an integer (0) and ${reboot} is also an integer (0) so the comparison works happily
in the first example log ${rc} is a string so the comparison with an integer causes an exception in Run Keyword if before the comparison even happens.
The easiest solution is to ensure they are always both strings, like this:
Run Keyword if '${rc}' == '${reboot}' Serial Login ${rc}=0
In the first case you would be comparing ‘0’ == ‘0’, still true
In the second case you would be comparing ''Password: ‘’ == ‘0’, not true and because they are both strings no exception.
Dave.
Hi Dave,
This ways issue was also not get resolved then tried with no. of different ways and finally worked as per expectation.
Thanks you very much for your reply and interest to keep motivation up. Robot Framework construct is not as much easy as we do think like other language but very good to learn also you have to keep on trying.
Solution:
${rc}= Run Keyword If "${rc.replace('\n','').strip()}" == "${reboot.replace('\n','').strip()}" Reconnect Serial Login
... ELSE Set Variable ${rc}
Manu.
Hi Manu,
I see it was a difference of new line characters that was causing your issue, I would not have guessed that without access to your systems, so I’m glad you figured it out.
When testing an application or device that uses less common libraries it can take a bit of effort to set up you low level keywords, but think of them as building blocks or lego bricks, one you have these basic low level keywords built and saved in a resource file then building your high level test cases will go much quicker as you re-use these low level keywords.
The other thing is that your keywords is named descriptively mean that anyone with no programming knowledge can read your test cases and understand exactly what it’s doing, likewise the test results are very easy to read and understand the failure case.
Stick with it and you should see the benefits quickly enough, also this issue you encountered is probably one of the harder issues i’ve seen anyone encounter, so you are now in a good position to solve any other issue you run into, and hopefully everything should be easy from here 
Good luck with your testing,
Dave.