Hi All,
I have been reading through the API Documentation but can’t seem to figure out how to achieve what I’m wanting to do.
Basically what I’d like to do is
- load robot file
- run suite setup
- run test setup
- run test a
- run test a
- run test a
- run test teardown
- run test teardown
but regardless if I use robot.run or robot.run_cli I can only do
- load robot file, run suite setup, run test setup, run test a, run test teardown, run test teardown
So I’m hoping someone can point me in the right direction, am I missing something?
Dave.
You cannot do this directly. Each suite and test you run has whatever setup/teardown that was specified for them in the data. Good news is that you can pretty easily create a pre-run modifier to adjust setups and teardowns as needed.
Hi Pekka,
Thanks, can you elaborate how I would approach this? I’ve not seen mention of pre-run modifiers anywhere in the documentation (if you can point to it would be great as I often refer back to the doco).
Basically I want to run the Setup once then run the test in a while loop until the while condition is met (in my case end of test run) then run the Teardown.
I would be ok with the suite setup and suite teardown being once and the test setup and test teardown being with every test run.
Dave.
Pre-run modifiers are explained in the User Guide:
http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#programmatic-modification-of-test-data
For details about the model objects you are editing and the visitor interface, you need to study the API docs. Relevant sections ought to be linked from the User Guide
Hi Pekka,
Thanks, that was really helpful I’m getting the hang of it now.
Dave.
Great! You can use the same API to rename your tests so you don’t get that warning about duplicate tests. That said, I’m not certain is that a useful warning in general. If you don’t like it, please submit an issue and we can consider removing it or making it configurable.
I’m not so worried about the warning, when I get to it i’ll probably rename the tests with iteration nn (nn being the number).
Another question, end_test and end_suite seem to get called before the first test is actually run, is this how it’s supposed to work? the documentation says:
end_suite ( suite )[source]
Called when a suite ends. Default implementation does nothing.
end_test ( test )[source]
Called when a test ends. Default implementation does nothing.
but when I actually run:
% robot --prerunmodifier Repeat.py repeatme.robot
suite: Repeatme
suite.tests: [robot.running.TestCase(name='Repeat Me')]
suite.tests[0]: Repeat Me
test: Repeat Me
test.parent: Repeatme
test.parent.tests: [robot.running.TestCase(name='Repeat Me')]
end suite: Repeatme
==============================================================================
Repeatme
==============================================================================
These were in my end_test
test: Repeat Me
test.parent: Repeatme
test.parent.tests: [robot.running.TestCase(name='Repeat Me')]
and this was in my end_suite
end suite: Repeatme
Docs seem to be a bit misleading. Visitor is executed for the whole model at once so that start methods are called before visiting possible children and end methods are called after that. Pre-run modifiers are run before test execution starts.
If you want to do something during execution, you need to use listeners. Listener API v3 operates with same model objects as modifiers so similar code works there as well. The major limitation of listener API v3 is that it doesn’t support start/end_keyword methods.
1 Like
Ok thanks,
I already have a V2 listener, do I need a V3 listener to access the API for adding tests to the test suite? Or can that be done in the V2 API as well? It doesn’t look like end_test has the test object or end_suite has the tests list object, so I guess I need V3 for this.
Also I know I can pass multiple listeners to the same test but mix a V2 and V3 listener on the same test, or do they all have to be the same version? I guess I’ll find out when I try it.
Dave.
I successfully ran the V2 and V3 listeners together, and got the trigger to add the test running from end_test which brings me to the next question.
From end_test I have the test object, and from the parent I can get the suite object, which has a metadata object. But how do I access the variables?
I pass some variables in on the command line, when I run the test again I would like to update one of these variables. I looked at the config object of the test but couldn’t see how to get the variables?
Dave.
I do not think you can do this directly.
You need to use v3 listeners for this. In the end_test method you ought to be able to use something like this to create a new test:
new = test.parent.tests.create(name='New test')
new.body.create_keyword(name='Log', args=['Example'])
Or something like this to copy an existing test:
copy = test.copy(name='New name')
test.parent.tests.append(copy)
Same stuff works also with modifiers. You are working with same model objects in both cases.
1 Like
Thanks Pekka, I’ll give it a try.
Dave.
I was revisiting this and decided to post a full working example incase it helps someone else.
BTW Thanks @pekkaklarck for all your help with this.
Run Command:
robot --listener TestRepeater.py repeatme.robot
TestRepeater.py
from robot.api import SuiteVisitor
import time
class TestRepeater(SuiteVisitor):
ROBOT_LISTENER_API_VERSION = 3
testname = None
count = 1
def end_test(self, test, result):
# print("test:", test)
# print("test.parent:", test.parent)
# print("test.parent.tests:", test.parent.tests)
if self.count < 5:
self.count += 1
if self.testname is None:
self.testname = test.name
newname = "{} {}".format(self.testname, self.count)
copy = test.copy(name=newname)
test.parent.tests.append(copy)
# test.parent.tests.append(test)
def end_suite(self, suite, result):
# This prevents the error:
# [ ERROR ] Calling method 'end_suite' of listener 'TestRepeater.py' failed: TypeError: end_suite() takes 2 positional arguments but 3 were given
pass
def start_suite(self, suite, result):
# This prevents the error:
# [ ERROR ] Calling method 'start_suite' of listener 'TestRepeater.py' failed: TypeError: start_suite() takes 2 positional arguments but 3 were given
pass
def start_test(self, test, result):
# This prevents the error:
# [ ERROR ] Calling method 'start_test' of listener 'TestRepeater.py' failed: TypeError: start_test() takes 2 positional arguments but 3 were given
pass
repeatme.robot
*** Settings ***
Suite Setup SS keyword
Suite Teardown ST keyword
Test Setup TS keyword
Test Teardown TT keyword
*** Test Cases ***
Repeat Me
${time} = Get Time
Log Repeat Me ${time}
Log To Console Repeat Me ${time}
Sleep 5
*** Keywords ***
SS keyword
Log Suite Setup
Log To Console Suite Setup
TS keyword
Log Test Setup
Log To Console Test Setup
ST keyword
Log Suite Teardown
Log To Console Suite Teardown
TT keyword
Log Test Teardown
Log To Console Test Teardown
Result
==============================================================================
Repeatme
==============================================================================
Suite Setup
Repeat Me Test Setup
...Repeat Me 2023-03-06 11:36:24
..Test Teardown
Repeat Me | PASS |
------------------------------------------------------------------------------
Repeat Me 2 Test Setup
...Repeat Me 2023-03-06 11:36:29
..Test Teardown
Repeat Me 2 | PASS |
------------------------------------------------------------------------------
Repeat Me 3 Test Setup
...Repeat Me 2023-03-06 11:36:34
..Test Teardown
Repeat Me 3 | PASS |
------------------------------------------------------------------------------
Repeat Me 4 Test Setup
...Repeat Me 2023-03-06 11:36:39
..Test Teardown
Repeat Me 4 | PASS |
------------------------------------------------------------------------------
Repeat Me 5 Test Setup
...Repeat Me 2023-03-06 11:36:44
..Test Teardown
Repeat Me 5 | PASS |
------------------------------------------------------------------------------
Suite Teardown
Repeatme | PASS |
5 tests, 5 passed, 0 failed
==============================================================================
Output: /Users/dave/tmp/RepeatTest/output.xml
Log: /Users/dave/tmp/RepeatTest/log.html
Report: /Users/dave/tmp/RepeatTest/report.html
2 Likes
