Adding test results directly suite_results.create doesn't work

Robot Framework
1

Hi all,

I am trying to add “test results” to robotframework dynamically, and have some problems occurred.
The below is a simplified test of what I am trying to do, the below is a test (to test if adding “test results” directly) will show up in the final output.xml

I created a listener v3, and imported it as a library.
I registered a

start_suite listener, to keep record of the current suite

_start_suite(self, stuite, result):
self.currentSuite = suite

end test listener

def _end_test(self, test, result):
# test for creating a test result directly (from a existing test)
test_result = self.currentSuiteResult.tests.create(test.name+‘APPENDED_TEST’,
test.doc,
test.tags,
None,
test.lineno,
test.start_time)
# apply all settings from test to test_result
test_result = test
# modify the name
test_result.name = test.name+'MODIFIED"

I tried copying the result testcase, with a different name, and adding it to the suite results as below

    test_result = self.currentSuiteResult.tests.create(test.name+'MODIFIED',
                                    test.doc,
                                    test.tags,
                                    None,
                                    test.lineno,
                                    test.start_time)
  test_result = test
  test_result.name = test.name+'MODIFIED"

But it doesn’t seem to show up in the final output.xml (or report.html)

Is there something else I need to do for it to work?

Thanks for the help!

Note:
on why I am trying to add test results without any test case: I used robotframework to execute a googletest executable, which results in a xml file. Since there are multiple test cases within the googletest, I would like to add them into the googletest results. I didn’t create a new “dynamic test case” since there isn’t a need to execute anything.

2

sorry for the update

the _end_test(self, test, result) test code is updated as below

def _end_test(self, test, result):
test_result = self.currentSuiteResult.tests.create(result.name+‘TEST’,
test.doc,
test.tags,
None,
test.lineno,
datetime.now())
test_result.status=‘PASS’

by adding this listener, I expect the output.xml will have duplicate test results of each test with different names(with the word TEST appended)

3

after digging into robotframework source code, it seems output.xml and other outputs are generated while the suiterunner is visiting the suite.

Ex: suiterunner.py visit_test
it calls self.output.end_test(data, result), which writes to the corresponding logger output files.
If by adding a entry to the “result” directly, the output logs will not trigger the write test start, test end to output files.

4

It’s hard to read your examples because they aren’t formatted correctly. Anyway, it seems that you have already found the reason why changes to, for example, test case name aren’t possible in the end_test listener method. The test name has been already written to output.xml earlier and thus the change change has no effect. Also if you create new tests results with tests.create(), they are only created in memory and Robot doesn’t know they should be written to output.xml.

I didn’t fully understand what you are trying to do, but here are some alternatives that may help:

  • If you want to create new tests dynamically during execution, you need to do them in start_suite or end_test so that you modify data, not result. These newly added tests will be executed and logged as expected. You can then modify them further when they are executed.
  • An alternative solution would be modifying results after execution. This can accomplished with the --pre-rebot-modifier option or by parsing results, modifying them and saving the modified results. How to use the former is explained in the User Guide and the result APIs are documented in the API docs.
5

Hi Pekka,

Thanks for spending time to reply to my question, it helped with clarifying my questions.

1 Like