List all failed tests as final execution step

I have quite a complicated setup and the one use case that we have is that when all the tests have completed, as a last step, I should list all the failed test case names.

This data will then be written to a 3rd party logger via API. I already post data to said logger but struggling to get the list of failed test cases.

What I do have is a python script that does get all the names as a list but I am unsure how to get that data to robot framework so that it can be used in the REST call. (Json payload.)

Working script:
robotAPI.py

    from robot.api import ExecutionResult, ResultVisitor
    
    class Visitor(ResultVisitor):
    
        def __init__(self):
            self.failed = []
    
        def end_test(self, test):
            if test.status == "FAIL":
                self.failed.append(test)
    
    visitor = Visitor()
    
    result = ExecutionResult('assets/results/output.xml')
    result.visit(visitor)
    
    for x in visitor.failed:
        print(x)

Any help at this stage will be welcomed.

Hi Orrin,

    ${result}=    Run Process    python    robotAPI.py|
    @{lines}=    Split To Lines    ${result.stdout}
  • Another way it track it in robot framework itself, create a test teardown, declared in the *** Settings *** section the test teardown will run at the end of every test case. Inside the test teardown the automatic variables ${TEST STATUS} and ${TEST NAME} are available for you, so you can append the ${TEST NAME} to a list if the ${TEST STATUS} is FAIL.
    Then you can simply use a suite teardown to send the list of failed tests to the 3rd party logger.

  • Yet another option, if the system you are sending to supports it, then send the Failed ${TEST NAME} to the 3rd party logger individually in the test teardown, this will make the coding much simpler, because it’s just an if statement in the test teardown;
    If ${TEST STATUS} = FAIL then send ${TEST NAME} to the 3rd party logger

Hopefully that gives you some ideas and options,

Dave.

Thank you @Dave

I have become blind to what is in front of me after all the struggle.

1 Like