I am looking into using Robot Framework to automate and extend a set of manual tests. These are hardware tests, think performance testing of a hardware device after it has been flashed with new firmware. In addition to pass/fail (does the DUT turn on? does it communicate with the tester?) I’d really like the generated logs to include non-boolean measurements such as power usage of various subsystems, load times, etc.
I believe I can pretty easily take these measurements and generate pass/fail by passing in threshold values, but is there any way to configure the log file to report these “variable” measurements in the statistics section?
In my travels through the documentation there are only 3 results (Pass, Fail, NotRun), so not really.
But Robot Framework is very flexible:
One option is you could have an external reporting tool like graphana or similar that you can get your test to push the extra info there, along with the pass fail results, then you can build a custom report with all the extra information you want.
Another option is to store the value in a variable with a special name e.g. ${measurement}, then run a post processor or XSLT over the xml output file to generate a report with the pass fail and measurement results
there are probably several other ways you could approach this, those are just a few ideas I thought of.
I don’t think it’ll be easy to add a separate section to the standard log with your measurements, but if you log the desired information (at the desired log level), it’ll be in the log. If it’s logged at INFO or lower, it’ll be visible if you expand the test case, if it’s at WARNING or higher, it’ll show up at the top of the report.