Hi, I am looking for test management solutions. This is not straight forward question, just interested how do you manage your testing integrating robot. My main problem is that I need some tool for the coverage management and ongoing development planning, solution to plan ongoing tests development and have ability to prioritize it.
Current approach is to implement the most valuable tests at the moment, this approach works but it’s not giving any long or even mid term distance planning though it seems to be the best approach)
I am not using manual test cases, because of lack of resources needed for it’s maintenance
I have an option to use Test Rails, though having robot test report integration only seems to be not valuable
I really don’t want to manage it in different tools doing redundant work
How do you approach your test management integrating robot framework?
Thanks in advance!
Robot Framework is a framework to specify and run / execute tests and generate a report for that run. That’s what it does, that’s what it for.
It’s not a test management tool. it’s not a planning tools. It’s not a backlog. It’s not a reporting platform. It’s not a (requirements) coverage management tools. If you want to do any of those things, you’ll have to select an appropriate tool for that particular task.
In my actual work, we have developed a Test Factory based on Robot Framework.
All the teams are using Jira to manage User stories, backlog, Defects, etc.
We had a TMS (SquashTM) which can be linked to JIRA, and offers all the functionalities you are looking for.
But 2 years ago, we have also deployed Jira Xray, which is a fully integrated plugin in JIRA which add all the functionalities of a TMS (Define tests, organize them, execute, reports, etc…).
The integration with Robot Framework is quite simple as Jira Xray will compute the report.xml and import the execution results in Jira. The only requirement is to put the JIRA key of a test, as a tag in your RF test.
As Jira Xray is integrated to Jira, if you have created a Test ticket then linked it to a User Story ticket (issue), then you will have your test coverage automaticaly available on the User Stories.
And when you’ve ran a test, the Jira Test issue will be updated with the status (PASS/FAIL/…).
Jira Xray Test issue can be described in Gherkin (thus you can use it to write your .robot files), automated test (free description), manual (Action/Dataset/Expected Result). All kind of tests can be updated automaticaly using RF execution.
If you create a bug issue, you can link it to the Test issue and/or U.S. issue.
Once again, Jira Xray is fully integrate to Jira so your Test is a new type of Jira issue.
One thing is you need to choose a single source of truth. Are your tests defined in the TMS, or in RF (scm) ?
This means that you can not do changes in Jira and hope this will update your RF test, or update RF and hope this will update your tests in the TMS, at the same time.
You can export/import tests from/to Jira xray but there are no mecanism to automaticaly ensure the consistency of both systems.
It’s not really a problem. But if it is a pain point for you, you will have to developp this kind of dual synchronization system (which I still hope to do in the future).
So, if you are using JIRA, I guess Jira Xray will be the simplest TMS which can helps you to manage your tests.
If you are using another tool (or no tools), thus you can check SquashTM (it has a free version), or other tools (TestLink may not be a good choice, seems dead…), TestMO (Saas solution), …
Each TMS tool pretend to be the #1, but you need to evaluate some of them to validate which ones are really fitting your needs, at a decent price.
Regards
Olivier