Can we create bug automatically in azure dev-ops when pipeline fails

Robot Framework
1

Hi All,

Is there a way to create bug automatically in azure dev-ops when pipeline fails?

Thanks,
Sudheer

2

Is it possible? Perhaps. But you shouldn’t call it a Bug but a Log since it won’t be anything more than that. If a pipeline fails, a human should take a look at the pipeline logs and determine the cause of the failure. Depending on the failure, that human could decide to file a Bug report.

1 Like
3

Hi @sudheer1987,

As I see it there are a few parts to this question:

  1. Does azure dev-ops have api calls to create a “bug”?
    I don’t know the answer to this part, perhaps someone here is more familiar with azure dev-ops and can answer this? Perhaps your dev team can help you with this?
  2. Can RobotFramework perform actions when a test case fails?
    This one I can answer, yes there are several ways this can be achieved:
    – you can use an if statement in a teardown keyword, the ${TEST STATUS} will tell you if the test has failed, then inside the if statement you can call an API to azure dev-ops to do what you need
    – you could use a listener to listen for a end_suite or end_test and then check the status for PASS or FAIL, then you can execute some python code based on this result status
  3. should we create a bug automatically?
    This is what @robinmackaij was raising, It probably shouldn’t be called a bug initially, I like the term “Test Incident” better but azure dev-ops may force you to use the term bug? I know ALM/Quality Centre forces the use of defect.
    Either way, you should initially assign any auto created bug to a tester (yourself?) to manually verify it’s actually a bug before passing it on to anyone else.
    – test automation can fail because of an issue with the test automation
    – test automation can fail because the test case has changed and the test automation needs updating
    – test automation can fail because the requirements have changed and the test automation needs updating
    – test automation can fail because a multitude of reasons that are not an actual bug with the software
    – test automation can fail because of an actual bug

While auto created bug reports can help to improve/standardise/speed up reporting of issues, I suggest you proceed cautiously down this path and ensure a tester verifies every auto created bug before they are passed on otherwise it can cause the test team to loose credibility and this is much worse than any benefit from auto created bugs.

Dave.

4

Thanks Dave

5

Thanks Robin