Suite Teardown stop on keyword error

Robot Framework
1

Hello,

I’m facing an issue on Suite Teardown. From what I read in the documentation, the teardown should always been run.

But I just discover than, in case of keyword issue (not a test failure) it is not the case. Here a simple example inspired from my test suite:

*** Settings ***

Suite Setup         Setup
Suite Teardown      Release

Force Tags          robot:recursive-continue-on-failure
Default Tags        all

*** Test Cases ***

Test1
    Log  test1

*** Keywords ***
Setup
    Log  setup

Release
    SubRelease
    Log  Release 4

SubRelease
    Log  release 1
    Set Test variable   ${toto}   4
    Log  Release 3

Here the result
image

I know that do a Set Test variable in a teardown is a mistake but still, teardown should not continue ? Or did I missed something ?

Thank you.
Best regards,
Max

2

I “think” that keyword errors are critical, that is why it Fails.

3

Screen Shot 2022-02-01 at 9.53.12 am

If you are asking why these 2 keywords ran? they didn’t Grey is “SKIP” status 3.2.2 Test and suite statuses

Does that answer your question or did I miss understand?

Dave.

4

I’m asking “why they did NOT ran?” => Teardown shall be considered as “Continue if failure” but in that case, it is not. Are we agreed that’s a bug ?

5

Thats beyond my knowledge of Robot Framework, sorry I’m not a Robot Framework developer. Maybe someone else can step in and answer?

6

It’s not a bug but according to design and documentation. The reserved tags continue-on-failure and recursive-continue-on-failure apply to Test Cases. The Suite Setup and Suite Teardown are not part of the Test Case scope. I’m not entirely sure if Test Setup and Test Teardown are considered in the Test Case scope, I would guess not.

I believe being able to use robot:continue-on-failure at a keyword level was discussed at some point, but it raises my issues that can lead to conflicting behavior.

7

It shall be the case, here an extract of the user manual :

image
image848×108 4.53 KB

8

Helio is correct, it’s the difference between a keyword failure and an error. If Set Test variable ${toto} 4 is changed to Fail (= regular keyword failure), Log Release 3 and Log Release 4 are executed as expected. If it’s changed to Fatal Error, we get the result you’re seeing.

While this could be more clear in the documentation, I understand why this difference is there; if a Fatal Error occurs, that’s a hard override the regular flow…trying to continue should be considered pointless at that point. This allows us to abort a Teardown, if needed / desired, which would be impossible if any error / failure was ignored during Teardown.

1 Like
9

Ok got it, thx !