Skip On failure option with a various tags

Robot Framework
1

Hi all,
I have a test suite with SMOKE tag for all the tests, and various tags for some tests in it.
to be clear, all the tests have the SMOKE tag, and some of the tests have another tags.
for example:

  1. tc1 suite - [tags] SMOKE TEST1
  2. tc2 suite - [tags] SMOKE TEST1
  3. tc3 suite - [tags] SMOKE TEST2
  4. tc4 suite - [tags] SMOKE TEST2

I want to use --skiponfailure flag in my robot command , so it will skip all the test suite with TEST1 tag (as I show in the example). since it is not working for me and the tests keep fail instead to skip.

this is the command I sent (in this order):
robot -V envsetup.py -i SMOKE --skiponfailure TEST1–skiponfailure TEST2 --loglevel trace -d ./test_result --name SMOKE functional_tests/*.robot

I have some ideas why its failed and didn’t skip , but Im not sure if its corret ideas:

  1. The order of arguments: the --skiponfailure options are placed after the test suite or test case names.
  2. There are 2 kinds of Tags in the test, SMOKE and TEST1, so the skiponfailure tag not working for 2 tags and above.

regards oriel :slight_smile:

1 Like
2

Hi, I do not use this option myself, but since nobody else answered, here are some ideas:

  1. the double dash in front of skiponfailure TEST2 seems to have been replaced we some wide dash special character. Although this probably only occurred when you copied and pasted it to the forum.
  2. The documentation about skiponfailure mentions that the option is compatible with tag patterns. You might want to give this a try.
  3. You could try asking on robotframework slack to get more answers.

If you already found how to fix your issue, then please share it with us!

1 Like
3

thanks for the response, I will try what you suggest and share results, thenks :slight_smile:

1 Like
4

Hi all,
In the end, I created a listener to handle with this issue.
The lisener is running during the test suite for each step and check if the Tags of the step are relevant ,if the test fail, to skip or not.
In the robot command itself I putted the path of the lisener and as argument the tegs that relevant for skip on failure.
here is example of the command:
"robot --listener /path/to/SkipOnFailureListener.py:TAG_1,TAG_2,TAG3 /path/to/robot_file.robot

now, to make the step to skip if the tag fail I created via the listener Global Variable so every step can took this variable and skip is is True.
I just wrote in the robot file this line
“Test Setup setup_skiponfailure”

and the keyword of setup_skiponfailure look like that

setup_skiponfailure
${is_should_skip} Get Variable Value ${G_SKIP_TEST} ${False}
Skip If ${is_should_skip} ${skip_msg}

and here is part of the listener (python file):

from robot.libraries.BuiltIn import BuiltIn

b = BuiltIn()

class SkipOnFailureListener:
ROBOT_LISTENER_API_VERSION = 3

def __init__(self, skip_on_failure_tags=''):
    self.tags_failed = []
    self.current_tags = []
    self.skiponfailure_tags = skip_on_failure_tags.split(',')
    print(self.skiponfailure_tags)

def start_test(self, data, result):
    b.set_global_variable('${G_SKIP_MESSAGE}', self.skip_msg)
    self.current_tags = result.tags
    self.current_tags = list(set(self.current_tags) & set(self.skiponfailure_tags))
    if any(tag in self.tags_failed for tag in self.current_tags):
        b.set_global_variable('${G_SKIP_TEST}', True)
    else:
        b.set_global_variable('${G_SKIP_TEST}', False)

def end_test(self, data, result):
    if result.status == 'FAIL':
        self.tags_failed.extend(self.current_tags)
        self.tags_failed = list(set(self.tags_failed))
    elif any(tag in self.tags_failed for tag in self.current_tags):
        b.set_global_variable('${G_SKIP_TEST}', True)

pay attention to check wich version you need of ROBOT_LISTENER_API_VERSION , there are 2 kinds 2 and 3.

3 Likes