# Using a Listener to Stop a Test When a Flag Is Set

**URL:** <https://forum.robotframework.org/t/using-a-listener-to-stop-a-test-when-a-flag-is-set/8558>\
**Category:** Robot Framework\
**Created:** [26 February 2025 16:14 UTC](https://forum.robotframework.org/t/using-a-listener-to-stop-a-test-when-a-flag-is-set/8558 "2025-02-26T16:14:48Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![wbeacham](https://avatars.discourse-cdn.com/v4/letter/w/77aa72/32.png) [@wbeacham](https://forum.robotframework.org/u/wbeacham)\
**Post date:** [26 February 2025 16:14 UTC](https://forum.robotframework.org/t/using-a-listener-to-stop-a-test-when-a-flag-is-set/8558/1 "2025-02-26T16:14:48Z")

</div>

Hey everyone, I’m hoping for some help with this!

I’m working with a **Robot Framework** test suite that interacts with a WebSocket server using a custom Python library. This library has an `expectedErrorSeen` flag, which is set to `True` when an expected error is see in the web socket reply. Im using a test template and there the error can happen at different keywords in the test.

### **What I Want to Achieve:**

- Whenever `expectedErrorSeen` is set to `True`, **stop the test immediately** but **mark it as passed**.
- This should work for **any keyword** , not just a specific one.

I wrote the following **listener** to detect when `expectedErrorSeen` is `True` in the library and i tried to stop the execution using BuiltIn().pass\_execution() function. I get the following errror and the keywords in the test are executed.

Calling method ‘end\_library\_keyword’ of listener HarrierListener.py’ failed: Expected error encountered, skipping test.

Test Template  
Send Test Email \<— error could happen here if it does i don’t want the rest of the test to run  
Process Release Approval Of Email  
Verify Test Email In Recipients Inbox  
Retransmit Test Message \<— error could happen here if it does i don’t want the rest of the test to run  
Verify Retrasmited Message In Senders Folder  
Process Release Approval Of Retransmitted Message  
Verify Retrasmited Message In All Recipients Folder

from robot.libraries.BuiltIn import BuiltIn  
from robot import result, running  
from robot.api.interfaces import ListenerV3  
from pprint import pprint

class HarrierListener(ListenerV3):

```
def __init__ (self):
    self.expected_error_seen = False
    self.test_started = False

def start_test(self, data, result):
    """Reset expected error flag at the start of each test."""
    self.test_started = True
    harrier_ws= BuiltIn().get_library_instance("HarrierUtilities.SMIME.HarrierWebSocket")
    setattr(harrier_ws, "expectedErrorSeen", False)

def end_keyword(self, data: running.Keyword, result: result.Keyword):
    """Check for the expected error flag after every keyword execution."""
    if self.test_started == True:
        # Retrieve the HarrierWebSocket instance and check for expectedErrorSeen
        harrier_ws= BuiltIn().get_library_instance("HarrierUtilities.SMIME.HarrierWebSocket")
        if hasattr(harrier_ws, "expectedErrorSeen") and harrier_ws.expectedErrorSeen:
            self.expected_error_seen = True
            BuiltIn().pass_execution("Expected error encountered, skipping test.")

```

My question is:  
is this even possible using a listener or do i need to look at managing this another way.

---

<div class="post-metadata">

**Author:** ![Many](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.robotframework.org/many/32/658_2.png) [@Many](https://forum.robotframework.org/u/Many)\
**Post date:** [26 February 2025 16:53 UTC](https://forum.robotframework.org/t/using-a-listener-to-stop-a-test-when-a-flag-is-set/8558/2 "2025-02-26T16:53:18Z")

</div>

Hi Willem,  
in my opinion this should work.

Are you able to run/debug this thing locally?  
I would be interested in the `result.status` during the `end_keyword` call.  
Is it `PASS` or is it `FAIL` ? Maybe it needs to be adjusted.

To Debug:  
If its a Listener Library and you are using RobotCode, you can enable the `Debug: Attach Python` option and set a breakpoint in your end\_keyword function.

If it is a normal listener, you can add a Debug Configuration to VS Code like  
{  
“name”: “Robot Framework: HarrierListener”,  
“type”: “python”,  
“request”: “launch”,  
“module”: “robot”,  
“args”: [“–listener”, “MyListener.py”, “-d”, “results”, “${file}”],  
“console”: “integratedTerminal”,  
“justMyCode”: false  
}  
and just use run your .robot suite in VS Code using that Debug configuration.

---

<div class="post-metadata">

**Author:** ![Many](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.robotframework.org/many/32/658_2.png) [@Many](https://forum.robotframework.org/u/Many)\
**Post date:** [26 February 2025 16:54 UTC](https://forum.robotframework.org/t/using-a-listener-to-stop-a-test-when-a-flag-is-set/8558/3 "2025-02-26T16:54:54Z")

</div>

But I have to admit that I never tried using ` BuiltIn().pass_execution` in a listener and don’t know the actual effect on the remaining keywords…

---

<div class="post-metadata">

**Author:** ![wbeacham](https://avatars.discourse-cdn.com/v4/letter/w/77aa72/32.png) [@wbeacham](https://forum.robotframework.org/u/wbeacham)\
**Post date:** [27 February 2025 09:10 UTC](https://forum.robotframework.org/t/using-a-listener-to-stop-a-test-when-a-flag-is-set/8558/4 "2025-02-27T09:10:05Z")

</div>

I simply think that its not possible to use `BuiltIn().pass_execution` in a listener, i would probably need to change some thing in one of the models but that’s a little over my head.

Something here i think.  
[https://robot-framework.readthedocs.io/en/master/autodoc/robot.running.html#robot.running.model.Keyword](https://robot-framework.readthedocs.io/en/master/autodoc/robot.running.html#robot.running.model.Keyword)

---

<div class="post-metadata">

**Author:** ![pekkaklarck](https://dub1.discourse-cdn.com/flex005/user_avatar/forum.robotframework.org/pekkaklarck/32/1019_2.png) [@pekkaklarck](https://forum.robotframework.org/u/pekkaklarck)\
**Post date:** [3 March 2025 19:48 UTC](https://forum.robotframework.org/t/using-a-listener-to-stop-a-test-when-a-flag-is-set/8558/5 "2025-03-03T19:48:20Z")

</div>

Using `BuiltIn.pass_execution()` like this doesn’t work. The reason is that it raises an exception and although that exception works when raised by a keyword, any exceptions raised by a listener is considered a failed listener call.

The way for listeners to affect the execution flow is setting `result.status`. Unfortunately for you, setting it to PASS only passes the current keyword, it doesn’t stop the whole execution. One way to accomplish that would be modifying `data` so that you add `Pass Execution` there. Not too pretty but ought to work.

Ideas how to enhance the listener API so that this would be supported are appreciated. The approach should work also with other special cases like continuable failures and stopping the whole execution.

---

<div class="post-metadata">

**Author:** ![wbeacham](https://avatars.discourse-cdn.com/v4/letter/w/77aa72/32.png) [@wbeacham](https://forum.robotframework.org/u/wbeacham)\
**Post date:** [4 March 2025 09:34 UTC](https://forum.robotframework.org/t/using-a-listener-to-stop-a-test-when-a-flag-is-set/8558/6 "2025-03-04T09:34:20Z")

</div>

Thanks for the confirmation Pekka
