Hi!
I have around 300 suites.
How to see in console amount of suites waiting to be run?
I am running cases with pabot.
For example I want to see in console something like 200 suites executed, 10 in process, 90 is not run?
Hello @Vitaly,
Pabot doesn’t show real-time suite progress by default, but you can enable it using the --verbose flag. For more detailed tracking, use --pabotlib and implement a custom listener to log suite states. Alternatively, this forum thread discusses community workarounds for tracking suite execution status dynamically. Hope that helps streamline your runs!.
Best Regards,
Amy Cross
I’m not sure if this does what you’re looking for and I probably over-engineered this a while back, but I wanted something similar so I could see what was going on in an Azure pipeline that runs for several hours. If a lot of things were failing, I could just abort, fix the problems, and run again rather than wait for the whole thing to complete with tons of failures. It is more Test-oriented than Suite-oriented, but maybe it is helpful enough for you as-is, or gives a starting point and you can hack away at it. I wrote a listener that would output the following before running anything (just as a sanity check).
- The name of each ‘Suite’ directory (single directory called Access in this example)
- The name of each Suite file within each Directory
- The name of each Test in each Suite file
Then it would output each…
- Suite Start
- Test Start
- Test End (Green for Pass, Red for Fail) with Test Duration
- Suite End with Counts of Pass and Fail
- Running Overall Test Stats with Total Duration, which shows how many Test remaining
import time
from datetime import datetime
from robot.libraries.BuiltIn import BuiltIn
class TestCaseCounterListener:
ROBOT_LISTENER_API_VERSION = 3 MAGENTA = '\\033\[95m' BLUE = '\\033\[94m' GREEN = '\\033\[92m' RED = '\\033\[91m' RESET = '\\033\[0m' def \__init_\_(self, filename='failedTests.md'): self.total_test_count = 0 self.executed_cases = 0 self.passed_cases = 0 self.failed_cases = 0 self.overall_start_time = 0 self.suite_start_time = {} self.suite_end_time = {} self.test_start_time = {} self.suite_test_counts = {} self.suite_passed_cases = {} self.suite_failed_cases = {} self.filename = filename self.fh = open(self.filename, 'w') self.fh.write("# Robot Framework Failed Test Report\\n") self.fh.write("|Test|Status|\\n") self.fh.write("|---|---|\\n") self.fh.close() def close(self): self.fh.close() def start_suite(self, data, result): \# Start timing the suite self.suite_start_time\[data.longname\] = time.time() self.suite_test_counts\[data.longname\] = len(data.tests) self.suite_passed_cases\[data.longname\] = 0 self.suite_failed_cases\[data.longname\] = 0 \# Print line with full coloring print(f"{self.\_get_colored_message(self.MAGENTA, 'Starting Suite: ' + data.name)}", flush=True) if data.parent is None: \# Overall start time for top-level suite self.overall_start_time = self.suite_start_time\[data.longname\] \# Count total test cases for all sub-suites self.total_test_count = self.\_count_test_cases(data) self.\_output_suite_structure(data) #input("\\nAbout to Begin Suite Execution. Please Confirm above tests and Press Enter to continue...") def start_test(self, data, result): \# Start timing the test self.test_start_time\[data.longname\] = time.time() \# Print line with full coloring print(f"\\n{self.\_get_colored_message(self.BLUE, 'Starting Test: ' + data.parent.name + ' - ' + data.name)}", flush=True) def end_test(self, data, result): \# End timing the test test_end_time = time.time() test_elapsed_time = test_end_time - self.test_start_time\[data.longname\] formatted_test_time = self.\_format_elapsed_time(test_elapsed_time) \# Track executed, passed, and failed cases self.executed_cases += 1 if result.passed: self.passed_cases += 1 self.suite_passed_cases\[data.parent.longname\] += 1 test_color = self.GREEN else: self.failed_cases += 1 self.suite_failed_cases\[data.parent.longname\] += 1 test_color = self.RED \# Print line with specific coloring for pass/fail print(f"{self.\_get_colored_message(test_color, 'Completed Test: ' + data.parent.name + ' - ' + data.name + ' (' + formatted_test_time + ')')}", flush=True) \# Write Failures to File in Append Mode if result.failed: self.fh = open(self.filename, 'a') self.fh.write(f"|{result.name}|{result.status}|\\n") self.fh.close() def end_suite(self, data, result): \# End timing the suite self.suite_end_time\[data.longname\] = time.time() \# Calculate suite elapsed time suite_elapsed_time = self.suite_end_time\[data.longname\] - self.suite_start_time\[data.longname\] formatted_suite_time = self.\_format_elapsed_time(suite_elapsed_time) \# Calculate total elapsed time from the start of the top-level suite total_elapsed_time = self.suite_end_time\[data.longname\] - self.overall_start_time formatted_total_time = self.\_format_elapsed_time(total_elapsed_time) remaining_cases = self.total_test_count - self.executed_cases suite_tests = self.suite_test_counts\[data.longname\] suite_passed = self.suite_passed_cases\[data.longname\] suite_failed = self.suite_failed_cases\[data.longname\] \# Conditional coloring for suite-level pass/fail counts suite_passed_color = self.GREEN if suite_passed > 0 else self.RESET suite_failed_color = self.RED if suite_failed > 0 else self.RESET \# Output summary after each suite print(f"\\n{self.\_get_colored_message(self.MAGENTA, 'Completed Suite: ' + data.name + ' (' + formatted_suite_time + ') - ' + str(suite_tests) + ' tests, ')}" f"{suite_passed_color}{suite_passed} passed{self.RESET}, {suite_failed_color}{suite_failed} failed{self.RESET}", flush=True) \# Conditional coloring for overall pass/fail counts overall_passed_color = self.GREEN if self.passed_cases > 0 else self.RESET overall_failed_color = self.RED if self.failed_cases > 0 else self.RESET \# Print overall stats with specific coloring for pass/fail print(f"{self.\_get_colored_message(self.MAGENTA, 'Overall Stats: ' + str(self.executed_cases) + ' (of ' + str(self.total_test_count) + ') tests executed after ' + formatted_total_time + ', ')}" f"{overall_passed_color}{self.passed_cases} passed{self.RESET}, {overall_failed_color}{self.failed_cases} failed{self.RESET}, {remaining_cases} remaining\\n", flush=True) def \_format_elapsed_time(self, total_seconds): """Format elapsed time as hours:mins:secs.""" hours = int(total_seconds // 3600) minutes = int((total_seconds % 3600) // 60) seconds = int(total_seconds % 60) return f"{hours:02}:{minutes:02}:{seconds:02}" def \_get_colored_message(self, color, message): """Return the message with the specified color.""" return f"{color}{message}{self.RESET}" def \_output_suite_structure(self, suite, indent=0): suite_indent = ' ' \* indent test_indent = ' ' \* (indent + 2) \# Print the suite name with the count of test cases in this suite and its subsuites suite_test_count = self.\_count_test_cases(suite) print(f"{suite_indent}- {suite.name} ({suite_test_count} test case{'s' if suite_test_count != 1 else ''})", flush=True) for test in suite.tests: print(f"{test_indent}- {test.name}", flush=True) for subsuite in suite.suites: self.\_output_suite_structure(subsuite, indent + 2) def \_count_test_cases(self, suite): """Recursively count the number of test cases in the suite and its subsuites.""" count = len(suite.tests) for subsuite in suite.suites: count += self.\_count_test_cases(subsuite) return count
Wow, thank you! I will take a look into it.
As already suggested, this can be accomplished with a listener. Enhancing Robot’s standard console output would be possible as well, but that requires someone to design how it should actually work.