Pabot executes all tests even only few tests are defined in ordering file

Extensions Pabot
1

I want to execute only few certain test suites in certain fashion from the test directory and I have chosen ordering would be easier to group the tests and execute only certain tests from the entire test suite.

When executing, I notice that tests are not obeying the ordering file created, rather it just creates a .pabotsuitenames file. Test execution is such a way that it executes the tests from the ordering file and also from the remaining tests from entire suite.

Also, documentation has this command to start the execution of test under grouping
pabot --process 2 --ordering order.txt data_1

process command does not work. [option --process not recognized]

order.txt file looks like this:
{
–suite Tests.Folder1.Test1
–suite Tests.Folder3.Test4
}
{
–suite Tests.Folder2.Test3
}

pabot --processes 3 --ordering ordering.txt tests
Storing .pabotsuitenames file
2025-06-10 11:29:38.549758 [PID:3758304] [0] [ID:2] EXECUTING Tests.Folder1.Test2
2025-06-10 11:29:38.550493 [PID:3758306] [1] [ID:1] EXECUTING Group_Tests.Folder2.Test3
2025-06-10 11:29:38.551134 [PID:3758307] [2] [ID:0] EXECUTING Group_Tests.Folder1.Test1
2025-06-10 11:29:39.151576 [PID:3758304] [0] [ID:2] PASSED Tests.Folder1.Test2 in 0.6 seconds
2025-06-10 11:29:39.153049 [PID:3758306] [1] [ID:1] PASSED Group_Tests.Folder2.Test3 in 0.6 seconds
2025-06-10 11:29:39.153334 [PID:3758307] [2] [ID:0] PASSED Group_Tests.Folder1.Test1 in 0.6 seconds
2025-06-10 11:29:39.152298 [PID:3758322] [0] [ID:3] EXECUTING Tests.Folder2.Test4
2025-06-10 11:29:39.153945 [PID:3758324] [1] [ID:4] EXECUTING Tests.Folder3.Test5
2025-06-10 11:29:39.160229 [PID:3758326] [2] [ID:5] EXECUTING Tests.Folder3.Test6
2025-06-10 11:29:39.654449 [PID:3758322] [0] [ID:3] PASSED Tests.Folder2.Test4 in 0.5 seconds
2025-06-10 11:29:39.755867 [PID:3758324] [1] [ID:4] PASSED Tests.Folder3.Test5 in 0.6 seconds
2025-06-10 11:29:39.762307 [PID:3758326] [2] [ID:5] PASSED Tests.Folder3.Test6 in 0.6 seconds
6 tests, 6 passed, 0 failed, 0 skipped.
2

Hi,

First thing is that the argument is --processes and not --process. It should solve you starting issue.

Then about the suites ordering file, you have to take care of the file content, especially:

  • Suites case naming
  • Suites structure in the file depending of where you run your pabot/robot command

See this topic, and the related one mentioned in it:

If the file is not set properly and read, all tests will be executed as you encounter.

Regards
Charlie

3

Thanks for the reply. I have attached the screenshot of the test folder structure and content of ordering file.

image
image871×373 76.6 KB

If my ordering file content is wrong, then the while executing the tests, the tests would not state about the executing Group_Tests specified in the ordering file. You can check the first 3 lines from the output.

pabot --processes 3 --ordering ordering.txt tests
Storing .pabotsuitenames file
2025-06-10 20:32:48.128432 [PID:3969354] [2] [ID:2] EXECUTING Tests.Folder1.Test2
2025-06-10 20:32:48.128614 [PID:3969355] [0] [ID:0] EXECUTING Group_Tests.Folder1.Test1
2025-06-10 20:32:48.129098 [PID:3969357] [1] [ID:1] EXECUTING Group_Tests.Folder2.Test3
2025-06-10 20:32:48.630066 [PID:3969354] [2] [ID:2] PASSED Tests.Folder1.Test2 in 0.5 seconds
2025-06-10 20:32:48.630585 [PID:3969360] [2] [ID:3] EXECUTING Tests.Folder2.Test4
2025-06-10 20:32:48.730951 [PID:3969355] [0] [ID:0] PASSED Group_Tests.Folder1.Test1 in 0.6 seconds
2025-06-10 20:32:48.731783 [PID:3969357] [1] [ID:1] PASSED Group_Tests.Folder2.Test3 in 0.6 seconds
2025-06-10 20:32:48.731542 [PID:3969362] [0] [ID:4] EXECUTING Tests.Folder3.Test5
2025-06-10 20:32:48.732678 [PID:3969363] [1] [ID:5] EXECUTING Tests.Folder3.Test6
2025-06-10 20:32:49.233243 [PID:3969362] [0] [ID:4] PASSED Tests.Folder3.Test5 in 0.5 seconds
2025-06-10 20:32:49.233511 [PID:3969360] [2] [ID:3] PASSED Tests.Folder2.Test4 in 0.6 seconds
2025-06-10 20:32:49.234169 [PID:3969363] [1] [ID:5] PASSED Tests.Folder3.Test6 in 0.5 seconds
6 tests, 6 passed, 0 failed, 0 skipped.

Do you still think my content in ordering.txt is wrong?

4

For me your pabot file seems correct, except the --suite Tests.Folder3.Test4 that doesn’t exist in your case.
Then, I assume your ordering.txt is in test_pabot root file.

From here, as you might have seen in the shared link, the suites provided in ordering.txt are gathered depending of the file, but if you specify .\tests folder, it will:

  • Run pabot with 3 processes, and use the groups in ordering.txt
  • But as you mention ./tests folder (as robot start folder), it will run the other suites in the folder (not grouped as not mentioned in .txt)

I have the same as you and indeed it starts all the tests, and works only if you use this command to start :

pabot --processes 3 --ordering order.txt --suite Test1 --suite Test4 --suite Test5 .\tests

It means you filter the suites to run. Another solution is to set a Test Tags to Test1, Test4 and Test5 for example:

  *** Settings ***
  Library             SeleniumLibrary
  Test Tags           MYRUN

Then run with tag include:

pabot --processes 3 --ordering order.txt --include MYRUN  .\tests

Hope this helps.

Regards.
Charlie

5

Thanks for explaining in detail.