RobotCode - whole project marked as test suite

[Oavio (David S.)]

Hey guys,

I had to start with my VSCode environment for RF preparation from scratch and it led me to many issues which I hope you could help me with.

I have the robotframework_sikulilibrary==1.0.8 installed (I don’t have the Java installed - intentionally) and I run any of these two test cases from this example robot file which is located in my Testautomation project (Testautomation/Test Sources/Example/Test Definitions/example_robot_file.robot):

*** Settings ***
Documentation     Example robot file
    
Suite Setup       Global Suite Setup
Suite Teardown    Global Suite Teardown
Test Setup        Global Test Setup
Test Teardown     Global Test Teardown


*** Comment ***
Additional comments or data. Ignored by Robot Framework.


*** Variables ***
@{EXAMPLE_OF_GLOBAL_LIST_VARIABLE}      value 1    value 2    value 3
${EXAMPLE_OF_GLOBAL_SCALAR_VARIABLE}    some value


*** Test Cases ***
Dummy TestCase Pass
    [Documentation]    This is a dummy test case that passes.
    [Tags]        dummy_pass
    [Setup]       Log    This setup overrides the global setup.
    Log    ${EXAMPLE_OF_GLOBAL_SCALAR_VARIABLE}
    Log    ${EXAMPLE_OF_GLOBAL_LIST_VARIABLE}
    Log    ${EXAMPLE_OF_GLOBAL_LIST_VARIABLE}[2]
    [Teardown]    Log    This teardown overrides the global teardown.

Dummy TestCase Fail
    [Documentation]    This is a test case that fails.
    Fail    Dummy TC
    [Tags]    dummy_fail
    No Operation
    Fail    Dummy TC


*** Keywords ***
Global Test Setup
    [Documentation]    This is the global test setup keyword.
    No Operation

Global Test Teardown
    [Documentation]    This is the global test teardown keyword.
    No Operation

Global Suite Setup
    [Documentation]    This is the global suite setup keyword.
    No Operation

Global Suite Teardown
    [Documentation]    This is the global test teardown keyword.
    No Operation

When using Python 3.8, RF 5.0.1, RobotCode - Robot Framework Support v0.83.0
The RobotCode thinks that the whole project is a large test suite, making the results unnecessarily complicated.

Plus, it somehow activates the installed sikuli which then starts to spam my root project folder with stderr files talking about not being able to get to Java. The sikuli is used in a different robot file in a different location in the same project.

Do you, guys, have any idea what did go wrong and how can I make the RobotCode extension / VSC stop acting weird?

I’ll be grateful for any response!
David

[CharlieScene (Charlie)]

Hi,

Edit : looking at your structure your run directly from TestAutomation folder, so each following folder might be considered as a suite.

You could run directly from Test Definitions, if you don’t have resource in higher folders and/or correctly set relative paths.

Also, RF doc gives this :

Robot Framework requires Python 3.8 or newer. The latest version that supports Python 3.6 and 3.7 is [Robot Framework 6.1.1]
The Robotcode v0.83 is the latest, and requires Python 3.8 which is OK, but you’re RF version does not seem to fit. Might be a hint.

For Sikuli Library be careful, I think some keywords are shared with Selenium Library (Capture Page Screenshot I think), which might be the reason of the error.

Regards.
Charlie

[Oavio (David S.)]

Hey Charlie,

Thank you for your response.

From what I know, the RobotCode v0.83 claims compatibility with RF 4+ and Python 3.8+, and I am using RF 5.0.1 & Python 3.8, therefore it should be OK.
RF 5.0.1 also claims to be compatible with Python 3.6+.

Let’s skip to your last point - the code I posted here is the code i tried to run. There’s not a single resource/library imported.

I was executing the test case via the Play button next to the test case name in VSC. I would really like to run it from the .robot file instead of the project root, but I just don’t know how. The Play button simply redirects the terminal to the root of the project and marks the whole project as a suite, while the suite is the .robot file I am clicking the Play button from.
When I execute a single test case from a single .robot file, I do not want the RobotCode to respect the content of the whole project.
There are more folders than just Examples which contain Test Definitons.

Enjoy your weekend everyone
David

[CharlieScene (Charlie)]

Hi,

Ok I see what you mean.
The launch.json file is usually set to start from the workspace folder for current work directory (cwd) :

Here you can maybe modify for “cwd”: “${fileDirname}” so only the .robot file directory will be used as the base for execution.
Again only thing I see would be dependencies to other folders or files, but you can give it a try.

Regards.
Charlie

[Oavio (David S.)]

Thank you for another hint! Unfortunately, it did not solve the whole situation.

My settings.json looks like this now:

and when I run the whole robot file using the RobotCode: Run Current from the lower part of VSC, it indeed starts only the local suite:

But when I run only a single test case from the suite, using the Play button next to a test case name, it still defines the whole test suite as the entire project folder:

With RobotCode extension disabled and Robot Framework Language Server enabled, using the Play button next to a test case name runs only the one test case as expected:

Is it possible to make RobotCode extension act as the RFLS extension when executing a single test case from a suite?

David

[daniel (Daniel Biehl)]

Hi @Oavio

Yes, RobotCode works like this, but RobotFramework works the same way
All files and directories under a root directory belong to a test suite. And yes, you read that correctly, a directory is also a test suite. Check the RF documentation for __init__.robot; with this, you can also set settings in a directory, e.g., set the same setup/teardowns on the suite or test level for all suites under this directory, etc.
Metadata defined at the root suite level, for example, can also be included in all test executions this way.
Additionally, this impacts the ID generation in reporting when you want to analyze or archive test results, or … or … or. Depending on the path or file you specify when starting Robot, it can lead to different behaviors. Internal variables, import paths, etc., also change, which impacts how you write your test cases, e.g., when you want to include resources/libraries or load test data.
To ensure consistent behavior, you should always start Robot from a root path, and if you have individual test cases, filter them with the usual Robot Framework filters --testcase, --suite, --include, --exclude. This ensures that RobotFramework always behaves consistently.

RobotCode works as described above to ensure consistent behavior. I will not change that.

You can also specify in VSCode where Robot should find your test suites. Search for robotcode.robot.paths in the VSCode settings. By default, this is always the root directory, as it seems to be in your case. I always put my tests in the tests directory.

I’m not sure what “Test Sources” and “Test Definitions” are in your setup; perhaps your directory structure is too complex, and you have multiple projects open?

But even that is possible. You can create different test profiles from a root directory using the robot.toml file and then easily switch between them in the test view. The documentation for this is still in progress, but you can check out this tutorial: https://www.youtube.com/watch?v=7Uad_250YuI

There is also a console-width setting where you can specify the width of the console. Increasing this value will allow more to fit on the screen.

The RobotFramework Sikuli Library is quite old, and there is actually a version 2 available. However, the library requires Java to initialize and expose its keywords. When RobotCode analyzes a project, it tries to learn all unknown libraries. If they throw an error, it will try again the next time it starts. Simply install Java and set up Sikuli correctly.

[Oavio (David S.)]

Hi Daniel

First of all, thank you for your comprehensive answer.
I could not find any documentation for __init__.robot - could you provide a link, please?

I’ll try to describe our project structure so that it’s less of a black box to you.

I understand that you can have a project/directory, with tests folder which contains several suites of suites, and see it as the robotcode.robot.paths, but it is not the only way of structuring the project.

To run the tests from the root directory - I can still be in the root directory and run only a single suite deep in my structure without having the suite explicitly linked to all the levels above, which makes the HTML report unnecessarily deep (even in your tutorial you have to go through two more levels to check a single screenshot - what benefits does it bring?), and that’s what I expect from the Play button.

My issue would be solved if I could set the robotcode.robot.paths as ${fileDirname}.

To the Sikuli and libraries stuff - I am not complaining about my Sikuli not working, I expect it not to work. What I don’t get is why the RobotCode even touches it when I am trying to run a test case from a test suite that does not even mention the library

[damies13 (Dave)]

Hi David,

The documentation for __init__.robot is here: Suite directories

Dave.