Debugging Robot files with Robot Code extension and remote process

Tools Visual Studio Code(ium)
1

Hello all,

First time posting here, hoping that someone can give me a hand with debugging my robot files for my specific scenario.

I am currently set in my environment to properly use the Robot Code extension. Everything works great, code completion, lookup/intellisense works after configuring resource/libraries and extension settings, albeit the extension is a bit slow for our larger project to parse everything. For my testing, I do system testing on a physical device. I have a module called run_tests.py that I execute from CLI that interfaces with this device. In this run_tests.py, it will configure the robot environment after talking with the device and launch robot to run testing on the device’s GUI/API. I can currently debug the python backend with the below launcher.json settings and this added in my main file to utilize debugpy.

#5678 is the default attach port in the VS Code debug configurations. Unless a host and port are specified, host defaults to 127.0.0.1
debugpy.listen(5678)
print(“Waiting for debugger attach”)
debugpy.wait_for_client()

I then run my run_tests.py script to start the process, it will hold for the debugger to attach. At this point, I go to the VS code debug tab and hit play. At this point the VS code breakpoints will work from the GUI from all of my sub Libraries that are utilized by my test suites. Great to step through everything in the back end.
I currently have my launcher.json file configured like so:

“version”: “0.2.0”,
“configurations”: [
{
“name”: “Python: Remote Attach”,
“type”: “python”,
“request”: “attach”,
“host”: “127.0.0.1”,
“port”: 5678,
}
]

Does anyone know how I could configure this extension for this scenario like I am for python backend to debug my robot cases/custom keywords? Would be a great bonus if I could step into the robot keywords directly to the implementation in the python backend. Would be happy if it only worked on robot side as well. Thanks in advance all.

2

He Jim,

as I wrote in the other thread, this is currently not possible.
The code on the python side (means robotcode) is already there, but I have to connect this with the VSCode side.

I’ve created an issue for this see [ENHANCEMENT] Debugger should support Remote debugging · Issue #86 · d-biehl/robotcode · GitHub

You can subscribe to this issue to get informed when it’s done.

> python -m robotcode.debugger --help
usage: robotcode.debugger [-h] [--version] [-p PORT] [-w] [-t TIMEOUT] [-c TIMEOUT] [--log] [--log-debugger] [-n] [--debug-asyncio] [--log-asyncio] [--log-colored]
                          [--log-config FILE] [--log-file FILE] [--log-level LEVEL] [--call-tracing] [--call-tracing-default-level LEVEL] [-d] [-dp PORT] [-dw] [-om] [-ol] [-og]
                          [-soe] [-- [ROBOT ARGS ...]]

RobotCode Debugger

options:
  -h, --help            show this help message and exit
  --version             shows the version and exits (default: False)
  -p PORT, --port PORT  server listen port (tcp) (default: 6612)
  -w, --wait-for-client
                        waits for an debug client to connect (default: False)
  -t TIMEOUT, --wait-for-client-timeout TIMEOUT
                        timeout to wait for an debug client to connect (default: 5.0)
  -c TIMEOUT, --configuration-done-timeout TIMEOUT
  -dp PORT, --debugpy-port PORT
                        sets the port for debugpy session (default: 6613)
  -dw, --debugpy-wait-for-client
                        waits for debugpy client to connect (default: False)
  -om, --output-messages
                        Send output messages from robotframework to client. (default: False)
  -ol, --output-log     Send log messages from robotframework to client. (default: False)
  -og, --group-output   Fold messages/log from robotframework to client. (default: False)
  -soe, --stop-on-entry
                        Stops on entry. (default: False)
  -- [ROBOT ARGS ...]   RobotFramework arguments. (see robot --help) (default: None)

this are the commandline argument of the robotcode.debugger. You can get, it if you install robotcode via pip in your remote environment, but please don’t install it in your environment where run vscode, because of conflicting robotcode python packages.

The robotcode.debugger supports already the usage of debugpy to connect with a python debugger, maybe you don’t need you run_tests.py where you start debugpy.

2 Likes
3

Hi Daniel,

Thanks for the quick response. Love your tool. It supports the pipe character separator much better than Robocorp’s extension from what I have trialed. If you have any tips for getting the environment to load quicker, let me know. It can take a good 45-60 seconds when opening VS Code and each file for your extension to parse all the .robot code for syntax/keywords. Our library is so large that sometimes I need to just restart the IDE so that it can restart the parsing.

To the main question, I can already debug the python code. For my specific situation, I must always use the run_tests.py utility to start my robot tests on my specific device under test, as I am testing physical hardware and its capabilities. The hardware must be connected to first before robot is started, which means I cannot go into any specific robot file and just press the green arrow to run a test case. I need to use run_tests.py to start my testing no matter what.

Using debugpy, I can add a listener on port 5678 and configure the python debugger as I describe above in order to attach to the python debug process. This works great for my needs. I can put breakpoints in the IDE in any python library that is called in robot and can debug that in the backend. This has nothing to do with RobotCode extension at the moment.

I want to know if I could also get support to allow debugging of the .robot files themselves via VS Code GUI, that I start robot in another process (run_tests.py from command line) and attach to it using your extension. Does that help clarify? I am not sure if we are talking about the same idea or not. Is your CLI debugger debugging python only or python and robot files? It would be great if somehow they could all link together in a debug session. For example, step into a keyword from robot that steps into a python library where that keyword is defined.

4

Ok, for the first point.

I cannot tell you why it takes so long to load your code, but here some questions and some tips.

How large is your library, how many tests, keywords, files,…? How many libraries are you using? And if you use resources, what is the file extension of the resources? And why you are using the pipe character? Which robotframework version?

And the tips:

  1. use the correct file extension
    • only files with the extension .robot can contain testcases, files with the extension .resource can only contain keywords
    • robotcode searches for files with the .robot extension in the workspace to get all tests
  2. import only that, what you are really use in that file
    • I offen see projects that contain one resource file where all imports are collected that are used in the project And this file is then imported in every test suite or resourcefile in the project
      This seems the easiest solution when you write your tests, but for parsing and analysing times and memory allocation, dependency resolution, … this is the dead.
      So only import that in your suite or resource file, what you really need, at the point where it is needed.
  3. Disable virus scanners for your project folder

To the main point:

With the next big release of RobotCode there is also support for remote debugging, means start your tests on another machine and connect it with your local debugger (VSCode). My plan is, that I release it at the same time the new RobotFramework version is released. Means in the next 2 or 3 weeks.

I don’t know exactly what your run_tests.py does and why it is needed, but I assume that you need to initialize your hardware in someway before you start your tests. So my question, what exactly does your run_tests.py and why you are not run your tests with the robot command?

Just an idea:

Did you hear about robotframework listener interface? with this interface you can attach you in the process of robotframwork test execution.
If you have something to do before the tests/suites starts, just implement this in the __init__ or the start_suite method and if you have something to do after the tests is running, implement the end_suite? or close` methods.

then you can start your test with something like:

robot --listener YourListener <whatever arguments you need>  test_suite.robot

And if RobotCode debugger is ready you can just replace this call with:

robotcode.debugger -w -soe -- --listener YourListener <whatever arguments you need>  test_suite.robot

and with debugpy enabled like this:

robotcode.debugger -w -soe -d -- --listener YourListener <whatever arguments you need>  test_suite.robot

And with a launch config like this:

    {
      "name": "RobotCode: Remote-Attach",
      "type": "robotcode",
      "request": "attach",
      "connect": {
        "host": "localhost",
        "port": 6612
      },      
    },

you then can connect you local VSCode to the remote running robot instance to step through your test cases.

Alternatively, maybe you can also use the internal API of the RobotCode debugger like:

asyncio.run(
        run_robot(
            args.port,
            robot_args,
            args.bind,
            args.no_debug,
            args.wait_for_client,
            args.wait_for_client_timeout,
            args.configuration_done_timeout,
            args.debugpy,
            args.debugpy_wait_for_client,
            args.debugpy_port,
            args.output_messages,
            args.output_log,
            args.group_output,
            args.stop_on_entry,
        )
    )

does this sounds good to you?

5

Thanks for the response. For the side point questions:

  • Our testing library has 13147 test cases total
  • 265 files encompass our tests and keywords and we have them labeled all with the .robot extension.
  • We have at least 292 keywords that are made accessible to our suites that are common. We are not importing all keywords to each file, we do have some separation here to import what is needed.
  • We are using .robot for each file, not using .resource extension at all
  • We are using the pipe character as the separator format described here: Robot Framework User Guide

For an easy fix, I can see about renaming the keyword only files with the .resource extension to see if that gives us any gains. We are running in Ubuntu 20.04, I think antivirus should be fine. Not sure what is happening different on robocorp’s language server but theirs is a great deal faster at loading the workspace. We like your extension better though because yours handles the pipe separator format. Let me try the low hanging fruit of renaming to .resource files.

For the main point, that is excellent you are planning to release remote debugging support. You are talking about debugging of the .resource/.robot files themselves, right? To debug keywords/test cases? Or are you talking about Python Libraries only? Based on your listener example, it sounds like you are implementing remote attach for the test cases themselves. Will this require the latest version of robot framework to use?

We are actually using the listener interface to connect with TestRail, we are familiar with it. The run_tests.py is connecting to the physical hardware to get device characteristics before we start our tests. This will also set tags in robot framework based on the device we are testing so certain suites will be excluded on a per device basis. I see what you are saying though, to implement a listener that handles our setup and we can call robot directly so that we are not using this run_tests.py utility to start robot. Let me maul that around and think about it over the weekend, especially your idea about using the listener. If your new feature works all the same, I probably could wait for that as well if it is in your roadmap.

6

yes, that’s what I’m talking about. And if you enable python code debugging, then then python debugger attaches automatically to the run.

which version of RobotCode do you use?

Do you use python virtual environments and if yes where is your venv saved?

There is a RobotCode setting robotcode.workspace.excludePatterns where you can exclude folders in wich robotcode searches for .robot and .resources file.

You can also disable codelenses for references and
and you can turn off the full workspace diagnostics.

Search for this two settings:

robotcode.analysis.referencesCodeLens
robotcode.analysis.diagnosticMode

robotcode.analysis.findUnusedReferences (depending if you use a preview version of RobotCode)

7

Understood. Thanks for explaining these features.

  • We are running RobotCode v0.12.1 with robot framework 5.0

  • Yes we are using a python venv - the venv is one directory higher than the repo that has our robot files that is shared across multiple repositories. Where does robotcode start looking for robot files? Does it start where the venv is defined?

  • All of your suggested settings were already disabled that you referenced above. Maybe changing the files to .resource and using robotcode.workspace.excludePatterns will support efficiency in the file parsing.

8

No it start from the project root.

9

Understand. Thank you.

By the way, do you have a recommended way to support the same repository, but cloned more than once? The purpose would be IDE use in code reviews

For example, if I have a repository that I have robotcode configured with and it is called system_testing_tools.
I want to clone system_testing_tools again and call it, system_testing_tools_Merge_Requests.

If I do this, when I am in my system_testing_tools_Merge_Requests it will jump to robot and python definitions in my original system_testing_tools due to the paths I have setup for Robot code->Robot:Python Path. Is there a way to fix this or is this a limitation?

Here is my python path as an example:
“robotcode.robot.pythonPath”: [
“./lib”,
“./resources”,
“home/user/test-automation/system_testing_tools/lib/py”,
“/home/user/test-automation/system_testing_tools/py/main”,
“/home/user/test-automation/system_testing_tools/py/dut”,
“/home/user/test-automation/system_testing_tools/lib/py/rigs”,
“/home/user/test-automation/system_testing_tools/lib/py/dut”,
“/home/user/test-automation/system_testing_tools/lib/py/main”,
“/home/user/test-automation/system_testing_tools/lib/rf”,
“/home/user/test-automation/system_testing_tools/lib/vars”,
“/home/user/test-automation/system_testing_tools/lib/py/main”,
“/home/user/test-automation/system_testing_tools/tools”,
“/home/user/test-automation/automation_testing_common/lib/py”,
“/home/user/test-automation/automation_testing_common/lib/py/dut”,
“/home/user/test-automation/automation_testing_common/lib/py/rigs”,
“/home/user/test-automation/automation_testing_common/lib/py”

10

I think using relative paths instead of absolute is the solution here.

“robotcode.robot.pythonPath”: [
“./lib”,
“./resources”,
“./lib/py”,
“./py/main”,
“./py/dut”,
“./py/rigs”,
“./lib/py/dut”,
“./lib/py/main”,
“./lib/rf”,
“./lib/vars”,
“./lib/py/main”,
“./tools”,
“/home/user/test-automation/automation_testing_common/lib/py”,
“/home/user/test-automation/automation_testing_common/lib/py/dut”,
“/home/user/test-automation/automation_testing_common/lib/py/rigs”,
“/home/user/test-automation/automation_testing_common/lib/py”

I don’t know why you have so many paths in you python path, if you use python packages (a python package is simply a folder with an __init__.py) it is enough to put one path and in your robot files you only need to use something like:

Library  dut.whatever.tools
Library  rigs.whatever.rigs.tools

The project automation_testing_common i think are libraries that are shared accross different projects?
If you have such libraries you can make a python package of them and install it to your project like a normal python dependency, maybe this is interessting for you: RoboCon 2022 - 1.08 Project and package management: Poetry for robots - YouTube

11

Hello,

Just to let you know, the new RobotCode version 0.14 is out. The most important feature for you is probably the remote debugging.

On your test machine you can install RobotCode via pip.

pip install robotcode

When that is done, you can call robotcode.debugger --help and look at the help.

To start a test, instead of robot, do this:

robotcode.debugger -b <ipaddress> -w -- <robotparameters> .

The -w means that the test starts only when a debug client is connected.
Everything after the -- are the parameters you would pass to robot.

In VSCode you can now create a new launch configuration “RobotCode: Remote-Attach” which looks like this:

{
      "name": "RobotCode: Remote-Attach",
      "type": "robotcode",
      "request": "attach",
      "connect": {
        "host": "localhost",
        "port": 6612
      },
      "attachPython": true,
      "pathMappings": [
        {
          "localRoot": "${workspaceFolder}",
          "remoteRoot": "."
        }
      ]
},

Hope it works!

Give me feedback if you have any questions.

1 Like
12

I will check this out and get back to you, thanks Daniel!