import python packages

Hallo,

I have been working with Robot Framework for a short time.
In Python, I installed two additional packages: pyvisa and pyserial.
These are located in the directory: C:\Program Files (x86)\Python39-32\Scripts.
This directory is assigned to the environment variable (system variable) “Path”.
Now I want to use (import) these two packages in Robot Framework.
I have created a Robotor: Basic Python Template.
At the beginning of the file task.py I import all the python-packages I need such as

import os
import sys
import serial
import pyvisa

When I run my Robot, the message "No module named serial" or "No module named pyvisa" appears.

My question is how to solve this problem?

Thanks a lot

Run what kind of file? A .robot?
Or are you only using Python?

Do you have a small example on how you use the library you created?

1 Like

Hello helio,

Thank you for your answer. In my example I am using a pure Python robot task.py (Basic Python Template) - not an A.robot.
In the test case I am trying to connect to an external device using a Modbus communication over a COM port.
In my planned test case several Python scripts have to be imported. But that’s not really the problem. Because it’s about importing installed Python packages.

I show a very simple, modified Python script in which I include the Python packages. You can already see from the Python code that
the Robot Framework does not recognize the two Python packages “serial” and “pyvisa” - see line 6 and 7.

After running the python robot, the following message appears in the terminal:

But with the command “pip freeze” I can show that the two packages are installed.

Thanks

Sometimes, we have more than one Python installation and we are not running the expected one.

You can try to install those modules again, but be sure to use the exact Python:

python -m pip install -U --force serial pyvisa
1 Like

Hello Helio,

thank you for your answer. I have a question. Could it be that the problem lies both with using the Python interpreter and with accessing the environment variables.
When I write a Python script and run it in VSCode (not in Robot framework),
the Python interpreter assigned to the “Path” system variable is used: C:\Program Files (x86)\Python39-32.
On the other hand, if I start a Python script in Robot-Framework, then Robot-Framework uses a Python interpreter from the directory: C:\ProgramData\robocorp\ht.

The same is with the environment variables. A normal Python script, which is not executed in the Robot Framework, accesses the directory
C:\Program Files (x86)\Python39-32\Scripts, which is also mapped to the “Path” environment variable.
The situation is different in the Robot Framework. Here folders are accessed which are located in the directory C:\ProgramData\robocorp\ht\ for example
C:\ProgramData\robocorp\ht\368d3ba_b1f3c24_d4ec2478\Scripts.

Here my question: Is it possible to change the Python interpreter as well as the access to the environment variables in Robot Framework?
I would like Robot Framework to use the interpreter from C:\Program Files (x86)\Python39-32\ and the environment variable Path with
C:\Program Files (x86)\Python39-32\Scripts.

Thanks

Maybe someone can help you with VSCode configuration.
I only use RIDE or robot from command line.

You can create a .cmd file that sets the PATH and then call VSCode, or to run robot.

1 Like

I am also having this problem. I believe I have identified the root cause, but don’t know what to do about it.

Here’s the summary

  • I also use PySerial
  • PySerial defines its module namespace as “serial”. This has been a long-time standard and is apparently reserved on PyPi.
  • It appears that RobotFramework includes its own package “serial”, so there is a namespace conflict. “import serial” does not pull in PySerial, but something else.

This problem first arose in the PySerial community when someone posted another package called “serial”:

I have opened up the specific Python interpreter that is used by RobotFramework, and have tried to uninstall “serial”, but that is not the name of any installed package (see below.)

I have tried to figure out where the “something else” is coming from but am not a Python package expert.

Can anyone help? Any idea where this namespace pollution is coming from? Can the offending package be safely removed from the RobotFramework Python environment?

At the moment, if I am understanding the situation correctly, there is no way to test an application that uses the (pretty much ubiquitous) PySerial library for serial I/O.

Thanks,
Brad

C:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988>.\python -m pip list
Package                            Version
---------------------------------- -----------
async-generator                    1.10
attrs                              22.2.0
beautifulsoup4                     4.11.1
cached-property                    1.5.2
certifi                            2022.12.7
cffi                               1.15.1
chardet                            3.0.4
charset-normalizer                 3.0.1
click                              8.1.3
colorama                           0.4.6
comtypes                           1.1.11
cryptography                       37.0.4
decorator                          5.1.1
defusedxml                         0.7.1
dnspython                          2.3.0
docutils                           0.19
et-xmlfile                         1.1.0
exceptiongroup                     1.1.0
exchangelib                        4.9.0
fire                               0.4.0
fonttools                          4.38.0
fpdf2                              2.6.1
furl                               2.1.3
future                             0.18.3
graphviz                           0.13.2
h11                                0.14.0
htmldocx                           0.0.6
hubspot-api-client                 4.0.6
idna                               3.4
importlib-metadata                 4.13.0
iso8601                            1.1.0
isodate                            0.6.1
java-access-bridge-wrapper         0.9.5
jsonpath-ng                        1.5.3
jsonschema                         4.17.3
lxml                               4.9.2
mss                                6.1.0
netsuitesdk                        1.24.0
notifiers                          1.3.3
ntlm-auth                          1.5.0
O365                               2.0.25
oauthlib                           3.2.2
openpyxl                           3.0.10
orderedmultidict                   1.0.1
outcome                            1.2.0
packaging                          21.3
pdfminer.six                       20201018
Pillow                             9.4.0
pip                                22.3.1
platformdirs                       2.6.2
ply                                3.11
psutil                             5.9.4
pycparser                          2.21
Pygments                           2.14.0
PyJWT                              2.6.0
pynput-robocorp-fork               4.0.0
pyotp                              2.8.0
pyparsing                          3.0.9
PyPDF2                             1.28.6
pyperclip                          1.8.2
PyQt5                              5.15.8
PyQt5-Qt5                          5.15.2
PyQt5-sip                          12.11.1
pyrsistent                         0.19.3
pyserial                           3.5
PySocks                            1.7.1
python-dateutil                    2.8.2
python-docx                        0.8.11
python-dotenv                      0.21.1
pytz                               2022.7.1
pytz-deprecation-shim              0.1.0.post0
pywin32                            303
pywinauto                          0.6.8
PyYAML                             5.4.1
requests                           2.28.2
requests-file                      1.5.1
requests-ntlm                      1.1.0
requests-oauthlib                  1.3.1
requests-toolbelt                  0.10.1
robocorp-dialog                    0.5.3
robotframework                     6.0.2
robotframework-pythonlibcore       3.0.0
robotframework-requests            0.9.4
robotframework-sapguilibrary       1.1
robotframework-seleniumlibrary     6.0.0
robotframework-seleniumtestability 2.0.0
rpaframework                       19.4.1
rpaframework-core                  10.1.0
rpaframework-dialogs               4.0.1
rpaframework-pdf                   5.0.1
rpaframework-windows               6.2.0
selenium                           4.5.0
setuptools                         66.1.1
simple-salesforce                  1.12.3
six                                1.16.0
sniffio                            1.3.0
sortedcontainers                   2.4.0
soupsieve                          2.3.2.post1
stringcase                         1.2.0
tenacity                           8.1.0
termcolor                          2.2.0
tqdm                               4.64.1
trio                               0.22.0
trio-websocket                     0.9.2
tweepy                             3.10.0
tzdata                             2022.7
tzlocal                            4.2
uiautomation                       2.0.17
urllib3                            1.26.14
webdriver-manager                  3.8.5
wheel                              0.38.4
wrapt                              1.14.1
wsproto                            1.2.0
xlrd                               2.0.1
xlutils                            2.0.0
xlwt                               1.3.0
zeep                               4.2.1
zipp                               3.11.0

C:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988>
C:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988>.\python -m pip uninstall serial
WARNING: Skipping serial as it is not installed.

And just to be clear, adding the PySerial package does not help, (I’ve tried removing and re-installing it several times):

C:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988>.\python -m pip install PySerial
Requirement already satisfied: PySerial in c:\programdata\robocorp\ht\lib\site-packages (3.5)

C:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988>.\python
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:51:29) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import serial
>>> SerialPort = serial.Serial('COM5',19200)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'serial' has no attribute 'Serial'
>>>

WORKAROUND FOUND, MAYBE?

I have yet more info.

Short Summary
I have things working now, but I’m not exactly sure why.

I found a problem with where Python was looking for its libraries - PySerial was getting installed but wasn’t in the search path.

I worked around it by updating PYTHONPATH, and things started working.

But then at some later point, I tried removing my PYTHONPATH mod, and it was still working. It’s possible that I changed which Python interpreter I was using? But not sure. I did at one point go into the Robot Framework Language Server extension settings, but I don’t think I actually changed anything.

At some point, VS Code ceased trying to run “conda activate” at the start of each test session, although things started working before that.

So it’s working now, and I don’t know why.

Full Details

What’ I discovered was going on is that PySerial is getting installed into

C:\ProgramData\robocorp\ht\lib\site-packages\serial

as can be seen here:

C:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988>.\python
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:51:29) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import serial
>>> print(serial.__dict__.keys())
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__path__', '__file__', '__cached__', '__builtins__', 'absolute_import', 'sys', 'importlib', 'serialutil', 'io', 'time', 'unicode', 'basestring', 'iterbytes', 'to_bytes', 'XON', 'XOFF', 'CR', 'LF', 'PARITY_NONE', 'PARITY_EVEN', 'PARITY_ODD', 'PARITY_MARK', 'PARITY_SPACE', 'STOPBITS_ONE', 'STOPBITS_ONE_POINT_FIVE', 'STOPBITS_TWO', 'FIVEBITS', 'SIXBITS', 'SEVENBITS', 'EIGHTBITS', 'PARITY_NAMES', 'SerialException', 'SerialTimeoutException', 'PortNotOpenError', 'Timeout', 'SerialBase', '__version__', 'VERSION', 'os', 'win32', 'serialwin32', 'Serial', 'protocol_handler_packages', 'serial_for_url'])
>>> print(serial.__path__)
['C:\\ProgramData\\robocorp\\ht\\lib\\site-packages\\serial']
>>> print(serial.__file__)
C:\ProgramData\robocorp\ht\lib\site-packages\serial\__init__.py

but that folder wasnot in the PYTHONPATH when RobotFramework runs. This is the PYTHONPATH that was being printed out at the start of each run.

PYTHONPATH:
          c:\Users\brad\Nextcloud\Documents\_TMT\Software\m1cs\UpperSegBox\UsebGui
          c:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988\lib\site-packages\win32
          c:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988\lib\site-packages\win32\lib
          c:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988\lib\site-packages\Pythonwin

I notice in the VS Code terminal window that there is a call to “conda” to start the process, which fails:

PS C:\Users\brad\Nextcloud\Documents\_TMT\Software\m1cs\UpperSegBox\UsebGui> conda activate c:\ProgramData\robocorp\ht\3ed6a10_b1f3c24_691db988
conda : The term 'conda' is not recognized as the name of a cmdlet, function, script file, or operable program. 

There is no conda.exe on my system, other than in a folder where I do some Julia programming, and it is not in my path. There is no Conda.exe is the robocorp tree.

The workaround fix would appear to be to augment PYTHONPATH to include C:\ProgramData\robocorp\ht\lib\site-packages by adding an “env” clause to my launch.json:

  "program": "${file}",
  "env": {
    "PYTHONPATH": "C:\\ProgramData\\robocorp\\ht\\lib\\site-packages;${PYTHONPATH}"
  }

The question is why conda is getting called, or maybe why it isn’t included in the robocorp tree? Or maybe this stuff should be being set up in the .bat file that runs just after Conda fails?

In any event, when I made that mod, things started working.

And then later when I removed that mod, they continued to work.

Very baffling, but perhaps this will help someone if they run into the same problem.

1 Like

I now understand the conda / no-conda thing.

If I click here to run the task, it tries (and fails) to activate a conda environment:

image

If I instead right-click the check mark and select “Run Test”, no conda.

[Imagine that I had pasted here an image with the green check-mark circled instead of “Run” circled. But the forum won’t let me since I’m a new user.]

I imagine that the difference between these two is significant somehow.