I understand that there must be an architecture below that supports multiple tasks. For example, I need to run a robot 100 times that extracts data from an API (first task) and processes it (second task); So I would like a task to run on a machine type (with low resources) and the processing task to run on a high CPU machine. Is it possible to configure robot framework (or RPA framework) to support this functionality? What type of architecture is recommended to work on and what tools could be used?
1 Like
Hi @amanosalva,
welcome to Robot Framework Forum!
I am not sure, if there is a best practice, yet, but I share my current architecture for such kind of tasks.
I would not let my software robots organize their environments. Look at each task of the robot as a software service. You usually have some kind of orchestrator that defines in what environment a service has to run (low resource environment, high resource environment).
In modern software engineering, you have plenty of good “orchestrators” running under different names: CI/CD, job scheduler and so on. If you are developing with Robocode LAB the fastest way to orchestrate so called “actitvties” is using Robocloud for orchestration. But you might as well use GitLab, Jenkins, Rundeck or what ever job scheduler you like, but then you need to take care about shareing data between jobs yourself.
I spread tasks of a software robot among different jobs or stages letting GitLab or Jenkins decide which task to run next. Those tools also decide which environment to use.
You probably do not need to look for an architecture, you might as just start with the architecture you have available and see how it is going. We started with virtual machines and dedicated server and moving more and mroe in to on-demand enviroments in cloud.
Regards,
Markus
Split the work to multiple stages…
One step is collecting data, once you have it acquired, you can then place it on some persistent storage with status that indicates that it’s newly added. Rince and repeat as many times as needed.
Then you have another step that fetches a single item from that persistent storage and you process it. If there are failures, you can then associate a failure state to that item so that you can later check what is the issue and maybe retry or do manual processing. If things work fine, you can then mark that entry as processed.
Robocorp most likely has similar “work queue” implementation but I’m not sure if they share it as foss library. Checking out their docs might be worthwhile.
Just couple of extra lines about the Robocorp part…
- Robocorp has a cloud “orchestrator” (Robocloud) with an extensive free tier - easy to check it out
- As said, you can configure processes that contain multiple activities and each activity can run in a separate environment (on-prem computer, cloud container). Each activity also has a work queue that provides load balancing if needed
- “Work items” are used to carry data between activities
- General purpose data storage is going to be available as part of the Robocloud in near future
RPA Framework is sponsored by Robocorp, but nothing requires you to use Robocorp’s orchestrator or development tools. E.g. Secret Management -library has an extension mechanism that allows anyone to contribute new backends to store secrets - but ofc there is a backend that makes it easy to utilise Robocloud’s Vault 
3 Likes