The application where I need to do test automation, unfortunately prevented programmatic testing like using Appium. The application recognizes if you simulate pressing the screen using any programmatic method as such and can’t be used.
So I need a physical device that simulates a real finger and performs the functions I want. I found automatic screen clickers that are otherwise suitable, but they work with the device’s own keys and can’t be controlled from any kind of custom scripts or frameworks. The purpose is to integrate the device with the Robot Framework and use for example Serial Port communication to control the device automatically only and when needed.
The problem with that solution is that the plan is to use this method simultaneously for many devices, maybe even dozens. At the moment I have one arduino-powered device for this, but I need a reliable, long-lasting and compact device that can be used repeatedly and easily for many devices. The real use case is really only one page, more precisely the login page. That’s why such a small and simple solution is suitable.
Although, that suggestion of yours seems interesting and I will definitely look into it! Who knows, maybe I’ll try it.
I would imagine with the solution I suggested you remove the base plate from the CNC device and replace it with your own “base plates” that you make from plywood / mdf or similar wood that would have a cut out for the type of device you are testing, the cut out should hold the device with the screen at a known relative height (level with the base plate?) and stop the device sliding around maybe with wedges/chocks/foam pads. Then it would be a matter of place the device in the slot test, remove and replace with the next device.
I also imagine the same could be achieved by modifying a 3d printer (either a Cartesian or Delta)
I also remember there was a thread a while back where someone was asking about controlling a cnc device from robot framework, If I remember correctly they were using a modified cnc device with the base plate removed for testing touchscreen monitors on kiosks
Basically you need a device that can move the stylus in 3 planes (x axis, y axis & z axis) rather than developing something, I’m suggesting to adapt an existing mass produced device that already does this, CNC & 3d Printers are already produced in higher volumes than any dedicated hardware test tool would ever be so a dedicated hardware test tool would be more expensive due to the low volumes of production.
Hopefully that explains why my answer is not quite what you were expecting,
How is the application (under test) run or where does the application run from? Because I’m wondering if you can just run it under a virtual guest OS and use the local virtual display on the host to automate the application. That should work unless the application detects for virtual OS environment and won’t run under such.
Another approach could be to remotely access the application over VNC or (RDP) remote desktop (for Windows), or x11 forwarding (if is Linux based GUI application). Using any of these approaches, you can do the UI automation on the remote client side rather than local to the (server) application’s desktop. Thus the application would have no knowledge it is being automated. This does have the trade-off that you can’t use direct UI automation tools like Appium because those will need to run from the application host. You can run tools however that use OCR and image recognition like Sikuli and others remotely, and work off a fixed specified/controlled desktop/window screen size. Those kind of tools work off a screen, it doesn’t matter if the application is hosted locally or remotely.
Just some thoughts in case the hardware based approach is tricky to do or expensive.
And for simultaneous use, except for case of say key presses or taps/clicks or finger movement across multiple devices at the same time, you could have a big desktop screen remoting to multiple instances of the application (over VNC or RDP) and handle one screen after another sequentially. There might be options to do it in parallel but you’d have to find some additional tooling for that.
I’m trying to do exactly what you described, executing an integrated test on multiple workstations simultaneously through vnc connections.
Do you know if Robot Framework has the capability to run tests on devices connected through vnc or has a compatible library that is able to do that?
I haven’t been able to find any resources about vnc connection with Robot Framework. I would love to know if you have a better insight or experience with this application of the framework.
I don’t think there’s any RF library or article/blog post that covers the VNC/RDP orchestration from within an RF test. You’d lilkely have to write your own code as RF keywords or create your own RF library.
But there are (python?) libraries out there for working with VNC and RDP I would think, so you can adapt those to work with RF.
For what you want, I don’t think there’s a pre-built solution, so you have to build it yourself. I would first figure out how you would do this test execution on each workstation, executing it remotely, then figure out how you can simultaneously execute across the workstations remotely, then finally figure out how to wrap this deployment execution with RF. I wouldn’t start from the beginning trying to get it all built and running from RF. Taking it as a more peace meal approach allows it to be flexible and not dependent on RF. You can simply make RF the “orchestrator” for the test but not the base component itself.
If you want to make it all within RF, I would first start trying to do for one workstation then see how you can scale it to multiple simultaneously. You may need pabot (the parallel execution of RF), something like that to assist in that.
It would be easiest if the tests run on the workstations themselves locally (selenium, appium, or custom code) and you only trigger them remotely via RF, and where VNC/RDP comes into play is to trigger/keep the “session” open/logged in for the automation to run. It could be more complicated if you have the automation test running remotely, and using the VNC/RDP screen as your basis for UI automation via image recognition tools (e.g. Sikuli).
Wow! Thank you for such a fast and detailed reply, David!
It’s sad to hear that RF doesn’t natively support VNC connection. My use case requires the tests be run remotely without running them on the system under test but on a separate workstation, which seems too specific in a direction that is not addressed by most test automation tools out there.
I am also considering building my own framework from scratch using python libraries, so your suggestions might actually make a better solution. Utilizing some of the features of RF and its libraries and mixing them with other libraries might reduce the overall workload. I will need to dig a bit more with your suggestions in mind.
I really appreciate your response and sharing of your ideas!
I hope you have a great rest of the day!
While not directly a VNC/RDP library, you can launch the client with command line options to get it to connect and login, you can then use SikuliLibrary or ImageHorizonLibrary to control the VNC/RDP session, so it’s not impossible just a bit of a clunky workaround.
Another option is to connect the VNC/RDP session and run robot framework inside the session, this works well if that’s an option.
I’ve actually looked into the idea of creating an RDP library for robot framework, I spent about 30-40 hours investigating what options there are
unfortunately what I found was all the python modules that claim to do RDP are very incomplete and don’t do much more than open a connection and send keystrokes blindly, no window interaction or verification
I found an opensource RDP client written in C that also supports VNC that’s quite complete but with limited documentation on how to call the api, but this looked like the best option, and was the way I planned to move forward but it’s on my back burner as it’s a nice to have not an urgent need for me.
Any research into RDP via remote library interface via another language? Maybe .NET/C# or Java? I’m thinking other language platforms might have better support of RDP. But of course, there is the matter of whether one is familiar with said language to code up the wrapper library to run over remote library/server interface.
Interesting idea, I didn’t see anything that was open source in those languages, but didn’t look for them either, but that could be a good option if you can find a suitable library/tool.
