ImageHorizonLibrary

Hello experts, maybe it has been asked before but could not find solution.
I have a very simple script on mac OS that uses ImageHorizonLibrary. This is first try out on using image comparison library for future possibilities of test automation. Script should try to click on search textbox on system settings window which is open. It throws error “ImageNotFoundException”.

refer the project itself for details,

ImageNotFound exception is thrown when the screen does not contain your reference image.

Without seeing your actual desktop and the reference image, its pretty hard to say what goes wrong.

Do you mean this?


I tried with other window as well but same error.

Thats a picture of your desktop. Where’s the reference image ?

Search. I have placed it in images folder “Search.png”. This Search text box is available in system settings window.

that image doesnt match because it has the blue edge around it while the original desktop image doesn’t have one. Lowering “confidence” might work but still, its not exact match and that reference image should be updated.

ok got it. I updated the image, matching exactly in required window but no luck.

Did you allow/modify the access controls (that mac should hint about on your initial run) ? If not, you get ImageNotFoundException …

Hi Jaggs,

A tip for tuning images for image recognition libraries like ImageHorizon and SikuliLibrary, is to make your reference images smaller to give you a better chance at a match without having to lower the confidence level.

As an example with your reference image:
image

Any variation in the length of the field will cause a no match, likewise and variation in the distance from the field to the pane frae (black vertical bar at the right of the reference image, will also cause a no match

I would use a reference image like this as you’ll get a much better match rate:
image

notice I don’t include the round corners, nor the full width of the field, just enough to uniquely identify the place on the screen you want.

Also note with macos, if you take the screenshot on a machine with a retina display it won’t work on a non-retina display (so take that into consideration if you plan on running the tests in a Ci environment (the same applies for screen scaling in Windows and Linux, a retina display is just 200% screen scaling)

Hope that helps,

Dave.

In SikuliLibrary we have a keyword to set the path where the images are located. Maybe ImageHorizon have one too.

You should try to use the full path to your image, because I believe that is the meaning of the error.

I read the code, if the reference image would not be found, exception would be different …

Anyway, i made a dummy project and only way to get the same exception where:

  1. Image was not really present in desktop.
  2. Didnt allow the app running python (in my case ghostty & iterm) to record screen and access accessibility features (for mouse interaction)… Worth to point out, just clicking allow at this point was not enough, one had to go to system settings and toggle the checkbox and for both options and then restart the application running robot.
  3. reference image was way too different than what the desktop has …

Im putting my 2 cents on step 2…

2 Likes

@HelioGuilherme66 , FYI - Image Horizon gives a InvalidImageException: Image path not found: error if the path is not found.

Also Jaggs put the path in as an argument to the Library import (first screenshot), this is the way Image horizon likes it, but there is a keyword as well if you need to change it during the run.

1 Like

Btw is the eficode repo the main for ImageHorizon ? They (eficode) had this on their “testomat” page which i think is dead now. And that repository hasnt seen any commits in 4 years. Same faith happened to some Siili/SAlabs projects…

And reason why I’m asking is that since IHL uses pyautogui under the hood, maybe suggesting better keyword library would be a good thing

I would love to know the answer, I submitted a few bug fixes for IHL a few years ago and nothing has happened with them, I don’t know who is the maintainer, I tried to contact eficode about this library but got no response.

It works as is for me for my needs, but I have a long term plan for RFSwarm to move from tkinter to a web based UI so I can use browser library for my UI tests.

Thought of taking a shot, I tried calculator app on Windows machine vs Mac. But even here , hit by roadblock ImageNotFoundException err.

If by happenstance, your windows experiment was via jenkins, Windows also has similar access restrictions depending on how jenkins node was started

Two points on that:

First, keep in mind that the Click keyword does not have a timeout.
It expects the screenshot to be present right now and throws an exception immediately.

I have written a resource with lots of user keywords which encapsulate waiting and clicking, for example Wait For And Click uses Wait For to save the matching location into a variable and then calls MoveClick. MoveClick takes the coordinates as argument, uses Move To to move the cursor and Click to perform the click action. And so on…

All these keywords can be parametrized with a timeout which is used by Wait For.


Secondly, there is an unfortunate dependency trap:

Pyscreeze is the sublibrary that Pyautogui uses to recognise the images.

In Pyautogui’s setup.py, Pyscreeze is only written like this:

pyscreeze>=0.1.21

It is therefore not pinned, and pip installs the most recent version.

PyScreeze versions from 0.1.30 have an annoying bug that has obviously not been fixed with the release of Pyscreeze 1.x either:

The bug causes an exception when comparing images. This is propagated in a way that the image horizon library exits the entire wait process (wait for) and immediately throws an error.

There is a trick: if you install 0.1.29 before installing the ImageHorizonLibrary , the dependency is already fulfilled and it is happy with this ‘old’ - but functioning - version.

3 Likes