i’m trying to find the proper Library that will help in comparing 2 images.
the scenario:
i will take a screenshot of an element
do an action which change the element image
take another screenshot of the element
try to compare between the 2 screenshots
I have looked over different solutions:
but for each of them, there are multiple steps for installing the dependencies and tools in order it to work ( and it is different for windows/ mac/ linux)
is there a generic / automatic way to compare images / screenshots on Robot Framework?
this is something that i really need in order to proceed in our company automation.
I created this smaller Image Comparison Library yesterday
If offers basic image comparison options and some simple masks.
No more dependencies with Tesseract, but because of that no Text Recognition is supported.
Maybe interesting for other users as well.
It’s still in a very early development stage
regardless of pass or fail return a difference score (0 being no similarity and 100 being 100% match) then people could do something like:
${Status} ${Score}= Run Keyword And Ignore Error Compare Images Reference.jpg Candidate.jpg
IF ${Score}>${ArbitraryNumber}
Do Something Keyword
ELSE
Run Some Other Keyword
END
Allow user to input a difference tolerance or score needs to be greater than to pass e.g.
Great idea @damies13 .
And both pretty simple to implement, so I’ll definitely look into it.
I think i’ll provide the possibility to set the threshold as a library argument and as a keyword argument.
Shall I return the similarity score for the Compare images keyword, or do you prefer a separate one , like Get Similarity ?
I don’t actually have a preference, It’s your library, so do it how you think is best, I just thought it would be useful.
If someone only wants the Get Similarity functionality they can just set the threshold to 0 so it never fails, but I guess providing both options wouldn’t be hard either.
Having the possibility to set the threshold as a library argument as well is a good idea
It might be useful to have keywords like Get Threshold and Set Threshold so the default value can be changed throughout the test, something like what SeleniumLibrary does with the selenium timeout. The same with the other kwargs, I’m guessing you set them as class level variables so it should be easy to implement the setter and getter functions.
But… the installation itself is probably not the problem, is it?
For me, the question is still open as to what exactly you want to achieve with the image comparison. Can you describe more precisely what your application is?
try to compare between the 2 screenshots
If you have changed the reference image meanwhile, the comparison will fail. What is your expected result? Do you want to measure the differing pixels?
Hi Simon,
I’m trying to compare between 2 elements and see if there is a change (on some steps the comparison should be the same & other steps it should be different)
i need to test when the elements are different/ equal during automation
Hi Many,
First thanks for your solution.
I used the Library but I had an ERROR "No OpenCV Image could be created for file c:\Users..\test1.jpg . Maybe the file is corrupt?
any help please ?
There is a Keyword Get Text From Document which will allow you to retrieve the text via OCR.
The text will be returned in a List which you could compare afterwards.
I have to say though, that a lot of fake texts are detected in an image, depending on the settings.
You need to have Tesseract installed to use that Keyword, you can find intructions in the GitHub repository
Thank you many. I need help in comparing two full images and sometimes finding template inside a image without anti aliasing. Is there any way to achieve this? Both images looks same but showing differences some time because of anti aliasing i think.
There is the Keyword Image Should Contain Template which you can use to search for a smaller image inside a bigger image.
The Keyword has a parameter “threshold” which allows you to define how “equal” the images should be. The value is between 0.0 and 1.0.
Default is 0.0 which is an exact match. Higher values allow more differences.
Start with very small values (like 0.01) and increase it in small steps until the comparison matches.
I see you have the show_diff which “shows differences in black and white”, I gather that’s black where differences are and white for the same?
When i played with ImageMagick years ago, it had an option to display an image (the original I think) with a red overlay (~40 transparent) over where the second image was different.
Could you add something like this, I guess the black differences can be used as a mask for the semi transparent overlay?
That was really useful for me when identifying changes for the project i was working on, so I imagine it would help others.
First thanks for this useful library and maintaining it
I’m using Compare Image Keyword, running tests on my computer (Win11) and on Docker.
I have comparison of images from Capture Element Screenshot, which are not full screen nor big images. They may contain some text sometimes.
My question (as you obviously know how the threshold works) is that between the 2 running platform I have to set a different threshold (not a big difference, let say 0.04 and 0.08).
I’m using same screen size, same color depth and xvfb on Docker.
So I’m wondering (also reading previous post on anti aliasing) if you have hints on what could lead to this difference.
Good idea @damies13 , I wasn’t so happy with the current visualizations of differences anyway and your suggestion might improve that. Will create an issue for it and look into it.
Good question about the root cause.
Typical reasons could be a different resolution, different anti-aliasing, compression artifacts because screenshot was saved as jpg.
I even saw cases where the screenshot was taken before the rendering worker of the browser was fully done which resulted in a visual difference.
Can you see some actual differences between the pictures?
Is your screenshot a jpg or a png?
I strongly suggest to use png as there are no compression artifacts.
Thanks for the feedback. For now I only use .png files, so I would discard this point.
About differences, with the same threshold parameter it detects mainly issues around text (that’s why I was raising anti aliasing issue), and also points out/red squares white/blank areas, corners where nothing is displayed.
Maybe this is linked to what you mention, i.e. screenshot taken before rendering. I have seen sometimes that the screenshot has some kind of blur effect/transparency on some buttons for example, letting me think that page is not fully loaded and just appearing.
I assume in this case the solution is to wait for one of the last enabled element, as I try to avoid sleep instructions
