How to perform Image comparison - selenium automation

Robot Framework
#1

Hi,

i’m trying to find the proper Library that will help in comparing 2 images.
the scenario:

  1. i will take a screenshot of an element
  2. do an action which change the element image
  3. take another screenshot of the element
  4. try to compare between the 2 screenshots

I have looked over different solutions:

but for each of them, there are multiple steps for installing the dependencies and tools in order it to work ( and it is different for windows/ mac/ linux)

is there a generic / automatic way to compare images / screenshots on Robot Framework?
this is something that i really need in order to proceed in our company automation.

Thanks

#2

Hi Moshe,

This topic might have some useful info: Library for automated visual regression testing

Dave.

#3

Hi @Moshed ,
as discussed in the Slack Channel:

I created this smaller Image Comparison Library yesterday

If offers basic image comparison options and some simple masks.
No more dependencies with Tesseract, but because of that no Text Recognition is supported.

Maybe interesting for other users as well.
It’s still in a very early development stage

1 Like
#4

Hi Many,

Nice Library :+1:t3:

Some feature suggestions for future versions:

  • regardless of pass or fail return a difference score (0 being no similarity and 100 being 100% match) then people could do something like:
${Status}    ${Score}=    Run Keyword And Ignore Error     Compare Images    Reference.jpg    Candidate.jpg
IF     ${Score}>${ArbitraryNumber}
     Do Something Keyword
ELSE
     Run Some Other Keyword
END
  • Allow user to input a difference tolerance or score needs to be greater than to pass e.g.
Compare Images    Reference.jpg    Candidate.jpg    tolerance=80
#5

@Many thanks again
@damies13 I agree that these feature will surely improve the image comparison library.

#6

Great idea @damies13 .
And both pretty simple to implement, so I’ll definitely look into it.
I think i’ll provide the possibility to set the threshold as a library argument and as a keyword argument.
Shall I return the similarity score for the Compare images keyword, or do you prefer a separate one , like Get Similarity ?

#7

Hi Many,

I don’t actually have a preference, It’s your library, so do it how you think is best, I just thought it would be useful.

If someone only wants the Get Similarity functionality they can just set the threshold to 0 so it never fails, but I guess providing both options wouldn’t be hard either.

Having the possibility to set the threshold as a library argument as well is a good idea :+1:t3:

It might be useful to have keywords like Get Threshold and Set Threshold so the default value can be changed throughout the test, something like what SeleniumLibrary does with the selenium timeout. The same with the other kwargs, I’m guessing you set them as class level variables so it should be easy to implement the setter and getter functions.

Dave.

#8

Hi,
I’m not quite clear what your point is.

But… the installation itself is probably not the problem, is it?

For me, the question is still open as to what exactly you want to achieve with the image comparison. Can you describe more precisely what your application is?

try to compare between the 2 screenshots

If you have changed the reference image meanwhile, the comparison will fail. What is your expected result? Do you want to measure the differing pixels?

#9

Hi Simon,
I’m trying to compare between 2 elements and see if there is a change (on some steps the comparison should be the same & other steps it should be different)
i need to test when the elements are different/ equal during automation