Running chrome test template does not respect ${EMPTY} variablevariable

Libraries SeleniumLibrary
1

I’m running a test with a template inside it. One of the arguments I’m passing is ${EMPTY} to simulate not putting in anything in the text field. The test case looks like this:

Log in with invalid credentials
[Documentation]    Simulate logging in with incorrect password and
...    verifying that correct error appears
[Template]    Invalid login of ${username} and ${password} should show error
bad_user    bad_password
${EMPTY}    other_bad_password
other_bad_user    ${EMPTY}

When attempting to run this against chrome, it seems to take in the previously entered field instead of an empty text field. However, when running against firefox, it is showing the correct error message stating that the field is empty. Is this a bug in SeleniumLibrary running for Chrome tests?

2

Well, hard to say anything concrete with that example. Because it works with Firefox and does not work Chrome, it feels the be a bug in the ChromeDriver or Chrome implementation. But that is just educated guess.

You could run the test with --loglevel trace set from the command line and show the output from the Chrome and Firefox runs, where the test fails. Perhaps then we are able to help you better.

3

Hi Tatu,

I’m actually now seeing 2 problems, but i’ll focus on just the one issue as we see here.
This is actually what should be seen and the firefox runs capture the problem correctly. I’m testing this site: Swag Labs

  • Also side note is that I have a line where I want to capture a screenshot with the username and password as part of the image name; I expected 3 different images, but see only 1 image when I run this script.

    Invalid login of ${username} and ${password} should show error
    [Documentation] Template to test invalid login scenario
    Enter username ${username}
    Enter password ${password}
    Click submit button
    Verify error message appears
    Capture Page Screenshot filename=Log-in-with-invalid-credentials(${username}-${password}).png

Running robot tests...
==============================================================================
TestLogin :: Test suite containing login scenarios
==============================================================================
Log in with invalid credentials :: Simulate logging in with incorr… | FAIL |
Several failures occurred:

1) Epic sadface: Username is required != Epic sadface: Username and password do not match any user in this service

2) Epic sadface: Password is required != Epic sadface: Username and password do not match any user in this service
------------------------------------------------------------------------------
TestLogin :: Test suite containing login scenarios                    | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================

However, when I’m running with chrome no errors appear; which is incorrect (I have a bug in my script, the expected error message should be different). However, it correctly shows 3 images when I run the same script (but with Chrome).

    Running robot tests...

==============================================================================
TestLogin :: Test suite containing login scenarios

DevTools listening on ws://127.0.0.1:56668/devtools/browser/5f542dbd-d2d5-4a88-bc38-48c120121b7b
Log in with invalid credentials :: Simulate logging in with incorr… | PASS |

TestLogin :: Test suite containing login scenarios | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed

I have also placed my repo here if you want to look at the full details:
https://gitlab.com/f00dog/robotframeworkweb-proofofconcept

4

That wasn’t the the output which I wanted to see, because I explained my self in unclear manner.

You could run the test with --loglevel trace set from the command line and show the stack trace from the log.html file. Both from Chrome and Firefox runs, where the test fails.