How can I automate web view flutter using the Canvas locators?

Robot Framework
1

I was trying to automate a web view flutter using a canvas locator, but it didn’t work when I scrolled, or swipe. Does anyone know about this?

2

You didnt mention your software stack at all and what sort of app you have? Appium does have flutter driver, is that what you are using ? Flutter driver says:

you’d like to test a release app, which can be released from app store as-is, Appium UIAutomator2/XCUITest driver is a good choice. Since Flutter 3.19, Flutter apps can expose identifier for SemanticsProperties as resource-id in Android and accessibilityIdentifier in iOS. They should help to achieve automation against release apps with Appium UIAutomator2/XCUITest as blackbox testing.

No mention of canvas locators but also that part mentions mac/ios/android only ..

Worth pointing out that there are few scrolling related questions/issues in flutter driver @ GitHub · Where software is built

3

Thank you for your input!
I’m currently automating a Flutter web application using Robot Framework with Selenium, running on a web browser like Chrome — not on mobile (iOS/Android). So I’m not using Appium or the Flutter driver at this point.

Since the app is rendered as a single canvas on the web, standard element locators like aria-label or XPath don’t always work as expected, which is why I’ve been exploring different strategies, including canvas locator approaches.

Appium with Flutter driver might help for mobile testing, but my use case is focused on web automation, where scrolling and element detection are proving to be challenging due to the way Flutter renders UI in the browser.

4

Pardon my ignorance, what do you refer to as canvas locator? I’m not aware of the details.

Have you tried javascript code execution to interact with the element(s)? In the past, that was a way to workaround handling elements that Selenium WebDriver wasn’t natively able to, having Selenium invoke the custom javascript code to do what you want instead. This can include scrolling actions among other things. Not sure how well it would work against flutter and the canvas related elements, but worth a try. Reference: Javascript is your ally for Selenium / WebDriver | autumnator