Failure to launch Robot test cases in Azure Dev Ops pipeline

Robot Framework
1

Hello everyone

I am trying to launch automated tests in Azure Dev Ops. I have setup the DevOps where appium and windows application driver are installed. Appium Driver version is 1.22.2. Testcases have been written in Robot Framework where a sample code is the following:

Enter Application for Animation using Vulcan Build
        [Documentation]    Open application for animation part of RockSolid SDK using Vulcan
        Open Application     http://localhost:4723/wd/hub    alias=Myapp1    platformName=Windows    platformVersion=Windows10Pro    deviceName='Windows'    app=${CURDIR}\\buildVulkan\\animation\\Release\\animation.exe

However, when I launch the tests in Azure Dev Ops pipeline, I get the following error

Failed to locate opened application window with appId: D:\_work\1\s\rocksolid-sdk\buildVulkan\animation\Release\animation.exe, and processId: 3036

Azure pipelines yml is shown below

resources:
  repositories:
    - repository: WindowsPackageManagement
      type: git
      name: 'DevOps/WindowsPackageManagement'
      ref: master

stages:
- stage: build_examples
  jobs:
  - job: build_examples_on_windows
    pool:
      name: Azure Self-hosted pool
      vmImage: gpu-nv-windows-
    steps:
    - checkout: WindowsPackageManagement
      clean: true
      fetchDepth: 1
    - script: powershell .\windows-package-management.ps1
      displayName: Run WindowsPackageManagement
      workingDirectory: $(Build.SourcesDirectory)/WindowsPackageManagement
    - checkout: self
      clean: true
      submodules: recursive
      fetchDepth: 1
    - task: Bash@3
      displayName: Build requirements
      inputs:
        filePath: rocksolid-sdk/build-requirements
        workingDirectory: $(Build.SourcesDirectory)
    - bash: |
        python -m pip install --upgrade pip
      displayName: Install pip
    - task: DownloadPipelineArtifact@2
      displayName: Download rssdk
      inputs:
        source: 'specific'
        project: 'Rocksolid SDK'
        pipeline: 28
        runVersion: 'latestFromBranch'
        runBranch: 'refs/heads/master'
        artifact: 'x64-windows.x64-windows'
    - task: CmdLine@2
      displayName: Install rssdk
      inputs:
        targetType: inline
        script: for %%i in (rssdk*.*.exe) do start /wait %%i /S /Q
        workingDirectory: $(Pipeline.Workspace)
    - task: Bash@3
      displayName: Build examples (Vulkan)
      inputs:
        targetType: inline
        script: |
          export ROCKSOLID_SDK="$PROGRAM_FILES/Rocksolid SDK"
          cmake -D RSGFX_GRAPHICS_API=Vulkan -B buildVulkan -S "$PROGRAM_FILES\Rocksolid SDK\share\RocksolidSDK\examples"
          cmake --build buildVulkan --config Debug
          cmake --build buildVulkan --config Release
        workingDirectory: $(Build.SourcesDirectory)/rocksolid-sdk
      env:
        PROGRAM_FILES: $(PROGRAMFILES)
    - task: Bash@3
      displayName: Build examples (OpenGL ES 3.1)
      inputs:
        targetType: inline
        script: |
          export ROCKSOLID_SDK="$PROGRAM_FILES/Rocksolid SDK"
          cmake -D RSGFX_GRAPHICS_API="OpenGL ES 3.1" -B buildOpenGL -S "$PROGRAM_FILES\Rocksolid SDK\share\RocksolidSDK\examples"
          cmake --build buildOpenGL --config Debug
          cmake --build buildOpenGL --config Release
        workingDirectory: $(Build.SourcesDirectory)/rocksolid-sdk
      env:
        PROGRAM_FILES: $(PROGRAMFILES)
    - script: npm install -g appium 
      displayName: 'Install appium'
    - bash: |
        pip install robotframework robotframework-pabot robotframework-appiumlibrary 
      displayName: 'Install dependencies'
    - script: sh -c "appium &"
      displayName: 'Start appium'
    - task: Windows Application Driver@0
      inputs:
        WADArguments: 127.0.0.1 4723/wd/hub
        OperationType: Start
    - powershell: |
        reg add "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock" /t REG_DWORD /f /v "AllowDevelopmentWithoutDevLicense" /d "1"
        Write-Host "Robot Framework tests"
        robot .\imagesDetect.robot
        cat .\output.xml
      failOnStderr: true
      displayName: 'Run Robot Framework Tests'
      workingDirectory: $(Build.SourcesDirectory)/rocksolid-sdk
    - task: PublishTestResults@2
      displayName: 'Publish Test Results'
      condition: always()
      inputs:
        testResultsFiles: $(Build.SourcesDirectory)/rocksolid-sdk/output.xml      
    - task: Windows Application Driver@0
      inputs:
        OperationType: Stop
2

At first glance to me that error looks like the robot code has the application executable under one path but the actual path under the Azure CI/CD system is different.

Can you share the snippet(s) of code for the open application keywords and possible any variables used to pass along the path?

3 
    Enter Application for Animation using Vulcan Build
    [Documentation]    Open application for animation part of RockSolid SDK using Vulcan
    Open Application     http://localhost:4723/wd/hub    alias=Myapp1    platformName=Windows    platformVersion=Windows10Pro    deviceName='Windows'    app=${CURDIR}\\buildVulkan\\animation\\Release\\animation.exe
    Take Screenshot
    Close Application

Enter Application for Animation using Open Gl Build
[Documentation] Open application for animation part of RockSolid SDK using Open Gl
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildOpenGL\animation\Release\animation.exe
Take Screenshot
Close Application
Enter Application for Compute using Vulcan Build
[Documentation] Open application for compute part of RockSolid SDK using Vulcan
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildVulkan\compute\Release\compute.exe
Take Screenshot
Close Application
Enter Application for Compute using Open Gl Build
[Documentation] Open application for compute part of RockSolid SDK using Open Gl
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildOpenGL\compute\Release\compute.exe
Take Screenshot
Close Application
Enter Application for Minimal using Vulcan Build
[Documentation] Open application for minimal part of RockSolid SDK using Vulcan
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildVulkan\minimal\Release\minimal.exe
Take Screenshot
Close Application
Enter Application for Minimal using Open Gl Build
[Documentation] Open application for minimal part of RockSolid SDHK using Open Gl
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildOpenGL\minimal\Release\minimal.exe
Take Screenshot
Close Application
Enter Application for Model Loader using Vulcan Build
[Documentation] Open application for model loader part of RockSolid SDK using Vulcan
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildVulkan\model_loader\Release\model_loader.exe
Take Screenshot
Close Application
Enter Application for Model Loader using Open Gl Build
[Documentation] Open application for model loader part of RockSolid SDK using Open Gl
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildOpenGL\model_loader\Release\model_loader.exe
Take Screenshot
Close Application
Enter Application for Text using Vulcan Build
[Documentation] Open application for text part of RockSolid SDK using Vulcan
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildVulkan\text\Release\text.exe
Take Screenshot
Close Application
Enter Application for Text using Open Gl Build
[Documentation] Open application for text part of RockSolid SDK using Open Gl
Open Application http://localhost:4723/wd/hub alias=Myapp1 platformName=Windows platformVersion=Windows10Pro deviceName=‘Windows’ app=${CURDIR}\buildOpenGL\text\Release\text.exe
Take Screenshot
Close Application

4

Thank you for your prompt answer EdManlove. Above is the snippets of code for the open application keywords and there was no variables used to pass along the path

5

From this error message,

Failed to locate opened application window with appId: D:\_work\1\s\rocksolid-sdk\buildVulkan\animation\Release\animation.exe, and processId: 3036

I was curious as to paths, which other being completely lost in a brand new system, things look … ok? Re-reading that error message several times it does seem to say the application was launched (thus the path should be valid) but couldn’t switch to the window. Oddly, or at least to me, it seems to use the path ass the appID.

Do you have a Robot Framework Slack account? I’d suggest maybe posting to the #appiumlibrary channel there as I know a couple people with experience with the appiumlibrary listen and respond there. I’m a little out of my knowledge zone here …

6

Hi thanks for your message. No I dont have Robot Framework Slack Account. I will try to get it and then post to the appium library channel there and lets see how it goes. Thanks again for your help