--rerunfailedtest with xunit file in Azure pipeline

Robot Framework
#1

Hi,

Is there a way to use a cmd or powershell command in the Azure build pipeline
Expected result of the command is:
rerun the failed testcases as a task (cmd or Powershell) in the Azure pipeline and merge the output files (xml) with initial outputxunit.xml and the output2 file.

See the example below.

Azure is using -x (xunit) to interpret the output

I use the following command

robot -x outputxunit TC*.robot | robot --rerunfailed outputxunit.xml -x output2.xml TC*.robot | rebot --output outputxunit.xml --merge outputxunit.xml output2.xml

the Error message is:
[ ERROR ] Reading XML source ‘outputxunit.xml’ failed: Incompatible root element ‘testsuite’.

Does anyone have an idea?

Thanks
Joost

#2

I would try to not use pipe to rebot, but instead to use a real file.

#3

like an input xml file?

#4

Yes, instead of one command like you have with piping the input to rebot, you would redirect to a new xml file, and then in another command use it for rebot.

#5

Hi, @HelioGuilherme66 @JoostW
Have you manage to fix this ?
I’ve pasted below my pipline.
Doesn’t matter what I will write in script bold below I always got the same "
“[ ERROR ] Collecting failed tests or tasks from ‘D:\a\1\s\TestOutput\ers_browser.xml’ failed: Reading XML source ‘D:\a\1\s\TestOutput\ers_browser.xml’ failed: Incompatible root element ‘testsuite’”

Python package

Create and test a Python package on multiple Python versions.

Add steps that analyze code, save the dist with the build record, publish to a PyPI-compatible index, and more:

Build and test Python apps - Azure Pipelines | Microsoft Learn

trigger:

  • main

schedules:

  • cron: 0 9 * * 1-5 # cron syntax defining a schedule
    displayName: ERS UI Automated Tests - browser library # friendly name given to a specific schedule
    branches:
    include: [ ERS ] # which branches the schedule applies to
    exclude:
    - Geet_Branch
    - Iza_branch
    - Piotr_Branch
    - Tiantian-branch
    - ERS_Jose # which branches to exclude from the schedule
    - Vehicle_and_BrokenUI # which branches to exclude from the schedule
    always: true # whether to always run the pipeline or only if there have bee

jobs:

  • job: job
    timeoutInMinutes: 90 # Sets the execution timeout in Azure to 90 min - maximum is 360 min
    pool:
    vmImage: ‘windows-latest’
    strategy:
    matrix:
    Python39:
    python.version: ‘3.10’

    steps:

    • task: UsePythonVersion@0
      inputs:
      versionSpec: ‘$(python.version)’
      displayName: ‘Use Python $(python.version)’

    • script: |
      python -m pip install --upgrade pip
      displayName: ‘Install Python’

    • task: CmdLine@2
      inputs:
      script: ‘pip install -r $(build.sourcesdirectory)\packages_browser.txt’ # This line is to
      displayName: “Install Packages”

    • task: CmdLine@2
      inputs:
      script: ‘python -m Browser.entry init’
      displayName: ‘Browser Dependencies’

    • task: CmdLine@2
      continueOnError: True
      inputs:
      script: ’ robot -d TestOutput -i “VTE(ERS)” -x ers_browser.xml --loglevel DEBUG --loglevel TRACE Tests/ERS.robot’
      displayName: ‘Execute ERS Tests’

    • task: CmdLine@2
      continueOnError: True
      inputs:
      script: ‘robot -d TestOutput --rerunfailed D:\a\1\s\TestOutput\ers_browser.xml -x ers_rerun_browser.xml --loglevel DEBUG --loglevel TRACE Tests/ERS.robot’
      displayName: ‘Rerun Ers Test’

    • task: CmdLine@2
      continueOnError: True
      inputs:
      script: ‘rebot -d TestOutput --merge D:\a\1\s\TestOutput\ers_browser.xml D:\a\1\s\TestOutput\ers_rerun_browser.xml -x merged_results.xml’

    • task: PublishPipelineArtifact@1
      inputs:
      targetPath: ‘$(System.DefaultWorkingDirectory)\TestOutput’
      artifact: ‘TestResults’
      publishLocation: ‘pipeline’

    • task: PublishTestResults@2
      inputs:
      testResultsFormat: ‘JUnit’
      testResultsFiles: ‘ers_browser.xml’
      searchFolder: ‘$(System.DefaultWorkingDirectory)\TestResults’

#6

Hi Piotr,

I have for now solution. See the yaml below. Error is : [ ERROR ] Collecting failed tests or tasks from ‘outputxunit.xml’ failed: Reading XML source ‘outputxunit.xml’ failed: No such file or directory

image
image845×1122 77.4 KB

#7

@JoostW Basically what you are trying to tell me that your YAML configuration is still doesn’t work, right ?
But instead of one error you are getting another >?

#8

@Piotr, yes that’s right, now get another error back.

I found something below: (it says, first there must to by merged the original output xml and use rebot to create xunit from the merged output)

#9

Hi @JoostW
I’ve found answer why we are getting this errors and I repaired my YAML.
There is very simply reason you can not us XUNIT file as you are doing there
-x outputxunit.xml you need to create in first run --output outputxunit.xml instead -x
always use OUTPUT file not Xunit file.
My pipline now works fine

#10

hi @Piotr, thats nice. How your yaml looks like?

#11

Hi @Piotr, I also found the answer, and the pipeline does what it should do (1st the run, rerun failed TC’s, merge and PublishTestResult), see the example of the yaml below:

image
image1309×768 26.8 KB

#12

@JoostW
Can you tell me what is for this part in your Yaml ?

image
image879×46 1.98 KB

#13

image
image1527×765 77.3 KB

1 Like
#14

@Piotr On that line of the yaml, you can move to the root directory
or another specified PATH, which contains, for example, your robot tests.

1 Like