Have you tried creating a continuous integration environment in Azure DevOps (formerly VSTS)?
Isn’t it a crazy challenge?
I spent a couple of weeks trying to track down the information so that I can actually have a complete working pipeline.
As a result, it no longer has to be so hard for you…
I documented everything so that any automation engineer can use Azure DevOps to create a continuous integration environment for automation.
How to create your first CI build chain
- Navigate to your VSTS account and team project. Click Pipelines
- Select New pipeline to create a new build definition.
- Choose GitHub from Where is your code
- Authorize communication with Github by clicking the Authorize button
- Select your repository
Executing the pipeline
- Select the ASP.NET Core (.NET Framework) build template
- A .yml file will be created for your repository
- Click Save and Run
Adding continuous integration
- Select Builds > Edit > Triggers. Under Continuous integration, select on the name of your repository.
- Toggle on the checkbox labeled Enable continuous integration.
- You can select CI for both merges and Pull Requests – https://prnt.sc/kb0g6n
How to create scheduled builds
- Picking up from the section above, click the Add button under Scheduled section
- Update the time and the day of when you want to run your builds
- Save & queue
How to set up your test .dll paths
The hardest time I had was how to configure my paths for my automation .DLL files. It’s not intuitive to know what is the working directory of your code base.
Here are some examples of what’s worked for me:
How to pass parameters to test code from a build or release pipeline?
A: Use a runsettings file to pass values as parameters to your test code. For example, in a release that contains several stages, you can pass the appropriate app URL to each the test tasks in each one. The runsettings file and matching parameters must be specified in the Visual Studio Test task.
How to add a status badge to your Github repo
The instructions on this are really good from Microsoft and you can follow them here in the Get the status badge section
All about YAML
Working YAML file for UI Automation
Possibly a useful YAML for test automation
I found this YAML snippet for executing tests with VSTest from here
Not sure if it’s helpful yet, but it might be.
# Create a secret variable - powershell: | Write-Host '##vso[task.setvariable variable=mySecret;issecret=true]abc' # Attempt to output the value in various ways - powershell: | # Using an input-macro: Write-Host "This works: $(mySecret)" # Using the env var directly: Write-Host "This does not work: $env:MYSECRET" # Using the mapped env var: Write-Host "This works: $env:MY_MAPPED_ENV_VAR" env: MY_MAPPED_ENV_VAR: $(mySecret)
How to set and read environment variables?
I literally spent days of research, trial, and error trying to figure out how to set and read environment variables from an Azure test agent.
Regardless, I figured it out. Enjoy the solution.
- First you need to create some environment variables in your Azure DevOps UI that you want to use for values. This is an example of a variable that I would like to set on the test agent for my automation scripts. sauce.userName
- I will use the value of this variable(sauce.userName) and set it in my Environment Variables of the test agent when my automation is running. That way, the value of this variable isn’t exposed to the public. Well, it’s exposed to you because I am writing this post haha. But pretend it’s some “critical information” like your credit card 🙂
- Next, you will want to create a Powershell script that you attach to your solution. Here’s my solution layout.
Here is what you want in your Powershell script. Basically, I’m going to pass in two variable values and then my Powershell script will set the Environment Variables on the agent for the user.
Finally, you want to have a powershell step in your YAML that executes this Powershell script and passes in the values from the variables that you set in the Azure DevOps UI.
Below is what my YAML step looks like and I’m basically setting the SAUCE_USERNAME and SAUCE_ACCCESSKEY variables.
In order for the YAML to understand where these variables come from, you need to convert sauce.userName to SAUCE_USERNAME. That’s the Azure convention. Read more about working with variables.
Basically the value stored in
sauce.userName is passed in as a variable called
$env:SAUCE_USERNAME . It’s the same exact variable, but when you convert it from the ADO UI to YAML, that’s the convention, I know, it’s weird…
How to create your first project
- Go here
- Click the Get Started For Free button
How to run NUnit tests in VSTS
Instead of using Nunit test filters, you need to use the MSTest filters. For example, you need to convert:
cat == unit to /TestCaseFilter:”Category=unit”
This will execute all Nunit tests that have the Category “unit”.