Azure Devops supports building CI/CD pipelines for your applications by providing set of inbuilt build and release steps which I have written about in the past. However, they also support Cake builds which is a very nice predictable and repeatable approach to CI/CD. It also puts more control in the hands of us developers because we can customise the entire build + release workflow to suit the needs of our team and standardise it across various CI/CD platforms.
In this not very long post I will show how I recently set-up a multi-environment (Testing, Acceptance and Production) CI/CD pipeline to build my application in Azure DevOps but deploy to Azure AppService using Octopus with all the steps scripted using Cake. For demonstration purposes and to keep things simple, I will deploy an ASP.NET Core 2.2 web api (that I have already built) without any other infrastructure but the concepts will apply to more complex apps all the same.
Sidebar: A good resource to get up to speed with Cake scripting is this.
Visualised, the pipeline I am going to build looks something like this:
Assuming the per environment resource groups and app services have already been provisioned in Azure, I will break this into 4 parts:
- Set up the project in Octopus to deploy the app to Azure.
- Create the Cake script with the build and deploy tasks
- Set up the CI build in Azure Devops to use Cake script to build
- Set up the CD release in Azure Devops to use Cake script to deploy app via Octopus.
Part 1. Set up the project in Octopus
Say the application is called “cicdapp” and the corresponding app service apps in Azure are: cicdapp-testing, cicdapp-acceptance and cicdapp-production.
In order to set up Octopus to deploy to these apps, I will first need to allow it access to Azure resources. The way to do this is to register an application in Azure Active Directory and give it permission as a “Contributor” at the subscription level. This way it will automatically be inherited down to the resource group level.
Give it a name, select the type as a WebApp/API, provide the name of your Octopus instance and hit Create:
Once the app is created, I need to copy the Application ID down somewhere for later reference:
Open the Settings blade and create a secret key that will be used by Octopus to talk to Azure (this is only a one time set up, all Octopus project deployments in one instance can share the same credentials):
I will now add this app as a contributor at the subscription level:
Adding Azure account to Octopus instance is detailed on Octopus website
Once the Azure account is added, I need to set up the deployment targets (one per environment) i.e. hook up the Azure app services with Octopus’ deployment infrastructure:
Pick Azure as the platform and Azure Web App as a target:
In addition to the basic details, I need to pick:
- The environment that this deployment target should deploy to and creat a new Target Role for the deployment. From what I can tell, Octopus uses the target role to link a deployment with an environment and in general I think its a good practice to have a separate role per app per environment.
- My Azure account that I have already linked to Octopus using the details from the Azure Active Directory application registration. This will load the various Azure Web Apps that are set up in my Azure subscription, from which I will pick the one that I am targetting (the testing one in this case).
Finally, hit Save. I can immediately see if this has been successful because Octopus runs a quick connectivity check to the Azure app and if its all green (as shown below), then the setup was successful:
I will simply repeat this process for the remaining environments : Acceptance and Production.
Next thing to do is to create a new Octopus project that will represent the application that I need to deploy. It will also house all the application level configuration settings such as database connection strings etc.
Next define the deployment process:
Hit “Add Step” (as if that wasn’t obvious enough):
Pick Azure Web App as deployment template:
Fill the application deployment details. Select all the target roles applicable for this project and specify the package id. The application will be bundled as a nuget package and therefore it needs to have a unique id in the Octopus instance. Here I have just given it an id even though the package doesn’t yet exist in Octopus, it will once I make my first release with that package name.
The final thing to configure here is the application configuration source, since I will be deploying an ASP.NET Core app, I will set appsettings.json as a configuration file and Octopus will automatically replace application settings per environment. For .NET Framework apps, this will be Web.config.
In the Settings tab, I will change one crucial setting and that is the release versioning scheme. I want to use the version numbering from the application package that I publish to Octopus and not have Octopus dictate that for me.
There is no particular reason for this, I just want to use the default version that comes out of my CI pipeline which happens to be the build number. For applications this might be ok, but if I was deploying a reusable library such as a nuget package then I would let Octopus pick the “major.minor.patch” scheme for versioning or make sure that my CI pipeline tags the version number appropriately. Build numbering can be configured in Azure DevOps.
Just to make sure that the whole deployment pipeline is correctly deploying the application to each environment, I will add a dummy application setting in the variables section and scope it by environment. This way when I navigate to the application on a certain environment, it will simply output the value of this variable to prove the pipeline worked correctly.
This concludes part 1 of Octopus set up and we have 3 healthy deployment targets:
Part 2. Write the Cake script
In this part I am going to write the Cake script that can be invoked locally just as well as by the CI/CD pipeline in Azure DevOps which I will create in part 3.
The Cake website actually offers a simple bootstrapper kit to get you started with Cake scripting and it consists of essentially 2 files:
build.ps1 (for us lowly Windows users) and
build.cake. So I will clone this bootrstrapper repository and copy these 2 files into the main solution folder:
Next I will open up the build.cake file in VS Code and add all the workflow steps needed:
Let me elaborate on some of the key aspects:
There are handy tools available for Cake to be able to talk to Octopus and they are referenced in the Cake file like so (much like
These tools bring in helper methods that expose Octopus’ API in a developer friendly way and makes it easy to work with it.
Here I am setting the package id to exactly what I set in my Octopus project, otherwise the app won’t deploy. I am also creating a default semantic version for the package that I will publish. I will be using the build number as a version for the package being deployed. The build number is available to the Cake script via the
BUILD_BUILDNUMBER environment variable which is being read in in the Setup task.
Then I am reading Octopus settings from environment variables of the build agent and finally, adding the
releaseTo argument, defaulting it to the
Testing environment. Eventually the value for this argument will come from the release stage.
What’s crucial to realise here is that I want to use the same Cake script to do the deployment as the CI build which means that the
build.cake file needs to be available to the release pipeline. By default this won’t happen because the release pipeline won’t have access to the source code folder, it can only work with published artifacts. Therefore, during the CI build I copy these 2 files into my published artifacts folder which can then be published to the release pipeline. This way it can access the build.cake file and invoke it with appropriate arguments.
Its a bit of a hack but it works!
Just to make sure that the script is working, I can invoke it locally from my local repository root with a target of
This will run through all the steps and will generate the following folders in the root (in order to make sure they don’t get pushed to remote repository, just add these folders to the
artifacts folder contains the published binaries from the ASP.NET Core project and the
publish folder contains the published Nuget package with a default version:
OctoPush step will upload this package to Octopus server and
OctoCreateRelease will create a release on Octopus server (without actually deploying it).
The final step
OctoDeploy will actually tell Octopus to start the deployment to correct environment on Azure and will wait until its finished, reporting any deployment errors as they happen.
That’s pretty much it for the Cake part. In the next part, I will set up the CI/CD pipeline and make the first deployment attempt.
Part 3. Set up the CI pipeline in Azure DevOps
To set up the CI pipeline for this amazing application, I will head over to the Pipelines section of the repository and hit New:
cicdapp project and hit Continue:
Scroll all the way to the bottom of the template list, pick Empty pipeline and hit Apply:
Give the pipeline a name:
Give the AgentJob1 a meaningful name:
Other settings can be left on their defaults.
Next, add a Cake task and point it to the
build.cake available in the source repo. To do that, click on the + icon next to the Run CI Build and select Cake from the list:
Give the step a meaningful name and set the target field to “OctoCreateRelease” (this will run all Cake tasks until this point but not deploy anything):
One odd step that I will need to add to this build pipeline is .NET SDK Installer step to install .NET Core SDK 2.2.101 because the application being deployed is an ASP.NET Core 2.2 app but the build agents in Azure DevOps, at the time of this writing, don’t support .NET Core 2.2 natively but I can install it as a part of my build pipeline. This will add to the overall build time but its not too bad, plus, I expect Microsoft to address this as .NET Core 2.2 becomes more widely used.
I am using version 2.2.101 so I will set that:
As a final build step, I added a Publish Build Artifacts step because this will make the published artifacts (along with the cake file) available to the release pipeline:
I will change the default “Path to publish” to “artifacts” that’s the folder the Cake script is publishing the built binaries to:
One last thing I will need to do is add the custom environment variables (for Octopus settings) that the Cake script needs and add them in a way that they are available in both CI and CD phases. It turns out Azure DevOps provides a handy way to do so through Variable Groups.
Variable Groups are set up per repository by navigating to Library tab and hitting + Variable Group:
I will give the variable group a meaningful name and add the variables:
Now I need to link these variables to my build pipeline:
Select the variable group just created and hit Link:
Save the configuration and enable the continuous integration on this repository and with that the build pipeline is all set:
Part 4. Set up the release pipline in Azure DevOps
The last piece of puzzle is to set up the release pipeline that will deploy the app to Testing, Acceptance and Production via Octopus.
Following a similar process as the CI pipeline, I will pick the Empty Job template:
Name the stage “Release to Testing” or just “Testing” is fine as well.
Add the artifact for the release pipeline to work on and have it use the latest version of the artifact:
I still need to set up the release pipeline to run the Cake script, so I will do that next:
And then just like the CI pipeline, add a Cake step to the Run on Agent job:
Change the field values as shown below:
This right here is the crux of the release pipeline, the cake script will be available in the published “drop” folder once I make the first build and then the release pipeline will be able to invoke the Cake script with the right arguments. This will allow it do deploy to appropriate environments.
And once again just like the CI pipeline, I will link the variable groups to the release pipeline as well.
Once this is done, enable the continuous deployment trigger:
This completes the “Release to Testing” phase of the release pipeline, I now need repeat the same process for Acceptance and Production environments.
In the end the release pipeline will look like this:
Its time to test the whole pipeline, so I will go ahead and queue a new build:
And wait for it to finish and kick off the release pipeline (or fail and stop):
Once the builds successful, the release pipeline will be kicked off starting with Testing -> Acceptance -> Production
If all the environments successfully deploy, then the status in Octopus and Azure DevOps will show all green:
As a final test, I will navigate to all the three endpoints and see whether or not each environment got its copy of the app correctly by looking at the value of the “EnvironmentCode” app setting in the API response and voila!:
Looks like it all went swimmingly on its very first attempt! Very uncommon occurrence in my line of work! 🙂
So there you have it folks a full multi-environment CI/CD pipeline using Azure DevOps+Cake+Octopus. Cake is actually quite versatile and with a large selection of addins and tools for it, you can do a whole lot in your CI/CD pipeline.
This took me a while to set it up the first time with a few trial and erros so I figured I will document it here for future reference (good thing that AzureDevOps allows you to save your CI build pipeline as a custom template so you can set it up for other projects very quickly). Hopefully this will be useful for others too.