This is the continuation of the previous article where I implemented the Pipes and Filters integration pattern but left out the continuous deployment aspect of it. This post is going to assume you have working knowledge of Cake, WebJobs and Kudu, although appropriate links have been added for reader convenience. The Cake script that I have written is still a work in progress so it may not be fully up to standards of Cake scripting (if any).

One key thing with Web Jobs deployment on Azure is that it can be deployed on its own, it doesn’t need a corresponding web app. The web job does however need to be under an AppService which translates to physical folder structure that looks like this on the server (for continuously running web jobs):

/site/wwwroot/app_data/jobs/continuous/{web-job-name}

and

/site/wwwroot/app_data/jobs/triggered/{web-job-name}

for triggered web jobs. I am using the former.

Whatever upload technique I want to use must be able to write to that location. It turns out Kudu provides a very simple to use REST API and corresponding Cake client that allows you to create and manage resources on Azure. One such API is the zip file upload API that can read your artifact folder, zip it up, upload it to the right location on the deployment destination and unzip it there and VOILA! You’ve just uploaded an app to Azure programmatically.

You do however need to make sure you provide proper credentials for  Kudu client for it to be able to successfully upload files, so let’s just dig in to the Cake script and start at the publish phase and go all the way down to the deploy phase:

Publish:

The publishing step uses DotNetCorePublish to publish the dlls and binaries for a .NET project but to be able to do this I need to first convert my .NET web job project to use the new csproj format. You can check out my article on how to do this and then come back here.

Also important during publishing the binaries are application settings like connection strings and other app settings that need to be applied to the deployable unit. I have chosen to store these potentially sensitive values in the environment variables of my machine, that way they never get checked into the source control. During CI/CD these same environment variables will be read from the VSTS CI build variables so I have create them there too.

VSTS vars

VSTS build time variables:

Azure Web-Deploy:

Now that I have my published artifacts with appropriate config settings applied, I am ready to upload it to Azure App Service that I have set up before hand. This is what the step looks like:

As you can see it depends on a “Get-PublishProfiles” step, this step essentially retrieves the authentication credentials that Kudu client will need in order to upload the published artifacts. A manual way to get these credentials is to download the publish profiles for the app service from Azure portal and copy the user name, password and the management URI and put it in the Cake script but this has 2 problems:

  1. I will be storing sensitive information in the Cake file.
  2. I will need to do this for every web job in the solution which kinda defeats the purpose of build and deployment automation. Also, these profiles will be regenerated everytime I provision my infrastructure using scripts which will mean even more manual work to update all the old profiles as they won’t be valid anymore.

To programmatically access Azure resources, Microsoft expose REST endpoints that can be called with appropriate API token. One such endpoint is this:

subscriptions/{azureSubscriptionId}/resourceGroups/loanbrokerage-testing/providers/Microsoft.Web/sites/{siteName}/publishxml?api-version=2016-03-01

which allows you to list the publish profiles for a given web app under an Azure subscription and resource group, as an XML document. I can then use Regex to parse the publishing credentials out of this response. To use this API though, I first need an access token that I can request from a well known OAuth2 endpoint that Microsoft expose. Here’s what that step in Cake looks like:

The client_id and secret are for the Azure AD app that I need to register first in Azure portal in order to access Azure resources programmatically (I am sure this step can be automated too but for this exercise I have just gone into the portal and manually created an web app/api type app registration and used the corresponding client_id and secret). The resource that I need access to is the Azure resource management API exposed at “https://management.azure.com”.

Using all this information I can make a POST request from within Cake to retrieve the access token and store it in memory. Once this access token has been retrieved I now have to load the publishing profile from another Azure API and pass it this access token. The function “PasswordFor” does exactly that:

I am simply parsing the XML response and extracting the “userPWD” field for the app currently being deployed. The management URI for each App Service in Azure follows this pattern: https://{appname}.scm.azurewebsites.net therefore I can easily construct this in Cake at build time because I already know the name of the app.

If you look back at the Azure Web-Deploy step that does the actual deployment, this function gets called once per web job because each of these web jobs live under a different app service on Azure and have different publish profiles. I need to make sure that each web job deployment is using the right credentials.

Once this information is retrieved the rest of code to do with Kudu is pretty straightforward. I create an instance of the KuduClient with the appropriate credential information retrieved in the publish profiles step, set the source as my artifact folder, set the destination as the folder path for the web job under Azure AppService and call ZipUploadDirectory() to send files from source to destination, effectively, deploying the application.

Jobsagudden!

You can check out the whole code along with all the Cake stuff is on my GitHub.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Category

architecture, design, ci/cd pipeline, continuous build, continuous deployment, Microsoft Azure, Uncategorized

Tags

, , , , , , , ,