Create Resource Group and Key Vault

The first thing that we need to do is create the new Resource Group (RG) and Key Vault. Go to the Azure Portal and launch Cloud Shell with a PowerShell Environment so we can get started.

Note: Cloud Shell requires a Storage Account to store configuration. So the first time that you launch it, you will be prompted to specify the Subscription and create the storage account. You can select the Show advanced settings options to set the details of an existing RG and Storage Account or create new ones.

1 – Create a new Resource Group by running New-AzResourceGroup -Name <RG Name> – Location <Azure Location for RG>.

Update the values between the <> with the name and location of your choosing.

2 – Create a Key Vault in our Resource Group by running New-AzKeyVault -VaultName <KV Name> -ResourceGroupName <RG Name> -Location <Azure Location for RG>.

Update the values between the <> with the values for <RG Name> and <Azure Location for RG> that you specified in the previous step.

3 – Add an Access Policy on Key Vault so that you can add the Self-Signed Certificate to it. Run the Set-AzKeyVaultAccessPolicy -VaultName <KV Name> -UserPrincipalName <Your User Principal Name> -PermissionsToCertificates get,list,update,create -PermissionsToSecrets get.

Update the values between the <> with the value of <KV Name> that you specified in the previous step. As well as your UPN name i.e. someone@example.com.

Note:  Cloud Shell uses it’s own token to authenticate users rather than user tokens. So you will need to login to Azure using Azure PowerShell to be able to add the Self-Signed Certificate to the Key Vault.

4 – Log in with the User Principal Name (UPN) that you specified when adding the Access Policy to the Key Vault and run the following command to create Self-Signed Certificate. And then add it to the Key Vault that you previously created.

Login

Connect-AzAccount -UseDeviceAuthentication

You will need to follow the instruction in the Cloud Shell console to log in i.e. open the URL displayed (https://microsoft.com/devicelogin) in a browser and enter the code provided.

Create Certificate

$Policy = New-AzKeyVaultCertificatePolicy -SecretContentType “application/x-pkcs12” -SubjectName “CN=<Certificate Common Name>” -IssuerName “Self” -ValidityInMonths 6 -ReuseKeyOnRenewal

Update the values between the <> with the value that you would like to use for the Common Name of your certificate. In my example I used “CN=test.com”.

Add Certificate to Key Vault

Add-AzKeyVaultCertificate -VaultName <KV Name> -Name <Certificate Name> -CertificatePolicy $Policy

Update the values between the <> with the value of the <KV Name> that you specified in the previous steps and specify a name for your certificate. In my example, I used “TestCert01”.

5 – Now you can create a second Resource Group that you can deploy resources to. Running New-AzResourceGroup -Name <RG Name> – Location <Azure Location for RG>.

Update the values between the <> with the name and location of your choosing. In my example I have called the Resource Group azdevops-aes-dev-deployment-rg and specified the australiaeast location.

Create Service Principal and Assign RBAC

In the steps below, you will be retrieving the certificate from the Key Vault and using it to create your Service Principal in Azure AD. After which, you will assign an Azure RBAC role to your Service Principal so that it has the necessary permissions to deploy resources to your deployment Resource Group.

1 – Now you need to retrieve the Certificate from the Key Vault and then use it to create the Service Principal.

Get the Certificate from the Key Vault

$cert = Get-AzKeyVaultCertificate -VaultName <KV Name> -Name <Certificate Name>

Update the values between the <> with the value of the <KV Name> and the <Certificate Name> that you specified in the previous steps.

Get the Secret from the Key Vault

$secret = Get-AzKeyVaultSecret -VaultName <KV Name> -Name $cert.Name -AsPlainText

Update the values between the <> with the value of the <KV Name> that you specified in the previous steps.

Create the Service Principal 

$sp = New-AzADServicePrincipal -DisplayName <SP Name> -CertValue $secret -EndDate $cert.Expires

Update the values between the <> with the value that you would like to use for the Service Principal. In my example, I used “AADAZDEVOPSDEV01”.

In the Azure Portal, you should see the Service Principal that you have created under the App registrations in Azure Active Directory. The Service Principal should also see the Self-Signed certificate associated with the Service Principal under Certificates & secrets.

2 – Now you can assign the Azure RBAC role to your Service Principal over the deployment Resource Group that you created in the previous steps. To make things easier you can use the AzureAD modules to get the application Id of your Service Principal and set as variable.

Set Service Principal Application Id Variable

Connect-AzureAD

$AppRegName = ‘<Service Principal Name>’

$appId = Get-AzureADApplication -Filter “DisplayName eq ‘$AppRegName'” | Select-Object -ExpandProperty AppId

Update the values between the <> with the value the Service Principal Name. In my example I used “AADAZDEVOPSDEV01”.

Assign Contributor Role to Service Principal

New-AzRoleAssignment -ApplicationId $appId -RoleDefinitionName “Contributor” -Scope /subscriptions/<Subscription ID>/resourceGroups/azdevops-aes-dev-deployment-rg

Update the values between the <> with the value of your Azure Subscription.

Create Personal Access Token Azure DevOps

Before you can create the ARM Service Connection, you will need to create a Personal Access Token (PAT) to use the Azure DevOp CLI from Azure Cloud Shell. Learn more about PATs here.

1 – Log in to your Azure DevOps Organisation and select Personal access tokens under User settings in the top left-hand navigation bar.

2 – In the Personal Access Tokens screen select New Token and enter the appropriate details in the Create a new personal access token modal. Click Create when finished.

Note: For the purpose of this article, set the Scope to Full Access and the Expiration value to a time in the future. This setting is only recommended for learning purposes. The best practice is to restrict the scope of the PAT to only what is required for the specific interaction.

3 – You should now see the token details appear. Azure DevOps does not store the details of the token. So please copy these details into a safe location. You will need to re-generate the token if you misplace these details.

Tip – To make the solution more robust, consider storing the token in Azure Key Vault and retrieving the details at run time.

Create ARM Service Connection

Now that you have created your Service Principal in Azure AD and your PAT in Azure DevOps, you are ready to head back to Azure Cloud Shell and create your ARM Service Connection.

Note: To complete these steps below, you will need the Application Id of the Service Principal, Azure Subscription Name, Azure Subscription Id and Azure Tenant Id.

1 – You now need to set the AZURE_DEVOPS_EXT_PAT environment variable in Azure Cloud Shell to allow the Azure DevOps CLI to login into your Azure DevOps Organisation. Run the following command, $env:AZURE_DEVOPS_EXT_PAT = ‘<PAT Token>’ .

Update the values between the <> with the details of your PAT that you created in the previous steps.

2 – Configure the default Organisation for Azure DevOps CLI by running the following command, az devops configure -d organization=<ORG URL>. Once you have configured Azure DevOps CLI run the az devops project list command to check that there are no connectivity issues.

Update the values between the <> with the details of your Azure DevOps Organisation i.e. https://dev.azure.com/example.

Note: The first time you run an Azure DevOps CLI command in Cloud Shell you will be prompted to install the extension.

3 – To be able to create the ARM Service Connection using the Self-Signed Certificate stored in the Key Vault that you created earlier you will need to convert it from a PKCS#12 format to a PEM format. In the steps below, you will retrieve the Certificate from the Key Vault, store it on your Cloud Shell drive as a PFX file then use OpenSSL to convert it to a PEM file that you can use when creating the ARM Service Connection.

Retrieve Certificate from Kay Vault and Store as PFX file

$cert = Get-AzKeyVaultCertificate -VaultName <KV Name> – Name <Certificate Name>

$secret = Get-AzKeyVaultSecret -VaultName <KV Name> – Name $cert.Name -AsPlainText

$secretByte = [Convert]::FromBase64String($secret)

[System.IO.File]::WriteAllBytes(“cert.pfx”, $secretByte)

Convert Certificate from PFX to PEM

openssl pkcs12 -in cert.pfx -out cert.pem -nodes -password pass:

4 – You are now finally ready to use the Azure DevOps CLI to create your ARM Service Connection in your Azur DevOps project with certificate-based authentication. Run the following command.

Create ARM Service Connection

az devops service-endpoint azurerm create `

–azure-rm-service-principal-id <Service Principal Application Id>  `

–azure-rm-subscription-name ‘<Azure Subscription Name>’ `

–azure-rm-subscription-id <Azure Subscription Id> `

–azure-rm-tenant-id <Azure Tenant Id>`

–azure-rm-service-principal-certificate-path ./cert.pem `

–project <Azure DevOps Project Name> `

–name <ARM Service Connection Name>

Update the values between the <> with the details of your Service Principal Application, Azure Subscription Name, Azure Subscription Id, Azure Tenant Id, Azure DevOps Project Name and specify a name for your Service Connection.

If you head over to your Azure DevOps Project, you should be able to see the Service Connection that you just created.

In Summary 

This article provided a step by step guide on how to create an ARM Service Connection in Azure DevOps with Azure Cloud Shell. Including how to create a Service Principal in Azure AD that uses a Self-Signed certificate for authentication.

While this article focused on the technical steps for creating an ARM Service Connection in Azure DevOps. It also provides a scalable and secure foundation that you can use to build an automated approach for creating and managing Service Principals in Azure AD and creating and managing ARM Service Connection in Azure DevOps.

Thank you for reading. I hope that this article has provided you with some useful information.