Last week I struggled (again) running ASP.NET Core integration tests for an ASP.NET Core Web API that uses DefaultAzureCredential in a Azure DevOps YAML pipeline. To avoid having to struggle again, I am writing down my findings here.
The initial situation
There was an existing ASP.NET Core (.NET 8) Web API project that uses DefaultAzureCredential to connect to an Azure Key Vault. The connection to the Azure Key Vault is established during startup to fetch certificates from the key vault.
var client = new SecretClient(new Uri(builder.Configuration["AzureKeyVaultEndpoint"]!), new DefaultAzureCredential()); var signingCert = GetCertificateWithKey(client, "signing-certificate"); var encryptionCert = GetCertificateWithKey(client, "encryption-certificate");
A Azure DevOps YAML pipeline had already been set up to build the application and execute unit tests. Furthermore a service connection of the type Azure Resource Manager using workload identity federation with openid connect was set up for the deployment.
The problem
After setting up integration tests according to Integration tests in ASP.NET Core the existing Azure DevOps YAML pipeline started failing. The pipeline runs failed during execution of DotNetCoreCLI@2 task with command test because all integration tests failed.
- task: DotNetCoreCLI@2
command: test
projects: ${{parameters.testProjects}}
arguments: '--configuration ${{parameters.buildConfiguration}} --collect "Code coverage"'
The logs were full of errors like the ones below.
Azure.Identity.CredentialUnavailableException: DefaultAzureCredential failed to retrieve a token from the included credentials. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/defaultazurecredential/troubleshoot
- EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot
- WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot
- ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource.
Status: 400 (Bad Request)
Content:
{"error":"invalid_request","error_description":"Identity not found"}
...
- Visual Studio Token provider can't be accessed at /home/vsts/.IdentityService/AzureServiceAuth/tokenprovider.json
- Please run 'az login' to set up account
- Az.Accounts module >= 2.2.0 is not installed.
- Azure Developer CLI could not be found.
---> System.AggregateException: Multiple exceptions were encountered while attempting to authenticate. (EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot) (WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot) (ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource.
Status: 400 (Bad Request)
Content:
{"error":"invalid_request","error_description":"Identity not found"}
...
) (Visual Studio Token provider can't be accessed at /home/vsts/.IdentityService/AzureServiceAuth/tokenprovider.json) (Please run 'az login' to set up account) (Az.Accounts module >= 2.2.0 is not installed.) (Azure Developer CLI could not be found.)
---> Azure.Identity.CredentialUnavailableException: EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot
at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
at Azure.Identity.EnvironmentCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
at Azure.Identity.EnvironmentCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
---> (Inner Exception #1) Azure.Identity.CredentialUnavailableException: WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot
at Azure.Identity.WorkloadIdentityCredential.GetTokenCoreAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
at Azure.Identity.WorkloadIdentityCredential.GetTokenCoreAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
at Azure.Identity.WorkloadIdentityCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)<---
---> (Inner Exception #2) Azure.Identity.CredentialUnavailableException: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource.
Status: 400 (Bad Request)
So basically all credentials combined by DefaultAzureCredential are not providing a token and therefore authentication fails. So far clear, after all the pipeline task does not use the service connection at all. However, even after setting the service connection on the task, the pipeline runs failed with the same error.
- task: DotNetCoreCLI@2
azureSubscription: YOUR-SERVICE-CONNECTION-NAME
command: test
projects: ${{parameters.testProjects}}
arguments: '--configuration ${{parameters.buildConfiguration}} --collect "Code coverage"'
NOTE: the service principal used by the Azure DevOps service connection is assigned to role Key Vault Certificate User on the corresponding key vault resource.
I also tried to use AzureCLI@2 task right before the DotNetCoreCli@2 task. Unfortunately without success as the service principal details were not accessible by the subsequent task as addSpnToEnvironment: true only allows to use the service principal details within the script executed by the Azure CLI task.
The solution
To make it working I had to execute the dotnet test command within an AzureCLI@2 task.
- task: AzureCLI@2
inputs:
azureSubscription: YOUR-SERVICE-CONNECTION-NAME
scriptType: pscore
addSpnToEnvironment: true
scriptLocation: inlineScript
inlineScript: |
dotnet-coverage collect "dotnet test ${{parameters.testProjects}} --configuration ${{parameters.buildConfiguration}}" -f xml -o "coverage.xml"
env:
ASPNETCORE_ENVIRONMENT: "Development"
Update from 10.12.2024
Instead of using DefaultAzureCredential you could make use of ChainedTokenCredential which allows you to explicitly specify the types of credentials to be used as well as the order in which they are called. In my opinion ChainedTokenCredential makes the code more obvious and understandable and therefore I’ll start using it as a replacement for DefaultAzureCredential.
To successfully execute integration tests in Az DevOps pipeline as explained above, make sure the following credential type is used in combination with ChainedTokenCredential when running in the Az DevOps pipeline.
new ChainedTokenCredential(new AzureCliCredential());
Leave a Reply