Testing via Azure DevOps Pipeline – BlazeMeter

BlazeMeter tests can now be executed via Azure DevOps Pipelines by using our BlazeMeter Azure DevOps Extension.

How do I install BlazeMeter Extension in Azure DevOps?
How do I configure the BlazeMeter Extension to start a BlazeMeter Test?
Select Tasks
How to get Azure File Path using PowerShell

How do I install BlazeMeter Extension in Azure DevOps?

Log in to Azure DevOps.
Select your project.
Navigate to Pipelines -> Releases.
Click on +New. Select New Release Pipeline.
On the Select a template screen, start with an Empty Job.
Click on the text 1 job, 0 task to add a new task to the stage.
Click on + to Add a task to Agent Job.
When you add a task to the agent job, the Add tasks screen appears on your right. Search for BlazeMeter to see the BlazeMeter Integration extension in the marketplace.
Click the Get it free button.
You will now be redirected to the Azure Marketplace in a new tab.
1. Click Get it free.
2. Click Install to install and add the extension to your Organization.
Once added, switch to your other browser tab and click the Refresh link on the Add tasks screen. You will notice that the extension is now installed and ready to be added to a task.

How do I configure the BlazeMeter Extension to start a BlazeMeter Test?

This procedure requires that you have installed the BlazeMeter Azure DevOps Extension. Now that you have the extension installed, you can add it to a task to kick-off BlazeMeter tests.

On the Add tasks screen, search for BlazeMeter and click the Add button on the BlazeMeter Integration.

bzm integration add button

Select Tasks

You have the following options under Select Tasks:

Run Existing Performance Test
Create Performance Test
Create Multitest

Run Existing Performance Test

By default, Run Existing Performance Test is selected.

To run an existing test:

Enter your Test URL.
1. To find your Test URL, log in to your BlazeMeter account.
2. Click on Tests.
3. Select the test you wish to use.
  Test details appear.
4. Copy the URL of that page and paste it into the Test URL text box.
For the API Key and API Secret text box, default values will appear.
Note: Azure DevOps supports the use of variables.
- If you wish to create your own variables and store the API Key and API Secret values within those variables, you may leave the defaults. Refer to the next section on how to create variables in Azure DevOps.
- Alternatively, you can clear the default values and configure the actual API Key and API Secret on the screen. For more information about key creation, see BlazeMeter API Keys to learn more about how to generate API Key and API Secret within BlazeMeter.
(Optional) Define a Report Name.
(Optional) For Do you want to update the test files before the test execution?, choose one of the following options:
- No. This is the default.
- Yes, I want to update main test files and its dependencies.
- Yes, I want to update several test files using JSON.
(Optional) Enable Show Log if you want to see a running log of the test in the console.
(Optional) Enable Ignore SLA if you want the build to succeed regardless of the SLA.
Click Save.
Click Create Release.
Your release is ready.
Start your BlazeMeter test from the pipeline by clicking the Deploy button.

If you want to update main test files and dependencies to Run Existing Performance Test, you choose between "Yes, I want to update main test files and its dependencies" or "Yes, I want to update several test files using JSON".

If you select Yes, I want to update main test files and its dependencies:

Update the main test files by checking Update Main Test File. Then under Upload Test Script, upload the main test file.
If you want to upload dependencies as well, you have to check Upload Dependencies. Then under Upload Dependent Scripts you have to upload the dependencies.

Or, if you select Yes, I want to update several test files using JSON:

Upload the JSON file with content in the following format:

{
  "mainTestFile": "", // provide the main test file path
  "dependency" : "" // provide the dependency file path 
}

If you want to upload only the main test file, use the following syntax:

{
  "mainTestFile": "${maintest file path}",  
  "dependency" : ""
}

If you want to upload or update the dependency file, use the following syntax:

{
  "mainTestFile": "${maintest file path}",
  "dependency" : "${dependency File Path}"
}

Create Performance Test

If you want to create a performance test, select Create Performance Test.

For the API Key and API Secret text box, default values will appear.
Azure DevOps supports the use of variables.
- If you wish to create your own variables and store the API Key and API Secret values within those variables, you may leave the defaults. Refer to the next section on how to create variables in Azure DevOps.
- Alternatively, you can clear the default values and configure the actual API Key and API Secret on the screen. For more information about how to generate API Key and API Secret within BlazeMeter, see BlazeMeterAPI keys.
(Optional) Define a Report Name.
For Select uploaded file type, you can choose one of the following:
- Upload main test files and its dependencies
- Upload several test files using JSON
- By default, Upload main test files and its dependencies is selected.
You can update main test files by checking Update Main Test File. Then, under Upload Test Script, upload the main test file.
If you want to upload dependencies as well, you have to check Upload Dependencies. Then under Upload Dependent Scripts you have to upload the dependencies.
Or, if you select Upload several test files using JSON:

Upload a JSON file under Upload JSON File For Update Main Test File Or Upload Dependencies. Use the following format:
```
{
  "mainTestFile": "", // provide the main test file path
  "dependency" : ""  // provide the dependency file path
}
```
Or if you want to upload only main test file, use the following syntax:
```
{
   "mainTestFile": "${maintest file path}",
   "dependency" : ""
}
```
If you want to upload or update dependency file, use the following syntax:
```
{
   "mainTestFile": "${maintest file path}",
   "dependency" : "${dependency File Path}"
}
```
Specify a Test Name.
Specify a Project Id.

Create Multitest

If you want to create a multi performance test, select Create Multitest.

In the Test Name Input field, specify a Test Name.
In the Project Id field, you have to specify a Project Id.

In the Multitest File field, upload a multitest JSON file in the following format:

{
  "multitest" : [
  {
    "url" : "",         //Provide the Test Url
    "mainTestFile": "", //Provide the main test file path
    "dependency" : "",  //Provide the dependency file path
    "testName": "",     //Provide the performance test name 
    "totalUsers" : "",  //Provide the total users
    "durationMin" : "", //Provide the duration in minutes
    "rampUpTime" : ""   //Provide the ramp-up time
  } 
  ]
}

Examples:

If you want to create new Taurus Performance test, use the following format:

{
"multitest" : [
{
"url" : "",
"mainTestFile": "${mainTestFilePath}",
"dependency" : "",
"testName": "${testName}", 
"totalUsers" : "",
"durationMin" : "",
"rampUpTime" : ""
}
]
}

If you want to create a new performance JMX test:

{
"multitest" : [
{
"url" : "",
"mainTestFile": "${mainTestFilePath}",
"dependency" : "",
"testName": "${testName}", 
"totalUsers" : "${totalUser}",
"durationMin" : "${durationMin}",
"rampUpTime" : "${rempUpTime}"
}
]
}

If you want to add an existing test:


{
"multitest" : [
{
"url" : "https://a.blazemeter.com/app/#/accounts/${accountID}/workspaces/${workspaceID}/projects/${projectID}/tests/${testsID}/edit",
"mainTestFile": "",
"dependency" : "",
"testName": "", 
"totalUsers" : "",
"durationMin" : "",
"rampUpTime" : ""
}
]
}

If you want to add an existing test as well as upload or update the main test file:


{
"multitest" : [
{
"url" : "https://a.blazemeter.com/app/#/accounts/${accountID}/workspaces/${workspaceID}/projects/${projectID}/tests/${testsID}/edit",
"mainTestFile": "${maintest file path}",
"dependency" : "",
"testName": "", 
"totalUsers" : "",
"durationMin" : "",
"rampUpTime" : ""
}
]
}

If you want to add an existing test as well as upload or update the dependency file, use the following format:

{
"multitest" : [
{
"url" : "https://a.blazemeter.com/app/#/accounts/${accountID}/workspaces/${workspaceID}/projects/${projectID}/tests/${testsID}/edit",
"mainTestFile": "",
"dependency" : "${dependency File Path}",
"testName": "", 
"totalUsers" : "",
"durationMin" : "",
"rampUpTime" : ""
}
]
}

How to get Azure File Path using PowerShell

Under Add Task, search Powershell.
Click Add to add Powershell in your task.
Options appear on the right side.
Under Type, select Inline.

Enter your script:

Write-Host "Working Directory Path==>"$(System.DefaultWorkingDirectory)

Save the task.
Create a release.
Copy the path.

Create the main test file path or dependency file path in the following format:

${workingDirectoryPath}_${gitUserName}_${repositoryName}\\${MainTestFileName}

For example:

{azureDirectoryPath}_{userName}_{repoName}\\{fileName or folderName}
azureDirectoryPath = D:\\a\\r1\\a\\

Using Azure DevOps Variable groups

Log in to Azure DevOps.
Select your project.
Navigate to Pipelines -> Library.
On the Variable Groups screen, select + Variable group.
Enter your Variable group name. For example: BlazeMeter Keys. The Description is optional.
Click + Add under Variables.
Add the two variables APIKEY and APISECRET. For more information about key creation, see BlazeMeter API Keys.
Click Save.
Return to your Release pipeline by navigating to Pipelines -> Releases.
Select the Pipeline you created. Click Edit.
Once in edit mode, click the Variables tab and select Variable groups.
Click Link variable group. Select the name of your variable group and click Link. Your variable group is now linked with the release pipeline. This allows you to use this variable group inside multiple releases and tasks within the pipeline.

Once you have set your APIKEY and APISECRET as variables, reference them within your tasks as $(APIKEY) and $(APISECRET) respectively.

Where can I view the BlazeMeter test reports?

Once a release is successfully deployed, you will see a screen like below. If any failure criteria have been triggered, the task would show up as Failed.
Click on BlazeMeter Integration to view the log.
The log displays the test Report URL at the beginning of the test execution.
Navigate to that URL to view the detailed report on BlazeMeter.
If you enabled the Show Log checkbox in your task configuration, you see a detailed log as shown in the following sample screenshot.