Start Testing Now BlazeMeter University Submit a Support Request My Support Tickets

BlazeMeter's Jenkins Plugin - A Guide

Avatar
BlazeMeter Support

BlazeMeter provides a plugin for executing automatic load test scripts in Jenkins, an open source continuous integration tool. The plugin is named the "BlazeMeter Plugin" or "BlazeMeter's Jenkins Plugin".

This BlazeMeter-Jenkins plugin allows you to load test your site with BlazeMeter and then to view and analyze the test report in Jenkins. It also allows you to set the final Build status as good, unstable or failed, based on the reported error percentage and response times KPIs.

What is Jenkins?

Jenkins is an open source continuous integration tool written in Java for automating tests. Builds can be started in various ways. For example, they can be triggered by a commit in a version control system, scheduled via a cron-like mechanism, or built when other builds have completed and by the request of a specific build URL.

How do I install BlazeMeter’s plugin for Jenkins?

  1. Login to Jenkins and go to the dashboard. Select ‘Manage Jenkins’ and then ‘Manage Plugins’. Then, from the top tabs, select ‘Available’. (You can also download BlazeMeter's Plugin to Jenkins).
  2. Hit 'Ctrl + F' / ‘CMD +F’ in your browser and search for BlazeMeter. Then mark the BlazeMeter check box.
  3. Scroll down and click ‘Download now and install after restart’.
  4. After the download starts, click to restart Jenkins.

jenkins plugin available plugins

How do I configure BlazeMeter's plugin for Jenkins?

  1. Once restarted, click 'Manage Jenkins' and then click Configure System.
  2. Scroll down to the 'BlazeMeter Cloud' section. You will see the URL 'https://a.blazemeter.com'.

    jenkins plugin server url

  3. Click Save.
  4. Click 'Credentials' --> 'System' --> 'Global Credentials (unrestricted)'.
  5. Click the 'Add Credentials' button and choose the "BlazeMeter Credentials" button. Add your BlazeMeter API Key Id and API Key Secret.
  6. Add a nickname for your key in the Description field.
  7. Leave the 'ID' field empty - it will be generated automatically once you save the credential.
  8. Click the 'Test BlazeMeter credentials' button.

jenkins plugin credentials

How do I configure a job using BlazeMeter's Jenkins plugin?

  1. Click on an existing job or create a new one, then press configure.
  2. Scroll down to the ‘Build’ steps. Click 'Add Build Step' and choose 'BlazeMeter'.

jenkins plugin add build steps

How do I set up the Build's test?

  1. If your keys were pre-configured correctly in the global settings, you will see your Workspaces and tests in two drop down menus.
  2. Click on the "Workspace ID" menu to choose the Workspace you want to see tests from.
  3. You will now see all your tests for your chosen Workspace in the drop down menu 'BlazeMeter Test'.
  4. Select a test to run.
  5. Now, you can either click 'Save' to save your Jenkins project and then click 'Build Now' to run the test. Or, you can use the plugin's additional configurations and only then save and build. There are a few options:
  • 'Download JTL report' checkbox - downloads the JTL report after finishing the test.
  • 'Download JUNIT report' - downloads the Junit report after finishing the test.
  • 'Advanced' - options include the following properties:
    - Path to the JTL and/or JUnit reports (if you checked the checkboxes)
    - Path to the main and additional test files. This files will be upload in your BlazeMeter test (override if this files already exists) before run test. (Read more here)
    - Set Notes and JMeter properties for pushing to the BlazeMeter load test
    -Interrupt FreeStyleProject build, if BlazeMeter Build Step was failed.
    - Set custom report link name to BlazeMeter report in your Jenkins build

jenkins plugin advanced button

jenkins plugin

Reports

Plugin supports downloading two kinds of reports: JUnit & JTL.

All reports are saved to ${WORKSPACE} by default, but this behavior can be overriden by setting paths. Path can contain Jenkins variables, which will be resolved into actual values.

When using plugin with master/agent Jenkins configuration, please keep in mind that permissions Jenkins have in master filesystem by default differ from agent ones. E.g. if you want to save jtl/junit artifacts outside of $JENKINS_DATA_FOLDER, you should set permissions to 757 for the folder you want to use at master.

On remote agent Jenkins agent by default has 777 permissions for folder in /home. E.g. if you're using ssh @ for connecting to agent from master, than jenkins agent will have 777 permissions for /home/<username>.

Adding notes to public report

There is an ability to add notes to public report. This feature is available both in Freestyle project and in pipeline mode.

  • open job;
  • go to BlazeMeter step;
  • expand Advanced section;
  • enter note to Notes field;

Pipeline

Refer to pipeline usage guidenotes:'a\n\b\n\t\r' will produce multiline note.

Adding JMeter session properties to test session

There is an ability for adding jmeter properties to test session.

  • open job configuration;
  • expand Advanced section;
  • enter session properties to Session properties field in the following format:key=value,key1=value1,key2=value2

Snippet for using session properties with pipeline

Sending properties for test session is currently not supported for multi tests.

How do I run a BlazeMeter test in Jenkins without a Plugin?

Jenkins, like any other CI tool for that matter, enables executing Shell commands. With shell commands you can execute API calls to start tests, create tests, update them, etc.


jenkins plugin execute shell

How do I build a job with BlazeMeter's Jenkins plugin?

jenkins plugin job menu

  1. Once inside the job, click 'Build Now'.
  2. Click the build that is running.
  3. Click 'Console Output' to see the build progress. You’ll see the test parameters and an update every few seconds.
  4. When the test finishes, you'll see a summary of the test results and their impact on the build.

You can now click on ‘BlazeMeter Report’ to see the test report , the same way you would see them on the BlazeMeter site.

You can also edit and compare test results to previous runs.

All logs and data generated during test runs are stored in a ZIP - which you can access directly from your BlazeMeter account, in the 'Logs' tab of a report.

Using BlazeMeter with Jenkins pipeline

See this link for more information:
https://github.com/Blazemeter/blazemeter-jenkins-plugin/blob/master/usingPipeline.md

Jenkins Reports

Jenkins’ Reports provide great out of the box intelligence and visualization. Let's take a look at them:

Performance Trend Report

The Trend Report presents information about the trends and robustness of the successful and failed test results, over time. This report is generated by fetching the JUnit.xml report when a test comes to an end. This capability is inherent to Jenkins and is provided by the common Performance Jenkins plugin.

jenkins plugin performance trend report


Performance Report

The Performance Report provides a basic KPI/Transactions report that visualizes a JTL that is an artifact of the test run. This report is built by fetching the JTL file when a test comes to an end.

jenkins plugin jtl report


Comprehensive Reporting Dashboard

A far more comprehensive reporting dashboard is available, during the test and after the test has ended. By pressing the BlazeMeter link, the BlazeMeter reports are visible providing a deep dive into the performance KPIs, allowing one to look and find performance bottlenecks.

jenkins plugin reporting dashboard

PASS/FAIL Criteria

The Plugin is using /ci-status request for setting the build result. Depending on this request, the plugin sets one of the available in Jenkins build result:

SUCCESS - This status is set in next cases:

  • test has no thresholds;
    test has thresholds and they were not violated;
    there were no errors in ci-status response;

UNSTABLE - Test was completed successfully, but there were errors in /ci-status response;

FAILURE - This status is set in next cases:

  • test has thresholds and they were violated;
  • test has no thresholds, but there were specific errors in /ci-status response:
    • "errors":[{"code": 70404,"message": "Session ended without load report data","details": null}]
    • "errors":[{"code":0,"Message":"Not enough resources"}]
  • jenkins job got error on /test/start request;
  • jenkins job got UNSTABLE status due to errors, but their were one of possible reasons, which caused FAILURE. E.g. FAILURE overrides UNSTABLE in all cases.

The default behaviour in case when request to BlazeMeter does not respond or fail is make job status as FAILURE. But you can change this status to UNSTABLE in 'Manage Jenkins' -> 'Configure System' -> 'BlazeMeter Cloud' section using checkbox 'Mark build as UNSTABLE if BlazeMeter does not respond with pass or fail'

jenkins plugin server url

The other way to make your build more stable is set number of request attempt that plugin will do to BlazeMeter. By default this value is 3 (plugin will repeat GET request to BlazeMeter if response was fail or BlazeMeter didn’t respond in time). Also you can set other number of request attempt using Java System properties:
Go to 'Manager Jenkins' -> 'Script Console' and set property 'bzm.request.retries.count' using command 'System.setProperty("bzm.request.retries.count", "5");'

jenkins plugin script console

Logging

This Plugin has <bzm-log> file, which can simplify debugging process in case of any issues with the running test.
This file is kept in a workspace. Its location differs depending on which kind of build (Freestyle job, or pipeline) is run.

If you run Freestyle job, then you can use the following URL:
http://<jenkins-host>/job/<job-name>/ws/<build-number>/

In case of pipeline, do the following to find log files:

  • go to http://<jenkins-host>/job/<job-name>/<build-number>/flowGraphTable/
  • in Step table click at Allocate node:Start
  • switch to Workspace and get into build-number folder.

Working with Proxy Server

Since version 4.2, the Setup Proxy configuration for master, and all agents are in the section,
‘Manage Jenkins’ > ‘Manage Plugins’ > ‘Advanced’

This config will be applied for each of the agents and master node.

jenkins proxy configuration

In case that you need to use some agent with other Proxy server, or without it, you should add JVM params to this agent in the section,
‘Manage Jenkins’ > ‘Manage Nodes’ > ‘Select Node’ > ‘Configure’ > ‘Launch method’ > ‘Advanced’ >’JVM Options’.

The argument -Dproxy.override=true means that this agent will use other proxy configuration.
In case that this agent wouldn’t use proxy server, simply pass this argument below:

jenkins proxy override

If you want to use other proxy server, after the 'proxy.override' option, also add the following options:

-Dhttp.proxyHost=<IP>
-Dhttp.proxyPort=<PORT>
-Dhttp.proxyUser=<USER>
-Dhttp.proxyPass=<PASSWORD>

jenkins jvm option proxy override

The other way for passing these Java properties to the agent without restart, is by using a Script Console, as described below,

System.setProperty("proxy.override", "true");
System.setProperty("-Dhttp.proxyHost", "<IP>");
System.setProperty("-Dhttp.proxyPort", "<PORT>");
System.setProperty("-Dhttp.proxyUser", "<USER>");
System.setProperty("-Dhttp.proxyPass", "<PASSWORD>");

jenkins script console system property

Set Custom report link name

Since version 4.2, you can set custom link name for link to your BlazeMeter report:

jenkins user test custom link

By default, the Max report link name length = 35, but you can increase this value using the Java System property, or Jenkins environment variable "bzm.reportLinkName.length" .

Using Job DSL - https://github.com/jenkinsci/blazemeter-plugin/wiki/Using-Job-DSL

Using BlazeMeter Jenkins plugin with Github - https://www.blazemeter.com/blog/how-to-keep-blazemeter-test-data-updated-with-github-and-jenkins

Learn more about running performance testing within Jenkins CI from out free on-demand recording, How to Scale Test Automation with Jenkins & JMeter.

Start performance testing now with BlazeMeter. Request a demo to learn more.

Have more questions? Submit a request

0 Comments

Article is closed for comments.