Glossary – BlazeMeter

Admin View

A page view that displays administration-specific information. Only accessible by account admins.

agent

An automated program that either gathers information or performs a task without user intervention. For example, a service that checks if specified tests have been updated. Formerly known as ship.

agent log

A detailed list of an agent's or application's activities, useful for tracking service use and troubleshooting.

API model

A list of operations consisting of GET, PUT, POST, DELETE methods and their parameters. Users can either import API definitions from a Swagger file or create the model manually. API models ensure consistency in API testing.

API Monitoring (formerly Runscope)

A feature that provides insight into the traffic running through customer apps and infrastructure. API monitoring runs API monitors on a continuous schedule to give visibility into API errors so that users can prevent, identify and solve issues fast.

API Test Creator

A testing tool that allows users to monitor public, private, and third-party APIs in seconds, as well as evaluate the uptime, performance, and correctness of an API via API tests.

API user key

A unique 20-character alphanumeric string for a BlazeMeter user. This key is used for user authentication when communicating with third-party tools.

Asset Catalog

A list of information about MAR files and transactions. MAR files are portable copies of virtual services that are proprietary to Broadcom's enterprise Service Virtualization product.

AWS (Amazon Web Services)

A collection of remote computing services that together make up a cloud computing platform, offered over the Internet by Amazon.com. BlazeMeter uses Amazon's EC2 service, which enables you to launch a dedicated cluster from one of AWS's regions.

baseline comparison

A method for identifying degradations and bottlenecks. Users select a test run as a baseline against which subsequent runs can be compared. In addition, subsequent test runs can automatically be marked as failed if the performance degrades compared to the baseline. Users can find the related changes in code, and quickly take action to resolve issues.

BlazeMeter agent

A server that performs load generation tasks within a Private location. Formerly known as 'ship'.

BlazeMeter Chrome Extension

A free tool, available from the Google Play Store, that enables users to record, upload, and run test scripts for their web applications. The Chrome Extension supports recording JMX, JSON or YML files, for running in JMeter, Taurus or BlazeMeter, as well as both JMeter and Selenium scripts, automatically and simultaneously.

BlazeMeter engine

A standalone dedicated environment for running tests. BlazeMeter engines run on Taurus and support various different open-source testing tools.

bucket

A logical grouping of one or more tests.

CA Bundle

The file that contains root and intermediate certificates. Together with your server certificate (issued specifically for your domain), these files complete the SSL chain of trust. The chain is required to improve the compatibility of the certificates with web browsers, email clients, and mobile devices.

cloud computing

The delivery of computing as a service rather than as a product, whereby shared resources, software, and information are provided to computers and other devices as a utility (like an electricity grid) over a network (typically the Internet). Clouds can be classified as public, private or hybrid.

cluster

A collection of virtual servers in the cloud. BlazeMeter reduces the time required to set up and run tests to minutes, allowing you to quickly scale capacity, both up and down, as your testing requirements change.

concurrent users

The number of users accessing the system simultaneously.

ConfigMap

A Kubernetes configuration that is used during either application initialization or runtime.

connect time

A measurement of the time taken to establish the connection, including the SSL handshake.

crane agent

An automated program that performs Crane-related tasks in BlazeMeter. Crane is a Docker orchestration tool that reads either JSON or YAML configuration files that describe how to run containers.

data dependency

A directed relationship between BlazeMeter and existing test data in the database. BlazeMeter uses this test data to prepare the environment against which a test runs. A data dependency has a name, description, sequence, and source.

data entity

A container for data parameters. A data entity can contain one or more data parameters from various data sources. GUI Functional tests currently support only a single data entity, Performance tests support multiple data entities.

data parameter

A constant, variable, or expression that is used to pass values between tests and BlazeMeter. The first letter of a data parameter is always a dollar sign, and the parameter name is surrounded by two braces, for example, ${my_variable} The same test can iterate over rows of test data multiple times and perform test runs with a fresh set of values each time. Parameter names must be unique within each test.

debug test

A small-scale test, run against a logical copy of a test configuration. Debug tests are free and allow users to validate test configurations.

dedicated IP

A static IP address, assigned to a BlazeMeter engine to allow connection behind a corporate firewall. Each IP address belongs to a specific region, such as US East (Virginia), EU West (London), or any other Google Cloud location.

dynamic response

A referenced value in a transaction request that is not constant and that changes with time. A dynamic response is used in dynamic testing to validate that the referenced value is repeated appropriately in the response body.

There are two key components in a dynamic response:

Parameters that control which data is returned, often based on request values
Helpers that control the format of the returned data

End User Experience Monitoring

A feature that provides insights into the user experience when running load tests to measure application server performance.

engine

A virtual server that generates the actual load and simulates the number of threads (virtual users) specified in the script provided. BlazeMeter generates load using the open-source Taurus architecture.

enhanced logging

The recording of detailed request and response information in the trace.jtl file that is added to the artifacts.zip file for debug test runs.

environment

A group of configuration settings (initial variables, locations, notifications, integrations, and so on) used when running a test. Every test has at least one environment, but users can create additional environments as needed. For common settings (base URLs, API keys) that can be used across all tests within a bucket, a shared environment can be created.

Functional Testing

A module that lets users ensure that the GUI (graphical user interface) functionalities of an application work as intended by checking screens and controls like menus, buttons, and icons. Functional Testing uses Selenium on the cloud, which in turn generates a robust report that allows users to review how a test performed step-by-step, even providing a recorded video of the test as it executed in an engine's web browser.

graceful shutdown

A process for sending a signal to close a test, archive test and log files, and then generating an artifacts.zip archive.

GUI

An abbreviation for "graphical user interface", referring to the graphical on-screen window a user clicks and types within. (Also referred to as UI, noted below in this glossary.)

HAR File

An HTTP Archive file is a JSON file used by BlazeMeter for logging a web browser's interaction with a site under test.

harbor (deprecated)

An on-premise private location.

helper

An application that lets users generate request matcher conditions for transactions. In the request body matcher, users can define conditions on the request of various types such as XML, JSON, RegEx, matches XPath, matches JSON Path, and so on.

hits per second (hits/s)

The number of HTTP/s requests per second to a Web page that are generated by a test. The number of hits a Web page receives equals the number of times a part of that page has been accessed. The number of hits per second captured in a report depends on the script configuration. Refer to the Summary Report guide for additional details.

iteration

The number of times an entire test (or an entire thread group, when using JMeter) is performed.

JMeter

An open-source software testing solution originally created by the Apache Software Foundation. Apache JMeter is a popular tool for performance testing.

JMeter (Apache JMeter™)

An open-source Java software application designed to load test functional behavior and measure performance. BlazeMeter is fully compatible with JMeter, and addresses its limitations such as scalability, stability, and reporting.

JTL file

A text file created by JMeter that contains the results of a test run. There are two types of JTL files: XML and CSV files. BlazeMeter reports are stored with the ".jtl" file extension.

KB/s (KBytes/s)

The speed of data transfer on a network, measured in multiples of 1,024 bytes per second.

KPI (Key Performance Indicator)

A measurable value, for example, an average response time, CPU, memory, bandwidth, and so on.

latency

The time interval between the instant at which a request is sent and processed on the server side, and the instant at which the client receives the first byte.

load configuration

A feature that lets users override frequently-changed parameters from within the BlazeMeter UI or API, eliminating the need to edit hard-coded values in their scripts and re-upload them between tests.

load distribution

A feature that lets users run load tests across multiple locations. This selection can be modified before each run. This means that users can run the same test several times and choose a different location each time.

load origin location

A specific geographic IP address from which users can run load testing. This selection can be modified before each run so that users can run the same test several times and each time choose a different location in the world. Users can load test from multiple locations with multiple tests with different network emulation values for maximum coverage.

low scale test run

A free option for executing a debug test with minimal resources, such as 10 threads, and for a maximum of 5 minutes or 100 iterations, whichever occurs first.

MAR file

A portable copy of a virtual service that is proprietary to Service Virtualization, a Broadcom product. Users can store MAR files in the BlazeMeter Asset Catalog so that they have a single place to find and view any available virtual service.

median

A standard statistical measure. The median is the same as the 50th percentile where half of the samples are smaller than the median, and half are larger.

mock service

A virtual web service that simulates a response to a web service request. A mock service creates a virtual environment to test a web service. This feature lets users test a web service without having access to a full test environment. There are two types of mock services: Transaction and MAR.

mock service template

A common collection of transactions used to generate a mock service for those transactions. Templates are persistent objects that can either directly create mock services or load transactions into a running mock service when associated with a test.

multi-test

A feature that lets users run multiple-scenario performance tests by combining existing BlazeMeter tests.

negative testing

The process of using invalid input data, or undesired user behavior, to check for unexpected system errors. Negative testing is a best practice to ensure complete test coverage in software development. BlazeMeter by default marks a test as "failed" if the response code returned is anything other than a 2xx code (such as a "200" code, meaning "OK"). For negative testing purposes, users can use the "assume-success" option to change the response codes that trigger a test to be marked as "failed".

network emulation

A technique for testing the performance of web applications over a virtual network. Network emulation allows users to impair the connection between a BlazeMeter test engine and the system(s) under test to observe the impact on key performance indicators (KPIs).

object

A GUI element of an application under test, such as a button or a text fields, to which users apply test actions.

OPL (deprecated)

An on-premise location. Former term for a private location.

PaaS (Platform as a Service)

A category of cloud computing services that provides a computing platform and a solution stack.

partial load

A failure to deliver the expected pressure on a system under test (SUT). This issue invalidates a load test because the SUT is not subjected to the intended normal and expected peak conditions. Partial load commonly occurs because a test-related BlazeMeter engine is either defunct or fails to start.

Performance Testing

A BlazeMeter module that lets users ensure that their application servers can handle a full load of users performing various actions concurrently when an application goes live. The Performance tab provides users with a wide range of options for testing, starting with either running a single performance test or a combination of multiple performance tests executed simultaneously, known as a Multi-Test. BlazeMeter cloud testing leverages cloud computing resources and models to enable all aspects of load testing in a highly cost-effective manner.

private location

An on-premise environment for testing applications or creating mock services behind a firewall.

project

A logical grouping of tests and reports within a workspace.

proxy recorder

A feature that lets users record and create JMeter performance test scenarios for mobile and web applications, even when using a secure connection. The proxy recorder works with both web mobile browsers and native mobile apps.

radar agent

An automated program that enables BlazeMeter API Monitoring to perform tasks on private APIs that are not available over the public internet. By running the agent within a customer's infrastructure, BlazeMeter API monitors can run against endpoints that are otherwise inaccessible to the cloud agents.

ramp-up

The period of time required to start all threads (virtual users).

remote control

A performance testing feature that allows users to remotely change the value of JMeter properties in real time. Remote control lets users change the behavior of tests that are running.

request per second (rps)

A set of individual visits to a website or Web page expressed as a measure of web traffic volume. The total number of requests divided by the number of seconds that elapse over the duration of the test. For example, a site that has 54,000 virtual visitors during a 30-minute period averages 30 requests (hits) per second during that span of time. A.k.a. Hit/s

reusable snippet

A chunk of JavaScript code that can be reused across requests. This feature is useful if users need to apply the same script logic to many different requests and tests (for example, schema validations, default assertions, and so on).

role-based access control (RBAC)

A feature that enables teams to manage user access to managing, editing, and viewing specific tests, buckets, and account features.

Runscope

A SaaS-based synthetic monitoring solution that enables users to ensure that APIs are up, performant, and returning the expected data. Proactive monitoring of APIs allows users to know if problems exist before their customers do. This solution allows users to minimize the business impact of application issues caused by APIs. sample An HTTP(s) request sent as a result of an action taken by a test script.

Script Engine

A tool for constructing API tests and validating API responses. The Script Engine is ideal for cases that require more flexibility than simple assertions and variables. Scripts, assertions, and variables work together seamlessly, each being evaluated at different points in the lifecycle of a test step.

scriptless functional test

A test method that lets users create and edit GUI functional tests manually by dropping pre-defined actions and objects into a test scenario. This approach helps testers avoid writing and uploading scripts manually. Every time a user changes the building blocks, the Taurus script behind the scenario editor is updated, and the test can be run immediately.

secret

A variable consisting of a key/value pair, where the value is encrypted and hidden. This feature allows all team members to use the variables in their tests with a built-in function {{get_secret(key)}}.

Selenium

An open-source testing framework that makes direct calls to the Web browser using the browser’s native support for automation. Selenium scripts are run at predefined intervals throughout the duration of a test, resulting in simulating an actual user browsing the web application. Selenium is a popular tool for GUI functional testing.

service

An application, routine, or process that performs a specific BlazeMeter function to support other applications.

shared environment

A group of common settings (base URLs, API keys, and so on) that can be used across all tests within a bucket.

ShiftLeft Converter

A software utility that changes test scripts from one format into another for use by BlazeMeter or open-source platforms supported by BlazeMeter. The converter can convert LoadRunner HTTP to JMeter, LoadRunner TruClient to Selenium, and SoapUI test scenarios to Taurus and BlazeMeter.

tag

A word or short phrase that users can associate with tests to describe their purpose.

Taurus

An open-source automation tool created by Blazemeter that provides an abstraction layer over JMeter, Grinder, Gatling, and even Selenium tests. Taurus tests are written using JSON or YAML. Taurus lets testers run existing scripts that are written in JMeter, Grinder, Gatling, or Selenium. An acronym for Test Automation Running Smoothly.

test data

The information that is used as input to perform the tests for identifying and locating application defects. BlazeMeter includes powerful capabilities for generating test data, either through proprietary functionality or through third-party integrations.

think time

A delay that is used in BlazeMeter to simulate slow human responses or network latency in mock services. The delay can be a fixed value or can be sampled from a random distribution which makes it easy for users to simulate different types of downstream latencies. Think time introduces irregular behavior that is difficult to produce from a real web service or a regular test script.

thread group

A group of threads that execute the same test scenario and a basic element of a Jmeter test plan. A thread group holds other test elements such as controllers, samplers, config elements, listeners, and so on.

transaction

A request/response pair that is associated with a given mock service. For example, in the Swagger Petstore, GET /pet/{petId} and the associated response is a transaction. A request in a transaction can be a full, valid request, like the Petstore example, or an expression that can be matched to a broader range of real requests. BlazeMeter comes with four pre-configured groups of transactions: AWS S3; Facebook Login; Salesforce Login; Open Banking.

trigger URL

A server-relative or absolute URL stored in the configuration settings of an environment. When BlazeMeter detects a trigger URL in an HTTP request, an action is initiated according to the URL configuration.

UI (user interface)

The graphical on-screen window a user clicks and types within. (Also referred to as GUI, noted above in this glossary.)

ultimate thread group

A group of threads that are highly customizable and that let users apply conditional load on the server. Ultimate thread groups allow users to create workload model for spike tests or step-up tests, unlike basic thread groups.

variable unit (VU)

A metric that measures usage of all capabilities across the entire BlazeMeter platform. A VU ceiling signifies the maximum concurrency that you can leverage across the platform as a whole at any time, taking into account various usage metrics such as the number of virtual users, browser session executions, running mock services, mock service transactions, test data usage, and running tests. See BlazeMeter's Credit Types and How They are Charged

variable unit hour (VUH)

Formerly known as Virtual User Hour. A metric used to measure performance testing and GUI functional testing usage. See BlazeMeter's Credit Types and How They are Charged.

virtual service environment

A containerized, cloud-native environment that is used to emulate the behavior of software components connected to an application under test.

virtual user (VU)

An emulation of the activities performed by an actual user of the application under test. Virtual users are used in performance testing metrics.

VPC (virtual private cloud)

An on-demand configurable pool of shared computing resources allocated within a public cloud environment, providing a certain level of isolation between the different organizations using the resources.

waterfall report

A type of data visualization that shows what a user would see in their web browser at different points during the load test. These reports are especially helpful when trying to debug why a certain page failed to load properly from a user point of view at a certain point in the load test.

workspace

A separate sub-account, with its own distinct set of assets and access control. Each workspace has a separate set of tests, reports, projects, shared folders, private locations, and dedicated IPs. A user can be assigned a different role (manager, tester, or viewer) in each workspace they have access to.