Manage Your Buckets

Your account is organized into Teams and Buckets. Buckets are a way of organizing your tests into logical groupings. Each bucket can contain multiple tests.

• Create a New Bucket in Your Team
• Manage Concurrent Tests in a Bucket
• Move a Test to a Different Bucket
• Bucket Utilization API

Create a New Bucket in Your Team

Follow these steps:

1. Log in to BlazeMeter and navigate to the API Monitoring tab.
2. Click Tests.
   You are taken to your dashboard.
3. Click the Bucket drop-down menu on the left side next to Tests.
   The list shows all your Teams and Buckets within the Teams.

   Note: You can also filter by Bucket name or a Key.

4. Click Create Bucket at the bottom of the list to create a new bucket. You will be taken to a Create Bucket page.
5. Select the Team to add the bucket to.
6. Name the Bucket.
7. Click Create Bucket.

Manage Concurrent Tests in a Bucket

As you start creating more tests in your buckets, there are certain limits that are good to keep in mind when creating and organizing your buckets and tests.

Currently, our buckets have certain limits when running a high number of tests, which is based on whether your team is using cloud locations and/or on-premise Radar Agents:

• 200 tests can be simultaneously running per cloud location (example: US Virginia) per bucket
• 200 tests can be simultaneously running amongst all on-premise agents (example: multiple tests running from 10 different agents) in a bucket

This is true for tests which are running or being executed concurrently. If you go over these limits, any additional test runs will be added to a queue, and will be executed as soon as the number of concurrent tests is less than 200. This does increase the chances of tests failing for taking more than 10 minutes to execute, which is the system limit for the execution time of a test. For more information, see Example Scenarios. To avoid potential issues with your tests such as delays or expiration, we recommend:

• Keeping the number of tests in a bucket to less than 150
• Running your tests from multiple cloud locations
• Splitting up tests into multiple buckets. For more information on how to move one or more tests to a different bucket, see Move a Test to a Different Bucket.
• Running your tests from multiple cloud locations. Set a different location for one or more tests. For more information, see Select Test Locations in Global Locations.
• Adjusting the schedule of your tests to minimize the number of tests running concurrently.

For each bucket with locations that are running greater than 85% of the maximum limit for concurrent tests per location per bucket, a warning message indicating the location name and the percentage of utilization of test concurrency is displayed on the dashboard for that bucket as follows:

Move a Test to a Different Bucket

Follow these steps:

1. Log in to BlazeMeter and navigate to the API Monitoring tab.
2. Click Tests.
   You are taken to your dashboard.
3. Click More... to expand the menu on the left side and click Settings.
4. Scroll down to the Move Test section.
5. Select a Bucket that you want to move the test to and click Move Test.
   

Note: When you move a test, all third-party service integrations will be removed and per-email notification settings will be reset.

Example Scenarios

The following example scenarios show real life situations where you go over the limit of the maximum number of tests running concurrently.

• Scenario 1
  Let's say you have 1 bucket with 250 tests in it, only one cloud location enabled for them of US Virginia, and every test is scheduled to run every minute. What is going to happen is, 200 of those tests will start running. The remaining 50 tests will be added to a queue, and as soon as the initial 200 tests finish running the remaining tests will start being executed. If your tests are long-running tests for any reason, this can lead to the queues in the system getting bigger, and that could lead to tests expiring because of timeout issues.
• Scenario 2
  Let's say you have 1 bucket with 200 tests in it, each test has two cloud locations enabled of US Virginia and US Illinois, and every test is scheduled to run every minute. Since each test has two cloud locations, every minute you'll have 400 tests running concurrently. But since each cloud location will only have a maximum of 200 tests running concurrently, as long as your tests finish running in under a minute, you shouldn't see any issues.
• Scenario 3
  Let's say you have 1 bucket with 100 tests in it, and each test has three Radar Agent locations, and every test is scheduled to run every minute. Since there are 3 locations, every minute 200 tests will start running among the 3 different agents, while 100 tests will be added to the queue and wait for the initial tests to be completed before they can start running. Like we mentioned above, on-premise agents work differently than our public locations. Even though in this scenario there are 3 separate agents representing 3 locations in your environment, they can still only run a maximum of 200 tests at the same time.

Bucket Utilization API

We have enhanced the Bucket Detail API to also list the percentage of concurrent tests running in each location compared to the limit of 200 tests per location per bucket. The API can be called with the list_utilizations_gt parameter that accepts a number (integer value) from 1 to 100 as follows:

v1/buckets/<bucket_key>?list_utilizations_gt=<value>

Note: If no value is specified (v1/buckets/<bucket_key>?list_utilizations_gt), the API will evaluate using the default threshold value of 85%.

The API output for threshold-percentage=80 with utilization will look like the following example:

api.runscope.com/v1/buckets/<bucket_key>?list_utilizations_gt=80

{
"data": {
 "auth_token": null,
 "default": false,
 "key": "ov2f2tq1floq",
 "name": "Mobile Apps",
 "team": {
 "name": "Mobile Team",
 "uuid": "7a7a0917-91d7-43ef-b8f4-fe31762167e0"
},
"verify_ssl": true,
"locations_utilization_%": {
 "remote": 82, 
 "us california": 86, 
 "us iowa": 90
}
},
"meta": {
"status": "success"
}

You can see that three locations exceeded the threshold value of 80%. All Radar agents 82%, US California: 86% and US Iowa: 90%.