Follow

Running Soak Tests

BlazeMeter gives you all the resources that you need to execute a test over a continuous availability period - a Soak test.
This article will show you the steps that you should take to run a Soak test successfully.

If you wish to know how to write your initial script, test it locally with JMeter and run BlazeMeter SandBox Testing please refer to this article.

Agenda

  1. Setup the amount of Users-per-Engine using 1 Console & 1 Engine.
  2. Setup and test your Cluster (1 Console & 1-14 Engines).
  3. Run a full length Soak test.

Step 1: Setup The Amount Of Users-Per-Engine Using 1 Console & 1 Engine

Now that we're sure the script runs flawlessly in BlazeMeter, we need to figure out how many users we can apply to one engine.

If you can use the SandBox data to determine that - that's great!

Here is a way to figure this out without looking back on the SandBox test data.

Set your test configuration to:

  • Number of threads: 500
  • Ramp-up: 2400 seconds  
  • Iteration: Forever
  • Duration: 50 minutes
  • Use 1 engine (1 console will be included automatically)

Run the test and monitor your test's engine via the Monitoring Report.

If your engine didn't reach either a 70% CPU utilization or 80% memory usage (one time peaks can be ignored):

  • Change the amount of threads to 700 and run the test again
  • Raise the amount of threads until you get to 1,000 threads or 60% CPU

If your engine passed the 70% CPU utilization or 80% memory usage (one time peaks can be ignored):

  • Look at when your test first got to 70% - and see how many users you had at this point
  • Run the test again. This time, instead of putting a ramp-up of 500, enter the amount of users you got from the previous test
  • Put the ramp-up time you want for the real test (5-15 minutes is a great start) and set the duration to 50 minutes
  • Make sure you don't go over 70% CPU or 80% memory usage throughout the test

You can also go with a safer option and decrease 10% of the threads per engine.

Step 2 : Setup and Test Your Cluster

We now know how many threads we can get from 1 engine.
If the amount of threads loaded successfully using 1 engine answers your scenario's needs, you can go ahead and skip to step 3. 
At the end of this step, we'll know the amount of users one Cluster (test) can get us.

A Cluster is a logical container which has one console and 0-14 engines.

Even though you can create a test with more than 14 engines, it actually creates two clusters and clones your test.

So, at this step, we'll take the test from step 1, and change only the number of engines by raising them to 14 or less if you prefer.

Run the test for 60 minutes. While the test is running, go to the Monitoring Report and:

  1. Verify that none of the engines pass the 70% CPU, 80% Memory limit
  2. Locate your console label (You can see its IP in the 'logs' tab, console's log). This should not reach the 70% CPU or 80% Memory limit

If your console reached that limit - decrease the number of engines and run again until the console is within these limits.

The most important indicator that your Soak test will run successfully is that the Memory and CPU graphs are stabilizing after the ramp-up. If they are inclining linearly, the console or engines will ultimately crash and test will terminate.
See the report below for a good example of stable Memory and CPU graphs.

By the end of this step, you should know:

  1. The users per cluster you'll have.
  2. The hit/s per cluster you'll reach.

For more information about your Cluster's throughput, look at the statistics in the Aggregated Report.

Step 3 : Run a Full Length Soak Test

We've got to the final stage!

We know the script is working, we know how many users one engine can sustain, and we know how many users we can get from one Cluster.
Let's change the Duration of the test to the number of hours we want in our Soak test.

Example 12 hour report:

Have more questions? Submit a request

0 Comments

Article is closed for comments.