Rectangle 27 0

Performing a Stress Test on Web Application?


Again, PAL reports help to analyze what happened during the tests - you often see very different bottlenecks on production environments.

For projects that don't have extreme performance requirements, we include basic performance testing in our testing; usually, we script out the tests using BadBoy, and import them into JMeter, replacing the login details and other thread-specific things. We then ramp these up to the level that the server is dealing with 100 requests per second; if the response time is less than 1 second, that's usually sufficient. We launch and move on with our lives.

For projects with extreme performance requirements, we still use BadBoy and JMeter, but put a lot of energy into understanding the bottlenecks on the servers on our test rig(web and database servers, usually). There's a good tool for analyzing Microsoft event logs which helps a lot with this. We typically find unexpected bottlenecks, which we optimize if possible; that gives us an application that is as fast as it can be on "1 web server, 1 database server". We then usually deploy to our target infrastructure, and use one of the "Jmeter in the cloud" services to re-run the tests at scale.

So, during development, we include very basic multi-user testing (using selenium), which checks for basic craziness like broken session management, obvious concurrency issues, and obvious resource contention problems. Non-trivial projects include this in the continuous integration process, so we get very regular feedback.

The key is to make sure you don't just run your stress tests, but also that you collect the information you need to understand the performance of your application.

We have developed a process that treats load and performance measurenment as a first-class concern - as you say, leaving it to the end of the project tends to lead to disappointment...

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


I played with JMeter. One think it could not not test was ASP.NET Webforms. The viewstate broke my tests. I am not shure why, but there are a couple of tools out there that dont handle viewstate right. My current project is ASP.NET MVC and JMeter works well with it.

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


  • Easy to get started with, and easy to use once you grasp the core concepts. (Ie, how to create a request, how to create an assertion, how to work with variables etc).
  • Has an active community and good resources to help you get up and running. Read the tutorials first and play with it for a while.
  • JMeter works by parsing the response text returned by the server. So if you're looking to validate any sort of javascript behaviours, you're out of luck.
  • Learning curve is steep for non-programmers. If you're familiar with regular expressions, you're already ahead of the game.
  • Open-Source/Free tool from the Apache project (helps with buy-in)
  • Reporting 'out of the box' leaves much to be desired, particularly for larger tests. In the test I mentioned above, I ended up having to write a quick console app to do some of the 'xml-logfile' to 'html' conversions. That was a few years ago though, so it's probable that this would no longer be required.
  • The UI is written in Swing. (ugh!)
  • There are large numbers of (insert expletive) idiots in the support forum asking stupid questions that could be easily solved if they'd give the documentation even a cursory glance. ('How do I use JMeter to stress-test my Windows GUI' shows up quite frequently).
  • Very scalable. I've run tests with 11 machines generating load on the server to the tune of almost a million hits/hour. It was much easier to setup than I was expecting.

Another JMeter related option to be aware of is JMeter as a service. These types of SaaS provide highly scalable JMeter together with much improved reporting.

I disagree that JMeter is very scalable. A million requests per hour is merely 278 requests/second, which - for being run on 11 machines - is extremely low compared to other tools. I would actually put JMeter's scalability on the Cons side.

It does however have a steep learning curve once you start getting to complicated tests, but it's well worth it. You can get up and running very quickly, and depending on what sort of stress-testing you want to do, that might be fine.

JMeter is an open-source load testing tool, written in Java. It's capable of testing a number of different server types (for example, web, web services, database, just about anything that uses requests basically).

please clarify, if JMeter can help you test application installed on a remote VPS ? I'm not sure as it is desktop version

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


A hard call as the learning curve is steep with both tools as you get into the more complicated scripting requirements for url rewriting, correlation, providing unique data per Virtual User and simulating first time or returning Users (by manipulating the HTTP Headers).

As this question is still open, I might as well weigh in.

Both tools will handle HTTP and HTTPS and have a proxy recorder to get you started. Both tools use the Controller model to drive multiple test agents so scalability is not an issue (given access to the Cloud).

From experience around 25 instances of 'headless browsers' can be executed from the Amazon M1 Small Instance.

Grinder can also use Clojure scripting.

Jmeter is driven from an XML style specification, that is constructed via a GUI.

Something new to add Headless browsers running multiple instances of Selenium WebDriver.

That said I would start with Jmeter as this tool has a huge following and there are many examples and tutorials on the web for using this tool. If and when you come to a 'road block', that is something you can't 'easily' do with Jmeter then have a look at the Grinder. The good news is both these tools have the same Java requirement and a 'mix and match' solution is not out of the question.

The good news is that over the past 5 or so years the Open Source tools have really matured and taken off in the space, the bad news is there are so many of them out there.

The scalability is compromised as more VMs will be needed to drive the load, as compared with a HTTP driver such as the Grinder or Jmeter. That said, if you are looking to drive 500 Virtual Users then with 20 Amazon Small Instances (6 cents an hour each) at a cost of just $1.20 per hour gives you load that is very close to the Real User Experience.

This is a relatively new approach because it relies on the availability of resources that can now be provisioned from the Cloud. With this approach a Selenium (WebDriver) script is taken and run within a headless browser (i.e. WebDriver = New HtmlUnitDriver()) driver in multiple threads.

What this means is that all of the correlation, url rewriting issues disappear as you repurpose your functional testing scripts to become performance testing scripts.

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


// your code starts here  
val scn = scenario("Scenario")  
     .exec(http("Page")
     .get("http://example.com")) 
// injecting 100 user enter code here's on above scenario.   
setUp(scn.inject(atOnceUsers(100)))
  • Easy to write the tests, don't scare if it's scala.
  • Using Akka and Netty which gives better performance compare to Jmeter Threading model

However you can make it as complicated as possible. One of the feature which stand out for Gatling is reporting which is very detail.

Let me give you simple example to write the code using Gatling Code:

We recently started using Gatling for load testing. I would highly recommend to try out this tool for load testing. We had used SOASTA and JMETER in past. Our main reason to consider Gatling is following:

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


A little late to this party. I agree that Pylot is the best up-and-coming open source tool out there. It's simple to use and is actively worked on by a great guy (Corey Goldberg). As the founder of OpenQA, I'm also happy that Pylot now is listed on our home page and uses some of our infrastructure (namely the forums).

However, I also recently decided that the entire concept of load testing was flawed: emulating HTTP traffic, with applications as complex as they have become, is a pain in the butt. That's why I created the commercial tool BrowserMob. It's an external load testing service that uses Selenium to control real web browsers when playing back load.

Sorry to blatantly pitch a commercial product, but hopefully the concept is interesting to some folks and at least gets them thinking about some new ways to deal with load testing when you have access to a bunch of extra hardware!

The approach obviously requires a ton more hardware than normal load testing techniques, but hardware is actually pretty cheap when you are using cloud computing. And a nice side effect of this is that the scripting is much easier than normal load testing. You don't have to do any advanced regex matching (like JMeter requires) to extract out cookies, .NET session state, Ajax request parameters, etc. Since you're using real browsers, they just do what they are supposed to do.

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


One more note, for our web application, I found that we had huge performance issues due to contention between threads over locks... so the moral was to think over the locking scheme very carefully. We ended up having worker threads to throttle too many requests using an asynchronous http handler, otherwise the application would just get overwhelmed and crash and burn. It meant a huge backlog could pile up, but at least the site would stay up.

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


You asked this question almost a year ago and I don't know if you still are looking for another way of benchmarking your website. However since this question is still not marked as solved I would like to suggest the free webservice LoadImpact (btw. not affiliated). Just got this link via twitter and would like to share this find. They create a reasonable good overview and for a few bucks more you get the "full impact mode". This probably sounds strange, but good luck pushing and braking your service :)

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


For a web based service, check out loader.io.

They also have an API.

This is a good alternative to testing your own machines with your own machines

loader.io is a free load testing service that allows you to stress test your web-apps/apis with thousands of concurrent connections.

Note
Rectangle 27 0

Performing a Stress Test on Web Application?


Note