Rectangle 27 4

We use SOAPUI to do the functional testing initially. However, SOAPUI is not good for regression and load testing.

Next step, we load the XML into JMeter.

It can drive load from multiple machines and we can easily stress our servers to the limits.

+1 neat, never used that before. very cool.

java - What are tools and best practices for testing web services? - S...

java web-services testing wsdl testing-strategies
Rectangle 27 1

Constant Timer would be best option as per your mentioned requirement.

JMeter: delay between each thread request a sampler - Stack Overflow

jmeter delay
Rectangle 27 1

I don't think that IOCBattle.com tells someone enough about real world use and best practices to rule out Ninject and Castle because they were much slower according to the results by MartinF who is the author of a little known IOC container.

Ninject is very easy to use and is in the Apress MVC 3 book. Unity is becoming popular as it is from Microsoft, and StructureMap is pretty nice (and has been favorable to many developers). They seem to perform pretty well. Unless someone with a heavy hit site has exposure to bench marking performance on each container or has done performance and load testing with concurrent users with a tool like JMeter.

I don't think some of these benchmarks accurately reflect the product. If your company must use the Microsoft stack, then hands down you must use Unity ( MEF lacks too many features for most situations). If you are "allowed" to use 3rd party / open source, then StructureMap, Ninject, Castle, and Autofac are the most popular. Again, Mark's book on DI has charts that break down which ones have auto-registration, xml configuration, lifetime.., this stuff becomes pretty important for an application.

.net - which IOC container is the most extensible and performant? - St...

.net ioc-container
Rectangle 27 22

A little late to this party. I agree that Pylot is the best up-and-coming open source tool out there. It's simple to use and is actively worked on by a great guy (Corey Goldberg). As the founder of OpenQA, I'm also happy that Pylot now is listed on our home page and uses some of our infrastructure (namely the forums).

However, I also recently decided that the entire concept of load testing was flawed: emulating HTTP traffic, with applications as complex as they have become, is a pain in the butt. That's why I created the commercial tool BrowserMob. It's an external load testing service that uses Selenium to control real web browsers when playing back load.

The approach obviously requires a ton more hardware than normal load testing techniques, but hardware is actually pretty cheap when you are using cloud computing. And a nice side effect of this is that the scripting is much easier than normal load testing. You don't have to do any advanced regex matching (like JMeter requires) to extract out cookies, .NET session state, Ajax request parameters, etc. Since you're using real browsers, they just do what they are supposed to do.

Sorry to blatantly pitch a commercial product, but hopefully the concept is interesting to some folks and at least gets them thinking about some new ways to deal with load testing when you have access to a bunch of extra hardware!

Performing a Stress Test on Web Application? - Stack Overflow

web-applications stress-testing performance webapplicationstresstool pylot
Rectangle 27 0

the best way to select JSON response in jmeter it is to do something like (?m) "nodeRef": "workspace://SpacesStore/idSpaceStore",\s* "name": "folder_for_testing-1372432881900",

(?m) - means start treat regexp as multi line oriented (\s*) - means any characters

jMeter RegEx JSON response - Stack Overflow

regex jmeter
Rectangle 27 0

For my below setup - HTTP Request A would be called by the first 4 threads, Threads 5,6,7 & 8 would call HTTP Request B

Parameterize the total no of users for the test.

cool thx for looking and answering my question. i've upvoted your answer but i can't select as the best yet b/c i'm sure i'll need to be able to increase the # of threads and I don't want to change the condition in the If controller every time.

Check the 'EDIT' - Please share if you find some other solution. Eager to learn.

@vlns, nothing new. I just took your idea and did something similar: ${__threadNum} % 4 == 1 or 0 in the 2nd if statement. it seems to be working for me. thx for your answer =)

How to alternate between two HTTP Requests in JMeter - Stack Overflow

jmeter
Rectangle 27 0

I recommend you to determine your requirements of web-site performance. But by default you can use "standard" response time limit about 10sec. Next, find user activity profile during day, week, month. Then you could choose base values of load for each profile. Find most frequent and most "heavy" user actions. Create test scenario with this actions (remember about percent ratio of actions, don't use all actions with one frequency). Limit load by throughput timer (set needed value of hits per minute). Number of threads in thread group select experimentally (for http load not needed many of them). Add plugins for jmeter from google-code. Use graphs "number of threads in second", "response time" and standard listener "summary report". Don't use standard graphs with high load, it's too slow. You can also save data to file from each listener (I strongly recommend this, and select csv format). Graphs show average points, so if you want precision pictures then build graphs manually in excel or calc. Grab performance metrics from server (CPU, memory, i/o, DB, net interface). Set ramp up period about 20-60 minutes and test about 1-2 hours. If all right, then test with 200% load profile or more to find the best performance. Also set test about 1-2 days. So you'll see server stability.

nice recommendation. i've dont have the SLA to read. I wish there's a nice article about this...

web stress and load testing with jmeter - How good is good? benchmark ...

jmeter load-testing
Rectangle 27 0

First, WebService(SOAP) Request is not the best way to test Webservices in JMeter, it will be deprecated in upcoming 2.9 version. HTTP Sampler is the one to choose as it performs much better.

Second, Connection Reset means your server has cut connection. It could be coming from the CPU which seems high but it's not sure.

If what you call "my comp" is the computer hosting JMeter started working slowly then your JMeter instance is overwhelmed by the number of threads (2003 or more?) you've configured. It can come from a lot of factors, read this:

She updated question. I think Response Latencies Over time looks as it looks as latenices haven't been saved. Is it the reason?

Error "Connection reset" in JMeter (SOAP XML web-service) - Stack Over...

jmeter
Rectangle 27 0

A few years ago I used JMeter for such a task. I generally enjoyed using it, though I never did much research on what else is available and I don't know if it's still actively developed.

user interface - Which is the best tool for automatic GUI performance ...

performance user-interface testing
Rectangle 27 0

ab still is the best choice. Chris Miles developed a patch which allows ab to take a list of request suffixes. Patch can be downloaded from here.

webserver - Static web pages (html) and files for benchmark web server...

webserver jmeter benchmarking
Rectangle 27 0

Make sure that the correct Classname/Test Method is selected in the JUnit Request Sampler. Make sure errors are appended by checking the boxes as in the image below. Use the fail() method to debug. Put it in your catch blocks or wherever needed. If that seems too tedious to you then write your own mini logger (or use an existing one) and append all errors to a file. (Probably the best option). Just throwing it out there: it is possible to use JOptionPane.showMessageDialog(null, e.toString()); to debug by adding the rt.jar from your jdk folder. This can get annoying though...

JUnit Scripts Does not run with JMeter - Stack Overflow

jmeter
Rectangle 27 0

The quick and dirty way would be to use an Interleave Controller and add Sampler A 4 times and Sampler B 1 time. But, that is very, very dirty.

A better way would be to use the Switch Controller. Then you can execute Sampler A or B depending on a variable you can set and change on a test and thread basis. (Initialise per test and change per thread.)

The variable you need can be found under the Config Elements and is named Counter.

Thank you for your answer, but I'm confused about changing variable value when running. How can I change this per thread? How should I use this Counter element for this mean?

The counter does that for your. The counter increments by n every thread.

How to configure jmeter to run sampler tests through percentage of thr...

jmeter
Rectangle 27 0

The best way to provide Jmeter with a list of parameters is to use a CSV file: http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config You can configure the CSV dataset config to do make every thread use a different line in the CSV. Each engine will need to have its own unique CSV file, because the sharing mode does nto work between engines in distributed testing (you can try to automate this part, this can be interesting to do :) ).

The login sampler will use the parameters loaded by from the CSV file, so for every login it will use a different line.

Distributed testing is pretty simple: http://jmeter.apache.org/usermanual/remote-test.html Keep in mind that running 100K concurrent users on a single Jmeter load engine will be hard (Jmeter consumes resources on the server, so you will need lots of CPU and memory). So you should also monitor the engines. Also 1M users will cause a lot of data that the engines will send back to the console, so you might need to start a bunch of distributed tests in parallel, and at the end aggregate the results.

I need the configurable data to be json object, since I am sending the JSON in the request body. Is that also possible to implement with CSV data config set?

Currently there is no config element which can read JSON format. You can use a beanshell script to try and format a JSON file to the parameters. Another option is to use the HTTP sampler 'body data' option to build a JSON request using the parameters in the CSV.

e.g. {     "name": "${name}",     "username": "${Uname}",     "password": "${Pword}" }

load testing - Simulate millions of users with JMeter - Stack Overflow

jmeter load-testing
Rectangle 27 0

Use JMeter's "HTTP Proxy Server" to record the WCF calls with your normal testclient, and then play them back later when testing. This is what I have experienced to be fastest, and gives the best test-cases (because you record them with your normal client, or test client of choice).

Set up JMeters HTTP Proxy Server as per instructions. Then, make sure the WCF (or any SOAP) client use that proxy. The important part of the WCF client configuation is (replace my ... with normal config):

<system.serviceModel>
    <bindings>
    ...
    <wsHttpBinding>
    <binding ...  proxyAddress="http://proxyServerName:8080" useDefaultWebProxy="false" ...>
    ...
        <security mode="None">
            <message establishSecurityContext="false"/>
            <transport clientCredentialType="None"/>
        </security>

proxyServerName is localhost, if the WCF client runs on the same machine as JMeter (normal when creating the test cases).

Also, I got an error message using HTTP Proxy, if I did not turn off security as shown above. The same security settings must also be at the WCF service server.

c# - How to test a WCF Webservice with JMeter? - Stack Overflow

c# web-services soap jmeter
Rectangle 27 0

The best sources of Jmeter documentation I know about are in Documentation section at JMeter website: user manual and best practices (for starters), Component Reference and Functions reference (once you are comfortable with basic Jmeter capabilities).

Especially you should read 4th, 5th and 6th sections of User Manual that will give you answers to those and many other questions that you'll have.

  • Use PostProcessor. I guess the most helpful will be Regular Expression Extractor that will extract data to variable and then you can use this variable as ${variable} in your test
  • I guess you mean test results by output. If this is case, use Listeners. In my experience the most helpful listeners are Simple Data Writer, View Results Tree (only for debugging), Response Latencies over Time from Jmeter plugins. Note that only Simple Data Writer should be enabled when you'll eventually run your test against SUT to minimize resources usage.

Once you are comfortable with JMeter capabilities note that some useful functionality is absent in it but exists in JMeter plugins:

performance - Writing a simple JMeter script - Stack Overflow

performance http automation load jmeter
Rectangle 27 0

You can use WebStressTool (WCAT) (from Microsoft, I think). From a Desktop client you will be limited to 10 concurrent connections (if you don't override the registry somewhere). I think you can "coordinate" "drones" using the web stress tool, i.e. do the stress from multiple pc's.

settings
{
    counters
    {
        interval = 10;
        counter = "Memory\\Available MBytes";
    }
}
{

    warmup   = 30;
    duration  = 120;
    cooldown  = 100;

    transaction
    {

        id = test;
        weight = 1000;

        request
        {
        url = /AAA_ST/Web/Index.aspx;
        }

    }
}

Also see: blogs.msdn.com/b/alikl/archive/2008/03/09/ . The link could eventually go missing, but it is MSDN, so it should be there for some time.

Will WCAT tool provide me data on Read latency - TTFB and Write throughputs after the runs?

According to the official site (iis.net/downloads/community/2007/05/wcat-63-(x86)), it supports these: "Supports Performance Counter integration" AND "Measures throughput and response time". Using the performance counter queues counters, you should see at what point it starts having to "stop" servicing requests as the queue length will grow.

You should be able to run a script for different sizes and then log them for comparison. I don't think it is unfathomable to consider that process scriptable.

Best HTTP Performance and Scale testing tool to measure throughput and...

performance http jmeter
Rectangle 27 0

The best way to provide Jmeter with a list of parameters is to use a CSV file: http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config You can configure the CSV dataset config to do make every thread use a different line in the CSV. Each engine will need to have its own unique CSV file, because the sharing mode does nto work between engines in distributed testing (you can try to automate this part, this can be interesting to do :) ).

The login sampler will use the parameters loaded by from the CSV file, so for every login it will use a different line.

Distributed testing is pretty simple: http://jmeter.apache.org/usermanual/remote-test.html Keep in mind that running 100K concurrent users on a single Jmeter load engine will be hard (Jmeter consumes resources on the server, so you will need lots of CPU and memory). So you should also monitor the engines. Also 1M users will cause a lot of data that the engines will send back to the console, so you might need to start a bunch of distributed tests in parallel, and at the end aggregate the results.

I need the configurable data to be json object, since I am sending the JSON in the request body. Is that also possible to implement with CSV data config set?

Currently there is no config element which can read JSON format. You can use a beanshell script to try and format a JSON file to the parameters. Another option is to use the HTTP sampler 'body data' option to build a JSON request using the parameters in the CSV.

e.g. {     "name": "${name}",     "username": "${Uname}",     "password": "${Pword}" }

load testing - Simulate millions of users with JMeter - Stack Overflow

jmeter load-testing
Rectangle 27 0

What do you mean by "Performance testing of Thick client"? If it is about how fast it launches, downloads or operates than you need to loop into desktop automation software. The best open source tools currently available for desktop testing automation are:

If your application does network communication and your scenario assumes simulating hundreds or thousands of simultaneous thick clients communicating with the server using the following open-source tools can help:

If your application supports proxy (given .jnlp - it is Java-based application and it surely does) and the protocol is HTTP(S) you will be able to record test scenario and replay. The Grinder tool recording proxy can go a lower level and perform TCP protocol recording.

For more information on main features comparison for performance testing tools listed above look into Open Source Load Testing Tools: Which One Should You Use? post.

If your application uses a form of secure communication or proprietary protocol you should be able to replicate it's behavior by plugging your application libraries to JMeter, Grinder or Gatling and use their multithreading and reporting features to perform massive load testing.

thx for the reply, i want to performance test application by sending many simultaneous client and then checking server performance.

jmeter - Performance testing of Thick client - Stack Overflow

client-server jmeter performance-testing jnlp loadrunner
Rectangle 27 0

If you're new to JMeter the best way to get things done is recording your user ID generation steps using one of the following approaches:

  • JMeter bundled HTTP(S) Test Script Recorder which acts like a browser proxy
  • A Google Chrome Extension which is capable of recording JMeter .jmx scripts right from your browser.

You can see what requests are being sent and HTML responses using handy View Results Tree listener.

However if clicking on link isn't being recorded, it's being handled by client-side Javascript. JMeter is not a browser and isn't capable of executing Javascript so you'll have to use WebDriver Sampler as Releasematic suggests.

JMeter - link clicking to generate a username - Stack Overflow

jmeter