News »Browse Articles »
Software Testing - Performance Testing Requirements
0
Software Testing - Performance Testing Requirements
Introduction
Performance is a "must have" feature. No matter how rich your product is functionally, if it fails to meet the performance expectations of your customer the product will be branded a failure.
Application architectural design decisions may be greatly influenced by the importance placed by the customer on one or more specific requirements. Incorrect design decisions, made at the outset of a project as a result of invalid assumptions, may become impossible to remedy downstream. The goal of performance testing is not to find bugs, but to eliminate bottlenecks and establish a baseline for future regression testing.
Where do you start?
Identify your customer needs
It is extremely important that you fully understand your customer`s intentions and requirements as early as possible regarding software performance i.e. the operating environment (both hardware and software) in which your product will be deployed and the manner in which it will be used.
So, how do you understand your customer’s requirements?
Classify their needs
End-User Requirements:
The users of your application don’t know or care what the results of the performance tests are, how many seconds it takes something to display on the screen past their threshold for "too long," or what the throughput is. The only thing application users know is that they either notice that the application seems slow or not based on what they have become accustomed to.
In order to quantify what the end user needs,
* Determine Application Functionality and the scenarios that they use frequently for their regular work.
* Verbalize and Capture Performance Requirements and Goals
* Quantify Captured Performance Requirements and Goals
* Record Performance Requirements and Goals based on their past and current experience
* Summarize it back to them reassure what you have recorded and ask for a formal sign off
Business Requirements
The Business Requirements would look like
*
Transaction response times < 3 seconds
*
Support 10000 users in standard conditions
*
The Amount of hardware they have currently that could be used to max capabilit
Technical Requirements
Technical requirements are usually not directly related to other categories of requirement .These would look something like
* CPU utilization rate of over 80% is observed for more than a few seconds is a defect
For knowing these kind of requirements ask the architects, or other team members who may be responsible for enforcing and/or making decisions regarding targets and thresholds, to share those decisions with you.
Standards, Compliance and Contracts
Determining performance characteristics in this category is conceptually Straight-forward, but there are often challenges in obtaining and interpreting the documents that spell out the characteristics of interest.
Often, contracts are viewed as proprietary and are not made available for review by non-executives. It is important to explain when facing roadblocks to reviewing contracts that the specific language and context of any statement related to application performance is critical to determining compliance.
The bottom line is simply this, don’t assume that you are done capturing goals and requirements until you have checked to see if some official or de-facto standard that your application is likely to be compared against.
Determining all the above information is particularly important when a customer is seeking your advice on purchasing suitable hardware and software specifications in order to best deploy your product or his product (in case you are a service provider). As there will undoubtedly be lead times for purchasing and configuring such items, your customer is likely to want this information at the earliest opportunity.
Now that you have identified the requirements, the following would be your action items.
* Derive Transactions mix (based on Application/System usage)
* Understand Data volumes needed.
* Understand Maximum allowable response times.
* Understand Minimum transaction throughput rates.
* Choose a Performance test tool which best suites your product needs
* Perform the tests
* Give updates on a regular basis.
Source:
http://www.testinggeek.com/index.php/testing-articles/131-performance-testing-re
Performance is a "must have" feature. No matter how rich your product is functionally, if it fails to meet the performance expectations of your customer the product will be branded a failure.
Application architectural design decisions may be greatly influenced by the importance placed by the customer on one or more specific requirements. Incorrect design decisions, made at the outset of a project as a result of invalid assumptions, may become impossible to remedy downstream. The goal of performance testing is not to find bugs, but to eliminate bottlenecks and establish a baseline for future regression testing.
Where do you start?
Identify your customer needs
It is extremely important that you fully understand your customer`s intentions and requirements as early as possible regarding software performance i.e. the operating environment (both hardware and software) in which your product will be deployed and the manner in which it will be used.
So, how do you understand your customer’s requirements?
Classify their needs
End-User Requirements:
The users of your application don’t know or care what the results of the performance tests are, how many seconds it takes something to display on the screen past their threshold for "too long," or what the throughput is. The only thing application users know is that they either notice that the application seems slow or not based on what they have become accustomed to.
In order to quantify what the end user needs,
* Determine Application Functionality and the scenarios that they use frequently for their regular work.
* Verbalize and Capture Performance Requirements and Goals
* Quantify Captured Performance Requirements and Goals
* Record Performance Requirements and Goals based on their past and current experience
* Summarize it back to them reassure what you have recorded and ask for a formal sign off
Business Requirements
The Business Requirements would look like
*
Transaction response times < 3 seconds
*
Support 10000 users in standard conditions
*
The Amount of hardware they have currently that could be used to max capabilit
Technical Requirements
Technical requirements are usually not directly related to other categories of requirement .These would look something like
* CPU utilization rate of over 80% is observed for more than a few seconds is a defect
For knowing these kind of requirements ask the architects, or other team members who may be responsible for enforcing and/or making decisions regarding targets and thresholds, to share those decisions with you.
Standards, Compliance and Contracts
Determining performance characteristics in this category is conceptually Straight-forward, but there are often challenges in obtaining and interpreting the documents that spell out the characteristics of interest.
Often, contracts are viewed as proprietary and are not made available for review by non-executives. It is important to explain when facing roadblocks to reviewing contracts that the specific language and context of any statement related to application performance is critical to determining compliance.
The bottom line is simply this, don’t assume that you are done capturing goals and requirements until you have checked to see if some official or de-facto standard that your application is likely to be compared against.
Determining all the above information is particularly important when a customer is seeking your advice on purchasing suitable hardware and software specifications in order to best deploy your product or his product (in case you are a service provider). As there will undoubtedly be lead times for purchasing and configuring such items, your customer is likely to want this information at the earliest opportunity.
Now that you have identified the requirements, the following would be your action items.
* Derive Transactions mix (based on Application/System usage)
* Understand Data volumes needed.
* Understand Maximum allowable response times.
* Understand Minimum transaction throughput rates.
* Choose a Performance test tool which best suites your product needs
* Perform the tests
* Give updates on a regular basis.
Source:
http://www.testinggeek.com/index.php/testing-articles/131-performance-testing-re