News »Browse Articles »
How Mature Are Companies Software Quality Management Processes In Today` Market?
0
How Mature Are Companies Software Quality Management Processes In Today` Market?
Executive Summary
Today, maintaining a competitive edge while managing costs is the challenge that most organizations face. High up on the IT Directors� to do list is innovation in the way costs are managed and contained, while at the same time providing a dynamic and responsive IT service to the business.
Based upon the industry standard Test Maturity Model integrated (TMMi), Experimentus undertook a survey across the IT Industry to understand the maturity of companies Software Quality Management processes.
the 100 plus companies, across 12 industry sectors who responded:
* 72.5% were at TMMi Level 1 heading for Level 2; meaning they are working in a chaotic, hero based way but starting to build project based processes.
* 27.5% were at TMMi Level 2 heading towards Level 3; meaning they have some established project based process and are moving towards implementing process at an organizational level.
* None of the respondents reached Level 3.
Notable Results
Interestingly, the Level 2 results suggest that although software testers believe they are good at designing tests and planning testing, they are not so good at setting goals, monitoring and managing the plans. They are also not very consistent with how they estimate for testing either.
The big surprise was to see how well planned test environments were, and that for the later stages of testing (e.g. User Acceptance Testing), �production like� test environments appear to exist fairly consistently.
Areas for concern
The most consistent weakness was around the collection and use of metrics, with a significant amount of respondents working without metrics altogether and therefore unable to say with any degree of confidence where they are or what they have done.
With over 100,000 qualified people through recognized examination boards, it is hard not to conclude that, despite being armed with the tools and knowledge to make an impact to the quality and cost of software delivery, no allowance is made to enable the skills learnt to be put into practice. With informed management, there is nothing stopping organizations benefiting from what the students have learnt through controlled management of change. After all, why invest in training and certification if they are then unable to put what they have learnt into practice.
Conclusion
The survey results reflect a view that Experimentus has had for a while, that �too many organisations are prepared to continue to fund poor, long winded, not repeatable, costly processes but will not seriously investigate making improvements that will bring about a significant increase in software quality together with considerable cost savings.�
At a time when Software Testing wants to be seen as a profession and Software Testers want to be seen as valued members of their delivery teams, the results of this survey have highlighted two areas for serious consideration by test teams and their management.
The first is around trust - given that over 100,000 people have qualifications gained through ISEB (Information Systems Examinations Board), and ISTQB (International Software Testing Qualifications Board), with the syllabi matching closely to the TMMi Foundations standards and methods, why is it that we appear unable to have some of the basics in place within the testing areas.
What is stopping these basic practices from being implemented?
Maybe focus is needed in ensuring the information learnt on courses is retained and practiced in the workplace through a controlled management of change. After all, why send them to be accredited if they cannot put into practice, best practice?
The second is around demonstrating that Testing is a profession.
In most professional disciplines, people are judged by their results and achievements. The survey indicated that more than 70% or respondents do not have metrics in place to monitor or manage testing goals. Metrics provide a valuable demonstration of results and achievements, which will help in raising their standing amongst their peers. If testing is serious about becoming a profession, it needs to get to grips with metrics to demonstrate its worth.
Some interesting results were also revealed: it�s the higher level tasks such as Test Policy, Test Strategy and the management and reporting of testing that Software Testers are not so good at. Maybe this is a direct result of software testers not getting the right training in management skills. A lot of the test managers we have met have had little or no formal training, in test or project management. The impacts of poorly managed projects are wide ranging, one example that comes to mind is the project that has been reporting good progress for 6 months, and the day before live suddenly says it is only 50% through testing. This actually happened on a project we were reviewing, the result was a 3 month delay to the go live date with major repercussions for the Project and Test Management team and additional cost of over �200k, as well as significant delays on projects that resources were due to move on to. This kind of situation is easy to fix with a little training and mentoring.
There are some bright lights shining in there around test design; however, the key issue that this survey highlights is our lack of focus on goals, objectives and metrics. Resolving these areas will make a significant difference to the benefits Software Testers and Software testing provide to their organizations.
Today, maintaining a competitive edge while managing costs is the challenge that most organizations face. High up on the IT Directors� to do list is innovation in the way costs are managed and contained, while at the same time providing a dynamic and responsive IT service to the business.
Based upon the industry standard Test Maturity Model integrated (TMMi), Experimentus undertook a survey across the IT Industry to understand the maturity of companies Software Quality Management processes.
the 100 plus companies, across 12 industry sectors who responded:
* 72.5% were at TMMi Level 1 heading for Level 2; meaning they are working in a chaotic, hero based way but starting to build project based processes.
* 27.5% were at TMMi Level 2 heading towards Level 3; meaning they have some established project based process and are moving towards implementing process at an organizational level.
* None of the respondents reached Level 3.
Notable Results
Interestingly, the Level 2 results suggest that although software testers believe they are good at designing tests and planning testing, they are not so good at setting goals, monitoring and managing the plans. They are also not very consistent with how they estimate for testing either.
The big surprise was to see how well planned test environments were, and that for the later stages of testing (e.g. User Acceptance Testing), �production like� test environments appear to exist fairly consistently.
Areas for concern
The most consistent weakness was around the collection and use of metrics, with a significant amount of respondents working without metrics altogether and therefore unable to say with any degree of confidence where they are or what they have done.
With over 100,000 qualified people through recognized examination boards, it is hard not to conclude that, despite being armed with the tools and knowledge to make an impact to the quality and cost of software delivery, no allowance is made to enable the skills learnt to be put into practice. With informed management, there is nothing stopping organizations benefiting from what the students have learnt through controlled management of change. After all, why invest in training and certification if they are then unable to put what they have learnt into practice.
Conclusion
The survey results reflect a view that Experimentus has had for a while, that �too many organisations are prepared to continue to fund poor, long winded, not repeatable, costly processes but will not seriously investigate making improvements that will bring about a significant increase in software quality together with considerable cost savings.�
At a time when Software Testing wants to be seen as a profession and Software Testers want to be seen as valued members of their delivery teams, the results of this survey have highlighted two areas for serious consideration by test teams and their management.
The first is around trust - given that over 100,000 people have qualifications gained through ISEB (Information Systems Examinations Board), and ISTQB (International Software Testing Qualifications Board), with the syllabi matching closely to the TMMi Foundations standards and methods, why is it that we appear unable to have some of the basics in place within the testing areas.
What is stopping these basic practices from being implemented?
Maybe focus is needed in ensuring the information learnt on courses is retained and practiced in the workplace through a controlled management of change. After all, why send them to be accredited if they cannot put into practice, best practice?
The second is around demonstrating that Testing is a profession.
In most professional disciplines, people are judged by their results and achievements. The survey indicated that more than 70% or respondents do not have metrics in place to monitor or manage testing goals. Metrics provide a valuable demonstration of results and achievements, which will help in raising their standing amongst their peers. If testing is serious about becoming a profession, it needs to get to grips with metrics to demonstrate its worth.
Some interesting results were also revealed: it�s the higher level tasks such as Test Policy, Test Strategy and the management and reporting of testing that Software Testers are not so good at. Maybe this is a direct result of software testers not getting the right training in management skills. A lot of the test managers we have met have had little or no formal training, in test or project management. The impacts of poorly managed projects are wide ranging, one example that comes to mind is the project that has been reporting good progress for 6 months, and the day before live suddenly says it is only 50% through testing. This actually happened on a project we were reviewing, the result was a 3 month delay to the go live date with major repercussions for the Project and Test Management team and additional cost of over �200k, as well as significant delays on projects that resources were due to move on to. This kind of situation is easy to fix with a little training and mentoring.
There are some bright lights shining in there around test design; however, the key issue that this survey highlights is our lack of focus on goals, objectives and metrics. Resolving these areas will make a significant difference to the benefits Software Testers and Software testing provide to their organizations.