Quality assurance is a process of ensuring the product that is being developed has the right level of quality and that it meets customers’ expectations in terms of its functionality, reliability, efficiency, price, design etc.

Close picture

In my previous post, Why is quality assurance important in software development?, I talked about the importance and benefits of a properly established quality assurance process.

When it comes to software development, the main risks related to not having the right level of quality assurance include:

• undetected bugs and errors,
• system misbehaviours,
• poor system security,
• unstable performance.

All those risks have an impact on how businesses operate and how they are perceived by their clients. Poor quality assurance may mean missed deadlines, financial losses, reputational damage, unsatisfied end users, less customers deciding to buy the product. But sometimes the consequences may be far more dramatic.

Real life examples of why quality assurance really matters

The history is full of events showing why quality assurance matters. Here are some of the most interesting ones.

The global financial crisis, 2008

The global financial crisis that happened in 2008 and the events that led to it are today known as the ‘greatest quality failure of all time‘. The phrase was first used by Paul Moore, Senior Risk and Compliance Executive at HBOS who was dismissed after having warned the board about the consequences of taking risks. Senior managers never saw the danger coming from the risky decisions they were taking, which led to the largest crisis that ever happened in the financial world.

As noted by the Reserve Bank of Australia, the crisis was caused by the excessive risk-taking in a favorable macroeconomic environment, increased borrowing by banks and investors, and regulation and policy errors.

There were several things that could have been done to improve the processes and quality to prevent the crisis:

• enhancing the quality of financial products and services, because the crisis was fueled in part by the proliferation of complex and risky financial products and services,
• strengthening the risk management processes, because developing better processes for identifying and mitigating risks, including stress testing and scenario analysis, could have helped to prevent the crisis,
• improving the quality and reliability of financial reporting, as investors and regulators were not fully aware of the true financial condition of many financial institutions,
• enhancing the quality and effectiveness of regulatory and policy frameworks,
• ensuring that all personnel are properly trained and competent.

Challenger Space Shuttle explosion, 1986

Close picture

The Space Shutter Challenger was launched on 28^th January 1986. Shortly after taking off, it exploded, destroying the vehicle, and killing all crew members. The explosion was a result of a bad decision-making process and the lack of quality assurance. A part of the vehicle, O-ring seals, was not ready to be used in cold temperatures, yet NASA officials, worried about political and financial consequences of not performing their task, decided to go ahead with their mission, despite the danger it was linked to.

In addition to the faulty O-ring seals, the disaster was also caused by several other factors, including the lack of communication and the lack of understanding within NASA about the risks associated with launching the Challenger in the cold weather.

There were also problems with the design and testing of the SRBs, as well as with the decision-making process within NASA that led to the launch being approved despite the known risks.

There were several things that could have been done to prevent the Challenger Space Shuttle disaster in 1986. Some of the most significant ones include:

• improving the design of the O-ring seals,
• testing the O-rings more thoroughly,
• developing better communication and decision-making processes within NASA,
• implementing better safety procedures and protocols,
• improving the design and testing of the solid rocket boosters.

BP Deepwater Horizon explosion and oil spill, 2010

BP’s Deepwater Horizon rig exploded on 20^th April 2010 which immediately became known as one of the biggest environmental disasters in history. It killed eleven people and caused the discharging of four million oil barrels into the Gulf of Mexico, devastating the entire ecosystem.

As explained by Graham Freeman on his blog:

Here are some specific measures that could have been taken:

• implementing thorough testing and inspection protocols to ensure that all equipment and systems are properly tested and inspected before being put into service,
• adapting to industry standards and best practices for offshore drilling operations, including the American Petroleum Institute (API) and the International Association of Oil & Gas Producers (IOGP),
• better training and education of workers in the offshore drilling industry can help to ensure that they have the necessary knowledge and skills,
• regular safety drills and exercises,
• implementing robust safety systems such as blowout preventers, which are designed to prevent the uncontrolled release of oil and gas, can help to reduce the risk of accidents.

Failure of Boeing Co. 787 Dreamliner project, 2013

The Boeing 787 Dreamliner was introduced in 2011 and was supposed to change the entire air travel industry as a less expensive, more customer friendly and simply better option for air passengers. Yet in 2013 the entire Boeing 787 fleet was grounded, following several incidents of fire aboard some of the planes.

The causes of those incidents were complex and included problems with manufacturing, supply chain and organisational factors. All of them happened because of poor quality assurance – numerous documents and reports confirm that Boeing did not adopt an effective Reliability Program Plan, where best practice tasks would be implemented to produce a reliable product. 

Some of the things that could have been done to prevent the failure of the project include:

• improving the design of the aircraft – The Boeing 787 Dreamliner had a number of innovative design features that also introduced new risks and challenges that were not adequately addressed,
• ensuring that manufacturing and assembly processes are robust and reliable,
• improving communication and decision-making processes within the company,
• developing better risk management processes.

Therac-25 killing patients

One of the scariest examples of a failure in quality assurance procedures was the Therac-25 case. This radiation therapy machine, which was produced in 1982, caused death of several patients due to massive overdoses of radiation it gave. As noted by Wikipedia, because of concurrent programming errors, the machine sometimes gave patients radiation doses that were hundreds of times greater than normal. The series of accidents showed the importance of software control standards, ethics, and due diligence to resolve software bugs.

It is important to note, that bugs in codes were not the root cause of this failure. The entire system design was flawed. The analysis of the failures aiming to understand how system will behave in different cases (the fault tree analysis) for the entire system was not conducted. Unit testing, which might catch bugs in codes on early stages, was not performed by the team.

The full analysis of the case can be found in the research paper: Medical Devices: Therac-25 by the Professor of Aeronautics and Astronautics Nancy Leveson from the University of Washington.

There are also some lessons learnt described, which may apply to different projects:

• the overconfidence in software,
• confusing reliability with safety,
• the lack of defensive design,
• failure to eliminate root causes,
• complacency and blind trust,
• unrealistic risk assessments,
• inadequate investigation or follow-up on accident reports,
• inadequate software engineering practices, especially in quality standards,
• software reuse does not guarantee safety,
• safe versus friendly user interface – sometimes making machines as easy to use as possible may conflict with safety.

Patriot Missile software accident

The accident occurred during the Gulf War in 1991, when the United States and its allies were engaged in a military conflict with Iraq. On February 25, 1991, a Patriot missile defense system operating at Dhahran, Saudi Arabia failed to track and intercept an incoming Scud missile, which subsequently hit the Army barracks, killing 28 Americans.

The failure of the Patriot missile defense system was a significant event that highlighted the challenges and limitations of the missile defense systems. It was later determined that the Patriot system experienced a software issue that caused it to ‘lock on’ to a single point on the ground rather than accurately tracking the movement of incoming missiles. This error was not detected during testing and was not identified until the system was deployed in a real-world scenario.

In response to the incident, the manufacturer of the Patriot missile defense system, Raytheon, implemented a number of changes to the system’s software and hardware in order to improve its performance and reliability.

The company also implemented more robust testing and quality assurance processes to help prevent similar problems from occurring in the future. The Patriot missile defense system has continued to be used by the United States and other countries around the world, but it remains subject to ongoing testing and improvement in order to ensure its effectiveness and reliability.

The detailed analysis of this incident can be found in the government report Patriot Missile Defense: Software Problem Led to System Failure at Dhahran, Saudi Arabia from 1992.

Conclusion

Quality failures can have serious and far-reaching consequences, as demonstrated by the numerous incidents described in this article. From the Challenger Space Shuttle explosion and the BP Deepwater Horizon oil spill to the global financial crisis and the Patriot missile defense system incident, these events highlight the importance of maintaining high standards of quality in all aspects of operations.

To prevent future incidents, it is essential to implement robust processes and procedures for ensuring the reliability and performance of products and services, as well as to continuously monitor and assess the quality. By taking these steps, organisations can prevent costly and potentially devastating incidents caused by quality failures.

Deliver high quality software solutions

Work with us and turn your idea into a great software solution for your company’s needs.

Contact us

A story of an efficiency

First Case

In the case of this project, we were approached by a client who already had an existing system in place. So the first step we took was getting to know that particular customer’s needs. That way, we were able to define the project’s objective: achieving the desired result within a given timeframe using available resources.

The client’s existing system was associated with financial solutions. It had been created based on older technologies and developed for many years by various teams. Additionally, it had been passed on numerous times from one contractor to another, who all approached Quality Assurance very differently. As a result, the code’s quality was very low, and it lacked suitable project templates, among other things. Consequently, the quality of the whole system suffered, rendering it an obstacle to the effective implementation of business objectives.

For example, the processing of some data points was measured in days. That also caused serious problems with using the software, which – in turn – significantly damaged the client’s reputation in the eyes of their end-users. Hence, improving the overall system efficiency became our highest priority. Its successful implementation would allow us to achieve the project’s main business objective. The timeframe for achieving the desired result – improving the efficiency – was also a part of that objective.

Additionally, to achieve the desired result within the given timeframe, we needed to clarify available resources in the context of the team and the tools. The team comprised a few experienced developers and QA representatives. Their task was to ensure that the quality of the process and the delivered solution would be maintained at all times. The tools were chosen according to the technology the whole system was based on. It was related to the project itself. Apart from the main application’s code, verifying the tests’ code was also one of the tasks. The tests were supposed to detect errors in the early stages of the implementation of new functionalities.

Unfortunately, in the case of this particular project, they were practically non-existent. All verifications and validations up until that point had been carried out manually. That made everything very costly and time-consuming, and, keeping in mind the strict upcoming deadlines, we couldn’t afford to lose more time. Therefore, our QA team was tasked with the following:

• developing a test approach for legacy type systems,
• defining system quality indicators to monitor the state of the system and its quality,
• designing and implementing tests to verify achieving the desired project result – improving the efficiency,
• devising a CI/CD process with emphasis on the tests,
• designing and implementing automatic regression tests to assure stakeholders of the lack of regression in all areas, i.e. in terms of financial calculations.

Action plan

We spent the first few weeks drafting an action plan. Manual testing was not financially viable, and establishing a full framework for automatic testing was impeded by insufficient resources. Therefore, we focused on the Rich Based Testing methodology, recommended when time, budget, and resources are limited. Thus, we defined the business, technical and project risks. Based on all that, we were able to determine the action plan.

Next, we designed a fairly unique set of tests which convinced us that the modifications in the application itself wouldn’t impact financial calculations. While creating the solution, we also carefully considered the ROI associated with the automation of all the remaining process elements.

We were able to reduce the time needed to process some operations from a few hours to a few minutes, while also being certain that no regression was introduced to the code itself. That was possible due to the properly designed and implemented test harness. Ultimately, we reached the objective within the designated timeframe. As a result, the improved product was delivered to the clients and end-users on time.

The dedicated QA specialists team ensured that:

• risks and dangers were well-defined, and then mitigated or eliminated,
• the process was constantly monitored, and potential dangers were detected in the early stages,
• efficiency was improved without the fear of regression,
• tests to reduce the number of errors in future improvements were designed and implemented.
• Integration is key

Second Case

In the second case study, I joined an already existing team working on a project in the transport sector. Its objective was transport improvement by introducing an innovative system to manage, analyse, and distribute permits, access points, and payments.

The concept was based on micro-services and the integration with external suppliers. The project was already in the advanced stages of development with many requirements set in stone. And some micro-services were ready to be deployed. A dedicated QA team was not involved in the process from the start, and that situation posed a real challenge. Because of it, the existing process didn’t include the following features:

• an approach to testing at different project stages;
• testing processes for each release;
• best practices for test automations;
• defining risks and taking actions to mitigate them.

Testing strategy

After getting acquainted with the project’s objective, documentation, and existing code, we proceeded to establish a testing strategy and general approach to testing.

The first essential deadline (and goal at the same time) was delivering the first version of the software for the meeting with the stakeholders. It was important to get it right – the success of that event would determine the funding for the whole project. One of the decisions we made was to invest heavily in automatic testing. The tests’ role was to confirm our assumptions that every module met the client’s needs and that the integration worked properly. At the same time, we decided to expand the QA team to run manual tests on the already existing components.

While testing, it turned out that some services returned integration defects, which negatively impacted the general payment calculations. This early error detection during the testing phase spared the client the cost of fixing those issues at later stages. It would have surely been much more expensive if the software had been already delivered to the clients and users. However, that cost could have also been reduced further had the QA team participated in the initial project phases when the testing requirements and documentation were being established.

Consequently, we were able to pass to the next implementation phases and develop the application further.

Having established a dedicated QA team, we were able to succeed in the following areas:

• devising a strategy and testing process for the project – from testing new functionalities and automation to regression tests and application quality reports;
• designing an early error detection process that prevented potentially lower confidence in the product and higher costs of fixes as opposed to spotting errors in the production phase;
• implementing QA in the requirement definition process to detect potential errors much quicker;
• devising an automatic testing solution to cover both individual modules and the integration between micro-services and external systems;
• introducing testing in the CI/CD process phase, thanks to which we were provided with quick feedback on the current application status;
• delivering high-quality software in a timely manner, both as a presentation for the stakeholder and later for the end customers.

Conclusion

The above case studies demonstrate that any team composed of QA experts can significantly improve the quality of the delivered software solution. While it is a fairly broad statement, here are some tangible advantages of having QA onboard your next project:

• improved satisfaction from the application’s end-users, time and costs savings in terms of potential bug fixes if they are detected early enough,
• protecting the reputation of the client who is co-operating with the team on the final product,
• avoiding missed deadlines and costs generated by potential delays,
• eliminating issues related to the system’s efficiency and security,
• identifying risks and taking appropriate actions to either mitigate or completely eliminate them.

To summarise, every software development team should always include experts who will ensure the highest possible quality of the final product.

Deliver high quality software solutions

Work with us and turn your idea into a great software solution for your company’s needs.

Let’s join forces

Close picture

Quality Assurance Versus Quality Control

The terms “quality assurance” and “quality control” are often used interchangeably when we speak about the processes used to maintain the quality of a product. However, there are some key differences as highlighted below:

• Quality assurance refers to process management activities which are aimed at reducing defects and errors for the end customer. QA involves looking at how the processes are performed and making sure that the quality requirements are being fulfilled. It is a proactive, process-based approach which aims to manage the quality of a product before and during the production process.
  
  
• Quality control (QC) is concerned with the end product and seeks to ensure that it is not defective or damaged before reaching the customer. The defects that quality control are interested in are likely derived from faulty manufacturing processes rather than inherently ineffective processes (as you would typically see with quality assurance). QC is a reactive process which is employed after the product has been created to verify its quality.

Close picture

Quality Assurance in Software Development

As well as being used when creating physical products, QA also plays a key role in software development and is applied by software companies in exactly the same ways as manufacturing organizations would. The QA team in a software company works alongside a Solution Architect to analyze the requirements of the system, define the parameters which are set to determine whether that product is meeting all the necessary needs, and to implement processes to make sure that not only is the end product suitable, it is of the highest quality possible.

Once the software product has been created, and is in adherence to all the QA processes and checks, the QA team will then hand over to the QC team (quality control) who devise and implement tests to make sure that the software is working without any bugs. They will test the product extensively before finally either releasing the product or handing it over to a client.

The Importance of Quality Assurance in Software Development

Quality assurance is hugely important when it comes to software development. Some of the most important features are:

QA helps to reduce cost and saves time

• Having QA systems in place during your production life-cycle allows you to anticipate and prevent issues such as bugs, defects, and errors from ever occurring. By identifying problems in the initial stages, you can put fixes in place early, helping you save time and money that would otherwise have been spent on fixing those issues during the later stages of the product lifecycle. In addition, if any bugs or errors do occur despite the QA processes implemented, they would likely be relatively minor and easy to fix.

QA helps to maintain the quality of your product

• Having QA processes in place allows you to maintain the reliability and stability of your product. As a proactive process, QA aims to anticipate issues before they occur so that potential problems are never allowed into the software build. Fewer problems means better quality, and the QA team works alongside the quality control team in order to carry out tests at different stages of the life cycle which focus on the usability, functionality, security, and usability of the product. The final result is that the finished product has minimal defects and, to the best of the company’s ability, works perfectly as intended.

QA ensures that your product is secure

• Security is always a huge priority when it comes to software, so it makes sense that a company’s QA processes should have a strong focus in this area. Software programs may work perfectly well, but this doesn’t guarantee that they are secure. Any weaknesses or defects in a software’s security could seriously compromise a company’s data, which is evidently a massive problem. The QA team installs safeguarding processes when the product is being developed to make sure that it is built as securely as possible before finally being tested by the QC team after product completion.

QA improves accessibility and usability

• With a QA engineer on board you can improve an existing application or the prototype, investigate the possible technical implementations and find optimal solutions to best satisfy your customers’ needs.
  

How? You can:

• Evaluate the project’s usability through different methods, like cognitive walkthrough or heuristic evaluation.
  
  
• Analyse the user’s flow related to keyboard navigation, tabulation and interaction together with potential blockers.
  
  
• Verify aspects like correct colour contrasts, colour use or support for keyboard navigation and screen readers.
  

A QA engineer can share with you recommendations of what needs to be improved in service or product so it can be used by people with visual, hearing, motor and cognitive disabilities. And when working closely with the development team all the information on priorities, recommendations, references to specific points of guidelines and tips are shared with web-designers, editors or developers. And you get the best quality product, that will have higher market share, will be accessible for society and will enhance the brand image.

QA runs performance testing

• When engaging a QA in your project you get even more – performance testing. It is a testing practice that helps to determine the way system performs in terms of responsiveness and stability under a particular workload. What’s more? With performance testing you can measure, validate or verify other quality attributes of the system e.g. reliability, scalability and resource usage. Quality is not only failure-free software, but also meeting the user’s needs in a timely manner.

QA increases customer satisfaction

• Regardless of whether you are creating software for a client or directly for the end customer yourself, customer satisfaction is key. Clients and customers alike require a product to fulfill their needs, as well as having a full set of features that function properly and without issue. It is the role of the QA team to put processes in place to ensure that their software gives customers exactly what they expect. If they do this well, customer satisfaction will be high and it will go down as a job well done.

QA protects your company’s reputation

• Providing a good product which results in satisfied customers reflects directly on the company who created it. By releasing high-quality software which boasts excellent features, uncompromising security, and flawless functions helps to build a company’s positive reputation in the marketplace. QA is a huge contributor to a company’s reputation for obvious reasons and it ensures that your star product is a fitting ambassador for your company’s international reputation.

The Risks of Neglecting Quality Assurance Activities

• Companies may be tempted to neglect various aspects involved in QA activities to save money or decrease production times. However, this is a very risky strategy that can seriously backfire. There are lots of technical and commercial dangers which could potentially be highly damaging to your company if QA is not taken seriously enough.

Close picture

Conclusion

Quality assurance ensures that software is efficient, contains minimal defects, and meets customers’ needs closely. It is extremely important to incorporate QA into the overall software development lifecycle and to weave it into every stage of the process.

QA shouldn’t be deferred to the QC testing phase because by this time, it will be too late and any problems here that could have been mitigated earlier on with sound QA processes will probably be very costly indeed in terms of both time and money.

Without QA, software development can be very unreliable, resulting in products being unviable and having to be taken back to the drawing stage if its flaws are too widespread. Therefore, having sound, effective quality assurance processes in place and a strong “QA driven philosophy” is a huge asset to any software company.

Deliver high quality software solutions

Work with us and turn your idea into a great software solution for your company’s needs.

Let’s work together