Skip to main content
Page Tittle
Ensuring Quality from Agile Software Development

The term ‘Agile Software Development’ is easily misapprehended in nomenclature-obsessed days. Most of the development firms call themselves Agile as soon as they step into iterative development and deployments. Before we discuss any further, I believe it is important we take a step back and refer to the Manifesto for Agile Software Development and the values proclaimed by the 17 signatories.


As much as the aspects listed on right hand side are still important, those on left hand side should be valued more if and when there is a scenario where you have to choose between both. Agility is not only releasing frequently, but about the idea of iterative development, where requirements as well as solutions evolve through collaboration among self-organizing and cross-functional teams.

When we say cross-functional software development teams, the culture comes into picture.

Every other thought leader is talking about culture, how important it is to have collaborative culture infused and demolish silos since they would inhibit value delivery. Even the software engineering companies are seemed to be adopting no-silo work culture but, as soon as a vendor or an external partner comes into the picture, somehow the transaction of work happens in waterfall style. Technically, in this environment, your organization is the customer and the very third value of agile software development manifesto promotes customer collaboration.

A discussion about ensuring ‘Quality’ in Agile Software Development cannot overlook the fact that a good deal of software production is outsourced in various levels for various reasons. Vendors or external development partners are generally hired for large initiatives that take a long time and incur high costs if done in-house. So, practicing centralized strategic decisions and decentralised frequent & time-critical decisions should not be a challenge (as endorsed in SAFe principles). So, in this piece, we have brought together the most successful Agile Software Development practices that can strengthen your Quality score that can directly influence your product/business value.

Re-define Vendor Work relationships

Try Distributed Agile in action

Work cascading from one step to another (or one sprint to another) should not mean the vendor throwing the finished project over wall and in-house team rolling the sleeves up to integrate it. The onsite as well as offshore teams must work closely, share information, ideas, solutions, fail fast and that is when you create a true agile development team that can produce a minimum viable product at the end of every single sprint. The one-team mindset builds a sense of ownership in the vendor side teams and a sense of control and confidence to actual product owners in the buyer company.

Empowering the vendor resources improves their judgement which works wonders in terms of speed. Think out of the box, create bigger team building activities, treat them as one, and thanks to the evolution of collaboration tools, creating a bond among teams who are working in different time zones is not a problem anymore.


Propose a T&M Contract

If you are following fixed-price and date model, you are likely to fail being Agile

A fixed price contract to a vendor to build certain deliverable within X months. Many of us has been dealing with the same or similar models and nearly all of us are victims of long hours, missed deadlines, delivering over-budget, or the horror - a complete re-do. Rather propose a pay by the hour, pay by the story point, or pay by the sprint approach (at least when you just started collaborating with the vendor).

In addition to having financial and quality control over the project,

  • The buyer company will have command over the design since the contract does not stop them from changing requirements.
  • Since there is no commitment of the whole price upfront, there is an opportunity to pivot or streamline the cost funnel as needed.
  • Iterative Development gets easier. Since the agreement says the vendor should deliver functioning software every sprint, the progress towards the goal is measurable. Now instead of just a piece of code, the result of each contract is actual value.
  • Since commitment is small, it is easier to respond to the changing market conditions.

Define the definition of 'Done'

It is the only way to reduce Technical Debt

A scrum master's definition of done should have a clear checklist when the code built meets all those conditions, it is accepted by a user, customer, team, and consuming system without any hiccups. As they say, Predictability is key in Agile development. Code passing functional and usability testing will not make the cut. The user stories, features and epics may have separate and distinct definitions of done. Ensure your DOD checklist includes these conditions to ensure quality consistency.

  • Code was peer reviewed
  • Automated unit tests are written
  • A tester has confirmed the functionality
  • Code is well written and consistent
  • Functionality fits the product vision
  • It has been reviewed with the customer

If ignored, a code that is marked 'Done' without obeying the checklist may lead to bugs at production stager along with leaving a trail of tech debt. If you are outsourcing external teams, ensure that they follow the same DOD checklist. All your quality expectations, cross-functional skills, and project related communications should be extended to the vendor teams as well. As we mentioned earlier, the output from your vendor teams is only as good as your collaboration with them.

Add Agility to Testing

Integrate Exploratory Testing with Conventional Testing Techniques

Understanding the personas of end-users and exploring the application to identify potential edge cases can add value to the conventional scripted tests. Even though the tools, technologies, goals, and strategies are different, there is one trait that we find common in all the high achieving agile teams in software development. That is steady focus on Continuous Improvement.

Exploratory testing could be a life saver when it comes to improving quality since the practice shares a lot of characteristics with Agile Software Development Methodology itself. Test Automation is great but it is highly scripted so there is a good chance that you might miss some unpredicted bugs and errors even after running your auto-tests. Motivating testers to do investigate the application manually to identify potential edge cases keep the software quality high. It is all about tester asking questions that scripted testing did not (can't).

Contrary to a popular belief that exploratory testing can slow down the software development process, it accelerates the test cycle speed. Because of less documentation and faster feedback. Working Software over Comprehensive Documentation, remember!

Metrics, of course!

Identifying right Agile Software Development metrics can be daunting. There are simply too many to choose from. But we believe, to delivery good quality software, it is important to keep a track of functional as well as non-functional metrics. A few obvious ones are:


  • User Story Acceptance = Number of user stories accepted by the customer/number of stories X 100
  • Review Effectiveness = (Number of defects found in Review)/Total number of defects found before Delivery (both Reviews and Testing) X 100
  • Defect Density = Defect count/size of the release
  • Test Coverage = % of code covered in automated testing


  • Performance = software responsiveness and stability under a particular workload
  • Functionality = how well the software meets the user's needs
  • Usability = degree of ease of use and learnability, even for people with disabilities
  • Reliability = the software design and functionality can be depended on to be accurate
  • Compatibility = the ability of a piece of software or system to work with another
  • Maintainability = ease with which code can be amended or maintained
  • Security = software is secure from malicious attacks and hackers

User Involvement

Making the end user part of Continuous Feedback loop

Since we all agree to test-driven development to achieve maximum quality, do not consider the end-user as a persona that comes into the picture at the end of software development process. The target audience is one of the most important stakeholder in the production process. Include your users in the form of acceptance testing and make it metric for release validation.

At Qentelli, we co-create an Agile Software Development methodology based on client’s requirements and the nature of the application. Broadly, a concise version of it would look something like this. Though it is not obligatory for everyone to follow it, this approach benefitted us and our clients to absorb user feedback from earlier stages and exploit it for the betterment of the application.


Draw how to make Toast

As delivering value to the customer/user is the key objective of agile projects, it is important for the main development company and the outsourced vendors to stay on same page. Choosing a reliable external team that is a challenge in itself. Agile Development is all about finding solutions through collaboration between self-organizing cross-functional teams. There is an exercise that can help you assess the problem-solving skills and create a collaborative bond between your (internal and vendor) teams. It is a simple workshop designed by Tom Wujec from Autodesk to assess problem solving skills. It is a 3-part exercise.

Step 1: Give a paper each to everyone and ask them to draw how to make a toast.

Step 2: Divide people into groups, give each group a bigger sheet of paper and ask them to collectively draw how to make a toast.

Step 3: Hide the names and classify the drawings only by individual and group outputs. Most commonly the group outputs turn out to be more strategic, tactical, and have more nodes and details. Compare the drawings and discuss the contrast between the efforts.

Feel free to read more about this exercise here.

Successful Agile Software Development teams must have as much information from as many resources as possible to work efficiently and develop effectively. The team’s success is dependent on their ability to coordinate with each other - developers, testers, business analysts, and partners.

To summarize our mantra for top-notch quality:

  • Track the defects, not incidents and prioritize them immediately.
  • Meeting non-functional requirements is as important as meeting the functional ones.
  • The vendor teams should be as agile as your internal teams.
  • The sooner you bring in the User feedback into loop, the better Quality you achieve.
  • During production, measure Quality in terms of defect counts and the mean time to resolve them.
  • Collaboration is the key, period.

Do you have any more questions regarding Quality Assurance, Quality Engineering Practices, Automation or Agile Software Development in general? We are happy to discuss. Drop us an email: [email protected]