How Small Businesses Should Define and Measure Agile Success

Ask any project manager and they’ll agree: Pleasing all stakeholders is near impossible.

Each stakeholder wants to track a different metric, which means that small businesses don’t always focus on the core success metrics that can define actual business growth.

Gartner says that tracking just the traditional productivity metrics (e.g. size/quantity of project deliverables ) hinders the ability of agile teams to successfully deliver projects (content available to clients). This, in turn, affects the team’s motivation levels.

Small and midsize businesses (SMBs) that practice agile management but fail to strategically track and measure project success will stifle employee motivation and lose customers.

SMBs need to define and understand project success with the right agile mindset, in the context of team happiness, customer value and agile fluency. This will help SMBs espouse a holistic approach to agile execution and gain a clear understanding of project performance.

This report looks at the example of software development firms solely because these companies have widely adopted agile. Our goal is to help SMBs define and understand agile metrics to define the success of their projects.

Here’s what we’ll cover:

Agile Triangle: The Three Sides of Success

1. Gauge Team Happiness, Don’t Compare Employees

2. Measure Customer Value, Not the Quantity of Work

3. Track Agile Fluency, Not Just the Speed of Delivery

Conclusion and Next Steps

Agile Triangle: The Three Sides of Success

Businesses typically define project success by how well they managed the scope, budget and timeline. While this approach works for businesses in the pre-agile stage, it’s risky to define agile project management success with just short-term profit metrics. That’s because it shifts the team’s focus from delivering value to savings on costs.

Instead, businesses should focus on the three critical areas that Steve Dennings (author of Age of Agile and independent management consultant) says define agile project management:

• The small team

• The customer

• The collaboration between larger teams in the company

Combining Denning’s insights with the concept of agile fluency (more on this later), as defined by Diana Larsen (agile/XP consultant and coach) and James Shore (agile coach and agility consultant), what we get is the agile success triangle.

The Software Advice agile success triangle

Let’s look at what each side of this triangle represents and why your small business should care.

1.Gauge Team Happiness, Don’t Compare Employees

According to a Software Advice study, the most common operation that agile teams automate with project management software is reporting. Fifty-one percent of small businesses say that they most often use the reports to “review individual/user productivity and monitor team performance.”

Why gauge team happiness? Measuring employee performance is vital. But in agile frameworks, tracking just the traditional productivity metrics, such as employees’ effective work hours or tasks completed in a day, paints a misleading picture of employee performance.

For example, one inaccurate way to measure employee performance success is measuring a software developer solely by the number of lines they code every day. They might also have good coaching skills, which are vital for a team’s success, but don’t contribute any actual hours to the team or project.

Software development firms should instead focus on metrics that measure the team’s collective efforts at driving product improvement and establishing long-term customer value.

 HOW SMBS CAN DO THIS:  A popular study on employee productivity is Google’s Aristotle project. The survey found out that the best teams aren’t ones with the best individuals, but those that possess the following five team dynamics:

(Source)

These five team dynamics are the foundation of healthy agile teams. Gartner’s report also notes that small businesses should conduct periodic “team health checks” to gauge their happiness. We recommend using the following two diagnostic metrics to track the five team dynamics listed above:

• Gauge cross-team cooperation: Use surveys tools that allow members to rate their team on dependability and psychological safety.

• Identify process issues: Align your organizational and employee goals with an objective and key results (OKR) program. OKR software can help you ensure transparency of information regarding the team’s structure, roles and objectives

2. Measure Customer Value, Not the Quantity of Work

Keeping customers happy is critical since you’re judged on the value your product offers, not the size of your brand. However, software development firms still measure project success on traditional metrics such as the number of product features built in a given sprint, not the value they offer customers.

 WHY MEASURE CUSTOMER VALUE?  Delivering customer value directly results in business growth. For instance, back in 2015, Spotify adopted user personification in its “Discover Weekly” feature, using data analytics to make personalized playlists. In just six months, users had streamed the playlists 1.7 billion times!

Realizing how user personification can drive customer value and boost revenue, competitors such as SoundCloud have now launched similar services.

Adopting strategies such as user personification makes customers feel rewarded by using your tools, which in turn brings you more customers.

Gartner stresses that businesses should constantly employ strategies to improve customer value and measure it by asking questions such as:

• Is our organization delivering something of value to customers (or business users) and stakeholders?

• Are the customers satisfied/happy, and are they using your product?

 HOW SMBS CAN DO THIS:  Tracking customer value isn’t all about how you’re delighting customers. In fact, businesses should stay away from delighting customers if they can’t do it “repeatedly, reliably, scalably and profitably,” say customer service experts Thomas A. Stewart and Patricia O’Connell.

Tracking value metrics means ensuring that your business has a truly customer-focused mindset. Here are some key metrics that allow you to measure customer value:

• • Evaluate customer satisfaction/adoption: Use surveys tools, such as the net promoter score and customer feedback surveys, to understand customer satisfaction levels.

• • Measure revenue/consumption: Calculate the number of active users/end customers to identify when you gain or lose users.

• Track customer issues**:** Identify the “defect rate” in your product by analyzing bug trends and tickets raised by customers.

In addition to tracking these metrics, ensure that your business is doing all it can to offer customer value. Here are some questions you can ask your team to find out whether they follow a customer-driven agile development process:

• • Is the team building product features that target the needs of the primary customers?

• • Are they involved in making small, but regular, product innovations to delight core customers?

• • Is the team constantly learning from customer feedback and product experiments?

• • Are they following a product release planning process by evaluating the features requested by core customers?

• Is the team designing products that can be easily customized and personalized by customers?

3. Track Agile Fluency, Not Just the Speed of Delivery

Small businesses adopt agile project management to get more work done with fewer people. Agile frameworks, such as scrum and kanban, help businesses improve employee productivity. In our survey on top technology trends, improving employee productivity is a top business goal for SMBs in 2018-2019.

 WHY TRACK AGILE FLUENCY?  Often, small businesses practicing agile software development mistakenly equate employee productivity with the rate of software delivery. As a result, they try to achieve faster time to market, releasing features that are incomplete and unfinished.

This happens when you prioritize selective success metrics, such as team velocity (how fast teams complete sprints), to track performance. Businesses misunderstand that when measuring velocity, they’re actually measuring how a team can deliver predictably, not how quickly they deliver.

It’s unfair to use velocity as the only metric to judge the performance of different teams. Instead, small businesses should track agile fluency to better understand the nature of their teams and establish fair success metrics to judge their value to the business.

 HOW SMBS CAN DO THIS:  In their blog, Larsen and Shore define their agile fluency model by categorizing teams that have adopted an agile culture into four “zones,” based on the value they deliver to the business.

This is not an “agile maturity” model but rather, a way to understand the current responsibilities of your agile teams and create their core success metrics. Here’s a brief summary of the agile fluency model:

Most small business agile teams will fall under the first three zones, and almost none in the strengthening zone. That’s because this zone is more of a prediction on where agile project management is headed.

Here are some quick ways to track the core success metrics, depending on which zone your team currently falls into:

• Track basic progress reporting for the focusing team: Project managers should look at basic reports, such as team capacity and sprint burn-up charts, to track progress. They should also make timely interventions to prevent the team from working on projects that aren’t a business priority.

• Measure the delivering team’s product quality and market cadence: Velocity and burndown charts are good for tracking your team’s capacity for delivering products on time. Also, use bug tracking software to review bug trends reports that show the number of defects in the product.

• Identify the optimizing team’s market expertise: Product ROI is a key metric to find out whether the optimizing team is proficient in building products that drive valuable business outcomes. Other metrics include the team’s ability to engage customers and identify market needs, based on previous customer reviews of the product. These reviews are collected from sources such as customer emails, customer feedback surveys and even reviews on product review websites.

Conclusion and Next Steps

No two businesses implementing agile project management practices can expect similar results. After all, they’ll have employees with varying skill sets, organizational structure and stakeholder influence on projects. They could also have different agile frameworks than scrum, kanban, extreme programming and DevOps—or even hybrid models.

However, these small businesses should not neglect the three core metrics that define agile project management success.

Whether your business is transitioning to an agile culture or has fully adopted the practice, it’s crucial to measure the happiness of your team, identify how they deliver customer value and understand their core capability to establish a fair method to define and measure agile success.

Small businesses should also remember that constant learning is the key to successful agile project management. Gartner’s “Become an Agile Superhero: 8 Attributes for Success” report (content available to clients) notes that a top attribute of successful agile teams is to develop a deep understanding of agile concepts and frameworks.

To better understand agile project management concepts, practices and trends, start by reading the reports recommended, created with a special focus on small businesses:

• What Is Agile Decision-Making in Project Management?

• The Best Scrum Project Management Tools for Your Agile Team

• Want a Truly Agile Software Development Team? Focus on These 3 Attributes

• How to Prepare Your SMB For a Transition to Agile Processes

Information on Gartner’s survey on technology trends:

We conducted this survey in April and May 2017 among 699 U.S.-based SMBs with more than 10 employees and annual revenue of less than $100 million. The survey excluded nonprofit organizations. The qualified respondents are decision-makers or have significant influence on the decisions related to purchasing technologies for their organization.