Metrics Definition Example Essays
The Importance of Metrics
- Length: 1524 words (4.4 double-spaced pages)
- Rating: Excellent
The Importance of Metrics
Establishing metrics is crucial to any organization, especially in technology related company projects. Metrics permit organizations to measure its performance against industry sectors to determine how well the company is doing. Furthermore, metrics allow organizations to evaluate and improve the effectiveness and efficiency of its processes. Metrics are designated in different categories. The categories identified in this document include output, in-process, and people. (Duris 2003) The organization must first determine exactly what the company is trying to accomplish or determine. Metrics are then identified based on what is relative to the subject matter. Finally, metrics are verified when tracking progress against previous records or a company given standards or goals.
Importance of Metrics
Establishing metrics is crucial to any organization, especially in technology related company projects. Metrics can be defined as a system of parameters or ways of quantitative and periodic assess of a process that is to be measured, along with procedures to carry out such measurement and the procedures for the interpretation of the assessment in the light of previous or comparable assessments. The results of the metrics can be used to record trends, efficiency, capital, and etcetera. Metrics permit organizations to measure its performance against industry sectors to determine how well the company is doing. Metrics allow organizations to optimize its productivity.
Categories of Metrics
Categories of metrics differ according to the type of company and its goals. An article written by Paul Germeraad (2003), describes in detail different segments of metrics that are used in measuring Research and Development processes which is a critical technical competency. Each category of metric is calculated or measured differently. The categories identified in the article Measuring R&D 2003 include output, in-process, and people.
Output metrics can come in tangible or intangible form.
How to Cite this Page
| Essay on Performance Metrics: Dell Computer Corporation - Dell Computer Corporation asserts a corporate responsibility philosophy geared towards “enabling human potential” through technology and making access to green technology easier for everyone, including the underprivileged (“Corporate responsibility,” 2012, para. 1-2). In order for this philosophy to result in meaningful practices and actions, it is necessary for Dell to utilize performance measures. As Epstein (2008) points out, “measurement is critically important because it links performance to the principles of sustainability and facilitates continuous improvement” (p.... [tags: Business Analysis ]|
:: 8 Works Cited
| Applications of a Connectionist Model of Poetic Meter to Problems in Generative Metrics - Applications of a Connectionist Model of Poetic Meter to Problems in Generative Metrics Meter is one of the most distinctive formal features of English verse. Yet theoretical approaches to metrical analysis have proved problematical for a number of reasons. Traditional metrics, based upon scansion systems derived from Latin forms, is strong and flexible in its ability to describe individual units of a line, but fails to describe well the dynamics of the line as a whole and the lexical and syntactic structures which underlie that line.... [tags: Poem Poetry Poet Meter Metrics Essays]|
:: 9 Works Cited
|The Return on Marketing Investments of Nestle Company Essay - ... However, marketing metrics focus in return of investment of campaign in contrast of their cost to the amount of business that they return in terms of financial information. The other metric focuses on how to create the brand awareness through campaign, how many new customers get attracted and how many old customers are maintain (WiseGeek 2014). A marketing and financial approach of marketing metrics is to focus on return of investment of marketing activity. Return on investment is analyzed by using the total revenue produced by a specific activity and after that minus its total cost of performance.... [tags: marketing, metrics, sales]||1044 words|
|Software Quality Assurance Essay - Software Quality Assurance ABSTRACT High complexity, invisibility of the product and opportunity to detect the defects provided the uniqueness which will meet the challenges for the development and operation of quality assurance for software. This paper provides a brief introduction to software quality assurance and discuses about the software quality metrics and their limitations. Keywords: SQA (software Quality Assurance), software quality metrics, process metrics, product metrics INTRODUCTION Definition: Software Quality Assurance is a set of activities designed to evaluate the process by which the products are developed or manufactured.... [tags: Software Quality Metrics]||1161 words|
| Essay on The Importance of Performance Appraisals - Introduction: Organization is a group of people brought to gather to achieve specific goals. Goals can be achieved if team member are performing well. Performance is the results of activities given to the employees in an organization to be achieved within specific period of time. Evaluating the current performance of employees against past performances and organizational standards is known as Performance Appraisal (Dessler, 2005). Furthermore performance appraisal helps the company know how individual employees are performing and how to improve their performance thus improving the performance of the company (Grubb, 2007).... [tags: effectiveness of management, staff performance]|
:: 1 Works Cited
| Importance of Performance Metric for IT Governance Essay - Importance of Performance Metric for IT Governance: The organization order may deliver good governance, which is capable to add true value to the projects. The metric may have performance with a well-defined management with the means of success to determine the areas to focus on the effectiveness improvement. The carrying out of the metric include the improvement of the quality of IT services, reduction of risks in IT, reduce the cost of delivery services in IT. They are two types of performance metrics: 1. Development Metrics 2. Services Metrics Development metrics are used to measure the functioning in the maturation of the IT projects.... [tags: investments, stakeholders, projects]|
:: 6 Works Cited
|The Importance of Planning Essay - The Importance of Planning It’s amazing how long the term “Garbage in, Garbage out” has been around, yet with the advent of the term agile teams (and typically developers) tend to embrace agile as a means of avoiding planning and detailed design. Equally harmful can be the outright of rejection of implementing “agile” methods on the basis of believing that “agile” means “unending, undefinable costs.” When done with best practices in mind, neither of these extremes should hold true. The quality of one’s estimation will naturally correlate with the amount of time spent in thinking about the design, approach, and level of detail discussed up front; and by the amount of change that is allowed o... [tags: garbage, harmful, agile implementation]||1082 words|
| The Importance of Data Visualization Essay examples - We acquire more information through vision than through all of the other senses combined.” “The 20 billion or so neurons of the brain devoted to analyzing visual information provide a pattern-finding mechanism that is a fundamental component in much of our cognitive activity”, according to Colin Ware, in his book Information Visualization. (Ware, 2013) Better decisions are made when backed by information reducing the uncertainty rather than the intuitive approach. It becomes overwhelming when the information comes from various sources in different formats presented in a variety of ways.... [tags: Data, Research, Graphs]|
:: 4 Works Cited
| Importance of Cyber Security Essay - The nation has become dependent on technology, furthermore, cyberspace. It’s encompassed in everything we deliver in our daily lives, our phones, internet, communication, purchases, entertainment, flying airplane, launching missiles, operating nuclear plants, and implicitly, our protection. The more ever-growing technology empower Americans, the more they become prey to cyber threats. The United States Executive Office of the President stated, “The President identified cybersecurity as one of the top priorities of his administration in doing so, directed a 60-day review to assess polices.” (United States Executive Office of the President, 2009, p.2).... [tags: technology, phones, internet]|
:: 11 Works Cited
| Essay Importance of Organizational Behavior - Importance of Organizational Behavior Organizational Behavior The environment of business is always changing. With increases in workload and responsibilities, managers have found that their importance in the work environment has also increased. In order to help managers and supervisors learn more about the complexity of the new workforce many different areas of study have been developed. One area of study that has increased in importance over the years is the study of organizational behavior.... [tags: Work Psychology Essays]|
:: 2 Works Cited
Some of the tangible output metrics include return on investment, capital gain, quality of products or service, quantity of products or service. The measure of the return on investment (ROI) is essential. The ROI is used to assess the effectiveness of an investment such as a technology related project. To calculate ROI, the benefit (return) of an investment is divided by the cost of the investment; the result is expressed as a percentage or a ratio. The ROI is an accepted metric across many companies due to easy calculation and comprehensiveness. Capital gain is also a good metric used by many companies. Using the capital gain as a measure will yield notification of whether or not the project is successful. The value of the project must be worth more than the cost of implementing the project in order to receive positive capital gains. The quality of products is another good output metric. A technology company can compare the difference of its previous products to its new products to determine whether or not the quality is better, the same, or worse. This would identify if the new project or process was a success or a failure. In addition, the quantity of products or services is also used as a metric. Technology companies can compare the amount of products produced in a specific time frame with its new investment to the amount of products produced by the previous process. For example, cell phone companies may find a new process of mass producing cell phones more beneficial than producing each cell phone individually providing that the quality is still the same if not better. The intangible metrics include customer satisfaction and knowledge gained.
The in-process category of metrics is tracked by setting benchmarks or milestones to ensure a project is completed in an effective and efficient manner. The milestones may include a variety of different items dependent upon the type of project that is given. Examples of milestones used in web-development include items such as developing the project plan, developing the prototype, meeting for client review and approval, construction and testing. Other in-process metrics that companies can use for managing its technology-related projects include tracking the number of meetings held, number of management actions, number of people trained, and number of reports and publications relative to the project.
People can also be used as a measurement for managing projects. The metrics can include the measurement of how many training programs were completed, how many trainees had passed the program, and how many trainees obtained a job due to the training. Furthermore, companies can track the progress of employee’s career progress by setting goals and meeting with employees at certain set times to make sure that the employees are meeting their goals or if additional training is necessary.
Who Should Be Involved in Metric Development
An essential point is to appoint certain individuals in the involvement of developing metrics. All individuals who are involved should be well versed in the company’s goals, mission, business acumen, and processes. For all projects, a Project Manager should be designated to lead the process in development. The Project Manager should be appointed by corporate. The Project Manager is also responsible for communicating these metrics and adequately informing the stakeholders of all the planning and how they may be affected by the metrics. In addition, the team members assigned to work on the project should also be involved in developing metrics. The team members should consist of individuals whoare experienced in their line of work and can offer insight of what types of metrics to use and when to measure. Furthermore, it may also be wise to include the client in developing metrics as well. When including the client, it not only keeps the client informed of any deliverables and its due dates, but also forewarns the client of what they may expect. Essentially, the individuals whoshould be involved in developing the metrics are those that are involved in the project itself (client and vendor) and those responsible for the work to be measured.
Categories of Metrics Useful For Managing Technology-Related Projects
First, the company must identify what type of metrics it would use. Once the types of metrics are determined, only then can the organization produce the data needed to be input into the metrics. Paul Germeraad (2003) gives plenty of examples of how metrics should be identified and verified. One example involves determining when the corporation should receive its money’s worth from its R&D dollars and what size organization is required so that only value-adding functions are left. From these efforts, it was found that the best overall metric for the outcome produced by incremental R&D was to measure divisional financial performance tracked monthly on a control chart (Germeraad, 2003). Furthermore, divisional products are outlined on a Value Chart, which tracks the relative costs of products and services manufactured by differing technologies as opposed to relative performance such as the level of sever, design, and quality for outcome measures. Another example provided includes how to determine whether or not R&D provides the best service to the operating divisions in addition to its customers in the form of new services and products. The best metric was to record the change in market share of the different businesses which can be recorded on control charges. The organization must first determine exactly what the company is trying to accomplish or determine. Metrics are then identified based on what is relative to the subject matter (ie: ROI, change in market share, career progress). Finally, metrics are verified when tracking progress against previous records or a company given standards or goals.
How Should Sources of Data Be Identified And Verified
The resulting performance data identified in metric monitoring should be taken seriously by organizations as it provides a foundation of whether or not the company is performing successfully and proficiently. If the metric results yield an outcome that is below the company standards or goals, the organization must find ways to mitigate the situation and determine what it can do to better itself. Each result and each risk is dependent upon the company and its type of project. If the metrics result in answers that are favorable to the company, it proves that the organization is doing well and the project is moving along successfully. The company should use the metrics to continue to progress in its level of superiority. Organizations should be able to use the output from the metrics to help guide each department that is involved and responsible for creating, managing, and obtaining value for the company.
How Resulting Performance Data Should be Disseminated and Used
In conclusion, using metrics for monitoring performance is fundamental for all organizations. Metrics help organizations focus on what is important for the company to survive and compete well in the market. (Duris 2003) In addition, metrics are used to determine how well the company’s processes are and what it can do to maintain high quality. By implementing metrics, companies are able to evaluate how to execute better quality performance activities.
Duris, R. (2003, July). Goal setting 101. Frontline Solutions, 4(7), 42.
Germeraad, P. (2003, November/December). Measuring R&D in 2003. Research Technology Management, 46(6), 47.
Metrics Defined. Retrieved on July 7, 2008 from www.wikipedia.com.
Return on investments defined. Retrieved July 2, 2006 from www.investopedia.com
What's wrong with how we use metrics?
Organizations looking at metrics from a management by numbers perspective follow a process that looks like this:
Management come up with a goal and work out a measure
Management establish a target over a large period (3-6 months up to a year) for the people doing the work
Management communicate only the target (in terms of the agreed metric)
People doing the work do everything in their power to meet the target number
This process encourages overloading a metric with the following purposes:
Metrics as a target – Numerical metrics make it particularly easy for people to use it as the only means for communicating a goal. It’s often much easier to tell people a scale and a number than explaining a much more complicated goal. The target is frequently an arbitrary number and some organizations even spend excessive amounts of time determining what that number should be.
Metrics as a measure of performance – With an established number in place instead of a well-articulated goal, it is now easy for managers to use that same measure to track how quickly people doing the work move towards the goal. Many organizations link these numbers to individual performance targets.
Metrics as best practice – Using metrics both as a target and a measure of performance results in an unintended side effect – an implication this metric is the best method of working towards the goal. When an independent party measures someone else using a numerical target, it applies more pressure on the person doing the work to simply meet an established number. Since they are only measured on performance to this metric, they do all that they can to achieve that particular metric. It implies no other method is best at achieving the end goal.
Overloading a single metric with multiple purposes causes many problems, particularly when dealing with knowledge work such as software. Metrics are simplifications of much more complex attributes. The cost of simplifying complexity comes at the cost of losing sight of the real end goal, and ends in a suboptimal result.
Let’s look at an example:
A test manger, let’s call her Mary, holds weekly meeting with the development lead, Dan. "Where are we at with our bug counts?" she asked at their most recent one. Dan answered, "We cleared our three priority one bugs, fixed four priority two bugs and cleared out a record twelve priority three bugs. A pretty good week right?"
Looking at the development lead, slightly shaking her head, Mary responded, "Unfortunately our customer reported five priority one bugs, six priority two bugs and fifteen priority three bugs. You’ll need to work harder next week." Exasperated and feeling overwhelmed at missing his target, Dan left the meeting thinking about asking his team to work yet another weekend.
In this very simple story, the chosen metric meets one benefit of making the meeting move very quickly. Both people quickly understand progress after Dan reports his results and when Mary responds. Unfortunately the implied goal of delivering useful software is missed, and Dan leaves the meeting with a solution more likely to cause further software issues and drag in software quality.
The way that Mary states her objective puts pressure on Dan to reduce the number of bugs. It seems like an admirable goal. While reducing the number of bugs is a good goal, it also leads to a very reactive solution. Dan leaves the meeting thinking how much harder to work. The question posed by Mary fails to neglect the broader goal, and she fails to ask the crucial question that guides Dan and his team towards fixing the underlying reason the bug exists. Without resolving this root cause, Dan and his team are destined to fix bugs for life.
Dan is experiencing single loop learning. Single loop learning is the repeated attempt at the same problem, with no variation of method and without ever questioning the goal. If Dan ever hopes to break out of this vicious bug cycle, he needs to do something differently. The inappropriate use of software leads Dan away from the end goal of delivering useful software and improving overall software quality. Einstein’s definition of insanity seems to fit well here: "doing the same thing over and over again and expecting different results."
Be careful what you measure
Organizations love metrics because it makes setting targets easier, and discourages people from questioning the goal behind the target. This leads managers into a false sense of organizational efficiency. Strong incentives tied to strong metrics force people to concentrate on just one part of the work, neglecting other contributing factors that might make a goal more successful. Organizations must be wary of this actively destructive focus that leads people to neglect other important factors.
Even agile techniques do not protect teams from the undesirable behaviors driven by measuring and tracking the wrong number. For example, agile teams often use story cards for development work. Teams often visualize these small increments of work on board as it moves through their organization’s software lifecycle. A typical process might look like this with the ideal flow of stories moving from left to right:
Figure 1: Example of a storywall
Management and product management often ask the question, "How soon will that feature be complete?" Teams often choose to interpret this as when coding finishes, succumbing to the idea that testing and the path to production are trivial and inconsequential parts of software process. Project management reinforces this perception by asking the question, "How many stories did we finish coding this week?" instead of the better question, "How many stories are we happy to release to end users?" or better yet, "How many stories did we release to end users?" An even better question is, "How much value have our users found from our recent releases?"
Teams want to do the right thing and these questions and metrics consequently drive developers to focus on getting storiesDevelopment Complete. Let’s look at the consequences of overtly focusing on this sub optimal goal alone:
Malcolm, a marketing representative always takes a keen interest in what developers built for him that he dropped by the team as often as he could. He often talked to Dan, the developer, asking when his features would be complete. Dan, not wanting to disappoint Malcolm worked hard to focus on finishing whatever Malcolm asked, knowing he wouldn't be far off from returning to ask on progress. He'd often think to himself, "This feature must be really important." Tim, the team's newest tester often needed to approach a developer, like Dan, to understand how to trigger the newly developed features.
Tim approaches Dan one day, "Hi Dan! I really need your help to understand how to test this feature you completed last week." Dan, under pressure to deliver snaps, "Can't you do anything by yourself? I need to get this feature complete so Malcolm gets off my back." Shocked at Dan's response, Tim returns to his desk, and waits. He thinks to himself, "I can't get anything done until Dan helps me out."
Each week this happens, and over time, the stack of stories waiting to be tested grows and grows. Eventually Malcolm calls a meeting with the team concerned he’s yet to see that feature he asked for two months ago in production. Surprised, Dan says he completed it over a month ago. Tim bashfully responds, "I couldn't test that story because I needed some help from Dan and he's been so busy with other work. I didn't want to interrupt him."
What can we learn from this story? Firstly, what matters to Malcolm is that the flow of work is getting done. Even though Malcolm asks when something will be completed, what he really wants is to be able to use it in production. We know that Tim didn’t have knowledge necessary to complete and his work and the pressures on Dan to complete more work prevented Tim from acquiring any more knowledge. The end result was a vicious cycle of work building up in testing, never getting released and with Malcolm puzzled why he hadn’t received the feature he’d asked for. This is why methods like Kanban Software Development encourage Explicit Work in Progress limits. These limits force people to help out other when bottlenecks appear. These WIP limits work to overcome the undesirable behaviors that emerge when people are measured by the wrong metric of their individual productivity instead of overall value delivered. The book, Lean Software Development, stresses the importance of measuring the end to end result instead of simply a small part of the process, calling out a principle they call ‘Optmize the Whole’. Optimizing the whole means ensuring the metrics in use do not drive sub optimal behavior towards the real goal of delivering useful software.
Guidelines for a more appropriate use of metrics
Given the undesirable behaviors that emerge due to the inappropriate use of metrics, does this mean there is no place for them? Of course there is a place for metrics. What is needed is a different method. Use the following guidelines to lead you to a more appropriate use of metrics:
Explicitly link metrics to goals
Favor tracking trends over absolute numbers
Use shorter tracking periods
Change metrics when they stop driving change
We’ll use the following sections to explore what these mean.
Explicitly link metrics to goals
In the traditional style, management decides what the best measure for a particular goal is. Management then set a target in terms of that measure. Management then articulate only this target to people doing the work, in its, often, numerical representation. The lines between the measure chosen to monitor progress towards the goal and the actual goal itself blur. Over time, the reason behind the measure is lost and people focus on meeting the target even if that metric is no longer relevant. A more appropriate use of metrics is to ensure that the chosen measure for progress, the metric, is teased out, yet related to its purpose, the goal.
For example, in a software development context, you might see metrics defined like this:
Methods must be less than 15 lines. You must not have more than 4 parameters to a method. Method cyclomatic complexity must not exceed 20.
With an appropriate use of metrics, every single measure should clearly be linked to its original purpose. The current mechanism for tracking and monitoring must be decoupled from its goal and that goal made explicit to help people better understand the metric’s intent. A metric in a richer context for its existence guides people in making more appropriate, pragmatic and ultimately useful decisions towards the goal. Without its purpose, the effort expended means people find ways to creatively game their system, ultimately detracting from the real goal. Here's what that looks like:
We would like our code to be less complex and easier to change. Therefore we should aim to write short methods (less than 15 lines) with a low cyclomatic complexity (less than 20 is good). We should also aim to have a small handful of parameters (up to four) so that methods remain as focused as possible.
Explicit linking the metrics to the goal allow people to better challenge their relevance, to find other ways of satisfying the need, and to help people understand the intent behind the numbers. Without this articulated purpose, people may find ways, unintentionally working against the implicit goal. For example, a number of techniques might help reduce a method length, but increase overall complexity by being harder to read if not applied with the correct intent.
The nature of software development means most work is knowledge work, and is therefore hard to observe. It is easy to monitor activity (how much time they sit at their computer) yet it is hard to observe the value they produce (useful software that meets a real need). The further that people move away from the code, the harder it is for them to appreciate the complexities involved. This implies that it is very difficult, if not impossible for people furthest away from the work to really know the best measure to monitor for progress towards the goal.
A shift towards a more appropriate use of metrics means management cannot come up with measures in isolation. They must no longer delude themselves into thinking they know the best method for monitoring progress and stop enforcing a measure that may or may not be the most relevant to the goal. Instead management is responsible for ensuring the end goal is always kept in sight, working with the people with the most knowledge of the system to come up with measures that make the most sense to monitor for progress.
Favor tracking trends over absolute numbers
Management find metrics too hard to resist because it distils down organizational complexity into something everyone can understand, a number. It’s easy to see one number is bigger or smaller than another, or how distant one number is from another. It’s much harder to see if that number is still relevant. This traditional approach to management likes using these metrics because it makes it easy to communicate when a target is met. "Just reach this number and we’ll be fine".
When you turn a qualitative and highly interpretive issue (think of productivity, quality, and usability) into a number, any figure is relative and arbitrary. There may be significant difference between code coverage of 5% and 95%, but is there really a significant difference between 94% and 95%? Choosing 95% as a target helps people understand when to stop, but if it requires an order of magnitude of effort getting that last 1%, is it really worth it? This is only something that people must work out subjectively in their own organizational context.
Looking at trends provides more interesting information than whether or not a target is met. Working out if a goal is met is easy. The difficult work, and one that management must work with people with the skills to complete is looking at trends to see if they are moving in the desired direction and a fast enough rate. Trends provide leading indicators into the performance that emerges from organizational complexity. It is clearly pointless focusing on the gap in a number when a trend moves further and further away from a desired state.
Focusing on trends is important because it provides feedback based on real data on any change implemented and creates more options for organizations to react. For instance, if the team is trending away from a desired state, they can ask themselves what is causing them to move away from their goal and what can they do about it. It pre-empts action much earlier than simply doing as much as they can before working out a number. If a team find themselves trending towards a desired state, they can ask themselves what is helping them move towards their goal and what else can be done to accelerate that rate. Measuring teams encourages people to experiment much more. Tweak one thing and observe its effect on the trend, monitoring where you are with the desired state and knowing when to stop.
Arbitrary absolute numbers also create helplessness, especially when progress towards a goal is slow and dependencies on other departments or corporate policies outside of a group’s control prevent more progress. Trends help focus people’s efforts on making movement in the right direction rather than being paralyzed between a gap that looks impossible to resolve.
A more appropriate use of metrics requires more management involvement in reporting and recording movements in trends because the ecosystem that surrounds a team is management’s responsibility. This ecosystem includes the organization’s policies, the way work is scheduled or planned and the way that teams and people are organized. This ecosystem often has much more influence on the trend that the efforts expended by individuals. Management should be interested in trends to observe the effects of changes to this ecosystem.
An appropriate use of metrics finds trends much more useful than absolute numbers. Arbitrary targets don’t really have much meaning without the right trend and better questions emerge when thinking about what affects a trend and what else can be done to affect the trend, rather than pointing about what the gap is between an arbitrary number and reality.
Use shorter tracking periods
Many organizations use metrics to set targets for very long periods, typically 3-6 months, even as long as a year or beyond. Managers establish this target, with the responsibility lying with the people doing the work to do whatever they can to meet that target. Management revisits this target at the end of the period to evaluate the people doing the work. In this system, the relationship between management and the workforce is, at best, described as confrontational. The workforce, doing their best endeavors, do whatever they can to meet the goal, with an implicit idea that management do not have any responsibility.
A consequence of revisiting metrics after long periods is that the failure to meet management’s arbitrary target becomes more and more unacceptable. I’ve heard managers say things like, "You had a whole year to meet your target and you missed it." The risk and cost of failure increases the longer the tracking period is.
Agile methods prefer shorter periods for review because any performance gap is less costly. Failing to make enough progress in a week is much less significant than failing to make enough progress over a whole year. Reviewing progress after each week generates many more options than reviewing progress after a year, simply because there are more opportunities to react and change. After a short period such as a week, you also have much more data about what actually happened instead of what was planned, and this should be used to influence the outcome by using it to drive change.
Organizations benefit from using shorter tracking periods as it creates more opportunities for re-planning that allows maximum value.
I worked with a team that released software into production every two weeks. The business liked regular releases because they could use the software almost immediately. On using the software deployed after the latest release, the business discovered they had enough features they could do almost everything they needed for a new marketing initiative. It was only a fraction of what they originally asked for.
Instead of the development team writing features that would probably never be used, the business picked a small subset of the leftover stories and started work on the next initiative.
An appropriate use of metrics tracks progress in smaller cycles because it gives much more information about where a project may end up further in the future. Tracking smaller periods helps identify trends and the pause gives organizations a more informed position to influence the environment and the rate/direction of a trend.
Tracking smaller periods also enables more collaboration because it provides more opportunity for management to be involved. Rather than simply evaluating people at the end of a larger period, tracking smaller periods provides more data about what is actually happening that influences the trends.
Change metrics when they stop driving change
If organizations reached goals easily, they would never need metrics. The organization could shift direction and they would reach their goal immediately. Unfortunately this doesn’t happen in reality which is why measures exist. Achieving a goal often takes much longer. The first guideline to an appropriate use of metrics separates the real goal from the measure selected to monitor progress towards that goal. The real goal must always be made explicit.
Guideline #2 and #3, monitoring trends and doing so over shorter periods is about helping organizations realize their goal faster. It isn’t achieved through the single-loop learning described earlier in the chapter. Organizations require is the double-loop learning Argyris writes about. An appropriate use of metrics drives people to question the goal and, based on collecting real data, implementing change to get there.
Here’s what double loop learning looks like:
Frustrated by fixing bugs every week Dan the developer considers why he is constantly fixing bugs. Over the last three weeks, Malcolm reports many issues about things not working as he expected. He steps back to think about what is really going on, less concerned about the bug count he is always asked about and more about why he has them to begin with.
When Dan picks up a story, he often has lots questions for Malcolm about how it should work. Dan knows Malcolm has his other marketing activities keeping him busy and understands Malcolm cannot sit with him to answer his questions. Dan is under enormous pressure to deliver something, so he makes several assumptions to ensure he can deliver something instead of nothing.
Looking at the bugs, Dan realizes that many of the bugs reported are based on those small assumptions he keeps making. The pressures to deliver something mean that Dan never builds the right thing the first time around.
When Dan explains this to Malcolm, they agree to sit down at the start of each new story to make sure all of Dan’s questions are answered before he starts coding. They try this the next week and the overall number of bugs reported that week decreases.
Double loop learning requires more data about what is actually going on. Shorter periods create more data points, making it easier to see any trends. The trends offer insight into the current performance of the system and should be used to trigger thinking and problem solving about the deeper underlying forces at play in the system, not simply for tracking performance measurement. Implementing real change helps accelerate organizations towards their current goal.
Changing the system that people work in often has a much greater impact than focusing on the individual’s efforts to work harder or faster. In our story, Dan could have spent more time each week trying to fix bugs, but by adjusting the flow of information and the working relationship between Malcolm and Dan, they changed the system to be much more effective.
Project post-mortems look back after a project finished, seeking lessons learned in the hope of applying them to future projects or spreading them across an organization. Conducting post-mortems at the end of the project offers no chance to actually apply these learnings to the project itself. Agile Retrospectives differ in their intent by seeking change while a project is in flight, where actions have more impact than they would at the end. These meetings create an opportunity for teams to look for opportunities for change though still rely on the people and organization to commit to those changes.
When an organization reaches its goals, it’s time to return the metrics used to achieve it. Remember that if organizations accomplished their goals instantaneously, there would never be a need for a metric. Defining, tracking and monitoring and interpreting metrics takes time and resources that could be better spent against new goals. Organizations need to drop metrics that are no longer relevant, instead of holding on to all the metrics they are used to collecting. With an appropriate use of metrics, understanding what metrics to retire will be easy because those metrics have been explicitly linked to the goal and constant monitoring of trends during periods encourage a continuous review of the state of the end goal.
You can look for some symptoms for potentially out-dated metrics by asking people, "Why do we need to collect this number?" A terrible response might include, "That’s the way we’ve always done it," or worse yet, "It’s our policy." This question does not necessarily discriminate between poorly explained goals, or out-dated metrics so it probably takes a little bit more digging. It is management’s responsibility to ensure that an organization’s time is not spent needlessly gathering, maintaining unnecessary metrics.
Metrics have a purpose and a place in organizations and teams. They cannot be used as a substitute for thinking. Nor can organizations think management by numbers is enough for effective software delivery. Organizations must be vigilant against the undesirable behaviors that emerge due to the inappropriate use of metrics. Double loop learning helps us understand focusing on the individual to behave differently cannot exist until the organization learns a more appropriate use for metrics.
With the appropriate use of metrics, organizations link each measure back to a well-articulated goal that everyone understands. The measure chosen to monitor progress must be decoupled from the goal, and challenging each metric’s relevance welcomed as time passes. Organizations using metrics more appropriately understand the value in watching the trends, monitoring in smaller periods in order to understand individual, management and organizational influences. A better use also means frequent inspecting and adapting these influences to ensure trends accelerate, decelerate and reverse in the context of an end goal that is constantly evaluated for fitness. The most appropriate use of metrics also means understanding when measures are no longer relevant, replacing them, or dropping them as progress is made towards the goal and the environment changes around it.