Business Performance Measurement
At the Crossroads of Strategy, Decision-Making,
Learning and Information Visualization
February 2003
Vince Kellen
CIO &
Faculty,School of CTI
DePaul University
Chicago, IL
U.S.A.
http://www.depaul.edu
Abstract
Business Performance Measurement (BPM) systems have grown in use and popularity over the past
twenty years. Firms adopt BPM systems for a variety of reasons, but chiefly to improve control over the
firm in ways that traditional accounting systems have not allowed. Several approaches, or frameworks,
for building and managing BPM systems have evolved with the balanced scorecard as the dominant
framework in use today. Despite the growing use of BPM systems in organizations of all kinds,
significant problems cause firms to experience difficulty in implementing BPM systems. The problems
range across a variety of topics: excessive diversity in the field of study, data quality and information
system integration problems, lack of linkage to strategy, fundamental differences in how a strategy is
formulated and executed in the firm, ill-defined metrics identification processes, high levels of change in
BPM systems, analytical skills challenges, knowledge as a social and non-deterministic phenomenon,
judgment and decision biases (from prospect theory literature) and organizational defenses that can
undermine successful BPM systems use. To help address these problems, a set of critical success factors
for BPM projects, derived from the literature, are identified. A minimal set of four criteria for designing
successful BPM systems along with 12 BPM system factors to be considered when building BPM
systems are discussed. Forty (40) software vendors with BPM related solutions are listed and the role of
data visualization and metaphor is discussed as a potential means for addressing cognitive problems with
BPM systems. Given the continual information processing and computing power improvements coupled
with the advances in business strategy theory, analysis of decision-making, organizational learning and
BPM systems overall, BPM systems are at a crossroads of difficult problems and interesting
opportunities.
Introduction
While this paper discusses business performance measurement (BPM), two larger questions must be
looked at in order to understand the core issues and opportunities in BPM:
1. How do we know?
2. What moves us to act?
In these questions, the “we” and the “us” referred to can stand for individuals, teams, business units,
corporations and organizations and cultures. As encompassing as this definition of “we” is, it is necessary
because of the ubiquity of communication technology, most notably the Internet, which is causing us (as
individuals, teams, business units and companies) to again wrestle with understanding how it is we know
who our competitors and collaborators are, what our core capabilities are, what makes us unique and what
we should be doing. The technology is allowing a radical increase in the potential gathering and
disseminating of information worldwide. The second question also follows from the first. Knowing
without acting properly can be fatal. For businesses, these epistemological concerns are increasingly
important as markets, competitors and customers are changing faster and causing businesses that had
previously been thought of as enduring to turn up as ephemeral. Business performance measurement is an
integral component to how businesses do know things and how it is they cause themselves to act in a
manner that helps them survive and thrive. It is natural, then, that businesses would show an increased
interest in performance measurement, especially since Internet technology makes diffusion of
performance measurement across the business or across businesses much simpler than in the past. The
measurement mantra continues to reverberate throughout nearly every corridor of business life.
In order to survive and succeed, firms need to set strategic directions, establish goals, execute decisions
and monitor their state and behavior as they move towards their goal. Once a firm becomes large enough
that a single manager cannot sense the firm’s current state and cannot control its behavior alone, the firm
must use performance measurement and control systems to replace the eyes and ears of the beleaguered
manager. Over the past few decades, firms have used information technology to provide this “sense and
control” capability. Several dozen vendors provide business performance measurement information
technology solutions. These tools have leveraged the latest advancements in data and application
integration approaches, web-based charting and reporting, statistical analysis, artificial intelligence,
machine learning and expert system technology.
Yet despite the technology’s improvement, availability and increasing adoption rate, many challenges to
successful adoption and use abound. The challenges in implementing performance measurement arise in
the following areas:
Technical Data quality & latency, application usability, visualization of data
Organizational Business culture, leadership, processes, strategic control and intent
Individual Gesturing, biases, framing and decision-making abilities
Overcoming these challenges are not as simple as finding the right software, establishing the set of best
practices and implementing a BPM system. Issues in each of these areas are teased out of some of the
recent BPM and related literature and discussed here.
What are Business Performance Measurement and Control
Systems?
Specifically, business performance measurement and control systems are the formal, information-based
routines and procedures managers use to maintain or alter patterns in organizational activities (Simmons
2000). A typical performance measurement helps businesses in periodically setting business goals and
then providing feedback to managers on progress towards those goals. The time horizon for these goals
can typically be about a year or less for short-term goals or span several years for long-term goals
(Simmons 2000). Since a BPM system measures performance, it is important to define what performance
is. Lebas and Euske (2002) provide a good definition of performance as “doing today what will lead to
measured value outcomes tomorrow.” BPM then is concerned with measuring this performance relative
to some benchmark, be it a competitor’s performance or a preset target.
Measurement systems are comprised of multiple measures. A measure (or metric) is a quantitative value
that can be used for purposes of comparison (Simmons 2000). A specific measure can be compared to
itself over time, compared with a preset target or evaluated along with other measures. Since a measure is
used for the purpose of comparison, it need not represent an absolute value. For example, in measuring
customer profitability, knowing the relative distance in profitability between two customers may be as
valuable (and more easily gotten) than knowing the absolute value for a customer’s profitability.
Moreover, many BPM systems normalize a measure into a value that promotes comparison not just with
itself, but also with other measures.
Following Simmons (2000), measures can be objective or subjective. Objective measures can be
independently measured and verified. Subjective ones cannot. Measures are also typically classified as
financial or non-financial. Financial measures are typically derived from or directly related to chart of
accounts and found in a company’s profit and loss statement or balance sheet, such as inventory levels or
cash on hand. Non-financial measures are measures not found in the chart of accounts, such as customer
satisfaction scores or product quality measures. Measures are also leading or lagging. Lagging measures
give feedback on past performance, such as last month’s profit, and typically do not provide insight into
future performance. Leading indicators, in contrast, are designed to measure future performance, and
more often than not, future financial performance. Some leading indicators to future performance might
include customer defection rate, customer satisfaction scores or changes in consumer confidence.
Measures are either complete or incomplete. Complete measures capture all the relevant attributes of
achievement, whereas incomplete measures do not. Measures are also responsive or not responsive.
Individuals can influence responsive measures, whereas non-responsive measures are outside the
influence or control of an individual (such as consumer confidence). Measures may be related to inputs
into a process, feedback on the performance of a process itself or they may be related to the outcomes or
outputs from the process. Measures may be related to human performance, process performance or market
conditions. Some, but not all, measures are directly related to the firm’s strategy and are critical for its
successful execution of its strategy. These are called critical or key performance indicators. Finally,
measures can refer to tangible things, often recorded in the chart of accounts, such as inventory levels,
accounts receivable balances, employee headcount, or can refer to intangibles such as level of skill or
knowledge, creativity and innovation.
In summary, below is a listing of attributes that can be useful in examining, selecting, designing and using
measures:
• Objective / subjective
• Financial / non-financial
• Lagging / leading
• Complete / incomplete
• Responsive / non-responsive
• Inputs / process / output
• Critical / non-critical
• Tangible / intangible
When discussing performance measurement, most practitioners (and software vendors) refer to the type of
measurement that helps companies monitor its current and past state. Thresholds, both low and high, for
key performance indicators (KPIs) are set and managed by exception. When data begins to move outside
the threshold limits, the performance measurement system can alert management, who then attempt to
diagnose the problem and address its causes. This type of measurement is referred to as diagnostic control
systems (Simmons 2000). While this type of measurement provides management with basic control over
the firm and an “auto-pilot” capability that can keep the firm on target with its goals, it is frequently
insufficient for success.
Interactive control systems provide additional control capabilities to help the firm deal with strategic
uncertainties. According to Simmons (200), interactive control systems “are the formal information
systems that managers use to personally involve themselves in the decision activities of subordinates.”
Interactive control systems help managers integrate new data and learning into the decision-making
process. Diagnostic and interactive control systems are not disjoint. In fact, an important synergy may
exist between the two as multiple diagnostic control systems serve as a basis for dialog between levels in
the firm (de Hass & Kleingeld 1999). This strategic dialog can aid in managers questioning the validity of
its control system, constituting double-loop learning which challenges controlling assumptions or
variables for the process, the business unit or the firm.
BPM systems need to provide insight into different units or levels of analysis. Performance can be
ascribed to corporations, business units, support or functional units, teams and workgroups and
individuals. One key benefit of BPM systems lies in their ability to help align these different levels of
analysis in the firm. Many corporations consist of several business units or divisions that compete in
different markets with differing strategies. A corporate-wide BPM system can help articulate the theory of
the firm (why different business units exist within the corporation) and improve overall performance by
exploiting synergies between the business units (Kaplan & Norton 2001). At the lowest level of analysis
lies measurement of human performance, for which the literature and examples are rich and long. In
between the business unit and the individual lie other layers, such as the functional or service group,
workgroup or team and the business activity. BPM systems are often designed to be a vehicle for strategic
dialog within the firms. Therefore, performance metrics and scorecards scattered horizontally and
vertically across a corporation, need to be coherent so that the conversations between people about the
strategy is consistent and all the different measurement units contribute to the performance of the
corporation overall (de Haas & Kleingeld 1999). BPM systems can help provide this firm-wide
coherency.
Why Measure Business Performance?
Business performance measurement has a variety of uses. Bititci, Carrie and Turner (2002) list the
following reasons companies measure business performance:
• To monitor and control
• To drive improvement
• To maximize the effectiveness of the improvement effort
• To achieve alignment with organizational goals and objectives
• To reward and to discipline
Simmons (2000) looks at business performance measurement as a tool to balance five major tensions
within a firm:
1. Balancing profit, growth and control
2. Balancing short term results against long-term capabilities and growth opportunities
3. Balancing performance expectations of different constituencies
4. Balancing opportunities and attention
5. Balancing the motives of human behavior
Looking at the firm as a complex organism seeking to survive or thrive in its competitive environment,
performance measurement systems serve as a key contributor to the perceptual and coordination/control
capabilities of the firm. Firms use BPM systems to help monitor and control specific activities; to predict
future internal and external states; to monitor state and behavior relative to its goals; to make decisions
within needed time frames; and to alter the firm’s overall orientation and/or behavior.
Related terms
While performance measurement is frequently used to refer to systems that track and provide feedback on
strategy execution and implementation, other related concepts touch upon performance measurement in
some manner. The field of cybernetics, now an interdisciplinary study of organization, regardless of the
form or material representation of the organization, touches upon measurement systems (Principia
Cybernetica). Firms today are more frequently intertwined with information technology that collects and
delivers data that is significant to the control of the organization. Cybernetic systems are autonomic; they
are self-regulating. Other writers and experts frequently point to BPM systems as providing businesses
with this autonomic capability. Beer (1966) uses cybernetics as a means to describe management control
as a way of coaxing a system towards optimal performance, and even better, arranging for the system to
regulate itself.
In 2001, Gartner coined the term business activity monitoring (BAM), which is “the provision of realtime
access to critical performance indicators” (Flint 2002). BAM delivers alerts and business metrics in
real-time or near-real time to increase efficiency of business processes, monitor shifts of priority and
conflicting goals, increase customer satisfaction through improved product and service quality. Gartner
restricts the term BAM to refer to “systems that draw upon and support the management of several major
business processes” (Flint 2002).
Similar to BAM, event management and alerting (EM&A) is a term that several software vendors have
used to describe information technology that deals more specifically with how business process events are
managed and how alerts are distributed to management. Not specifically a performance measurement
tool, EM&A is used to deliver performance measurement data throughout an enterprise, typically through
messaging middleware and out to different devices, including cell phones, pagers, email boxes, web sites
and database entries. Depending on the breadth of each vendor’s offering, both BAM and EM&A
technology can be used in combination with performance measurement software. The BAM or EM&A
technologies become the integration and messaging layers of technology getting data to and from the
performance measurement application.
BPM Frameworks and Reference Models
Different frameworks and reference models for measuring business performance have evolved from a
variety of origins. Frameworks are approaches to measurement that businesses frequently adopted, often
with significant diversity in their design and use. Reference models are more rigorous standards, typically
around specific performance metrics and associated business processes, adopted by an industry or by a
common functional unit. Discussed in this paper are:
Frameworks
Balanced Scorecard
Economic Value Added
Activity-based costing
Quality Management
Customer Value Analysis
Action-Profit Linkage Model
Reference models
Supply chain management
New product development
The Balanced Scorecard
Perhaps the most widely used BPM framework is the balanced scorecard. Introduced by Robert S. Kaplan
and David P. Norton in 1992, balanced scorecards have found widespread adoption in Fortune 1000
companies. Initially focused on finding a way to report on leading indicators of a business’s health rather
than traditional accounting measures which are lagging indicators, the balanced scored was refocused to
measure the firm’s strategy. Instead of measuring anything, firms should measure those things that
directly relate to the firm’s strategy (Kaplan & Norton, 2001). Normally (although not required) the
balanced scorecard is broken down into four sections, called perspectives:
The financial perspective The strategy for growth, profitability and risk from the shareholder’s
perspective.
The customer perspective The strategy for creating value and differentiation from the perspective
of the customer.
The internal business
perspective
The strategic priorities for various business processes that create
customer and shareholder satisfaction.
The learning and growth
perspective
The priorities to create a climate that supports organizational change,
innovation and growth.
Economic Value Added
Developed by the Stern Stewart Corporation as an overall measure of financial performance, EVA is both
a specific performance measure and the basis for a larger performance measurement framework (Otley
1999). According to its creators, EVA is a financial performance metric that is most directly linked to the
creation of shareholder value, over time (Stern Stewart 2002. EVA is net operating profit less an
appropriate charge for the opportunity cost of all capital invested in an enterprise. Mathematically it is:
EVA = ( Net Operating Profit After Taxes ) – ( Capital X Cost of Capital )
It is designed to give managers better information and motivation to make decisions that will create the
greatest shareholder wealth. The EVA framework is typically used as a manager incentive plan. Since
EVA is a single metric (although it can cascade down and across an enterprise to evaluate the
performance of specific investments) it is complementary to the balanced scorecard and can be included
in one (Otley 1999). Using EVA alone can cause managers to invest in less risky, cost-reducing activities
rather than in growth activities and as a pure financial model, EVA cannot serve as a vehicle for
articulating a strategy. But coupled with the BSC, the trade-offs between short-term productivity
improvements and long-term growth goals can be managed (Kaplan 2001).
Activity-Based Costing
Activity-based costing (ABC) was developed to provide better insight into how overhead costs should be
allocated to individual products or customers. Typically, businesses make simple adjustments to allocate
overhead costs that do not accurately model how the product or cost consumes those overhead activities.
ABC links expenses related to resources supplied to the company to the activities performed within the
company. Expenses flow from resources to activities and then to products, services and customers. Using
the approach, companies get insights into profitable and profitless activities based on a customer or a
product viewpoint (Kaplan 2001). ABC, then is a way of measuring which of the firm’s activities
generate revenues in excess of costs and as a result, provide keen insight into what is really providing
value for customers (Meyer 2002).
Again, just as with EVA, ABC can be complementary to the BSC. Companies with large and growing
indirect and support expenses may benefit from an ABC measurement scheme first. Companies with a
low return on capital and a weak financial structure may start with EVA first. If the organization wishes
to implement a major change in its strategy, a BSC scheme may be embarked upon first. Over time, EVA
and ABC metrics can find their place within the BSC (Kaplan 2001).
Quality Management
Over the past few decades, many firms have adopted various quality programs, such as Total Quality
Management (TQM), Six Sigma, European Foundation Quality Management (EFQM) and The Baldridge
National Quality Program to improve the quality of the manufacturing and service offerings. A central
tenant for all of these programs is business performance measurement. For example, The Baldrige
National Quality Program measures businesses in seven categories and the EFQM in nine (Kaplan &
Lamotte, 2001):
Baldrige categories
Leadership
Human Resource Focus
Strategic Planning
Process Management
Customer and Market Focus
Information and Analysis
Business Results
EFQM Criteria
Leadership
People
Policy and Strategy
Partnerships and Resources
Processes
People Results
Customer Results
Society Results
Key Performance Indicators
Kaplan and Lamotte (2001) argue that quality program performance measurement need not be exclusive
of balanced scorecard measurement systems. They point out differences and synergies between the
frameworks:
• The BSC emphasizes explicit causal links through strategy maps ad cascaded objectives more
than the quality programs do.
• The BSC targets breakthrough performance whereas the quality programs rely on benchmarking
approaches
• The BSC sets strategic priorities for process enhancements.
• The BSC integrates budgeting, resource allocation, target-setting, reporting and feedback on
performance into ongoing management processes.
Quality programs, while grounded in product quality improvement and applied to many other business
processes (Wruck & Jensen, 1998), are continuous improvement frameworks that might not be best suited
to help manage discontinuities in business strategies (Kaplan and Lamotte, 2001). Despite the differences
between quality programs and the BSC, Kaplan and Lamotte (2001) see a symbiosis between the two
frameworks. They point out similarities in the four perspectives of the BSC and the four sub-categories
(customer-focused results, financial and market results, human resource results and organizational
effectiveness results) in the Baldrige Business Results category. The EFQM is also converging on more
detail in assessing the organization’s results. Proponents of the Baldrige frameworks also note the shift
over the years to integrated measurement. In 1997, the Baldrige criteria moved “further away from a
perceived narrow focus on ‘managing quality’ to a comprehensive framework for improving overall
organizational performance excellence” (Evans, 1997).
Customer Value Analysis
With the recent strong focus on customers, businesses have begun to deploy technologies, and
measurement systems, to manage business activities that directly or indirectly interact with the firm’s
customers. These customer relationship management (CRM) technologies are providing firms with better
data integration and hence better measurement regarding customers. With the obvious strategic
importance of customers, it is natural for businesses to begin exploring more robust ways of measuring
customers and the related business activities. Gale (1994) explains the role of managing customer value in
the context of the Baldrige National Quality Award’s customer focus and satisfaction criteria, which
comprises about 30% of the overall score for the award. Customer value analysis is sufficiently rich and
complex to require more robust analytical tools and frameworks for measurement and Gale (1994) offers
seven tools:
1. The market-perceived quality profile which breaks down the firm’s offering(s) into a set of
attributes that are scored and weighted from the customers’ perspective.
2. The market perceived price profile which breaks down the firm’s offering price (in many
businesses and markets price is composed of multiple aspects) into a set of attributes that are
scored and weighted from a customers’ perspective.
3. The customer value map depicts the firm’s relative perceived offering price and relative
perceived offering performance along and 2-dimensional grid against competitors.
4. Won/lost analysis researches further reasons and facts about why a customer defected or decided
to buy the firm’s offering.
5. Head-to-head area chart of customer value is a graphic display of how the firm’s offering is
performing against a single competitor.
6. Key events timeline depicts how the firm’s and competitors’ actions change the markets
perception of performance of each offering attribute.
7. A what/who matrix is a tool for tracking who is responsible for what actions that will change the
firm’s ability to improve its ability to manage customer value.
Rust, et al (2000) decompose the customer problem down to three top-level areas (with further
decomposition beneath each of the three):
1. Value equity refers to the customers’ perceptions of value
2. Brand equity refers to the customers’ subjective appraisal of the brand
3. Retention equity refers to the firm building relationships with customers and encouraging repeat-
purchasing
These three areas correspond to three distinct disciplines in the CRM and marketing literature (brand
management, customer value analysis and customer loyalty analysis), each with its own detailed
measurement approaches. Numerous other CRM measurement frameworks exist (Kellen, 2002). The
implications for BPM systems are clear: measuring business activities and outcomes regarding customers
is becoming increasingly complex and increasingly important to the firm’s strategy.
Action-Profit Linkage
Epstein and Westbrook (2001) developed the Action-Profit Linkage (APL) model to help firms identify,
measure and understand the causal links between company actions and profits. This framework is a
multiple-stakeholder behavior model that lays out the chain of effects as a result of changes in stakeholder
behavior. The APL model starts with the corporate strategy and moves to the four main components:
company actions, delivered product/service, customer actions and economic impact. Behaviors (and
perceptions and attitudes) are measured in each of the components.
For example, management and employee behavior can be measured as activities in the company action
component by measuring learning, workload, reward and recognition and culture. Product/service
characteristics such as price and quality can be measured along with employee or customer perceptions of
the product/service. Customer behaviors such as purchase rates, share of requirements, repeat purchases,
cross-sell rates and new referrals can be measured as well as attitudes such as customer satisfaction and
intent to purchase. Finally, economic impact such as customer revenues and profitability, number of new
customers and market share can be measured.
By presenting managers with an explanation of behavioral linkages between strategy, company actions,
product/service improvements, customer behavior and economic impact, Epstein & Westbrook (2001)
claim that firms can show some dramatic improvements. Their model shares features in common with
BSCs and ABC. Like ABC, APL looks at a firm as a collection of activities. Unlike ABC, APL describes
these activities in a strict behavioral linkage. Like the BSC, the APL model accommodates measures from
different aspects of a firm and combines internal and external measures and lagging and leading
indicators. While the BSC does have this notion of causal linkages between elements within the BSC, the
APL model ascribes behavioral causal linkages and because of this focuses more on the company’s
actions (Epstein & Westbrook, 2001). Despite the differences between APL and the BSC, Epstein &
Westbrook stress the compatibility between the APL and the BSC frameworks.
Reference models
Well-established standard measures for financial performance already exist. Not so for non-financial
measures. Different standards groups are developing standard approaches to non-financial measures.
TQM, Six Sigma and the EFQM Excellence model are methodologies and management systems that can
provide leading indicators of financial performance. The International Organization for Standardization
(ISO), along with the large public accounting firms, is working on the processes that generate these nonfinancial
numbers. The Supply Chain Council and the Product Development and Management
Association are developing reference models for business processes, which also include specific
performance measurements (Smith, 2001).
Just a few BPM reference models exist. Most likely this is due to popularity of the balanced scorecard as
a strategic measurement tool and the various quality programs that have become integrated business
performance measurement systems more closely supporting the business strategy. Since strategies are
often highly differentiated, BPM systems that support the strategy become highly differentiated and
propriety as well, including specific sequences of metrics. Hence, reuse on specific metrics and
arrangement of metrics is less likely to occur in these strategically aligned frameworks. A reference
model is more likely to cover non-core processes or processes that are highly consistent and diffuse across
many companies, such as new product development and supply chain management. While many
practitioners may desire reference models to ease the initial design and implementation of a BPM, they do
not address how to construct a BPM system that is tightly integrated with a differentiated strategy.
Issues with BPM
Commentary on performance measurement can be placed along a continuum ranging from views
supporting conventional approaches based on a rational view of decision-making at one end to
unconventional approaches that derive from a view based on ambiguity, irrationality or boundedrationality.
In addition, commentaries cover a wide range of topics from data integration and quality, best
practices to cognitive psychology that address multiple levels of analysis – including the individual, the
workgroup and team, the firm and cultures.
Diversity
Perhaps the first and most important issue with BPM is its diversity. Neely (2002) cites 12 million web
sites dedicated to performance measurement, up from 200,000 in 1997, a significant rise in the number of
conferences world-wide on the subject and wide-spread adoption of the BSC in large organizations. He
also cites tremendous diversity in the academic field as well, with experts in accounting, economics,
human resource management, marketing, operations management, psychology and sociology all
exploring the subject independent of each other. In 1998 at a multi-disciplinary conference on
performance measurement in the U.K., the 94 papers presented cited 1,245 books and articles, of which
less than 10% were cited more than once and only 0.3% were cited more than five times. More
importantly, Neely argues that there is little agreement on what are the most important themes and
theories in performance measurement.
Adoption rates
Citing studies by technology research firm Gartner, Frigo and Krumwiede (2000) report that the balanced
scorecard approach is in use at about 40% of Fortune 1000 companies. In the public sector, only 33% of
U.S. counties with populations of more than 50,000 were using performance measurement in any form
with a similar adoption rate among cities (Berman, 2002). Performance measurement may be following
similar diffusion patters as other productivity improvements, which can take a generation to achieve
widespread acceptance. Wide variation in the use of information technology may be part of the problem
as well. Performance measurement efforts may have more success in measuring activities and outputs,
versus outcomes. Outcomes require stakeholder or customer perceptions of timeliness, quality and
usefulness of services, which involve data not widely gathered (Berman, 2002).
Data quality
BPM systems typically draw their data from data warehouses that in turn draw their data from source
enterprise systems and numerous ancillary software and data sources throughout an enterprise. Bad data
quality is affecting the usefulness for data warehouses in general. The Data Warehousing Institute
(TDWI) reports in its study of 647 companies on data quality reports that 40% of the companies surveyed
have suffered losses, problems or costs due to poor data quality (Eckerson, 2002). Sources of data quality
are: lack of validation routines in data entry systems or in system loading; mismatched syntax (first name,
last name versus last name, first name), data formats (6-byte versus 4-byte data fields) and code structures
(male/female versus m/f); unexpected changes in source systems; the number and complexity of system
integration interfaces; poor system design; data conversion errors (Eckerson, 2002).
Strategy and measurement
BPM systems come in two distinct flavors: strategic and operational. The balanced scorecard is an
example of a strategic BPM system. Operational BPM systems help managers with specific operational
process control issues that may or may not be directly related to the strategy. Numerous researchers have
discussed the link between strategy, measurement and success.
Measurement plays a crucial role in translating business strategy into results (Lingle & Schiemann, 1996).
In the area of executive management, only 6 in 10 executives place confidence in the data presented to
them. Factors that prevent successful measurement include fuzzy objectives (more precise objectives
needed), unjustified trust in informal feedback systems, and existing entrenched measurement systems.
Those that measure gain agreement on the strategy, clarity of communication, focus and alignment and
organizational culture advantages (Lingle & Schiemann, 1996). The survey included 203 executives, 72
percent top executives. 50% of the respondents were from companies with more than 500 employees.
Not all measures are good ones to include in a strategic measurement system. Strategy and performance
measurements need to be intertwined, and as such are likely to be unique for each company. Companies
should measure how parts of their value chain actually fit together for an overarching advantage instead
of relying on process-by-process metrics (Porter, 2002).
Three forces are at work in shaping BPM: increased demands from capital markets for forecasting
accuracy; shorter product cycles, quicker market shifts and expanding partnerships; and the growing
sophistication and availability of information technology, including ERP software, and improved database
and analytic capabilities (Krell, 2002). Despite the widespread understanding of the link between strategy,
measurement and success and the need for some balance between internal/external, leading/lagging
indicators, internal, lagging metrics dominate performance measurement, rather than external and leading
indicators (Krell, 2002).
BPM system satisfaction
The Institute of Management Accountants (IMA) conducted an annual survey of its 1,300 members on
performance measurement systems, practices and trends and performed follow-up interviews with
respondents. The 2001 IMA survey reported that 80% of its respondents reported making changes in their
performance measurement systems during the last three years. Changes ranged from radical (dumping an
existing system) to incremental (adding or deleting performance measures, refining the mix of measures).
33% reported the change as a major overhaul. Overall, only 31% of the respondents reported their
existing performance measurement system as less than adequate to poor in supporting management’s
business objectives and initiatives. Only 15% considered their PM systems as very good to excellent in
communicating strategy. Balanced Scorecard (BSC) users fared much better in perception.
The key challenges for performance measurement is in intangible assets (human and information capital)
and innovation. (Frigo, 2001). In the IMA survey, 60% of the respondents said that innovation was a key
part of the firm’s mission statement yet more than 50% rated the BPM system as poor or less than
adequate in this area. Overall, less than 10% of the respondents rated performance measures for intangible
assets as very good or excellent (Frigo, 2002).
Percent rating PM system
Either “very good” or “excellent”
BSC users Non-users
Supporting management’s objectives and initiatives 52% 5%
Communicating strategy to employees 48% 3%
Supporting innovation 22% 2%
The American Institute of Certified Public Accountants and Lawrence Maisel, from the Balance
Scorecard Collaborative, conducted a study to determine current perceptions and practice regarding
performance measurement systems. The study included 2000 respondents to a survey and on-site
interviews with a smaller number of companies. Only 35% rate their BPM systems as effective and 80%
consider the information from their BPM systems as merely “adequate” if not “poor.” The study cited
many other points. Many respondents indicated using BSCs even though their BSC system fails to meet
the criteria set by the BSC creators (Kaplan and Norton). Performance measurement involves change
management, and therefore staff buy-in, education and leadership are all required. Benefits include
improved organizational development and leadership, financial performance, operating performance,
decision-making, and strategy and alignment. Common barriers cited include issues related to buy-in,
leadership, education and the measurement process itself. Better information quality and technology was
cited as one of the areas of needed improvement. Financial professionals see PM systems as more
effective than their accounting counterparts (Maisel, 2001)
A recent KPMG study of U.S and European business and government executives revealed that one of the
most common disappointments reported was the lack of data integrity and the inability of their system to
product meaningful information to support decision-making. The study also discovered that BPM systems
are not aligned with strategic business measures, dependant on lagging, not leading indicators, are poorly
integrated with internal and external information and rely too heavily on financial measures. Some factors
for failed BPM systems included: measuring things that are easily measured versus what should be
measured, data inaccuracy; measures that were too complicated, and users didn’t understand the system
and its measures (KPMG, 2001).
Building measurement systems
BSCs work best if employees have input into the formation of its parameters (Ho & McKay, 2002). The
number and type of parameters must be well thought out in advance. Firms should avoid an
unmanageable number of parameters so that the using the BSC does not become too cumbersome and
time-consuming.
Van Aken & Coleman (2002) identify a process for building BPM systems. After defining the need for
measurement and improvement, the process proceeds through the following steps:
• Creating a common understanding of what the organization does (its mission, key processes, and
key outputs).
• Defining key performance areas and understanding the metrics (so everyone knows if the process
was successful) are next.
• After a balanced and focused set of metrics has been defined, the measurement system must be
implemented, taking into account required resources, technology, training, and communication.
• In the remaining steps, the management team must systematically use the measurement system to
assess performance, determine improvement actions, and review the impact of these actions.
Van Aken & Coleman (2002) also report that the time it takes to design and implement a measurement
system varies and can take as little as one to two months if no major technologies or tools are needed.
Kaplan and Norton (2001) also report that the time to create a BSC can be, on average, eight weeks,
which is down from the 16 weeks setting up a BSC took in 1996. The use of industry templates for
strategy maps and more experience with the BSC contribute to the improvement in time. However,
Kaplan and Norton (2001) report that many companies find the BSC harder than it initially looked. Many
companies do not implement a complete BSC. Many measures are not collected and over time measures
are added and refined. Kaplan and Norton (2001) encourage firms to think of the BSC as a living
document that evolves over time.
Jensen & Sage (2000) describe a process for selecting and refining metrics (depicted in Figure 1). Since
metrics in a BPM system do change over time, firms need to establish the process for accommodating this
change.
Figure 1. Metrics identification and refinement process flowchart (Jensen & Sage, 2000)
Kaplan & Norton (2001) make the argument that BSCs are superior to other forms of performance
measurements, such as key process indicators (KPI scorecards) and stakeholder scorecards. BSCs link
performance measurement to the strategy, and cause and effect mapping (strategy maps) helps firms
identify the indirect links between business activities and overall performance. They warn that BSCs
should not just be a collection of financial and nonfinancial measures organized into three to five
perspectives. Metric selection should be driven by the firm’s strategy. While formulating a strategy is an
art form, describing the strategy (which BSCs do) should not be (Kaplan & Norton, 2001).
Causal and correlative measurement
A key attribute for the BSC and other BPM frameworks is the support for identifying and communicating
causal linkages between components of the business that fulfill the strategy. Identifying the causal
linkages may be a bottleneck. It might take firms too much time to statistically prove causal linkages
between performance metrics, thus severely impairing the causal diagnostic benefit that BPM systems are
supposed to supply (Maisel, 2000).
While some firms have begun establishing closed loop measurement approaches, which by their nature
seek to understand causality in a deeper fashion, many firms struggle with the more subjective correlative
measurement that might provide insight with less effort and in less time. Nail (2002) points out that
marketing departments need to master correlative measurement, not just closed loop measurement.
Complexity in integrated marketing campaigns is making closed loop measurement harder to do.
However, correlative measurement requires a more complex data store and higher level of analytical
skills than causal analysis (Nail, 2002).
Successful violations of accepted BPM principals
The widely accepted model of performance measurement adheres to three basic principles: performance
should be clearly defined; performance should be accurately measured; rewards should be contingent
upon measured performance (Austin & Gittell, 2002). However, continued acceptance on these three
principles may be misplaced. Austin and Gittell document cases where firms deliberately violated one or
more of these three principles but still exhibited high levels of performance relative to competitors. The
widely accepted model of performance is based on compliance and extrinsic motivation. Following
Austin & Gittell, these approaches tend to give rise to undesirable outcomes including the distortion of
information quality, the displacement of employee effort from real organizational goals to ones that can
be more easily pre-specified and measured and a deterioration of performance. In contrast, the firms in the
anomalous cases used a performance measurement approach that was based on ambiguity and intrinsic
motivation. In these approaches, information quality tends to be high and require dialog across levels in
the organization to determine what is the right thing to do in a particular context. Moreover, since
individuals are frequently not identified in specific performance situations, the “information-reducing”
effect of fear is minimized.
Intrinsic motivations have other benefits. Intrinsic motivation is often required to transfer tacit
information. Intrinsic motivation enhances learning and is needed for creativity. Intrinsic motivation can
help overcome those situations where it is impossible to specify all the relevant employee behaviors and
outcomes needed. Intrinsic motivation is also not without its faults, however. Changing intrinsic
motivation is hard and the outcomes are less certain. Intrinsic motivation has its ugly side as well. Envy,
vengeance and the desire to dominate are intrinsic motivations. Also, manipulating extrinsic motivations
enables more flexible firm behavior (Osterloh & Frey, 2002).
Discriminatory power and coherence
Almost all measures lose their ability over time to discriminate between good and bad performance
(Meyer, 2002). Causes include: improved performance, learning how to meet the measure without
improving the performance that is sought (perverse learning or gaming); replacing low performers with
high performers (selection); and withholding performance data when differences persist (suppression).
This requires firms to change measures and search for new measures that can discriminate better.
Activity-based approaches might provide measurement coherence. Meyer (2002) suggest that we think of
a firm as a bundling of activities which incur costs and may or may not add value to the customer. “The
problem for the firm is finding those activities that add value for the customer and generate revenues in
excess of costs, extending those activities, and reducing or eliminating those activities that incur only
costs,” (Meyer, 2002). He has proposed activity-centric performance measurement as an approach that
can aid in establishing the right measures and improving lateral and horizontal measurement coherency in
the organization. His notion of activity-centric performance measurement is derived from activity-based
costing, and is referred to as activity-based revenue (ABR). ABR renders individual accountability
independent of organization design. This approach is not without problems either. Gathering activitybased
measures can be difficult. ABR is also more suitable for complex situations such as when a firm
supplies many products to many customers and the product specifications adding value to the customer
are not fully understood.
Bounded Rationality
Technology and process considerations aside, decision-making based on measurement data is fraught
with individual biases, depending on how the measurement data and problem is presented within the
relevant decision-making context. Most managers make decisions about future hypothesized choices that
make little sense from the utility theory perspective of a rational assessment of probabilities (Kahneman
& Tversky 2000). Prospect theory attempts to explain why people make what appear to be unsound
decisions, especially under uncertainty and risk. How decision problems are described (framed) can lead
to decision outcomes that deviate from standard decision-making theory (utility theory). These issues
around framing and biases can affect decision-making based on performance measurement data.
Managers consistently exhibit unwarranted risk aversion and a propensity to look at decisions in narrow
terms often isolated from future or past decisions (narrow framing), quite possibly leading to, in
aggregate, incorrect management choices. This narrow framing and excessive risk aversion may be
unintended consequences of excessive insistence on measurable short-term successes (Kahneman &
Lovallo, 1993). A performance measurement system project that proceeds unaware of the framing issues,
the heuristics people employ when making judgments under uncertainty and the cognitive biases that even
statistical experts possess and employ, could be headed for little impact on the business, or worse still,
accelerated faulty decision-making.
When making decisions, many organizations are often overly optimistic (the optimism bias). Despite the
use of measurement, analysis of data and worst-case scenarios, decision-makers paint rosy pictures that
give an ungrounded illusion of control (Kahneman & Lovallo, 1993). Realism, however, may have its
costs. Kahneman and Lovallo (1993) point to research that indicates “the deeply disturbing conclusion
that optimistic self-delusion is both a diagnostic indication of mental health and well-being, and a positive
causal factor that contributes to successful coping with the challenges of life.”
While the work of Kahnamen, Tversky and others in the prospect theory strain have contributed much to
understanding judgments and decision-making, some of the biases and heuristics that prospect theory
“uncovers” may be an artifact how information in these probabilistic reasoning tasks is represented
(Gigerenzer, 2000). Some of the biases that prospect theory exposes disappear when the information in
the problem is represented as a frequency problem rather than a probability problem. Gigerenzer (2000)
argues: “While the standard probability format has become a common way to communicate information,
… it is only one of many mathematically equivalent ways of representing information.”
Gigerenzer shows how the dramatic removal of one such fallacy, the conjunction fallacy, is achieved with
a frequentist representation of the information. Following Gigerenzer (2000), the original conjunction
fallacy problem is shown below:
Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student
she was deeply concerned with issues f discrimination and social justice, and also participated in
antinuclear demonstrations.
Participants are asked with of the two alternatives was more probable:
1) Linda is a bank teller
2) Linda is a bank teller and is active in the feminist movement
His transformed problem asks participants in the study the following:
There are 100 persons who fit the description above (i.e., Linda’s). How many of them are:
a) bank tellers
b) bank tellers and active in the feminist movement
In Gigerenzer’s (2000) research, the conjunction fallacy largely disappears with this reframing.
While the classic depiction of the fallacy above is constrained and not usually naturally found, as is, in
BPM environments, this type of reasoning employed by people may in fact be behind many decisionmaking
problems. The implications for business performance measurement is that information
representation can have a significant impact in the number of decision errors made as a result of common
cognitive limitations.
Use of Metaphor
A further source of errors in reasoning may lie in how the human brain actually processes information and
reasons. Assuming that to reason is to construct and manipulate symbolic representations in a language of
thought may be wrong. Based on our biology and evolution, our use of visual, spatial and kinesthetic
cognitive abilities to process information may be the basis for reason and language (Potts, 2001). The role
of metaphor (image schemas and perceptuo-motor mappings) is to provide a structural environment for
this reasoning.
Citing Lakoff & Johnson (1999), Potts enumerates some metaphors and associated abstract concepts:
Metaphor Abstract Schema Concrete Schema
Important is big Significance in situation Seen/felt size
Difficulties are burdens Obstacle to intention Felt weight
More is up Quantity or degree Seen or felt elevation
Categories are containers Classification Seen or felt containment
Similarity is closeness Diagnostic/predictive
similarity
Seen or felt proximity
Help is support Assistance toward intention Felt firmness underneath
Time is motion Passage of time Seen/felt movement
States are locations Situational equivalence Seen/felt place
Change is motion Variation over time Seen/felt movement
Action is self-propulsion Autonomous activity Intentional movement
Purposes are desired objects Intention Reinforcing object
Causes are physical forces Causes and origins Felt pressure and weight
Relationships are enclosures Relational dependency Seen/felt enclosure
Controlling is being above Causal dependency Vertical alignment
Seeing is understanding Knowledge Objects seen
Understanding is grasping Knowledge and
comprehension
Objects actively felt
Potts (2001) suggests that “if all cognition, from everyday preconscious thinking to abstract
professionalized mathematical reasoning is grounded in perceptual and performance schemas rather than
arbitrary symbolic codes, perhaps we should consider the ways in such schemas can be used in
professional descriptions of desired and actual computing and information artifacts.”
Indeed. While Potts was considering the use of metaphor to enrich requirements specification of
computing systems, the approach is especially noteworthy with regard to BPM systems. Visualization and
metaphor, if is the basis for reasoning skills, may be the best form of further reducing errors in decisionmaking
due to problems of cognition.
Others have classified metaphors differently. Nesbitt (2000) classifies metaphors into five categories:
1. Spatial metaphors. These relate to scale, location and structure and can carry quantitative
information. Relationships can be described by a position on a map or a two or three-dimensional
grid. Structures such as tree maps or data maps can carry broader overview information.
2. Temporal metaphors. These are concerned with how data changes over time. It includes concepts
of movement, animation, rhythms and cycles.
3. Sight metaphors. These use direct mappings from information to the attributes of sight. These
include color, light, shape and surface texture. Icons are an example of how abstract shapes can
be used to convey information using intuitive symbols.
4. Sound metaphors. These deal with direct mappings of typical sound properties such as pitch,
amplitude, timbre and also more musical qualities such as rhythm and melody. Auditory symbols
are less common, but good examples exist, such as the Geiger counter.
5. Touch metaphors. These relate to tactile properties such as force, inertia and vibration. Other
properties such as weight, density, hardness and surface texture can be used to encode
information.
With abstract data (data that does not conform to the three-dimensional world, such as business
information), metaphor classifications would first help guide the analysis and transformation of the data
(Nesbitt, 2000).
Defensiveness and Organizational Learning
More danger lurks still. Cognitive limitations and problem framing is just one component of the overall
issues in BPM. How organizations deal with threatening information is another substantial concern. It is
one thing to have the problem framed appropriately and have the performance data represented properly.
It is another to get human beings to discuss the information and its implications and take appropriate
action. Argyris and Schön (1996) examine the role defensiveness plays in organizational learning.
Organizational inquiry (the intertwining of thought and action that proceeds from doubt to the resolution
of doubt) can produce a change in thinking and acting within an organization. This learning, however, has
two distinct “flavors” – single-loop and double loop learning. Single loop learning produces a change in
behavior, but not a change in the underlying assumptions within the organization. For example, normal
product process improvement inquiry may yield changes in the manufacturing process, but not impact
basic assumptions about the organization. Double-loop learning on the other hand results in a change in
the strategies and assumptions governing the activities (Argyris and Schön, 1996). The relationship
between the two forms of learning is depicted in Figure 2.
Figure 2. Double-loop and Single-loop learning (adapted from Argyris, 1999).
Argyris and Schön describe two models of theories in action. These are theories that organizations
actually put in to use (theories of action put to use, or action theory), versus theories that organizations
say they put into action (theories of action espoused, or espoused theory). The first model, (called ModelI),
is one they claim to have observed in many organizations in many environments throughout the world.
Model-I theory in use has four main governing variables or values and the accompanying action
strategies:
1 Define goals and try to
achieve them
Design and manage the environment unilaterally (be persuasive,
appeal to larger goals.
2 Maximize winning and
minimize losing
Own and control the task (claim ownership of the task, be
guardian of the definition and execution of the task).
3 Minimize generating or
expressing negative
feelings
Unilaterally protect yourself (speak in inferred categories with
little or no directly observable data, be blind to impact on others
and to incongruity; use defensive actions such as blaming,
stereotyping, suppressing feelings, intellectualizing)
4 Be rational Unilaterally protect others from being hurt (withhold
information, create rules to censor information and behavior,
hold private meetings)
An organization with this learning model (which Argyris and Schön contend is a great many
organizations) is “highly unlikely to alter its governing variables, norms and assumptions,” (Argyris and
Schön, 1996). Argyris and Schön offer Model-II theory in use has three main governing variables and
accompanying action strategies:
1 Valid information Design situations where participants can be origins of action
and experience high personal causation.
2 Free and informed choice Task is jointly controlled
3 Internal commitment to the
choice and constant
monitoring of its
implementation
Protection of self is a joint enterprise and oriented toward
growth (speak in directly observable categories, seek to reduce
blindness about own inconsistency and incongruity). Bilateral
protection of others
If this model is followed, the degree of defensiveness between individuals and groups will tend to
decrease and double-loop learning will be enhanced (Argyris and Schön, 1996). Argyris contrasts the
organizational reasoning associated with these two models as defensive reasoning and productive
reasoning. He points out the productive reasoning in dynamic environments is especially difficult for
people because it requires them to “reexamine their basic assumptions and test their judgments against
changing conditions” (Argyris, 1997). Interestingly, he comments on the need to move beyond a static
conception of the world that is reflected in deterministic causality to forms of probabilistic reasoning.
“Because the world of action is dynamic and uncertain, probabilistic reasoning is more realistic and
accurate in assessing the likelihood of accomplishing our intended result,” (Argyris, 1997). Action can
more easily follow from probabilistic reasoning for Model-II organizations. Organizations holding to
defensive reasoning is more likely to dismiss probabilistic evidence that challenges the organization’s
espoused theory.
For BPM systems, organizational defensiveness has profound implications. While cognitive biases can be
overcome by reframing and representing problems so they are more intuitively understood, organizational
biases due to defensive behaviors are much harder to root out and change. Argryis points to some hope in
the form of management information systems. He states several benefits for these systems (Argyris,
1997):
1. Technology allows the design of information practices that support individual and organizational
learning
2. Storing and retrieving relevant actual performance information is relatively easy and timely
3. Individuals can use information technology tools to record and discover discrepancies between
stated goals and actual performance in a nonthreatening setting.
4. All members of the organization can have access to confirming and disconfirming data, lowering
the cloak of secrecy and control
5. By linking accurate, timely information to the sense of stewardship among decision makers, the
likelihood of learning increases. As organizations begin to change their practices, individuals
within them will feel less threatened and more willing to correct their mismatches between intent
and action as part of an ongoing development process.
To-date, most BPM systems excel in assisting single-loop learning, that is, correcting specific processes
so that they meet stated goals. BPM systems are currently not designed specifically or solely to help
manage the double-loop learning problem. Since organizational environments (markets and competitive
situations) can rapidly change and extinguish even the largest or seemingly durable entity, enhanced
double-loop learning is critical for long-term survival.
Competing Views of Strategic Intent, Strategic Control and Knowledge
In actual practice, companies craft their strategies differently. In some cases, strategy is a planned and
deliberate process. In others it is unplanned and emergent. Where one finds strategy crafted in a top-down
fashion, one can also examples of firms that created their strategy bottom, up, using grass-roots efforts
(Mintzberg, 1987). How a firm actually crafts, executes and controls its strategy can have a significant
effect on its BPM system. How firms construct their strategic BPM systems (ones that measure the
strategy) will most likely need to mirror how they constructed the strategy itself. Is it conceivable that a
top-down strategy in which the details of the strategy are known to a few at the top of the firm can be
measured with a BPM system built using bottom-up approaches that involve asking managers and
directors closer to the front-lines how to measure the strategy? Emergent strategies will have provisional
BPM measurement that must evolve over time, either using top-down and/or bottom-up conversations in
the firm to describe how formulate the strategy and how to best measure it
Business environments rarely change on a regular and orderly basis. Firms (and BPM systems) are
obviated when the environment shifts in subtle or profound ways. Turbulent changes are easy to detect
Some argue that subtle discontinuities are not and can only be dealt with by minds that are in tune with
existing patterns yet still able to perceive breaks in them (Mintzberg, 1987). With most knowledge of the
business environment today codified in digital data and presented as digital information, personal and
direct observation or trusted anecdote via word-of-mouth might be insufficient for detecting this change.
Perhaps BPM systems can play a significant role in not just monitoring changes in internal states, but also
understanding changes in the external environment as well.
Knowledge as a social phenomenon is emerging as a significant contribution to knowledge management
(Thomas, Kellogg & Erickson, 2001). Knowing what comprises the social and cognitive context that
contributes to individuals understanding things is critical for successfully managing knowledge in that
organization. Some social techniques for managing or creating knowledge include Bohm dialogs,
systematic use of metaphor, strategy mapping, story-telling and narrative, expressive communication.
Thomas, Kellogg & Erickson (2001) go so far as to re-label knowledge management as knowledge
socialization. Viewed in the light of the need for enhanced organizational learning, it is not surprising the
social nature of knowledge viewpoint is challenging more traditional thinking.
Stacey (2000) criticizes what he calls mainstream thinking on knowledge management for many
oversimplifications or inadequate explications. The key concepts in mainstream thinking – double-loop
learning, tacit and explicit knowledge, systems dynamics, sender-receiver models of knowledge
transmission from information theory, dialog as a special form of communication – have the following
problems: they treat individual learning different than organizational learning and hence require not one,
but two theories of how learning takes place, they fail to account for how new knowledge is created and
they cannot explain the unpredictable patterns of knowledge that may emerge outside of the control of the
managers. Stacey argues that “systems, databases, stored and written artifacts” are simply “records that
can only become knowledge when people use them as tools in their process of gesturing and responding
to each other.”
Stacey (2001) goes further, arguing that knowledge is not designed but emerges from the conscious and
unconscious interactions and gestures between individuals, and as such can be thought of as a complex
adaptive system. Systems thinking, which attempts to understand things by examining and reducing the
interactions between components of the overall system, is inadequate for understanding how knowledge
actually is created and diffuses. Stacey concludes that “from this perspective, it becomes impossible to
think of designing such a process and it makes no sense to think of managing it.” In this view, knowledge
is intractable.
What this means for BPM is that the art of identifying, linking and gathering data for a BPM system is
only part of the problem and not the thorny problem at that. Getting knowledge regarding the
performance measurement data diffused and used throughout an organization is at the core of what
knowledge management and BPM is all about. That, according to Stacey, may very well be an intractable
problem.
Success and Fail Factors
Business performance measurement literature abounds with do’s and don’ts of measurement. Some
classic problems include the use of too many metrics; the use of primarily or exclusively financial and
cost metrics; the use of only short-term focused metrics; a lack of alignment between operational metrics
and organizational objectives and strategies; and metrics that drive the wrong behaviors and performance
(Van Aken & Coleman, 2002). In their literature review and industry research, Bititci, Carrie and Turner
(2002) identified the following aspects companies look for a performance measure system to address:
• Reflect stakeholder requirements to maximize their satisfaction
• Reflect the external/competitive position of an organization
• Focus on competitive criteria of the organization’s markets in order facilitate strategies and
actions to improve the organization’s competitive position
• Provide an input into strategy development
• To help ensure alignment if strategy, actions and measures
• Differentiate between control and improvement measures
• Focus on critical areas of the business
• Use locally meaningful terminology to encourage understanding and ownership
• Facilitate resource bargaining
• Promote proactive measurement by focusing on leading measures
• Accommodate both quantitative and qualitative measures
• Measure organizational capability and learning
• Ensure that measures are used at the correct levels
• Promote understanding of the causal relationships between various measures
• Facilitate simple reporting
• Be dynamic and change in response to the internal and external environments
Based on their experience, Kaplan and Norton (2001) identify the following key pitfalls for BSC
performance measurement systems:
• Changes in executive leadership due to mergers and acquisitions in which BSC loses executive
sponsorship or priority
• Design problems in which a poor BSC is built
o Too few measures (two to three) per perspective
o Too many measures without clear identification of the critical few measures
o Lack of linkage to the BPM system and sustainable competitive advantage
• Process failures in implementing a BSC
o Lack of senior management commitment
o Too few individuals involved
o Keeping the scorecard at the top of the hierarchy and not distributing it
o Too long of a development process, not incremental
o Treating the BSC as a systems project
o Hiring inexperienced consultants
o Introducing the BSC for compensation purposes only
Based on the current issues regarding BPM systems discussed above and elsewhere in this paper, key
success/fail factors that have been cited in the literature are summarized in Table 1.
1. BPM systems need to be understandable and not overly complex
2. Employees should be involved and leadership must be present. The BPM system needs
to be contextualized and relevant to all parties, up and down in the organization
3. The organizational learning and knowledge management environment must be
consistent with and enhanced by the BPM system.
4. Data integration, latency and quality must be acceptable
5. Intangible, predictive and outcome measurement practices need to improve relative to
current practices
6. BPM systems need to enhance and/or be preceded with some level of organizational
alignment or coherence
7. BPM systems need help reveal linkages between the firm’s strategy and the activities
the firm must manage
8. Problems related to lack of reference models and uniqueness of strategies must be
overcome
9. The BPM system needs to be easily changed
Table 1. BPM critical success factors
Since today performance measurement data is almost always derived from or communicated with
computers and information system technology, it is not surprising that some of the critical success factors
are essentially technology problems.
A Framework for Evaluating Performance Measurement Systems
Over the last 20 years or so, established frameworks and reference models, along with specific, lesserknown
measurement approaches, APL, IPMS, OPTIMAS (Jensen & Sage, 2000) and others, have
populated the business world. Companies have choices when building a BPM. Despite the large amount
written about how to measure, much less has been written on what are the key attributes that are desirable
in a BPM system. De Haas & Kleingeld (1999) mention seven pre-existing measurement system criteria
from other studies and add their concept of coherence (discussed below) to the list to make eight:
1. Controllability
2. Validity
3. Completeness
4. Cost-effective measurability
5. Specificity
6. Relevance
7. Comprehensibility (Van Tuijl)
8. Coherence
In another study, Jensen & Sage (2000) enumerate metric design attributes (goals) and metric set goals
and measurement system infrastructure goals. The metric goals include:
1. Cost-effectiveness
2. Strategic alignment
3. Acceptability (buy-in)
4. Usefulness
5. Acquirability and implementability
6. Consistency
7. Accuracy
8. Reliability
9. Repeatability
10. Believability
11. Timeliness
12. Responsiveness
13. Known responsibilities
14. Security
The Jensen & Sage (2000) metric set goals include:
1. Balance across types of metrics
2. Organizational coverage
3. Completeness, minimum overlap
4. Cost-effectiveness
5. Total number, number per measurement area
6. Standardization
7. Documentation
8. Coverage of strategic thrusts
9. Current status and trend measures
10. Communications to staff
The Jensen & Sage (2000) measurement system infrastructure goals include:
1. Automation
2. Repository, communications and other security (access to archival information)
3. Labor hour reduction
4. Information dissemination
As can be seen, design attributes vary from author to author. This paper attempts to enumerate a minimum
set of BPM system design attributes (comprised of criteria and factors) and foregoes discussion of
specific metric design attributes, metric set attributes or their linkages to a defined strategy. Metric and
metric set design attributes can be derived from the BPM system design attributes. In addition, some
useful and successful BPM systems are operational in nature and may not be designed to clearly link to
and communicate a firm’s strategy. In the interest of minimalism, the criteria and factors offered in this
paper are silent on this matter of strategic linkage. Underlying the following design attributes is the notion
that BPM systems provide a key component to a firm’s ability to sense and respond to its internal and
external environments. Data in them is often tied to key motivational aspects for both the firm and its
employees. In addition, the term BPM system refers to the information technology and the human process
that interact with the technology. The two are conceived as joined in a symbiotic relationship with each
other and hence design attributes must take into account both aspects. Using this biological organism
metaphor, this paper recasts the prior design criteria described elsewhere into the following four key
measurement criteria (Table 2). In addition, 12 factors that link to these four criteria are discussed (Table
3).
1. The BPM system should help the firm accurately perceive relevant internal and
external phenomenon. These include threats and opportunities, shortcomings in its
ability to perceive phenomenon as well as shortcomings in its ability to control its
actions (breadth, depth, coherence and predictability).
2. Measurement information needs to be delivered, processed and acted upon within the
time frame needed for market survival (latency: propagation and response).
3. The BPM system must aid the decision-making process (provability, explainability,
believability, communicability).
4. The BPM system needs to operate self-reflexively and largely below the threshold of
the firm's awareness (adaptability, measurability, autonomic).
Table 2. Measurement system design criteria
Breadth Refers to how much of the total set of activities needed to be measured are
actually measured. Breadth needs to be balanced between internal state
and activities inside of the firm and activities and items external to the
firm such as customers, suppliers, competitors, market conditions,
environmental conditions, etc.
Depth Refers to the unit of analysis. Levels of analysis, or granularity, can
include the employee, the workgroup or team, the functional unit, the
business unit, the product, the customer, the firm as a whole, the
marketplace or the economy at large. BPM systems can and typically do
cover multiple levels of analysis.
Coherency Refers to the how much breadth and depth factors combine together to
improve performance. How do lower levels of measurement contribute to
higher levels? How do units of measurement at the same level coordinate
together to contribute higher levels?
Predictability Refers to how accurately and far into the future a BPM system can
project.
Provability Refers to how the BPM system can show the relationship between causes
and effects. Identifying causes and effects helps managers better
understand where (which object) to apply attention.
Explainability Refers to how easily people in the firm can explain relationships between
measurements and how the BPM system functions.
Believability Refers to how much people in the firm trust the BPM system. Do people
in the firm believe what the BPM system is expressing? Data quality and
overall measurement trust (reliability, consistency, accuracy) are key
components.
Communicability Refers to how well can people in the firm communicate measures and
discuss them amongst themselves?
Adaptability Refers how easily and completely the BPM system can be altered. Is the
BPM system automatically self-changing? How much intervention is
required to change it? Is the human component capable of changing?
Measurability Refers to how the BPM system itself is measured (meta-measurement). Is
the BPM system working within normal parameters? What is the quality
of service? How effective is the BPM system? Where is improvement in
the BPM system warranted? Is it measuring the right things?
Autonomic How much does the BPM system help the firm self-correct? How much
management attention and effort does operating the BPM system require?
Table 3. Measurement system design factors
The measurement criteria are non-gradated; that is the BPM system either meets the criteria or it does not.
If anyone of the four BPM system design criteria is not met, the BPM system may not be successful in
contributing to the success of the firm or may fall into disuse. The 12 factors are gradated. Individually
they vary depending on the constraints inside or outside the firm but collectively they meet the criteria
threshold.
Austin & Gittell’s (2002) discussion of the three conventional attributes (performance should be clearly
defined; performance should be accurately measured; rewards should be contingent upon measured
performance) is unnecessary to explicitely include here. Nor is a discussion of intrinsic, ambiguous or
extrinsic and unambiguous metrics/motivations. The factors in Table 3 relevant to the topic of
ambiguous-unambiguous metrics and intrinsic-extrinsic motivations can be scaled to either direction to
satisfy the criteria. In addition, causality is folded into the model as a factor, not a criteria, under the
notion that it might be possible (albeit remotely) for a BPM system to satisfy all the four criteria without
the need for strict causal proof or even causal reasoning. While managers generally intend to do things
with a causal framework in mind, the BPM system may not be able to capture (or need to capture) the
causal linkages. These design attributes (criteria and factors) make a clear distinction between what
managers intend with regard to causality and what the system is capable of detecting.
BPM Software
Through literature review and web category searches, 40 vendor solutions have been classified in the
following categories:
1. Balanced scorecard or general business performance measurement systems (BSC/BPM)
2. Business activity monitoring (BAM)
3. Event alerting and management (EM&A)
Table 4 lists the vendors by type:
A) BSC/BPM vendors
1) ABC Technologies, Inc.
2) Accrue Software
3) ActiveStrategy
4) Cognos
5) ComShare
6) CorVu
7) Crystal Decisions
8) Dialog Software
9) Ergometrics
10) FiberFlexBI
11) Gentia
12) Hyperion
13) Host Analytics
14) IC Visions
15) INPHASE
16) Online Development
17) Oracle
18) Panorama Business Views
19) PBViews
20) PeopleSoft
21) PerformanceSoft
22) Pilot Software
23) Procos Professional Controlling Systems AG
24) Prodacapo AB
25) QPR
26) SAP AG
27) SAS Institute
28) Show Business Software Ltd
29) SIMPEL Systems
30) Solvision B.V.
31) Stratsys A
B) BAM Vendors
1) Mentisys
2) Quantive
3) Accenx
4) Presence
5) Praja/TIBCO
6) Catagoric Software
C) EM&A Vendors
1) Adeptra
2) Categoric
3) Objective Edge <neXus/>
Table 4. Software vendors
Software Issues
Obviously BPM systems need data. Several researchers have pointed out the need for timely access to
quality data. Companies implementing BPM systems need to choose the appropriate data movement,
integration and transformation approach.
Data integration and transformation
Data for performance measurement systems can come from a variety of sources: enterprise systems
(including supply chain, demand chain, CRM, and point of sale systems), ad-hoc systems (including
spreadsheets, desktop database applications, word processing documents) or direct data entry into the
BPM software. In addition, data architectures to house the BPM data can vary. Most of the BPM
software, especially the BSC niche vendors such as QPR and Dialog Strategy, has specific data
architectures into which BPM data must be entered or transferred. Enterprise software vendors, like
PeopleSoft, have a specific data warehouse architecture to support enterprise performance data that is, to
some extent, integrated with the transaction systems within the suite. BAM vendors are more likely to
rely on other data architectures for BPM data since their role is to disseminate BPM data, not aggregate it.
In this case, the BPM system will use integration and messaging techniques to present BPM data without
the aid of a pre-built or custom BPM data warehouse. The data architecture approaches are summarized
here:
• Data architecture embedded within the BPM tool. Here the BPM tool contains all the necessary
data tables specifically for its purpose of producing output.
• Built-in BPM data warehouse architecture integrated with enterprise suites. Many ERP solutions,
such as PeopleSoft, have a general-purpose data warehouse architecture that is also used for the
BPM system.
• Custom BPM data warehouse with custom integration to data sources. Companies that build their
own performance measurement systems need to design and build the data tables for the BPM
system and determine the best integration approach that will bring enterprise data into the BPM
data architecture.
• Multiple application and database messaging and integration. The EM&A approach can use
messaging and point-to-point application integration to deploy performance measurement data
without a significant repository for BPM data. For specific process control-type measurement
situations, this is often the model used.
Firms have a variety of approaches available to them to move data through the BPM system:
• Extraction, transformation and loading (ETL) tools that move large amounts of data into data
warehouse, typically in nightly batches
• Message-oriented middleware (MOM) tools that move smaller amounts of data between systems
(and into a data warehouse) in near-real time. MOM tools can provide transactional integrity
benefits as well, which ETL tools do not provide.
• XML and web services tools that move smaller amounts of data, often using MOM tools,
between systems. (The BSC has an XML standard for moving data into a BSC system.)
Moving data between systems nearly always involves transforming the data in some way.
Transformations can occur at many levels and for different reasons:
Computing
representation
Differences between representations of data due to different CPU
architectures, operating systems and programming languages.
Formatting Combining or splitting values (such as first name and last name),
managing different representations of dates, currencies, numbers, logical
values and codes.
Semantic meaning Data may not be directly comparable to each other due to differences in
level of aggregation (monthly versus weekly summaries), differences in
relational representation (how tables link to each other in primary key and
foreign key relationships), differences in meaning that users ascribe to
data (two users can have very different meanings for the term ‘customer’
or different competing coding schemes may be in use for the same
attribute)
Errors and omission Data quality may vary between systems with incorrect data entered or
missing data causing other problems.
BPM systems are suspect to all the normal data management problems found in other information
processing systems. In highly uncertain environments where the organization may be testing and
changing basic assumptions, data quality and trust in the data is critical. Fortunately there are data quality
and data conversion tools that can aid in this aspect.
Data visualization
Tufte (1997) provides a compelling argument for proper data visualization to support decision-making in
his analysis of the documents NASA engineers used to unsuccessfully convince NASA managers to scrub
the doomed Challenger launch in January 1986. On the day of the launch, managers and engineers
discussed canceling the launch due to unusually cold weather. Engineers familiar with the problem had
correctly anticipated the problem – the now infamous O-rings would fail in the cold weather – but did not
make an appropriate visual presentation of the data to management.
Data visualization, while practiced in some form for centuries (Tegarden, 1999), has developed rapidly
since the 1980s, starting out initially as visualization of scientific information (Card et al, 1999).
Technological advances have also contributed to development of data visualization as a field of research
due to the widespread availability of low-cost, high-performance workstations (DeFanti et al, 1999).
Visualization as a field is divided into two categories: visualization of scientific data that tend to be based
on physical data such as the human body, molecules, the earth, and so on, and visualization of abstract
data that are based on financial data, business information and, document collections (Card et al, 1999).
This distinction is important for BPM systems because the key challenge for visualization of abstract data
has to do with representing more than 2 or 3 dimensions of data in an intuitive form. Scientific data is
more easily and more frequently depicted in 2 or 3 dimensions, often corresponding to the dimensions of
space. Abstract data, of which BPM data is an example, is not easily represented in the 2 and 3 spatial
dimensions. There is no physical geography to give the data an intuitive structure (Wright, 1999). Since
no common framework yet exists, most of the literature in business information visualization has attempts
to provide new metaphors and visualization approaches to abstract data rather than coalescing around a
common framework for business information visualization. With this distinction in mind, Card et al
(1999) define information visualization as:
The use of computer supported, interactive, visual representations of abstract data to amplify
cognition.
Cognition here is referred to as the acquisition of use of knowledge. Cognitive amplification is useful for
tasks that are “characterized by use of large amounts of heterogeneous data, ill-structured problem
solving, but a relatively well-defined goal requiring insight into information relative to some purpose,”
(Card et al, 1999). These tasks are called “knowledge crystallization” tasks that tend to follow the
following six steps:
1 Information foraging.
2 Search for schema (representation)
3 Instantiate schema with data. Residue is significant data that do not fit the
schema. To reduce the residue, go to step 2 and improve the schema.
4 Problem solve to trade off features.
5 Search for a new schema that reduces the problem to a simple trade-off.
6 Package the patterns found into some output product.
Each of these steps has a cost (time, effort) associated with performing the step. Visualization can help
reduce the cost for each of these steps.
BPM tasks fit this description of knowledge crystallization. They often deal with large amounts of data.
The data is typically heterogeneous. The problem-solving context can be ill-structured and the goals are
usually well-defined (revenue growth, product quality, etc.). While some portions of BPM are routinized,
especially operational measurement, strategic BPM is not. The data can be collected and presented with a
completely repeatable process but insight into the data cannot. For this reason, strategic BPM needs to be
supported with this kind of iterative process in which meaning is created by refining schemas of analysis.
Card, et al (1999) describe a reference model for visualization that might prove in understanding issues
with providing visual representations of BPM data (Figure 3).
Figure 3. Reference model for visualization (Card et al, 1999).
Raw data is transformed into one or more data tables (relational mappings of raw data). Tables
themselves are iteratively manipulated and then transformed into one or more visual structures (a base
spatial design to which marks of various kinds are added). Users manipulate the visual structure in a view.
This is a highly interactive model for visualization.
Beyond the cost-savings benefit that visualization provides, visualization research over the past two
decades has illuminated many other benefits. Card et al (2000) lists six main benefit types:
1. Increasing the memory and processing resources available to the user
2. Reducing the search for information
3. Using visual representations to enhance the detection of patterns
4. Enabling perceptual inference operations
5. Using perceptual attention mechanisms for monitoring
6. Encoding information in a manipulable medium
Several factors related to each type are listed below (Card et al, 2000):
Parallel perceptual processing Some attributes of visualizations can be processed in parallel
compared to text.
Offload work from cognitive to
perceptual system
Some cognitive inferences done symbolically can be recoded into
inferences done with simple perceptual operations.
Expanded working memory Visualizations can expand the working memory available for
solving a problem.
Expanded storage of
information
Visualizations can be used to store massive amounts of information
in a quickly accessible form (e.g., maps).
Locality of processing Visualizations group information used together, reducing searching.
High data intensity Visualizations can often present a large amount of data in a small
space.
Spatially indexed addressing By grouping data about an object, visualizations can avoid symbolic
labels.
Recognition instead of recall Recognizing information generated by a visualization is easier than
recalling that information by that user.
Abstraction and aggregation Visualizations simplify and organize information, supplying higher
centers with aggregated forms of information through abstraction
and selective omission.
Visual representations make
some problem obvious
Visualizations can support a large number of perceptual inferences
that are extremely easy for humans.
Perceptual monitoring Visualizations can allow for the monitoring of a large number of
potential events if the display is organized so that these stand out by
appearance or motion.
Manipulable medium Unlike static diagrams, visualizations can allow exploration of a
space of parameter values and can amplify user operations.
Since BPM data is a good candidate for visualization, BPM systems, if they support visualization, should
enable user manipulation of data tables, visual mappings and view transformations to support this
reference model. For rote BPM needs, such as operational BPM, user interaction with data
transformations and visual mappings is not likely to be frequent. However, providing these interactive
visualization capabilities may be needed since metrics and measurement systems undergo a continual
change. In addition, manipulation of visual views for operational BPM might prove highly useful since
perceptual abilities can be useful for detecting patterns while monitoring data.
There may also exist a synergistic relationship between ongoing operational monitoring and more
interactive visualization tasks. For those BPM systems that let managers monitor operational activities,
detection of problems often leads to further interactive analysis into the causes and effects of the problem.
Visualization not only can help in detecting operational problems, it can be useful for researching the
problem causes further.
Despite the numerous benefits for information visualization, none of the commercially available BPM
systems offer advanced information visualization capabilities of the kind hinted at here. Part of the
challenge in providing visualization in BPM systems has to do with the structure and dimensionality of
the base data. Since business information is not constraint by the law of physics, representing the data in
an intuitive three-dimensional form is challenging. Which dimensions of the business data should be
mapped to the three vectors of spatial representation? Should dimensions greater than three of business
data be reduced to a three dimensional model? Mapping metaphors can make use of multiple dimensions
but at a cognitive cost. Conventional topographic maps can encode many dimensions of data into a single
two-dimensional display. However, for business data, of the many dimensions, which two dimensions
should be placed on the prominent x/y axes? Can other metaphors, richer in meaning, be useful? Perhaps
to avoid these essentially arbitrary difficulties, to bow before the sustained use and acceptance of simpler
visualizations (bar charts, line charts, scatter plots and the numerous variants), and to reduce the cost of
developing the software, BPM software vendors have not ventured too deep into the visualization waters.
Current BPM Visualization Approaches
To deal with the arbitrariness and multi-dimensionality of business data, visualizations in BPM software
rely on simple marks, charts and in some cases, metaphors. Current metaphors common in visualization
software include.
Marks
Textual manipulation (color, font size and style, blinking)
Symbols (up/down arrow, +/-, empty/half/full circles, a.k.a. “harvey balls”)
Charts
Pie chart (2-D, 3-D)
Bar chart (2-D, 3-D)
Line chart
Scatter plot (2-D, 3-D)
Surface chart
Radar or spider web chart
Visual metaphors
Thermometer (high/low)
Fuel gauge (Empty/full)
Speedometer (fast/slow)
Cause-Effect Influence diagram (network, hierarchy)
Experimental approaches
Map (multiple dimensions coded in marks and colors)
Tree (as a visual metaphor showing time linkages)
Human face (because of our evolutionary enhanced visual apparatus for facial recognition)
Virtual reality
Animation, video
While many academics are researching some of these advanced metaphors and experimental approaches,
none so far have found their way into widely accepted BPM software.
Future directions
Business performance measurement suffers from the weaknesses of the antecedent disciplines of business
strategy, decision analysis, organizational learning and data visualization: relative newness and lack of a
widely accepted foundational theory to describe and predict. Business strategy has multiple disciplines,
each using a different framework to describe it, like the blind men describing an elephant (Mintzberg &
Lampel, 1999). Decision and judgment analysis from the prospect theorists’ perspective, while receiving
plenty of attention from cognitive psychology and economics, also may suffers from inconclusiveness.
Gigerenzer (2000) characterizes prospect theory as an important but a provisional step along an
undermined path. Organizational learning and management has fragmentation of its own to contend with.
Argyris points out the need for an integrative approach to research in management problems (Argyris,
1996). Visualization of abstract data, being quite new as a discipline, has been tied to the neurosciences or
specific task-related outcomes more than it has been tied to decision analysis as a whole and not at all to
the problems that prospect theory seeks to solve. It too seems to evolve without theoretical foundations
and useful linkages to other disciplines. Gigerenzer (2000) discusses the need for cross-discipline
collaboration bluntly: “Intellectual inbreeding can block the flow of positive metaphors from one
discipline to another.” This form of territorial science, says Gigerenzer (2000), creates distrust and
disinterest in anything outside one’s subdiscipline. Is this Argyris’s Model-I theory in action?
The pieces, however, are in place. Business strategy is rapidly evolving to accommodate more complex
and sophisticated views of how firms determine and execute their strategies. Measurement related to
those strategies is bringing multiple pieces of the organizational puzzle together into an integrated
framework, most likely enabled by recent advances in distributed information technology. Prospect theory
proponents and its critics have explored the inadequacies of human cognition in decision and judgment
making. Organizational learning has exposed the difficulties and problems in getting organizations and
individuals to learn. Information visualization researchers have demonstrated how we can utilize human
visual (and kinesthetic) intelligence for better understanding. Business performance measurement, which
needs to address and include the theories of strategy, decision-making, learning and cognition, needs to
grasp these linkages and exploit the synergies that lie in their convergence.
Conclusion
BPM systems capture and disseminate strategic information that matters most to the firm in the form of
strategic process and outcome measurement, and most to the individuals within the firm in the form of
performance measurement, incentives and motivation. Because of this, BPM systems are a primary means
of “knowing” (coordinating what a firm knows and learns) and “doing” (how it alters what it does). Over
time, they may perhaps become the single most strategic information system resource in the
firm. Technological advances in data processing and integration, application deployment via the web and
in analysis and visualization have helped BPM systems advance significantly. However, the best may be
yet to come. While adoption rates of BPM systems are high in larger companies, actual success in larger
firms is more limited and adoption in smaller firms is embryonic. Measurement frameworks such as BSC,
EVA and ABC have found their way in to higher education curricula and into the accounting functions in
those adopting firms. Measurement reference models, which promise to diffuse BPM approaches even
further – albeit at the expense of competitive differentiation, are just beginning to sprout. Key software
manufacturers are aboard the analytical and performance measurement bandwagon. The market appears
big enough to support a rather large number of very different software vendors.
Despite the optimism, much work needs to be done. BPM systems could benefit from a dominant theory
of judgment, decision-making and organizational learning. Economic decision analysis researchers
(prospect theory and its critics) need to connect with data visualization researchers to see what impact
using different mental “machinery” can have on judgments and decision-making. Distributed and social
knowledge gathering and knowledge creation activities, enabled chiefly through distributed information
technology (the Internet) needs to be understood, accepted and adopted by firms to complement to
historical tendencies towards top-down, engineered strategic planning. All of these areas need to connect
with the organizational learning discipline. Right now, BPM seems much more art than science. While
this bodes well for consultants and gurus, it does not for businesses. How to make BPM successful for
businesses is the challenge for the next thirty years.
References
Argyris, Chris (1996, December). Actionable knowledge: Design causality in the service of consequential
theory. Journal of Applied Behavioral Science. Vol 32. No. 4. 390-406
Argyris, Chris. (1999). Why Individuals and Organizations Have Difficulty in Double-Loop learning, in
On Organizational Learning, 2nd
Edition. Blackwell Publishing.
Argyris, Chris. (1997, January). Initiating Change That Perseveres. American Behavioral Scientist. Vol.
40, No. 3. 299-309.
Argyris, Chris, & Schön, Donald, A. (1996). Organizational Learning II, Theory, Method, Practice.
Addison-Wesley Publishing Co.
Austin, Rob & Gittell, Jody Hoffer. (2002). When it should not work but does: Anomalies of high
performance, in Business Performance Measurement: Theory and Practice. Neely, Andrew,
editor. Cambridge University Press.
Beer, Stafford (1966). Decision & Control: The Meaning of Operational Research & Management
Cybernetics. John Wiley & Sons, Ltd.
Berman, Evan. (2002, June). How Useful Is Performance Measurement. Public Performance &
Management Review, Vol. 25 No. 4.
Bititci, Umit, Carrie, Allan & Turner, Trevor. (2002). Integrated performance measurement systems:
Structure and dynamics, in Business Performance Measurement: Theory and Practice. Neely,
Andrew, editor. Cambridge University Press.
Card, Stuart K., Mackinlay, Jock D. & Shniederman, Ben. (1999). Readings in Information Visualization:
Using Vision to Think. Academic Press.
De Haas, Marco & Kleingeld, Ad. (1999). Multilevel design of performance measurement systems:
enhancing strategic dialog throughout the organization. Management Accounting Research. Vol
10, 233-261.
DeFanti, T. A., Brown, M. D. & McCormick, B. H. (1999). Visualization – Expanding Scientific and
Engineering Research Opportunities, in Readings in Information Visualization: Using Vision to
Think. Academic Press.
Eckerson, Wayne W. (2002). Data Quality and The Bottom Line. The Data Warehouse Institute.
Epstein, Marc J. & Westbrook, Robert A. (2001, Spring). Linking actions to profits in strategic decision
making. MIT Sloan Management Review.
Evans, James R. (1997). President’s Letter. Decision Line. Vol. 28(3).
Flint, D. (2002, March) Research Note: BAM: Evaluating Tomorrow’s Management Technology.
Gartner.
Frigo, Mark L. & Krumwiede, Kip R, (2000, January). The balanced scorecard. Strategic Finance. Vol.
81 No. 7. 50-54.
Frigo, Mark L. (2001, November). The State of Strategic Performance Measurement: The IMA 2001
Survey. The Balanced Scorecard Report.
Frigo, Mark L. (2002, May). Strategy, Business Execution and Performance Measures. Strategic Finance.
Vol. 83, No. 11.
Gale, Bradley T. (1994). Managing Customer Value. The Free Press.
Ho, Shih-Jen & McKay, Ruth B. (2002, March). Balanced Scorecard: Two Perspectives. CPA Journal 20-
25 72, No. 3.
Gigerenzer, Gerd. (2000). Adaptive Thinking: Rationality in the Real World. Oxford University Press.
Jensen, Anne J. & Sage, Peter B. (2000). A systems management approach for improvement of
organization performance measurement systems. Information, Knowledge, Systems Management.
2 (2000) 33-61.
Kahneman, Daniel & Lovallo, Dan. (1993, January). Timid Choices and Bold Forecasts: A Cognitive
Perspective on Risk Taking. Management Science. Vol. 39, No. 1.
Kahneman, Daniel & Tversky, Amos. (2000). Choices, Values, and Frames, in Choices, Values, and
Frames. Cambridge University Press.
Kaplan, Robert S. & Lamotte, Gaelle. (2001). The Balanced Scorecard and Quality Programs. Balanced
Scorecard Report. March 15, 2001.
Kaplan, Robert S. & Norton, David P. (2001). The Strategy-Focused Organization. Harvard Business
School Press.
Kaplan, Robert. (2001). Integrating shareholder value and activity-based costing with the Balanced
Scorecard. Balanced Scorecard Report. January 15, 2001.
Kellen, Vince. (2002, March). CRM Measurement Frameworks. http://www.crm-forum.com.
KPMG. (2001). Achieving Measurable Performance Improvement in a Changing World: The Search for
New Insights. KPMG.
Krell, Eric. (2002, June). The State of Corporate Performance Management. BusinessFinanceMag.com.
Lakoff, Mark & Johnson, George. (1999) Philosophy in the Flesh: The Embodied Mind and its
Challenges to Western Thought. Basic Books.
Lebas, Michel & Euske, Ken. (2002). A conceptual and operational delineation of performance, in
Business Performance Measurement: Theory and Practice. Neely, Andrew, editor. Cambridge
University Press.
Lingle, John H. & Schiemann, William A. (1996, March). From Balanced Scorecard to Strategic Gauges:
Is Measurement Worth It? Management Review. Vol. 85(3)
Meyer, Marshall W. (2002). Finding performance: The new discipline of management, in Business
Performance Measurement: Theory and Practice. Neely, Andrew, editor. Cambridge University
Press.
Maisel, Lawrence S. (2001, May – June). Performance Measurement Practices: A Long Way from
Strategy Management. The Balanced Scorecard Report.
Mintzberg, Henry. (1987, July-August). Crafting Strategy. Harvard Business Review.
Mintzberg, Henry & Lampel, Joseph. (1999, Spring). Reflecting on the Strategy Process. Sloan
Management Review.
Nail, Jim. (2002, September). The TechStrategy Report: Mastering Marketing Measurement. Forrester.
Neely, Andy. (2002). Business Performance Measurement: Theory and Practice, Andy Neely editor.
Cambridge University Press.
Nesbitt, Keith, V. (2000). A Classification of multi-sensory metaphors for understanding abstract data in
a virtual environment. The Proceedings of the IEEE International Conference on Information
Visualization (IV’00). IEEE.
Osterloh, Marget & Frey, Bruno S. (2002). Does pay for performance really motivate employees? in
Business Performance Measurement: Theory and Practice. Neely, Andrew, editor. Cambridge
University Press.
Principia Cybernetica (Accesseed November 2002). http://pespmc1.vub.ac.be.
Porter, Michael. (2002, March-April). The Importance of Being Strategic. The Balanced Scorecard
Report.
Potts, Colin. (2001). Metaphors of Intent. Proceedings of the Fifth International Symposium on
Requirements Engineering. (IV’01). IEEE.
Rust, Roland T., Zeithaml, Valarie A. & Lemon, Katherine E. (2000). Driving Customer Equity. The Free
Press.
Simmons, Robert. (2000). Performance Measurement and Control Systems for Implementing Strategy.
Prentice Hall.
Smith, Michael. (2001, November). Fixing the Balanced Scorecard’s Missing Link. GartnerG2.
Stacey, Ralph, D. (2000). The Emergence of Knowledge in Organizations. Emergence, 2(4) 23-39.
Stacey, Ralph, D (2001). Complex Responsive Processes in Organizations: Learning and Knowledge
Creation. Routledge.
Stern Stewart Corporation (2002). What is EVA? http://www.sternstewart.com/evaabout/whatis.shtml.
Accessed Novemeber, 2002.
Tegarden, David P. (1999, January). Business Information Visualization. Communications of the
Association for Information Systems. Vol. 1, Paper 4.
Thomas, J.C., Kellogg, W.A &; Erickson, T. (2001). The Knowledge Management Puzzle: Human
Factors and Knowledge Management. IBM System Journal. Vol. 40 No. 4.
Tufte, Edward R. (1997). Visual Explanations: Images and Quantities, Evidence and Narrative. Graphics
Press.
Van Aken, Eileen M. & Coleman, Gary D. (2002, July/August). Building Better Measurement. Industrial
Management. Vol. 44, No. 4 (28-33).
Wright, William. (1999). Information Animation Applications in the Capital Markets, in Readings in
Information Visualization: Using Vision to Think. Academic Press.
Wruck, Karen Hopper & Jensen, Michael C. (1998). The two key principles behind effective TQM
programs. European Financial Management. Vol. 4. No. 3.