The basic steps to be taken when establishing a set of performance indicators are set out below.
Specify your objectives
What do you want to achieve?
Set your targets
Have you identified a target for each objective?
Identify the required outputs
What goods, service or condition do you need to produce or provide to meet your objective?
What outcome is to be achieved?
What is the desired impact on society?
Consult the intended users
Are users satisfied with the relevance of the information? Remember, users needs will vary.
Consult with other selecting indicators
If an activity or output is part of a chain, check with those selecting indicators for related areas, to ensure that the indicators are consistent.
Is the required information available?
Check that the information required to provide the indicator is available and where necessary can be combined.
Do the benefits of the information outweigh the costs of collection?
Collecting data can be difficult, time-consuming and resource-intensive so time should be given to considering data collection costs and requirements.
3.2 Linking Individual and Organisational Performance Indicators
Middle managers (Principal Officers or Assistant Principal Officers) have a pivotal role in defining performance indicators for the Department and for linking these indicators to the performance of staff and other resources they manage. Their role and responsibility in this regard is to gain the support and participation of staff in planning, implementing and using better performance information in order to achieve the improved performance of the organisation. They should aim to agree staff performance indicators and targets (included in PMDS) which are aligned and congruent with those of their Department. The aim should be to maximise the contribution made by all members of the team to achieve their Department’s goals.
Managers are responsible for managing the performance and development of staff. They may also be involved in helping to define key performance indicators and targets for inclusion in their Department’s Strategy Statements which will be cascaded down, included in Business Plans, and reported in Department’s Annual Reports. To undertake this role effectively, therefore, managers need to be clear about:
Ÿ What the Department is seeking to achieve;
Ÿ Their role in selecting, implementing and using individual performance indicators and targets;
Ÿ Their role as Business Manager in selecting organisational indicators and measures.
3.3 What is the Department seeking to achieve?
To address this question managers need to be clear about:
Ÿ The time horizon - one week? One month? One year? Three years? Three to five years? Ten years?
Ÿ The Inputs, Outputs, Outcomes framework;
Ÿ What is most important to measure?
Ÿ What is cost effective to measure?
3.3.1 Time Horizon
The time horizon will determine whether there is a need to address strategic (Strategy Statement) issues (more than one year) or operational (Business Plan) issues (less than one year) and provide a feel for whether the achievement is in respect of outcomes (impacts upon society) or outputs (amounts of products or services provided) or inputs (the resources used to deliver the outputs and outcomes).
3.3.2 Inputs, Outputs and Outcomes
Frequently these terms are misunderstood. They are defined in Appendix 1. Managers need to understand them in order to organise their management information and reporting requirements logically within their Department’s Management Information Framework. Managers need to establish if their organisation is seeking to achieve improvements in inputs, outputs or outcomes.
3.3.3 What is important to measure and to whom?
Managers need to be sure that their organisation is measuring the right things and be clear about the purpose of the measure, and who is the main stakeholder in the information to be reported.
3.3.4 What is cost effective to measure?
Managers need to be sure that the cost of collecting, monitoring and using the information does not exceed the benefit to the organisation and/or stakeholder. They need to be careful to avoid possible overlap, duplication and/or waste in holding information which is not used.
3.3.5 Some steps to take to reduce the risk of error
Managers need to get involved in discussions on how the Strategy Statements affect their responsibilities, and about the resource implications of achieving goals and objectives set out in their Business Plans. As performance indicators/measures are cascaded down, a manager, at whatever level, will want to check that measures for which they are responsible do not:
Ø Contradict each other;
Ø Conflict with each other;
Ø Undermine priority given to critical measures;
Ø Over-measure - keep measurement to the minimum to tell an accurate story.
A manager will know their Department well and will therefore be well placed to plan for and drive improvements in value-for-money. In order to make the most of their knowledge managers need to:
Ø Know what they are trying to do - driven by the Strategy Statement and Business Plans;
Ø Select a range of financial and non-financial measures - compare to past performance, best known internally and externally;
Ø Report results regularly - promoting knowledge and action;
3.4 Selecting, implementing and using performance indicators and targets
The purpose of performance indicators and measures is to help:
(a) Translate strategy into business plans
(b) Measure progress against strategic goals and objectives;
(c) Plan for, and drive improvements in economy, effectiveness and efficiency;
(d) Assess whether the outputs are, in fact, being delivered
(e) Make better decisions about the allocation and use of resources.
Ideally, start with specifying desired outcomes for public services rather than outputs of Departmental activities. Ask this question:
Is there a clear understanding of how programmes and activities impact on desired outcomes?
Have the main influences on the desired outcome been clearly identified?
Link Departmental activities and outputs to outcomes
Have Departmental activities and performance measures been chosen with regard to:
Ø The main influences on outcomes linked to Strategy Statements and Business Plans
Ø The motivations and perceptions of people involved in delivering and receiving services (Customer Service Action Plans)
Ø The results of relevant research and programme evaluations (NDP/CSF)
Chose the most feasible and cost-beneficial option
Use internal and external evaluations to devise appropriate performance measures. Are they:
Relevant The measure describes either quantity timeliness or cost per unit of specific output or unit of good or service. In some cases location may be a necessary addendum measure. The measure assists Government in deciding how to allocate resources
Robust he measure will facilitate comparison with:
□ similar outputs delivered by other
□ the agency’s own performance
Manageable Data can be collected and reported against the measure at the end of each reporting period. The Department will have the capacity to collect data and report the performance information within agreed time frames.
Complete Do the output measures cover all important aspects of performance so that a full and accurate picture is given of success in delivery of the output? Do the quantity output measures fully describe activities under outputs and are they specified at the highest possible level?
Consistent Where there is more than one output quantity measure are other output measures consistently applied? For example, total quantity costs must add to total accrual cost of the output.
If possible, integrate short-term managerial objectives of performance measurement with the long-term policy based objectives of evaluation.
Does the performance measurement of the Department support the Strategy Statement?
Do measures specify a cross-cutting approach?
Are there overlaps with other Departments?
Can targets be shared across Departments?
Vary targets so that they closely address key issues (some Departments may find it useful to supplement averages with minimum standards or measures of variations).
Have the Department considered the form of the target that most closely supports the objective?
Minimise the potential for unwanted skewing of performance (actively review and reject targets).
Are targets actively reviewed by stakeholders and/or professionals/experts;
Have you guarded against over-emphasis on targeted areas? (Remember what gets measured gets done!)
Use the Strategy Statement and Business Planning process to communicate aspirations and priorities of SMI strategies - the programme for Prosperity and Fairness, Quality Customer Service, and the National Development Plan/Community Support framework.
Are the measures Focused?
Are the measures Appropriate?
Are the measures Balanced?
Are the measures Robust?
Are the measures Integrated?
Are the measures Cost Effective?
When implementing targets Departments should:
w Consult stakeholders - seek their help in establishing targets and measures;
w Get ownership and reward good performance (these can be non-financial incentives)
w Consider the potential value of a Balanced Scorecard approach as part of the Business Planning process;
w Consider carefully the data collection requirements for measuring inputs, outputs and outcomes.
A Checklist for Selecting Good Performance Indicators and Measures
Issue Things to Consider
Has the quality of performance Þ Choose an option which is most feasible
data been considered. and cost-beneficial.
Has specialist advice been sought Þ Have specialists been consulted about
on the design of performance the behavioural and incentive effects of
Who is responsible and accountable Þ The need for a clear understanding of
for performance data used in roles and responsibilities in collecting
strategy statements and business and using performance data.
Does management play an active Þ Does management use its routine
role in ensuring performance data monitoring of performance data to help
is used in managing resources? ensure data quality by, for example,
seeking explanations of variations
between outturn and target?
3.5 The role of the Business Manager in selecting organisational indicators and measures
Business Managers need to gather information about what their Department is seeking to achieve from a variety of sources, including published Strategy Statements, Business Plans and Annual Reports. They need to choose the outcomes, outputs and inputs to be measured, and specify indicators for each of the outcomes, outputs and inputs. They should prepare to collect data on their indicators, and try out their suggestions on colleagues who are insightful about what the organisation is aiming to achieve. Once they are confident that their key indicators are acceptable, and can be set in motion, they should implement the changes.
There are a number of techniques which can help a manager overcome the obstacles of reconciling different information needs, and these are highlighted below.
Managers need to be clear about:
Their understanding of:
- The information requirements of Strategy Statements and Business Plans
- Performance information terminology
- Customer Service
- The limits and constraints of Performance Indicators
- Good practice in selecting Performance Indicators
Their involvement in:
- Identifying useful performance indicators to contribute to improved organisational performance
- Specifying precise performance indicators
- Monitoring and reporting on performance indicators
And they need to provide guidance and reassurance to staff in:
- Implementing these changes
- Providing the opportunities for training where appropriate
- Communicating the benefits (and costs) of improved performance information.
3.6 Managing the Information needs of Stakeholders
Management are the “stewards” of Departmental resources, and are held accountable for them. Balancing the needs of competing stakeholders often requires management to resolve conflicts in order to maximise the use of resources to achieve priorities. Essentially, this can often mean that managers need to look for “Win/Win” solutions which may not always be present. Managers may therefore seek to avoid confronting issues which need to be addressed and brought out into the open, which may constrain performance to sub-optimal limits.
It is therefore essential that managers involved in designing performance indicators and measures should be especially alert to the following lessons:
Ø The need for a clear vision of organisational goals, even though outcomes may be difficult to assess or interpret;
Ø The need to ensure that we as organisations are competent to deliver our strategies - we have the right people to act in various situations - the right skills, education, experience, values, and social skills;
Ø What gets measured gets valued;
Ø What gets measured gets done;
Ø The probability that you may have solved the wrong problem by measuring the wrong thing;
Ø The need to not only know what to do, but actually get on and do it;
Ø The need to integrate financial and non-financial indicators and measures into Business Plans, Strategy Statements and Annual Reports;
Ø The need for all staff to see the benefits of the performance information system, so that it is not perceived as a control mechanism but as a process whereby all staff can play an active part in improving their own, and organisational, performance;
Ø The need to motivate teams beneath us by communicating critical success factors and key performance indicators on a regular basis so that a dialogue about performance becomes a routine way of managing day to day business;
Ø Opportunities to improve learning and dialogue about performance.
3.7 Balance Competing Demands: Helpful Techniques
There are a number of helpful techniques (briefly explained below), which can be used to help balance the competing demands of stakeholders.
Framework for the Strategic Management Process;
The Balanced Scorecard;
Activity Based Costing; and
3.7.1 Framework for the Strategic Management Process
The framework at Appendix 3 is derived from work undertaken by the Department of Finance. It outlines the Strategic Management cycle of the Civil Service. The framework clearly describes the process for providing information on progress with regard to the implementation of strategies and objectives.
3.7.2 Balanced Scorecard
The Committee for Public Management Research (CPMR, Institute of public Administration) Discussion Paper 3 “Developing an Integrated Performance Measurement Framework for the Irish Civil Service” (1997) suggests building upon the Balanced Scorecard approach (The Balanced Scorecard - Measures that Drive Performance by Robert Kaplan and David Norton, 1992). The Balanced Scorecard aims to give a balanced presentation of both financial and operational measures. It aims to allow managers to look at their businesses from four key perspectives, providing answers to four basic questions:
1. How do customers see us? (Customer perspective)
2. What must we excel at? (Internal perspective)
3. Can we continue to improve and create value? (Innovation and learning perspective)
4. How do we look at shareholders? (Financial perspective)
The key requirement of the Balanced Scorecard approach is that, through a structured approach, it links performance measures and goals of each perspective to the strategy for the organisation. It focuses on both measuring the business strategy and managing the business strategy, whilst creating and maintaining a learning environment.
So, how is the Balanced Scorecard technique useful to you? There are two main ways (“Lessons for the Public Sector?” By Richard Tonge, 1996):
On the macro level it recognises that performance measurement in purely financial terms fails to appreciate the various strategic objectives required in order to be successful in the future. An organisation has a range of objectives which will need appropriate recognition within the measurement framework. The public sector has the difficult task of reconciling multiple objectives. The Balanced Scorecard analysis can reflect this fact in the information system.
At the micro level, contractors/providers operating within market or quasi-market situations have to compete against the private sector to provide services. Management and accountants, therefore, need to consider the practical adoption of the latest tools that are believed to improve the process of strategic management.
A good example of the use of the Balanced Scorecard approach can be viewed in The Foundation For Performance Measurement paper “The Role of Performance Measures and the impact of Corporate Structure” (Kevin Bounds, Finance Director, Nationwide Life, UK, March 1998, www.fpm.com/script/UK/Mar98/980301.htm).
The article asks the question if the performance of the Finance function cannot be measured simply by financial measures, can the performance of the entire enterprise be measured in just financial terms? The article then describes the evolution of the financial management process to focus on a corporate model including the Balanced Scorecard, and means of measuring success by reference to Key Performance Indicators, and Strategic Action (programmes costed and prioritised by Activity Based Management techniques).
The article gives an account of how the Balanced Scorecard is used by Nationwide Life, the world’s largest Building Society:
Ø Integrated assessment of performance
Ø Alignment on areas to address
Ø Communicate key areas to all staff
Ø Rewards, appraisals, plans
Ø Education and alignment.
And asks the question is the Balanced Scorecard the Perfect Solution? The simple answer is “no”.
Ø Still requires judgment and balance
Ø Only works if constant dialogue and communication
Ø Targets need to be set
Ø External surveys needed
Ø Mainly addresses tactical decisions.
The main benefit is “alignment”: “If you go in the same direction you are incredibly powerful. It doesn’t matter what direction you are going in as long as you are all going together. You get somewhere and you get there effectively. So I think that is one of the main benefits of the main Balanced Scorecard.”
3.7.3 Costing, Outputs and Performance Measures
art of the role of the new MIF is to provide management with accurate information about the full costs of their activities and performance for which they are responsible, and are held to account. For such information to be useful to management the framework should be designed to enable management to:
Ø Determine the costs of different activities and scenarios
Ø Highlight those costs which drive the organisation
Ø Have a better understanding of which costs are controllable in the short and longer-term, and
Ø Differentiate between low-priority tasks in delivering their personal and organisational objectives.
For Departments it is, therefore, essential that when designing their Management Information Framework that
1. Actual information requirements, particularly about outputs are determined at an early stage, and
2. Opportunities for gathering statistical information are identified where currently there are omissions.
Difficulties likely to be encountered in identifying performance indicators are:
Ø Non-financial performance data may vary considerably between Departments
Ø Policy areas will encounter greater difficulty than non-policy areas, as the latter lend themselves more easily to measurement
Ø Defining outputs and outcomes may be difficult at the outset but that programme evaluations should be helpful in the future development and refinement of such measures over time.
The general approach to costing in the Management Information Framework is as follows:
Ø A standard approach to cost allocation across Departments, facilitating the possibility of cost and performance measurement comparisons (yet to be finalised)
Ø Cost allocation is the process of attributing all relevant costs, both indirect and direct, to outputs so as to determine the full output cost
Ø Costs of compiling information should not outweigh the benefits
Ø Cost centres, organised as necessary, on an individual, programme or project basis, to continue as the basis for cost allocation. (Programme costs could be allocated to a cost centre or a number of cost centres)
Ø Examples of such costs are salaries and wages, including increases in superannuation and leave liabilities; depreciation and amortisation; other direct costs, including materials and services
3.7.4 Activity Based Costing
Activity Based Costing (ABC) is a management accounting idea that appeared in the 1980s. It is associated with work done by Robert Kaplan and David Norton (The Balanced Scorecard - Measures that Drive Performance, 1992). ABC is an approach to the costing and monitoring of activities which involves tracing resource consumption and costing final outputs. Resources are assigned to an activity. This involves tracing resource consumption and costing the final outputs. The final outputs utilise cost drivers to attach activity costs to outputs. This approach is particularly useful when comparing the estimated costs of achieving defined objectives and/or an agreed level of service within a given cost, to the actual and forecast costs at different levels of service. The aim of ABC is to generate improved cost data for use in managing an organisation’s activities.
You may well need to seek the accountancy advice and/or expertise in attributing costs to particular activities and outputs, so that appropriate costs are ascertained for decision making purposes.
For more information about Activity Based Costing and Budgeting see Value-based Management, Chapter 5: Activity-based Costing and Activity-based Management, Edited by R C Scarlett, Chartered Institute of Management Accounting (CIMA), 1997.
The number of businesses that would now claim to be involved in benchmarking is vast (Scott, Management Accounting, July/August, 1996).
Benchmarking is the process of rating an organisation's practices, processes, and products against the best and then emulating them. It involves seeking out best-in-class performers inside or outside the organisation, studying them to determine why they are the best at what they do, and applying what is learned. Rank Xerox defines benchmarking as “finding and implementing best practices”. It is about practices or how things are done at least as much as what level of performance is achieved.
Benchmarking can be very helpful in planning and driving performance improvements. Benchmarking involves the establishment, through data gathering, of targets and comparators, through whose use, relative levels of performance (and particularly areas of under performance) can be identified. By the adoption of identified best practices it is hoped that performance will improve. Benchmarking usually include two categories:
What is to be measured?
Who is to be measured?
Within these two categories are “types “ of benchmarking, including:
Strategic benchmarks measure and compare the relative position of a particular organisation within an industry and are the results of an organisation’s performance at the functional and operational levels. These are benchmarks for the organisation’s core competencies, key business processes, and key success factors.
Internal benchmarks compare an organisation’s own similar processes, products, or services. This is perhaps the easiest form of benchmarking since the potential benchmarking partners can easily be identified and are usually willing to share the information.
Functional benchmarks identify products, services, and work processes from organisations that are not necessarily direct competitors. These are benchmarks at a level blow the strategic benchmarks. Internal functions are compared with those of the best external practitioners of those functions, regardless of the industry they are in (also known as operational benchmarking or generic benchmarking)
Competitive benchmarks identify the products, services, and work processes of an organisation’s direct and strongest competitors in the industry. (The best in the industry are usually easy to identify). The objective is to compare the competitor’s products, processes, and business results with your own. Competitive benchmarking is useful in positioning an organisation's products, services, and processes relative to the marketplace.
For further information on benchmarking see “Value-based Management, Chapter 8: “Benchmarking”, CIMA, 1997.
The Balanced Scorecard, Activity Based Costing and Benchmarking can assist the Strategic Management process by providing Departments with the techniques to produce better performance information linking Strategy Statements, Business Plans and SMI strategies.
The framework for the Strategic Management process is itself a framework for systematically reviewing and updating Strategy Statements and Business Plans.
Committee for Public Management Research, Developing an Integrated Performance Measurement Framework for the Irish Civil Service, Discussion Paper 3, 1997 http://www.irlgov.ie/cpmr/papers/paper3.doc
CSF Performance Indicators: Proposals fro 2000-2006 Programming Period http://www.eustructuralfunds.ie/htm/publications/evaluation/csfindicators.doc
Delivering Better Government: Second Report to Government of the Co-ordinating Group of Secretaries - A Programme of Change for the Irish Civil Service, May 1996 http://www.bettergov.ie/files/dbg.htm
European Commission, Evaluating Socio-Economic Programmes: selection and use of indicators for monitoring and evaluation, Means Collection, Volume 2, 1999
Gowers, Sir Ernest, The Complete Plain Words, HMSO, 1996
HM Treasury, Cabinet Office, National Audit Office, Audit Commission & Office for National Statistics, Choosing the right FABRIC: A Framework for Performance Information, March 2001 http://www.hm_treasury.gov.uk/performance_info/index.html
Institute of Public Administration, Guidelines for the Preparation of Strategy Statements under the Public Service Management Act, 1997, 2000 http://www.bettergov.ie/files/strategy.htm
Integrating performance Management in the Irish Civil Service, Trinity College, Public Sector Class 1997/98
Kaplan, Robert S & Norton, David P, The Balanced Scorecard, Harvard Business School Press, Boston, 1996 (ISBN: 0875846513)
Kaplan, Robert S & Norton, David P, The Balanced Scorecard - Measures that Drive Performance, Harvard Business Review, 1998 (ISBN: B00005BK30)
Management Accounting Official Terminology, 2000, CIMA, (ISBN 185971 3475)
Performance Management and Development System, General Council Report 1368 http://www.bettergov.ie/taoiseach/publication/smi/performance/keypapers.htm
Practical Guide for the Development of Quality Customer Service, December 2000 http://www.bettergov.ie/files/CAP.htm
Programme for Prosperity and Fairness, February 2000 http://www.irlgov.ie/taoiseach/publication/partnership/default.htm
Report of the SMI Working Group on Financial Management to the SMI Implementation Group, July 1999 http://www.bettergov.ie/finance2.htm
SMI Implementation Group 3rd Report, March 2001 http://www.bettergov.ie/files/3rd Report.htm
The Role of Performance Measures and the impact of Corporate Structure, Kevin Bounds, Finance Direct, Nationwide Life, UK, March 1998 http://www.fpm.com/script/UK/Mar98/980301.htm
The Users Role in Systems Development (CCTA) (1989)
Treasury Department, Western Australia, Purchasing and Analysing Outputs June 2000 (ISBN 07307 44973)
UK National Audit Office, Good practice in Performance Reporting in Executive Agencies, March 2000 http://www.nao.gov.uk/publications/nao_reports/9900272.pdf
Value Based Management (1997) CIMA, (ISBN1874784752)
Appendix 1: Some Key Definitions
Performance information is intended to describe how inputs are translated into outputs and how outputs are translated into outcomes. The meanings of these and related terms in the context of the Management Information Framework are:
The resources that contribute to production and delivery. Inputs commonly include things such as labour, physical assets, and IT systems.
Outputs are the final goods and/or services produced as a result of pursuing the specified objectives and strategies for delivery to the customer in order to bring about the outcomes desired by government. They may be either generic outputs of an office or section, or specific outputs arising out of objectives and strategies.
Intermediate outputs are goods or services of one part of an agency, delivered to another part of the same agency or steps along the way in the internal production process, that directly contribute to the delivery of an external output.
Outcomes are the impact, or consequences for the community, benefits or changes for individuals or groups arising from Departmental outputs. They are the effects that government wants to achieve through its policies.
Outcome measures address whether or not the service is meeting the policy goals. Outcomes reflect the actual results achieved and the impact. Both intermediate and long- term outcomes can be evaluated.
Intermediate outcomes can be useful in assessing early results when the primary goals will not be realised for several years.
Key Performance Indicators
High level or strategic indicators - key performance indicators - are used for external reporting, for example, in the Strategy Statement.
Operational indicators are used for internal management purposes - for example, the indicators included in the work plans for each Assistant Secretary area.
The enhanced management information systems should be capable of capturing information to enable value-for-money in delivering projects or programmes to be assessed. Value-for-money is a combination of economy, effectiveness and efficiency. These terms mean:
The acquisition of resources of appropriate quantity and quality at minimum cost.
The utilisation of resources such that Departmental outputs achieve the
The achievement of either
(i) Maximum useful output from the resources devoted to an activity,
(ii) The required output from the minimum resource input.
Part 3 of this document
|Users who read this document also viewed|
|06 January 2002 The following is a discussion draft entitled 'Central Guidance: Performance Indicators and Measures' which has been...|
|06 January 2002Discussion Draft - Performance indicators - Part 3|
|06 January 2002 This is a short factual guide to Performance Indicators. It suggests what to do before selecting indicators and...|
|01 June 2003Welcome from the Minister|
|04 August 2003 An A-Z index of all the Document Categories within the Department of Finance's website. Each link provides a list of...|