The basic steps to be taken when establishing a set of performance indicators are set out below.

3.2              Linking Individual and Organisational Performance Indicators  

Middle managers (Principal Officers or Assistant Principal Officers) have a pivotal role in defining performance indicators for the Department and for linking these indicators to the performance of staff and other resources they manage.  Their role and responsibility in this regard is to gain the support and participation of staff in planning, implementing and using better performance information in order to achieve the improved performance of the organisation.  They should aim to agree staff performance indicators and targets (included in PMDS) which are aligned and congruent with those of their Department.   The aim should be to maximise the contribution made by all members of the team to achieve their Department’s goals.  

Managers are responsible for managing the performance and development of staff.  They may also be involved in helping to define key performance indicators and targets for inclusion in their Department’s Strategy Statements which will be cascaded down, included in Business Plans, and reported in Department’s Annual Reports.  To undertake this role effectively, therefore, managers need to be clear about:

Ÿ         What the Department is seeking to achieve;

Ÿ         Their role in selecting, implementing and using individual performance indicators and targets;

Ÿ         Their role as Business Manager in selecting organisational indicators and measures.  

3.3     What is the Department seeking to achieve?  

To address this question managers need to be clear about:

Ÿ         The time horizon - one week?  One month?  One year?  Three years?  Three to five years?  Ten years?

Ÿ         The Inputs, Outputs, Outcomes framework;

Ÿ         What is most important to measure?

Ÿ         What is cost effective to measure?      

3.3.1       Time Horizon

The time horizon will determine whether there is a need to address strategic (Strategy Statement) issues (more than one year) or operational (Business Plan) issues (less than one year) and provide a feel for whether the achievement is in respect of outcomes (impacts upon society) or outputs (amounts of products or services provided) or inputs (the resources used to deliver the outputs and outcomes).

3.3.2       Inputs, Outputs and Outcomes 

Frequently these terms are misunderstood.  They are defined in Appendix 1.  Managers need to understand them in order to organise their management information and reporting requirements logically within their Department’s Management Information Framework.  Managers need to establish if their organisation is seeking to achieve improvements in inputs, outputs or outcomes.

3.3.3       What is important to measure and to whom?    

Managers need to be sure that their organisation is measuring the right things and be clear about the purpose of the measure, and who is the main stakeholder in the information to be reported.     

3.3.4       What is cost effective to measure?

Managers need to be sure that the cost of collecting, monitoring and using the information does not exceed the benefit to the organisation and/or stakeholder.  They need to be careful to avoid possible overlap, duplication and/or waste in holding information which is not used.        

         3.3.5                   Some steps to take to reduce the risk of error

Managers need to get involved in discussions on how the Strategy Statements affect their responsibilities, and about the resource implications of achieving goals and objectives set out in their Business Plans.  As performance indicators/measures are cascaded down, a manager, at whatever level, will want to check that measures for which they are responsible do not:  

Ø       Contradict each other;

Ø       Conflict with each other;

Ø       Undermine priority given to critical measures;

Ø       Over-measure - keep measurement to the minimum to tell an accurate story.

A manager will know their Department well and will therefore be well placed to plan for and drive improvements in value-for-money.   In order to make the most of their knowledge managers need to:  

Ø       Know what they are trying to do - driven by the Strategy Statement and Business Plans;

Ø       Select a range of financial and non-financial measures - compare to past performance, best known internally and externally;

Ø       Report results regularly - promoting knowledge and action;           

3.4     Selecting, implementing and using performance indicators and targets

The purpose of performance indicators and measures is to help:

      (a)      Translate strategy into business plans

      (b)      Measure progress against strategic goals and objectives;

     (c)      Plan for, and drive improvements in economy, effectiveness and                     efficiency;

      (d)      Assess whether the outputs are, in fact, being delivered

      (e)      Make better decisions about the allocation and use of resources.         

Ideally, start with specifying desired outcomes for public services rather than outputs of Departmental activities.  Ask this question:

Is there a clear understanding of how programmes and activities impact on desired outcomes?

Have the main influences on the desired outcome been clearly identified? 

Link Departmental activities and outputs to outcomes

Have Departmental activities and performance measures been chosen with regard to:

Ø       The main influences on outcomes linked to Strategy Statements and Business Plans

Ø       The motivations and perceptions of people involved in delivering and receiving services (Customer Service Action Plans)

Ø       The results of relevant research and programme evaluations (NDP/CSF)  

Chose the most feasible and cost-beneficial option  

Use internal and external evaluations to devise appropriate performance measures.  Are they:  

Relevant                The measure describes either quantity timeliness or cost per unit of specific output or unit of good or service.  In some cases location may be a necessary addendum measure.  The measure assists Government in deciding how to allocate resources    

Robust                    he measure will facilitate comparison with:

                           □ similar outputs delivered by other

                               providers, and/or   

                            □ the agency’s own performance

                               overtime     

Manageable           Data can be collected and reported against the    measure at the end of each reporting period.  The Department will have the capacity to collect data and report the performance information within   agreed time frames.  

Complete               Do the output measures cover all important aspects of performance so that a full and accurate picture is given of success in delivery of the output?  Do the quantity output measures fully describe activities under outputs and are they specified  at the highest  possible level?  

Consistent             Where there is more than one output quantity measure are other output measures consistently applied?  For example, total quantity costs must add to total accrual cost of the output.  

If possible, integrate short-term managerial objectives of performance measurement with the long-term policy based objectives of evaluation.

Does the performance measurement of the Department support the Strategy Statement?

Do measures specify a cross-cutting approach?

Are there overlaps with other Departments?

Can targets be shared across Departments?  

Vary targets so that they closely address key issues (some Departments may find it useful to supplement averages with minimum standards or measures of variations).

Have the Department considered the form of the target that most closely supports the objective?

Minimise the potential for unwanted skewing of performance (actively review and reject targets).

Are targets actively reviewed by stakeholders and/or professionals/experts;

Have you guarded against over-emphasis on targeted areas?   (Remember what gets measured gets done!)

Use the Strategy Statement and Business Planning process to communicate aspirations and priorities of SMI strategies - the programme for Prosperity and Fairness, Quality Customer Service, and the National Development Plan/Community Support framework. 

Are the measures Focused?

Are the measures Appropriate?

Are the measures Balanced?

Are the measures Robust?

Are the measures Integrated?

Are the measures Cost Effective?[1]    

  A Checklist for Selecting Good Performance Indicators and Measures

                   Issue                               Things to Consider

Has the quality of performance        Þ      Choose an option which is most feasible

data been considered.                            and cost-beneficial.  

Has specialist advice been sought    Þ      Have specialists been consulted about

on the design of performance                   the behavioural and incentive effects of

indicators?                                           Measures?  

Who is responsible and accountable   Þ       The need for a clear understanding of

for performance data used in                    roles and responsibilities in collecting

strategy statements and business            and using performance data.

plans?

Does management play an active      Þ     Does management use its routine

role in ensuring performance data          monitoring of performance data to help

is used in managing resources?                 ensure data quality by, for example,

                                                          seeking explanations of variations

                                                          between outturn and target?

3.5     The role of the Business Manager in selecting organisational    indicators         and measures 

Business Managers need to gather information about what their Department is seeking to achieve from a variety of sources, including published Strategy Statements, Business Plans and Annual Reports.  They need to choose the outcomes, outputs and inputs to be measured, and specify indicators for each of the outcomes, outputs and inputs.  They should prepare to collect data on their indicators, and try out their suggestions on colleagues who are insightful about what the organisation is aiming to achieve.  Once they are confident that their key indicators are acceptable, and can be set in motion, they should implement the changes.

There are a number of techniques which can help a manager overcome the obstacles of reconciling different information needs, and these are highlighted below. 

Managers need to be clear about:

3.6  Managing the Information needs of Stakeholders

Management are the “stewards” of Departmental resources, and are held accountable for them.  Balancing the needs of competing stakeholders often requires management to resolve conflicts in order to maximise the use of resources to achieve priorities.  Essentially, this can often mean that managers need to look for “Win/Win” solutions which may not always be present.  Managers may therefore seek to avoid confronting issues which need to be addressed and brought out into the open, which may constrain performance to sub-optimal limits.

3.7   Balance Competing Demands:  Helpful Techniques  

There are a number of helpful techniques (briefly explained below), which can be used to help balance the competing demands of stakeholders. 

Framework for the Strategic Management Process;

The Balanced Scorecard;

Activity Based Costing; and

Benchmarking.

       3.7.1  Framework for the Strategic Management Process   

The framework at Appendix 3 is derived from work undertaken by the Department of Finance.  It outlines the Strategic Management cycle of the Civil Service.  The framework clearly describes the process for providing information on progress with regard to the implementation of strategies and objectives.

        3.7.2     Balanced Scorecard

The Committee for Public Management Research (CPMR, Institute of public Administration) Discussion Paper 3 “Developing an Integrated Performance Measurement Framework for the Irish Civil Service” (1997) suggests building upon the Balanced Scorecard approach (The Balanced Scorecard - Measures that Drive Performance by Robert Kaplan and David Norton, 1992).  The Balanced Scorecard aims to give a balanced presentation of both financial and operational measures.  It aims to allow managers to look at their businesses from four key perspectives, providing answers to four basic questions:  

1.       How do customers see us? (Customer perspective)

2.     What must we excel at? (Internal perspective)

3.     Can we continue to improve and create value? (Innovation and learning perspective)

4.     How do we look at shareholders? (Financial perspective)

The key requirement of the Balanced Scorecard approach is that, through a structured approach, it links performance measures and goals of each perspective to the strategy for the organisation.  It focuses on both measuring the business strategy and managing the business strategy, whilst creating and maintaining a learning environment.  

So, how is the Balanced Scorecard technique useful to you?  There are two main ways (“Lessons for the Public Sector?” By Richard Tonge, 1996):  

On the macro level it recognises that performance measurement in purely financial terms fails to appreciate the various strategic objectives required in order to be successful in the future.  An organisation has a range of objectives which will need appropriate recognition within the measurement framework.  The public sector has the difficult task of reconciling multiple objectives.  The Balanced Scorecard analysis can reflect this fact in the information system.  

At the micro level, contractors/providers operating within market or quasi-market situations have to compete against the private sector to provide services.  Management and accountants, therefore, need to consider the practical adoption of the latest tools that are believed to improve the process of strategic management.      

A good example of the use of the Balanced Scorecard approach can be viewed in The Foundation For Performance Measurement paper “The Role of Performance Measures and the impact of Corporate Structure” (Kevin Bounds, Finance Director, Nationwide Life, UK, March 1998, www.fpm.com/script/UK/Mar98/980301.htm).  

The article asks the question if the performance of the Finance function cannot be measured simply by financial measures, can the performance of the entire enterprise be measured in just financial terms?  The article then describes the evolution of the financial management process to focus on a corporate model including the Balanced Scorecard, and means of measuring success by reference to Key Performance Indicators, and Strategic Action (programmes costed and prioritised by Activity Based Management techniques).  

The article gives an account of how the Balanced Scorecard is used by Nationwide Life, the world’s largest Building Society:  

Ø       Integrated assessment of performance

Ø       Alignment on areas to address

Ø       Communicate key areas to all staff

Ø       Rewards, appraisals, plans

Ø       Education and alignment.

And asks the question is the Balanced Scorecard the Perfect Solution?  The simple answer is “no”.

Ø       Still requires judgment and balance

Ø       Only works if constant dialogue and communication

Ø       Targets need to be set

Ø       External surveys needed

Ø       Mainly addresses tactical decisions.

  3.7.3                   Costing, Outputs and Performance Measures  

art of the role of the new MIF is to provide management with accurate information about the full costs of their activities and performance for which they are responsible, and are held to account.  For such information to be useful to management the framework should be designed to enable management to:

Ø       Determine the costs of different activities and scenarios 

Ø       Highlight those costs which drive the organisation  

Ø       Have a better understanding of which costs are controllable in the short and longer-term, and  

Ø       Differentiate between low-priority tasks in delivering their personal and organisational objectives.

  3.7.4 Activity Based Costing

Activity Based Costing (ABC) is a management accounting idea that appeared in the 1980s. It is associated with work done by Robert Kaplan and David Norton (The Balanced Scorecard - Measures that Drive Performance, 1992).  ABC is an approach to the costing and monitoring of activities which involves tracing resource consumption and costing final outputs.  Resources are assigned to an activity.  This involves tracing resource consumption and costing the final outputs.  The final outputs utilise cost drivers to attach activity costs to outputs.  This approach is particularly useful when comparing the estimated costs of achieving defined objectives and/or an agreed level of service within a given cost, to the actual and forecast costs at different levels of service.  The aim of ABC is to generate improved cost data for use in managing an organisation’s activities.

You may well need to seek the accountancy advice and/or expertise in attributing costs to particular activities and outputs, so that appropriate costs are ascertained for decision making purposes.  

For more information about Activity Based Costing and Budgeting see Value-based Management, Chapter 5:  Activity-based Costing and Activity-based Management, Edited by R C Scarlett, Chartered Institute of Management Accounting (CIMA), 1997.

3.7.5  Benchmarking

The number of businesses that would now claim to be involved in benchmarking is vast (Scott, Management Accounting, July/August, 1996).

Benchmarking is the process of rating an organisation's practices, processes, and products against the best and then emulating them.  It involves seeking out best-in-class performers inside or outside the organisation, studying them to determine why they are the best at what they do, and applying what is learned.   Rank Xerox defines benchmarking as “finding and implementing best practices”.  It is about practices or how things are done at least as much as what level of performance is achieved.

Benchmarking can be very helpful in planning and driving performance improvements.   Benchmarking involves the establishment, through data gathering, of targets and comparators, through whose use, relative levels of performance (and particularly areas of under performance) can be identified.  By the adoption of identified best practices it is hoped that performance will improve.   Benchmarking usually include two categories:

                   What is to be measured?

Who is to be measured?  

Within these two categories are “types “ of benchmarking, including:

Strategic benchmarks measure and compare the relative position of a particular organisation within an industry and are the results of an organisation’s performance at the functional and operational levels.  These are benchmarks for the organisation’s core competencies, key business processes, and key success factors.

Internal benchmarks compare an organisation’s own similar processes, products, or services.  This is perhaps the easiest form of benchmarking since the potential benchmarking partners can easily be identified and are usually willing to share the information.

Functional benchmarks identify products, services, and work processes from organisations that are not necessarily direct competitors.  These are benchmarks at a level blow the strategic benchmarks.  Internal functions are compared with those of the best external practitioners of those functions, regardless of the industry they are in (also known as operational benchmarking or generic benchmarking)

Competitive benchmarks identify the products, services, and work processes of an organisation’s direct and strongest competitors in the industry. (The best in the industry are usually easy to identify).  The objective is to compare the competitor’s products, processes, and business results with your own.  Competitive benchmarking is useful in positioning an organisation's products, services, and processes relative to the marketplace.   

For further information on benchmarking see “Value-based Management, Chapter 8: “Benchmarking”, CIMA, 1997.

The Balanced Scorecard, Activity Based Costing and Benchmarking can assist the Strategic Management process by providing Departments with the techniques to produce better performance information linking Strategy Statements, Business Plans and SMI strategies.

The framework for the Strategic Management process is itself a framework for systematically reviewing and updating Strategy Statements and Business Plans.  

References  

Committee for Public Management Research, Developing an Integrated Performance Measurement Framework for the Irish Civil Service, Discussion Paper 3, 1997 http://www.irlgov.ie/cpmr/papers/paper3.doc

CSF Performance Indicators: Proposals fro 2000-2006 Programming Period http://www.eustructuralfunds.ie/htm/publications/evaluation/csfindicators.doc

Delivering Better Government: Second Report to Government of the Co-ordinating Group of Secretaries - A Programme of Change for the Irish Civil Service, May 1996 http://www.bettergov.ie/files/dbg.htm

European Commission, Evaluating Socio-Economic Programmes: selection and use of indicators for monitoring and evaluation, Means Collection, Volume 2, 1999

Gowers, Sir Ernest, The Complete Plain Words, HMSO, 1996

HM Treasury, Cabinet Office, National Audit Office, Audit Commission & Office for National Statistics, Choosing the right FABRIC: A Framework for Performance Information, March 2001 http://www.hm_treasury.gov.uk/performance_info/index.html

Institute of Public Administration, Guidelines for the Preparation of Strategy Statements under the Public Service Management Act, 1997, 2000 http://www.bettergov.ie/files/strategy.htm

Integrating performance Management in the Irish Civil Service, Trinity College, Public Sector Class 1997/98

Kaplan, Robert S & Norton, David P, The Balanced Scorecard, Harvard Business School Press, Boston, 1996 (ISBN: 0875846513)

Kaplan, Robert S & Norton, David P, The Balanced Scorecard - Measures that Drive Performance, Harvard Business Review, 1998 (ISBN: B00005BK30)

Management Accounting Official Terminology, 2000, CIMA, (ISBN 185971 3475)

Performance Management and Development System, General Council Report 1368 http://www.bettergov.ie/taoiseach/publication/smi/performance/keypapers.htm

Practical Guide for the Development of Quality Customer Service, December 2000 http://www.bettergov.ie/files/CAP.htm

Programme for Prosperity and Fairness, February 2000 http://www.irlgov.ie/taoiseach/publication/partnership/default.htm

Report of the SMI Working Group on Financial Management to the SMI Implementation Group, July 1999 http://www.bettergov.ie/finance2.htm

SMI Implementation Group 3^rd Report, March 2001 http://www.bettergov.ie/files/3rd Report.htm

The Role of Performance Measures and the impact of Corporate Structure, Kevin Bounds, Finance Direct, Nationwide Life, UK, March 1998 http://www.fpm.com/script/UK/Mar98/980301.htm

The Users Role in Systems Development (CCTA) (1989)  

Treasury Department, Western Australia, Purchasing and Analysing Outputs June 2000 (ISBN 07307 44973)  

UK National Audit Office, Good practice in Performance Reporting in Executive Agencies, March 2000 http://www.nao.gov.uk/publications/nao_reports/9900272.pdf

Value Based Management (1997) CIMA, (ISBN1874784752)    

             Appendix 1: Some Key Definitions

Performance information is intended to describe how inputs are translated into outputs and how outputs are translated into outcomes.  The meanings of these and related terms in the context of the Management Information Framework are:

Inputs

The resources that contribute to production and delivery.  Inputs commonly include things such as labour, physical assets, and IT systems.   

Outputs

Outputs are the final goods and/or services produced as a result of pursuing the specified objectives and strategies for delivery to the customer in order to bring about the outcomes desired by government.  They may be either generic outputs of an office or section, or specific outputs arising out of objectives and strategies.

Intermediate Outputs

Intermediate outputs are goods or services of one part of an agency, delivered to another part of the same agency or steps along the way in the internal production process, that directly contribute to the delivery of an external output.

Outcome

Outcomes are the impact, or consequences for the community, benefits or changes for individuals or groups arising from Departmental outputs. They are the effects that government wants to achieve through its policies.

Outcome Measure

Outcome measures address whether or not the service is meeting the policy goals.  Outcomes reflect the actual results achieved and the impact.  Both intermediate and long- term outcomes can be evaluated.

Intermediate Outcomes

Intermediate outcomes can be useful in assessing early results when the primary goals will not be realised for several years.

Key Performance Indicators

High level or strategic indicators - key performance indicators - are used for external reporting, for example, in the Strategy Statement.

Operational Indicators

Operational indicators are used for internal management purposes - for example, the indicators included in the work plans for each Assistant Secretary area.    

The enhanced management information systems should be capable of capturing information to enable value-for-money in delivering projects or programmes to be assessed.   Value-for-money is a combination of economy, effectiveness and efficiency.  These terms mean:

 Economy 

The acquisition of resources of appropriate quantity and quality at minimum cost.

 Effectiveness

  The utilisation of resources such that Departmental outputs achieve the 

  policy goals.

 Efficiency

  The achievement of either

(i)                  Maximum useful output from the resources devoted to an activity,

or

 (ii)      The required output from the minimum resource input.  

Part 3 of this document