MOOC: Auditing waste management

4.2. Audit design matrix

The audit design matrix (ADM) is a very good tool for systemising the auditing process as a whole. There are many ways of creating one, but its main elements – the audit questions, the ideal situation (criteria) and how the team assesses the situation (methodology) – should be covered. Read more about the ADM from the MOOC ‘Introduction to Environmental Auditing in the Public Sector’.

Below is one practical example of an ADM from the already familiar NAO Estonia municipal waste audit (see 4.1) and also two additional, theoretical example matrices illustrating potential audits connected to package waste recovery and waste prevention.

Audit title: Activity of the State and Local Governments in the Collection and Recovery of Municipal Waste
Audit objective: To determine whether cooperation between the state and local governments guarantees that the municipal waste is collected, whether separate waste collecting is promoted and whether the target quantity of waste is recycled.
Audit question 1: Have government actions related to waste prevention, collection and recycling been successful?
*Sub-questions*	*Criteria*	Methods	*Expected findings*
1. Has the government set clear goals and rules and given financial support that would ensure the reduction an recycling of municipal solid waste?	Rules have been set, are understandable to all parties, and an action plan for waste reduction and recycling has been adopted. The regulation supports waste prevention and recycling. The government has implemented waste reduction and recycling subsidy schemes. Whilst updating waste management rules, impact of these on municipal waste management in local governments has been analysed and changes made were justified.	Documentary analysis Analysis of taxing principles, costs of services, subsidies, international practices Interviews	The government has not done much to help reduce waste generation and the state will not not meet the EU recycling target by 2020. Recycling of less valuable waste types (household organic waste) needs to be promoted, for example via creating a demand for separatelt collected organic waste on the biogas market.
2. Have the roles of the state and local governments been set clearly and these are understood to enable achieving strategic municipal waste management goals?	Tasks are clearly defined in legislation. Tasks intrinsic to the central government are financed by the central government.	Documentary analysis Analysis of relevant court cases Poll among local governments Interviews	The division of tasks is in many aspects not clear. Local governments lack freedom to make strategic waste management decisions and financial resources for organising waste management.
3. Has the infrastructure built for sorting (for public use) and recycling (end users) waste met its goals?	Waste infrastructure (waste stations and recycling plants): – have been built according to strategic documentation and the effectiveness of their operations has been assessed; – have received the planned amount of waste; – are open for public use; – are working efficiently; – are used for their intended purpose; – are accessible to the public (the location of the waste station is at least 10-20 km from every household). If not, the collection of sorted waste is otherwise organised.	Documentary analysis Analysis of waste data GIS analysis Interviews Poll among citizens	Investments have been made without considering the existing capacities and the amount of waste generated. Consequently, many objects have overcapacity.
4. Does the packaging waste collection system help to improve waste sorting?	The proportion of packaging waste in the total volume of municipal waste is decreasing. Observable improvements in the collection and reuse of packaging waste have taken place during the last 5 years. Packaging waste containers are located according to the Package Law (accessible to people).	Analysis of waste data Documentary analysis Review of NAOE’s earlier recommendations GIS Analysis

Theoretical example of a packaging waste audit matrix

Audit title: Packaging waste recovery
Audit objective: To evaluate the effectiveness of packaging waste collection and recovery
Audit question: Do the activities of the state support the separate collection and recovery of packaging waste?
*Sub-questions*	*Criteria*	Methods	*Expected findings*
1. Are waste generators motivated to separately collect and dispose of packaging waste?	Disposal of separately collected packaging waste is easy and accessible. Separate collection and disposal of packaging waste is economically more viable for waste generators compared to mixed waste disposal.	Analysis of legal acts and documents; Interviews; Public poll; Review and comparison of the prices for packaging waste and mixed waste disposal	Packaging waste collection network does not have a sufficient coverage and collection points are not easily accessible to people. Prices for mixed waste disposal do not motivate separate packaging waste collection.
2. Has the state ensured sustainable operations of producer responsibility organisations (PROs)?	PROs are properly accredited and controlled to ensure the maintenance of packaging waste collection points. The revenue of PROs is used for developing the network of collection points. The state is able to ensure the continuance of activities of PROs in case of their dissolving.	Analysis of legal acts and PRO accreditation documents; Analysis of annual reports of PROs; Review of monitoring documents; Interviews	Accreditation and control systems do not have significant shortcomings.
3. Does the state have reliable information on the amount of packaging waste generated and recovered?	All products that enter the market are registered. State has an overview of the amount of packaging waste collected. Recovery of packaging waste is being systematically controlled.	Review and analysis of data in the packaging waste register and national statistics; Review of activity reports of PROs, monitoring documents; Interviews	Data on collected and recovered packaging waste are not complete due to lack of systemic control over data compilation process in PROs.
4. Will the state be able to achieve the target packaging waste recovery indicators?	X% of the total amount of packaging waste is recovered annually (national target). PROs not fulfilling their obligations are subject to packaging excise duty or fines.	Review of reports and data of PROs and relevant authorities, comparison with packaging waste register; Interviews	While the recovery rates have increased, the state has not been able to achieve the national recovery target. Supervisory system of PROs has not guaranteed that all PROs failing to meet target recovery rates pay the excise duty or fines.

Theoretical example of a waste prevention audit matrix

Audit title: Prevention of waste
Audit objective: To evaluate the effectiveness of government activities for ensuring waste prevention
Audit question 1: Has the government established requirements for encouraging waste prevention, reuse and recycling?
*Sub-questions*	*Criteria*	Methods	*Expected findings*
1. Does the government have a strategy for promoting waste prevention?	Goals and principles of the national waste prevention strategy/other strategic documents help to achieve the goals of SDG 12 (or e.g. European Union Circular Economy Package)	Analysis of strategies and programmes; Interviews in responsible authorities	Government is aware of the SDGs (SDG 12)/EU Circular Economy Package but has not developed a national strategy for achieving waste prevention.
2. Has the government enforced legal acts to encourage waste prevention?	Goals and principles of the national waste prevention strategy are reflected in national laws	Analysis of legal acts	Some of the strategic goals and principles are presented in laws, but not all. Responsibilities of institutions are not clearly set.
3. Have government or municipalities introduced economic incentives for encouraging waste prevention?	Subsidies exist for waste prevention, reuse or recycling. Other support mechanisms for waste prevention, reuse or recycling have been introduced	Documentary analysis Interviews	There are some support programmes in some municipalities.
Audit question 2: Do waste data give evidence of the reduction of waste?
4. Has total waste generation decreased? If not, then what are the reasons.	Generation of waste has decreased by x%* Generation of municipal waste has decreased by Y % Landfilling has decreased by Z%	Analysis of waste data; Interviews with relevant authorities, local governments	Waste data is difficult to obtain Total waste generation is increasing Municipal waste generation is stable Landfilling has not decreased
5. Has the reuse of waste increased? If not, then what are the reasons.	Reuse of waste has increased by M%	Analysis of waste data Interviews with relevant authorities, local governments Public poll	There are no data on waste reuse
6. Has waste recycling increased? If not, then what are the reasons.	Waste recycling has increased by N%	Analysis of waste data Interviews with relevant authorities, local governments, waste operators Public poll Survey of waste operators	Recycling has not increased by N% Separate waste collection system is not convenient for the citizens Waste operators have little economic incentive to organize separate waste collection

* – values for % in criteria can be derived from laws, strategies or agreed with auditee.

THINKING EXERCISE

Does your SAI use an Audit Design Matrix? What elements does it include? Revisit practical examples of ADMs below from cases presented earlier in the course to find similarities and differences.

Examples of Audit Design Matrices

Estonia – Activities of the state and local governments in the collection and recovery of municipal waste (full)

Latvia – Municipal Waste Management Compliance with Planned Objectives and Legal Requirements

Lithuania – Application of the producer responsibility principle

Lithuania – Hazardous waste management

Slovakia – Effectiveness and efficiency of separate collection of municipal waste