4.3. Audit methods

Methodology is selected for ensuring a systematic approach to answering the audit question. The methods applied in an audit should therefore be closely related to the identified risks and the topic of the audit. There are a variety of methods available for auditors for collecting and analysing data. Most of these methods may also be applied in compliance audits. This chapter does not provide an exhaustive list of methods, but gives an overview of some of the methodological possibilities when conducting a waste audit, based on INTOSAI WGEA waste audit guidelines [Ref 6] and complemented by practical examples from the already familiar NAOE municipal waste audit.

File examination

Reviewing documents is an efficient way of collecting data and may provide important evidence. Relevant files may include the decisions of officials, ‘case records’ of programme beneficiaries and records of government programmes. Prior to collecting documents, it is important to assess the nature, location and availability of the documents. The analysis of official files and documents may involve collecting citations and relevant descriptions and/or systematically analysing the text to help answer the audit question. However, it is important to bear in mind that document reviews restrict the analysis to the existing documentation. It is therefore often necessary to collect data from other sources.

In NAO Estonia’s waste audit, file examination was wide-ranging. Sources of information included local governments’ plans, regulations and document registries but also documents of ministries and even the parliament. Parliament’s internal documents helped to understand the reasons behind major legislative changes.

Interviews

Interviews are normally used to gather specific and detailed information to answer the audit question. This method is commonly used as a supplement to questionnaires, and may be used to obtain documents, gather opinions and ideas related to specific waste issues, confirm facts, affirm data and explore potential recommendations. Face-to-face and telephone interviews can be used for measuring public opinion. Interviews may form part of the planning phase or the investigation itself. It is important to bear in mind that the interviewees should, as far as possible, represent people with different positions, perspectives and insights. Focus groups may be used to help an auditor achieve a comprehensive overview of problems and risks as well as their root causes, or also for discussing audit plan or the results of the audit.

In NAO Estonia’s waste audit, all major interest groups were interviewed, including the managers of recycling plants, local politicians, officials and experts. This provided a very good overview and gave an idea of the practical situation with waste management.

Observation

Observation is not one of the most common methods applied in performance audits. However, it is frequently used in waste audits to document the actual handling and control of waste during practical site visits. Observation may therefore provide physical evidence in terms of photos and recordings. Furthermore, observation may be used to gain an insight into waste management and an understanding of the functioning of various waste processes.

In NAO Estonia’s waste audit, the audit team visited recycling plants landfills and waste stations to observe real-life waste management operations close up.

Ecuador – Hazardous waste site inspection SAI of Ecuador carried out a waste audit where the level of control risk obtained was high. Therefore, substantive audit tests were applied, which made the in-depth analysis of all available information, as well as the execution of procedures aimed at verifying the coherence between the documented information and what was found on site. Through a technical inspection of the site, the level of compliance with the applicable legislation and the technical requirements for waste management were established, such as the presence of waterproofed floor, spill containment basins, leachate collection systems, covers and signage, among others. The presence of waste that had not been managed was verified with the objective of identifying the effectiveness of the adopted measures and to avoid or mitigate the environmental impacts that the inadequate management can cause. To view third-party content, please accept cookies. Change consent

Use of statistical information

Statistics may be retrieved from the databases of public agencies or from central institutions producing official statistics. Although databases and statistics from secondary sources may provide valuable information, it is important to assess the reliability and relevance of the content. Is the information objective, and does it cover the audit question? Statistics retrieved from databases and secondary sources may provide a basis for simple analyses such as frequencies, mean and other types of distribution. Statistics may also provide a basis for impact studies or cost-benefit studies. These should both consider tangible and intangible costs (ghost costs). Estimating intangible costs will be relevant in many waste audits, as health costs, biodiversity costs and climate change costs are commonly associated with waste.

In NAO Estonia’s waste audit, data about waste generation and recycling were essential for assessing the country’s problems.

Surveys

Surveys can provide a means of systematically collecting the required information. Surveys are useful when quantifying information and are normally used in cases where the information is not available from other sources. Public opinion may be obtained using public poll or e-mail/Internet surveys. A panel survey involves measuring the same variables with the same individuals at several points over time, and subsequently measuring changes in public opinion over time. When preparing a survey questionnaire/questions, the auditor must decide whether to collect data from a defined population or a sample of the population.

In NAO Estonia’s waste audit, a public poll was conducted to profile waste generators: whether they sorted waste (why/why not), whether the waste collection system was accessible and convenient and the cost of the service affordable to them. This helped the audit team to point out problems and come up with recommendations.

Innovative methods

A variety of methods from different sources may be applied when conducting a waste audit. Due to the global flow of waste and the flexible nature of criminal waste activity, waste is becoming an increasingly complex issue. This may in turn require auditors conducting waste audits to adopt new methods. Web-monitoring and tracking systems such as GIS may be helpful in identifying waste-smuggling methods and the destination of various types of waste. As hazardous waste commonly consists of complex chemical compounds, laboratory analysis may be used as a method of identifying various types of hazardous and chemical waste. The involvement of experts may be a way of gathering information about waste streams and waste crime. Case studies could provide a chance to study problem-cause relationship close up and gain a more deepened understanding of issues, while international benchmarking could offer a better understanding of the scale of national issues and ideas for policy solutions.

In NAO Estonia’s waste audit, statistical information and geodata were combined (covered in more detail in the introductory video of Module 4). This innovative method allowed the audit team and the experts to assess the seriousness of the problem of households being excluded from the waste collection system, as well as to detect potential littering sites.

AUDIT CASE: UTILISING NATIONAL CENSUS IN A WASTE AUDIT

Costa Rica – Use of the national census The national census 2011 was used in SAI Costa Rica’s waste audit for two different purposes. Firstly, data about the total amount of houses by canton was used as part of the following indicator (denominator of the formula): Percentage of houses that receive the waste collection service by canton (coverage) = Total of houses that receive the waste collection service in one specific canton (provided by municipalities) Total of houses registered in the canton (provided by national census 2011 because it was not available in the municipalities)   This indicator was estimated for the 81 Costa Rican cantons and it allowed to create a map with ranges that depict the situation of every canton in terms of the coverage of waste collection:                   Secondly, the national census of year 2011 provided key information regarding the percentage of houses that use non-sanitary practices such as waste burning or burying. This piece of information helped the audit team to identify a tendency between the waste collection coverage in every canton and the percentage of houses that burn or bury their solid waste. * – English subtitles are available (see under video settings) FULL CASE (pdf)