Impact Assessment

Title: Project and Program Evaluation Author: Jennifer L. Tavis

I. Impact Assessments What is an impact assessment? Impact assessments are conducted in the early stages of a proposed project to try to predict the potential positive and negative effects of the project on the environment and society. They are defined by the International Association for Impact Assessment (IAIA) as “the process of identifying the future consequences of a current or proposed action.” (www.iaia.org) Carrying out an impact assessment involves conducting a thorough analysis of current conditions, reviewing the effects of past changes and projects in the affected area, and working with experts and stakeholders to extrapolate what may happen as a direct or indirect consequence of carrying out the proposed project.

How are impact assessments used? Impact assessments are used by NGOs, governments and corporations as means of ensuring that the major changes resulting from a proposed project will have a net positive effect, and that negative consequences can be foreseen and mitigated.

If your proposed project will have a significant impact on the environment or on a community, it is a good practice to perform an impact assessment as a part of carrying out the project. Indeed, impact assessments may be mandated in certain cases. For example, if you are planning to build a hydroelectric plant to provide electricity for an underserved area, the government may require that an environmental impact assessment be conducted to ensure that the plant and its construction will not significantly disrupt the local ecosystem. A social impact assessment may also be required to verify that farmers downstream will still have sufficient clean water to irrigate their fields, since the hydroelectric plant may reduce water flow.

There are various types of impact assessments, and you will need to consider which ones make sense for your project. While this article provides an overview of the various types of impact assessments and their basic elements, the skills needed to carry them out effectively are fairly specialized. This article is intended as an introduction to impact assessments and their uses, not as a manual for conducting them.

If you feel that your project requires an impact assessment, look into bringing someone on board with formal training and experience in the type of impact assessment you will be conducting. If that is not possible, make sure someone on your team has a strong grounding in sociology, data analysis, and whatever specialization is required (for example, environmental science, medicine, hydrology, and so on), and conduct thorough research on impact assessment methodologies and approaches. At the end of this article, you can find a list of resources on the various types of impact assessments. Why are impact assessments important? Before your project has even begun, impact assessments allow you to: • ensure that the project has a strong chance of resulting in the desired outcomes • take steps to mitigate negative consequences • build support for a project among stakeholders, donors and other interested parties

It is not possible to foresee all of the big and small ways in which a project might affect the people and environment where it is carried out, but impact assessment techniques can help dig beneath the obvious implications and uncover concerns, risks and benefits that might not otherwise be recognized.

By engaging the community in a dialogue about the project prior to its execution, you build the case for why the project is useful and how they will benefit from it. You can address concerns before they become objections, impediments or resentments. Often it is possible to avoid major conflicts with the community and enhance the project’s chances of success by making minor adjustments to the plan based on what is learned in the assessment process.

Impact assessments also provide assurance that major proposed changes have received due consideration prior to implementation. A famous example is the series of studies conducted prior to the construction of the Trans-Alaska Oil Pipeline in the 1970s. The United States National Environmental Policy Act (NEPA) required that the potential environmental and social impacts be investigated before construction on the pipeline began, and the developers were required to reconsider certain aspects of their plan as a result.

Another relevant case is the proposed Atlantic Yards development project to build large apartment towers and a new basketball arena in Brooklyn, New York. Impact assessments revealed that the project would stretch local transportation, education, emergency services and other infrastructure beyond its intended limits. As a result, community activists have pressed the developer to rethink the scale of their project and work with the city to address the infrastructure concerns.

While neither of these projects involved NGOs, they demonstrate the value that can be created by impact assessment efforts.

Types of Impact Assessments There are three major types of impact assessments: • Environmental Impact Assessments • Social Impact Assessments • Health Impact Assessments Though these three types of impact assessments share certain themes and approaches in common, each has its own history, goals and formal methodology. They can be used separately or together, depending on the type of project under analysis. Large or complex projects often have diverse consequences that cannot be adequately analyzed through any single type of impact assessment. The descriptions below will help you to determine which types of impact assessments are most useful for a given project.

1. Environmental Impact Assessments (EIAs) An environmental impact assessment is “a procedure for assessing the environmental implications of a decision to enact legislation, to implement policies and plans, or to initiate development projects, [and] has become a widely accepted tool in environmental management.” (Wathern 1992, 3) It takes into account potential human and ecological consequences related to the environment.

Methodology According to the IAIA, the major activities that make up an environmental impact assessment are: • Screening to determine whether a proposal should be subject to an EIA and, if so, at what level of detail • Scoping to identify the issues and impacts that are likely to be important, and to establish terms of reference for the EIA • Examination of Alternatives to establish the preferred or most environmentally sound and benign option for achieving proposal objectives • Impact Analysis to identify and predict the likely environmental, social and other related effects of the proposal • Mitigation and Impact Management to establish the measures that are necessary to avoid, minimize or offset predicted adverse impacts and, where appropriate, to incorporate these into an environmental management plan or system • Evaluation of Significance to determine the relative importance and acceptability of residual impacts (that is, impacts that cannot be mitigated) • Preparation of an Environmental Impact Statement (EIS) or Report to document clearly and impartially the impacts of the proposal, the proposed measures for mitigation, the significance of the proposal’s effects, and the concerns of the interested public and the communities affected by the proposal • Review of the EIS to determine whether the report meets its terms of reference, provides a satisfactory assessment of the proposal and contains the information required for decision-making • Decision-making to approve or reject the proposal and to establish the terms and conditions for its implementation • Follow-up to ensure that the terms and conditions of approval are met, to monitor the impacts of development and the effectiveness of mitigation measures, to strengthen future EIA applications and mitigation measures, and, where required, to undertake an environmental audit and process evaluation to optimize environmental management It is desirable, whenever possible, to design monitoring, evaluation and management plan indicators so they also contribute to local, national and global monitoring of the state of the environment and sustainable development. (International Association for Impact Assessment in cooperation with the Institute for Environmental Assessment, UK 1999, 4)

2. Social Impact Assessments (SIAs) Social impact assessments are used “to inform the public about proposals and their implications; to assemble information from locals; [and] to solicit public opinion on proposals, alternatives, trade-offs, etc.’ (Barrow 2000, 3)

Methodology According to the IAIA: SIA comprises most of the following activities. It: • participates in the environmental design of the planned intervention; • identifies interested and affected peoples; • facilitates and coordinates the participation of stakeholders; • documents and analyses the local historical setting of the planned intervention so as to be able to interpret responses to the intervention, and to assess cumulative impacts; • collects baseline data (social profiling) to allow evaluation and audit of the impact assessment process and the planned intervention itself; • gives a rich picture of the local cultural context, and develops an understanding of local community values, particularly how they relate to the planned intervention; • identifies and describes the activities which are likely to cause impacts (scoping); • predicts (or analyzes) likely impacts and how different stakeholders are likely to respond; • assists evaluating and selecting alternatives (including a no development option); • assists in site selection; • recommends mitigation measures; • assists in the valuation process and provides suggestions about compensation (non-financial as well as financial); • describes potential conflicts between stakeholders and advises on resolution processes; • develops coping strategies for dealing with residual or non-mitigatable impacts; • contributes to skill development and capacity building in the community; • advises on appropriate institutional and coordination arrangements for all parties; • assists in devising and implementing monitoring and management programs. (Vanclay 2003, 4) 3. Health Impact Assessments (HIAs) Health impact assessments are used to “identify how development induces unintended changes in health determinants and resulting changes in health outcomes. HIA provides a basis to proactively address any risks associated with health hazards.” (Quigley 2006, 1) Methodology According to the IAIA: The following represents key steps in the HIA process and suggested responsibilities for each step: 1. Screening: deciding what scale, if any, HIA is required (desk exercise by ministry/authority). 2. Scoping: setting the boundaries in time and space for the assessment and formulating Terms of Reference (TOR) for a full scale HIA accordingly (usually by Ministry of Health (MOH) (central, province and/or district) and key stakeholders). 3. Full scale HIA (by HIA team according to specifications in TOR). 4. Public engagement and dialogue (initiated by MOH or other relevant authority). 5. Appraisal of the HIA report (compliance with TOR, quality control of independent criteria) and the feasibility/soundness/acceptability of its recommendations (MOH or another MOH assigned independent consultant). 6. Establishment of a framework for intersectoral action (MOH and relevant ministries). 7. Negotiation of resource allocations for health safeguard measures (Ministry of Finance and relevant ministries). 8. Monitoring (of compliance and of pertinent health indicators), evaluation and appropriate follow-up (MOH and line ministries). (Quigley 2006, 3)

Designing and Implementing an Impact Assessment Exactly what shape each step in the assessment process takes, and who the ultimate decision-makers are, will vary from project to project. In some cases, the national or local government may take an active role in the decision about whether a given project will go ahead. In other cases, the political and social situation may be such that it is difficult to convince the government or the local community to participate and provide input. If you are having difficulty getting the right people to take part in the process, it may be due to a lack of understanding of what the project is and why it matters to the people who will be affected. While you don’t want to bias the process, you do want to give stakeholders a reason to participate. Be clear about the potential benefits, and risks, that the project holds for them.

Be careful to ensure that the impact assessment is structured to address the project’s scope and doesn’t get sidetracked by ancillary agendas. Impact assessments are often carried out for political reasons, and it can be a challenge to keep the assessment focused and objective. Careful scoping prior to implementation can help keep the assessment from being derailed or co-opted by special interests.

Information collection techniques should be standardized, and thought should be given to how to avoid biasing outcomes. When possible, take advantage of existing reliable information sources such as census data or existing academic studies. When you are collecting your own data, use established information-collection techniques. Surveys, questionnaires, focus groups, community meetings and interviews can all be useful means of gathering information. • Surveys: If you need to gather statistical data on a particular topic, surveys can be an effective tool. They are cheap and easy to administer, and the results are easy to parse. Surveys are not the best choice if the questions you are investigating are complex and require dialogue. But if you have a few simple questions and need feedback from a large population, surveys can provide you with the information you need. Surveys can be conducted on paper (see “Questionnaires” below), by phone, or in person. Conducting surveys by phone or in person can increase response rates, but it also tends to push up costs. Consideration should be given to sample size and diversity in determining the approach. Be sure to test your surveys (on yourself or a co-worker) before you start using them with your target population. • Questionnaires: These remain the cheapest and easiest way to collect quantitative data for statistical analysis. If the population you are targeting has Internet access, you can do them online using free or low-cost tools (for example, www.surveymonkey.com); of course, there is always the paper option too. When you are designing a questionnaire, give consideration to the reading level of your target population. If literacy is low in the target population, consider a verbal survey instead. If education levels aren’t high on average, keep vocabulary simple. Always keep the questions and response options as straightforward as possible. There shouldn’t be much room for interpretation. Ambiguous questions lead to useless results. As with the surveys, give careful thought to sample size and diversity, and always test your questionnaires before use. • Focus Groups: Focus groups are facilitated discussions on particular topics with pre-selected representatives of the larger population you are trying to understand. If part of what you seek to understand is the group decision-making process, these discussions can be tremendously useful. This technique collects qualitative rather than quantitative information, and the real value comes from the details of the discussion rather than any conclusions the participants reach. Focus group facilitation is a specialized skill; if you are considering taking this route for information gathering, it may be worth considering hiring a professional. • Community Meetings: These can be an excellent forum for educating the community about the project and for learning about their questions and concerns. Typically there will be a speech or two describing the project and then a question-and-answer period. You may want to have a panel for answering questions, or you may want to designate a knowledgeable, able communicator from your team. Be aware that this sort of forum tends to favor the bold and boisterous. Those who are shy or nervous about standing up in front of a group will probably not participate. The most active participants are likely to be those with strong opinions, and those opinions may not always be favorable to your cause. If you know that the meeting is likely to be contentious, it may be helpful to meet individually with your most vocal critics beforehand to answer their questions and try to assuage their concerns. At the very least, this will allow you to prepare to address the issues they raise, so you don’t get blind-sided. • Interviews: Individual interviews, while time-consuming and expensive, yield valuable insights into how people view issues and approach problems. Experts disagree about how many interviews are enough to be representative. In general, if you have selected sufficiently diverse interviewees to represent the population you want to learn about, you have done enough when the interviews stop yielding substantial new insights. In general, it is best to err on the side of thoroughness, but time or budget constraints may limit what is possible. Be sure that the interview questions are clearly worded, relatively free of bias, and open-ended. “Yes/no” questions aren’t very conducive to conversation. As the interviewer, try to be an active listener. If possible, use an audio recording device during the interview so that you aren’t distracted by note-taking. Try to be calm and consistent in your approach, body language, and tone of voice so as not to bias the interviewee. Also, be aware of the relationship you are building with your interviewee. Consider their expectations, hopes and fears around the interview experience. They may desperately want to stay anonymous, or, conversely, they may want to be kept informed regarding the project’s progress and have their role recognized. Interviewing is a complex tool, and, as with focus groups, may be best used by experienced professionals. The tools described above are just a few of the options available for gaining feedback from people. The descriptions provide an overview, but books have been written on each of these techniques — their strengths, their weaknesses, and their variations. Consider which tool or tools will best meet your needs as you carry out the assessment process.

Impact Assessments in Post-conflict and Conflict-ridden Areas A Peace and Conflict Impact Assessment (PCIA) “is a means of anticipating, monitoring, and evaluating the ways in which an intervention may affect or has affected the dynamics of peace or conflict in a conflict-prone region.” (Kenneth Bush).

It is all too easy for well-intentioned aid and development efforts to go astray, supporting the aims of political or military factions rather than the populations they are meant to serve. It can be difficult to be certain that services and resources provided are actually reaching the target population. It can be difficult even to collect reliable information. Under these challenging circumstances, it is especially important to use a careful and rigorous approach.

Challenges and Successes with Peace and Conflict Impact Assessments In a conflict or post-conflict zone where your project team may feel threatened or uncertain, it can be tempting to work only with those who seem most willing and able to work with you. In fact, it can be very difficult to do otherwise. However, that tendency poses certain dangers, especially when NGOs are offering goods and services that factions can easily misdirect for their own benefit. Kenneth Bush, author of a useful how-to guide to PCIAs, identifies the following ways in which “Development can create conflict: • By increasing socio-economic inequalities — or fueling the belief that such inequalities are increasing • By benefiting certain groups more than others • By increasing competition for development resources and political control • By introducing new structures and institutions that challenge existing ones (social, political or economic)” (Kenneth Bush) These concerns can be addressed by methodically engaging the community and working to open communication channels. Try to ensure that both men and women from the impacted communities take part. Include in the dialogue not only leaders but people from all walks of life. Work to build bridges and improve mutual understanding among groups. Take the time to investigate the current situation as thoroughly as possible in order to avoid placing valuable resources in the hands of a single group. If you are using local service vendors, screen them carefully and supervise work to ensure that money is not simply being embezzled.

Make use of the resources at the end of this article to ensure that you are on the right track. There is no single approach that is correct in all cases, and each situation will present unique challenges. You will need to tailor your assessments case by case.

There is much debate over what constitutes success in environments that pose so many challenges. With such a variety of factors at work, it is nearly impossible to link NGO efforts directly to increases or reductions in levels of violence.

Nonetheless, the PCIA process is an effort to create metrics for success. It takes into account the failings of past and current methodologies and asks how we can do this better. By actively trying to investigate the situation, engage the communities that will be affected and mitigate the risks uncovered by their work, NGOs are taking steps to ensure that their efforts are thoughtful, responsibly carried out, and genuinely beneficial.

Microfinance Impact Assessments Microfinance impact assessments measure “how the services of a microfinance institution (MFI) impact the lives of its clients in such areas as employment, income, nutrition, education, health, and gender equity. It is the primary tool used to determine the effectiveness of microfinance as a development intervention.” (Anton Simanowitz) They borrow from the methodologies for health, environmental, and social impact assessments, but also face unique challenges. Primary among them is the fact that money is fungible (that is, it is interchangeable with other money), so it can be difficult to track loan money and be sure it is actually spent according to the borrower’s stated purpose. That in turn makes it more difficult to measure the economic impacts of the money lent.

Challenges and Successes with Microfinance Impact Assessments In recognition of the complex realities they face, microfinance institutions have changed the focus from “proving” to “improving” results in order to come up with meaningful statements regarding the impacts of their efforts. Rather than to trying to prove direct cause and effect between an individual’s receipt of a loan and his or her becoming more economically successful, the goal is to show improvements in overall social indicators for the target group, as compared to a control group. Also, longitudinal indicators (comparisons of where people were prior to, during, and after the loan) are aggregated to gain a clearer understanding of impacts over the whole life-cycle of loans. In a sense, microfinance institutions are conducting individual impact assessments for each loan they provide, then using those unique stories to piece together the overarching themes.

The popularity of microfinancing has been increasing in recent years because of its apparent successes around the world. The website www.microfinancegateway.com, an excellent resource on microfinance in general and impact assessments in particular, cites numerous examples of success with microfinance initiatives. According to the site, “studies of microfinance programs and their clients indicate the following impacts on poverty and hunger. (Click on the links to read the full studies.) • In Indonesia, 90 percent of BRI clients surveyed on the island of Lombok had moved above the poverty line, with income increases averaging 112 percent. See Panjaitan-Drioadisuryo et al.(1999) • Extremely poor Zambian clients of Zambuko Trust, a local MFI, increased their consumption of high-protein foods at a time when food expenditures across the country as a whole were decreasing. See Barnes (2001) • In addition to increased economic well-being, a study of SHARE clients in India documented a marked shift from irregular, low-paid daily labor to more diversified sources of income, with a strong reliance on small businesses. See Simanowitz and Walters (2002)” (http://www.microfinancegateway.org/section/resourcecenters/impactassessment/impacts/mdg_1/) These impacts demonstrate the value not only of microfinance projects, but of the impact assessment process, without which we would not be aware of the positive changes brought about by these projects, and would therefore be less motivated to replicate that success.

Methodological Problems with Impact Assessments Impact assessment methodologies are evolving constantly because so much is at stake and the problems are so complex. Cause and effect are seldom straightforward and measurable, and you can only follow the train of potential consequences so far before it becomes absurd. Impact assessments are not crystal balls for seeing the future, but when carried out effectively, they provide valuable insights into the strengths and weaknesses not only of the project, but of the methodology itself. Several issues have been identified. • Impact assessments do not have a common methodology overall, due to the independent development of impact assessment specialties. • Practitioners don’t often enough share their learning and experience in order to improve methodologies. • There is no formalized certification for practitioners, so implementations of existing methodologies vary significantly in quality and approach. • Decision-makers often ignore the recommendations of impact assessments. None of these is an easy problem to solve. Keep them in mind as you conduct your assessment, and give thought to how you might be able to contribute to the ongoing effort to solve them.

External Links: International Association for Impact Assessments: www.iaia.org Berghof Handbook for Conflict Transformation: http://www.berghof-handbook.net/ Hands-On PCIA: http://action.web.ca/home/cpcc/attach/Hands-On%20PCIA%20--%20Handbook%20X%20--%20BUSH%20Final%20Author%5C's%20Version1.pdf WHO on Health Impact Assessments: http://www.who.int/hia/en/ U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Marine Fisheries Service on Social Impact Assessments: http://www.nmfs.noaa.gov/sfa/social_impact_guide.htm Microfinance Gateway (website for CGAP): http://www.microfinancegateway.org Online Survey Tool: www.surveymonkey.com

References: Barrow, C. J. Social Impact Assessment : An Introduction /. London: Arnold, 2000. http://www.questia.com/PM.qst?a=o&d=102534049. (accessed January 30, 2007) Bush, Kenneth. “Hands-On PCIA” Under the Auspices of the Federation of Canadian Municipalities and the Canada-Philippines Local Government Support Programme. http://action.web.ca/home/cpcc/attach/Hands-On%20PCIA%20--%20Handbook%20X%20--%20BUSH%20Final%20Author%5C's%20Version1.pdf (accessed January 30, 2007) "Environmental Impact Statement." In The Columbia Encyclopedia 6th ed., edited by Lagass, Paul. New York: Columbia University Press, 2004. http://www.questia.com/PM.qst?a=o&d=101242845 (accessed January 30, 2007). Day, Kevin. "Can Risk Assessments Be Fact-Based? Subjectivity Can Never Be Removed Completely from the Risk Assessment Process, but It Should Be Stripped from the Data-Gathering Phase." Security Management, September 2003, 248+. http://www.questia.com/PM.qst?a=o&d=5002554102. (accessed January 30, 2007) DeMarrais, Kathleen and Stephen D. Lapan, eds. Foundations for Research: Methods of Inquiry in Education and the Social Sciences. Mahwah, NJ: Lawrence Erlbaum Associates, 2004. http://www.questia.com/PM.qst?a=o&d=110599546. (accessed February 20, 2007) Derman, William and Scott Whiteford, eds. Social Impact Analysis and Development Planning in the Third World. Boulder, CO: Westview Press, 1985. http://www.questia.com/PM.qst?a=o&d=51258517. (accessed January 30, 2007) "Evaluation and Use of Epidemiological Evidence for Environmental Health Risk Assessment: WHO Guideline Document." Environmental Health Perspectives 108, no. 10 (2000): 997. http://www.questia.com/PM.qst?a=o&d=5001146299. (accessed January 30, 2007) International Association for Impact Assessment in cooperation with the Institute for Environmental Assessment, UK. Principles of Environmental Impact Assessment Best Practice. Fargo, USA: International Association for Impact Assessment, 1999. http://www.iaia.org/Members/Publications/Guidelines_Principles/Principles%20of%20IA.PDF (accessed January 30, 2007)

“MDG 1: Eradicate Extreme Hunger and Poverty.” CGAP. http://www.microfinancegateway.org/section/resourcecenters/impactassessment/impacts/mdg_1/	(accessed February 20, 2007)

Paffenholz, Thania. “Peace and Impact Assessment Tip Sheet.” SDC. http://www.sdc.admin.ch/ressources/resource_en_92756.pdf (accessed January 30, 2007) Quigley, R., L. den Broeder, P. Furu, A. Bond, B. Cave and R. Bos. Health Impact Assessment International Best Practice Principles. Special Publication Series No. 5. Fargo, USA: International Association for Impact Assessment, 2006. http://www.iaia.org/Non_Members/Pubs_Ref_Material/SP5.pdf (accessed January 30, 2007) Simanowitz, Anton. “The What, Why, and How of Impact Assessments.” CGAP. http://www.microfinancegateway.org/section/resourcecenters/impactassessment/what (accessed February 20, 2007) Vanclay, Frank. Social Impact Assessment International Principles. Special Publication Series No. 2. Hobart, Tasmania: International Association for Impact Assessment, 2003. http://www.iaia.org/Members/Publications/Guidelines_Principles/SP2.pdf (accessed January 30, 2007) Wathern, P. "1 An Introductory Guide to Eia." In Environmental Impact Assessment: Theory and Practice, edited by Wathern, Peter, 3-30. London: Routledge, 1992. http://www.questia.com/PM.qst?a=o&d=103493021. (accessed January 30, 2007) Wathern, Peter, ed. Environmental Impact Assessment: Theory and Practice. London: Routledge, 1992. http://www.questia.com/PM.qst?a=o&d=103493001. (accessed January 30, 2007)