Academic Editor:Tongmeng Jiang, Guangxi Engineering Center in Biomedical Materials for Tissue and Organ Regeneration, Guangxi Medical University, Nanning, Guangxi, China.
Checked for plagiarism: Yes
Review by: Single-blind
Evaluation of the Effects of Health Impact Assessment (Hia) Practice in Monteregie
This study explores the effects of the collaborative model of health impact assessment (HIA), as deployed in Monteregie (Quebec), on the development, adoption and implementation of municipal projects that include health considerations.
Nine HIA processes were studied in nine territories and 35 individuals were interviewed. Data collection was based on the six steps of contribution analysis, and included document analysis, semi-structured interviews, and on-site observations. The study design is cross-sectional design were every HIA was analysed at least six month after completion. The individuals interviewed where those implicated into the HIA process (no matter at what point of the process). No exclusion criteria were applied considering that all points of view were important for this analysis. The Contribution Analysis (CA) was used to analyze the data.
The study results emerged form by the interviews, the field observations and document analysis. They showed that the HIAs had varying results. First, the actors involved acquired new knowledge. However, the HIAs had little impact in terms of increasing the municipal actors’ awareness of health issues. Rather, it helped them acquire arguments for raising awareness among and convincing their municipal council members of the merits of certain actions and their potential positive impacts on citizens’ health. In fact, the HIAs were generally undertaken by municipal actors already aware of the importance of promoting citizen health. Second, as observed in the document, in a few of the HIAs, some recommendations were integrated into planning documents, but usually, as reported by the actor, the HIA report constituted an additional planning document and was not merged with the original planning documents. Lastly, following the HIAs, document analysis and interviews showed that most of the municipal actors continued to include health considerations in their subsequent planning of public policies and projects.
Prerequisites for effective HIA include the presence of municipal actors, who are aware of the importance of their role in their local population’s health, municipal policies that include health considerations, and the municipality’s active participation in the HIA process.
This study sheds light on the complexity of the factors that ensure HIA impact on municipal decision making and decisions. The particularities of each HIA process play a major role.
Health impact assessment (HIA) is a “combination of procedures, methods and tools” that provides impetus to decision makers to include health considerations when planning and implementing projects, programs and public policies. Its aim is to estimate the health effects of such projects, programs and policies in order to minimize potential negative impacts and maximize potential positive impacts 1, 2. It is based on a broad definition of health that includes the social, economic and environmental determinants influencing health 3, 4.
A handful of studies have shown that HIA practice can influence decision makers and induce them to consider the impacts of their decisions on citizens’ health, while also improving their understanding of health determinants 5, 6, 7, 8, 9. HIA can thus bring about positive changes in the planning and implementation of public projects 6, 9, 10. It can be also associated with indirect effects, such as improving intersectoral collaboration 11, 12, 13 and ownership of decisions by the community 8, 10.
Since 2011, the Monteregie regional public health department (PHD) has been collaborating with municipalities in the region in an HIA process. The Monteregie HIA model is based on the five steps proposed by the World Health Organization (WHO) 1, but includes two additional steps: initial scouting to identify a project in the planning stages for which an HIA process could be beneficial, and a final step that involves supporting implementation of the recommendations (which are in the final report) emerging from the HIA. This final step is offered by local public heath actors.
Given that HIA is a valuable prospective impact assessment tool, it seemed highly relevant to examine the effects of its implementation at the municipal level. Support from the Monteregie PHD enabled the municipalities to evaluate the effects of their HIAs on both the collaborative process and the municipal project enhancement process. Moreover, there is scant literature that evaluates HIAs or their effects on local decision making 14. Our study helps fill this gap in the literature. As the HIA process is a complex intervention that involves several actors with many often-differing issues and that has long-term, non-linear effects, Rydin et al. 15 go even further by proposing a collaborative evaluation model that takes all these factors into account, deeming them both pertinent and important. This collaborative model was adopted in our study.
The aim of this research project was therefore to explore the effects of the collaborative model of HIA, as deployed in Monteregie, on the development, adoption and implementation of municipal projects that include health considerations. The objectives of this study were as follows:
1. To determine if and how the knowledge produced and shared during the HIAs was used by decision makers in the development, adoption and implementation of public policies or municipal projects that included health considerations.
2. To identify the contextual factors (political or economic) and personal factors (commitment, values, and beliefs) that influenced the decision-making process and decisions of the municipal decision makers.
3. To determine to what extent the observed effects on decision making were attributable to the HIA.
The research protocol is detailed in an earlier article by Nour et al. 16. The protocol was submitted to and approved by the Ethics Committee of the Charles Lemoyne Hospital Research Centre (MP-HCLM-14-036) in Quebec. No major ethical issues were raised in this study.
The Contribution Analysis (CA) and the theoretical framework of the Advocacy Coalition Framework (ACF) provided the structure used in our study to evaluate the effects of the HIAs.
CA is an evaluative approach that analyzes effects by applying a theoretical logic model to determine the extent to which the observed outcomes can be credibly attributed to a policy or program.
The ACF is a theoretical framework that aids in understanding the contextual elements surrounding a decision-making process. It structures the system of influences within which municipal actors operate and that impact their ability to understand, take ownership of and include health considerations in their reflection and decision-making processes 16. It has been used in many studies on health and public policy impact assessment 17. The ACF divides influencing factors into several subsystems: stable parameters, external events, personal and interpersonal factors, institutional factors, structural factors, and systemic factors.
A theoretical logic model was developed by the research team, based on work by Bourcier et al. 18 and the ACF, to identify the desired short-, medium- and long-term outcomes in terms of the effects of the HIA process on decision making.
Data Collection and Analysis
The HIAs had to have been completed at least six months prior to the start of data collection. The objective was to explore the changes made in municipal decision-making from the beginning of the HIA until at least six months after presentation of the appraisal and recommendations report. As the practice of HIA by the Monteregie PHD was relatively new when the research project began, all the HIAs conducted during the study period were included in the research project.
Data collection was carried out according to the six steps of contribution analysis 19, partially adapted as follows: Develop a theory of change (chain of results); Collect and assess the existing evidence on the chain of results; Propose and assess alternative explanations (AEs); Assemble and assess the contribution story; Seek out additional evidence; Revise and strengthen the contribution story.
Data was collected through semi-structured interviews with various municipal actors and others, on-site observations, and document analysis.
Generally speaking, for each HIA process, the knowledge broker from the Monteregie PHD, the local public health actors, and the municipal actors who participated or were implicated in the HIA were met. In some municipalities, actors outside the local and regional public health networks were also met.
To identify the actors to be interviewed for each targeted HIA process, the research team first called upon the PHD knowledge broker to suggest potential participants, whom were then approached. Next, a snowball recruitment strategy was used by asking these individuals to refer us to other potential actors they regarded as important. Semi-structured interviews were conducted either in person or by telephone, in the respondent’s language of work. The semi-structered questionnaire was developed by the research team. It was designed to correspond to the logical model of the HIA in order to gather the information on short, medium and long term outcome. It was pretested on HIA process, before the study, in order to insure that the questions were clears and that the information gathers was useful for the analysis. Written consent was obtained from the actors who agreed to participate and the interviews were recorded. The individuals interviewed where those implicated into the HIA process (no matter at what point of the process). No exclusion criteria were applied considering that all points of view were important for this analysis. Considering that each HIA implicated a different number of actors, no sample size was targeted or measured. Interviews were conducted by two trained interviewers: one did 5 HIA process and the other one, 4 HIA process.
A search was conducted of documents produced by the municipalities before and after the HIA, to identify indicators that would reveal the effects the HIA may have had on the municipal actors’ decision-making processes. Different aspects were assessed: 1) how many times ‘’ HIA’’, ‘’healthy policy’’, ‘’healthy habits’’, ‘’environmental health’’, “PHD’’, etc. were mentioned in various of municipal documents before/after the HIA process, 2) type of policy or document or regularly rules made before/after the HIA process, 3) collaboration or projects with the PHD, citizen or health organization before/after the HIA process, 4) all other relevant information.
We observed the field implementation of HIA recommendations regarding land use projects. As our evaluation was conducted shortly after completion of the HIAs, only one municipality had actually implemented the project. On-site observations were not, therefore, carried out in this municipality. The observations were made by one the interviewer with a grid based on all the recommendations made in the HIA document. The interviewer had to indicate if the recommendation was put in place (e.g. a light on a corner of a street ton increase security).
Our research project faced several challenges that limited the scope of the study. First, the project involved HIAs carried out in the Monteregie region (Quebec, Canada), where the context may differ from that in other countries. Again, depending on the characteristics of the HIA we studied, the HIAs focused primarily on land use development projects, which may limit the extrapolation of our results to other types of projects.
Moreover, all the HIA processes studied were conducted voluntarily by the municipalities. The participants were interviewed on a voluntary basis and were generally those with a positive perception of their HIA experience. Refusals to participate were noted among actors who were less involved in the HIAs. Despite the limited sample size, i.e. the small number of HIAs studied (n = 9) and of different actors met (n = 36), we reached data saturation in the analysis.
The interviews were transcribed and the verbatim transcripts analyzed using QSR N.Vivo 10 software. The research team performed double coding using a node structure that cross-tabulated the observed outcomes and some of the influencing factors identified in the ACF 20. The documents used in the document analysis were also included in the data analysis.
1. The contribution Analysis (CA), to guide the evaluation of the degree of influence of the alternative explanations (AEs) on the chain of expected results 21. Three selection criteria were applied to the AEs: redundancy, theoretical pertinence and originality.
2. The Relevant Explanation Finder (REF) framework for evaluating the degree of influence of the influencing factors and alternative explanations 22, in order to evaluate the impact of each influencing factor on the chain of expected results.
The degree of influence of each AE was evaluated by applying the REF 22, allowing for transparent evaluation of the weight to be attributed to each AE 23. When the degree of influence of a given explanation was considered low, the AE was discarded and the main hypothesis retained. The simplicity of the measurement scale made it possible to apply the deductive process and reinforce the conclusions drawn regarding each AE’s degree of influence. This increased the reliability of the hypotheses and of the contribution story.
Nine HIAs carried out in Quebec’s Monteregie region between 2012 and 2016 were studied. They involved two social policies and seven land use projects. Nine territories participated in the study: one agglomeration, six cities and two rural municipalities.
Between January 2015 and February 2017, 36 individuals were met for a total of 43 interviews. The 36 participants included 9 elected municipal officers, 12 directors or municipal civil servants, 10 practitioners from the local services networks and five other types of actors who were involved in the HIAs. Among the 36 participants were 27 municipal representatives, 15 of whom were active participants in the various steps of the HIA. Some participants were interviewed two times. Interviewed could take for 30 minutes to three hours long. For each HIA, a chain of results of the influencing factors and AEs was established. Using these chains of results, profiles were compiled to highlight the broad lines or findings regarding the HIAs’ effects.
The results are presented below for all the HIAs, with a distinction made (for the 27 municipal actors interviewed) between those who were participants (n = 15) and non-participants (n = 12) in the HIA process. No demographic characteristic were gather in this study. The participants were steering committee members who had attended most of the meetings held in the context of the HIA. We have decided to present the results in an integrative way (interview, document and observation) as it was analysis as a whole.
In this section, the results are detailed according to the logic model for each of the HIAs.
The HIA increased various knowledges and raised awareness, among most of the municipal actors who participated in the process, of the roles a municipality can play and the positive influence it can have regarding health. However, even prior to the HIA, these actors were often aware of health consideration and open to including them at the municipal level. One added value of an HIA is that it demonstrates a municipality’s ability to act on other, less traditional health determinants and to use various means to achieve this. Nevertheless, the HIA did not increase awareness among the non participant. This therefore attenuates the HIA’s real contribution to the attainment of this outcome.
Commitment was first explored from two angles: participation of the actors involved, and disclosure of the HIA process. In eight out of the nine HIAs, actor participation was good. In only one case, other factors pertaining to the municipal context resulted in poorer participation.
All the municipalities passed a municipal resolution about their participation in the HIA, hence the process was disclosed internally in all cases. However, in only four of the nine HIAs was there external disclosure to citizens of the intention to participate in an HIA or disclosure of the HIA report following receipt by the municipality. Regarding internal disclosure, a post-HIA report was presented to all municipal officers in only two processes, while in four other cases, disclosure was informal.
The HIA reinforced the initial values and beliefs of participants who were already aware of health issues prior to the HIA. The desired change in values and beliefs was not therefore observed among non-participants of the HIA process but who were involved in the project that underwent the HIA. Moreover, it has been notice that municipal actors seemed to reject ideas of the HIA that did not coincide with their personal values or institutional view of the municipality’s development. Therefore, no change of values was observed even in participant of HIA for some concepts.
Integration of the recommendations into planning documents refers to the changes made to the municipal project after the HIA, in light of the recommendations offered. We observed this integration in only two of the nine HIA processes. In the remaining seven, the planning documents had already been written, published and ratified by the municipal councils. In these cases, the HIA was used as a complementary tool.
Most of the municipalities implemented the recommendations or intended to do so, with a preference for recommendations concerning land use projects rather than social themes. There was also a clear tendency among the municipal actors to drop recommendations that did not fit with their personal and/or institutional beliefs or values.
In seven out of the nine HIAs, either the recommendations were integrated or the municipal actors had the intention of integrating them into other planning documents that were not the subject of the HIA. This type of result is easily attributable to the HIA, but depends on influencing factors such as the institutional vision of the municipality’s development.
Complex contexts like those surrounding the development of policies, which include health considerations, are the result of numerous influencing factors, as described in the Advocacy Coalition Framework. It is impossible to show that any single factor has caused the observed outcome. However, HIA may be an important change factor, much like other influencing factors 24.
Each of the observed outcomes of the HIA that are detailed in the previous section may therefore be associated with specific influencing factors.
Personal beliefs and values, prior experiences with HIA, and field of professional practice are characteristics that can influence the observed outcomes positively or negatively. In terms of the professional functions of the individuals in charge of the project that underwent the HIA, they often involved land use planning. In these instances, we generally noted a greater interest in promoting and defending the HIA recommendations concerning land use projects.
In five out of the nine HIAs, the person in charge of the project that underwent the HIA had a limited degree of municipal influence. In three other HIAs, the degree of influence was limited to certain types of decisions, and in the remaining HIA, the person had full decision-making power. This was one of the key influencing factors on the outcomes of the HIA, as the actors with no, or only some, decision-making power had to convince others of the merits of including health considerations in municipal projects and policies. In seven municipalities out of the nine, the leaders of the projects undergoing the HIA mentioned being the sole champions of including health considerations. They encountered difficulties when people in more influential positions were unwilling to implement certain recommendations, most likely for reasons that had nothing to do with health.
To maximize the outcome of HIA process, it should be to start during the development phase. In two processes, the HIA took place during the project development phase, and thus the writing of these projects was influenced by the HIA recommendations. In one process, timing was a truly problematic since the project was already under way when the HIA report was filed. In the remaining projects, final planning documents were submitted for the HIA, and the HIA report and recommendations were therefore used in tandem.
The more actively the municipal actors participated in the different steps of the HIA, the more likely they were to have been made aware of their role in health issues and to display an openness to consider implementing the HIA recommendations. In HIAs process studied, participation did varied a lot from actively implicate, to barely implicate.
The HIA follow-up step was often not understood and therefore not done. Yet failure to carry out follow-up reduces the likelihood that the recommendations will impact decision making. For the processes where this follow-up was carried out, the local public health actor was involved in various municipal projects and made a point of promoting the ideas and recommendations of the HIA.
The acceptability of the recommendations produced in the HIA was enhanced in cases where the municipal actors had previously collaborated with the PHD. Conversely, when these actors were not committed to the issue of health prior to the HIA, the process often generated new ideas that would be harder to integrate.
This factor can have a very significant impact, either positive or negative on implementation of HIA recommendation. For example, in one municipality, the change in municipal council greatly aided the continuation of the HIA and enhanced its sustainability, as the elected municipal officer who had requested the HIA became mayor.
This constituted an influencing factor mainly regarding implementation of some of the HIA recommendations concerning land use projects, more specifically in the smaller municipalities.
These municipalities had little leeway for development projects, as their budgets were devoted to ongoing municipal operations. For three municipalities, the initial motivation for participating in the HIA stemmed from the fact that scientific arguments were given to show that the process could strengthen applications for subsidies in the context of various calls for tenders.
This was found to be a positive, but less weighty influencing factor that may have facilitated both implementation of the HIA recommendations and a change in values. In fact, as the majority of the actors confirmed that the HIA report was credible and based on scientific facts, this legitimized the actions proposed.
Using a contribution analysis approach was fundamental to measure the extent to which the observed outcomes were attributable to the HIA, but still showed the difficulty to generalize the conclusions reached in this regard to all the HIAs studied. While we were able to classify the influencing factors according to their general degree of influence on attainment of the outcomes, it was difficult to obtain an overview of the degree of influence of the HIA process as a whole on the attainment of the observed outcomes.
Our study nonetheless yielded a number of findings regarding the effects of HIA on decision making and the influencing factors that hinder or facilitate the process.
There is currently no consensus regarding the definition of an effective HIA. The results we obtained suggest that the earlier the involvement of key municipal decision makers in the HIA process, the more beneficial the HIAs. Strategies should be deployed to ensure the commitment and engagement of these decision makers. It is also recommended that they invest the necessary time and take active part in formulating the recommendations. In this way, the actors ensure that any recommendations are pertinent and adapted to the local context. Health impact assessments are likely to be more influential if they directly address issues of concern to decision makers 28. Lastly, it is recommended that awareness be raised among all municipal council members and municipal civil servants. The largest possible number of municipal decision makers should take part in order to obtain maximum impact. This recommendation is in length with what is founded in the literature. In fact, according to the document Minimum Elements and Practice Standards for Health Impact Assessment (2013)29 comprehensive Health Impact Assessments (HIA] should include, as a minimum elements, the involvement and engagement of stakeholders affected by the proposal.
According to Quigley and Taylor 27, to maximize the effects of an HIA, the process should, right from the outset, define the desired outcomes and the goals sought by each of the participating actors, as the latter may have their own particular reasons for participating. This provides greater insight into the particularities of each type of HIA conducted and ensures that “the right thing” is evaluated. Along the same lines, Haigh et al. 5 recommend that time be spent early in the HIA process inquiring about the municipality’s initial motivations. It becomes all the more important to discuss and clarify the objectives, values and desired outcomes.
The HIA should therefore be conducted at a timely moment, preferably in the early planning stages of a policy or project, when certain aspects can still be modified.
As one of the main objectives of an HIA is to promote inclusion of health considerations in all municipal policies and actions, it is essential that local and regional public health actors offer support to and do follow-up in the municipality.
In addition, to promote application of the recommended actions, the HIA team should first provide clear explanations of the roles of the regional vs the local public health actors regarding post-HIA follow-up, and verify that they are understood. Local public health actors should be equipped to do proper follow-up. In other words, they must enter into dialogue with each other to ensure the best possible follow-up in order to support the municipality in carrying out the HIA process and implementing the recommendations. It is therefore important to monitored (via the local public health actors) or to propose to the municipality a monitoring plan in order follow the actions that promote positive health impacts and mitigate the negative health impacts of the decision 29. This recommendation is also made by O'Mullane (2013)30 who stated that the ‘’follow-up and monitoring of the result of a HIA, is critical to ensuring the quality of the HIA’s carried out as well as ensuring that the investment and application of the HIA is generating the intended or desired result.’’
The aim of this study was to explore the effects of HIAs conducted in Monteregie on the municipal actors’ decision-making processes. It identified the effects of the collaborative HIA model, as deployed in Monteregie, on the development, adoption and implementation of municipal projects that included health considerations.
The study sheds light on the complexity of the factors involved in ensuring HIA impact on municipal decision making and decisions. It shows HIA to be a promising means of influencing health at the municipal level in already-favourable contexts. In such contexts, the impact of the HIA can be maximized by involving important decision makers in the process, conducting the HIA on the right project at the right time, and ensuring proper follow-up. In other contexts, factors such as budget restrictions may reduce the effects of the HIA. The same applies if the leader of the project undergoing the HIA is the only person aware of the importance of including health considerations, or if the HIA process is conducted or the HIA report (with its recommendations) is issued when the project is already too far advanced for modification.
Several bias and limitations need to be mentioned. First, even thus there was a strict interview canvas and standardized interview analysis grid, two interviewers made the interviews, which may lead to different interpretations. Secondly, HIA done in Monteregie are different from the one made elsewhere with a huge implication of the PHD into the process, which may influence the results. Thirdly, some (even thus it was few) actor who did participated into a HIA process refused, for various reason (i.e. change of job, lack of time), to participated in this evaluation. Their point of view might be different from those who had accepted to participate. Finally, future research should investigate the effectiveness of HIA after conducting case studies (several years later) such as those in this study. It should also look closely at the cost-effectiveness variable of HIA.
This project was funded by the Canadian Institutes of Health Research and Quebec’s Health Research Fund.