NLHHP - Final Evaluation Report

PROCESS EVALUATION

Process evaluation requires description (in as much detail as feasible) of the materials, activities, processes and administrative arrangements that characterize a particular program. This type of evaluation required multiple data collection activities and flexibility in approaches, permitting program monitoring to shift back and forth between formal data collection, informal conversations with participants and analysis of recorded notes.

Selection of the types of data collection approaches and information to be obtained was been guided by (a) the theoretical perspectives of community development and innovation-diffusion, (b) the Guidelines for Heart Health Evaluation (regarding the five major strategies) and (c) the feedback of project staff. The NLHHP had particular interest in those processes that impacted on (1) coalition building, (2) public policy development, (3) sustainability of initiatives, and (4) diffusion of new programs throughout the community health system in the province. Each of these is discussed below.

4.1 Coalition Building

Three major coalitions - building endeavours were pursued in the course of the demonstration phase of the NLHHP: (1) the Coordinating Committee; (2) the partnerships related to each sub-project and (3) the Regional Coalitions.

The process of creating these coalitions shared a common feature in the NLHHP, in that there was a commitment to strengthening public health system leadership through active recruitment of community health professionals in leadership roles:

The NLHHP team was led by 2 Public Health professionals (Drs. Kevin Hogan and Catherine Donovan).
The project coordinator position was filled by a practising community health nurse (Gerri Thompson, followed by Eva Laing).
The Coordinating Committee had representation from numerous community agencies and the Community Health Boards.
All community-based projects were notified that a pre-condition of qualifying for funding was liaison with local community health professionals.

The net result of this strategy was that the NLHHP had, from the beginning, extensive linkages with the people who previously demonstrated commitment to the philosophy and practices espoused by the heart health project. By building on existing community health leadership capacity, the NLHHP ensured that individuals who possessed key characteristics for developing and maintaining a coalition were in place i.e. those who had (a) firm commitment to the idea, (b) previous experience with health promotion projects and (c) access to infrastructure to support activities as necessary, (access to telecommunications, funding, legitimacy in the eyes of the community at large).

There were also differences between the coalition - building processes embraced by each of the 3 endeavours. The most striking of these was the extent to which true public participation in the coalition was fostered. The Provincial Coordinating Committee actually had little to no true consumer representation, but rather, focussed on linkages across key government sectors and community voluntary agencies (such as Heart and Stroke, and the Lung Association). The community projects were mostly at the other end of the continuum, with generally high levels of general public participation, in keeping with the community development/mobilization framework which was the foundation of the project. The Regional Coalition efforts were somewhere in the middle, with strong community health professional leadership but also general public/consumer involvement through representation of local community projects within the region. All 3 coalition building efforts were successfully sustained. The professional/public mix in the regional coalition and community projects did however sometimes lead to tension between these different groups of members as they sought to clarify their respective roles but not to the extent that the successful functioning of the coalition was impeded.

4.2 Public Policy Development

The activities of the NLHHP with respect to healthy public policy focused on providing leadership to the development of a process for introducing healthy public policy as an agenda item for local communities and the provincial government. A document was developed through the Public Policy Subcommittee of the NLHHP Coordinating Committee. This document was originally intended to provide information for individuals or groups who wish to create a policy to benefit the health of the community, or to support, oppose or advocate for a change in a policy which may have health effects in the community. Partners in developing the manual were intersectoral representatives (Social Policy Secretariat of Executive Council, Newfoundland Group Against Poverty, Advisory Council on the Status of Women, Division of Health Promotion, Government of Newfoundland, and academia) and the manual was written by a consultant sub-contracted for the task. The resulting document was officially launched by the Minister of Health in October 1994 and has received broad distribution within the province and across Canada. To date, the major success of the manual is that it has been instrumental in the development of guidelines for the assessment of Cabinet Submissions for health implications. There has, however, been little report of active use of the manual by grassroots groups in the community. Thus, the manual has been successful in introducing, in a formal manner, government consideration of the health implications of public policy but its impact in terms of local community mobilization has yet to be determined.

4.3 Sustainability of Initiatives

Sustainability of initiatives and the heart health agenda overall has been the predominant focus of the NLHHP since its inception and the project has achieved remarkable success in this area. The continuation of community project activities, the development of regional coalitions and the adoption of heart health goals and philosophies by the regional community health boards all attest to the solid foothold heart health has secured in this province. Questions addressing sustainability have been included in all key informant interviews with project personnel over the course of the demonstration phase. The NLHHP evaluation and project teams have identified the primary factors which account for the sustainability of the various components of the initiative. These include:

Integration of activities into the existing community health system whenever possible.
Strong leadership presence, and continuity of leadership, particularly in the initial phases of the program.
Cohesiveness of the coalition/partnership, as evidenced by long term commitment of partners to the task at hand and ability of the coalition to (a) tolerate diversity of viewpoints and (b) maintain flexibility in response to unforeseen circumstances which arise and (c) agree upon overall goals and approaches.
Experience of early successes, combined with an expectation of success which is firmly held by the majority of coalition members/partners involved.
Use of the community development/mobilization approach to generate the community based components of the project.

The major threats to sustainability identified in this project include:

inadequate base of personnel, particularly volunteers, which can lead to burnout and loss of momentum in the initiative.
introduction of competing agendas which cannot easily be overcome (i.e. the continuing high unemployment rate, low levels of economic activity and outmigration have competed with heart health on the local level for attention and investment by scarce volunteers).

The significance of several of these factors can be illustrated by references to 3 project activities which met with varying degrees of success, ranging from highly sustainable to discontinuation. The Single Parents Association of Newfoundland (SPAN) addresses many issues associated with poverty and health promotion. This group, which benefitted from a dedicated leader, demonstrated a high degree of flexibility and creativity in continually modifying their project in order to achieve a feasible and sustainable outcome. They moved their project focus away from formal low-budget cooking classes to a more community mobilization approach of community kitchens, and, successfully disseminated their activity to other regions which included both low income and average income populations. In contrast, a project which is no longer active, Templeton Collegiate, experienced a high turnover of key personnel within one year of initiating their school- based project. Links with the community health system were not vigorously pursued, the school as a whole did not "own" the project and the project eventually floundered under the burden of inconsistent leadership and competing agendas on the local level. The Professional Education Committee, a third project, lies somewhere in between the 2 extremes of sustainability and discontinuation. The project, although it had consistent and committed leadership at the provincial level, encountered significant problems when attempting to mobilize interest at the local level. However, project personnel continuously re-evaluated their approach and persisted in their efforts to create multi-disciplinary professional consensus on heart health promotion. Today, an active project is underway in the Northern region and expectations of success are high among the volunteers and the NLHHP project and evaluation teams.

Based on extensive review of all project components, the NLHHP has helped to develop a model for promoting sustainability which is currently being tested in a NHRDP peer-reviewed project involving 3 provinces and 9 communities involved in the heart health project across Canada. Findings from this study will help to collaborate or question the findings from the demonstration phase in Newfoundland.

4.4 Diffusion of New Programs

Numerous programs or activities developed in one project or region were adopted implemented and maintained in other areas of the province. Examples include:

Community kitchens were initiated by SPAN and adopted by Dunfield Park.
Walking clubs were started in St. Mary's and were taken up in Baine Harbour and St. Barbe; they were also established in various communities where there was no original Heart Health project.
The Heart Smart Restaurant Program was initiated in St. John's and was adopted in the Port Blandford Heart Health project, the Western Region, St. Barbe and the Central Region.
Lifestyle clinics were started in St. Barbe and were taken up by Baine Harbour and Port Saunders, and, eventually, all regions.
Line dancing commenced in Dunfield Park and was followed up on in St. Mary's and St. Barbe.
Aerobics was initiated in Baine Harbour; Port Blandford, St. Mary's and St. Barbe subsequently included aerobics in their programs.
Cooking classes were started by SPAN and adopted by Dunfield Park and St. Barbe.
Community activity days (winter carnival, fall festival) were first held by Port Blandford followed by St. Barbe, Dunfield Park, and Baine Harbour.
Heart Health ski runs were started by St. Barbe and adopted by Port Blandford.
Baine Harbour received an Active Living/Go for Green program grant and subsequently Dunfield Park and Port Blandford applied for this grant.

Most of these activities were slightly modified/adapted for application in new communities but remained overall consistent with the original project. The NLHHP attributes this high rate of diffusion of community based projects to several key characteristics consistent with diffusion theory:

The ideas were compatible with the needs of Newfoundland groups and communities because they were developed and tested by other Newfoundland groups through a community development /mobilization model (and hence were appropriate and feasible).
The networks for information exchange fostered by the NLHHP (including the Heart Health Newsletter, teleconferencing, regional coalition and Provincial Exchange Workshops) reinforced positive expectations about the projects and conveyed the message that these initiatives were "do able" within the constrained resources of most Newfoundland communities.
The linkages with the formal community health system proved invaluable, both as a vehicle for dissemination and a source of reinforcement/support for new ideas.
The NLHHP benefitted from relative stability of most key personnel throughout the project and consistent level of resources available to the project though the NHRDP funds, matched by in-kind contributions from the health and public sector at the provincial, regional and community levels.
Although local media activity proved to be a useful adjunct to project activity, provincial media activity (including a special insert in all province-wide print media early in the project and a later television advertising campaign) did not appear to impact substantially on awareness of the project. Instead, the above noted diffusion networks and locally- focused media efforts were the more important supports to diffusion efforts.