More and more are policy evaluations used as a tool to understand and improve the policies implemented on achieving a specific goal. For fire prevention activities, the utmost important aim is to change the peoples’ behaviour to decrease the number of fire casualties and emergencies. In case an emergency still occurs, they would have to have the knowledge and skills to act in a way to minimize the possible losses. In order to analyse, whether the undertaken fire prevention activities have the assumed effect in pursuing their aim, a policy evaluation would be of great help. This subsection will focus on the first eight months of the project period and to which state the framework has thus far been developed.
To come up with a tool to evaluate the impact of fire prevention activities in the project partners countries (Estonia, Latvia, Lithuania, Denmark and Finland), we introduce as a result of document analysis a somewhat similar protocol to Magro and Wilson (2013). The protocol follows these steps:
As a first step to gather knowledge of the system, the descriptions of different fire prevention activities were collected from each country with the help of the officials. The descriptions covered the three basic and big questions in economics introduced by A. Smith: what? to whom? and how?
Based on the feedback, short country overviews were developed, which would be the starting point for each country’s’ analysis. These country overviews will be covered in the final report as an introduction to project partners’ systems.
After collecting the basic information on each fire prevention activity, the different activities were analysed and classified into bigger groups. A natural way would be a division of five, which also was confirmed by each project partner during the Riga meeting (August 16-17):
1) Personified activities, which means, that something is targeted specifically to an individual or household. Such activities would be consultations with elderly people, specific professionals (e.g. school directors, mayors), house visits and so on.
2) Standardized activities to a (fixed/closed) group, which includes teaching/consulting a group of people, following a beforehand decided framework. Such activities would be trainings for school children, standardized informational/safety days with fixed invitation lists and so on.
3) Event-based activities to a (random/open) group, which includes providing information in an open setting. Such activities would be participating in exhibitions and fairs, other project-based events.
4) Large-scale media campaigns, which target a wide group of population. Such activities would be multi-platform (tv-, web-based, social media advertisement) media campaigns.
5) General information provision. Such activities would be op-eds in newspapers, press releases, updating web page/social media, etc.
Such a classification would be the basis of evaluating the impact of single prevention activities – each group has a somewhat specific framework for measuring and evaluating the impact. The following graph covers each group of activity, where it is implemented, to whom targeted, how implemented and how such activity’s impact could be measured.
To cover the sub-goals of the project, the comprehensive evaluation framework to assess the impact of fire prevention consists of three interrelated components, which would characterize the prevention activities in a holistic way. In addition to the impact component, which will characterize the impact of each single fire prevention activity based on the previously explained framework, it consists of the general statistics component, which will characterize the long-term impact of fire prevention (and trends) and development of society. And the population survey component, which will characterize the general knowledge and development level of fire prevention in a country in a specific point of time. The first phase of the project has developed (a) the country overviews, which describe and explain in a systematic manner, which fire prevention activities are implemented by each partner country; (b) the framework in which the fire prevention activities’ impact will be measured and (c) a classification for different fire prevention activities with basic guidelines on how to measure the impact of an activity in each group.
During the first eight months of the project, the initial framework, of how to measure the impact of fire prevention activities in a systematic and comprehensive way, was developed. As a next step, an individual impact evaluation for each prevention activity in each partners’ country is conducted. The impact evaluation methodology follows a guideline dependent to which “group” the activity belongs to. Simultaneously, the preparations to carry out the population survey are made as well as the information for the general statistics component of the evaluation is collected and analyzed.