Decide on evaluation measures

Typical questions asked in evaluation

1. Was the intervention selected on the basis of a review of the type of trespass, its causes and motivations?
2. Did the design of the intervention follow recognised good practice?
3. Has the intervention been implemented and maintained as intended? 
4. What were the inputs and outputs of the intervention?
5. What influenced the extent and quality of implementation?
6. What is the perception of the intervention amongst people implementing it and / or receiving the intervention? 

• Do they think it is practical and effective? 
• Do they want to carry on engaging with the intervention?
• How effective do stakeholders think the intervention is? Why is this?

7. What could be done differently to increase the effectiveness of the intervention? 

Typical data and information collection methods

• Intervention planning documentation, such as description of the intervention and underpinning assessment.
• Data on the volume of work done, such as how many patrols have been performed or how many school visits have been completed?
• Interviews, questionnaire surveys and workshops with people implementing the intervention.
• Interviews, questionnaire surveys and workshops with the target population, such rail passengers, residents and school pupils.
• Case studies of the intervention
• Observation or inspection of activities, such as observing an education session or inspection of a newly installed fence.

Typical questions asked in evaluation

1. To what extent has the number of incidents changed in the target location(s)?
2. To what extent has the number of incidents changed (i.e. more so in the target location) when compared to other similar areas lacking this intervention?
3. Is there evidence that the impact will be sustained?
4. Was the intervention cost-effective?
5. To what extent has there been a change in outcomes, such as attitudes, knowledge, capacity and behaviours?
6. What influenced the impact of the intervention?
7. Were there unintended impacts?

Typical data and information collection methods

• Incident data: Before and after comparison of incident data (SMIS) in the target location compared to trends in other areas (without the intervention).
• Data on other factors, such as number of passengers, changes in train throughput and changes in local crime levels, which may have influenced the impact of the intervention.
• Interviews, questionnaire surveys and workshops with people implementing the intervention.
• Time and cost data: Cost of materials, amount of time involved in implementation
• Schedule 8 payments data.

Typical questions asked in evaluation

1. What processes helped achieve success?
2. What were the barriers to implementation and impact?
3. What could be done differently to increase the effectiveness of the intervention?

Typical data and information collection methods

Same data as for process assessments.

Typical questions asked in evaluation

1. How does the cost-effectiveness of this intervention compare to other interventions?
2. How feasible and worthwhile is it to implement this intervention elsewhere?
3. What needs to be done to secure the legacy impact of this intervention?
4. What could be done differently to increase the effectiveness of this type of intervention in the future?

Typical data and information collection methods

Same data as for process and impact assessments plus evaluation results for other interventions..