bc.tobaccofacts Evaluation Summary
Evaluation of Grades 4 to 7 Resources
About bc.tobaccofacts
bc.tobaccofacts is a set of tobacco prevention resources for Kindergarten to Grade 12 teachers, intended to help keep youth tobacco-free. Many students, teachers and health professionals guided the development of these resources. The Ministry of Health and the Ministry of Education collaborated to ensure that the resources connected to the education curricula being taught throughout British Columbia.
The Grades 4 to 7 bc.tobaccofacts program consists of four individual binders written at the intellectual and interest level of each age level. Each binder contains a series of five or six 30 to 90 minute lessons leading to an event to celebrate the students’ learning. Everything required to teach the lessons; background information, student handouts, overhead transparencies and evaluation strategies is included.
" We had to make some modifications
because of time constraints
and class composition. " Teacher
Purpose of Evaluation
The purpose of the evaluation of the Grades 4 to 7 bc.tobaccofacts was to document how the resources were being used in the schools, how teachers and students viewed the efficacy of the resources, and what suggestions teachers and students had to keep the resource viable.
" The kids never once said, 'Oh, no!'
And they couldn't stop talking
about the lessons. " Teacher
Evaluation Methodology
Both qualitative and quantitative methods were used in the evaluation. They included:
118 completed teacher surveys,
27 structured telephone interviews with teachers,
7 unstructured face-to-face interviews with teachers, and
5 student focus groups.
" ...my daughter is in grade 6
and has recently done the lessons. She does not generally do well
in school (she's got attitude!)
but this made
a huge impression on her - to the point of quoting
facts to our
smoking friends and relatives! " Teacher
Questions Addressed In This Evaluation:
1. Is there a logical link between the mission, objectives, activities and expected outcomes of the program?
2. Does the program delivery conform to the program design?
3. Is the resource being used in a comprehensive way in elementary schools?
4. Do teachers perceive the resource meets their needs and the needs of their students?
5. What is the student reaction to the program?
6. What is the perception of the students about how effective the program is in keeping them from using tobacco?
7. What suggestions do students/teachers have to improve the program or implementation of the program?
" I don't think I would change
anything except supplement with video activities.
I still can't
teach the whole thing at a time. " Teacher
Key Findings
Results suggest that teachers and students believe the decision to use tobacco is very complex, but that the activities in bc.tobaccofacts provide children with the knowledge and skills to assist them to remain tobacco-free. They made several suggestions regarding improvements to the program when it is revised: shorten the lessons, update statistics and advertisements, and produce a video to support the program.
Based on the program logic model, constructed from program materials, it is concluded that the activities address the objectives outlined in the resources and reflect what is currently known from research and theory about best practices in tobacco prevention.
Teachers liked the program, but one of the biggest challenges to program implementation related to program fidelity. Results of this evaluation suggest that a substantial number of teachers did not implement the program as intended, leaving out parts of lessons or not using the culminating activity designed to celebrate students’ learning. Data from the written surveys, telephone interviews and face-to-face interviews all agreed that time was the greatest factor influencing teachers not using the program as it is written.
Students are very enthusiastic about the program. Not only could they remember a great deal about what they were taught without any prompting, but also they shared their knowledge outside the classroom. This led to an unintended consequence of some family members and friends actually quitting smoking, cutting down on the amount of cigarettes they smoked or smoking outside the house, as reported by students.
Written Survey Statistics
91.5% of teachers indicated that they would use the resource again.
93.3% of teachers would recommend the resource to other teachers.
94.9% of teachers rated the quality of the materials as good to excellent.
95% of teachers indicated that the resource was age-appropriate.
86.4% of teachers rated the resource either excellent or good in terms of cultural appropriateness.
91.5% of teachers rated the relevance of the resources to the curriculum as good to excellent.
97.4% of teachers answered ‘yes’ when asked if their students enjoyed the program.
94% of teachers rated their students’ knowledge of the materials good to excellent after teaching the tobacco prevention unit.
It is important to keep in mind when interpreting and using these data that for a variety of reasons the response rate to the written surveys was low, at only 23.6%
" I talked to my sisters and my family about everything we were learning in school and my mom quit smoking." 10-year-old girl
Recommendations
1. In marketing the program, emphasize the curriculum connections in the resources. If teachers realize they are meeting prescribed learning outcomes by using bc.tobaccofacts, the “lack of time” issue is much less likely to be a barrier.
2. Emphasize the key activities that are required at each grade level in future program orientations to ensure learning outcomes are achieved.
3. Update statistics and advertisements to ensure the program is current and consider supplementing the written resource with a tobacco prevention video.
>>>> Barb Tarbox