ECUR 809 Assignment #1

PLEA Evaluation Report

I first heard about PLEA when I applied to work as a teacher in the Daughters and Sisters program early in my teaching career.  The Daughters and Sisters program serves young girls that are recovering from substance addictions. The girls receive an intense treatment program which includes detox, counselling and education. I was offered an interview but unfortunately, I had already decided to accept another teaching position and had to turn it down.  I have always wondered what teaching there would have been like.

Starting in 2008, and continuing over a three year period, an external evaluation of the PLEA programs was performed by the McCreary Centre Society (MCS) in partnership with Douglas College and funded by Justice Canada.  Tools used in the data collection included surveys of participants, a small focus group of participants from one of the programs, and phone surveys of the caregivers. The ethics of the project were examined by Douglas College's Research Ethics Board and the evaluation proceeded with the care and propriety required when dealing with young people.

MCS used a repeated measure design, where the same youth were surveyed three times: prior to intake, at the completion of the program and a short time after. It should be noted that the number of youth surveyed decreased from intake, where 261 were surveyed, to post completion, where only 105 youth were surveyed.  Qualitative information collected from a focus group was presented anecdotally throughout the report.

This report outlined a monitoring and outcome evaluation that provided a summative assessment of the program.  As a result of the inquiry by MCS, some formative feedback was collected.  This feedback may have an impact on the development of the program, but the primary purpose of the evaluation was to comment or pass judgement on the efficacy of the program.   It was determined that PLEA was indeed serving their intended population of high risk youth and that they were able to make a positive change in the behaviours of these youth.

The evaluation had many strengths: it was very thorough; it met the needs of the PLEA administration who initiated the process; having an outside agency perform the evaluation prevented bias; and having the support of the local collage students helped to make the project economically feasible.  

However, a weakness in the evaluation was created by the low completion rates of the second and third surveys.  Only 40% of the youth initially surveyed completed the final survey; it is possible that the youth that did not complete the survey did not present improved behaviour. The completion of the final surveys by less than half of the participants could have led to inaccurate conclusions. Although the evaluation answered the question of program efficacy in the short term, the conclusions drawn were generalized, which left me wondering about the actual long term effects of the program.

The very positive way in which the evaluation report was written left me once again wondering what it would have been like if I had interviewed for and accepted the position working with the PLEA Daughters and Sisters program.  The impact of the program in the lives of young girls struggling with addictions was evident.  Then again, the low rate of survey completion speaks to one of the challenges of being involved with "high risk" youth.