Program Evaluation
Evaluation Proposal
SAER 8370
[University of Houston]
[College of Education]
[October 31, 2013]
Katina Thomas
Elizabeth Lee
TABLE OF CONTENTS
Abstract #
Summary
Synopsis of Ark Program
Description of Project
Central Research QUestions
Explanation of how propose work fits within goals outlined by primary stakeholders
The relationship of proposed research to the literature
Brief summary of relevant literature
Introduction
The Importance of the ARK Program
Definition of student-teacher relationship #
Qualities of teachers to form positive teacher-student relationships
Behavior and student-teacher relationships
The far reaches of student-teacher relationships
Knowledge expected to result from proposed research
Conceptual framework
Establishing broad goals or objectives
Classifying the goals or objectives
Defining objectives in behavioral terms #
Finding situations in which the achievement of objectives can be shown #
Developing measurement technique
Collecting performance data
Comparing performance data with the behaviorally stated objectives
Research methods
Data collection schedule
Data collection instrument
Personal inventory ARK survey
Anecdotal records
Participant-oriented observations
Focus groups
Teacher self-evaluation form
Modes of analysis
Principal investigators
Principal investigators
Support team
University of Houston Student Teaching Directors
Student Teaching Field Supervisors/ARK Facilitators
Data Analysts
Budget
References
Abstract
The objective of the ARK program evaluation is to ensure that the program is meeting the needs of student-teachers to its fullest ability. The objective of this evaluation is to modify current lessons if necessary so that they address the needs of student-teachers revealed through situation circle discussions. Meeting that objective means that after completing the ARK program through the University of Houston the student-teachers will utilize the skills taught during the ARK group meetings in forming positive relationships with their students.
Summary The purpose of this evaluation is to ensure that the ARK program is meeting the needs of student-teachers to its fullest ability. Through modification of current lessons to address the needs of student-teachers revealed through situation circles, the ARK program can better ensure that after completing completion of the program the student-teachers at graduating from the University of Houston will utilize the skills taught during the ARK group meetings. By utilizing the skills taught through the program the student-teachers will understand the importance of building positive relationships with their students. The data for this evaluation were obtained through mixed-methods analysis employing both quantitative and qualitative techniques for measurement. A pretest/posttest will be used to analyze the influence of ARK. Observations of the ARK situation circle discussions will be employed in assisting with measurement. Anecdotal records, emails from ARK facilitators, participate-oriented observations, and student-teacher focus groups will also be analyzed for evidence. References for further information concerning the ARKGroup can be found at http://www.thearkgroup.org/.
Synopsis of ARK Program
The ARKGroup was established in 1992 as 501(c)(3) Texas non-profit organization. The ARKGroup offers programs to teachers, parents, college students, teens, kids, divorced parents, and childcare providers. ARK stands for Adults Relating to Kids. The goals of the ARK program as stated in their program guide are, to teach and model the revolutionary power of unconditional love and respect, to build better families and classrooms, to reduce juvenile and adult violence by improving self-esteem, and to make ARKparenting the norm for the 21st-century school, family, church, and community. The ARK program is provided in various venues including public-private and alternative schools, juvenile Justice programs, churches, prisons, home shelters, women’s shelters, childcare centers, and corporations.
The ARK program is grounded on the belief that teachers and students benefit from student-teacher relationships based on an established foundation of love, respect, and relationship in the classroom.
Description of Project
The purpose of this project is to help the ARK program reach its fullest potential in addressing concerns and questions of student-teachers through the combined use of lessons and situation circles.
One goal of this evaluation is to examine the ARK program lessons to determine whether or not they are addressing questions and concerns asked by student-teachers during situation circles.
Situation circles are an allotted amount of time given to the student-teachers during an ARK group meeting during which they have the opportunity to ask questions to fellow student-teachers and a situation circle facilitator. Situation circles often bring up challenging, emotionally charged, confrontational, or otherwise harder questions that student-teachers may not feel equipped to handle. Student-teachers may not feel that some of the situations that they are encountering in their student teaching experience are being addressed in their preparatory classes. They may also feel uncomfortable asking a coworker or their cooperating teacher for advice. For example, during a situation circle a student-teacher may ask for advice on dealing with a student through a detailed account of a particular situation and they are giving feedback from members of the circle.
Another goal of this evaluation is to determine whether or not the ARK lessons are helping teachers create relationships with their students in their classrooms.
The ultimate goal of helping the ARK program improve or feel confident about proceeding in its existing format can be examined through asking the following type of questions:
1. Are the ARK lessons centered around topics that truly help student-teachers understand how to foster love, respect, and relationship in their classrooms?
2. From the help of the ARK program, do student-teachers feel they have the power and knowledge to create relationships with their students?
3. When difficult situations arise do they feel empowered through these relationships to establish steps to take to address the situations?
4. Are the ARK lessons performed acting as catalysts for questions and concerns presented in situation circles?
5. Likewise, did the ARK lesson help the student in any way concerning this question?
6. Are the lessons correlating with the majority of the concerns and questions on the minds of these student-teachers?
7. Through preparation on building relationships, are the ARK lessons assisting student-teachers in confidently and knowledgeably addressing situations they are encountering in their field experience and bringing to light during situation circles?
The overarching goal of this project is to take such questions into consideration in order to help the ARK program modify the existing program or to feel confident in proceeding as is.
Central Research Questions
1. To what extent are student-teachers utilizing the content from the ARK lessons in their student teaching classrooms in order to help them build relationships with their students?
2. To what extent are the ARK curriculum expectations and objectives aligned with the topics that are discussed in the small group segment of their ARK sessions?
Explanation of How Proposed Work Fits Within Goals Outlined By Primary Stakeholders In a meeting held on September 9, 2013 the primary stakeholders sat with the evaluation creators to discuss goals for the ARK program and for the evaluation. The primary stakeholder expressed a desire to make the ARK program better. He stated that a teaching toolkit consists of three main components, curriculum, pedagogy, and relationships, and that the ARK program’s aim was to focus on improving the latter. The following is a statement made by the primary stakeholder concerning the goal of the program. “The idea for teachers to think about is how important is the relationship factor to them in their classroom? I’d like this program to help increase awareness of the importance of relationships. Then there are two questions. There is the why and there is the how.” This evaluation aims to help answer those questions by taking into consideration the expressed concerns and questions of the student-teachers participating in the program. This evaluation will examine the extent to which ARK lessons are being used by student-teachers to help them build relationships in their classrooms. It will also look at whether or not the program is meeting the needs of the student-teachers through the structural alignment of lessons and situation circle topics, questions, and concerns.
The Relationship of Proposed Research to the Literature
With the teacher retention rate as low as it is researchers continue to study the causes of teacher turnover. This area of research lays the foundation for educational reform based on the needs of teachers.
According to an Article published by the National Education Association, the National Center for Education Statistics (NCES) investigated and published reasons they found for high teacher turnover rates. Based on interviews conducted by the NCES with more than 7,000 current and former teachers, they found the following to be the top reasons: accountability mandates for standardized testing, a lack of teacher community and support, a lack of funding, low salary, a lack of influence or respect in the field, and a lack of skills or support concerning student discipline.
This literature review contributes previous research related to the needs of teachers. The needs addressed in this research are centered upon the effects of student-teacher relationships and how those relationships help empower both the student and the teacher.
Brief Summary of Relevant Literature
Introduction
There is a variety of research that covers the main ideas presented in the four main sections of the ARK program lesson guide. These sections lie under the categories of: self-esteem and unconditional love, classroom environment, understanding the dynamics of student behavior, and enabling students to monitor their actions.
This literature review will cover the importance of the ARK program with regard to current educational needs based on research covering the effects of student-teacher relationships and how those relationships help empower both the student and the teacher.
The Importance of the ARK Program
The ARK program’s foundational basis is set upon the idea of informing teachers about the importance of relationships with their students. A sizable literature provides evidence that strong and supportive relationships between teachers and students are fundamental to be healthy development of all students in schools (Hamre & Pianta, 2001). Student-teacher relationships are important during the early years for transition into the school setting. The impact of student-teacher relationships in the early years follows students for years to come. They continue to be important throughout elementary, intermediate, and high school and extends far beyond academic success.
Definition of Student-Teacher Relationship
The definition of a student-teacher relationship is not a professional relation between instructors and those they instruct. Positive student-teacher relationships are characterized by open communication as well as emotional and academic support that exist between students and teachers (Pianta, 1999).
Qualities of Teachers who Form Positive Student-Teacher Relationships Teachers’ abilities to form positive relationships with students are impacted by their personality type, experiences, and the quality of their own personal relationships (Baker, 2006). Teachers who engaged in frequent social conversation with students, made themselves available to students who were struggling, displayed regard for students perspectives and ideas, used behavior management strategies that clearly communicated expectations and caring, and took time to nourish the social emotional development of students displayed the most positive student-teacher relationships (Hamre & Pianta, 2006). Furthermore, teachers who learned about the personal interest, backgrounds, culture, and religion of their students were more effective at building positive relationships with them.
Behavior and Student-Teacher Relationships The dynamic of behavior on the part of the student and the teacher play a very important role in the development of a positive relationship. The ARK program devotes a great deal of its lessons to equipping teachers with strategies on how to positively interact with students who are exhibiting various forms of difficult behaviors. These lessons include: Understanding the Misbehaving Student, Dealing with An Attention-Seeking Student, Handling a Rebellious Student, Working with a Power Seeking Student, and Disciplining with Natural and Logical Consequences.
One study out of University of Virginia provided four key elements of a well-designed behavior management system. These key elements include: providing clear limits in tolerances that help regulate students behavior, reinforcing the idea that teachers will respond in expected and fair ways, creating opportunities to give students positive feedback about their behavior, and implementing the behavior system in a way that communicates care and respect of students (Pianta, 1999).
The Far Reaches of Student-Teacher Relationships
Positive student-teacher relationships extend far beyond success in academics. The positive student-teacher relationship affects adaptation to school regardless of gender or grade level across the elementary age range (Baker, 2006). They view school as a positive experience, and as a result, exhibit fewer behavior difficulties and display better social skills (Buyse et al., 2009). They are also more active participants in class, express a greater interest in college, and maintain higher grade point averages (Hallinan, 2008).
Jones (2008) proposes that school leaders should even use the term relationship building in lieu of classroom management to shift the thinking away from the idea of managing students and toward the notion of collaborating with them.
The ARK program’s philosophy on forming close student-teacher relationships in order to create a more fulfilling and enriching experience for both the student and the teacher were established on laudable, credible and well researched philosophy. Actions taken with regard to that philosophy have proven to have positive effects on both the teacher and the student, as well as the entire school community. In the words of Teddy Roosevelt, “People don’t care how much you know until they know how much you care”.
New Knowledge Expected to Result from Proposed Research From this project we expect to gain insight on the strengths and weaknesses of the ARK program. We will work to gain an understanding about the level of effectiveness of the ARK program lessons and situation circles with regard to the needs of student-teachers. We aim to discover ways to help the art program become a program that addresses the needs of student-teachers by looking deeply at their questions concerning their relationships with their students. By analyzing their questions and concerns we hope to help the ARK program facilitate lessons and situation circles that play an instrumental role in the development of teachers. The ultimate goal of evaluating this program is to help ARK contribute to the growth and production of teachers who understand the value of love, respect, and relationship in their classrooms, by which creating and environment whereby both the student and the teacher have access to reaching their highest level of social, emotional, and academic competence.
Conceptual Framework
The ARK program will be evaluated using an objective-oriented approach. Incorporating a Tylerian Model will determine a relationship between the goals of the program and its outcomes. The purpose is to conduct a formative evaluation to determine the quality of the teacher preparation program and if there are any characteristics within the lessons that aid in the achievement of ARK’s goals. This is designed as an internal evaluation that will be conducted by two doctoral student members of the University of Houston staff who serve in the roles of Student Teaching Field Supervisors and ARK Facilitators. Discrepancies between performance and objectives would lead to modifications intended to correct the deficiency, and the evaluation cycle would be repeated (Fitzpatrick, Sanders, & Worthen, 2011). The framework for conducting an evaluation founded on the Tylerian Model will include: (1) establishing broad goals or objectives, (2) classifying the goals or objectives, (3) defining objectives in behavioral terms, (4) finding situations in which achievement of objectives can be shown, (5) developing measurement techniques, (6) collecting performance data, and (7) comparing performance data with the behaviorally stated objectives. The primary stakeholders in the evaluation design are the ARK creator, ARK Liaison, and the Student Teaching Directors from the University of Houston. Participants include the Student Teaching Field Supervisors who serve as ARK Program Facilitators and the student-teachers at the University of Houston. Although outcomes and any discrepancies will be examined following the completion of program implementation, the evaluation will be guided by goals established by the ARK creators and University of Houston Student Teaching Directors prior to program engagement.
Establishing broad goals or objectives
The broad goals of the ARK program were established by the creator of ARK prior to its implementation. ARK’s goals are listed in the instructional books for teachers and support staff that accompany the video lessons. The program’s two broad goals include (1) tools for working with the kids; and (2) a support network for fellow teachers (Wilkerson, 2007). These goals are based on Glasser’s Choice Theory that promotes the idea that the most important need is love and belonging, as closeness and connectedness with the people we care about is a requisite for satisfying all of the needs. The goals of the program are also based on the assumption that the student-teacher has had no prior training in building relationships with students, and need formal training in building positive relationships with students.
Classifying the goals or objectives
Through dialogue with the ARK creator, ARK Liaison, and university Student Teaching Directors, the evaluation team was able to determine a specific goal for the ARK program at the University of Houston. The specific goal is to equip student-teachers with strategies for building and maintaining positive relationships with students prior to entering the teaching profession full time.
Defining objectives in behavioral terms
The objectives for evaluation were devised collaboratively between ARK creators and the University of Houston Student Teaching Directors. Stated behaviorally, the objectives are: (1) that all students that completed the student teaching program at the University of Houston (Quest 3 and Student Teaching Parts 1 and 2) during Fall 2013 and Spring 2014 will utilize the skills taught during the ARK sessions during their student teaching assignments, and (2) that all students that completed the student teaching program at the University of Houston (Quest 3 and Student Teaching parts 1 and 2) during Fall 2013 and Spring 2014 will utilize the skills taught during ARK sessions in their classrooms once they become a classroom teacher full time.
Finding situations in which achievement of objectives can be shown
Evidence of achievement of Objective 1 can be shown can be found through observation of the small group segment of the ARK sessions when student-teachers discuss techniques used while student teaching in the field, and from suggestions from their peers for resolving situations within the classroom. Evidence of achievement can also be found during observations of the student-teachers in their classrooms by their Student Teaching Field Supervisors. Additional evidence can also be found through the summaries of ARK sessions that are emailed to the ARK creators from the ARK Facilitators. Situations in which achievement of Objective 2 can be found will be limited to surveys of the student-teachers and focus groups after the completion of the program.
Developing measurement techniques
The evaluation design will employ a mixed method of quantitative and qualitative techniques for measurement. Quantitatively, a pretest/posttest method of existing surveys developed by the ARK Corporation will be used to analyze the influence of ARK. Surveys will be administered to all student teaching participants before the initiation of ARK lessons, and a new survey will be administered to them following the conclusion of the last ARK lesson. Pretest surveys will be used to assist in measuring the achievement of Objective 1 and posttest surveys will be used to assist in measuring the achievement of Objective 2. Existing surveys that were developed by the ARK Corporation will be administered as the pretest and posttest surveys following the conclusion of the ARK curriculum.
Qualitatively, participant observations of the ARK small group discussions will be employed to assist in measuring the achievement of Objectives 1 and 2. Participant observations will occur as a result of the evaluation team also serving in the roles of Student Teaching Supervisors and ARK Facilitators. A chronological collection of anecdotal emails from the ARK facilitators summarizing small group discussions, in addition to focus groups of the student-teachers will support measurement of both Objectives 1 and 2.
Collecting performance data
Performance data will include: summaries of the ARK sessions from ARK facilitators, pretest and posttest surveys from student teaching participants, posttest surveys from ARK facilitators, participant observation notes of the ARK small group discussions, and records from the student teaching focus groups. ARK session summaries and ARK facilitator surveys will be collected by the ARK liaison, pretest and posttest surveys will be collected by ARK facilitators, and observation and focus group records will be collected by the evaluation team.
Comparing performance data with the behaviorally stated objectives
After all data has been collected, the evaluation team will collaborate with the ARK creator, ARK Liaison, and University of Houston Student Teaching Directors to analyze and compare the outcomes of program implementation with the following objectives: (1) that all students that completed the student teaching program at the University of Houston (Quest 3 and Student Teaching Parts 1 and 2) during Fall 2013 and Spring 2014 will utilize the skills taught during the ARK sessions during their student teaching assignments, and (2) that all students that completed the student teaching program at the University of Houston (Quest 3 and Student Teaching parts 1 and 2) during Fall 2013 and Spring 2014 will utilize the skills taught during ARK sessions in their classrooms once they become a classroom teacher full time.
Research Methods A mixed approach of quantitative and qualitative methods will be employed within a Tylerian evaluation design to compare the outcomes of the ARK program with the objectives that were collaboratively determined by the ARK Corporation and University of Houston Student Teaching Directors. Data collected during implementation of the ARK lessons will be used to measure how the participant outcomes compared to the specific objectives established by the primary stakeholders. The data will also be examined for discrepancies within the components of the program that may have influenced its outcomes. A pretest/posttest design of survey data will be used to compare the use of ARK strategies by student teaching participants before ARK lessons to the use of ARK strategies by student teaching participants after the completion of ARK. Anecdotal emails from ARK Facilitators, participant-oriented observations, and student-teacher focus groups will be analyzed for evidence of discrepancies.
The student-teachers will serve as the participants that will be examined in the ARK program. These are under graduate and Master’s level graduate students who have met university guidelines to student teach in a school within the Region IV area of Texas to complete graduation requirements on their degree plan. There are approximately 500 student-teachers, and these student teaching participants were pre-determined by the university Student Teaching Department. Three days during the Fall semester and three days during the Spring semester have been designated for student-teachers to return to their university campus to engage in two consecutive hours of ARK lessons.
An ARK lesson will begin with a choral reading of expectations known as “ARK Covenants.” An 8-10 minute lecture about a specific relationship building topic is then broadcast on a DVD. Students can also follow along in an ARK book that was distributed during their student teaching orientation. The 8-10 minute lecture is viewed by approximately 60 students at a time. Each group is assigned according to student teaching assignment. Participants that are completing a K-6 teaching assignment are assigned to a room that will broadcast the elementary ARK curriculum, and participants that are completing a grade 7-12 assignment are assigned to a room that will broadcast the secondary ARK curriculum.
Following the video, students divide into groups of approximately 16-18 students to engage in a small group discussion, known as a Situation Circle. Groups were predetermined by the ARK Liaison and are randomly divided within their teaching category (elementary education or secondary education). Small group sessions last for 45-55 minutes and are facilitated by a Student Teaching Field Supervisor that serves as an ARK Facilitator. During Situation Circle, participants are guided by the ARK Facilitator to give feedback regarding the 8-10 video presentation. Facilitators also allow time for participants to discuss any concerns that they may have regarding relationships within their student teaching assignment. Once a participant has expressed a specific situation that brings concern, other participants are allowed to ask clarifying questions, and participants also take turns around the group circle offering suggestions to resolve the concern. While suggestions are being offered, one participant is transcribing the suggestions for them.
At the close of the ARK session, the small group will choral read the ARK Covenants to conclude the lesson. To maximize the amount of time that the participants are available, the Student Teaching Directors have scheduled two ARK lessons to be completed consecutively for two hours on each of the six scheduled days.
Student Teaching Field Supervisors and Directors from the University of Houston received ARK Facilitator Training from the ARK Corporation regarding the best practices for implementing the curriculum to student-teachers. The professional development session consisted of four hours of modeling and discussion of strategies for facilitating lessons and situation circles with student-teachers. University staff also received individual copies of ARK materials, which included: one ARK Facilitator Guide, one ARK Lesson Guide, one set of ARK Covenants, a DVD of Elementary ARK lessons, and a DVD of Secondary ARK lessons.
Prior to the beginning of the first video in the first ARK session, each ARK Facilitator will administer a survey designed by the ARK Corporation to their participants before the ARK Covenants are read. These surveys are found in the ARK Facilitator training manuals. ARK Facilitators will also email the ARK Liaison following each of the six sessions with a brief summary of the topics discussed in their small group circles. During the third ARK session, the evaluation team will engage in a participant observation of a randomly selected elementary ARK small group and a randomly selected secondary ARK small group to note evidence of situations that illustrate achievement of the objectives. Following the fifth ARK session, eight members from one elementary ARK small group will be randomly selected for a focus group and eight members from one secondary ARK small group will be randomly selected for a focus group and interviewed by the evaluation team for additional evidence of performance that can illustrate achievement of the objectives being measured. At the conclusion of the final ARK session, ARK Facilitators will administer a different survey designed by the ARK Corporation to student teaching participants regarding feedback about the ARK program.
Data Collection Schedule
Method
Timeframe
Responsible Party
Personal Inventory ARK Survey
First ARK session
ARK Facilitators
Anecdotal Summary/Emails
Within three days following each ARK session
ARK Facilitators
Participant-oriented Observations (2)
Third ARK session
Internal Evaluators
Focus Groups (2)
Immediately following the fifth ARK session
Internal Evaluators
Teacher Self-Evaluation Form
Sixth ARK session
ARK Facilitators
Data Collection Instruments The following data collection instruments will be used for program evaluation: a Personal Inventory ARK Survey, anecdotal records, participant-oriented observations, two focus groups, and a Teacher Self-Evaluation Form. Each instrument is expected to yield data that can be measured and compared to performance objectives.
Personal Inventory ARK Survey
A pre-program survey developed by the ARK Corporation is designed to inventory the extent of training and experiences that student teaching participants have had regarding self-perception and relationship building. The survey lists fifteen statements and offers multiple choices for each statement in the format of a Likert scale. Answer choices include: Strongly Agree, Agree, Uncertain, Disagree, and Strongly Disagree.
Anecdotal Records
Brief summaries outlining the topics discussed during each ARK Facilitators’ lessons will be emailed to the ARK Liaison no more than three days following each of the six ARK days. These emails will serve as anecdotal records that will list general feedback from student teaching participants regarding the DVD video lessons, as well as any concerns that were discussed in Situation Circles.
Participant-oriented Observations
During the third ARK session, the two members of the evaluation team that also serve as ARK Facilitators will observe one ARK lesson from a randomly selected elementary ARK group during the first hour of instruction and one ARK lesson from a randomly selected secondary ARK group during the second hour of instruction. Notes will be taken regarding verbal cues, verbal interactions, and nonverbal cues.
Focus Groups
Following the fifth set of ARK sessions, the members of the evaluation team that serve as ARK Facilitators will randomly select eight student teaching participants from elementary ARK groups and conduct a focus group to extract feedback regarding the influence that ARK has had in their current classroom assignments and on their strategies for building student relationships when they enter the teaching profession full time. A similar focus group will also be constructed, consisting of eight randomly selected student teaching participants from secondary ARK groups. Leading interview questions will be pre-determined by the evaluation team and asked to both focus groups. Each group will be led by both members of the evaluation team, and will last for 30-45 minutes each.
Teacher Self-Evaluation Form
At the conclusion of the sixth scheduled ARK session, ARK Facilitators will administer a Teacher Self-Evaluation Form to student teaching participants to complete regarding their feedback about the effects that ARK has had on their personal and professional relationship building skills. Developed by the ARK Corporation, the Teacher Self-Evaluation Form has one section that consists of a survey with multiple answer choices numbered from one to three. Student-teachers are expected to choose one answer choice that most closely matches their belief. The second section of the form is a questionnaire consisting of three open-ended questions that encourage participants to elaborate in detail on the influence that ARK has had on their attitudes towards the students in their classroom assignments, their existing relationships with those students, and the behaviors of those students.
Modes of Analysis To analyze the quantitative data, a pretest/posttest will be employed to compare the performance of student teaching participants in their teaching assignments prior to ARK and following ARK. It has yet to be finalized which specific data analysis (sample t-test, ANOVA) will be used to determine the exact outcome. Qualitative data will be analyzed by categorizing feedback from the anecdotal records, participant-oriented observations, and focus groups; however the most efficient form of data analysis has yet to be determined.
Principal Investigators The principal investigators responsible for implementing the evaluation design will be two doctoral students from the University of Houston. The principal investigators will be conducting an internal evaluation, and they also serve in the capacities of Student Teaching Field Supervisors and ARK Facilitators. Supporting the principal investigators include: the ARK Liaison, the two university Student Teaching Directors, Student Teaching Field Supervisors who also serve as ARK Facilitators, and two data analysts. Ultimately, the principal investigators will be collaborating with those that serve as the primary stakeholders to measure and compare outcomes to the stated objectives. The primary stakeholders include: the ARK Corporation and the University of Houston Student Teaching Directors.
Principal Investigators
Two doctoral students will serve as the principal investigators throughout the evaluation. They also serve dual roles within the ARK program--as evaluators and as ARK Facilitators. The principal investigators will be responsible for conducting the participant-oriented observation of the two ARK Situation Circles during the third ARK session. The principal investigators will also use ethnographic methods to analyze the data collected from the observations. Conducting the focus group interviews will also be an additional responsibility for the principal investigators. They will design the leading interview questions so that they are open-ended and aligned with the information that is needed to be collected. The evaluators will randomly select eight elementary student teaching participants and eight secondary student teaching participants for the two focus groups.
Support Team
ARK Liaison
The ARK Liaison, who is a primary stakeholder, will assist in data collection. She will archive the emails from ARK Facilitators that summarize topics and feedback that student teaching participants share during their Situation Circles. These emails will be given to the principal investigators for analysis and measurement.
University of Houston Student Teaching Directors
The two student teaching directors, who are also primary stakeholders, will be responsible for corresponding with the ARK Facilitators to send reminders of the appropriate times to collect data according to the proposed schedule. The Directors will also be responsible for contacting the student teaching participants to inform them of upcoming ARK sessions, as well as contacting the randomly selected participants for the focus groups. They will also collect the completed Personal Inventory ARK Surveys and Teacher Self-Evaluation Forms from the Student Teaching Field Supervisors/ARK Facilitators following ARK Session 1 and ARK Session 6.
Student Teaching Field Supervisors/ARK Facilitators
The primary responsibilities of the field supervisors/facilitators will be to lead the ARK lessons, and administer the surveys. They will be expected to administer the Personal Inventory ARK Survey to the student teaching participants in their assigned ARK groups prior to the first ARK lesson. Following the final ARK lesson, the field supervisors/facilitators will also be expected to administer the Teacher Self-Evaluation Form to the student teaching participants. They will collect each survey immediately after completion and turn them in to the Student Teaching Directors. Following each of the six sessions, the field supervisors/facilitators will also be expected to email the ARK Liaison a brief summary of the feedback and topics discussed during each lesson within three days of the meeting.
Data Analysts
There will be two data analysts. Each analyst will code the Personal Inventory ARK Survey into the Statistical Packages for Social Sciences program for categorization and analysis of the results. The two reports will be compared by the analysts for discrepancies with the results, and corrected within the program as needed. The same approach will be utilized for analysis of the Teacher Self-Evaluation Form.
Budget
Staffing
Cost-1,000.00 per evaluator
At a cost of $1,000.00 per evaluator the full evaluation including follow up meetings will cost $2,000.00 total.
Materials and Supplies
Cost- $100.00
This cost covers printer paper, printer ink, binders, folders, and staples for the evaluation, surveys, and anecdotal notes.
Travel
Cost-$50.00
This cost covers gas to and from meetings with stakeholders and student-teachers.
References
Baker, J. A. (2006). Contributions of teacher-child relationships to positive school adjustment during elementary school. Journal of School Psychology, 44(3), 211-229.
Buyse, E., Verschueren, K., Verachtert, P., & Van Damme, J. (2009). Predicting school adjustment in early elementary school: Impact of teacher-child relationship quality and relational classroom climate. Elementary School Journal, 110(2), 119-141. Retrieved from http://dx.doi.org/10.1086/605768
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Boston: Pearson.
Hamre, B., & Pianta, R. (2001). Early teacher–child relation-ships and the trajectory of children’s school outcomes through eighth grade. Child Development, 72, 625–638.
Hallinan, M. T. (2008). Teacher influences on students ' attachment to school. Sociology of
Education, 81(3), 271-283. Retrieved from http://dx.doi.org/10.1177/003804070808100303
Jones, R. D. (2008). Strengthening Student Engagement. [White paper.] Retrieved from http://www.leadered.com/pdf/Strengthen%20Student%20Engagement%20white%20paper.pdf Kopkowski, Cynthia (2008, April). Why They Leave. NEA Today Magazine. Retrieved from http://www.nea.org/home/12630.htm Pianta, Robert C. Enhancing relationships between children and teachers. Washington, D.C.:
American Psychological Assn., 1999.
Wilkerson, B. G. (2007). Adults relating to kids for teachers and staff (elementary level). The
Houston: The ARKGroup, Inc.
References: Baker, J. A. (2006). Contributions of teacher-child relationships to positive school adjustment during elementary school Buyse, E., Verschueren, K., Verachtert, P., & Van Damme, J. (2009). Predicting school adjustment in early elementary school: Impact of teacher-child relationship quality and Hamre, B., & Pianta, R. (2001). Early teacher–child relation-ships and the trajectory of children’s school outcomes through eighth grade Hallinan, M. T. (2008). Teacher influences on students ' attachment to school. Sociology of Education, 81(3), 271-283 Jones, R. D. (2008). Strengthening Student Engagement. [White paper.] Retrieved from http://www.leadered.com/pdf/Strengthen%20Student%20Engagement%20white%20paper.pdf Kopkowski, Cynthia (2008, April). Why They Leave. NEA Today Magazine. Retrieved from http://www.nea.org/home/12630.htm Pianta, Robert C. Enhancing relationships between children and teachers. Washington, D.C.: American Psychological Assn., 1999 Wilkerson, B. G. (2007). Adults relating to kids for teachers and staff (elementary level). The Houston: The ARKGroup, Inc.
SAER 8370
[University of Houston]
[College of Education]
[October 31, 2013]
Katina Thomas
Elizabeth Lee
TABLE OF CONTENTS
Abstract #
Summary
Synopsis of Ark Program
Description of Project
Central Research QUestions
Explanation of how propose work fits within goals outlined by primary stakeholders
The relationship of proposed research to the literature
Brief summary of relevant literature
Introduction
The Importance of the ARK Program
Definition of student-teacher relationship #
Qualities of teachers to form positive teacher-student relationships
Behavior and student-teacher relationships
The far reaches of student-teacher relationships
Knowledge expected to result from proposed research
Conceptual framework
Establishing broad goals or objectives
Classifying the goals or objectives
Defining objectives in behavioral terms #
Finding situations in which the achievement of objectives can be shown #
Developing measurement technique
Collecting performance data
Comparing performance data with the behaviorally stated objectives
Research methods
Data collection schedule
Data collection instrument
Personal inventory ARK survey
Anecdotal records
Participant-oriented observations
Focus groups
Teacher self-evaluation form
Modes of analysis
Principal investigators
Principal investigators
Support team
University of Houston Student Teaching Directors
Student Teaching Field Supervisors/ARK Facilitators
Data Analysts
Budget
References
Abstract
The objective of the ARK program evaluation is to ensure that the program is meeting the needs of student-teachers to its fullest ability. The objective of this evaluation is to modify current lessons if necessary so that they address the needs of student-teachers revealed through situation circle discussions. Meeting that objective means that after completing the ARK program through the University of Houston the student-teachers will utilize the skills taught during the ARK group meetings in forming positive relationships with their students.
Summary The purpose of this evaluation is to ensure that the ARK program is meeting the needs of student-teachers to its fullest ability. Through modification of current lessons to address the needs of student-teachers revealed through situation circles, the ARK program can better ensure that after completing completion of the program the student-teachers at graduating from the University of Houston will utilize the skills taught during the ARK group meetings. By utilizing the skills taught through the program the student-teachers will understand the importance of building positive relationships with their students. The data for this evaluation were obtained through mixed-methods analysis employing both quantitative and qualitative techniques for measurement. A pretest/posttest will be used to analyze the influence of ARK. Observations of the ARK situation circle discussions will be employed in assisting with measurement. Anecdotal records, emails from ARK facilitators, participate-oriented observations, and student-teacher focus groups will also be analyzed for evidence. References for further information concerning the ARKGroup can be found at http://www.thearkgroup.org/.
Synopsis of ARK Program
The ARKGroup was established in 1992 as 501(c)(3) Texas non-profit organization. The ARKGroup offers programs to teachers, parents, college students, teens, kids, divorced parents, and childcare providers. ARK stands for Adults Relating to Kids. The goals of the ARK program as stated in their program guide are, to teach and model the revolutionary power of unconditional love and respect, to build better families and classrooms, to reduce juvenile and adult violence by improving self-esteem, and to make ARKparenting the norm for the 21st-century school, family, church, and community. The ARK program is provided in various venues including public-private and alternative schools, juvenile Justice programs, churches, prisons, home shelters, women’s shelters, childcare centers, and corporations.
The ARK program is grounded on the belief that teachers and students benefit from student-teacher relationships based on an established foundation of love, respect, and relationship in the classroom.
Description of Project
The purpose of this project is to help the ARK program reach its fullest potential in addressing concerns and questions of student-teachers through the combined use of lessons and situation circles.
One goal of this evaluation is to examine the ARK program lessons to determine whether or not they are addressing questions and concerns asked by student-teachers during situation circles.
Situation circles are an allotted amount of time given to the student-teachers during an ARK group meeting during which they have the opportunity to ask questions to fellow student-teachers and a situation circle facilitator. Situation circles often bring up challenging, emotionally charged, confrontational, or otherwise harder questions that student-teachers may not feel equipped to handle. Student-teachers may not feel that some of the situations that they are encountering in their student teaching experience are being addressed in their preparatory classes. They may also feel uncomfortable asking a coworker or their cooperating teacher for advice. For example, during a situation circle a student-teacher may ask for advice on dealing with a student through a detailed account of a particular situation and they are giving feedback from members of the circle.
Another goal of this evaluation is to determine whether or not the ARK lessons are helping teachers create relationships with their students in their classrooms.
The ultimate goal of helping the ARK program improve or feel confident about proceeding in its existing format can be examined through asking the following type of questions:
1. Are the ARK lessons centered around topics that truly help student-teachers understand how to foster love, respect, and relationship in their classrooms?
2. From the help of the ARK program, do student-teachers feel they have the power and knowledge to create relationships with their students?
3. When difficult situations arise do they feel empowered through these relationships to establish steps to take to address the situations?
4. Are the ARK lessons performed acting as catalysts for questions and concerns presented in situation circles?
5. Likewise, did the ARK lesson help the student in any way concerning this question?
6. Are the lessons correlating with the majority of the concerns and questions on the minds of these student-teachers?
7. Through preparation on building relationships, are the ARK lessons assisting student-teachers in confidently and knowledgeably addressing situations they are encountering in their field experience and bringing to light during situation circles?
The overarching goal of this project is to take such questions into consideration in order to help the ARK program modify the existing program or to feel confident in proceeding as is.
Central Research Questions
1. To what extent are student-teachers utilizing the content from the ARK lessons in their student teaching classrooms in order to help them build relationships with their students?
2. To what extent are the ARK curriculum expectations and objectives aligned with the topics that are discussed in the small group segment of their ARK sessions?
Explanation of How Proposed Work Fits Within Goals Outlined By Primary Stakeholders In a meeting held on September 9, 2013 the primary stakeholders sat with the evaluation creators to discuss goals for the ARK program and for the evaluation. The primary stakeholder expressed a desire to make the ARK program better. He stated that a teaching toolkit consists of three main components, curriculum, pedagogy, and relationships, and that the ARK program’s aim was to focus on improving the latter. The following is a statement made by the primary stakeholder concerning the goal of the program. “The idea for teachers to think about is how important is the relationship factor to them in their classroom? I’d like this program to help increase awareness of the importance of relationships. Then there are two questions. There is the why and there is the how.” This evaluation aims to help answer those questions by taking into consideration the expressed concerns and questions of the student-teachers participating in the program. This evaluation will examine the extent to which ARK lessons are being used by student-teachers to help them build relationships in their classrooms. It will also look at whether or not the program is meeting the needs of the student-teachers through the structural alignment of lessons and situation circle topics, questions, and concerns.
The Relationship of Proposed Research to the Literature
With the teacher retention rate as low as it is researchers continue to study the causes of teacher turnover. This area of research lays the foundation for educational reform based on the needs of teachers.
According to an Article published by the National Education Association, the National Center for Education Statistics (NCES) investigated and published reasons they found for high teacher turnover rates. Based on interviews conducted by the NCES with more than 7,000 current and former teachers, they found the following to be the top reasons: accountability mandates for standardized testing, a lack of teacher community and support, a lack of funding, low salary, a lack of influence or respect in the field, and a lack of skills or support concerning student discipline.
This literature review contributes previous research related to the needs of teachers. The needs addressed in this research are centered upon the effects of student-teacher relationships and how those relationships help empower both the student and the teacher.
Brief Summary of Relevant Literature
Introduction
There is a variety of research that covers the main ideas presented in the four main sections of the ARK program lesson guide. These sections lie under the categories of: self-esteem and unconditional love, classroom environment, understanding the dynamics of student behavior, and enabling students to monitor their actions.
This literature review will cover the importance of the ARK program with regard to current educational needs based on research covering the effects of student-teacher relationships and how those relationships help empower both the student and the teacher.
The Importance of the ARK Program
The ARK program’s foundational basis is set upon the idea of informing teachers about the importance of relationships with their students. A sizable literature provides evidence that strong and supportive relationships between teachers and students are fundamental to be healthy development of all students in schools (Hamre & Pianta, 2001). Student-teacher relationships are important during the early years for transition into the school setting. The impact of student-teacher relationships in the early years follows students for years to come. They continue to be important throughout elementary, intermediate, and high school and extends far beyond academic success.
Definition of Student-Teacher Relationship
The definition of a student-teacher relationship is not a professional relation between instructors and those they instruct. Positive student-teacher relationships are characterized by open communication as well as emotional and academic support that exist between students and teachers (Pianta, 1999).
Qualities of Teachers who Form Positive Student-Teacher Relationships Teachers’ abilities to form positive relationships with students are impacted by their personality type, experiences, and the quality of their own personal relationships (Baker, 2006). Teachers who engaged in frequent social conversation with students, made themselves available to students who were struggling, displayed regard for students perspectives and ideas, used behavior management strategies that clearly communicated expectations and caring, and took time to nourish the social emotional development of students displayed the most positive student-teacher relationships (Hamre & Pianta, 2006). Furthermore, teachers who learned about the personal interest, backgrounds, culture, and religion of their students were more effective at building positive relationships with them.
Behavior and Student-Teacher Relationships The dynamic of behavior on the part of the student and the teacher play a very important role in the development of a positive relationship. The ARK program devotes a great deal of its lessons to equipping teachers with strategies on how to positively interact with students who are exhibiting various forms of difficult behaviors. These lessons include: Understanding the Misbehaving Student, Dealing with An Attention-Seeking Student, Handling a Rebellious Student, Working with a Power Seeking Student, and Disciplining with Natural and Logical Consequences.
One study out of University of Virginia provided four key elements of a well-designed behavior management system. These key elements include: providing clear limits in tolerances that help regulate students behavior, reinforcing the idea that teachers will respond in expected and fair ways, creating opportunities to give students positive feedback about their behavior, and implementing the behavior system in a way that communicates care and respect of students (Pianta, 1999).
The Far Reaches of Student-Teacher Relationships
Positive student-teacher relationships extend far beyond success in academics. The positive student-teacher relationship affects adaptation to school regardless of gender or grade level across the elementary age range (Baker, 2006). They view school as a positive experience, and as a result, exhibit fewer behavior difficulties and display better social skills (Buyse et al., 2009). They are also more active participants in class, express a greater interest in college, and maintain higher grade point averages (Hallinan, 2008).
Jones (2008) proposes that school leaders should even use the term relationship building in lieu of classroom management to shift the thinking away from the idea of managing students and toward the notion of collaborating with them.
The ARK program’s philosophy on forming close student-teacher relationships in order to create a more fulfilling and enriching experience for both the student and the teacher were established on laudable, credible and well researched philosophy. Actions taken with regard to that philosophy have proven to have positive effects on both the teacher and the student, as well as the entire school community. In the words of Teddy Roosevelt, “People don’t care how much you know until they know how much you care”.
New Knowledge Expected to Result from Proposed Research From this project we expect to gain insight on the strengths and weaknesses of the ARK program. We will work to gain an understanding about the level of effectiveness of the ARK program lessons and situation circles with regard to the needs of student-teachers. We aim to discover ways to help the art program become a program that addresses the needs of student-teachers by looking deeply at their questions concerning their relationships with their students. By analyzing their questions and concerns we hope to help the ARK program facilitate lessons and situation circles that play an instrumental role in the development of teachers. The ultimate goal of evaluating this program is to help ARK contribute to the growth and production of teachers who understand the value of love, respect, and relationship in their classrooms, by which creating and environment whereby both the student and the teacher have access to reaching their highest level of social, emotional, and academic competence.
Conceptual Framework
The ARK program will be evaluated using an objective-oriented approach. Incorporating a Tylerian Model will determine a relationship between the goals of the program and its outcomes. The purpose is to conduct a formative evaluation to determine the quality of the teacher preparation program and if there are any characteristics within the lessons that aid in the achievement of ARK’s goals. This is designed as an internal evaluation that will be conducted by two doctoral student members of the University of Houston staff who serve in the roles of Student Teaching Field Supervisors and ARK Facilitators. Discrepancies between performance and objectives would lead to modifications intended to correct the deficiency, and the evaluation cycle would be repeated (Fitzpatrick, Sanders, & Worthen, 2011). The framework for conducting an evaluation founded on the Tylerian Model will include: (1) establishing broad goals or objectives, (2) classifying the goals or objectives, (3) defining objectives in behavioral terms, (4) finding situations in which achievement of objectives can be shown, (5) developing measurement techniques, (6) collecting performance data, and (7) comparing performance data with the behaviorally stated objectives. The primary stakeholders in the evaluation design are the ARK creator, ARK Liaison, and the Student Teaching Directors from the University of Houston. Participants include the Student Teaching Field Supervisors who serve as ARK Program Facilitators and the student-teachers at the University of Houston. Although outcomes and any discrepancies will be examined following the completion of program implementation, the evaluation will be guided by goals established by the ARK creators and University of Houston Student Teaching Directors prior to program engagement.
Establishing broad goals or objectives
The broad goals of the ARK program were established by the creator of ARK prior to its implementation. ARK’s goals are listed in the instructional books for teachers and support staff that accompany the video lessons. The program’s two broad goals include (1) tools for working with the kids; and (2) a support network for fellow teachers (Wilkerson, 2007). These goals are based on Glasser’s Choice Theory that promotes the idea that the most important need is love and belonging, as closeness and connectedness with the people we care about is a requisite for satisfying all of the needs. The goals of the program are also based on the assumption that the student-teacher has had no prior training in building relationships with students, and need formal training in building positive relationships with students.
Classifying the goals or objectives
Through dialogue with the ARK creator, ARK Liaison, and university Student Teaching Directors, the evaluation team was able to determine a specific goal for the ARK program at the University of Houston. The specific goal is to equip student-teachers with strategies for building and maintaining positive relationships with students prior to entering the teaching profession full time.
Defining objectives in behavioral terms
The objectives for evaluation were devised collaboratively between ARK creators and the University of Houston Student Teaching Directors. Stated behaviorally, the objectives are: (1) that all students that completed the student teaching program at the University of Houston (Quest 3 and Student Teaching Parts 1 and 2) during Fall 2013 and Spring 2014 will utilize the skills taught during the ARK sessions during their student teaching assignments, and (2) that all students that completed the student teaching program at the University of Houston (Quest 3 and Student Teaching parts 1 and 2) during Fall 2013 and Spring 2014 will utilize the skills taught during ARK sessions in their classrooms once they become a classroom teacher full time.
Finding situations in which achievement of objectives can be shown
Evidence of achievement of Objective 1 can be shown can be found through observation of the small group segment of the ARK sessions when student-teachers discuss techniques used while student teaching in the field, and from suggestions from their peers for resolving situations within the classroom. Evidence of achievement can also be found during observations of the student-teachers in their classrooms by their Student Teaching Field Supervisors. Additional evidence can also be found through the summaries of ARK sessions that are emailed to the ARK creators from the ARK Facilitators. Situations in which achievement of Objective 2 can be found will be limited to surveys of the student-teachers and focus groups after the completion of the program.
Developing measurement techniques
The evaluation design will employ a mixed method of quantitative and qualitative techniques for measurement. Quantitatively, a pretest/posttest method of existing surveys developed by the ARK Corporation will be used to analyze the influence of ARK. Surveys will be administered to all student teaching participants before the initiation of ARK lessons, and a new survey will be administered to them following the conclusion of the last ARK lesson. Pretest surveys will be used to assist in measuring the achievement of Objective 1 and posttest surveys will be used to assist in measuring the achievement of Objective 2. Existing surveys that were developed by the ARK Corporation will be administered as the pretest and posttest surveys following the conclusion of the ARK curriculum.
Qualitatively, participant observations of the ARK small group discussions will be employed to assist in measuring the achievement of Objectives 1 and 2. Participant observations will occur as a result of the evaluation team also serving in the roles of Student Teaching Supervisors and ARK Facilitators. A chronological collection of anecdotal emails from the ARK facilitators summarizing small group discussions, in addition to focus groups of the student-teachers will support measurement of both Objectives 1 and 2.
Collecting performance data
Performance data will include: summaries of the ARK sessions from ARK facilitators, pretest and posttest surveys from student teaching participants, posttest surveys from ARK facilitators, participant observation notes of the ARK small group discussions, and records from the student teaching focus groups. ARK session summaries and ARK facilitator surveys will be collected by the ARK liaison, pretest and posttest surveys will be collected by ARK facilitators, and observation and focus group records will be collected by the evaluation team.
Comparing performance data with the behaviorally stated objectives
After all data has been collected, the evaluation team will collaborate with the ARK creator, ARK Liaison, and University of Houston Student Teaching Directors to analyze and compare the outcomes of program implementation with the following objectives: (1) that all students that completed the student teaching program at the University of Houston (Quest 3 and Student Teaching Parts 1 and 2) during Fall 2013 and Spring 2014 will utilize the skills taught during the ARK sessions during their student teaching assignments, and (2) that all students that completed the student teaching program at the University of Houston (Quest 3 and Student Teaching parts 1 and 2) during Fall 2013 and Spring 2014 will utilize the skills taught during ARK sessions in their classrooms once they become a classroom teacher full time.
Research Methods A mixed approach of quantitative and qualitative methods will be employed within a Tylerian evaluation design to compare the outcomes of the ARK program with the objectives that were collaboratively determined by the ARK Corporation and University of Houston Student Teaching Directors. Data collected during implementation of the ARK lessons will be used to measure how the participant outcomes compared to the specific objectives established by the primary stakeholders. The data will also be examined for discrepancies within the components of the program that may have influenced its outcomes. A pretest/posttest design of survey data will be used to compare the use of ARK strategies by student teaching participants before ARK lessons to the use of ARK strategies by student teaching participants after the completion of ARK. Anecdotal emails from ARK Facilitators, participant-oriented observations, and student-teacher focus groups will be analyzed for evidence of discrepancies.
The student-teachers will serve as the participants that will be examined in the ARK program. These are under graduate and Master’s level graduate students who have met university guidelines to student teach in a school within the Region IV area of Texas to complete graduation requirements on their degree plan. There are approximately 500 student-teachers, and these student teaching participants were pre-determined by the university Student Teaching Department. Three days during the Fall semester and three days during the Spring semester have been designated for student-teachers to return to their university campus to engage in two consecutive hours of ARK lessons.
An ARK lesson will begin with a choral reading of expectations known as “ARK Covenants.” An 8-10 minute lecture about a specific relationship building topic is then broadcast on a DVD. Students can also follow along in an ARK book that was distributed during their student teaching orientation. The 8-10 minute lecture is viewed by approximately 60 students at a time. Each group is assigned according to student teaching assignment. Participants that are completing a K-6 teaching assignment are assigned to a room that will broadcast the elementary ARK curriculum, and participants that are completing a grade 7-12 assignment are assigned to a room that will broadcast the secondary ARK curriculum.
Following the video, students divide into groups of approximately 16-18 students to engage in a small group discussion, known as a Situation Circle. Groups were predetermined by the ARK Liaison and are randomly divided within their teaching category (elementary education or secondary education). Small group sessions last for 45-55 minutes and are facilitated by a Student Teaching Field Supervisor that serves as an ARK Facilitator. During Situation Circle, participants are guided by the ARK Facilitator to give feedback regarding the 8-10 video presentation. Facilitators also allow time for participants to discuss any concerns that they may have regarding relationships within their student teaching assignment. Once a participant has expressed a specific situation that brings concern, other participants are allowed to ask clarifying questions, and participants also take turns around the group circle offering suggestions to resolve the concern. While suggestions are being offered, one participant is transcribing the suggestions for them.
At the close of the ARK session, the small group will choral read the ARK Covenants to conclude the lesson. To maximize the amount of time that the participants are available, the Student Teaching Directors have scheduled two ARK lessons to be completed consecutively for two hours on each of the six scheduled days.
Student Teaching Field Supervisors and Directors from the University of Houston received ARK Facilitator Training from the ARK Corporation regarding the best practices for implementing the curriculum to student-teachers. The professional development session consisted of four hours of modeling and discussion of strategies for facilitating lessons and situation circles with student-teachers. University staff also received individual copies of ARK materials, which included: one ARK Facilitator Guide, one ARK Lesson Guide, one set of ARK Covenants, a DVD of Elementary ARK lessons, and a DVD of Secondary ARK lessons.
Prior to the beginning of the first video in the first ARK session, each ARK Facilitator will administer a survey designed by the ARK Corporation to their participants before the ARK Covenants are read. These surveys are found in the ARK Facilitator training manuals. ARK Facilitators will also email the ARK Liaison following each of the six sessions with a brief summary of the topics discussed in their small group circles. During the third ARK session, the evaluation team will engage in a participant observation of a randomly selected elementary ARK small group and a randomly selected secondary ARK small group to note evidence of situations that illustrate achievement of the objectives. Following the fifth ARK session, eight members from one elementary ARK small group will be randomly selected for a focus group and eight members from one secondary ARK small group will be randomly selected for a focus group and interviewed by the evaluation team for additional evidence of performance that can illustrate achievement of the objectives being measured. At the conclusion of the final ARK session, ARK Facilitators will administer a different survey designed by the ARK Corporation to student teaching participants regarding feedback about the ARK program.
Data Collection Schedule
Method
Timeframe
Responsible Party
Personal Inventory ARK Survey
First ARK session
ARK Facilitators
Anecdotal Summary/Emails
Within three days following each ARK session
ARK Facilitators
Participant-oriented Observations (2)
Third ARK session
Internal Evaluators
Focus Groups (2)
Immediately following the fifth ARK session
Internal Evaluators
Teacher Self-Evaluation Form
Sixth ARK session
ARK Facilitators
Data Collection Instruments The following data collection instruments will be used for program evaluation: a Personal Inventory ARK Survey, anecdotal records, participant-oriented observations, two focus groups, and a Teacher Self-Evaluation Form. Each instrument is expected to yield data that can be measured and compared to performance objectives.
Personal Inventory ARK Survey
A pre-program survey developed by the ARK Corporation is designed to inventory the extent of training and experiences that student teaching participants have had regarding self-perception and relationship building. The survey lists fifteen statements and offers multiple choices for each statement in the format of a Likert scale. Answer choices include: Strongly Agree, Agree, Uncertain, Disagree, and Strongly Disagree.
Anecdotal Records
Brief summaries outlining the topics discussed during each ARK Facilitators’ lessons will be emailed to the ARK Liaison no more than three days following each of the six ARK days. These emails will serve as anecdotal records that will list general feedback from student teaching participants regarding the DVD video lessons, as well as any concerns that were discussed in Situation Circles.
Participant-oriented Observations
During the third ARK session, the two members of the evaluation team that also serve as ARK Facilitators will observe one ARK lesson from a randomly selected elementary ARK group during the first hour of instruction and one ARK lesson from a randomly selected secondary ARK group during the second hour of instruction. Notes will be taken regarding verbal cues, verbal interactions, and nonverbal cues.
Focus Groups
Following the fifth set of ARK sessions, the members of the evaluation team that serve as ARK Facilitators will randomly select eight student teaching participants from elementary ARK groups and conduct a focus group to extract feedback regarding the influence that ARK has had in their current classroom assignments and on their strategies for building student relationships when they enter the teaching profession full time. A similar focus group will also be constructed, consisting of eight randomly selected student teaching participants from secondary ARK groups. Leading interview questions will be pre-determined by the evaluation team and asked to both focus groups. Each group will be led by both members of the evaluation team, and will last for 30-45 minutes each.
Teacher Self-Evaluation Form
At the conclusion of the sixth scheduled ARK session, ARK Facilitators will administer a Teacher Self-Evaluation Form to student teaching participants to complete regarding their feedback about the effects that ARK has had on their personal and professional relationship building skills. Developed by the ARK Corporation, the Teacher Self-Evaluation Form has one section that consists of a survey with multiple answer choices numbered from one to three. Student-teachers are expected to choose one answer choice that most closely matches their belief. The second section of the form is a questionnaire consisting of three open-ended questions that encourage participants to elaborate in detail on the influence that ARK has had on their attitudes towards the students in their classroom assignments, their existing relationships with those students, and the behaviors of those students.
Modes of Analysis To analyze the quantitative data, a pretest/posttest will be employed to compare the performance of student teaching participants in their teaching assignments prior to ARK and following ARK. It has yet to be finalized which specific data analysis (sample t-test, ANOVA) will be used to determine the exact outcome. Qualitative data will be analyzed by categorizing feedback from the anecdotal records, participant-oriented observations, and focus groups; however the most efficient form of data analysis has yet to be determined.
Principal Investigators The principal investigators responsible for implementing the evaluation design will be two doctoral students from the University of Houston. The principal investigators will be conducting an internal evaluation, and they also serve in the capacities of Student Teaching Field Supervisors and ARK Facilitators. Supporting the principal investigators include: the ARK Liaison, the two university Student Teaching Directors, Student Teaching Field Supervisors who also serve as ARK Facilitators, and two data analysts. Ultimately, the principal investigators will be collaborating with those that serve as the primary stakeholders to measure and compare outcomes to the stated objectives. The primary stakeholders include: the ARK Corporation and the University of Houston Student Teaching Directors.
Principal Investigators
Two doctoral students will serve as the principal investigators throughout the evaluation. They also serve dual roles within the ARK program--as evaluators and as ARK Facilitators. The principal investigators will be responsible for conducting the participant-oriented observation of the two ARK Situation Circles during the third ARK session. The principal investigators will also use ethnographic methods to analyze the data collected from the observations. Conducting the focus group interviews will also be an additional responsibility for the principal investigators. They will design the leading interview questions so that they are open-ended and aligned with the information that is needed to be collected. The evaluators will randomly select eight elementary student teaching participants and eight secondary student teaching participants for the two focus groups.
Support Team
ARK Liaison
The ARK Liaison, who is a primary stakeholder, will assist in data collection. She will archive the emails from ARK Facilitators that summarize topics and feedback that student teaching participants share during their Situation Circles. These emails will be given to the principal investigators for analysis and measurement.
University of Houston Student Teaching Directors
The two student teaching directors, who are also primary stakeholders, will be responsible for corresponding with the ARK Facilitators to send reminders of the appropriate times to collect data according to the proposed schedule. The Directors will also be responsible for contacting the student teaching participants to inform them of upcoming ARK sessions, as well as contacting the randomly selected participants for the focus groups. They will also collect the completed Personal Inventory ARK Surveys and Teacher Self-Evaluation Forms from the Student Teaching Field Supervisors/ARK Facilitators following ARK Session 1 and ARK Session 6.
Student Teaching Field Supervisors/ARK Facilitators
The primary responsibilities of the field supervisors/facilitators will be to lead the ARK lessons, and administer the surveys. They will be expected to administer the Personal Inventory ARK Survey to the student teaching participants in their assigned ARK groups prior to the first ARK lesson. Following the final ARK lesson, the field supervisors/facilitators will also be expected to administer the Teacher Self-Evaluation Form to the student teaching participants. They will collect each survey immediately after completion and turn them in to the Student Teaching Directors. Following each of the six sessions, the field supervisors/facilitators will also be expected to email the ARK Liaison a brief summary of the feedback and topics discussed during each lesson within three days of the meeting.
Data Analysts
There will be two data analysts. Each analyst will code the Personal Inventory ARK Survey into the Statistical Packages for Social Sciences program for categorization and analysis of the results. The two reports will be compared by the analysts for discrepancies with the results, and corrected within the program as needed. The same approach will be utilized for analysis of the Teacher Self-Evaluation Form.
Budget
Staffing
Cost-1,000.00 per evaluator
At a cost of $1,000.00 per evaluator the full evaluation including follow up meetings will cost $2,000.00 total.
Materials and Supplies
Cost- $100.00
This cost covers printer paper, printer ink, binders, folders, and staples for the evaluation, surveys, and anecdotal notes.
Travel
Cost-$50.00
This cost covers gas to and from meetings with stakeholders and student-teachers.
References
Baker, J. A. (2006). Contributions of teacher-child relationships to positive school adjustment during elementary school. Journal of School Psychology, 44(3), 211-229.
Buyse, E., Verschueren, K., Verachtert, P., & Van Damme, J. (2009). Predicting school adjustment in early elementary school: Impact of teacher-child relationship quality and relational classroom climate. Elementary School Journal, 110(2), 119-141. Retrieved from http://dx.doi.org/10.1086/605768
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Boston: Pearson.
Hamre, B., & Pianta, R. (2001). Early teacher–child relation-ships and the trajectory of children’s school outcomes through eighth grade. Child Development, 72, 625–638.
Hallinan, M. T. (2008). Teacher influences on students ' attachment to school. Sociology of
Education, 81(3), 271-283. Retrieved from http://dx.doi.org/10.1177/003804070808100303
Jones, R. D. (2008). Strengthening Student Engagement. [White paper.] Retrieved from http://www.leadered.com/pdf/Strengthen%20Student%20Engagement%20white%20paper.pdf Kopkowski, Cynthia (2008, April). Why They Leave. NEA Today Magazine. Retrieved from http://www.nea.org/home/12630.htm Pianta, Robert C. Enhancing relationships between children and teachers. Washington, D.C.:
American Psychological Assn., 1999.
Wilkerson, B. G. (2007). Adults relating to kids for teachers and staff (elementary level). The
Houston: The ARKGroup, Inc.
References: Baker, J. A. (2006). Contributions of teacher-child relationships to positive school adjustment during elementary school Buyse, E., Verschueren, K., Verachtert, P., & Van Damme, J. (2009). Predicting school adjustment in early elementary school: Impact of teacher-child relationship quality and Hamre, B., & Pianta, R. (2001). Early teacher–child relation-ships and the trajectory of children’s school outcomes through eighth grade Hallinan, M. T. (2008). Teacher influences on students ' attachment to school. Sociology of Education, 81(3), 271-283 Jones, R. D. (2008). Strengthening Student Engagement. [White paper.] Retrieved from http://www.leadered.com/pdf/Strengthen%20Student%20Engagement%20white%20paper.pdf Kopkowski, Cynthia (2008, April). Why They Leave. NEA Today Magazine. Retrieved from http://www.nea.org/home/12630.htm Pianta, Robert C. Enhancing relationships between children and teachers. Washington, D.C.: American Psychological Assn., 1999 Wilkerson, B. G. (2007). Adults relating to kids for teachers and staff (elementary level). The Houston: The ARKGroup, Inc.
You May Also Find These Documents Helpful
-
A general survey of scholarship practiced within three broad cultures of inquiry: the Humanities, Social Sciences and Natural Sciences. Through a cross-disciplinary framework, this course explores the diverse ideas, values, and practices…
- 1886 Words
- 8 Pages
Powerful Essays -
This assignment is intended to provide evidence of a candidate’s knowledge and understanding of communication and professional relationships with children, young people and adults. By completing all tasks within the assignment, the candidate will provide evidence that meets the Learning Outcomes and assessment criteria of Unit 203, Communication and professional relationships with children, young people and adults.…
- 1554 Words
- 7 Pages
Powerful Essays -
The Kentucky Suicide Prevention Group (KSPG) has established them as a program that helps with suicide prevention. According to KSPG the program began as a planning group and has developed into in a group program (2008). During the planning process a needs assessment was performed at the beginning phase and was included in the evaluation plan. Information was collected throughout the beginning phase to answer why, who, how, what, and when during the needs analysis. Also Kentucky Suicide Prevention Group used evaluation design to conduct beginning phase of program planning.…
- 1206 Words
- 4 Pages
Powerful Essays -
Communicating with children, young people and adults in an educational environment is an important part of a professional relationship. To raise your awareness in this area:…
- 622 Words
- 3 Pages
Good Essays -
E2: Discuss TWO (2) issues which contribute to maintaining professional relationships with children and adults…
- 4363 Words
- 11 Pages
Better Essays -
Task 1: Establishing a respectful, professional and comfortable relationship with children and young people can be developed in numerous ways. This is normally down to common sense but other factors are included which can include principles and values. Children are very responsive human beings which are wide eyed to a lot of things going on around them. In order for children to achieve their highest potential they have to be surrounded by a constructive and reassuring environment.…
- 3950 Words
- 16 Pages
Powerful Essays -
Pianta, R. C. (1999). Enhancing relationships between children and teachers. Washington, DC: American Psychological Association.…
- 2730 Words
- 11 Pages
Good Essays -
Working with children requires us to build relationships with them quickly but also in ways that are professional. The quality of relationships that we have with children and young people has a huge effect on the way in which we can work with them. When children feel comfortable with us they can separate more easily from their parents and they are much more likely to take part in play and learning activities.…
- 546 Words
- 3 Pages
Satisfactory Essays -
Working with children requires us to build relationships with them quickly, but also in ways that are professional.…
- 640 Words
- 3 Pages
Satisfactory Essays -
COMMUNICATION AND PROFFESIONAL RELATIONSHIPS WITH CHILDREN, YOUNG PEOPLE AND ADULTS 1.1 Information from supporting teaching &learning n schools by Louise Burnham…
- 821 Words
- 4 Pages
Satisfactory Essays -
As a Human Service worker you work with special populations which are in need of mediation and advocates. Having the ability to mediate, advocate and facilitate are important tools as well as desired skills within the field of human services. In order to help these special populations you must be aware of groups that bring resources as well as advocate in a national and international basis for special populations. As an advocate you are the voice for someone who is unable to speak for themselves or unable to defend their rights and as mediator you assist as a neutral party in the agreement to a dispute on differences two parties might have. The contents of this assignment will attempt to address how three different organizations advocate for the populations they work with as well as what these organizations do.…
- 1126 Words
- 5 Pages
Good Essays -
There are many different issues which contribute to maintaining professional relationships with children and adults, one of the most important is…
- 2162 Words
- 9 Pages
Good Essays -
When I had first volunteered at a preparatory school as a teacher’s assistant, I was thrilled and excited. I enjoyed working with children and I admit, was slightly disillusioned with expectations that the children would be dutiful and effortless to watch over. Unfortunately, all my prior imaginations crashed the second I entered the classroom. Everywhere I saw, it was pure chaos. I dodged a flying crayon as I saw children running around throwing paper airplanes, balls, while others just stared at their iPads, focused on the latest game. Not a single one of them even listened to me or glanced at me as I tried to yell over the loud chatter.…
- 492 Words
- 2 Pages
Good Essays -
A program and evaluation of a program are not separate activities. When carefully planned a program and the evaluation of the program can be integrated into an organized set of ongoing operations that promote and support each other. Program planning is an organized process in which a set of coordinated activities or interventions is developed to address and facilitate change in some or all of the identified problems. It is an ongoing process of development with the intention of designing a plan that will tackle the identified needs and problems. Program planning is a means for program development and is a goal oriented activity based on the assessed needs and problems. Program planning requires that decisions be made. Program evaluation provides the needed information because it is the process of gathering and analyzing the data necessary for appraising alternatives. Program evaluation is a cooperative undertaking of all those concerned with improving vocational education, and is the foundation upon which planning is built.…
- 468 Words
- 2 Pages
Satisfactory Essays -
The first thing we did in our community was to identify the problems in our area. Then, we prioritized and ranked them. The highest is sanitation family planning so we focused our heath teaching on those two. Majority of the families we interviewed have three to five children without proper birth spacing. Based on our survey, we came to realized that most of them are not aware of the different family planning methods. We did a research on the different contraceptives and made leaflets that shows them the different methods. For sanitation we bought broomsticks, sacks and rugs. We also bought groceries for prizes. We also prepare sandwich and juice for them. We coordinate with the officials in the barangay hall for the sound system, chairs and for the security of everybody including us. We went to the association president and ask her help for the invitation and we gave out some invitations to the families in our community including the families we didn’t interviewed.…
- 289 Words
- 2 Pages
Satisfactory Essays