Florida Industry Certification Study: What did the survey ask of districts and schools?
By Elizabeth Parish and Elizabeth Glennie
In the Florida Industry Certification Study, RTI examined the strategies that districts and schools use to help high school students earn industry-recognized certifications.1 Florida’s Career and Professional Education (CAPE) Act is one of the nation’s most comprehensive, state-sponsored certification programs for high school students. Since its inception, the number of Florida students earning certifications has dramatically increased. With the CAPE Act, the state of Florida pays for examination costs and provides incentives to schools by awarding points on the School Report Card for the number of students earning certifications. The Florida Department of Education determines which certifications are appropriate for this program and exam retake policies. Districts and schools have leeway in implementing the program—deciding which courses to offer, coordinating logistics for exam taking, providing supports to teachers of these classes, and promoting opportunities to students. To date, limited research has examined which of these practices best prepare students for certification exams, promote equitable access, and lead to positive academic outcomes.
Studies of other programs have shown that implementation strategies can influence an initiative’s success. Setting clear program goals contributes to student learning.2 Clearly communicating goals can motivate staff and help them focus activities. Well-designed professional development can support teachers in instructional practices, and students whose teachers receive high-quality professional development may then have more successful outcomes.3 With common planning time, teachers can collaborate and share resources as they determine the best practices for their students.4 Finally, the ways that schools promote a program may influence students’ decisions to participate and thereby increase levels of participation. Schools can broadly recruit students to participate and recognize those who succeed. Schools can also help parents learn about the program by putting information on a school website and sending informational materials home.
To learn about the extent to which districts and schools employ these strategies, RTI administered a web-based survey to career and technical education (CTE) or curriculum leaders within each Florida district and high school. (For more information about the survey design and response rates, visit https://flcertificationstudy.org/survey-methods.) We developed two surveys for the study, one for district-level respondents and another for school-level respondents. These surveys were almost identical with only minor differences based on districts’ and schools’ roles in program implementation. During spring 2018, CAPE Act administrators and CTE staff had the opportunity to complete the survey, providing valuable input about their strategies for promoting this program. Some questions focused on more general implementation practices such as leadership/administration, teacher qualifications/support, and strategies for promoting the program. Other questions focused on specific aspects of the CAPE program, such as exam retake policies and transportation to exam sites.
Districts and schools are typically responsible for implementing state and federal policies. If districts and schools use different strategies, then the implementation of a new program or policy may look very different from school to school. Many of these questions from the Florida Industry Certification Study survey can be modified and adapted to other programs. Readers may be interested in these types of questions and consider similar questions for a program implementation of their own.
General implementation questions
FOCUS: Leadership and AdministrationOBJECTIVE: To understand the role and responsibilities of the person who coordinates the program at the district or school.
- Who is responsible for administering the certification program?
- What are the responsibilities of this program administrator?
- Does your district/school have goals for the number of students taking or passing certification exams?
FOCUS: Teacher Qualifications and Support OBJECTIVE: To learn about required qualifications for teachers of courses that lead to industry-recognized certifications and the support provided to them, such as specific training or professional development and common planning time with other teachers.
- What qualifications are teachers required to have to teach courses that lead to certifications?
- What forms of support does the district/school provide to teachers who teach courses leading to certifications?
FOCUS: Promoting Certifications OBJECTIVE: To learn about district and school requirements for students who take certification exams and strategies that promote or encourage students to complete certification programs.
- Does your district/school actively encourage students to take certification exams?
- Which students does your school encourage to take certification exams?
- How does your school promote certifications to students and parents?
- What are the challenges associated with encouraging students to obtain certifications?
FOCUS: Program Costs OBJECTIVE: To better understand start-up and ongoing costs for programming in terms of expenditures associated with curriculum, professional development, or technology.
- Has your district/school incurred additional costs to operate the certification program?
- To begin having the CAPE program, what initial costs did your district/school have?
- To continue the CAPE program, what ongoing costs does your district/school have?
Questions specific to the CAPE Act
FOCUS: Policies for Retaking Certification Exams OBJECTIVE: To understand district and school policies for retaking certification exams. The Florida Department of Education has exam retake policies, but districts and schools can add necessary steps for students who fail exams to retake them.
- If a student does not pass the certification exam, what requirements does your district/school have for the students to retake the exam beyond the state-mandated waiting period to retake exams?
FOCUS: Exam Locations and Transportation OBJECTIVE: To learn if certification exams occur primarily on-site at schools or off-site, and the coordination of transportation to off-site locations.
- About what percentage of certification exams taken by students in your district are offered in school? What percent of exams are offered off-site?
- If an exam is off-site, how do students get to the exam location?
Staff buy-in is critical to successful program implementation. We wanted to gain a better understanding of staff perceptions of the benefits and challenges of implementing certification programming in terms of scheduling, equipment needs, and time; staffing qualified teachers and professional development; and school culture related to staff and student buy-in and outcomes. We created a series of statements about possible benefits and challenges with which survey respondents could agree or disagree.
PERCEIVED BENEFITSDo you agree or disagree that this program...
improves student engagement in high school coursework
increases the job prospects of students
provides the potential for college credit at no charge
provides information to employers about students' job skills
motives students to remain enrolled in school
increases student morale
improves a school's state grades
increases teacher and staff morale
is valued by industry partners
PERCEIVED CHALLENGESDo you agree or disagree that this program...
requires significant professional development
is difficult to staff with qualified teachers
meets resistance from teachers who are reluctant to teach certification exams
shifts resources or attention from other academic or CTE programs
is difficult to supply with appropriate equipment or facilities
is of little interest to students
produces challenges in scheduling exams
is difficult to manage overall
Use of Results
RTI has prepared a series of issue briefs on survey findings to give policymakers, researchers, and practitioners a better understanding of how districts and schools implement the statewide certification program, the factors supporting effective and equitable certification opportunities, and the outcomes that result from having students earn certifications. For example, we relied on our questions about teacher supports and qualifications to examine that topic (https://flcertificationstudy.org/qualifications-of-and-supports-to-teachers). We also linked survey data to academic data from the Common Core of Data Public School Universe to see what school factors influence certification earning. For example, in our study of promotion strategies, we found that in high-poverty schools, a higher percentage of CAPE leaders said they promoted certifications in particular career areas in response to needed local job skills. In contrast, in low-poverty schools, a higher percentage of CAPE leaders said they promoted certifications in career areas that are popular with students (https://flcertificationstudy.org/promotion-strategies). We have also linked the survey to student data from Florida’s Education Data Warehouse to show how implementation factors influence certification earning. We found that more CAPE leaders in schools with low certification rates report challenges with meeting resistance from teachers and a lack of student interest. Students and teachers in low certification rate schools may not fully understand the value of earning industry certifications. (https://flcertificationstudy.org/program-challenges).
By asking these survey questions, RTI was able to learn about the strategies Florida districts and schools use to implement the CAPE program. We also learned about staff perceptions of the benefits of and challenges to this program. Those initiating other educational programs could use a similar approach to understand better how those in the field are applying those components.
END NOTES
Overall Notes
Results reported in this issue brief include school survey data collected through the Florida Industry Certification Study and are based on the schools that offer certifications.
References
1 For more information about the study, visit https://flcertificationstudy.org/about-the-study.
2 Day, C., & Simmons, P. (2014). Successful school leadership. Education Development Trust. The University of Nottingham, Berkshire, U.K.
3 Darling-Hammond, L., Hyler, M.E., & Gardner, M. (2019). Effective teacher professional development. Palo Alto, CA: Learning Policy Institute.
4 Merritt, E.G. (2016). Time for teacher learning, planning critical for reform. Phi Delta Kappan, 98(4), 31-36. https://journals.sagepub.com/doi/10.1177/0031721716681774
