Untitled Document

INTRODUCTION

The Summary Evaluation Questionnaire was developed to assist in evaluating the instructional processes of vocational programs. It is designed for use by both the local teacher in conducting a comprehensive self-evaluation and by an external evaluation team. The questionnaire is divided into 11 sections, corresponding to the 11 standards of quality program operations. These standards or “best practices” have been endorsed by the State Board of Career and Technology Education.

Each standard is followed by a series of rating scale questions or Quality Indicators that further define the standard. Some questions are identified as State Board Rules, and these questions must be met in order to meet the standard. The Quality Indicators may vary among occupational divisions due to unique consideration within those divisions. The sum total rankings of the State Board Rules and Quality Indicators below each standard will serve as a ranking of the standard. The ranking of each standard and the corresponding narrative section will be recorded and reported in the final evaluation report.

DIRECTIONS

Evaluations using this instrument should strive to rate the Quality Indicators for each program evaluated in relationship to the standards. Comparing one vocational program with another may result in distorting evaluative information and is highly discouraged.

Questionnaire Rating

To complete the questionnaire, carefully read each standard and the Quality Indicators that follow. Questions related to State Board Rules are enclosed in a shaded area and are grouped at the beginning of each standard. For each question, the evaluator should indicate if the State Board Rules are met or not met by marking the appropriate box in the column on the right.

o Met

o Not Met

All other Quality Indicators are assessed by circling the appropriate rating in the column on the right. The numbers on the rating scale indicate the following:

5=Outstanding

Exceeds Standard

4=Above Minimum Standard

Meets Standard 3=Meets Minimum Standard

2=Below Minimum Standard

Falls Below Standard

1=Poor (Major Improvement Needed)

Suggested Activities for Evaluator

For each standard, there is a suggested list of documents to review or activities to perform as the evaluator rates the standard. The evaluator should use these suggestions as a guide but not as a limitation in reviewing the program.

Narrative Comments

The Comments Section of the questionnaire should be used to note specific program characteristics for which recommendations or commendations need to be made. Items rated below standard should be accompanied by realistic statements on how they can be improved. These notes should be used to write the Commendations and Strengths Section, the Specific Recommendations for Improvement Section, and the General Suggestions for Improvement Section at the conclusion of the evaluation (see pages following Standard 11.)

Scoring Totals (Evaluators Only)

To determine if a standard is met, the evaluator should:

1 Determine if all State Board Rules are met.

2 a Sum the ratings in the column on the right to find the total points.

b Compare the Total Points to Minimum Points Needed to Meet the Standard as stated at the end of the standard.

c If any questions were not applicable, use the chart at the end of the standard to determine the Minimum Points Needed to Meet the Standard. (The Total Number of Questions Rated should include the questions related to State Board Rules.)

Example

Sum the ratings in the preceding columns ........................................................................................... 45 Total Points

In order to meet the standard, all State Board Rules must be met and the total points must equal or exceed 36___

If any questions are not applicable, the chart below can be used to determine the total points needed to meet the standard. (The Number of Questions Rated should include the questions related to State Board Rules.)

Total Number of Questions Rated

2 1

Minimum Points Required to Meet Standard

Pts.

All Must

Be Met

Based upon the Quality Indicators, which reflect State Board Rules and factors influencing the quality of the occupational program, the standard is o Met o Not Met.

PROGRAM STANDARDS

FOR THE EVALUATION

OF OCCUPATIONAL PROGRAMS

STANDARD 1 Instructional Planning and Organization

STANDARD 2 Instructional Materials Utilization

STANDARD 3 Qualified Instructional Personnel

STANDARD 4 Enrollment and Student/Teacher Ratio

STANDARD 5 Equipment and Supplies

STANDARD 6 Instructional Facilities

STANDARD 7 Safety Training and Practices

STANDARD 8 Advisory Committee and Community Relations

STANDARD 9 Leadership Development

STANDARD 10 Coordination Activities

STANDARD 11 Student Accounting and Reports

INSTRUCTIONAL PLANNING AND ORGANIZATION

STANDARD ONE

The instructional program should be designed to impart knowledge and develop skills that are essential for success in meeting the students’ educational objectives. There should be evidence that each instructional unit has been properly planned and organized and is being implemented in a sequential manner. The programs should implement a fairly administered grading system that is based upon identified criteria.

The course of study for each program shall include both theory and skill lessons along with other supplemental learning activities that will serve to guide the program toward desirable learning outcomes. In addition, the course of study shall be integrated and organized in such a manner as to effect an overall well-balanced program of instruction.

Levels of Assessment Interpretation of Level

4 – Excellent Exceeds program standards.

3 – Good Meets program standards. No need for additional technical assistance at this time, although some areas, as indicated, could be improved.

2 – Needs Improvement Generally falls below program standards. Demonstrates limited effectiveness; additional technical assistance and/or resource utilization is needed for improvement.

1 – Poor or Missing Evidences little or no effectiveness. A great deal of technical assistance and/or resource utilization would be an immediate need.

N/A – Not Applicable Does not apply to the program being reviewed.

INSTRUCTIONAL PLANNING AND ORGANIZATION

STANDARD ONE

A.	Instruction
1.	Which version of the PLTW™ curriculum is being used? Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review current curriculum being taught.
2.	Each student is provided with the grading criteria for projects and assignments in advance. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review grading system, evidence of students participation and competency based grade book.
3.	Students are actively engaged in learning by working on relevant projects. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review program objectives and partnership activities.
4.	PLTW™ end-of-course exams are being administered to students at the conclusion of each course. Comment:	1	2	3	4	NA	Met Not Met
5.	Gender-neutral teaching practices and materials are used for instruction. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review instructional content for instruction that encourages the elimination of bias and stereotyping. Review marketing materials, etc.
6.	All students are required to individually solve equations and apply mathematic concepts covered in the curriculum. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review lesson plans and course outlines.
7.	All student portfolios are available and reflect the progression of skills developed in the PLTW course(s). Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review student portfolios
8.	All students are required to keep written logs or journals for project activities. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review to journals and lesson plans

Implementation Schedule of PLTW courses:

Describe how sequence of courses were determined and how math and science requirements for state and post-secondary will be met.

INSTRUCTIONAL MATERIALS UTILIZATION

STANDARD TWO

The availability and proper utilization of instructional materials is considered essential to conducting quality training. Schools offering programs shall ensure that adequate amounts of such materials, including state instructional materials, audio visual aids, reference texts, competency profiles, etc., are made available for instructional purposes. Teachers shall utilize a variety of instructional materials and methods in accordance with student needs and the goals and objectives of the program.

B.	Instruction
1.	Is there funding for teachers, counselors, administrators to attend PLTW Professional Development. Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Determine if teachers, counselors and administrations have funding available to attend professional development.
2.	A balanced and up-to-date selection of relevant resources as recommended by PLTW™ is provided to meet the instructional needs of students. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Determine the quality and quantity of instructional materials.
3.	Effective teaching methods are used to enhance the PLTW™ instructional program. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review instructional activities being utilized in the classroom.

Concurrent Mathematics:

Describe the procedures in place to ensure each student is concurrently enrolled in a college preparatory mathematics course.

Extra Help:

What types of academic extra help do students have access to?

Instruction –cont.

There is a budget for PLTW™

program operation?

Comments:

Met

Not Met

Suggested Activities: Review current budget for PLTW program and replacement schedule.

Have Plans been developed in order to keep all essential equipment (i.e., furniture, equipment, tools, technology, etc.) up-to-date and high-tech? Please explain.

QUALIFIED INSTRUCTIONAL PERSONNEL

STANDARD THREE

Selecting, developing, and retaining instructional staff who are competent to meet the needs of career and technology education is vital to the success of the instructor program. As evidence or proper preparation, all CareerTech instructors shall meet or exceed state certification requirements and will hold a current teaching certificate or credential appropriate to their subject area. In addition to these minimums, all instructors should continuously strive to upgrade their skills and knowledge by meeting recertification requirements and attending professional improvement meetings, state conferences, etc.

Teacher Selection

Describe the criteria used to identify and recruit teachers for PLTW courses.

		Counselor Conference Attendance
		Location	Year
A.	Names of Counselors on Campus

	B. Names of PLTW™ Teachers on Campus		Certification Fields	PLTW™ Course(s) Currently Teaching	PLTW™ STI's Attended and Year(s)			Participating in ongoing professional development (Virtual Academy or other)





C.		Administration
1.		Participation in PLTW professional development activities:
		Administrators				1	2		3	4	NA
		Counselors				1	2		3	4	NA
2.		Administrators, counselors, and teachers work collaboratively to provide continuous improvement to the PLTW program.				1	2		3	4	NA

For any score of “2” or below, please provide a brief explanation:

ENROLLMENT AND STUDENT/TEACHER RATIO

STANDARD FOUR

The teaching load (student/teacher ratio) will vary with the program, contingent upon the number/kinds of students to be served, the specific skills to be taught, the size of the facility, and the methods of instruction to be used. However, reasonable enrollment limits must be maintained in order to ensure that programs objectives may be met in an efficient and effective manner. Unless otherwise specified, program enrollment limits shall be maintained in each program in accordance with the state operations and procedures requirements.

Students Enrollment Data

Pre-Engineering

Selection and Recruitment:

Current Year

Building Enroll #

PLTW COURSES

IED

POE

CIM

CEA

EDD

A. Gender

Females

Males

B. Race/Ethnicity

African Americans

American Indians/Alaskans

Asians

Caucasians

Hispanics

Other

Totals

Total Building

Enrollment: _____

Total PLTW Enrollment: ______

% Enrolled in PLTW: _______

% of Free and Reduced Lunches: _____

Describe the process and criteria used to enroll students in the PLTW™ program.

BioMedical

Selection and Recruitment:

Current Year	Building Enroll #	PLTW COURSES
Current Year	Building Enroll #	Prin.of BioMed. Science	Human Body Systems	Scientific Research Capstone

A. Gender
Females
Males
B. Race/Ethnicity
African Americans
American Indians/Alaskans
Asians
Caucasians
Hispanics
Other
Totals
Total Building Enrollment: _____			Total PLTW Enrollment: ______ % Enrolled in PLTW: _______

% of Free and Reduced Lunches: _____

Describe the process and criteria used to enroll students in the PLTW™ program.

EQUIPMENT AND SUPPLIES

STANDARD FIVE

Proper equipment and adequate supplies must be made available to support the program. Schools offering CareerTech programs shall provide funds for equipment and supplies in accordance with acceptable standards and at a level to ensure quality education. Equipment selected should be representative of the grade and must meet or exceed all appropriate safety standards.

A.	Equipment and Supplies						Suggested Activities
1.	A current software lease agreement has been signed by a school district official (provide a copy for the visit) Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review software agreement.
2.	The current revision of each software package is being used. Record titles and revision level below Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review software package
3.	Equipment (including computer hardware), software, tools, and supplies meet or exceed PLTW standards. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Compare PLTW and course equipment list
4.	Students have access to the internet to conduct research and communicate with mentors. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Look at classroom internet access
5.	Students have telephone access, when necessary, to support curricular activities. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Look at classroom phone access.
6.	The district has provided a laptop computer(s) meeting or exceeding the PLTW™ minimum standards for each PLTW™ teacher Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Look at classroom laptops.
7.	The annual PLTW™ Purchasing Manual is made available to PLTW™ teachers and administrators for planning purposes. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review purchasing manual

INSTRUCTIONAL FACILITIES

STANDARD SIX

Physical facilities for programs shall include adequate space and utilities in classrooms, laboratories, and shop areas that provide for safe and orderly, quality instruction to meet the program’s objectives. Both instructional and non-instructional areas, including storage areas, restrooms, and offices, shall be adequate for the number of students and staff using such areas. Special consideration shall be given to meeting the needs of students with disabilities as well as providing for the special needs of co-educational classes. Clinical facilities shall be adequate in number and quality to provide the experiences necessary for an effective program and must meet the requirements of the respective accrediting agency.

A.	Instructional Facilities						Suggested Activities
1.	The space and physical arrangements are conducive to effective instruction. Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review and observe classroom
2.	Considering maximum class enrollments, the space/facility, physical arrangement, furniture, equipment, tools, materials, and supplies in the classroom(s) and laboratories are adequate to provide a safe learning environment. Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review and observe classroom and laboratories.
3.	Space, furniture, equipment, tools, materials, and supplies are accessible to permit individual student participation in the instructional program. Comment:	1	2	3	4	NA	Met Not Met Suggested Activities: Review and observe classroom and laboratories.
4..	Adequate classroom/laboratory space is provided for students to participate in the PLTW™ curriculum. Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review and observe classroom and laboratories.
5..	Classroom/laboratory facilities are appropriate for student safety. Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review and observe classroom and laboratories.
6.	Students use safety equipment and follow safety rules. Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review and observe classroom and laboratories. Look for safety posters and MSD sheets
7.	Safety rules are displayed in appropriate places in the classroom/laboratory. Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review and observe classroom and laboratories. Look for safety signs/posters and MSD sheets

SAFETY TRAINING AND PRACTICES

STANDARD SEVEN

Due to the nature of career and technology education and related employment, student safety and safety education are considered essential to quality program operations. Schools offering programs shall ensure that safety features in the instructional facilities and equipment are properly implemented and maintained. In addition, adequate lighting, temperature, and ventilation shall be provided to ensure a safe and healthy learning environment. The Occupational Safety and Health Administration (OSHA) standards shall be used to guide the implementation of environmental health and safety features.

Instructors shall model correct safety practices and ensure that safety training is incorporated into the instructional content of the educational program. Student safety practices shall also be encouraged and continuously monitored by the instructor.

Safety and Practices

Suggested Activities

Does the institutional facility provide adequate heat, light, ventilation, dust control, and noise control to provide a safe environment conducive to learning?

Comments:

Met

Not Met

Suggested Activities: Review instructional facilities, equipment and appropriate safety features.

Are accidents resulting in injury recorded and reviewed in order to correct any problems?

Comments:

Met

Not Met

Suggested Activities: Check for accident report forms and documentation

3.	Is safety instruction planned, presented, demonstrated and practiced by the instructor in laboratory activities? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review lesson plans and curriculum.
4.	Are eye safety glasses utilized when necessary? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Observe classroom and safety practices.
5.	Does the program have adequate and accessible fire extinguishers (correct type and serviced within the last year) correctly identified and displayed in the facility? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Check fire extinguishers
6.	Are the student safety tests and evidence of hazardous materials handling, right to know instruction, and universal precautions training retained on file to verify that appropriate training has taken place? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review curriculum, lesson plans and view MSD sheets and safety notification items.

PROGRAM ADVISORY COMMITTEE AND COMMUNITY RELATIONS

STANDARD EIGHT

An effective and continuous program of community relations is necessary to maintain a close working relationship with business, industry, and other organizations and individuals, as well as to ensure that education is relevant to the needs of students. The purpose of a community relations effort is to inform the public of the program’s operations and capabilities and to generate active participation in the development and implementation of the program’s goals.

Each program should actively strive to encourage community involvement and to promote a greater understanding of the program’s needs and accomplishments. In addition, each program shall have a formally organized program advisory committee that is broadly representative of the school community and of the business and industrial community it serves. Each local program shall maintain on file a list of the names and occupations of the advisory committee members and the minutes of each meeting.

III. Partnership Team (Documentation to be provided at site visit.)

A. Membership

Provide a list of the Partnership Team members that include the following:

"Member's Name "Title/Role (must include non-school members, i.e., business/industry) "Name of Business

B. Frequency

Provide copies of the agendas from two previous meetings and a calendar or meeting dates of the next two meetings.

C. Minutes: Provide the minutes of the previous two Partnerships Team Meetings.

E. Faculty and Administration Support
1.	Teachers, administrators, and counselors actively engage PLTW partnership team members to optimize the operation of the program and to provide continuous improvement.	1	2	3	4	NA

Communication and Outreach

A. Awareness
The school counselors, administrators, teachers, and partnership team members have made the community aware of the program.	1	2	3	4	NA
Administrators have informed school faculty of the PLTW™ program.	1	2	3	4	NA
B. Recruitment
Students are aware of the opportunity to take PLTW™ classes.	1	2	3	4	NA
Parents understand the requirements and rigor of the program.	1	2	3	4	NA
Feeder pattern schools are included in the recruitment process.	1	2	3	4	NA
C. Celebration
Student, class and teacher success are recognized.	1	2	3	4	NA
Plans are in place for a press release announcing the outcome of the visit.	1	2	3	4	NA

For any score of “2” or below, please provide a brief explanation.

LEADERSHIP DEVELOPMENT

STANDARD NINE

Student leadership development activities are considered an integral part of the course of instruction in all programs. Each student shall be afforded the opportunity to become an active member of an appropriate student organization. The leadership development activities associated with the student organization shall be directed and supervised by the local teacher with guidance from the local school administration and the state supervisory staff.

A.	Leadership						Suggested Activities
1.	Is each student afforded the opportunity and encouraged to become actively involved in leadership, personal development, and employment skills attainment? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Discuss membership development and recruitment procedures.
2.	Are students involved in Robotics, Skills USA or Hosa? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Verify membership
3.	Does the program assist with projects that make contributions to the improvement of the school or community? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review activities. Check for evidence of student involvement.

COORDINATION/CLINICAL ACTIVITIES

STANDARD TEN

Cooperative education is uniquely designed to meet students’ career major objectives through supervised experiences, coupled with regular classroom instruction. Each student participating should be placed in an experience related to his/her area of study. The instructor should ensure that each student’s cooperative experience provides skills that allow the student to meet the goals and objectives of his/her program. A “Memorandum of Training” shall be signed by the student, parent (if applicable), and teacher-coordinator and shall be maintained on file.

In order to ensure quality of training and procedural compliance, the local teacher-coordinator shall visit or oversee the experience as outlined by state and national accreditation standards. Students should be encouraged to develop teacher-assisted entrepreneurship programs, where such programs are related to the instructional content of the training program.

If this standard is not applicable, check here __________.

A.
2.	Does each student have the opportunity to participate in an internship or capstone project? Comments:	1	2	3	4	NA	Met Not Met
3.	Are objectives for the internship or project based on and validated by industry standards? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review validation process and/or partnership committee recommendations.
4.	Does the internship supervisor/project manager complete a written evaluation on the employed student(s) Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review documentation of evaluation.

STUDENT ACCOUNTING AND REPORTS

STANDARD ELEVEN

Recognizing that the objective of CareerTech education is to educate a student to be work ready/college bound, a systematic program of follow-up is essential. As evidence of efforts to achieve this objective, each local educational institution shall maintain student enrollment, placement, and follow-up records on all students who have enrolled in programs. These records must be maintained in accordance with divisional requirements and reported as requested to the Oklahoma Department of Career and Technology Education.

In exploratory programs or in programs where students move from Program I to Program II, completers should be assisted in continuing their education. Enrollment, placement, continuing education, and follow-up records should be maintained for these programs.

Student Accounting and Reports

Does the one-year follow-up indicate that completers from the previous year are employed in a job related to training or are continuing their education?

Comments:

Met

Not Met

Suggested Activities: Review placement data for previous year’s program completers and/or PLTW data

Does each student have an opportunity to compile a résumé, complete a job application, follow the plan of study, plan for future education, prepare for a job search, or complete any other application as a part of his/her instructional program?

Comments:

Met

Not Met

Suggested Activities: Review program objectives and lesson plans.

3.	Is the student retention rate maintained at an appropriate level? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review the number of students returning from the 1^st year to the 2^nd year.
4.	Is student follow-up used for program evaluation? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review evaluation procedures
5.	Does each student evaluate the program as well as each course for instructor effectiveness, facilities, course content, supportive services, equipment and supplies, clinical component, and leadership opportunities? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review tabulated evaluations, survey, exit interviews, and other student summary evaluation.
6.	Is the program meeting the minimum criteria for pass rates on certification exams? Comments:	1	2	3	4	NA	Met Not Met Suggested Activities: Review previous years pass rate on exams

Participation in Data Collection

Project Lead The Way® has contracted with True Outcomes to conduct annual assessments of the PLTW™ program. PLTW™ student data is collected through an online registration process with teacher grade input. Long term, the data will be used to determine the success of the PLTW™ program in relation to increasing the number and diversity of high school graduates who successfully pursue careers in engineering and engineering technology. For more information, go to www.trueoutcomes.net/ pltw.

1. The Number of Teachers Participating in the PLTW True Outcomes National Assessment
2. The Number of Students Participating in the PLTW True Outcomes National Assessment

Provide a copy of the printed page from the true outcomes website which shows the school’s participation.

Oklahoma Department of Career and Technology Education

PROGRAM EVALUATION PROFILE

Instructor______________________________________________________________________

Program_______________________________________________________________________

School________________________________________________________________________

Date__________________________________________________________________________

Chairman______________________________________________________________________

Standard Standard

Met Not Met*

STANDARD 1 Instructional Planning and Organization
STANDARD 2 Instructional Materials Utilization
STANDARD 3 Qualified Instructional Personnel
STANDARD 4 Enrollment and Student/Teacher Ratio
STANDARD 5 Equipment and Supplies
STANDARD 6 Instructional Facilities
STANDARD 7 Safety Training and Practices
STANDARD 8 Advisory Committees and Community Relations
STANDARD 9 Leadership Development
STANDARD 10 Coordination/Clinical Activities
STANDARD 11 Student Accounting and Reports

*Refer to Specific Recommendations for Improvement

INTRODUCTION

Include a brief description of the program, number of teachers, type of students served, enrollment, program length, and any unique information about the program.

COMMENDATIONS OR STRENGTHS

Summarize below the major strengths of the program. Include outstanding accomplishments of students and/or the student organization.

SPECIFIC RECOMMENDATIONS FOR IMPROVEMENT

Outline specific recommendations for program improvement that should be implemented in order for the program to meet minimum standards. Each standard that was found to be deficient should be identified and recommendations for improvement made.

Standard Recommendations

GENERAL SUGGESTIONS FOR IMPROVEMENT

Outline general suggestions for the overall improvement of the program.

Standard Recommendations

School __________________________________________________