EDTECH 505: Evaluation Proposal - Response to RFP

img source

Evaluation of Determining Instructional Purposes (DIP) training program

A Proposal Submitted to Far West Laboratory for Educational & Research Development

Submitted by:

Silicon Valley Education Group

Matthew Doyle

EDTECH 505-4202

Boise State University

Spring 2015

Submitted to:

Far West Laboratory for Educational & Research Development

and

Dr. John Thompson

EDTECH 505

Boise State University

Summary

In March 2015, Far West Laboratory for Educational & Research Development issued a request for proposal (RFP) for evaluation of its Determining Instructional Purposes (DIP) training program. This proposal from Silicon Valley Education Group of Mountain View, California is submitted in response to the FLW RFP.

Description of the Program Evaluated

Far West Laboratory’s Determining Instructional Purposes (DIP) training package consists of a Coordinator’s Handbook and three training units: Unit 1 – Setting Goals, Unit 2 – Analyzing Problems, Unit 3 – Deriving Objectives. Each unit is comprised of four-to-six modules led by a qualified coordinator that offer training to guide participants towards meeting instructional objectives. Modules typically consist of relevant reading materials, activities in which participants develop relevant skills, and feedback for these activities. The delivery timeline for these units is flexible and FLW estimates the training time required for Units 1 and 3 is 10-15 hours each and 12-18 hours for Unit 2.

While each unit is self-contained and therefore not a necessary part of other units, the units were designed to work together as part of a comprehensive package. This training package is sold either by individual unit ($8.95) or as a complete set of 3 units ($24.95). Each unit package includes print copies of the Coordinator’s Handbook and all relevant reading and training materials.

Evaluation Method

The primary purpose of this evaluation is to determine if Far West Laboratory (FWL) should allocate resources towards the marketing and sales of their Determining Instructional Purposes (DIP) training program. Silicon Valley Education Group (SVEG) will conduct the proposed evaluation in consultation with FWL and provide the evaluation results to stakeholders to be identified by FWL.

In order to determine if FWL should invest resources in the market and sales of DIP, it will be necessary to address the following:

• Are the instructional materials for all units designed in a way that will assist the target audience (both course participants and the course coordinator) with meeting the stated instructional objectives?
• Do participants think the instruction was adequate in meeting their learning needs?

In order to answer these questions, an evaluation of this scope will require multiple assessments and sources of data. For the assessments, SVEG proposes 1) assessing course participants and training coordinators throughout the delivery of the units and 2) assessing course participants and training coordinators at the completion of each unit. The frequency and scope of assessments will be determined in consultation with FWL after FWL provides SVEG with unit materials and a proposed workshop schedule; however, based on the information provided, SVEG proposes the following assessments:

• Formative Assessments
• Online surveys to be completed by workshop participants and training coordinator as they progress through the course
• Online or telephone interviews with workshop participants and the training coordinator at regular intervals as they progress through the course
• Summative Assessments
• Online or phone exit interview with workshop participants to gather qualitative data about whether or not participants thought the instruction was adequate in meeting their learning needs
• Course participants will complete an end-of-course skills assessment to determine if the stated workshop learning objectives were met
• To further determine if the stated workshop learning objectives were met, school administrators will complete two online surveys, one prior to the course start date and one after the completion of the course. Both surveys will be designed to collect information pertaining to how well the educator meets the unit’s stated learning objectives, with the expectation that prior to participation in the workshop, the participant will not have met the learning objective. The first assessment may be administered any time prior to the workshop start date; the second assessment should be administered two months after the completion of the units. If necessary, follow up interviews will be scheduled with administrators to collect more qualitative data

Task Schedule

As for the duration of the evaluation, preliminarily suggest administering the workshop in one-hour sessions, three times a week. Given the projected unit time requirement of 10-18 hours, SVEG recommends running the units, and subsequently the evaluation, for six weeks. Preparation for the evaluation and follow up assessments with administrators two months after the evaluation will require a total of eight months. Since the start and end dates for this project are yet to be determined, the following schedule uses January 15th, 2015 - September 15, 2016.

Task	Meeting Date or Deadline
SVEG evaluation team and FWL principal stakeholders meet via video conference to determine all relevant stakeholders for future SVEG evaluation discussions	1/15/2016
FWL provides SVEG with unit materials, any existing end-of-unit assessments, and information regarding any schedule restraints for administering the units	2/1/2016
If no end-of-unit assessments exist as part of the training materials, FWL will create, and provide to SVEG, end-of-unit assessments	2/15/2016
SVEG evaluation team and FWL-identified stakeholders meet via video conference. SVEG present preliminary evaluation project for review, including survey/interview tools and questions, and projected schedule of assessments.	2/15/2016
FWL and SVEG meet via video conference to discuss any necessary changes to proposed evaluation materials, including survey/interview questions and schedule	3/1/2016
SVEG submits final evaluation, including finalized versions of survey/interview questions and unit schedule.	3/20/2016
For all three DIP units: FWL identifies and registers workshop participants; FWL identifies administrators; FWL identifies unit coordinators	4/15/2016
SVEG delivers pre-unit surveys to administrators	4/20/2016
Six-week training begins. SVEG conducts all formative and summative assessments except the follow-up interview with administrators, which is to take place in on 8/10/2016	5/1/2016 - 6/10/2016
SVEG present final report to all FWL-identified stakeholders and discuss report findings	9/15/2016

Project Personnel

Matt Doyle

Chief Evaluator at SVEG. Mr. Doyle has over 10 years of experience conducting evaluations for educational service providers in both public and private sectors. He holds master’s degrees in education and educational technology from Boise State University.

Bob Gladwell

Senior Data Analyst at SVEG. Mr. Gladwell has over 15 years experience working in data analysis and provides leadership and consulting in survey design. Mr. Gladwell holds a master’s degree in data analysis from Boston University.

Amy Chun

Junior Evaluator and Data Analyst at SVEG. Amy has three years of experience conducting evaluations for both private and public sector institutions. She holds a bachelor’s degree in data analysis from Carnegie Mellon University.

Budget/Fee

Note: The proposed evaluation project requires no travel. All surveys and assessments will be conducted online or through telephone. The following represent estimates for personnel expenditures and supply costs

Personnel

Personel	Daily rate (8 hours)	Subtotal
Matt Doyle	50 days x $400 day	$20,000
Bob Gladwell	30 days x $400 day	$12,000
Amy Chun	50 days x $250 day	$12,500
	Total	$44,500

Supplies and Materials	Cost	Subtotal
Print and binding of final evaluation report	$2 per printed and bound copy x 20	$40
	Total	$44,500

Total Estimate: $44,500