EDTECH 503: Mod 7 - Formative and Summative Evaluations

(C) Matt Doyle

Making Sense of New Information

As a lifelong learner, I’ve often had those moments when new information must find a home among the cul-de-sacs of existing schema. For example, this week we took a look at the Instructional Design definitions of formative and summative assessments, two concepts that already exist in my mind as a former K-12 educator. So as I read through Smith & Ragan’s (2005, pg 326-369) discussion on evaluations, I experienced all the familiar signs that my existing understanding was being challenged and that space must be made to accommodate the new information. Immediately I put up my guard, defending my desire to leave good enough alone, a terrible but unconcious reaction to working and straying away from the path of least resistance! It took a few moments of mental preparation, but soon enough I was ready to begin finding ways to make new connections. What you'll find here is my attempt at reconciliation between new and old understanding.

To elaborate, as a teacher I understood formative assessments to be opportunities to check for student understanding periodically throughout the course, or those moments when a teacher must determine if their instruction is helping a student reach a learning goal. I think the following definition from Carnegie Mellon’s Eberly Center for Teaching Excellence and Educational Innovation is apt:

The goal of formative assessment is to monitor student learning to provide ongoing feedback that can be used by instructors to improve their teaching and by students to improve their learning. More specifically, formative assessments: 1) help students identify their strengths and weaknesses and target areas that need work; and 2) help faculty recognize where students are struggling and address problems immediately.(Formative vs Summative Assessment - Teaching Excellence & Educational Innovation)

It is possible my understanding is flawed, but I think Smith & Ragan’s definition of formative assessments is related to the Carnegie Mellon definition, yet distinct. As I see it, the focus is more on the materials and procedures, it must occur prior to officially launching the instruction, and it is separated from any evaluation of instructor effectiveness (because it is out of the scope of the roles and responsibilities of an ID).

Smith and Ragan (2005, p. 236) state formative evaluation occurs when “the designer evaluates the materials to determine the weakness in the instruction so that revisions can be made to make them more effective and efficient”. This is not unlike what a teacher must do to formatively assess a student’s mastery of a learning goal periodically throughout a learning “program,” lesson, or unit. However, since an ID cannot be present during instruction, the evaluation and feedback must occur in simulated environments with test groups and during trial periods, all before the instruction is officially launched. A teacher has the luxury, in a sense, of being present during instruction to watch the learning take place, find areas of struggle, and make adjustments. This creates an environment that makes the design process constantly and rapidly iterative; every day a teacher learns something new about how the designed learning is functioning that can be incorporated into future iterations of the instruction.

Bringing it Together

My take away this week is that it would be ideal for an ID to watch the course take place, record feedback and assessment data, and then return to improve upon the instruction. This, however, seems difficult to do. Therefore, the best approach is to incorporate as part of the ID a way to gather assessment results from instructors, as well as other data regarding attitudes and perceptions, as a way to inform future iterations of the instruction. In other words, the instructor in the classroom is a valuable resource for IDs and should be consulted when revising Instructional Design programs.

Finally, this chapter has made me enthusiastic about taking EDTECH 505: Evaluation for Educational Technologists. I think the science behind testing validity is fascinating, and after attempting to craft a number of questions for my own ID project, I further understand how important it is to ask the right questions and know why they are being asked. I look forward to combining the skills learned in this course with those of EDTECH 505; I think it will help make me an outstanding ID.

References

Formative vs Summative Assessment - Teaching Excellence & Educational Innovation - Carnegie Mellon University. (n.d.). Formative vs Summative Assessment - Teaching Excellence & Educational Innovation - Carnegie Mellon University. Retrieved from https://www.cmu.edu/teaching/assessment/basics/formative-summative.html


Smith, P., & Ragan, T. (2005). Instructional design. (3rd ed.). Hoboken, NJ: Wiley Jossey-Bass Education.