Formative Evaluation
Names of members
- Amy Papst
- Michael Hawkes
- Kathryn Thompson
- Charity Jackson
- Jennifer Whitt
Subject Matter Expert (SME)
Purpose: Provide meaning feedback and evaluation of our approach and tools. Providing real life experience and direction in how it might be facilitated in a large organization.
Characteristics: Our SME is a warehouse leader with 5 supervisors, 15 leads and over one-hundred-line level direct reports. She is a corporate warehouse leader with over 10 years of people leading experience including onboarding, performance management, and accountability coaching experience.
Target Learner
Purpose: Provide critical understanding of the learner’s perspective in our approach, accessibility, and stickiness of information.
Characteristics: One of our target learners is a corporate employee with 20 years’ experience in corporate environments. He was also part of a Learning and Development team with regards to occupational safety in a previous position.
Another target learner is a manufacturing employee with 5 years experience in a warehouse and production environment. He is a line level employee producing products and maintaining quality standards while leading a rotating team of other plant employees.
Formative Evaluation Process
We reached out to an SME and two target learners to share our instructional materials and reflect on how the target learners would relate to the training approach.
SME
- Receiving & Returns Manager, Stitch Fix Warehouse, Brownsburg
SME Interview Questions
- What about this approach works well?
- What would you add or change?
- How would your employees/team receive this info?
Target Learner
- Senior project manager at Covance
- Warehouse team member
Target Learner Interview Questions
- Would you expect to have this type of training from your employer?
- Are the bingo and coloring helpful to encourage focus?
- Are the bingo and coloring too “campy”?
Outcomes
| Instructional Strategy Source | Problem | Proposed Change in Instruction | Evidence and Source |
|---|---|---|---|
| Hands-on activities: BINGO game | For some employees, it may not seem genuine | Make the BINGO game an optional activity; create a high-value prize to incentivize participation | SME, Target Learner |
| Coloring of the mask | No follow-through | Provide actual plain white masks to color with fabric markers | SME, Target Learner |
| Course Content | Slides 14-24 are text-heavy | Add graphics and videos to increase visual interest | Target Learner |
| Course Content | Lack of references on slides | Add a reference slide at the end of the PowerPoint | Target Learner |
| Course Content | Double negative on slide 6 | Correct wording | Target Learner |
| Instructor Handbook | Lack of reference page | Add a reference page to the end of the instructor’s guide with URLs to course content | SME, Target Learner |
Revision Plan
The editorial changes noted above will be completed prior to a pilot test at a single corporation. Additional changes to hands-on activities will be considered after feedback based on completion at pilot site. Further revisions will be considered after feedback from HR and managers after implementation is complete.