Assessment Development, Revision & Implementation
Can we upgrade our participant experience?
The assessment itself can be a personalized web-based virtual experience interwoven with role-play exercises, in-box activities and situational judgment items. As participants move forward in the integrated simulation, they watch videos, meet with role-players (video or phone-based) and respond to telephone calls/voicemails all centered around business challenges in a fictitious environment. This represents an opportunity to enhance face validity and create an immersive multi-media experience that is contextualized to your work environment, business challenges and competency model.
What will our assessor experience be like?
Update assessor and coach training programs to align with any virtual changes. Consider how assessors are going to access assessment data in a single location (e.g., simulation observations, third-party test results, interview notes, ratings). Identify how assessors or coaches will store completed reports. Implement an audit strategy to verify that all data is in the correct location and that correct user access is maintained.
Should we record it?
Consider recording all interactions (e.g., role plays, interviews, presentations) to allow assessors to review for rating and evaluation purposes. This can also be a very powerful feedback tool with participants and support future rater calibration exercises. Recordings can also be used to standardize the messaging presented to participants by leveraging tools like a video “trailer” for orientating participants to the process.
How can I mitigate risk and ensure that the solution is still valid?
Collaborate with appropriate subject matter experts (e.g., industrial/organizational (I/O) psychologists) who can evaluate the impact of any changes on the integrity of the assessment solution and the need to update the job analysis and collect additional validation evidence. If the assessment data is being used to inform selection and promotion decisions, please remember this is considered a selection procedure under the federal Uniform Guidelines for Employee Selection Procedures (U.S. Equal Employment Opportunity Commission) which govern the defensibility of selection procedures. Note this also includes algorithms based on artificial intelligence, machine learning and big data.
How can we ensure successful implementation?
Consider the CHRO or another executive sponsor as an initial pilot participant. Take time to anticipate risks, mistakes and low-probability events. Build contingency plans to offer real-time solutions (e.g., no connectivity, hosting issues, inappropriate data access, firewalls, distractions in the assessment or feedback environment), and create checklists & tips for each administrator, assessor, coach, or role player. Consider working with a consulting firm or assessment provider who has proven the concept and demonstrated a track record of success.
Are we meeting stakeholder expectations?
Focus attention on measuring the ROI of the initiative. Determine how the program currently, as well as in the future, will define and deliver value to stakeholders within and outside the company. Evaluate how the measurement strategy may change given the transition to a virtual assessment approach.