WF Rethinking Hiring

RETHINKING HIRING

WORKFORCE 20NEXT

The Power of Structured Interviews in Selection Systems

Valid selection systems are critical for organizations to make informed hiring decisions and identify individuals who are likely to succeed in their roles and stay with the organization longer.

Over the years, cognitive ability tests have been widely viewed as the most effective predictors of job performance [i] [ii]. However, two recent articles published by Sacket and his colleagues [iii] [iv] challenge this long-argued belief and provide significant new insights that should be considered by anyone involved in designing and implementing hiring processes in any organization for any role.

These articles are an update to Schmidt and Hunter’s [i] (1998) seminal meta-analytic research, which summarized the predictive validity of commonly used selection tools. Sackett, Zhang, Berry, and Lievens (2023) [iv] provide not only an updated summary of the predictive validity of commonly used selection tools but also a very useful visual summary of common selection tools’ validity and “black–white” differences. In other words, it summarizes hundreds of studies to help organizations design hiring processes that leverage tools that best predict future job performance while also balancing diversity objectives.

While Sackett et al. (2023) [iv] provide a lot of great information and recommendations regarding broader principles of selection system design, there are a few key points the authors make in regard to structured interviews that are worth calling out.

Structured interviews have the highest predictive validity of any selection tool.

Which tool is the best predictor of job performance?

The selection tool that best predicts future job performance is the structured interview. This is a significant shift from the previous belief that cognitive ability tests are the best predictors of job performance. Structured interviews involve a standardized set of questions and evaluation criteria to assess a candidate’s job-related knowledge, skills, abilities and other characteristics (KSAOs). According to Sacket et al. [iv], structured interviews are more effective in predicting job performance because they better measure how an individual will behave in a specific job. The interview questions are tailored to the specific job and reflect the critical KSAOs required for success in that role.

The efficacy of structured interviews varies WIDELY based on several key variables.

Are all structured interviews equally effective?

Sackett et al. (2023) made a second important point about structured interviews in their research. They found that the effectiveness of structured interviews varied significantly in the studies they analyzed. They looked at the predictive validity of structured interviews across dozens of studies and found that the credibility interval for structured interviews ranged from 0.18 to 0.66.

To put this into perspective, the Department of Labor [v] considers a tool with a predictive validity coefficient below 0.20 to be of uncertain value in a selection process. On the other hand, any tool with a predictive validity coefficient above 0.35 is considered useful for selection. In simpler terms, some structured interviews are definitely more valid, and effective than others.

Are your interviews worthless?

How do you know if your interview process genuinely helps optimize the outcomes of your interviewing process (e.g., improves quality of hires, reduces turnover, mitigates bias, enhances the candidate experience, and ensures the legal defensibility of the process)?

Many organizations believe that as long as all candidates get the same set of questions, they have a “structured interview” that will be highly useful in selecting candidates. Unfortunately, this is not the case. In order for organizations to optimize the outcomes of their interviewing process, they need to focus on three key areas:

Design

Organizations need to make sure the interviews are focused on the true requirements and differentiators of success in the role. These requirements and differentiators are typically identified through a job analysis. If your interviews are focused on the wrong KSAOs/competencies (or something even more abstract like “fit”), the desired outcomes of the interview will be be significantly suboptimized.

Build

Organizations need to construct interview guides that have questions linked back to the targeted KSAOs/competencies; clear evaluation standards (e.g., behaviorally anchored rating scales) to ensure consistency in evaluating candidate responses across interviewers; and a clearly articulated process that is consistently applied across candidates.

Train

The last area often given short shrift by organizations is interviewer training. Providing a well-constructed interview guide based on the competencies identified in a rigorous work analysis for the targeted role without training interviewers is like giving the keys to your Lamborghini to your 15-year-old child who just received his/her learner’s permit. Research has shown that training interviewers is critical to ensure your organization optimizes the outcomes of your interviewing process. [vi]

How to Optimize Your Interviewing Program

Step 1. Design – Ensure, through a job analysis, that interviews are designed to effectively measure the true requirements and differentiators of success in the role.

Step 2. Build – Create a structured process, interview guide(s) and rating guidelines that measure critical KSAOs/competencies required upon entry into targeted roles.

Step 3. Train – Ensure your training plan meets the needs of your learners and achieves the targeted learning outcomes (e.g., understanding the organization’s hiring process; recognizing and addressing bias in the interview process; knowing how to open, conduct, and close the interview; effectively managing the flow of the interview; and objectively evaluating interview data).

Structured interviews have the potential to be our best tool in making good hiring decisions that are fair to all applicants. However, the extent to which corners are cut in identifying the true requirements and differentiators of success for the targeted role, building quality interview guides/processes, and training interviewers will determine the outcomes and impact of your interviewing process.

Please reach out to APTMetrics if you are interested in discussing how you can improve your interviewing process through designing, building, and training.

[i] Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262-274. DOI: 10.1037/0033-2909.124.2.262.

[ii] Schmidt, F. L., Oh, I.-S., & Shaffer, J. A. (2016). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years of research findings. Fox School of Business Research Paper. Retrieved from https://ssrn.com/abstract=2853669

[iii] Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107, 2040-2068.

[iv] Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (Forthcoming). Revisiting the Design of Selection Systems in Light of New Findings Regarding the Validity of Widely Used Predictors. Industrial and Organizational Psychology: Perspectives on Science and Practice. Advance online publication. https://www.siop.org/Research-Publications/IOP-Journal/IOP-Focal-Articles.

[v] U.S. Department of Labor Employment and Training Administration (2000). Testing and Assessment: An Employer’s Guide to Good Practices.

[vi] Levashina, J., Hartwell, C.J., Morgeson, F.P., Campion, M.A. (2014). The Structured Employment Interview: Narrative and Quantitative Review of The Research Literature. Personnel Psychology, 67, 241-293. DOI:10.1111/peps.12052DOI:10.1111/peps.12052