The Interviewer Experience, developed by APTMetrics in partnership with our clients PepsiCo and Walmart, offers an immersive approach to learning structured interviewing techniques that put a participant into a realistic, engaging hiring scenario. Video, email, text messages and attachments are used to create a realistic work environment of an urgent need to fill a critical position on their team.
BEST ADVANCE IN INTERVIEWING STRATEGY FOR TALENT ACQUISITION
In partnership with PepsiCo and Walmart, APTMetrics wins a Talent Acquisition Gold Award for Best Advance in Interviewing Strategy.
Summary
The Brandon Hall Group category Best Advance in Interviewing Strategy in the Talent Acquisition program is for best practices in creating an interview strategy that aligns with the overall talent acquisition objectives including improving efficiencies, strengthening the candidate experience, and improving the quality of hires.
Business Need
Though the term is widely used, the true impact of inclusion— cherishing and utilizing the differences that all bring— in corporate America is still largely a concept more than an actuality.
Consequently, though corporate America has utilized the term “inclusion” in a variety of different solutions; organizational policies, promotions and cultures have generally remained hard to navigate slow to change.
One of PepsiCo’s core strengths is a focus on people. A seminal paper reflecting this orientation was published by Andy Pearson (CEO at the time) in 1987 titled “Muscle-build the organization” in the Harvard Business Review. Since then, people development has remained at the heart of the business and the human capital agenda. PepsiCo’s dedication to its associates is exemplified by the third pillar in the Winning with Purpose strategy (Associates and Communities), as well as embedded in the core values as outlined in The PepsiCo Way, an aspirational framework that describes the elements that shape PepsiCo’s shared culture (Raise the Bar on Talent and Diversity), and most recently, PepsiCo’s Racial Equality Journey which invests over $500 million dollars into creating a more equitable and inclusive workplace, and where the recruitment and selection processes play a key role.
Growth and Inclusion are two key pillars of Walmart’s Global People Strategy. To support associate and business growth, Walmart is focused on incorporating best-practice approaches for identifying leaders with the capabilities to drive their business into the future. To support these initiatives, Walmart collaborated with APTMetrics to design an immersive Interviewer Experience for hiring managers to improve the accuracy and objectivity of the data collected through behavioral interviews.
The Teams
The core team for this project consisted of 17 people that represented three groups from Walmart, PepsiCo and APTMetrics.
Walmart’s team consisted of three Ph.D.-level Industrial-Organizational (I-O) psychologists along with a director of diversity, equity and inclusion (DE&I), a consultant from the Global Selection and Assessment team and a human resources (HR) project Manager.
PepsiCo’s team consisted of three Ph.D.-level I-O psychologists along with an I-O intern.
The APTMetrics team consisted of two Ph.D.-level I-O psychologists, the vice president (VP) of the Talent Acquisition practice area, the VP of Innovation, the director of Marketing, the director of Product Development and the VP of Product Strategy and Solutions. There were several other participants that supported the core team at different stages of its development and implementation.
Approximately 30 additional subject-matter experts (SMEs) from Walmart and PepsiCo participated in initial pilots of the simulation. Several additional individuals from Walmart, PepsiCo and APTMetrics are involved in the ongoing implementation.
The project took one year to initially launch. The implementation is ongoing.
Project Concept
The solution was envisioned as a self-guided Web-based multimedia simulation where participants “sit in” on different interviewing situations, answer questions about the process, and discover where a particular path of questioning can lead.
If the results of the participant’s choices are deemed incorrect, they are provided immediate feedback and have the option to re-take a module and choose a different path to fully understand the how and why of the recommended actions. This is in clear contrast to many traditional interview training options that offer simple factual presentations and artificial role playing.
The solution transforms traditionally biased interviewing and hiring practices by providing an effective, engaging way for participants to gain realistic experience applying structured interviewing techniques. It trains hiring managers how to conduct predictive and unbiased structured interviews.
The immersive multimedia simulation allows participants to engage in simulated hiring decisions and tests their learning throughout the exercise, leading to higher quality real-world hiring decisions with measurable reductions in biased outcomes.
Both Walmart and PepsiCo felt that in order to truly hire the best talent and achieve their respective people strategies, they need their hiring managers to understand and fully embrace their structured approach. For that to be achieved, both organizations felt they needed an additional approach to structured interview training.
Design & Delivery
The Interviewer Experience offers a unique, immersive approach to learning structured interviewing techniques that put a participant into a real-life business case where there is an urgent need to fill a critical position on their team.
During the training, the participant takes on the role of Riley Garcia, the newly-hired senior VP of Strategy and Operations for the “Officer” version designed for more senior employees. For the “Manager” version designed for front-line employees, Riley’s role is the newly hired manager of Operations. In both the Officer and Manager versions, Riley will be leading a team of high-performing employees who solve some of the most difficult challenges facing the organization, but the team has some open positions that need to be filled. While the roles for both versions are fictitious, the context was branded specifically for Walmart and PepsiCo respectively. A fictitious organization, Palladium, is used for the context for the generic Officer and Manager shelf versions available to other organizations.
In the first part of the Interviewer Experience, learners become familiar with their new role and the immediate need to fill a critical position on their team. They are introduced to structured interview process that they will need to follow to fill the open position on their team. Before actually conducting interviews with three finalists for the open position, they get insights from their peers who made hiring mistakes in the past and also observe others applying the structured approached for a different role. Thus, they learn how to conduct an interview and then observe others applying what they just learned before they have to apply it themselves.
Next, the learner applies what they just learned and observed earlier in the simulation by conducting interviews with the three finalists for the open position on their team. Learners ask questions in a standardized order, determine follow-up questions based on candidate responses and then use structured evaluation guidelines to make final ratings for each of the three candidates. The simulation ends with a discussion involving Riley’s peers who sat in on the three interviews. The discussion helps to reinforce the structured interview process, provide insights about how bias can impact ratings and provide guidance on how to properly eliminate bias in the interview process.
Integration
A major goal of the Interviewer Experience solution is to help support the talent acquisition strategy by creating a robust structured interview process. The x-factor of this program is that it was developed in complete partnership with end-user clients.
This provided a powerful design process and superior results driven by multiple perspectives from Walmart and PepsiCo combined with the scientific pedigree of the behavioral experts at APTMetrics. This helped create an organic direct link from their talent acquisition strategy to the design of the program. Hiring experts from the partner companies helped create highly authentic scenarios and provided extensive input on areas where bias is most likely to affect hiring decisions.
The Interviewer Experience simulation will play a crucial role in achieving the PepsiCo Winning with Purpose strategy by training associates in interviewing best practices to help increase objectivity and reduce bias, inject rigor, and ensure standardization of their processes. By having managers engage in an immersive experience that allows their associates to learn in a low stakes simulation and course correct mistakes, associates can take this knowledge and apply it when hiring top talent. This contributes to their best-in-class selection strategy that places a strong emphasis on the candidate experience.
To support their Global People Strategy, Walmart agrees that by reinforcing the fundamentals of behavioral interviewing, they are also reinforcing their culture of associate growth and inclusion, reducing the potential for bias to impact selection decisions, and ensuring a fair and equitable hiring processes for their candidates.
Measurable Benefits
The Interviewer Experience was designed to address three broad learning objectives:
- understanding the value of a taking a structured interview approach including the impact on the candidate experience;
- understanding how detect and address interviewer bias in the interviewing process;
- and acquiring basic interviewing skills (e.g., how to gather and evaluate data, proper note taking, how to effectively manage an interview to enhance process efficiency and the candidate experience).
In addition to enhancing knowledge and skills associated with the learning objectives, a key objective of the Interviewer Experience was to engage learners and create articulate advocates in the organization that will help build organizational support not only for the training, but also for the continued application of the structured approach taught in the simulation across the organization.
In order to measure the impact of the program on these targeted areas, participants in the program were provided an evaluation survey to complete at the end of the simulation. The survey uses a five-point rating scale ranging from strongly disagree (1) to strongly agree (5). The expectation was that ratings 3.5 and higher would be good while ratings above 4.0 would be considered excellent.
All the ratings provided by participants from Walmart and PepsiCo are above 4.0 with an overall average of 4.4 across all survey items and respondents.
These results are particularly impressive given that 85% of the survey respondents had conducted 21 or more interviews (65% of the respondents had conducted over 40 interviews). These were very experienced interviewers, yet 91% indicated they increased their knowledge of how to conduct an effective interview while 94% indicated they planned to apply the knowledge and skills they learned in the Interviewer Experience.
Participant Comments
- “I am impressed with this entire experience. This is what learning should feel like. It’s as close to a consumer grade simulation that I’ve seen, and it captivated my attention and even challenged me in some parts despite completing an extensive volume of interviews…”
- “The method of a role-playing training approach utilizing, videos, reading and other mediums really made this an engaging program!”
- “I thought the use of videos and pictures was great. The media greatly enhanced the sense of being in the action.”
- “Questions were not ‘gimme’ questions! You had to pay close attention to details and think through responses.”
- “The detailed feedback was helpful. Even when it was ‘almost’ right but could have been better, there was specific feedback about what to consider.”