Strategies for Remote Interviews with Team Candidates
In a New York Times column, Adam Grant wrote:
Credentials are overrated, and motivation is underrated. It doesn’t matter how much experience people have if they lack the drive to think creatively, work collaboratively and keep on learning. We’re not just hiring people to do a job today — we’re hiring them to make their team and their organization better tomorrow.
Once upon a time - last century, actually - employers could rely on the conferring of a college degree as evidence of a certain level of competence in the degree subject. In some areas, this is probably still true. Generally speaking, this would apply to the scientific areas of study: chemistry, physics, mathematics, etc. Unfortunately, even these area are becoming suspect as academic rigor is eroded in the interests of removing perceived barriers to this or that special interest group. To be very clear, I'm referring to the importance of thorough and complete understanding of the subject. The mine field that academia has become is indeed rife with self-inflicted and often insurmountable barriers to learning. The egregious rise in the cost of tuition, grade inflation, and credential dilution are but a few examples.
There are other factors in play. The speed at which society moves in the 21st Century is simply too fast for the four-year degree to to have any hope of staying relevant, let alone keeping up. Almost every major university offers free courses in a wide variety of subjects so it is possible for a high school graduate to craft the equivalent of a Bachelors or Masters and complete it for a fraction of the cost and in half the time. Ah, but without having completed the paper chase, how can such an industrious individual establish for a potential employer that they have the requisite competence?
Adam Grant has it right. Credentials are overrated. So how can we assess the quality and potential of team candidates? Grant identifies three key mistakes interviewers make in the interview process.
They ask they wrong kinds of questions.
They focus on the wrong criteria.
They're overly influenced by the best talkers.
If, as Grant suggests, job interviews are broken than conducting remote job interviews in the midst of a pandemic are significantly more challenging. In this post, I wish to speak to the second mistake identified by Grant and write about what we can do to identify our criteria, what we can do during an interview to elicit information about the candidate's qualifications, and a strategy for improving the efficacy of remote job interviews.
Identify Important Criteria
For the sake of example, we'll engage in a little time travel into the future and imagine having hired the perfect product owner candidate. What tasks encountered in your work day are no longer an issue with the new candidate on board? Is the product backlog now well-maintained and in a healthy state? Does the sprint runway extend out 4-5 (or more) sprints? Has a stable sprint velocity emerged (suggesting that the user stories are of higher quality and understood better by the team)? Do conflicts between areas of the business occur less frequently than in the past? Are stakeholders pleased with the results they see at sprint and increment reviews?
If our example were for a scrum master candidate, we would ask ourselves different questions for eliciting important criteria for the position. Is there less conflict among team members? Does the team understand the purpose and value for determining the effort involved to complete a user story? And again, has a stable sprint velocity emerged?
In addition to considering what hasn't been working well (and therefore illuminating what skills you want a candidate to bring to the table) it is also important to include what has been working. It will not serve the organization if one set of problems are swapped for another. Perhaps, for example, the previous product owner was well liked by the team and helped the team maintain a positive morale, but had a poorly maintained product backlog that prevented a good approximation for a release date. It wouldn't be much of an improvement if the new product owner kept a healthy product backlog but did so by driving the team as a tyrant might.
Test for Matching Skills
With a good feel for the criteria needed to hire the best candidate you can then craft a strategy for determining how well the candidate's abilities satisfy your criteria. Prepare tasks for the candidate that will verify congruity between what a candidate says they can do and what they can actually do. One approach, which I use frequently, is to present the candidate with a series of scenarios, each designed to build on how the candidate responded to the previous scenario. While I may only present a candidate 3-4 scenarios, I have several dozen in the queue and present the sequence based on how well the candidates responses to the challenge.
For example, for a scrum master role - a high-touch role that requires consummate communication skills, flexibility, and the ability to solve people problems - I may present an initial scenario as follows:
"I'm going to give you several scenarios. You are free to ask any questions you wish about the scenario and state any assumptions you are making in your responses.
You are being considered for a position as scrum master for a team that is developing a healthcare related web application for use in hospitals. This team is responsible for developing the UI/UX components and works closely with another team responsible for much of the database and middle tier components. As a new scrum master, what questions would you ask of anyone in the organization to help you quickly understand what you need to do to become effective as a scrum master for your team?"
There are many things I would hope to hear in the candidate's answer. To mention a few, I'd like to hear that they want to speak to the product owner, the stakeholders, and, of course, each of the team members. I'd like to hear that they plan to spend time in information gathering mode rather than work immediately to shape the team into some version of teams they've worked with at other jobs. I'd like to hear questions from them about what kinds of metrics does the team use and what have they shown.
There are no right and wrong answers to a scenario like this. Just answers that are better than others. And I don't expect the candidate to deliver an exhaustively thorough response.
From their responses, I might learn that they are a recipe follower or that they are flexible in adapting to the needs of the business while working to establish good scrum practices. I might learn that they really don't know scrum at all and are only good at parroting text book examples and jargon. I might hear how they would attempt to leverage several things from previous experience while acknowledging those attempts would be experiments and subject to adaptation based on feedback.
Assuming the candidate responded to the first scenario in a way that scores high marks for satisfying my criteria, I might offer the next scenario as follows:
"Assume you have been serving successfully as scrum master for this team for six months now. The product owner calls the team together and says 'I need to swap out some of the stories in the sprint for work that marketing wants done before the end of the week.' As scrum master, how would you respond to this development?"
As with the previous scenario, the candidate's response would be measured against the criteria I have established for the position. Depending on what I've heard, I may continue to offer additional scenarios that build on the candidates developing experience with the scenario scrum team.
This strategy is pursued until I'm satisfied the candidate knows what they claim to know or not. A short interview does not bode well for the candidate. A long interview does.