Anyone who has been through the calibration process knows it can get messy…fast. Learn how to improve these meetings so that they lead to a better performance review process.
In a previous blog, I wrote about building performance reviews that are, among other things, designed to mitigate bias. This includes defining clear and consistent expectations against which all employees are assessed and limiting open text questions that are proven to introduce bias. By doing this, we can feel more confident in the objectivity and consistency of the outcome or our review process.
Unfortunately, even the most perfect performance review system will not make the human being writing the reviews any less human. They will naturally bring their deeply rooted biases into the process. Everyone needs support to ensure these biases are identified and addressed. This is the goal of a performance calibration.
Anyone who has been through the calibration process knows it can get messy…fast. Hours-long meetings, complicated spreadsheets (or, worse, no data at all) and plenty of rabbit holes have led calibration meetings to have a bad reputation and unimpressive results. Often, these marathon sessions, which cost the company tens of thousands of dollars in leaders’ time, lead to few changes to review cycle outcomes.
How might we improve the calibration meeting such that they lead to a better performance review process? How might we make the time spent truly valuable? I have run hundreds of calibration sessions over the past 20 years and have seen good, bad and ugly. Below is a summary of what I have learned about how to make performance calibration worthwhile for employees, managers and leaders.
Calibration is famously opaque. Twice a year, managers are locked in conference rooms with the (now virtual) blinds drawn. Employees know they are talking about them, but don’t know what is happening or whether it is helping or harming their chances for the performance or promotion outcomes they are hoping for.
As you talk to your employees about the performance review process, I encourage you to be transparent about why and how you calibrate. Talk about the outcomes related to fairness and consistency that you are trying to achieve. Consider sharing something like this:
“After reviews are written and before they are finalized, we conduct calibration meetings with managers. We use this time to align on how we are thinking about expectations and ensure we are consistent in how we are assessing performance. We also take time to talk about and help each other identify any biases that may have unconsciously entered the process. The goal with these calibration sessions is to ensure a more fair and consistent outcome for you.”
Not only will this transparency instill a great sense of trust and confidence in employees, it will also ensure managers take their role in calibration seriously.
Top down? Bottom up? The best way to sequence your calibration meetings depends on your goals.
Top down calibration means starting calibrations with your sr. leadership team, calibrating the leaders that report to them, and then moving to functional level sessions. The benefit in tops down calibration is alignment. Your leadership team will leave their own calibration session aligned on expectations and bring these expectations to their functional calibrations, thus driving more consistency. Tops down calibration is great for:
Bottom up calibration means conducting functional level calibration prior to calibrating with the sr. leadership team. The benefit of bottom up calibration is visibility. Your leaders will be able to review where the rest of the organization landed and consider overall rating and promotion rates as part of the session. Bottom up calibration is great for:
Whichever you choose, consider how you might reap the benefits of the other. For example, if you calibrate top down, find ways to report back on organizational performance following the completion of calibration. If you decide to calibrate bottom up, consider a conversation ahead of time to ensure alignment on the expectations of performance.
One of the biggest mistakes made in calibration is failing to focus conversation on the right people. When we aim to talk about everyone, we usually run out of time. When we just talk about the top and bottom performers, we miss the opportunity to identify middle performers who may be miscategorized.
Rather than identifying certain categories of performance to guide the discussion, I recommend focusing on the employees for whom there are questions. In other words, surface conflict. Spend less time in your calibrations agreeing on how great Joe is and more time wrestling with whether Sam is truly exceeding expectations when their peers find it hard to work with them.
You can do this by sending ratings and promotion recommendations ahead of time and asking the calibration participants to “flag” those they want to discuss. They may flag someone because they:
It’s ok to be fluid and add other employees to the list along the way. Just be clear about why you are spending time on each employee as part of the conversation.
As you enter your calibration session, be clear with managers about whether there will be an expectation to meet some sort of forced distribution of ratings. There are several approaches to managing the distribution of ratings in a performance review cycle:
Regardless of how you handle rating distributions, it is useful to think about whether the distribution of ratings is an accurate reflection of the outcomes of the business for a team, department or company. For example, if a department missed all or most of its goals, but skewed highly toward meeting or exceeding expectation ratings, there may be an important signal that the bar is not being held as high as expected.
The goal of calibration is to ensure consistency and mitigate biases. But those biases will not bust themselves. Be intentional about identifying the kinds of biases you are looking for and empowering participants to take an active role in calling them out. Here are three simple ways to integrate bias busting into your calibration conversation:
If resources allow, it can be effective to have an objective third party (usually a member of the HR team) act as a dedicated “bias buster” in the room. Their job is to listen carefully for bias language, call out inconsistency in understanding of expectations and ratings, ask questions like, “Tell me more about what it looks like to be a ‘superstar?’”
One important way to make calibration more objective is to rely on data, rather than narrative, to review and assess individual performance across the team. Ideally, this takes the form of any data that can be pulled from the review. For example, if you have rated questions in your review, along with an overall rating, you can assess them for alignment and consistency. A review of the following two employees demonstrate inconsistencies with how review data were interpreted in determining an overall rating.
Additional useful data include past review ratings, tenure in role and any other performance data, such as goal or target results, that may not be captured in the review form.
The calibration conversation is perhaps more prone to rabbit holes, side bars and circular logic than any other. The goal of calibration is to surface disagreement and use it as a way to align on performance expectations. Guiding the conversation this way takes active facilitation. Here are a few tools to use to keep the conversation focused and productive.
“We have reviewed all level 1 and 2s and talked through promotion to level 2. We will plan to move on to our large group of level 3 now, and calibrate their ratings first, then review promos from 2 to 3. Given the time, we should spend about 30 minutes on this next group”
“Sandy, it sounds like you got some really good feedback about Joe that may lead to an adjustment in his rating. I would suggest you and Katie take this offline and let me know by the end of the week what you decide. [To the group] Is this group comfortable with Sandy making this decision, based on the feedback you shared?”
Calibrations are a huge time investment. With thoughtful planning, preparation and facilitation, however, these meetings have the potential to take your performance review process and outcomes to a new level. It won’t be perfect the first time, but you will find that over time, your organization will build this calibrating muscle. Ultimately, leaders and managers will gain more clarity and alignment about expectations and the bar for performance. They will open lines of communication for feedback in the future. And most importantly, the outcomes of your performance reviews will be less biased and more consistent and fair.