Making

Multi-rater
Feedback Work in
Professional Services Firms
 2007

T he Missing Boss
On TV shows an office is a place where the boss is near at hand, drifting in and out
of the workplace to observe, coach and direct. This may represent life in some offices,
but it isn’t the case for people in most professional services firms. Professionals, such as
consultants, accountants, architects and lawyers typically work on many different accounts
or projects, continually moving on to new teams with different leaders. At Clark Nuber, an
accounting firm based in Bellevue, WA, an associate might work for ten or more managers over
the course of a single quarter.

For many firms, continually moving people on to new projects under different leaders is the only
possible way to operate. But the implication is that the traditional boss is missing. The formal
boss on the org chart is only loosely involved in directing day-to-day work and may rarely get a
chance to observe an employee in a work situation. At Jones Walker Waechter Poitevent Carrere
& Denegre, a New Orleans based law firm, while there is a single group practice head, an
associate in reality has between four and twenty bosses. This isn’t normally a problem; direction
is provided by whoever is leading the project. Where it is a problem is when it comes to
performance appraisal‑there is no one person who can do that appraisal.

When it is time to give formal feedback to employees on how well they are working, the fact
that the boss has been missing undermines the process in professional services firms. This is why
multi-rater feedback from project heads, partners and peers is essential.

T

he Big Choices to Make
There are three big choices professional services firms need to make in using multi-
rater feedback:

1. Do you use multi-rater feedback only for developmental feedback or do you

also have it affect someone’s performance appraisal and hence their
compensation and career advancement opportunities?
2. Do you use upward feedback where the employees rate their bosses?
3. What sort of technology do you use to make the administration fast and easy?

T he First Choice: Appraisal or Development

In HR circles the common wisdom is that multi-rater feedback (particularly if it includes
upward feedback) should only be used for development. This stems from a concern
that as soon as you attach pay to the discussion, the pay issue becomes so salient that it drowns
out the feedback people are supposed to be paying attention to. This is in fact not just a multi-
rater issue, it is one of the classic dilemmas of all employee performance management systems:
as soon as you talk pay‑whether that is done solely in the context of a boss’s assessment or
in a multi-rater assessment-thoughts about how to develop skills or improve behavior go out
the window.


 2007

At this point readers will be hoping

for a sentence to the effect, “The Business Value
solution to this classic dilemma is.”
However, if there really were a tidy, “Our use of multi-rater feedback has caused employees

widely applicable solution it wouldn’t to re-evaluate how they interact with each other and
mindful of how they do their work. Sometimes the
be a classic dilemma. A few companies
feedback has led to conversations that begin, “What? I
have two separate discussions, one
can’t believe that people feel that way about me!” But
on development and one on pay, but
the intent is never to be critical; it is to offer constructive
that means spending twice as long
feedback. It’s the people who are surprised who have
working on the performance appraisal
the most to gain from the process, and by adjusting their
process, covering much the same
behaviors they can improve their teamwork and their
material in each discussion.
own careers. A couple of people didn’t get good reviews
and they’ve taken it as a challenge and completely
The problem is only really avoided
reformed themselves. If the feedback had just been from
if performance appraisal is an on-
one person, their manager, they might have dismissed it.
going process, not a once-a-year
event‑and that is what professional
Renee Boyce
services firms should aim for. If a
Human Resources Manager
professional is getting feedback
J.H. Findorff & Son Inc.
over the course of the year then
the formal appraisal process will
contain no surprises and becomes
a much less intense discussion. Of course, HR departments have always argued that employee
performance management should be an on-going process, but in most organizations that didn’t
happen because it seemed like too much work. This problem does have a solution. Employee
performance management technology makes on-going performance management much
easier. Organizations still need a culture that supports this, but the technology helps create a
breakthrough for many companies. Aside from automating the complex task of gathering multi-
rater feedback, for example, the technology facilitates doing reviews at the end of every project,
so everyone can get timely feedback on their performance. Technology also makes it easy for
both managers and professionals to track relevant performance information on an ongoing
basis. Sometimes this is simply a matter of copying part of an email into a performance log or
making a quick note about a client’s comments so that when the performance evaluation is
done managers don’t have to rely on memory alone.

Circling back to the specific choice facing professional services firms: do we use multi-rater
feedback for appraisal or just for development? The answer is to do what the culture will sustain.
Some organizations may say, “I don’t see what the problem is, we can just fold the multi-rater
feedback into the appraisal as one component and it works fine.” Other organizations will say,
“We can’t possibly do that without sowing discord and undermining the intent of giving people
helpful feedback.” Each firm must assess what works for them.


 2007

The choice is not quite so stark as it may appear. The problem of the missing boss means that
performance appraisal and subsequent compensation and promotion decisions will inevitably
depend on feedback from the people who have worked with an employee. If an employee gets
negative feedback from a multi-rater process then they should know that it does not bode well
for their career, even if it is not formally part of the appraisal process. Most professional services
firms opt to stress that the main purpose of multi-rater feedback is for development, but this is
not always the case. At Jones Walker Waechter Poitevent Carrere & Denegre the main point of the
process is to have multiple bosses appraise an associate.

The anxiety some HR people may feel about making these kinds of choices actually stems
from the fact that they are trying to solve a problem that is not theirs to solve. The design of an
employee performance management system will not solve the tension between development
feedback and appraisal any more than the design of a tennis racket will win you a match at
Wimbledon. Employee performance management is really a matter of managerial skill. The
design of the system merely provides consistency across the organization, compliance with legal
issues and hopefully a helpful framework that skilled managers appreciate.

What managers need is training in how to do performance management and particularly

how to make it an on-going process so that there are no surprises at the end of the year. It is
up to mangers, not HR, to create a climate where honest feedback is appreciated and where
employees feel that managers are
being fair to them in appraisal.
Giving Associates Helpful Feedback

T he Second Choice:
Upward Feedback
The second big choice, whether
or not to use upward feedback, is a lot
simpler than the first one. Traditionally
a boss judges employees, not vice
versa. In some organizations it is
culturally unthinkable for employees to
rate their boss, in others feedback on
how to improve is appreciated.
The problem with upward feedback
is the realistic fear that bosses will
take revenge on employees they
think are giving them bad feedback.
This is handled by making feedback
anonymous. However, in some
environments employees don’t feel
“It’s in the nature of audit and tax work that our teams
change all the time. Our employees, both in audit
and tax, expect a lot of feedback so we do multi-rater
feedback quarterly. The forms are pretty simple, we
ask questions about business management, practice
development, personal productivity and so on; seven
areas in all with two to five questions per area. It takes
someone about ten minutes to do the quarterly feed-
back. We also do a more comprehensive multi-rater
feedback annually. This takes about 30 minutes per
person reviewed. The process makes our teams more
efficient because if a team member doesn’t realize they
are doing something wrong they can’t improve.”
Lizzie Rahm
Human Resources Generalist
Clark Nuber PS


 2007

safe giving upward feedback even when it is anonymous. The other concern is that if leaders are
rated by employees then they will be afraid to make unpopular decisions.

There is no question that feedback from employees can be very helpful for a manager’s
development. There is also good reason for feedback from employees, whether obtained
formally or informally, to affect a manager’s performance appraisal. If employees think someone
is a lousy manager they are almost certainly right and it will hurt the firm if that person is given
more managerial responsibilities.

Again, the answer is to do what the culture of the organization will sustain. Upward feedback
is desirable, but it may not be practical. If the culture is hostile to upward feedback then the
managing director, with HR’s help, should try to change the culture, but not introduce upward
feedback before the firm is ready.

T he Third Choice: Technology and Administration

Not so long ago the question would have been whether or not to use technology to
support multi-rater feedback, but the issue now is more a matter of what technology
to use. The administration of multi-rater feedback can be a real headache; much greater than it
appears at first sight. The problem is
that the number of feedback forms to
be managed can be very large. At J.H. A Focus on Development
Findorff & Son, a construction company
”We use multi-rater feedback for development
based in Madison, WI, half of the staff
not appraisal. We don’t even like to use numbers
have more than 10 multi-rater feedback
because we don’t want it to seem like a score. We use
forms to fill out; a quarter have 25 to 35
categories such as “exceeds at”,“sufficient for position”
forms to complete, and some even have
and “training needed”. The implication is never that
close to 50 forms. For HR, tracking these
someone is failing, only that they would benefit from
forms and averaging the scores is very
further development.”
time consuming without technology
and relatively easy when technology Ileana Prado
is in place. Technology also makes Director of People & Facilities
life easier for the people providing PainePR
multi-rater feedback, and since a
professional’s time is very valuable,
anything that cuts the time needed to spend on this is of high value.

In choosing a technology the critical thing to understand is that every firm uses multi-rater
assessment a little differently. Organizations need to test how well a given software product
handles their particular workflow and how flexible the software will be in accommodating future
needs. This can be difficult to assess. The best way to understand it is to sit with the vendor and
carefully step through the process in detail. You need not necessarily walk through the steps


 2007

using the software itself. It may be easier to simply talk through each step and have the vendor
explain how the software would handle it, perhaps making sketches as necessary. Sketches help
because sometimes there is so much detail on a screen that it distracts from the core point you
are trying to learn. The trick is to spend enough time and go into enough detail that you end up
with a very specific understanding of how each stakeholder will interact with the system and
what administrative steps you will need to take to get the output you want.

Good systems accommodate almost any kind of process and should be configurable without
needing to go back to the vendor or relying on internal IT resources. This is important because
the assessment process that your organization uses today-may not suit you tomorrow.

T he Good News
HR professionals rightly have to worry about all the issues that can arise when they
implement multi-rater feedback. But if we focus too much on difficulties then we
can lose sight of the real purpose. In a professional services firm the employees are typically
smart, ambitious, diligent people doing complex work. They need to develop a wide variety
of technical, client management, project management, personal efficiency and team skills.
Multi-rater feedback gives them the feedback they both need and crave. It can also be critical in
reinforcing the sort of team culture you are striving for. If employees get regular feedback from
their peers and managers they have an opportunity to improve. If managers are willing to accept
upward feedback then that is a great way to develop people-management skills, something
typically not covered in law school, a CPA program or other professional training.

The great advantage of technology is that it makes the administrative burden manageable,
which allows HR to spend more time on the cultural and training aspects so important in an
effective employee performance management system. Professional services firms are ultimately
based on their human capital, and investing in HR systems that make that human capital
stronger is a wise strategic choice.

Creelman Research www.creelmanresearch.com specializes in research and writing on

human-capital management. Based in Toronto, Canada, Creelman Research serves clients in
North America, Asia, Europe and Africa. The most significant work of Creelman Research is on
improving practice in reporting to the financial markets about human-capital intangibles.

NOTE: The professionals quoted in this white paper are clients of Halogen Software. Halogen
Software is a leading provider of EPM software and other talent-management solutions.

© 2007 Creelman Research and Halogen Software. All rights reserved.