The Foreign Service Journal, April 2020

THE FOREIGN SERVICE JOURNAL | APRIL 2020 35 promotion recommendations, the stakeholders group recognized that boards need more than just one perspective (the supervi- sor’s), and they need input that would more clearly differentiate FSOs. After much discussion and debate, the group determined that promotion boards would need three different documents to reliably assess the best candidates for promotion: • The Annual Accomplishment Record (completed by all FSOs as part of the Employee Performance and Development process). • A Promotion Input Form, consisting of five 250-word blocks employees complete and one 250-word block supervisors complete to help boards assess candidates on the six promo- tion decision factors [see the graphic, opposite page]. • Multisource Ratings (MSRs) of behaviors that reflect the four core skills, collected from supervisors, colleagues and subordinates. The Annual Accomplishment Record is designed to provide a fairly objective account of the employee’s significant contri- butions and accomplishments over the course of the appraisal period. It is written by the employee with input from the supervi- sor, who also has to sign off on its accuracy. The new Promotion Input Form asks employees to describe how they have demon- strated each of the four core skills (these are four of the six pro- motion decision factors), and to also show their understanding of and ability to advance the agency’s mission. Supervisors have just one block to complete, and this is where they can recommend or not recommend someone for promo- tion, and share what they believe the promotion board needs to know to accurately assess the employee on the six promotion decision factors. Multisource Ratings: How They Work The promotion process also involves collecting data frommul- tiple sources on the extent to which promotion-eligible FSOs are displaying specific behaviors related to the four core skills. This information is collected via short surveys where respondents use a five-point scale to rate the extent the FSO does things like “seeks information from relevant sources to develop and implement appropriate solutions when problems arise” and “communicates effectively and respectfully with people at all levels of USG and host country.” These Multisource Ratings are completed by the FSO’s immediate supervisor and all direct reports, if they have any. Ratings are also collected from a carefully selected group of colleagues. Supervisors are instructed to meet with their promotion- eligible FSOs to review the clearly defined criteria for peer/other raters and identify which colleagues in the pool of potential raters best meet the criteria. The goal is to only include individuals who have a strong working relationship with the FSO and thus are in a position to have observed most of the behaviors they are asked to rate. The peer/other and subordinate ratings are aggregated across the individuals in those groups, and averages are only shared with FSOs if there are at least three respondents in the group. This is to preserve the confidentiality of the peer/other and subordinate ratings—one of the most critical elements of the process. Supervisor ratings are not confidential. Summaries of the MSRs are provided to promotion candi- dates to support their professional development, as well as to the promotion boards. Of the three new components of the promotion package, the Multisource Ratings are the most controversial. The anxiety and confusion over them are reflected in the fact that when asked if including the ratings in the promotion process would help boards better assess candidates’ core skills, more than 40 percent of the October survey respondents said no. The survey also revealed some possible reasons for this skepticism. For example, aver- age ratings, particularly from supervisors, were very high; and most FSOs did not believe the five-point scale was adequate to differentiate candidates. We also noted that too many supervi- sors (approximately 15 percent) failed to follow the rater selection process and guidance, and that the large number of peer raters we allowed (up to 15) encouraged including raters who did not truly meet the selection criteria. The Office of Human Capital and Talent Management is addressing each of these issues. For example, we are using item analyses to identify those that are “too easy” and fail to adequately differentiate FSOs. We are also modifying the rating scale anchors and reducing the maximum number of peer and subordinate raters from 15 to 10. The other major focus for 2020 is communication. Only 60 percent of our FSO respondents said their supervisor encouraged the team to participate in the MSR We named the new system “Employee Performance and Development,” leaving no doubt that developing our people was the primary objective.