To compete effectively with the private sector and attract the best candidates, the State Department must continuously improve processes.

BY PATRICK T. SLOWINSKI

The Foreign Service Officer Assessment (FSOA), formerly the Foreign Service Oral Assessment, moved to a virtual platform in May 2024. This change represents the completion of a historic move from the long-established in-person assessments, which often required candidates to travel to Washington, D.C., at their own expense to complete the exercises in one day.

Reducing this candidate burden, and thereby expanding access to the Foreign Service career, was the primary driver in moving the FSOA online. The modification will increase the diversity of the applicant pool by leveling the playing field for candidates of all socioeconomic groups and geographic locations. It also modernizes the FSOA to align with best current evaluation practices.

The State Department competes with the private sector and other government agencies to attract candidates with the knowledge, skills, and experience needed to represent the U.S. government and its citizens overseas. To keep an edge in this competition, the Foreign Service Board of Examiners (BEX) must continuously modernize and improve Foreign Service candidate assessment processes and materials while maintaining the State Department’s reputation as a gold standard in employee hiring assessments.

During the last two years, BEX has shifted that modernization into overdrive.

The Transition in Detail

BEX previewed the change from the in-person FSOA to a virtual platform in the October 2022 edition of The Foreign Service Journal. In the interim, BEX leadership, assessors, contracted industrial and organizational psychologists (IOPs), and other testing experts evaluated each step in the progression toward a virtual platform.

Experience with the Foreign Service Specialist Assessment (FSSA)—which has been virtual since 2020, an innovation that was prompted by the COVID-19 pandemic—was important. The successful transition of the FSSA from an in-person assessment to a virtual platform demonstrated that interruptions to the process were rare and the virtual platform worked reliably.

Correspondingly, specialist candidates voiced approval of the change in anonymous surveys, and their scoring patterns remained comparable to those in in-person settings. The feedback from the nearly 3,000 specialist candidates BEX assessed virtually has been overwhelmingly positive, including on the perceptions of fairness in the virtual process and the clarity of its presentation.

Building on the success of conducting thousands of FSSAs in a virtual environment, BEX endeavored to provide a similar experience for FSOA candidates. The FSOA comprises three exercises. Candidates complete the first part of the test, the case management exercise, separately, several weeks in advance of the second and third parts. The group exercise and the structured interview are completed later, both on the same day. Details for each exercise are available at careers.state.gov/downloads/FSOA.

Expanding access to the Foreign Service career was the primary driver in moving the FSOA online.

BEX takes a data-centered approach to its regular review of assessment materials, procedures, and candidate accessibility. Preparing for the FSOA’s move to a virtual platform gave BEX further opportunity to review materials and procedures to determine how they would mesh with a virtual platform and still fulfill their rigorous standards and best practices.

To achieve this modernization, BEX sought the vital feedback of current and retired Foreign Service Officer (FSO) and Specialist (FSS) assessors to review the “dimensions” (criteria) used to rate candidate performance. Since 2001, BEX had utilized 13 dimensions to evaluate the performance of FSO candidates and 12 dimensions to evaluate the performance of FSS candidates.

New, Common Dimensions

In October 2023, assessors began evaluating both FSOA and FSSA candidates using 11 dimensions, the same for both generalists and specialists. The current Foreign Service dimensions are critical thinking, cultural adaptability, experience and motivation, information integration and analysis, leadership, negotiation, objectivity and integrity, planning and organizing, presentation skills, teamwork, and written communication.

BEX modernized the dimensions to measure candidate performance on the three exercises based on comprehensive job analyses. The group exercise (in which candidates discuss the planning of various projects) is largely a measure of social/interaction skills, and candidate effectiveness is assessed based on the teamwork, leadership, and negotiation dimensions. The case management exercise (where candidates read a hypothetical case study, integrate information, and write their recommendations) is measured with dimensions such as critical thinking, information integration and analysis, and written communication.

The structured interview measures nearly all the dimensions. Specialists are evaluated similarly on the same dimensions in their exercises. The revised dimensions and assessment materials improve the measurement of critical skills required for FS personnel to succeed at the start of their career.

These changes are part of the Secretary’s modernization agenda.

Testing and Revision

BEX conducted six comprehensive FSOA pilot tests and several partial pilot tests before May 2024 to ensure that a virtual platform would allow assessors to accurately evaluate candidate performance related to the FS dimensions. BEX used the assistance of contractors who had specific experience transitioning other federal agencies—the Foreign Commercial Service and the Department of Agriculture’s APHIS—moving their employee selection assessments to virtual platforms. They provided insight and training in line with best practices and standards for employee selection.

Data from the pilots indicated that candidates could demonstrate all dimensions on virtual platforms and assessors could assess candidate performance accurately. BEX also provided additional assessor and program assistant training to ensure their ability to administer and score the exercises accurately and without any potential for bias.

Building on the success of conducting thousands of FSSAs in a virtual environment, BEX endeavored to provide a similar experience for FSOA candidates.

Together with its contracted IOPs, BEX regularly validates and revises all FSOA and FSSA assessment materials and procedures. BEX uses a content-oriented approach and a series of comprehensive job analyses to document the need for each FS dimension. BEX will continue this standard of excellence in evaluating the efficacy of the virtual assessments, making revisions based on data and statistical analysis.

To that end, BEX now has a full-time data analyst to track assessment material performance and ensure that procedures remain effective and limit adverse impact. (According to the Society of Human Resource Management, adverse impact refers to employment practices, including employee selection procedures, that appear neutral but have a discriminatory effect on a protected group.)

Acknowledging Concerns

An innovation of this magnitude did not come without its share of reasonable doubts and concerns. BEX focused on questions regarding internet bandwidth, test question security, and accessibility for candidates with disabilities (as with in-person FSOAs, candidates may request reasonable accommodations). Of particular interest across the foreign affairs community was: How will you assess group dynamics if candidates no longer gather in-person?

BEX acknowledges that candidate internet bandwidth and connection speeds will vary. To mitigate this, pre-FSOA testing protocols include candidates testing bandwidth on their devices, with BEX staff confirming connections before exercises begin. While BEX has committed to rescheduling candidates who lose connection during an exercise, we note that in two years of conducting the FSSA, bad connections and rescheduled assessments have occurred less than 1 percent of the time.

BEX feels confident in test security thanks to robust coordination with testing professionals and IOPs. All candidates sign a nondisclosure agreement, and BEX staff remind candidates that failure to keep assessment materials and procedures confidential risks canceling their candidacy. Further, BEX rotates test materials from a catalogue of options, and if any materials are leaked, those materials are immediately changed out.

Regarding group dynamics, the same rigorous standards and procedures will apply in the virtual context as used for the in-person process. Candidates will continue to be assessed according to the dimensions matching current Foreign Service job skills and that were developed to reflect the eventual implementation of a virtual process.

Long known as the gold standard in employee selection, the BEX selection process remains an exemplar for both public- and private-sector organizations seeking the best candidates. With the move to a virtual FSOA, our nation’s future diplomats can participate in one of the most effective and rigorous employee selection processes without incurring additional cost or hardship and increasing accessibility to all candidates.

Patrick T. Slowinski, Ed.D., Ph.D., is a recently retired career member of the Senior Foreign Service. His last assignment at the State Department was as deputy director for policy and planning at the Board of Examiners (GTM/TAC/BEX), where he led BEX’s modernization and innovation efforts. He holds doctoral degrees in education administration and advisement as well as industrial and organizational psychology, and is an adjunct faculty member of psychology at Liberty University.

Read More...

• “Who is the ‘Total Candidate’? FSO Hiring Today” by Shawn Dorman, The Foreign Service Journal, June 2008
• “Needs of the Service and Those of Our People: The New HR” by Philip W. Kaplan, The Foreign Service Journal, April 2020
• “FSO Selection: Changing the Path to the Oral Assessment” by Deidi Delahanty, The Foreign Service Journal, October 2022