48 JULY AUGUST 2024 | THE FOREIGN SERVICE JOURNAL The structured interview measures nearly all the dimensions. Specialists are evaluated similarly on the same dimensions in their exercises. The revised dimensions and assessment materials improve the measurement of critical skills required for FS personnel to succeed at the start of their career. These changes are part of the Secretary’s modernization agenda. Testing and Revision BEX conducted six comprehensive FSOA pilot tests and several partial pilot tests before May 2024 to ensure that a virtual platform would allow assessors to accurately evaluate candidate performance related to the FS dimensions. BEX used the assistance of contractors who had specific experience transitioning other federal agencies—the Foreign Commercial Service and the Department of Agriculture’s APHIS—moving their employee selection assessments to virtual platforms. They provided insight and training in line with best practices and standards for employee selection. Data from the pilots indicated that candidates could demonstrate all dimensions on virtual platforms and assessors could assess candidate performance accurately. BEX also provided additional assessor and program assistant training to ensure their ability to administer and score the exercises accurately and without any potential for bias. Together with its contracted IOPs, BEX regularly validates and revises all FSOA and FSSA assessment materials and procedures. BEX uses a content-oriented approach and a series of comprehensive job analyses to document the need for each FS dimension. BEX will continue this standard of excellence in evaluating the efficacy of the virtual assessments, making revisions based on data and statistical analysis. To that end, BEX now has a full-time data analyst to track assessment material performance and ensure that procedures remain effective and limit adverse impact. (According to the Society of Human Resource Management, adverse impact refers to employment practices, including employee selection procedures, that appear neutral but have a discriminatory effect on a protected group.) Acknowledging Concerns An innovation of this magnitude did not come without its share of reasonable doubts and concerns. BEX focused on questions regarding internet bandwidth, test question security, and accessibility for candidates with disabilities (as with in-person FSOAs, candidates may request reasonable accommodations). Of particular interest across the foreign affairs community was: How will you assess group dynamics if candidates no longer gather in-person? BEX acknowledges that candidate internet bandwidth and connection speeds will vary. To mitigate this, pre-FSOA testing protocols include candidates testing bandwidth on their devices, with BEX staff confirming connections before exercises begin. While BEX has committed to rescheduling candidates who lose connection during an exercise, we note that in two years of conducting the FSSA, bad connections and rescheduled assessments have occurred less than 1 percent of the time. BEX feels confident in test security thanks to robust coordination with testing professionals and IOPs. All candidates sign a nondisclosure agreement, and BEX staff remind candidates that failure to keep assessment materials and procedures confidential risks canceling their candidacy. Further, BEX rotates test materials from a catalogue of options, and if any materials are leaked, those materials are immediately changed out. Regarding group dynamics, the same rigorous standards and procedures will apply in the virtual context as used for the in-person process. Candidates will continue to be assessed according to the dimensions matching current Foreign Service job skills and that were developed to reflect the eventual implementation of a virtual process. Long known as the gold standard in employee selection, the BEX selection process remains an exemplar for both public- and private-sector organizations seeking the best candidates. With the move to a virtual FSOA, our nation’s future diplomats can participate in one of the most effective and rigorous employee selection processes without incurring additional cost or hardship and increasing accessibility to all candidates. n Building on the success of conducting thousands of FSSAs in a virtual environment, BEX endeavored to provide a similar experience for FSOA candidates.
RkJQdWJsaXNoZXIy ODIyMDU=