The Foreign Service Journal, October 2022

44 OCTOBER 2022 | THE FOREIGN SERVICE JOURNAL regarding retention, tenure, assignments, promotion at all levels, and language incentive pay. Given its outsized impact on employees and their families, the Foreign Service Institute is cognizant of the impor- tance of language testing to Foreign Service personnel—including assignment and transfer. That is why it is imperative that FSI maintain a language testing program that is fair, transparent, and relevant to the vital mission of America’s diplomats. Changes Overdue Changes to language testing were over- due. While FSI’s language test was ground- breaking when introduced in the 1950s and has been incrementally updated since then, FSI leadership acknowledged that the test had not kept pace with current developments in language assessment and that significant innovations in the field went well beyond any previ- ous revisions. Separately, questions and concerns had arisen about limitations or potential biases associated with the testing program, particularly against individuals who identify as native or heritage speakers—terms for which there is no single, set definition—and perceptions that the department was not fully valuing their contributions to the workforce. Other concerns were raised about potential bias in testing based on accents, language variety, or educational or demographic background. (Please go to the LTU intranet site —http://fsi.state.gov/ SLS/6951— for detailed information.) Finally, the nature of diplomacy and the work of Foreign Ser- vice professionals have changed significantly over the decades through the use of email, text, social media, and the internet, resulting in shifts in the language skills required in overseas postings. To address these concerns, FSI in 2018 commissioned a major study by the National Academies of Sciences, Engineer- ing, and Medicine (NAS). Following receipt of the report, FSI conducted outreach to stakeholders throughout the State Department and convened a 20-person Task Force on the Future of Language Testing. Based on its review of the report, the task force proposed changes to language testing structure, content, administration, and scoring. The task force also discussed concerns about bias and made recommendations to address them, while FSI undertook a parallel yearlong effort to exam- ine training and testing of heritage speakers to ensure that they are tested fairly and their skills are recognized, including addressing concerns about unconscious bias. The task force produced a new Statement of Test Purpose, Use, and Impact, to serve as a guide for the test redesign process. The Changes, in Detail In July 2022, FSI’s Language Testing Unit (LTU) began implementing the changes rec- ommended by the task force and will continue with substantial revisions through 2023. For test takers, changes are both to scoring and the for- mat and content of the test. Changes to scoring include updates to the assessment criteria and greater transparency through score reports. Scoring. Tests will be scored with a new, updated version of the 0-5 Interagency Language Roundtable (ILR), often referred to as the ILR Skill Level Descriptions (SLDs). The new SLDs no longer refer to the “native speaker,” as definitions vary, and native speakers show a range of language proficiency across ILR skill levels. FSI now caps scores at 4, with 4, 4+, and 5 col- lapsed into one category, because the department’s highest requirement is also ILR 4 and the distinctions between these levels are challenging to identify in testing. These scores now appear as “AP” for Advanced Proficiency in the language skills inventory and the employee’s profile. The LTU is developing Foreign Service–specific rubrics and scoring procedures that will be implemented in early 2023 to tailor the assessment more closely to Foreign Service work and capture more information about the test. These new scoring criteria give greater weight to listening comprehension on the speaking test and reweight FSI’s goal is to ensure that language testing reflects the real-world requirements of Foreign Service work.