The Foreign Service Journal, September 2005

Testing: How to Get It Right? “The Case for Outside Examin- ers,” in the same issue, tackles the issue that is probably most painful for students. The writer echoes long- heard complaints that FSI is not rig- orous in maintaining objectivity, valid- ity and reliability in its testing system and suggests that the perceived ran- domness in assigning proficiency scores could be solved by bringing in outside examiners. This is neither practicable nor necessary. First, an assurance. FSI gets no “brownie points” from HR or any- where else for holding down the scores or boosting them up. We have nothing to gain from any deviation from fair and objective testing. Each year our School of Language Studies administers some 4,000 tests in over 90 languages. Unlike academic achievement tests, which measure students’ mastery of material that has been taught, our proficiency test mea- sures the ability to use a language to accomplish real-world tasks and trans- actions. FSI has a highly-structured, lengthy training program for both “testers” (the native speakers of the language who interact with the examinee) and “examiners” (experienced language professionals who direct and adminis- ter the tests). Testing teams aim to obtain as broad a language sample as possible in the necessarily limited amount of time. They also seek out examinees’ upper limit of proficiency to ensure a fair rating. This is one instance where “no pain, no gain” works to examinees’ advantage, because often they can produce sur- prisingly good results when pressed. It is true that in a testing situation some individuals perform better than others, and the same person may function quite well in one test and not so well in another. Because they are based on the sample elicited in a given test, test scores reflect these dif- ferences. And, in fact, they must do so, since injecting other factors, like classroom performance, into the final score would add what we all would regard as a random and unwelcome element of subjectivity. In all but the smallest language sections, FSI’s students are not tested by anyone who was their teacher dur- ing their last eight weeks. This pre- vents “rehearsed speech” or “chem- istry” (positive or negative) between the examinee and the testing team that could potentially affect the test results. The score is based entirely on 20 F O R E I G N S E R V I C E J O U R N A L / S E P T E M B E R 2 0 0 5 S P E A K I N G O U T u 2000 N. 14th Street • Suite 500 Arlington, VA 22201 Telephone (703) 797-3259 Fax (703) 524-7559 Tollfree (800) 424-9500