The Foreign Service Journal, April 2005

All six metrics used in the area of “rul- ing justly,” for instance, are either untested (the four from the World Bank Institute) or subjective (the two fromFreedomHouse). The Freedom House measures are based on Lichert scales, which are highly subjective. They fail to measure exactly what they are supposed to, to describe it, and to qualify it with any degree of objectivi- ty or accuracy: they lack internal valid- ity. Moreover, they are non-paramet- ric. As such, they lack external validi- ty and cannot be employed for either cross-country or cross-temporal com- parison purposes, using summary sta- tistical analysis and inference. For example, if Country A had scores of 3, 4 and 5 for the previous three years, using the FreedomHouse “civil liberties” measure, and Country B had scores of 1, 2 and 3 for the same three years consecutively, could one reasonably and objectively say that Country A was “ruled more justly” by dint of its higher scores? What, exact- ly, does it mean if a country is graded “3” rather than “4”? These numbers are symbolic at best, summarizing highly subjective interpretations of phenomena in a country and assigning them with an otherwise unrelated numeric value. For purposes of com- parison, however, they are completely useless, because there is no objective- ly measurable difference directly related to a difference in scores. As far as the World Bank Institute indicators are concerned, Kaufman and Kray, the creators of the “linch- pin” indicator, “control of corrup- tion,” have stated in recent publica- tions that this indicator still requires further refinement and testing. It lacks sufficient empirical application and would be inappropriate for deter- mining such a major policy decision as whether or not a country has met the basic requirement for eligibility in the MCA, they say. Yet this one indicator will determine a country’s initial and continued eligibility, overriding what might be good performance in any number of other indicators. Another problem lies in our own backyard. The U.S. Congress has his- torically prided itself on its vigilant oversight of USAID and other organi- zations spending taxpayer monies overseas. It is highly unlikely that the relevant committees will be willing to allow such major amounts of money (up to $3 billion per year by FY 2006) to be disbursed without regular, detailed progress reports being fun- neled back to them. Even if they were statistically valid and reliable, the 16 indicators and whatever additional measures may be stipulated in each country’s compact are not likely to be sufficient to satisfy the majority of lawmakers sitting on the various oversight committees. Members of Congress are naturally concerned that MCC monies might simply become a slush fund for the president to reward strategic allies who demonstrate their value through 42 F O R E I G N S E R V I C E J O U R N A L / A P R I L 2 0 0 5