Efficient and secure information technology processes and platforms are the primary requirements for State’s operational modernization. Here is a candid look at the challenges and suggestions for a way forward.

BY JAY ANANIA

First, the good news. For all the justifiable complaints employees and customers may have about the Department of State’s electronic systems, State manages generally reliable and secure global systems connecting several hundred U.S. locations and overseas posts, many in countries with poor telecommunications infrastructure. Very few organizations of any type confront the complexities faced by the department. That accomplishment is worth appreciating, especially given some of the inherent and unusual challenges briefly mentioned in this article.

And yet the performance of State’s information technology systems remains a sore spot for many employees from all serviced agencies. Customers accustomed to rapid developments in e-commerce and mobile computing chafe at using systems that often don’t share data or simplify routine processing. At many overseas posts, personnel are frustrated by poor performance as applications become more centralized, outrunning the quality of the connections to servers in the United States. Worse, malevolent intruders constantly threaten State IT systems as they seek (and at times, obtain) sensitive information and opportunities to derail U.S. initiatives. Even a cursory glance at the Office of the Inspector General’s online archive reveals persistent problems with State’s IT planning and execution, including issues that affect system performance and the integrity, confidentiality and access to data.

At the heart of the difficulties is the fact that State, like many other federal agencies, lacks a centralized authority that is empowered to establish and enforce an enterprise-wide IT architecture for domestic offices and overseas missions. Such a centralized authority is needed to set standards for efficiency and data sharing, to guide specific IT initiatives, to prioritize spending and to direct cybersecurity operations among the myriad IT systems “owned” by individual bureaus, departments and posts.

There are no easy solutions to the strategic failure of State’s IT systems. In this article, I discuss the various challenges, how they arose and their implications for efficiency and security. Finally, I offer some recommendations that—if backed by sustained, high-level management commitment—can set the department on an effective path for the future. Revamping the organization to reshape and manage State’s information technology underpinnings is critical to supporting overdue, broader management reforms necessitated by dramatic changes in the nature of foreign affairs challenges.

Past as Prologue

The current situation reflects the history of IT at the State Department and the consequences of decisions, and non-decisions, taken over the past three decades. Broad-scale computing at State started in the early 1980s with the introduction of Wang mainframe and word processing systems, often at the initiative of “early adopters” who saw the utility of computers over typewriters. There was little centralized capability or organization to manage these systems, and individual offices and bureaus purchased them and used them as they saw fit. Bureaus paid for these computers, typically made the decisions about what and when to buy, and expected employees to share computer terminals.

State management gradually recognized the utility of having a unified information technology organization. In 1998, it created the Bureau of Information Resource Management from some— but not all!—elements of the Bureau of Administration’s Office of Information Management. From the start, IRM was playing catch-up: State IT was already decentralized, and bureaus continued to fill the vacuum by creating solutions to meet their needs. “Functional” bureaus (including the Bureau of Administration) set about building core IT systems for accounting, human resources, logistics, etc. This decentralization had the advantage of putting bureaus in charge of systems that met their specific needs; but it came at the cost of duplicating efforts, and creating and institutionalizing inefficiencies.

With no coherent centralized initiative to unite systems “owned” by various bureaus and posts, it was the Bureau of Consular Affairs—with its obvious requirement for consistent consular systems globally, and utilizing funds retained from consular collections—that finally set a consistent standard for desktop computers, albeit only for consular personnel. Finally, with the (mostly imaginary) Y2K threat looming, IRM was funded to establish consistent global standards for Microsoft-based desktop computers.

As federal IT evolved, laws came into force creating chief information officers (CIOs) at each agency and defining their roles. But, like other federal agencies, the State Department was slow to adjust its policies and bureaucracy and is still far from complying with current law and standards. State first created the CIO as a solely advisory position in the Office of the Under Secretary for Management, only later making the CIO the head of IRM, a bureau still focused on managing core communications systems. Even today, State’s CIO is often viewed as akin to the head plumber or electrician rather than a critical business leader. By contrast, law and executive orders direct agencies to empower CIOs with broad authority over IT investments and cybersecurity.

Given the functional bureaus’ responsibilities, it is reasonable that they should serve as “business owners” and play a major role in managing IT systems. However, a weak CIO and the lack of effective enterprise-wide strategies led, perhaps inevitably, to a reality in which individual bureaus zealously guard their traditional prerogatives and funding. With IRM’s history of budget and human resources limitations, it is no surprise that other bureaus continue to directly create and manage core IT systems to carry out State’s critical HR, financial, consular, logistics, security and other functions.

A Tower of Babel

In practice, then, absent direction and assistance in aligning investments for the greater good of employees and organizational efficiency, bureaus can and do develop systems that respond to narrow requirements. Many vendors offer similar IT applications and platforms. Without the strategic guidance they often yearn for, bureaus inevitably end up picking differing, sometimes incompatible, tools to the detriment of overall efficiency and cost-effectiveness.

This is especially damaging to operations at overseas posts, which typically do a better job than headquarters of integrating internal and interagency operations. Unfortunately, the “Washington solutions for Washington problems” approach, in which bureaus focus on their own narrow requirements, leads to IT systems that actually hamper these needed collaborative efforts.

State lacks a centralized authority empowered to establish and enforce an enterprise-wide IT architecture for domestic offices and overseas missions.

Indeed, bureaus and posts have created thousands of systems— some from scratch and others using commercial software modified to meet “unique” requirements (perceived or legitimate). The result is a technical Tower of Babel that the department’s limited technical workforce cannot properly understand, manage or even catalog.

Contractor personnel created and still manage most major systems. State never invested in or retained the technical staff needed to properly document and apply the knowledge associated with both the administrative policies and technical details of these systems. The recent hiring freeze exacerbated the problem, as bureaus watched critical personnel—both technical and subject matter experts—retire, transfer or take lucrative private-sector positions. This lack of staffing continuity makes it impossible for State to effectively manage its IT systems.

Further, bureaus struggling with staff and budget shortages are understandably hesitant to make major changes given the risks of “breaking” increasingly obsolete systems or, worse, opening up new cyber vulnerabilities and exposing themselves to public criticism from Congress and the media. Some bureaus have literally dozens of interrelated applications, built using different tools over many years.

This is a root cause of the frustrations employees feel when trying to accomplish seemingly routine activities, especially when the tasks cut across bureaucratic lines. An obvious example is the difficulty of the Foreign Service transfer process, which requires human resources, finance and logistics personnel and systems to work together. Instead, many IT systems reinforce bureaucratic lines and impede productivity.

A Positive IRM Initiative, But…

While IRM’s consolidation of productivity tools on the Microsoft Office 365 platform should prove a highly positive initiative, individual bureaus continue to create systems based on other commercial products from competing companies. In many cases, customizations over time make it difficult to upgrade these systems or migrate and/or share the data in a manner compatible with department-wide objectives. Bureaus have the money to maintain current systems, but neither the resources nor the direction to develop and implement strategies to modernize them to make customers’ jobs easier.

The good news is that modern IT application development tools permit greater flexibility to configure systems without customizing underlying applications. This permits managers to plan for the inevitable “like it or not” technical upgrades required by commercial vendors without upsetting integration with other applications.

The bad news is that by permitting bureaus to choose from a smorgasbord of competing products, with little encouragement or incentive to consolidate systems, the department continues to increase IT complexity. Bureaus develop systems using different tools, in some cases migrating from State-managed data centers to competing commercial cloud-based platforms, such as those managed by Amazon and Microsoft. Worse, some bureaus operate software so altered to meet State’s needs that it cannot be updated away from obsolete technology, exposing data to cyber-intrusion and the more mundane risks associated with software no employee understands how to manage.

Attempting to track, control and set realistic configuration standards for so many systems (including data-center platforms) and manage the interconnections between them is a Sisyphean task, as critical OIG reports document. For example: so-called customer relationship management (CRM) software underlies many modern applications. There are several excellent options, including Remedy, ServiceNow, SalesForce and Dynamics. But lacking an enterprise IT architecture, the department is choosing all of them, “hosting” some applications on department-managed infrastructure while outsourcing others to competing commercial “cloud” data centers.

This exponentially multiplies complexity and perpetuates the past mistake of institutionalizing “fragmented decentralization.”

The Data Management Spider Web

Some State Department IT leaders now tout a data-centric approach to systems. This is long overdue, because information— data—is the asset at the heart of the department’s programmatic and administrative missions. Data—not IT systems—should be the starting point. System “owners” must understand and demonstrate a commitment to properly integrating data to ensure efficient, State-wide operations. Maintaining overlapping data sources with disparate underlying systems is wasteful, hurts data quality and increases the risk of data loss.

Yet, with individual bureaus “owning” these systems, there has never been an effective scheme to share data among them. Bureaus and posts, both internally and among themselves, implement “point-to-point” connections to share (or worse, not to share) data, creating hundreds of unique connections using multiple tools—a spider web of uncontrollable complexity. The department can neither track nor manage these data flows, which increases costs (it takes highly paid people to manage these systems), decreases efficiency (customers must work through multiple systems) and exposes data to the possibility of loss or misuse. This makes the job of the IRM chief information security officer nearly impossible.

The department already has commercial middleware (known as an enterprise service bus) software to coordinate centralized data sharing, but isn’t using it widely.

Amazingly, the department already has commercial middleware (known as an enterprise service bus) software to coordinate centralized data sharing, but isn’t using it widely. In this software’s hub-and-spoke configuration, each application connects to a central system. With data “on the bus,” IRM could work with system owners to appropriately share it, securely and efficiently, eliminating hundreds of current connections.

But without an effective, empowered CIO directing implementation, this isn’t happening. Nor is there a clear plan to make it happen. This is, quite simply, a strategic failure.

Moving Toward a Solution

Based on the observations and discussion above, there are a number of seemingly obvious steps that could be taken to overhaul, strengthen and rationalize State’s IT architecture.

1. Define goals. Codify department-wide “first principles” to affirm that the purpose of IT systems is to enhance the efficiency of secure global operations. Systems are tools to reduce employee and customer effort. Data is a corporate asset that all IT systems owners must share appropriately, eliminating duplicative data stores whenever possible. IT systems must enhance secure global access to data and meet defined performance levels.

2. Empower the CIO, consistent with federal law and executive orders, to establish and enforce an enterprise-wide IT architecture, prioritize spending and direct cybersecurity operations. Make the CIO responsible for advancing enterprise goals.

3. Direct the CIO, in consultation with bureaus, to set standards and limit the number of options for software applications, development tools and IT platforms. Set short-term and longer-term goals to converge systems, especially critical core systems.

Strictly limit cloud platforms and duplicative tools, permitting exceptions only where standard assets cannot be used. Expand department-wide licensing agreements, replacing bureau-by-bureau purchasing, to reduce overhead, improve internal controls and increase incentives for standardization.

4. Invest in hiring, retaining and continuously refreshing the skills of State’s IT employee workforce. While building a strong cadre of Civil Service staff who can manage core systems, integrate Foreign Service personnel with expertise managing overseas systems throughout IRM and other bureaus that manage major IT systems. As a priority, strengthen IRM’s core capacities to properly manage IT strategy and core IRMmanaged systems while providing guidance and oversight to bureaus managing other critical systems.

Provide career paths to ensure the department can count on an experienced team with strong technical and managerial skills and, critically, a broad understanding of enterprise business requirements and interrelationships between bureau missions. Especially for core financial, human resources, consular and logistics systems, ensure that managers are subject matter experts, with substantive experience, who can expertly inform system development carried out by qualified IT staff.

5. Replace fragmentation with federation. Maintain a degree of decentralization, with bureaus continuing to play lead roles in defining business requirements for IT systems; but empower the CIO to enforce strategy and funding flows, insisting that bureaus receive consistent, achievable direction and resources based on agreed priorities.

Over time, transfer technical resources to IRM to permit the bureau to properly provide technical support, reducing IT elements within other bureaus. Assistant secretaries, their deputies and executive directors are not and never will be IT experts. Just as they should manage bilateral relations, consular affairs, security, intelligence and financial operations, the CIO and IRM should manage the underlying technical aspects of IT.

Information technology processes and platforms are the primary requirement for State’s operational modernization. Unfortunately, there is no magic bullet to resolve the IT challenges State and other agencies face, which go back many years. But with a serious, sustained commitment from top-level management, a decisive start can be made toward significant improvement in this critical area.

Jay Anania retired in 2015 after a 31-year career as a management-coned FSO at the Department of State. His final tour was as U.S. ambassador to the Republic of Suriname from 2013 to 2015. Among his earlier assignments, Ambassador Anania served under Secretary of State Colin Powell as director of the Offices of Management Policy and Rightsizing the USG’s Overseas Presence, and as the Bureau of Information Resource Management’s acting chief information officer. He was executive director for the Bureau of Near Eastern Affairs and the Bureau of South and Central Asian Affairs, and then served as management counselor for U.S. Mission Iraq (2011-2012). In Iraq, Amb. Anania led the successful management transition as State took responsibility from the U.S. Army for supporting 17,000 personnel at 12 locations throughout the country. He also served in Tijuana, Havana, Amman, Abu Dhabi, Hong Kong and Berlin. The opinions and characterizations in this piece are those of the author and do not necessarily represent those of the U.S. government. The author can be reached at Parbonia-1@yahoo.com.

Read More...

• “Chugging Up the Onramp of the Info Interstate”
• “Anywhere, Anytime Diplomacy”
• “IT: Spending is Not Enough”