Note: Jeremy Weinstein & Joshua Goldstein's response to this Article can be viewed here: The Benefits of a Big Tent: Opening Up Government in Developing Countries

Introduction

The internet’s power to make government information more available and useful has, in the last several years, become a topic of keen interest for citizens, scholars, and policymakers alike.  In the United States, volunteers and activists have harnessed information that the government puts online in key domains, ranging from the federal legislative branch to local city services, and have created dynamic new tools and interfaces that make the information dramatically more useful to citizens.  These new tools have sparked significant academic and popular interest and have begun to prompt a fundamental shift in thinking: Policymakers have begun to consider not only the citizens who may ultimately benefit from government information, but also the third parties who can play a valuable mediating role in getting the information to citizens.

The primary concrete result of this trend is that governments have made a growing range of public sector data available in machine-processable electronic formats that are easier for others to reuse.  Information that enhances civic accountability, including pending congressional legislation and federal regula­tions, is indeed more readily available.  But more mundane and practical gov­ernment information, from bus schedules to restaurant health inspection data, is also being provided in friendlier formats.  Such data can be used to improve qual­ity of life and enhance public service delivery, but may have little impact on political accountability.

Recent policy initiatives that promote or reinforce this trend have been described as “open government” projects.  These initiatives usually include the provision of reusable data as one among a range of steps designed to increase overall governmental transparency.  For example, President Obama’s Open Government Directive, which was designed to implement the new administration’s overall “principles of transparency, participation and collaboration,”¹ instructed execu­tive branch agencies, inter alia, to “publish information online in an open format . . . . An open format is one that is platform independent, machine reada­ble, and made available to the public without restrictions that would impede the re-use of that information.”²  Similarly, the multilateral Open Government Declaration,³ signed by the United States and seven other countries in September 2011,⁴ situates these new technologies of data sharing in the context of political accountability.⁵  It begins with an acknowledgement that “people all around the world are demanding more openness in government.”⁶  Among their promises, the signatories commit to “provide high-value information, including raw data, in a timely manner, in formats that the public can easily locate, understand and use, and in formats that facilitate reuse.”⁷

These new “open government” policies have blurred the distinction between the technologies of open data and the politics of open government.  Open government and open data can each exist without the other: A government can be an open government, in the sense of being transparent, even if it does not embrace new technology (the key question is whether stakeholders know what they need to know to keep the system honest).⁸  And a government can provide open data on politically neutral topics even as it remains deeply opaque and unaccountable.  The Hungarian cities of Budapest and Szeged, for example, both provide online, machine-readable transit schedules,⁹ allowing Google Maps to route users on local trips.  Such data is both open and gov­ernmental, but has no bearing on the Hungarian government’s troubling lack of accounta­bility.  The data may be opening up, but the country itself is “sliding into authoritarianism.”¹⁰

The popular term “open government data” is, therefore, deeply ambiguous—it might mean either of two very different things.  If “open government” is a phrase that modifies the noun “data,” we are talking about politically important disclosures, whether or not they are delivered by computer.  On the other hand, if the words “open” and “government” are separate adjectives modifying “data,” we are talking about data that is both easily accessed and government related, but that might or might not be politically important.  (Or the term might have a third meaning, as a shorthand reference to the intersection of data meeting both definitions: governmental data that is both politically sensitive and computer provided.)

In this Essay, we acknowledge that this ambiguity may sometimes be bene­fi­cial, but ultimately argue that the term “open government” has become too vague to be a useful label in most policy conversations.  Open data can be a powerful force for public accountability—it can make existing information easier to analyze, process, and combine than ever before, allowing a new level of public scrutiny.  At the same time, open data technologies can also enhance service delivery in any regime, even an opaque one.  When policymakers and the public use the same term for both of these important benefits, governments may be able to take credit for increased public accountability simply by delivering open data technology.

In place of this confusion, we offer a stylized framework to consider each of these two questions independently.  One dimension describes technology: How is the disclosed data structured, organized, and published?  We describe the data itself as being on a spectrum between adaptable and inert, depending on how easy or hard it is for new actors to make innovative uses of the data.  The other dimension describes the actual or anticipated benefits of the data disclosure; the goals of disclosure run on a spectrum between service delivery and public accounta­bility.  This is admittedly a simplification of reality: In practice, many disclosures serve both objectives.  However, it is common for one of the two motives to predominate over the other, and we believe this provides a useful starting point for thinking about the competing goals of disclosure.

On the diagram, the vertical axis describes the data itself, and the hori­zontal axis describes the extent to which service delivery or public accountability predominates as a goal or anticipated result of the disclosure.  Along the vertical dimension, there is broad political consensus in favor of adaptable data; but, horizontally, there are differences of opinion about the relative political importance of service delivery and public accountability as end goals for public disclosure.  (Our discussion in Part III, below, illustrates these dimensions by populating the graph with examples of concrete public policies.)

We have organized our discussion as follows: Part I.A explains the concep­tual origins of the relatively modern idea of open government as a public policy, starting with the first recognized use of the term in the mid–twentieth century.  The phrase is of fairly recent vintage, but it reflects a particular perspective on the issues it describes—and it was well established before the internet came into being.  Part I.B, correspondingly, explores the conceptual roots of open data, an idea that has always included, but has always applied far beyond, the kinds of information associated with civic transparency.  Part II follows the story forward in time, as these concepts begin to merge and give rise to the ambiguous idea of open government data, and details some of the confusions that have ensued in the wake of this ambiguity.  Part III describes our alternative proposal, which differentiates the widely shared goal of adaptable data from the more contro­versial choice between enhanced service delivery and enhanced public accounta­bility as the end goals of disclosure.

I.          Conceptual Origins

Open government and open data each have rich conceptual histories with independent origins.  These histories are indispensable tools for understanding the current debate.

A.        Conceptual Origins of Open Government

The idea of open government, as a synonym for public accountability, is part of the peacetime dividend that America reaped after the Second World War.  After the war ended, the federal government was left in a state of relative opacity.  Having grown accustomed to wartime information restrictions, the federal workforce was “fearful of Cold War spies, intimidated by zealous loyalty investigators within and outside of government, and anxious about” workforce reductions following the war.¹¹  As a result, “the federal bureaucracy generally was not eager to have its activities and operations disclosed to the public, the press, or other governmental entities.”¹²

The opacity surrounding World War II was not, as wartime opacity might be today, a deviation from a clearly established statutory requirement of federal government transparency.  Instead, prior to World War II, the key federal law controlling disclosure of government information was the archaic Housekeeping Statute of 1789,¹³ which gave “[g]overnment officials general authority to operate their agencies” and withhold records from the public.¹⁴  The Administrative Procedure Act of 1946,¹⁵ while it did contain a general requirement of access to public records, empowered agencies to restrict access “in the public interest,” with or without “good cause found”—a faint precursor of the robust justificatory requirements and procedural assurances of modern administrative law.¹⁶

The period from 1945 to 1955 was a “crucial decade” of early pressure toward greater openness, driven in part by the American Society of Newspaper Editors (ASNE).¹⁷  In 1953, ASNE commissioned a report, prepared by a prominent newspaper attorney named Harold Cross, titled The People’s Right to Know: Legal Access to Public Records and Proceedings.¹⁸  The report’s foreword noted that Cross had “written with full understanding of the public stake in open government”¹⁹—one of the earliest known uses of the term.  The report became “the Bible of the press and ultimately a roadmap for Congress regarding freedom of information,”²⁰ and it served as “a call to battle . . . aimed primarily at the needs of news editors and reporters.”²¹

In 1955, the U.S. Congress created the Special Subcommittee on Government Information, also known as the Moss Committee,²² which incu­bated the legislation that became the Freedom of Information Act a decade later.²³  Wallace Parks, who served as counsel to the subcommittee,²⁴ gets credit as the first to expound on the term “open government” in print, thanks to his posthu­mous 1957 article, The Open Government Principle: Applying the Right to Know Under the Constitution.²⁵  Parks does not explicitly define the term “open government” in the article (in fact, he uses the phrase just four times in twenty-two pages), but his usage makes clear that he sees open government as a matter of accountability:

Parks’s thinking, and perhaps his choice of words,²⁷ was part of a long campaign of legislative pressure that would culminate with the passage of the Freedom of Information Act (FOIA)²⁸ in 1966.  Although President Lyndon B. Johnson “hated the very idea of journalists rummaging in government closets, hated them challenging the authorized view of reality, [and] hated them knowing what he didn’t want them to know,”²⁹ he nonetheless signed the FOIA bill, professing “a deep sense of pride that the United States is an open society in which the people’s right to know is cherished and guarded.”³⁰

Over the next several decades, policy stakeholders used the term “open government” primarily as a synonym for public access to previously undisclosed gov­ernment information.  When Congress amended FOIA in 1974,³¹ it noted that “[o]pen government has been recognized as the best insurance that gov­ernment is being conducted in the public interest.”³²  Similarly, the Privacy Act of 1974 aimed to achieve the ideals of “accountability, responsibility, legislative oversight, and open government” together, while respecting citizen privacy in government-held information.³³  Congress also considered open-meeting laws—like the Government in the Sunshine Act,³⁴ which threw open the doors of federal agency meetings—to be under the umbrella of open government.³⁵  As the case law of FOIA and related statutes developed through the 1970s and 1980s, federal court decisions began to use the term “open government” as well, likewise referring to governmental transparency.³⁶

B.        Conceptual Origins of Open Data

The internet holds obvious promise as a tool for sharing more data, more widely, than has ever been possible before.  Across a wide range of technical fields, the adjective “open” has become a powerful, compact prefix that captures information technologies’ transformative potential to enhance the availability and usefulness of information.

Parallel explorations of the possibilities have been unfolding in a number of areas, accelerating in tandem with the growing uptake of the internet.  For example, the Open Access movement aims to make peer-reviewed scientific literature freely available online.³⁷  The Open Educational Resources campaign seeks to create digital repositories of free learning materials to support global access to knowledge.³⁸  Open technological standards create pools of patent rights, relieving individual innovators of the need to negotiate patent licenses.³⁹  The Creative Commons system of copyleft licenses, which makes it easier for creative artists to share and reuse each other’s work, aims toward “an Internet full of open content, where users are participants in innovative culture, education, and science.”⁴⁰

Similarly, a programmer’s decision to release her software under an “open source” license means that the program’s source code will be freely available to its users.⁴¹  The phrase “open source” has also, more broadly, become shorthand for the collaborative innovation strategy that underlies many open source software projects—an ethos in which anyone can contribute, abundant scrutiny can help to find and resolve bugs,⁴² and a community of creators can take pride in a useful, freely available end product.⁴³

Across each area, there is a common thread: When many individuals or groups are able to access information themselves and interact with it on their own terms (rather than in ways prescribed by others), significant benefits can accrue.  Each of these movements is focused on certain classes of information, and each one leverages new technology to make that information more freely and readily available and useful.

The label “open,” as applied to various kinds of information, thus inherits both a technological and a philosophical meaning.  At a technological level, the term suggests using computers to handle information efficiently in place of manual human processing, greatly extending the range of logistically feasible ways in which information can be used.  The extent to which this is possible often turns on technical details, as computers can more readily transform information that is provided in standard, structured formats.

Philosophically, the term suggests participation and engagement—all the people who might benefit from information can share and reuse it in a democra­tized, accessible way.  This implies an absence of legal barriers to innovative new projects, and a larger cultural enthusiasm for innovative and sometimes unex­pected developments.⁴⁴

The label “open data” combines both senses of the word “open”—both the term’s technological meaning and its philosophical meaning—with a focus on raw, unprocessed information that allows individuals to reach their own conclu­sions.  Before its civic uses, scientists used the term to refer to raw, unprocessed scientific data.

The earliest appearance of the term “open data” in a policy context appears to come from science policy in the 1970s: When international partners helped NASA operate the ground control stations for American satellites, the operative interna­tional agreements required those partners to adopt an “open-data policy comparable to that of NASA and other U.S. agencies participating in the program, particularly with respect to the public availability of data.”⁴⁵  The agreements also required that data be made available to NASA “in the NASA-preferred format.”⁴⁶

Later, a 1995 National Academy of Sciences report titled On the Full and Open Exchange of Scientific Data elaborated the idea of sharing data from environmental monitoring satellites, perhaps reflecting its shared lineage with those earlier NASA agreements: “International programs for global change research and environmental monitoring crucially depend on the principle of full and open exchange . . . . Experience has shown that increased access to scientific data, information, and related products has often led to significant scientific discoveries and the opportunity for educational enhancement.”⁴⁷

The term “open data” has also appeared in the life sciences context, princi­pally in relation to genetic data.  A feature on Jim Kent, the graduate student whose programming work allowed the publicly funded Human Genome Project to finish its work before competing private efforts did, said in part: “Kent’s work illustrates the need to think about more than just open source code; in the scien­tific community there is a growing awareness of the importance of open data.”⁴⁸

II.       “Open Government” Meets “Open Data”

A.        Early Roots of the Convergence

Government data started going online almost as soon as the internet opened to individual users in the early 1990s.  The earliest pioneer was Jim Warren, a sixties radical from Silicon Valley.  Warren was well known as the founder of the West Coast Computer Faire, one of the first venues to showcase the personal computer.⁴⁹  He was also known as an open government activist, but his partic­ular flavor of transparency had a high-tech twist.⁵⁰  In 1993, he “show[ed] California Assembly Member Debra Bowen how public access to state legis­lative records could be accomplished via the Internet at low cost and high benefit to the public.”⁵¹  Bowen introduced A.B. 1624⁵² in March 1993, and Warren “single-handedly launched a crusade to ensure the bill’s passage,” which suc­ceeded later that year.⁵³  California became “the first state in the nation to put its legislative information, voting records, and state laws online.”⁵⁴  Following California’s lead, open government advocates in at least a dozen other states began to push similar grassroots proposals.⁵⁵

At the federal level, when the Republicans gained control of Congress in 1994, they enjoyed a fresh opportunity to overhaul that body’s infrastructure—the first such opportunity since widespread public use of the internet began.  The website THOMAS, launched in 1995, provided public access to proposed legislation, directory information about members and committees, and daily hearing schedules, among other useful documents.⁵⁶  Although today discussions of open government in Congress often begin with THOMAS, the website was not clothed in the language of “open government” at its launch.⁵⁷  Before the convergence, “open government” referred narrowly to the initial release of previ­ously undisclosed government information or the effort to get such information released.  At its inception, THOMAS simply increased the accessibility of congressional work that was already publicly available.⁵⁸  While this increase in accessibility was dramatic, it arguably did not fall within the then-current meaning of the term “open government,” because it did not disclose any previ­ously unavailable material.

THOMAS was not what would now be called an open data project either, because the information it provided was accessible only via a government-supplied interface.  The website was designed to serve the needs of citizens—not to open the door for third parties to innovate.  By contrast, although they may not have used the term “open data,” several other key government offices have long pursued open data policies, providing key data online in machine-readable formats that (unlike THOMAS) facilitate third-party analysis and reuse.  The greatest example may be the U.S. Census, which was providing public information through Census.gov as early as 1996.⁵⁹The first major project to take advantage of open data for an open government purpose—that is, to make data machine readable and accessi­ble in order to promote government transparency and accountability—was OpenSecrets.org, a website that allows users to search and analyze campaign finance disclosures.⁶⁰  It launched in 1998 under the auspices of the Center for Responsive Politics, combining government data with third-party innovation.  From the beginning, the website aimed to let users adapt the data to their own purposes.  On the site’s early home page, its creators explained that they planned on “expanding the interactivity of the site, making it possible for you to ask your own questions—how much did the tobacco industry give in the last election, for example, or where does your congressman rank in dollars from labor unions, defense contractors, or phone companies.”⁶¹  True to that promise, the site quickly emerged as a powerful and popular tool for members of the public, researchers, and journalists—a role it still enjoys today.

GovTrack.us, a website launched in 2004 as a side project of then–graduate student Joshua Tauberer,⁶² was a landmark in the convergence of open government and open data.⁶³  It focused on the same core information as THOMAS: legislative data about Congress.  The website included bills, votes, biographical information on members, and reusable digital maps of congressional districts, and it offered new functionality beyond THOMAS’s own for people to search, sort, and monitor legislation of interest to them.

The data in THOMAS was not freely available in bulk at the time of GovTrack’s launch—instead, Tauberer had to painstakingly write computer code to systematically scrape and reassemble the data in THOMAS.  But once he had reassembled the data for his own use, Tauberer did not keep it to himself.  Instead, he made it freely available, both in bulk and through an application programming interface (API) so that other websites could dynamically access his database and provide up-to-the-minute legislative information themselves, in whatever format or context they judged best.  A partial inventory on GovTrack lists at least thirty current and former online projects that rely on GovTrack’s data, including prominent sites like OpenCongress and MAPLight.org.⁶⁴

Well into the 2000s, however, the concept of “open government” among public officials still centered on fresh disclosures, rather than improved access to already-public data.  The Honest Leadership and Open Government Act of 2007⁶⁵ dealt with requirements related to lobbying waiting periods and disclosures, earmark requests, and gifts to Congress.  That same year, another law with a similar title, the OPEN Government Act of 2007,⁶⁶ modified FOIA’s fee structure and established an ombudsman to oversee FOIA’s processes.  Neither of these bills approached “open government” in the technologically innovative mode of sites like GovTrack.

B.        “Open Government” Becomes a Label for Both Technological Innovation and Political Accountability

In recent years, participants in the policy debate—first in the United States, and then internationally—began to use the term “open government” in a more ambig­uous way.

President Obama and his team, both during the campaign and in gov­ernment, have shown a major commitment to both open government and open data—and they have also been the leading force behind the conceptual merger of the two ideas.

On the campaign trail, then-Senator Obama promised to “restore the American people’s trust in their government by making government more open and transparent,”⁶⁷ responding in part to his predecessor’s perceived lack of transparency.  At the same time, the technology and internet industries based in Silicon Valley served as a key source of financial and logistical support for the campaign, both through their own financial contributions and by helping to build a record-setting, web-based fundraising machine.⁶⁸  Obama was no stranger to the power of the internet: As a Senator, he sponsored the legislation that established USASpending.gov, an online portal that gave internet users an unprecedented degree of insight into the federal budget.⁶⁹  His background as a grassroots organizer also helped him appreciate the power of online networking to connect his supporters with the campaign and with each other.

Alongside their specific policy impulse toward transparency, therefore, the candidate and campaign harbored a powerful, if general, sense that internet technologies could open doors for innovation, efficiency, and flexibility in gov­ernment.  In effect, this was a commitment to open data.  “From a policy standpoint, there [were] many reasons for tech-minded types to support Obama, including his pledge to establish a chief technology officer for the federal gov­ernment and to radically increase its transparency by making most government data available online.”⁷⁰  The campaign itself embraced a data-driven approach to its fundraising appeals, rigorously tested alternative fundraising and outreach messages, and devolved to its supporters a significant degree of autonomy in interacting with their friends to build support.⁷¹

The Obama transition team created a high-level working group on technol­ogy and innovation, alongside similar working groups on economics, national security, health care, and other major issues.⁷²  The group had an ungainly name but an endearing acronym: the Technology, Innovation & Government Reform Policy Working Group, or TIGR (pronounced like Tigger, the friendly tiger from Winnie the Pooh).  The group’s charter was to help prepare the incoming administration to implement its Innovation Agenda, which included a range of proposals to

This charter was squarely focused on technological innovation rather than on civic accountability.⁷⁴

Meanwhile, the communities of technological and political openness had continued to merge outside of government.  A key meeting took place in the San Francisco Bay Area a year before the transition team’s work.⁷⁵  The recom­mendations drawn up by attendees at the meeting speak in merged terms of “open government data”:

The language here is telling: Participants understood themselves as “open government advocates,” but the principles they produced specify circumstances under which “[g]overnment data shall be considered open” (emphasis added), rather than government itself.  The eight principles, which include completeness, timeliness, and freedom from license restrictions, are requirements that attach to disclosures, not to regimes.⁷⁷  It may be true in some sense that a regime becomes more open whenever it provides additional open data, even for mun­dane and apolitical topics,⁷⁸ but it is easy to imagine that a closed regime might disclose large amounts of data conforming to these eight requirements without in any way advancing its actual accountability as a government.⁷⁹

There was also an emerging scholarly literature on the benefits that gov­ernment might enjoy from fuller use of the internet, encompassing but reaching well beyond technology-driven enhancements of public accountability.  Beth Noveck, who played a leading role in the Obama administration’s open gov­ernment initiatives, wrote a book in this vein arguing not only for transparency, but also for new modes of “collaborative participation” that leverage citizens’ expertise.⁸⁰  We ourselves made similar arguments in our paper, Government Data and the Invisible Hand.⁸¹  There, we advocated for the release of machine-readable, structured government data to help close “the wide gap between the exciting uses of Internet technology by private parties, on the one hand, and the government’s lagging technical infrastructure, on the other.”⁸²

On President Obama’s first day in office, he issued two memoranda that dealt with “open government,” using the term to refer both to increased trans­parency and to technological innovation.  The first, a memorandum on the Freedom of Information Act,⁸³ was designed to encourage agencies to be more responsive to FOIA requests.  It stated that FOIA

The creators of FOIA, as described above, had political objectives, not tech­no­logical ones, and this memorandum focuses squarely on those political goals—transparency and accountability.⁸⁵  The word “innovation” does not appear, and technology earns a mention not as an end itself, but rather as one of the key means of achieving the political objective: “All agencies should use modern technology to inform citizens . . . . [Future Office of Management and Budget (OMB) guidance should] increase and improve information dissemination to the public, including through the use of new technologies.”⁸⁶

The second memorandum, on Transparency and Open Government,⁸⁷ took a much broader view.  Whereas the FOIA memorandum suggested that a “new era of open Government” could be achieved through the transparency that FOIA compliance entails,⁸⁸ the Open Government memorandum treated trans­parency as just one among a trio of goals, setting out in separate paragraphs that an open government is transparent, participatory, and collaborative.⁸⁹  Trans­parency was just one of the features of open government, and public trust was just one of the benefits: “We will work together to ensure the public trust and establish a system of transparency, public participation, and collaboration.  Openness will strengthen our democracy and promote efficiency and effec­tiveness in Government.”⁹⁰

The new administration thus began to move toward a broader conception of open government than had existed before—one that drew on the tech­nological and philosophical commitments to innovation that the word already carried in technical circles.  The president’s memoranda set the stage for the Open Government Directive and the Initiative that were to follow.  Being accountable was just one part of what made a government “open”—participatory or collab­orative measures that enhanced efficiency or effectiveness might equally claim to be making the government more “open.”

The central practical mandate of the Open Government Directive,⁹¹ issued eleven months later, was an open data requirement, not a political transparency requirement: The directive required agencies to “publish online in an open format at least three high-value datasets” via the new federal data portal at Data.gov.⁹²  High value, in turn, did not necessarily mean politically sensitive: Aside from making the agency more transparent or accountable, data might also be high value if it would “improve public knowledge of the agency and its operations”⁹³ or “create economic opportunity.”⁹⁴

Predictably, agencies responding to this mandate have tended to release data that helps them serve their existing goals without throwing open the doors for uncomfortable increases in public scrutiny.  In many cases, agencies published datasets on Data.gov that were already available in other online locations.⁹⁵  While agencies packaged some of these datasets into more usable machine-readable formats, critics questioned how these disclosures added to the public’s “insight into agency management, deliberations, or results.”⁹⁶  Critics saw the repackag­ing of old information as providing only “marginal value” and urged the gov­ernment to make available “public data that holds an agency accountable for its policy and spending decisions.”⁹⁷  A broader study of Data.gov in 2011 noted a significant downward trend in agency dataset publication over the site’s first year.⁹⁸  It concluded that most federal agencies “appear[ed] to cooperate with the program while in fact effectively ignoring it,” and that Data.gov had become “the playground for a tiny group of agencies.”⁹⁹

Even as the administration’s political momentum for its Open Government Initiative waned, local and state governments began to adapt these ideas for their own purposes.  From New York to San Francisco, city and state leaders launched new websites devoted to sharing public data, often describing them as “open data” projects.¹⁰⁰  But the rhetoric among localities was more focused on service delivery than on accountability.  City leaders in particular put an emphasis on improving communities through better services.  San Francisco mayor Gavin Newsom expressed his hope that DataSF.org would “stimulate local industry, create jobs and highlight San Francisco’s creative culture and attractiveness as a place to live and work,” and only briefly acknowledged the possibility for greater accountability.¹⁰¹

Meanwhile, similar ideas have gained momentum internationally, reflecting other nations’ growing recognition of the new technological realities.  The European Union’s 2003 Directive on the Re-use of Public Sector Information instructed that “[w]here possible, documents shall be made available through electronic means,”¹⁰² and the EU now operates a website and program to encourage member states to develop their own national data portals.¹⁰³  Independent efforts were underway in the United Kingdom by 2007,¹⁰⁴ leading to the creation in 2008 of a “Power of Information Task Force” to explore the benefits of adaptable government data.¹⁰⁵  Data.gov.uk, launched in October 2009, appears to have been the first site of its kind outside the United States.¹⁰⁶

A new multilateral initiative, instigated by the United States, has dramat­ically accelerated the spread of these ideas over the past year.  In October 2010, President Obama addressed the United Nations General Assembly and urged member states:

Following up on this idea, the U.S. State Department organized a series of meetings leading to what became the multilateral Open Government Partnership (OGP).¹⁰⁸  As conditions of entry into the OGP, prospective member countries are required to meet a minimum set of standards that are based on traditional contours of government accountability: timely publication of essential budget documents, an “access-to-information” law that allows the public to obtain key government information, anticorruption disclosure requirements for public officials, and measures to promote citizen participation and engagement.¹⁰⁹  These fac­tors are fundamentally political, so the “open government” goals of the OGP initially appear to be centered on public accountability.

However, the Open Government Declaration that OGP member countries sign takes a broader approach toward “openness,” as signatories commit to “seeking ways to make their governments more transparent, responsive, accountable, and effective.”¹¹⁰  In addition to transparency and accountability, OGP member countries promise to “uphold the value of openness in our engagement with citi­zens to improve services, manage public resources, promote innovation, and create safer communities.”¹¹¹  Thus, the stated goals of the OGP include making gov­ernments both more efficient and more accountable, and it remains to be seen how much focus each of these disparate goals will receive.  By casting a wide net, the OGP has received the “open government” pledges of more than 55 countries,¹¹² including historically closed regimes like Russia.¹¹³  The practical impact of such pledges remains to be seen.

The framing value of “open government” has not gone unnoticed in the pri­vate sector, either: A growing list of companies have repackaged their gov­ernment-oriented information technology products under this attractive new label.  Microsoft, for example, has created an “Open Government Data Initiative,” which promotes the use of Microsoft’s Windows Azure online platform as a tech­nol­ogical underpinning for open data efforts.¹¹⁴  Adobe is best known in the government data context as the creator of the PDF document format, which is the baseline digital format for scanned paper documents (and which, like paper, tends to be difficult for downstream innovators to reuse).  Notwithstanding the fru­strations associated with the PDF format, however, the company undertook a major federal government marketing campaign in 2009 under the tagline “Adobe Opens Up,” triggering consternation among some activists.¹¹⁵  One company, Socrata, has even dedicated itself exclusively to the governmental open data mar­ket, with a “Customer Spotlight” on its website that touts its product’s adop­tion by Data.gov, Medicare, the State of Oregon, and the cities of Chicago and Seattle.¹¹⁶  These businesses have an incentive to sell open data technologies for the broadest range of governmental uses; their decision to brand their efforts in terms of “open government” is powerful evidence of how vague the term has become.

C.        Assessing the Merger

Taken together, these developments have caused a major change in the concep­tual landscape: “Open government” policies no longer refer to those that only promote accountability.  New modes of citizen engagement and new effi­ciencies in government services now share the spotlight with the older goal of governmental accountability, which once had this felicitous phrase all to itself.

The shift has real-world consequences, for good and for ill: Policies that encourage open government now promote a broader range of good developments, while policies that require open government have become more permissive.  A gov­ernment can now fulfill its commitment to be more “open” in a wider variety of ways, which makes such a promise less concrete than it used to be.  Whether used as a campaign slogan, in a speech or policy brief, or in a binding national or international policy instrument, the phrase “open government” no longer has the clarity it once had.  Existing documents and historical arguments that refer to open government may have lost some of their precision, becoming more ambiguous in retrospect than they were when first authored.

This new ambiguity might be helpful: A government could commit to an open data program for economic reasons—creating, say, a new online clea­ringhouse for public contracting opportunities—only to discover that the same systems make it easier for observers to document and rectify corruption.  In any case, there is much to like about economic opportunity, innovation, and effi­ciency, and a conve­nient label could be a good way of promoting them all.  Also, the new breadth of the “open government” label creates a natural cognitive association between civic accountability and the internet, which may be for the best.  Accoun­tability policies that embrace the internet are often a great deal more effective than those that do not.  (It might even make sense to say that if a government is not transparent through the internet, it is effectively not transparent at all.¹¹⁷)

But this shift might also allow government officials to placate the public’s appetite for accountability by providing less nourishing, politically low-impact substitutes.  If the less specific idea of “open government” displaces accountability as the conceptual focus of public reform efforts, less accountability may be achieved.¹¹⁸

In April 2011, in response to criticism that its Open Government Initiative was not doing enough for transparency and accountability, the Obama admin­istration launched a new site on “Good Government.”¹¹⁹  The new site focuses on harder-edged issues like shutting down superfluous federal buildings, publi­cizing the White House visitor logs, and strengthening ethics rules that restrict the lobbying activities of former administration staff.

Meanwhile, the Open Government Initiative and Data.gov appear to be focusing more and more on technological innovation and service delivery.  Beth Noveck, who launched and led the program as the U.S. Deputy Chief Technology Officer (CTO) for Open Government, has returned to private life; her successor, Chris Vein, is described instead as the Deputy CTO for Government Innovation, a title seemingly more appropriate to Data.gov’s accomplishments.¹²⁰

Noveck herself now regrets the decision to adopt “open government” as the umbrella term for internet technologies’ transformative potential in the public sector:

Instead, she writes, the term was meant to be “a shorthand for open innovation or the idea that working in a transparent, participatory, and collaborative fashion helps improve performance, inform decisionmaking, encourage entrepre­neurship, and solve problems more effectively.  By working together as [a] team with government in [a] productive fashion, the public can . . . help to foster accounta­bility.”¹²²  She suggests that the new White House structure, with separate focuses for transparency and for public sector innovation, may be more effective.¹²³

Notwithstanding a possible change of heart at the White House, however, the ambiguity of open government remains alive and well in the international sphere.  In some foreign countries, the need for public accountability is far more acute, and the opportunity cost of deprioritizing it may be far greater.  One of the clearest statements of this view comes from Nathaniel Heller, who directs an NGO called Global Integrity and was a key participant in the creation of the Open Government Partnership.  He raised the question after Kenya launched an open data website:

The common thread to these observations is that “open government” is vogue but vague, an agreeable-sounding term with an amorphous meaning.  We need better conceptual and linguistic tools, both for keeping governments honest and for exploring the transformative potential of information technologies in civic life.

To some ears, the idea of “open government data” has also developed a more threatening cast.  Wikileaks, first launched in 2008, has created what some call “involuntary transparency,”¹²⁵ reshaping the conversation over leaks of secret government information to the press.¹²⁶  In earlier instances such as the Pentagon Papers, secret government documents reached a single journalist or a small group of journalists, and the public gained access not directly to the secret infor­mation itself but instead to the finished journalistic product.¹²⁷  The raw material was summarized, adapted, or otherwise filtered before it reached the masses, and sometimes it included changes that reflected the requests of incumbent government officials.  Now, however, Wikileaks has made a series of large-scale disclosures of secret government information readily available to individual mem­bers of the public, often with little or no redaction of sensitive information.  The site has provoked complaints from sources as diverse as the U.S. Department of Defense and Amnesty International, particularly after a trove of 250,000 unredacted documents—apparently released by accident—put the lives of some foreign supporters of U.S. policy at risk.¹²⁸

But even for voluntary government disclosures, increased privacy risk may be a fundamental objection to these new technologies: The more easily disparate sources of information can be analyzed, combined, and cross-referenced, the greater the chance that previously pseudonymous information can be tied to the iden­ti­ties of particular real people.¹²⁹  On the other hand, a rush to limit adaptability to reduce the risk of privacy harms could create a “tragedy of the data commons,” in which privacy fears foreclose valuable new insights into public issues.¹³⁰

“Mosaic” risks in national security present an analogous problem: Even if it is not sensitive when considered in isolation, a release of seemingly innocuous data may become useful to America’s adversaries if it can be combined to yield sen­sitive inferences about America’s defense and intelligence posture.¹³¹

Our goal here is not to take a position as to the salience or implications of these risks but rather simply to point out that they can complicate the cost-benefit calculus of the governmental “open data” trend.

III.    Our Proposal

Clearer language is possible, and it will serve everyone well.

From civic accountability to transit data to health statistics, online disclosures of government data across the world share one exciting feature: They are far more adaptable than ever before.  Statistics can be mapped, schedules automated, dis­parate trends cross-referenced, and useful information localized and personalized to a historically unprecedented extent.  Online data—particularly if it is structured, machine readable, and available for interested users to download in bulk—can be more readily adapted to new formats, new uses, and new com­binations than ever before.  Adaptability is independent of subject matter: Any subject—including transit, regulation, schools, crime, or housing—can be a source of data, and that data may be more or less adaptable depending on the format in which it is gathered and presented.

Offline data is very different: They gather dust in filing cabinets, often disorganized and disregarded.  An obscure bit of information remains apart from the handful of people who might really benefit from knowing it because it would cost too much to search, sort, or reorganize.  Offline data, though available in principle, is physically and psychologically heavy, encumbered by brick and mortar logistics, and tucked away in rooms with limited opening hours.  Offline data is inert.

Public disclosures thus occupy a spectrum, from the most adaptable data to the most inert.  Adaptability may depend on not only the format of the data itself but also on the prevalence and cost of the human and technological capital neces­sary to take advantage of it.

Disclosures also vary in a second dimension: They differ markedly in their actual or anticipated impact.  A machine-readable bus schedule aims to promote convenience, commerce, and a higher quality of life—it enhances service delivery.  Core political data, such as legislative or campaign finance information, serves a more purely civic role, enhancing public accountability.  Disclosures of public contracting opportunities play a dual role, potentially enhancing both economic opportunity and public integrity.

The diagram displays this conceptual model and gives several examples.  The vertical axis describes the data itself, in terms of its degree of adaptability.  The lateral axis is a continuum from purely pragmatic to purely civic disclosures.

Conclusion

The vagueness of “open government” has undercut its power.  Separating technological from political openness—separating the ideal of adaptable data from that of accountable politics—will make both ideals easier to achieve.  Public servants can more readily embrace open data, and realize the full range of its ben­efits, when it is separated from the contentious politics of accountability.  At the same time, political reformers—no longer shoehorned together with technologists—can concentrate their efforts on political accountability, whether or not they rely on new technology.  And governments will be less likely to substi­tute technol­ogy initiatives for hard political change.