They're from the government and they're here to help you. No, really. Some of the biggest collectors of data are the various governments, whether on a national, state and regional or local level. Increasingly, governments are making this data available to developers, who are using it to produce applications.
On President Barack Obama's first day in office, he issued the Memorandum on Transparency and Open Government, which instructed the Director of the Office of Management and Budget (OMB) to issue an Open Government Directive and to direct executive departments and agencies to take specific actions to implement the principles of transparency, participation and collaboration.
Transparency promotes accountability by providing the public with information about what the Government is doing. Participation allows members of the public to contribute ideas and expertise so that their government can make policies with the benefit of information that is widely dispersed in society. Collaboration improves the effectiveness of Government by encouraging partnerships and cooperation within the Federal Government, across levels of government, and between the Government and private institutions.
In response, the Office of Management and Budget issued a more detailed Open Government Plan with more detailed instructions to department heads. The government also created a portal, Data.Gov, to all sorts of freely available federal datasets, which is updated on a regular basis with new datasets. The federal government also installed Vivek Kundra as the first federal CIO in the nation's history.
Kundra recently resigned, but the new CIO, former Microsoft executive and FCC managing director Steven VanRoekel, is likely to continue and even extend the open government effort. Wrote Alex Howard on the GovFresh blog:
He brought a .com mentality to the FCC, including a perspective that “everything should be an API” that caught some tech observers’ eye. He worked with an innovative new media team that established a voice for social media for the @FCC on social media where that had been none and a FCC.gov/live livestream that automatically detected what device you’d used to access it."
In general, government data on the state level tends to be less available than on either the federal or local level. This was also the finding of an open government survey by Socrata, a company that licenses infrastructure for open data initiatives, which found the following:
- Only 39.3% of respondents said that state government had an open data mandate, compared with 44.4% on the county level, 55.1% on the federal level and 42.9% on the municipal level.
- Only 17.3% of respondents said that individual state departments publish data on their own web pages, compared with 45% on the county level, 25.8% on the federal level and 20% on the municipal level.
- Similarly, only 23.1% of respondents said that state governments had already set up a datasite, compared with 24.7% on the federal level and 28.9% on the municipal level. On the other hand, only 10% of respondents said that such datasites were available on the county level.
- In the good news front, though, 32.7% of respondents said the state government was in the planning stages, compared with 15% on the county level, 24.7% on the federal level and 28.9% on the municipal level. Similarly, only 17.3% of respondents said the state government had no plans to set up a datasite, compared with 30% on the county level, 11.3% on the federal level and 22.2% on the municipal level.
Consequently, it's likely that more data on the state level will be forthcoming in the future.
A number of cities have also implemented various open data initiatives, as well as various initiatives intended to encourage developers to write applications that make use of them.
Similarly, the Metropolitan Transit Authority is challenging developers to create new software applications that improve the transit experience of its 8.5 million daily riders, with prizes of up to $5,000 -- as well as a behind-the-scenes tour of Grand Central Station to all applicants. Prizes will be awarded for apps that do the most to improve the transit experience of the MTA’s by helping riders better navigate the system, the agency says, adding that it has made more than a dozen data sets available for the challenge, including General Transit Feed Specification (GTFS) schedule data; current service status; real-time information for some bus lines; elevator/escalator status; turnstile and fare data; bridge and tunnel traffic data; and subway entrance GIS data.
San Francisco, with DataSF, regularly publishes new datasets, such as the location of all public art or information about historic sites. In addition, the Apps Showcase helps publicize applications that make use of the datasets, such as one that helps people find vacant parking spaces.
Washington D.C.'s Data Catalog offers access to 484 datasets, including visualization tools, and offers the data to developers as well.
Third-party developers are also getting into the act. Sunlight Labs, a project of the Sunlight Foundation, which promotes government transparency, offers a regular blog talking about how its developers are making use of open government datasets to create applications. Nonprofit organizations such as GovLoop also help encourage development of open government applications.
More targeted organizations make use of the data as well. For example, the Environmental Working Group has a series of applications having to do with data from the Department of Agriculture, such as tracking farm subsidies.
Finally, for-profit developers are also working to help people write applications using government data. For example, Microsoft's Open Government Data Initiative offers a data repository of 60 government datasets, to which it adds, as well as information about how professional and amateur developers can best make use of it.
What's definite is that there's going to be more such developments in the future, not less. Going back to the Socrata survey, 67.9% of respondents believe that government data is the property of taxpayers and should be free to all citizens; 67.5% believe that if government data is supposed to be public, it should be available online; 61.4% believe that entrepreneurs will create new products and services based on the data; 61% are more likely to vote for politicians who champion data transparency; 60.3% believe that broad access to government data will help identify and reduce inefficiencies in government; and 56.3% would trust their government more if it would put the majority of government data online.
On the other hand, only 47% believe that data transparency is an important enough initiative to fund with taxpayer dollars. Apparently the importance of data transparency only goes so far.