In an October 9th article in the New Republic, Lawrence Lessig authored an important critique of the transparency movement. Primarily asking the questions – transparency is all fine and good, but what is the end?
Lessig’s article and a related article by Jill Lepore, which takes a critical look at the history of scientific management, in The New Yorker point out that what seems to be a good idea at one time may not be in the long run, especially if not examined. As the saying goes “the road to hell is paved with good intentions.” Thus, without careful review the rush to transparency may lead to unintended consequence that we do not want and may jeopardizes the good that could have been achieved by opening up government data.
In fact, Lessig points out the non-contextual nature of raw data. Without context, data can saying anything. An example that comes to mind is the Denver Post’s effort to publish the salaries of all Colorado state government workers in the Post’s data center (the information is no longer available). As a pure transparency matter, such publication is not a bad idea. However, this data cannot speak the whole story alone. The data that was release simply stated the name of the employee, the employee’s agency or department, the employee’s state classification, and salary. The data did not say if the employee worked for a full year or not. The data did not say if the employee was promoted or demoted during the year. The data did not say what the employee’s job was. The data was without context. So yes a few department of transportation drivers appear to have huge salaries. Why? They drive snow plows and work lots of over time. Was the information useful? Yes. But without the context it does not tell the whole story.
Neither Lessig’s article nor this post on O’Reilly Radar, which highlights both Lessig and Lepore articles, condemn open government data/transparency, rather they point to the need for goals and to ask the important question of “what is the end?” I support transparency fully and think government does need to open up more information. But we do need to think about what will be the benefits and the costs. And if we are republishing, redisturbing, or mashuping any of the data for use as a policy tool, we must provide context and avoid bias whenever possible.