Improving User Access to Metadata for Public and Restricted Use US Federal Statistical Files

Abstract

The US federal statistical system produces prodigious amounts of public and restricted use data. The restricted use data can be difficult to interact with due to poor documentation. Documentation across agencies and the public use/restricted use divide that has been produced does not adhere to a single standard, making the metadata useful, but insular. The Data Documentation Initiative (DDI) is an emerging metadata standard that is used internationally to describe data in the social sciences. It has the potential to unify the metadata managed by separate organizations into a comprehensive searchable set. Researchers from the Labor Dynamics Institute, in collaboration with the Cornell Institute for Social and Economic Research (CISER) received funding from the National Science Foundation to improve the documentation of federal statistical system data with the goal of making it more discoverable, accessible and understandable for scientific research. The scope of this paper is a subset of the overall project, and reports on development of the web interface for user searches and the search API. The primary data model being utilized in this application is DDI 2.5 (Codebook), which contains elements and attributes to describe the contents of a data set.