The government's own statistics and various studies from the National Security Archive and others have shown the massive delays in processing FOIA requests. Drafting and application of FOIA exemption claims and classification policy has been demonstrated to depend too heavily on the attitude of the particular administration towards openness. Governments frequently confuse what they perceive to be the interest of the administration with that of the public interest. The purposes and objectives of the FOIA are unclear, and the law fails to define measurable outcomes for openness, accountability and pre-decisional participation by the public in the policy-making process.
Experience around the world teaches us that government commitments to strengthen openness tend to be once in a generation opportunities to achieve - and embed - positive change. Changes to policy alone are meaningless if they are not supported by embedding changes within the law, as the next administration can alter policy far more easily than law (cf Att. Gen. Reno's FOIA memorandum vs. that from Att. Gen. Ashcroft).
It is therefore time for a root and branch re-write of the FOIA.
First, the law should contain a 'purposes' clause which sets out the aims of the law, and which agencies receiving requests and the mechanism to enforce the law must have regard when determining how to respond to requests. These should encapsulate the outcomes desired from the law, and other sections of the law should mandate the collection of such data as will facilitate the measurement of whether the outcomes are being achieved.
A good example of a purposes clause can be found in section 4 of the New Zealand Official Information Act 1982
The purposes of this Act are, consistently with the principle of the Executive Government's responsibility to Parliament,—
(a) To increase progressively the availability of official information to the people of New Zealand in order—
(i) To enable their more effective participation in the making and administration of laws and policies; and
(ii) To promote the accountability of Ministers of the Crown and officials,—
and thereby to enhance respect for the law and to promote the good government of New Zealand:
(b) To provide for proper access by each person to official information relating to that person:
(c) To protect official information to the extent consistent with the public interest and the preservation of personal privacy.
Second, the classification of a document for security purposes must be immaterial when taking a decision on whether it should be disclosed in response to a FOIA request. The Act itself should set out the sole grounds for justifiable withholding, in terms of specific harms to particular interests. There should be no scope for information to be withheld simply because it falls within a particular category: each exemption from the right of access should be subject to a test of harm. In the event that disclosure would, or would be likely, to cause harm to one of those interests, withholding should still only be permitted if the public interest in maintaing the claim for exemption outweighs the public interest in disclosure. The FOIA must override all other provisions in other statutes that provide for secrecy. It must also override secrecy provisions in international treaties, the bilateral 'security of information' treaties negotiated by the USA included. The FOIA should be the paramount legal instrument for arbitrating claims to secrecy and openness.
Third, the law must set out those categories of information which each agency is affirmatively mandated to publish. The Mexican federal FOI law provides a good example of 17 key categories of information, including all government contracts, research grants etc. However, this approach can be too static in the long run, as the shape and nature of government evolves. The UK FOIA approach of 'publication schemes', which are documents setting out the categories of information each authority will publish either proactively, or on request without further consideration of exemptions, is also worth adding to the affirmatory categories. 'Publication schemes' must be approved by the law's enforcement mechanism, thereby providing for flexibility and for the level of openness to be ratcheted upwards over time. (see s.19 of the UK FOIA http://www.opsi.gov.uk/acts/acts2000/ukpga_20000036_en_2#pt1-pb4)
This is the area of an FOI law most open to interface with provisions for agencies actively making available for scrutiny and re-use the datasets of information that they hold. Making them available in accordance with this provision of the FOIA provides the enforcement mechanism with a role in underpinning the Open Data initiative taken by this administration.
Fourth, the primary mechansim for adjudication of claims under the FOIA should no longer be the courts. The experience since 1966, and in other countries where claims are resolved in the courts shows that this, to put it bluntly, a broken mechanism if one is to faciliate the participatory goals of a FOIA. This was recognised in the FOIA Amendment Act promoted by Senators Cornyn and Leahy which created the office of a pseudo-Ombudsman in the National Archives and Records Administration. Although it was along the right lines, it was flawed. Instead, a Federal Information Commission should be created to receive complaints regarding non-compliance with the FOIA. It should have the duty to investigate those complaints, the power to compel agencies to supply it with the information it needs to do so, and to take evidence on oath if necessary, and to order partial or full disclosure of the information requested. It should have the duty to approve or reject agencies' 'publication schemes', depending on whether they will result in desired levels of proactive disclosure. It should have the duty to promote awareness of the Act and to deliver training, both to government officials and to the public. It should have duties to collect data which will demonstrate whether the desired outcomes of the FOIA are being achieved. It should have the duty to report annually to Congress, and to have the power to conduct 'own motion' (sui generis) investigations into systemic failings on the part of agencies to comply with or provide high standards of administration of the FOIA. The Commisson should be mandated to collaborate with NARA to improve standards of record-keeping and information management in government. It should also be empowered to collaborate with the GAO and other oversight and integrity institutions where this would drive up standards of governance, accountability and integrity. In particular the Commission should be mandated to pay particular attention to the issue of ensuring that requests for information needed by the public (or their proxies such as NGOs) to participate in the policy making and service design process are expedited. It is too easy for agencies and enforcement bodies to pay more attention to post-decisional accountability issues than to pre-decisional participation. To this end, the government should consider the desirability of placing time-limits within the FOIA for resolution of complaints about non-disclosure of requested information, and recognise that the Commission will need to be resourced appropriately in order to do so.
The Commission should have the power to instigate proceedings for contempt of court if its orders for supply of information or disclosure are not complied with.
Appeals against the decisions of the Information Commission to the Appeals and Supreme Court should be permitted by both the request and the agency, both on a point of law and on the merits. However, if the agency appeals, it should fund the legal representation of the requester, to ensure 'equality of arms' before the Court.
In conclusion, the US FOIA has proved a valuable but flawed tool to assist open government. But considerable experience, both in the US and overseas, has demonstrated that the drafting of the law is no longer fit for purpose and has fallen behind the lessons learnt by others. There are many benefits to be gained from overhauling the statutory guarantees of citizens' rights to information, and the current opportunity should be seized firmly.
I'd be happy to provide more detail upon request.