PDF Redaction -- some Q&A

A couple weeks ago a reporter contacted me regarding the PDF-related aspect of the story of the US Army report on the killing of an Italian agent in Iraq. I'm not sure if he's going to get published, or if I'll get a sound-bite in his final article, so I thought I'd post an edited version of our email "interview."

-How was this mistake made with the PDF by the military?
It seems like a lack of fundamental knowledge about the structure of a PDF file, and what you can and can't do with the various Acrobat tools. As I tell lawyers, it is helpful to visualize that PDF files, like onions (and ogres), have "layers." With a paper document, drawing across text with a black marker doesn't *remove* the text, it just obscures it. In the case of a PDF file, it is possible to either remove the cover-up markings, or, as the reports of this case implied, slip the text out from under the covers.

-Is it a common mistake?
Unfortunately, it seems to be. I don't have any stats on how often it happens with lawyers, but there have been several high profile instances in the past few years. It's not just the military -- both the Washington Post (the Washington sniper notes) and New York Times (a CIA document) have each managed a similar snafu. DOJ did it awhile back too.

-How do you avoid it?
You could print your redacted documents to paper . . .

First, there are tools on the market that deal specifically with this task. A company named Appligent sells one called "Redax" that is widely used by federal agencies.

Second, learn about and understand your tools. Get some training; implement some quality control in your organization; standardize a workflow; employ a geek that knows and cares about security. This isn't the result of some sort of hidden flaw in the PDF format or the Acrobat application.

-How do you protect your documents?
If I produced redacted things, I would use a commercial tool that does that task well, and make it part of my workflow. Just locking them with password protection isn't going to do the job -- you have to remove the text.

-Were you aware of this potential problem with PDF?
Hell, yes I was aware that using a black high-lighter tool in Acrobat doesn't "redact" anything. However, I think it is inaccurate to characterize this as a "potential problem with PDF." The issue is not with the file format, or with Acrobat.

I'd characterize it as a lack of skills and awareness on the users' part, which is generally related to a lack of training, lack of curiosity about and time to use and understand the tools, and also to a failure by organizations to provide proper tools and methods. Hey, somebody at CentCom IT must have been aware (I hope!) that you can't redact a document in this fashion, but either the proper tools weren't available, the user didn't know they were available, or he didn't have access to them.

There was a lot of nerd-chatter about what bone-head the user must have been, but I think it was primarily a failure at an institutional level -- it's not like this type of file goes out into the world (unless it's leaked) without a whole bunch of people approving it. But in all the approval process, where was the "quality control" that asked "is this document properly secured?" Where was the geek that should have QC'd it for security? The print industry spends a huge amount of money and effort assuring that a PDF has all the right fonts embedded in it -- so why don't legal and government institutions have even cursory checks for things like document security, metadata removal, etc.?

-Do you think the general public is aware?
I'd say not, based on recent events. I would assume that some (many? most?) users of Acrobat 6 (who are definitely not the general public) are similarly unaware of how properly to use the software. This isn't some solo practioner in East Poisonspider, Wyoming screwing up -- this is the US Army, NY Times, Washington Post, etc.

-What could some of the consequences be if a document you thought was properly redacted could actually be read in its entirety?
Most documents are redacted to remove privileged information, or confidential business or personal information. From a legal/litigation standpoint, you certainly have some serious issues with privilege claims. But, as with redacting with a magic marker that redacts poorly and allows the underlying text to be read, what are you gonna do? Even if the information can't be admitted in evidence in a proceeding, it doesn't mean it's not useful to other parties.

I haven't seen any cases where an attorney faced a malpractice claim for failing to understand the basics of his office software, but surely that day must come. There are certainly penalties for government entities and personnel that violate the Privacy Act and other statutory and regulatory requirements.

Plus, you look inept, which can't be good from a professional standpoint.

~~ Dave

Comments

[M. Sean Fosmire]

One rather easy way to have avoided this problem would have been: after making the obliterations, print the PDF file using pdfFactory or another similar product to create another PDF file representing an e-version of the original as printed.

[Traverse City Attorney]

I have instructed my staff to ensure that they print the redacted .pdf for production. If you 'show comments' on the print job, you redact. I would not send a redacted .pdf to opposing counsel.

[Dave Fishel]

I'm not sure what Mr. Fosmire means by "creating another PDF file." If by doing so you essentially create an image-only file, I guess that would work. I'd have to know more about it before being comfortable with doing it that way.

Printing a hard copy will certainly do the trick. Be quite sure you have no *other* comments that show up! This method will pose a problem if you are producing huge amounts of documents, or if you have to e-file it (I guess you could print and re-scan). The great advantage of paper is that what you see is what you get.

Appligent's Redax apparently works by extracting the highlighted text from the file entirely, and replacing the area with a block of color. It seems to be the product of choice for high volume jobs. For example, a government agency that is responding to a FOIA request may have to produce a zillion documents to a gaggle of requesters. It makes sense (tax money-wise) to do that electronically rather than on paper.

I agree that people should be very, very careful if they produce a redacted pdf to anyone.

[Richard]

We've been very cognizant of this issue and recently purchased Docudesk's deskPDF. Their new system in 2.5 has built in profiles which allow users to set different settings (i.e. internal PDF / external PDF / court PDF /etc.). It also integrates with Active Directory for Administrative control of the various settings and profiles.

While creating a "flattened PDF" image would work you lose the ability to do contextual search.

[council4baytown]

I found the company referenced above intriguing and thought I would share the link I hunted down for them. www.docudesk.com

[Roy Brookes]

There are Redaction software products available to prevent this type of incident from occurring. Nearly all of the Redaction products available have a flaw in that they can do only one type of document. Since using RapidRedact I have found that it can do all document types, including emails, word documents, PowerPoint’s etc. This is by far the best redaction software available on the market today.
www.rapidredact.com