ediscovery faceplant

5 Ways To Avoid An eDiscovery Faceplant

A Case for eDiscovery Keywords


The Electronic Discovery Keyword Search

Elder statesman of the Electronic Discovery world.  Simple to understand and easy to implement, keyword searching has played a central role in eDiscovery projects for more than a decade.  But as advances in eDiscovery expand the capabilities of technology-assisted review (“TAR”) and introduce us to concepts such as predictive coding and visual analytics, the good old fashioned keyword search has begun to lose its luster.  U.S. Magistrate Judge Andrew Peck, the judge who first approved TAR, wrote that a legal team’s choice of keywords often “is the equivalent of the child’s game of ‘Go Fish.’” Accomplished eDiscovery attorney, writer, and speaker Ralph Losey notoriously opined that “keyword search sucks”.  For these and other vocal critics, the keyword search is analogous to the 2015 version of Peyton Manning – sluggish and error-prone, with questionable accuracy.

Alt: Poorly Thought Out Keywords Are Time Consuming and Expensive


Know Thy Keywords

Most of the shade being thrown at poor keyword search is defensible to a certain degree.  After all, keyword searches behave like blunt instruments.  Attorneys who aren’t familiar with proper search methodologies run the risk of creating lists of search terms that are both under- and over-inclusive.  For example, searches conducted for the word “record” could yield results for a conversation about the new Kanye West record, a news article documenting Usain Bolt’s world record, a copy of a court record, a company’s safety record, or an employee’s spouse’s request to record Game of Thrones.  These so-called false positives decrease the percentage of relevant documents reviewed, increase client costs, and accelerate reviewer fatigue.

Alt: Finding Hidden eDiscovery Gems


Know Thy File Types

Litigators and document review attorneys often blame keyword searches for erroneously returning keyword hits.  A common refrain I hear during document reviews goes something like this: “The eDiscovery software says that Keyword “X” is somewhere in this document, but I’ve looked everywhere and it’s just not there!”  The truth of the matter is that the keyword has almost always been correctly identified.  It’s just that…

  1. The review software doesn’t do a good enough job of alerting the reviewer to where the keyword is and…
  2. The reviewer almost never understands how to find it.

I once joined a document review about two weeks after the project began.  Some of the client’s sales force had been accused of negotiating kickbacks with foreign agencies.  Although the names of the suspected individuals continued to show up as keyword hits in a large number of spreadsheets, I was told by more than one attorney that the keyword search wasn’t working and that I should ignore these documents.  It was only after I opened the documents in their native Excel format that I discovered the keywords (and proof of collusion) had been concealed in hidden rows, columns, and sheets.  Without proper training in the strengths and weaknesses of the eDiscovery software being used, reviewers can also miss critical, sometimes privileged, information left in the comments of Word, Excel, and PowerPoint files.


Take Heart, O’ Keyword Search

Inherent limitations aside, the question the legal community should be asking isn’t whether, but rather how keyword searches should be used in eDiscovery.  In fact, the selective use of keyword searches can still be the cornerstone of an efficient and defensible search methodology.  A properly deployed search takes advantage of a unique feature of electronic information –  the ability to quickly and accurately find relevant people, words, and concepts in large data sets.  But successful keyword searches require foresight, communication, and flexibility between litigators and ESI personnel.  


Keyword Search Best Practices

One of the most frequently cited guidelines regarding eDiscovery, the Sedona Principles Addressing Electronic Document Production, Second Edition, and its corresponding Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery provides a thoughtful commentary on the use of traditional keyword searches.  It suggests the following primary components can be utilized in an effective and defensible search methodology.

Testing.  Initially, keyword searches must be tested for accuracy and effectiveness, with particular emphasis on whether the search is producing over- or under-inclusive results. As stated by the Principles: “[m]ore advanced keyword searches using ‘Boolean’ operators and techniques borrowed from ‘fuzzy logic’ may increase the number of relevant documents and decrease the number of irrelevant documents retrieved.”

Sampling. After initial testing, the primary way to test the efficacy of keyword searches is through sampling.  Statistical sampling can assist attorneys in strengthening cases, reducing costs, increasing confidence in relevance, and improving defensibility.

Iterative feedback. The Principles also recommend creating an iterative feedback loop in order to reformulate and refine keyword searches.   With collaboration between lawyers and eDiscovery experts, searches can be conducted in stages to hone and validate results as the project continues.  

Because actual data is being examined when testing, sampling, and creating iterative feedback, the keyword search ceases to be a guessing game and begins to increase coding precision and recall (i.e., over- and under-inclusiveness).


 Additional Uses For The Keyword Search:

Most document reviews tend to employ a “let it ride” mentality to keyword searches.  Once the initial keywords have been tested and sampled, they remain the same throughout the duration of the project.  However, modifying and adding to the list of keyword terms during the course of a document review can pay huge dividends for reviewer accuracy and efficiency.  

For example, let’s assume all meeting invitations are non-responsive.  Let’s also assume every meeting invitation contains the relatively unique term “standard time”.  By adding “standard time” to the keyword search, you could filter most of your results to meeting invitations with a fair amount of certainty.  

Did a document reviewer discover a group of attorneys from a law firm that wasn’t included on the initial privilege list?  By adding the law firm’s domain name to a keyword search you decrease the likelihood that privileged documents will accidentally make their way into a production.  Didn’t know that the key defendant’s nickname was “Night Train” until four weeks into the review?  An extended keyword list could come in very handy.



Keyword search has unfairly found itself under fire by a number of judges, attorneys, and eDiscovery experts.  Sure, it’s not perfect.  But it’s alleged weaknesses often shine a light on the shortcomings of litigators and document review attorneys who aren’t communicating and don’t understand how to get the most out of their keyword searches.  What’s needed is better training on the intricacies of creating keyword lists and a greater understanding of the how document review platforms work.  So while the eDiscovery experts continue to develop new concepts and shiny new software, it’s important to recognize that the keyword search is still a useful – even essential – piece of equipment in the modern attorney’s tool box.

No Comments

Sorry, the comment form is closed at this time.