Golden Oldie #6 – Four Historical Search Decisions

There are countless decisions, law review articles, books, and guides written on search, search design, search methodology, search tools, and search metrics.  See, e.g., EDRM Search Guide; The Sedona Conference Best Practices Commentary on the Use of Search and Information Retrieval Methods in E-Discovery

The following are a few exemplar decisions that set the stage:

• In Re Seroquel Prod. Liability Antitrust Litigation, 244 F.R.D. 650 (M.D. Fl. 2007).
• Equity Analytics, LLC v. Lundin, 248 F.R.D. 331 (D.D.C. 2008).
• Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251 (D. Md. 2008).
• A. Gross Construction Assoc., Inc. v. Amer. Mfrs. Mut. Ins. Co., 256 F.R.D. 134 (S.D.N.Y. 2009).

As early as 2007, Seroquel held that search must be a cooperative and informed process.  In that case, one litigant designed its search in secret.  The court held that the Rules required discussion of search terms and the use of sampling for quality control.

AZ purported to embrace the requirements of Rule 26 and the Sedona Principles. However, the reality was to the contrary. For example, while key word searching is a recognized method to winnow relevant documents from large repositories, use of this technique must be a cooperative and informed process. Rather than working with Plaintiffs from the outset to reach agreement on appropriate and comprehensive search terms and methods, AZ undertook the task in secret. Common sense dictates that sampling and other quality assurance techniques must be employed to meet requirements of completeness. If AZ took such steps, it has not identified or validated them….  In this case, AZ never discussed with Plaintiffs which search terms to use as part of the search. There was no dialogue to discuss the search terms, as required by Rules 26 and 34.

Id. at 662, 664 (Emphasis added).

Equity Analytics made clear that search design was complex.  “Whether search terms or ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics.”  Equity Analytics established the need for specifics, not mere conclusions, in defense of search methodology: “I recently commented that lawyers express as facts what are actually highly debatable propositions as to efficacy of various methods used to search electronically stored information.”

Victor Stanley added:

• “[W]hile it is universally acknowledged that keyword searches are useful tools for search and retrieval of ESI, all keyword searches are not created equal; and there is a growing body of literature that highlights the risks associated with conducting an unreliable or inadequate keyword search or relying exclusively on such searches for privilege review.”
• “Common sense suggests that even a properly designed and executed keyword search may prove to be over-inclusive or under-inclusive….”

The Victor Stanley Court made it clear that an expert in search design was not required; however, as in Equity Analytics, “ipse dixit pronouncements from lawyers unsupported by an affidavit or other showing that the search methodology was effective for its intended purpose are of little value….”

Thus, the Victor Stanley Court wrote: “Parties should expect that their choice of search methodology will need to be explained, either formally or informally, in subsequent legal contexts (including in depositions, evidentiary proceedings, and trials).”

The Victor Stanley Court explained that:

The only prudent way to test the reliability of the keyword search is to perform some appropriate sampling of the documents determined to be privileged and those determined not to be [i.e., the discard pile] in order to arrive at a comfort level that the categories are neither over-inclusive nor under-inclusive.

See also Historical ESI Highlights – Part VI – Victor Stanley and Keyword Searching (May 29, 2022).

In Wm. Gross, the court issued a wake-up call to the Bar and stated that lawyers cannot design searches by the seat of their pants.  It pointed to the need for quality control to test for false negatives and false positives.  The Wm. Gross court also made the important point that a well-designed, pre-litigation information governance program could have avoided what became a costly and contested litigation issue.  The Wm. Gross court noted that the complex search problem “would have been avoided, of course, if Hill used a standard ‘Re’ line in its Bronx Courthouse emails to distinguish that project from its other work. It did not do so….  That left the Court in the uncomfortable position of having to craft a keyword search methodology for the parties….”   The court added:

The Court acknowledges that this result is less than perfect, and that there is a risk that as information later comes out at depositions of the Hill employees, another search may have to be done.

The opinion concluded by stating:

Of course, the best solution in the entire area of electronic discovery is cooperation among counsel. This Court strongly endorses The Sedona Conference Cooperation Proclamation (available at www.TheSedonaConference.org)….  Electronic discovery requires cooperation  between opposing counsel and transparency in all aspects of preservation and production of ESI. Moreover, where counsel are using keyword searches for retrieval of ESI, they at a minimum must carefully craft the appropriate keywords, with input from the ESI’s custodians as to the words and abbreviations they use….