Historical ESI Highlights – Part VI – Victor Stanley and Keyword Searching

The prior blog discussed the historical acceptance of technology assisted review.  However, keyword searching remains an important and useful tool.  Mark Twain wrote: “The reports of my death are greatly exaggerated….”  The same may be true of keyword searching.

While there are many decisions and articles on keyword searching, one of the leading ones is a decision of the Hon. Paul W. Grimm, in Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251 (D.Md. 2008).  There are many aspects of the multi-faceted series of cases that followed this decision; however, this blog focuses on the keyword search discussion.[1]

The decision cites authority that basic keyword searching has been widely accepted and continues in use.  It states: “While keyword searches have long been recognized as appropriate and helpful for ESI search and retrieval, there are well-known limitations and risks associated with them, and proper selection and implementation obviously involves technical, if not scientific knowledge.”  Victor Stanley, 250 F.R.D. at 260.  It then quotes the Sedona Conference to show the best practices.

A search must be designed by qualified persons, with quality-assurance testing, and the proponents must carry the burden of explaining what they did and why it was sufficient.  Id. at 262.

This is accomplished by using concepts of recall, precision, and richness, and other “Q.C.,” or “quality control” measures, such as testing (“sampling”) the discard pile.

One key issue in Victor Stanley was defensibility of a keyword search after 165 assertedly privileged documents had been produced.  The proponent failed to carry its burden. The decision begins by noting:

Use of search and information retrieval methodology, for the purpose of identifying and withholding privileged or work-product protected information from production, requires the utmost care in selecting methodology that is appropriate for the task because the consequence of failing to do so, as in this case, may be the disclosure of privileged/protected information to an adverse party, resulting in a determination by the court that the privilege/protection has been waived. Selection of the appropriate search and information retrieval technique requires careful advance planning by persons qualified to design effective search methodology. The implementation of the methodology selected should be tested for quality assurance; and the party selecting the methodology must be prepared to explain the rationale for the method chosen to the court, demonstrate that it is appropriate for the task, and show that it was properly implemented. In this regard, compliance with the Sedona Conference Best Practices for use of search and information retrieval will go a long way towards convincing the court that the method chosen was reasonable and reliable, which, in jurisdictions that have adopted the intermediate test for assessing privilege waiver based on inadvertent production, may very well prevent a finding that the privilege or work-product protection was waived.

The backdrop was two decisions by the Hon. John M. Facciola in the District for the District of Columbia and an article cited by Judge Grimm that “raised eyebrows” over those decisions.

Briefly stated, Judge Facciola had written:  “Whether search terms or ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics…. Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread.”

A commentator then suggested that the decisions mandated a Daubert-qualified expert to design a keyword search.  Judge Grimm disagreed with the commentator.

Judge Grimm’s decision laid out the framework for a defensible keyword search:

Judge Facciola made the entirely self-evident observation that challenges to the sufficiency of keyword search methodology unavoidably involve scientific, technical and scientific subjects, and ipse dixit pronouncements from lawyers unsupported by an affidavit or other showing that the search methodology was effective for its intended purpose are of little value to a trial judge who must decide a discovery motion aimed at either compelling a more comprehensive search or preventing one…. It cannot credibly be denied that resolving contested issues of whether a particular search and information retrieval method was appropriate—in the context of a motion to compel or motion for protective order—involves scientific, technical or specialized information. If so, then the trial judge must decide a method’s appropriateness with the benefit of information from some reliable source—whether an affidavit from a qualified expert, a learned treatise, or, if appropriate, from information judicially noticed. To suggest otherwise is to condemn the trial court to making difficult decisions on inadequate information, which cannot be an outcome that anyone would advocate.

The decision continues:

 Viewed in its proper context, all that [Judge Facciola’s decisions] required was that the parties be prepared to back up their positions with respect to a dispute involving the appropriateness of ESI search and information retrieval methodology—obviously an area of science or technology—with reliable information from someone with the qualifications to provide helpful opinions, not conclusory argument by counsel…. The message to be taken from O’Keefe, Equity Analytics, and this opinion is that when parties decide to use a particular ESI search and retrieval methodology, they need to be aware of literature describing the strengths and weaknesses of various methodologies, such as The Sedona Conference Best Practices, supra, n. 9, and select the one that they believe is most appropriate for its intended task. Should their selection be challenged by their adversary, and the court be called upon to make a ruling, then they should expect to support their position with affidavits or other equivalent information from persons with the requisite qualifications and experience, based on sufficient facts or data and using reliable principles or methodology. [emphasis added].

Victor Stanley shows that keyword searching can be defensible.  Reports of its death may be greatly exaggerated.

For a discussion of the use of keyword searches in combination with TAR, see Craig Ball, Train, Don’t Cull, Using Keywords | Ball in your Court (craigball.net) (Aug. 5, 2012).

History is important.

_____

[1] See, e.g., 2021 WL 4641586 (Oct. 7, 2021), and 269 F.R.D. 497 (D. Md. 2010), among other later decisions. The latter is famous for describing the “gang that couldn’t spoliate straight.” The Court also relied on Paul W. Grimm, Michael D. Berman, Conor R. Crowley, Leslie Wharton, “Proportionality in the Post–Hoc Analysis of Pre–Litigation Preservation Decisions,” 37 U. Balt. L.Rev. 381, 388 (2008).