What Does Counsel Really Think About Predictive Coding?

5 minute read
Sophie Ross avatar

Predictive coding for e-discovery
While the promise of predictive coding is alluring, many questions remain for corporations and law firms: Where does the software end and the importance of workflow begin? Are companies using it successfully? Is the technology well-suited for all types of legal matters?

Predictive coding has sparked considerable discussion among e-Discovery practitioners. From conference panels to blog posts, attorneys debate the extent to which it will reduce the role of human reviewers and question its capacity to defensibly automate and reduce the cost of e-Discovery.

To make sense of this dialog, FTI Technology recently conducted a survey of law firm leaders and senior corporate counsel that identifies key trends and perspectives on the emergence of predictive coding. Thirteen in-house counsel from Fortune 1000 companies and 11 Am Law 200 (American Lawyer) law firm partners and senior counsel were interviewed. More than half of the respondents have used some type of predictive coding technology.

Respondents were asked a wide range of questions relating to predictive coding, including their thoughts on high-profile court rulings, cost savings estimates and adoption inhibitors. The feedback reveals widespread interest in the technology, evidence of successful predictive coding pilot projects and optimism about its ability to reduce e-Discovery costs in the long term. However, skepticism and uncertainty remains.

Among all of the survey responses, a number of recurring themes emerged:

Lawyers are trying predictive coding with some promising results

The survey indicates that predictive coding may not be ready for “prime time” use but there are positive signs for future adoption. While 54% of participants reported use of predictive coding, the majority of these appear to be through pilot projects with predictive coding vendors. Of the 54% that have used predictive coding, 91% of them described the project as successful, for a variety of reasons. One respondent explained how predictive coding helped on an internal investigation in which 1.6 million documents needed to be reviewed in three weeks. For those that have yet to use it, many are considering pilot programs and anticipate making a decision about its use in the coming year.

For most, predictive coding really means fast culling and prioritizing

Predictive coding is not a silver bullet for automatically coding an entire data set. In fact, very few respondents indicated they use the technology for actual document coding. The majority discussed using predictive coding technology as advanced keyword search functionality so that reviewers could narrow in on important materials faster. As a result, several mentioned the ability for predictive coding to support early case assessments (ECA).

The verdict is still out on cost, as well as potential savings.

Perhaps because the majority of predictive coding projects were conducted on a trial basis, respondents had a hard time quantifying the costs on a real-world matter, or even potential cost savings. In fact, about a quarter of those using predictive coding could not assess the costs. Only one respondent was able to assess the organization’s savings in terms of an exact figure (US$ 580,000). In lieu of calculations, some simply remarked that they must have achieved cost savings.

Learning Opportunities

"Garbage in, garbage out”

Much of the dialogue surrounding predictive coding relates to its potential for eliminating or reducing human review, the most expensive e-Discovery phase. However, with predictive coding, the respondents overwhelmingly agreed that humans become an even more important part of the process since they determine the reference set, test and refine the software, and conduct additional quality control to ensure defensibility. Quite simply, the quality of the predictive coding results is dependent upon the quality of the predictive coding input, as the software will propagate bad coding decisions as well as good coding decisions.

Some matters are better suited to predictive coding than others

Although predictive coding is not necessarily designed for a specific type of matter or case size, 88% of respondents agreed there are certain parameters that may make its usage successful. To provide a bright line, the participants generally agree that the minimum number of documents for which it is appropriate is 100,000.

Among survey respondents, top ranked benefits of predictive coding include the ability to prioritize documents to be reviewed by humans, cost-effectively eliminate irrelevant documents, test the results of human review to ensure accuracy of coding decisions and find more responsive documents. Top ranked concerns include the notion that some predictive coding solutions may be somewhat “black box” and difficult to explain in court. Others highlight that it might not be well-suited for investigations or finding the “needle in the haystack.” Because predictive coding is just as much a process as a technology, there is widespread acknowledgement that experts are an integral part of the process.

The survey results show corporations and law firms are taking a measured and pragmatic approach to the adoption of predictive coding technology and workflow. Pilot projects, a reliance on “techno-lawyer” experts, as well as a focus on the types of matters best suited for predictive coding, demonstrate a keen understanding of the technology’s promise and potential challenges. As organizations transition from pilot programs to regular use, the industry as a whole will benefit from broader focus on and awareness of these potential issues, especially around cost calculations and defensibility.

Title image courtesy of Maksim Kabakou (Shutterstock)

Editor's Note: To read more on e-Discovery issues, see Richard Medina's How to Develop and Implement Your Discovery Readiness Program

About the author

Sophie Ross

Sophie Ross is a senior managing director in the FTI Technology practice and is based in San Francisco. Ms.