E-Discovery Insights – Clearwell Systems, Inc.

Automated Review in Electronic Discovery Re-Visited

by Dean Gonsowskion June 28th, 2010

Almost two years ago I wrote one of my first blog posts entitled
“Review-less E-Discovery Review.” Despite the tongue twister
of a title, the post posited that “there is a very real possibility
that we’re on the cusp of computers taking over a significant e-
discovery task for attorneys.” I’d like to take a look and see
how much (if at all) my prognostications have materialized.

A cynic might think that this is the moment where E-Discovery

2.0 jumps the shark. But no, this isn’t one of those sitcom
episodes where they flashback to previous shows as an easy
way to recycle content. Instead, it seems useful to see how the
legal E-Discovery market has evolved from a litigation workflow
perspective, particularly with some vendors touting the
benefits of review-less technologies like predictive coding.

In the original blog, I noted that there was a “scenario where a

non-manual review methodology may make sense” (while importantly noting that “this
approach is not without risk”). Since my last post there has been the successful adoption of
Evidence Rule 502,which makes this methodology (at least conceptually) safer.

But again (imagine dreamy flashback mode), here were the guidelines I previously proffered:

1. Large data set. This may sound a bit obvious, but a non-manual approach is best suited for
large, unwieldy data sets. The corpus doesn’t need to be in the terabytes, but the data set
should be evaluated in term of electronic discovery processing costs and attorney review estimates.
2. Short Production Timelines. Once the above calculations are conducted, the next step is to
determine if a human based review could even conceivably be conducted in the given time
frame. In many instances, an eyes-on review process just won’t be feasible since there won’t be
enough bodies to throw at the problem.
3. Next Gen “PAR” Tools. In order to pull this “review-less” review process off, both safely and
quickly, the responding party needs to have access to fast, robust processing, analysis
and review (“PAR”) tools. Certainly, it’s possible to have this scenario work with an e-discovery
service provider, if they have the capability.
4. Relatively Small Amount in Controversy. For the time being, this approach should not be
considered for any “bet the company” ediscovery litigation support, nor anything with significant downside risk
(governmental inquiries, punitive damages, class actions, 2nd requests, etc.). Yet, for many
standard commercial lawsuits, corporate investigations, HR claims, etc. this review-less
approach may be worth considering.
5. Ability to Use a Clawback Provision. Entering into a clawback provision with the opposition is
mandatory in this methodology since the chances of an inadvertent production are statistically

To read more visit www.clearwellsystems.com/e-discovery-blog/ 1

E-Discovery Insights – Clearwell Systems, Inc.

ever-present. Yet, until Evidence Rule 502 is resolved, there will always be a risk that the
clawback won’t be enforceable against 3rd parties.
6. Non-governmental Production. Most information in governmental productions becomes part
of the public record, meaning that a clawback isn’t going to be feasible. Here, trade secret
information, personally identifiably data and the like would be disastrous if pushed out into the
public domain.

The goal of this post is to see if this dog is any more ready to hunt than it was two years ago.
The short answer (right now) appears to be: No.

We all know that litigators are both risk adverse and generally slow to adopt new technology
approaches. This is particularly true when there’s a perception that they won’t have insight
into the technological black box behind automated coding/tagging decisions. Litigators are
understandably sensitive about the ability to prove up the reasonability of their search and
review processes. This “reasonableness” requirement lines up both with the Victor Stanley
requirements and FRE 50(b), which eliminates the chance of a waiver only “if the holder of the
privilege or work product protection took reasonable precautions to prevent disclosure.”

Given this ongoing hesitancy, the question remains shouldn’t we be seeing more movement in
automated review than the glacial progress that’s been achieved to date, particularly with the
known shortcomings of the eyes-on review process? Most are familiar with the 1985 STAIRS
study by Blair and Marion where the percentage of relevant documents lawyers thought they
had found using Boolean Keyword searches was 75% – when the percentage they actually
found was 20%.

But, despite the known deficiencies of eyes-on review e-discovery it follows into the “go with the devil you
know” mindset that often makes sense when dealing with judges and juries who aren’t likely to
grok newer-fangled approaches.

In addition to these high-level, almost dogmatic challenges, there is one other tactical element
I’d add to my previous list (of 6 factors).

7. All documents processed up-front (no rolling collection). I’ve heard some in the trenches e-
discovery experts claim that they’ve never had a case that didn’t involve at least some level of
incremental data collections. Whether this is an overstatement is immaterial. The fact is that a
large number of e-discovery projects involve ESI that is collected (and then processed) in dribs
and drabs. This if often a good thing, largely attributable to the incremental (start slowly)
nature of a well thought out e-discovery project where a smaller number of initial custodians
are processed, then ECA is conducted and only then is the additional ESI added to the corpus.
This common methodology causes some significant heartburn for a review-less methodology
since the ever changing nature of the corpus makes it difficult/impossible for a sample to be
truly extensible to what will eventually be the entire data set. For this reason, the review-less
approach should be limited to where the entire corpus is collected and processed at once.

To read more visit www.clearwellsystems.com/e-discovery-blog/ 2

E-Discovery Insights – Clearwell Systems, Inc.

In sum, the seven foregoing factors appear to still be largely valid and create an environment
where an automated, review-less methodology will only make sense in a relatively rare set of
circumstances. This may change in the future, but given the risk adverse DNA of most litigators
I can’t imagine this tipping point happening any time soon.

To read more visit www.clearwellsystems.com/e-discovery-blog/ 3