High Volume Document Storage Creates New Headaches for Content Managers

By Steve Jones Oct 26, 2009 Comments

1 comment

Corporations produce millions of customer-facing documents every month and store them in document management software for subsequent online access and records management compliance. The ramifications on storage infrastructures are massive and continue to grow exponentially.

It is evident that enterprise storage needs have extended well beyond traditional compression solutions. And single-instance storage offers a viable option for organizations looking to drive down the costs of the physical storage requirements. In this guest article, Steve Jones from Xenos Group explains how.

Single Instance Storage Offers Huge Savings Opportunity

Each month, millions of high volume customer-facing documents — such as statements, invoices, and bills — are produced for mailing or online presentment. This wealth of documentation is generated by a huge and complex infrastructure of corporate back office applications and composition engines on a recurring basis.

In most cases, these documents are stored in an Enterprise Content Management (ECM) system with individual PDFs quite often exceeding 1MB in size.The result is that millions of documents with remarkably similar components are stored over and over again, putting a rapidly escalating strain on storage requirements and costs.

While this way of storing individual documents prevails, organizations are now looking at the concept of single instancing to apply to their high volume transaction output (HVTO) needs, to drive down storage costs and improve overall efficiency. In fact single instance storage (SIS) represents the single largest operational cost reduction opportunity for budget constrained IT departments.

Single Instance Storage Fundamentals

Single instancing is based on the principle of keeping one copy of content that multiple users share. This can include design elements (headers, logos, etc.) as well as marketing and advertising messaging — all components that account for a majority of the storage requirements for any given document.

By eliminating duplication across a variety of storage-related solutions (file systems, data backup and e-mail servers for example), SIS is playing a significant role in helping organizations manage the explosion in data and document volumes — and the skyrocketing costs associated with the physical storage requirements to support this increase.

High Volume Transaction Output = New Challenges

When talking HVTO, many of today’s single instancing solutions are not optimized to deal with this particular area of concern for enterprises. Most are designed to reduce the storage needs for office documents such as Word, Excel or PowerPoint, or music wav files or movie AVIs, or email attachments. HVTO documents are a different matter.

HVTO documents are produced by many of the largest companies in the world, month in and month out. Traditionally, they are stored in ECM systems or what used to be known as IDARS or COLD solutions.

Available solutions for optimizing storage resources for these types of documents have typically been compression type applications. However, these fall far short of the mark, especially when dealing with the newer, more contemporary, high-resolution, graphically-rich customer-facing content now being produced by today’s corporate marketing departments.

If one were to look at the way in which high volume documents are created and the make-up of those documents however, it becomes very apparent that a single instancing approach can be applied.

HVTO Documents Present Unique Opportunities

High volume documents are produced repeatedly in massive batches (hundreds of thousands or millions of documents) by corporate applications and composition engines. Documents such as a credit card statement, an electrical utility bill, or an insurance explanation of benefits are produced every billing period and mailed or made available online through corporate self-service channels.