Corporations produce millions of customer-facing documents every month and store them in document management software for subsequent online access and records management compliance. The ramifications on storage infrastructures are massive and continue to grow exponentially.

It is evident that enterprise storage needs have extended well beyond traditional compression solutions. And single-instance storage offers a viable option for organizations looking to drive down the costs of the physical storage requirements. In this guest article, Steve Jones from Xenos Group explains how.

Single Instance Storage Offers Huge Savings Opportunity

Each month, millions of high volume customer-facing documents -- such as statements, invoices, and bills -- are produced for mailing or online presentment. This wealth of documentation is generated by a huge and complex infrastructure of corporate back office applications and composition engines on a recurring basis.

In most cases, these documents are stored in an Enterprise Content Management (ECM) system with individual PDFs quite often exceeding 1MB in size.The result is that millions of documents with remarkably similar components are stored over and over again, putting a rapidly escalating strain on storage requirements and costs.

While this way of storing individual documents prevails, organizations are now looking at the concept of single instancing to apply to their high volume transaction output (HVTO) needs, to drive down storage costs and improve overall efficiency. In fact single instance storage (SIS) represents the single largest operational cost reduction opportunity for budget constrained IT departments.

Single Instance Storage Fundamentals

Single instancing is based on the principle of keeping one copy of content that multiple users share. This can include design elements (headers, logos, etc.) as well as marketing and advertising messaging -- all components that account for a majority of the storage requirements for any given document.

By eliminating duplication across a variety of storage-related solutions (file systems, data backup and e-mail servers for example), SIS is playing a significant role in helping organizations manage the explosion in data and document volumes -- and the skyrocketing costs associated with the physical storage requirements to support this increase.

High Volume Transaction Output = New Challenges

When talking HVTO, many of today’s single instancing solutions are not optimized to deal with this particular area of concern for enterprises. Most are designed to reduce the storage needs for office documents such as Word, Excel or PowerPoint, or music wav files or movie AVIs, or email attachments. HVTO documents are a different matter.

HVTO documents are produced by many of the largest companies in the world, month in and month out. Traditionally, they are stored in ECM systems or what used to be known as IDARS or COLD solutions.

Available solutions for optimizing storage resources for these types of documents have typically been compression type applications. However, these fall far short of the mark, especially when dealing with the newer, more contemporary, high-resolution, graphically-rich customer-facing content now being produced by today’s corporate marketing departments.

If one were to look at the way in which high volume documents are created and the make-up of those documents however, it becomes very apparent that a single instancing approach can be applied.

HVTO Documents Present Unique Opportunities

High volume documents are produced repeatedly in massive batches (hundreds of thousands or millions of documents) by corporate applications and composition engines. Documents such as a credit card statement, an electrical utility bill, or an insurance explanation of benefits are produced every billing period and mailed or made available online through corporate self-service channels.

The look and feel of each document in each batch is identical. The only thing that differs from one document to the next is the individual transactional data within the body of the document, such as banking line items, phone call lists, automobile information, or usage statistics.

Most of the Data is Redundant

The common composition elements such as the branding, the graphics, the fonts, the overlays, the marketing messages on the other hand, are identical across every single document. If one considers that a typical graphically-rich document can be 1MB versus 15Kb to 80Kb for a text-based document, it is easy to understand the strain newer electronic formats are placing on storage requirements.

Yet, as mentioned earlier, these common composition components, which can compromise 90% of the total document size, are stored electronically over and over and over again inside the document management solution, for every single customer. This has become an all-too familiar story -- and costly IT headache -- for many enterprises today.

Records Management Compounds the Problem

Given that large corporations produce millions of these customer-facing documents every month and store them for online access, the ramifications on storage infrastructures are massive and continue to grow exponentially. The problem compounds itself with current records retention requirements that demand documents be stored anywhere from 13 months to seven years and beyond.

Down to the Brass Tacks: The Cost of Storage

To get a sense of how much of a strain the storage requirements place on IT budgets, consider that the cost for sustained digital storage ranges from $20 to $30 per GB per month. Doing the math, it’s not surprising that large enterprises today are spending as much as 45% of their IT budgets on their storage infrastructures alone. Since data volumes are expected to increase tenfold from 2006 levels by 2011, enterprises must re-evaluate their HVTO needs and look for effective ways to lower related costs.

When it comes to HVTO, single instancing is a relatively simple concept -- in theory. The key is to leverage technology that can intelligently separate the common composition resources from the unique transactional content that makes up an HVTO document. In doing this, only one single copy of the “common resources” is stored for any given batch of documents.

Pointers are inserted within the transactional document that link it to the composition resources. When the document is retrieved, the two pieces are integrated in near real-time to present a perfectly reconstituted, complete document exactly as was originally produced.

Determining HVTO storage costs and potential savings involves a simple calculation: Multiply the number of documents in a batch by the average size, by the number of times the batch is run each month. Then multiply the total by the average storage costs ($20 to $30 per GB). Finally, multiply the storage costs by the percentage reduction that HVTO SIS can achieve to receive the total savings.

For example: 100,000 PDF documents x 750Kb x 22 days x .000025 per Kb = $41,250 per month storage cost. Reducing those requirements by 80% (which is a conservative number) can deliver a monthly savings of $33,000 for those 100,000 documents.

HVTO is an issue that global organizations have had to contend with for years. At the same time, exponential growth in data and storage requirements have driven many to the breaking point in terms of what they can manage feasibly. It is evident that enterprise storage needs have extended well beyond traditional compression solutions. As a result, more and more businesses are looking to single instancing as a means to significantly reduce storage demands -- and free up budget resources for other IT projects.