Information Management, SharePoint Large Enterprise ECM
The concept of Enterprise Content Management (ECM) can be traced back to the 70’s and 80’s when the use of mainframe, midrange and desktop computer systems was in its infancy. These primitive systems quickly gained traction largely due to their ability to make business processes run more efficiently and cost effectively.

Throughout the evolution of these systems, ECM has been a significant driver for technological innovation. That trend continues today as Microsoft SharePoint is quickly becoming the most prolific Enterprise CMS in history.

The Association for Information and Image Management (AIIM) defines ECM as follows:

Enterprise Content Management (ECM) is the strategies, methods and tools used to capture, manage, store, preserve, and deliver content and documents related to organizational processes. ECM tools and strategies allow the management of an organization's unstructured information, wherever that information exists.”

Admittedly, spending an early paragraph on the definition of ECM is a pretty boring way to kick things off, but it’s important to understand exactly what ECM is in order to really grasp how relevant SharePoint has become in the ECM space.

SharePoint has a History of Adaptation

SharePoint is not a new product. It has been massaged and refined over the last 15 years or so into the stable product that it is today. It is important to know the history of SharePoint in order to understand how successfully it has evolved with the changing landscape of Information Technology. Large organizations have to be confident that a platform can be flexible as corporate needs change over the years.

The ancestry of SharePoint goes back to the 1990’s. The primitive core technology can be traced all the way back to a product called FrontPage and an underlying technology called FrontPage Server Extensions developed by a company called Vermeer Technologies Incorporated. Those of us familiar with the SharePoint programming interfaces may recognize the “_vti_bin” location for calling SharePoint web services where the “VTI” reflects the name of the technology creator. Recognizing the incredible potential for this powerful web technology, Microsoft acquired VTI in 1996.

The functional ability of FrontPage to surface dynamic web content was blended with another Microsoft product called Site Server to provide primitive corporate intranet portal capabilities. With the real value of the solution now being evident, Microsoft focused more on delivering a product that could help organizations rapidly stand up these intranet portals with the release of SharePoint Portal Server 2001. This version of the product would improve out-of-the-box navigation and search capabilities that really brought the platform to life. The ECM nature of SharePoint was rapidly evolving.

In 2003, Microsoft released Windows SharePoint Services 2.0. With it came modularized server side components called “web parts” and an improved security model. More importantly, the site collection and library approach to storing content was employed to bring powerful new content management capabilities to SharePoint. But there were still some shortcomings. Users couldn’t perform metadata searches across sites or libraries. There were also significant out-of-the-box limitations with site and library scalability.

With WSS 3.0 and Microsoft Office SharePoint Server 2007 (MOSS), many of the functional performance limitations were addressed and the capabilities of the platform increased by an order of magnitude. User controlled list and library indexes would improve view performance. An improved Search Service would allow cross-farm metadata based searches based on managed properties. A Windows Workflow Foundation (WinWF) based workflow engine became part of the core infrastructure.

Perhaps most importantly, the concept of Content Types to classify content by specific metadata along with other document publishing features provided the basic infrastructure for the true document management features of an ECM system. Additionally, the web content management (WCM) feature functions of Microsoft Content Management Server (CMS) were blended into MOSS to provide a unified product that would serve all of the collaboration and portal needs of an organization.

Even with the significant improvements in MOSS 2007, there was still a lot of work to be done. There was a significant limitation that impeded the performance and fault tolerance capabilities of SharePoint. This issue was the search subsystem. With MOSS, the platform supported only a single search index server. In addition to severely limiting scalability, fault tolerance was not possible.

With SharePoint Server 2010, this issue was resolved. Multiple index and query servers would allow the Enterprise Search subsystem to scale to support 100 million documents. Additional ECM feature functions included a farm level Managed Metadata service that would facilitate global content types and a centralized term store for organizational specific metadata terms.

Late in the SP2010 product development cycle, Microsoft acquired another company called FAST Search and Transfer. Their product, FAST ESP, was an industrial strength search platform that could scale to incredible size. FAST was not acquired in time to be integrated into the core search codebase, but there was time to build a bypass solution into the product.

By standing up a FAST Search Server for SharePoint 2010 farm and enabling the search bypass to take advantage of the FAST farm, SharePoint 2010 could expose an enriched search experience and a significant improvement in scalability. A FAST enabled SharePoint 2010 farm could support up to 500 million documents, far more scalability than most large organizations would ever need.

That brings us to the current platform version, SharePoint Server 2013. Architecturally, much of the infrastructure is similar to SharePoint Server 2010 with one significant exception. Much of the search subsystem has been re-architected to include the FAST core code. This means that all of the vastly superior capabilities of the FAST feature set such as improved linguistics handling, entity extraction, and the scalable/fault tolerant server topology are now all baked right into the core of SharePoint Server 2013.

There are many other improvements in SharePoint 2013, including:

  • Continued improvements in the claims based authentication model allowing the inclusion of security trimmed search results to be returned to the end user from a remote SharePoint result source.
  • Workflow improvements including new ways to visually create processes.
  • A continuing trend to cloud based architecture and a new application deployment model that enhanced the way custom solutions can be bolted onto SharePoint. In addition to still being able to add components using the legacy solution deployment model, a new “App Store” type deployment model gives SharePoint administrators full control over applications that authorized users can deploy.
  • Significant improvements to mobile content delivery. In addition to a vastly improved mobile browser viewing experience, industry standardized client side development models allow organizations and vendors to easily build mobile applications that interface with SharePoint 2013.

The key takeaway from the history of SharePoint is that it is here to play and it is here to stay. Aside from the fact that SharePoint has proven to be flexible with the wide variety of organizational needs over the last decade, there are still several other important reasons that large enterprises should consider SharePoint for their ECM platform.

SharePoint is a Framework

SharePoint has always been designed to be the central portal location that all members of an organization can turn to when they need to find the information that allows them to do their job. Unfortunately, there are some organizations that don’t give SharePoint the proper due diligence with regard to solution development and governance. For these organizations, SharePoint becomes a glorified file share. But with a little love and attention and then some regular care and feeding, SharePoint can be a powerful ally in the war of efficiency in the workplace.

Of course, SharePoint offers amazing collaborative and social capabilities out of the box. Before everyone knew what SharePoint could do, they would just stand up a site collection for every department and tell their users to just use SharePoint for all their content. Sure this is a good start, but we can do more. With a little planning, SharePoint can be customized and extended to be so much more than just the base out-of-the-box experience:

  • We can customize the libraries so that when users add documents, they must add metadata that can later be searched for “exact relevance” results or to drive faceted navigation.
  • We can completely redesign the interface to seamlessly align with the corporate brand.
  • We can use SharePoint as a foundation for a line of business system. For example, with a bit of custom workflow and the configuration of a records center, we can construct a comprehensive and compliant records management solution. A more specific example might be an Accounts Payable solution with a SharePoint content management foundation. By bolting on a vendor solution for content ingestion, GL coding, and business specific workflow to a secured SharePoint site, we can improve AP efficiency and, as a result, increase cash flow into the coffers and improve profit margins.
  • With the web content management capabilities of SharePoint, we can leverage the available Application Programming Interfaces (APIs) to construct a scalable, geo-distributed, and fault tolerant public facing internet presence.
  • Using the robust and flexible SharePoint development and deployment platform, virtually any Line of Business (LOB) system can be customized for an organization.

The SharePoint platform does all the heavy lifting related to content storage, content security, authorization and authentication. The included user interface also provides a powerful foundation for exposing content to end users without a single line of code. But the flexibility is there to bend, stretch, mold and otherwise enhance a solution that is much more than the sum of the out-of-the-box parts.

SharePoint has a Healthy Ecosystem

Perhaps one of the biggest reasons to consider SharePoint for ECM is the fact that you can always count on support from the SharePoint ecosystem. There are literally millions of .NET and web developers out there who could potentially be a resource for maintaining or developing a SharePoint solution. There are thousands of Independent Software Vendor (ISV,) Systems Integrator (SI) and Value Added Reseller (VAR) companies out there to help you design and deploy a SharePoint solution that meets your needs.

These types of vendors have been mentioned a few times in this article because they are key to extending the base capabilities of SharePoint. Of course, as with any service organization or product vendor, some are better than others and all must be properly vetted. But the important point here is that there is an ocean of knowledge out there that can help you get your solution off the launch pad.

More importantly, in three, five or 10 years when it’s time to upgrade that system, there will still be plenty of resources that can facilitate the next evolution of your system.

SharePoint is Flexible and Scalable

There is no such thing as a perfect software solution. It’s important to set that expectation early and often with the folks that write the checks or there will always be dissatisfaction. Software solutions are typically targeted at either a vertical industry or at horizontal integration.

Vertical software solutions are specifically designed to accomplish a narrow set of tasks. They are usually very good at doing exactly that, no more and no less. They are not flexible enough to do anything else. Vertical solutions can be very cost prohibitive over time due to the narrow specialization resulting in high maintenance costs. When money is no object and features/performance are the top priority, then vertical solutions may be the best answer. But more often, an organization needs to get more value for the dollar spent. That’s where horizontal software solutions come into play.

Horizontal integration type software tends to have features that are less targeted to a specific industry and more targeted to common feature requirements across all industries. SharePoint fits into this category by laying the foundational features for all verticals while exposing integration and extension capabilities for vertical specialization. This is where custom development and third party vendors come in. When an organization chooses SharePoint as the central portal technology, they enable other vertical solution vendors to take advantage of the shared infrastructure in order to provide a solution.

Horizontal flexibility does come at a cost though. With flexibility comes complexity and with complexity comes scalable boundaries. There are limits to what SharePoint can do because it does so much for so many use cases. That said, SharePoint can scale to hundreds of millions of documents with proper architecture. This takes care of an extremely high percentage of the small, medium and even large organizational needs. There will occasionally be extreme situations where hundreds of millions of documents is not enough, but the flexibility of the SharePoint platform will be more valuable to most organizations.

You Probably Already Own It

One of the biggest reasons to consider using SharePoint as an ECM platform is the fact that most large enterprises already have licenses for it! These days Microsoft is big into Enterprise Agreements for licensing. Most, if not all, of these agreements bundle in SharePoint CALs along with the Windows, Windows Server, SQL Server and Office licensing.

If an enterprise already has licensing in place for the centralized ECM platform, it is far more cost effective to use a common platform for multiple functional solutions and then purchase the additional components needed for a specific functional purpose. Overall implementation cost will be lower and the maintenance costs for the custom portion of the overall solution will also be lower.

The SharePoint platform is a living, breathing and evolving entity. It is a flexible and scalable framework that can be used as a base infrastructure that can be extended in a cost effective way to meet the needs of any enterprise.

Title image courtesy of Everett Collection (Shutterstock)

Editor's Note: Interested in reading more? Here's an argument for SharePoint as Web CMS: SharePoint 2013 Consolidates Its Position as a Mainstream Web CMS Player