looming silos

That people like me write about siloed systems, and that you are reading this suggests two things: 1. there's a lot of them out there, and 2. dealing with them can be a real challenge.

The very term reeks of pejorative and is normally considered a way of criticizing decisions involved in systems approaches and implementations across an organization. The implication is that silos must be “fixed” if things are to work properly. Admitting that you have siloed systems is tantamount to admitting that somehow you didn’t do your automation correctly.

Siloed systems, in the current view, come from siloed thinking in which each major component of an organization acts to solve its own technology problems with systems that do not easily (or at all) communicate with their counterparts across the organization. We should remember, however, that many environments viewed as siloed today wouldn’t have been viewed that way when they were developed only a few years ago.

Most organizations are designed to distribute responsibility via a hierarchy, so that each part of the organization is responsible and rewarded for meeting its own needs with its own staff and its own budget, which creates a strong incentive for it to act in its own interest. The result is systems and technology that meet each group’s particular needs (often quite well), but often without considering how they will play as part of the enterprise.

It would be great if the entire organization could create a highly interoperable and transparent environment in which all components share information and support each other’s functionality … but that’s rare. We’ll use the term interoperable instead of “integrated” because actually integrating disparate systems has turned out to be ephemeral by any measure.

Criticisms of siloed systems usually fail to take into account the dynamics and history of the involved organization and its components. Each of those hated silos may be an effective approach to its owner’s requirements and that’s the way those owners view them.

Criticize them from the outside as “siloed,” and you may find yourself in a war.

How to Fix Them? Don't Try to 'Fix' Them

The easiest way to start down the path toward interoperability is to start from scratch: involve the entire organization in the effort, selecting only systems and technology designed to interoperate. But that seldom happens.

Another approach is the mandate from on high that proclaims a particular system environment will become the standard across the organization, forcing everyone using something else to re-tool. But that nearly always creates chaos with the result that things don’t get better, but rather get a lot worse.

So if we can’t start over at the system and technology level, how can we create an interoperable environment from those entrenched, and often passionately defended, silos?

What We Have Here is a Failure to Communicate

The answer, while not easy, is pretty straightforward: find a way for the systems to talk to one another using an interchange content vehicle. This avoids the need for radical surgery on systems environments. 

Easy? No, but in today’s technology-intensive world, almost always possible.

Every modern system provides tools to import and export data in various formats. While they may not all provide the same level of capability, once you can get the data you want out of a system, you can transform it into the format you need. Similarly, not all systems can import data in the same way, but you can probably transform the exported data into a format for other systems to import. Backing up these internal capabilities is a robust transformation tool industry to support the export and import tasks.

So what is this standard interchange vehicle? While there are differing opinions, my vote is for XML (eXtensible Markup Language), the international standard first published in 1996 as a simplification of SGML (Standard Generalized Markup Language) dating back to the early '70s. JSON (JavaScript Object Notation,) a newer and more condensed notation that is easy to program but lacks some of the richness of XML, is also a candidate. 

Call me old fashioned, I prefer XML.

XML is everywhere: Microsoft Office tools use it as their content format, XHTML on the Web is built on it, Amazon uses it as the interchange vehicle with their thousands of vendors, and even Walmart offers it as an optional interchange format, the other being JSON.

So confronting your field of silos, you can think through what intercommunication is needed to make the environment work as one, design an interchange form in your chosen standard and develop an architecture that lets each system export what is asked of it and get what it needs from its peers.

But while the vehicle is important, the human and organizational setting is equally so. If you can’t engender a level of shared ownership and cooperation among the involved — and impacted — organizations, no matter how elegant your approach, you won’t get far.

The Content Hub Connects the Silos

Once you have decided on your interchange vehicle, you get to confront the other head of the silo monster: how to establish communications among all those individual systems without ending up in a mass of content spaghetti.

For that, we can take a lesson from Fred Smith, the founder of FedEx and father of hub-based delivery systems. As an undergraduate at Yale, Smith wrote a paper suggesting that delivery be organized like a telephone exchange: all callers connected to a central office that could make the desired connections on the output side. He got a C on that paper but revolutionized delivery. Packages arrive at a FedEx hub each night, then are redirected, reloaded and shipped to their destination the next morning in a single, integrated flow.

Taking our cue from FedEx, we can establish a content hub using the exported content received from each system, making it searchable and letting each system get what it needs in return.

The Devil’s in the Doing

So how do you move down the content interchange path?

Start by getting all the players in a room. Lay out both the challenges they face and your commitment to leave them alone to the maximum extent possible. While this may seem to go without saying, it is potentially the most difficult step and neglecting it is often the cause of failure.

When those silo managers sit down, they will already have heard rumors that you are about to turn their world upside down. Some will come ready to fight, and if they leave that way, you are in trouble. You must convince them that while there will be an impact, it will be limited to the export and import of data to and from a content hub you will set up (and they won’t have to pay for).

Make a clear case of how they will benefit from what you are proposing. If there’s nothing in it for them, the best you will get is neglect, and not necessarily the benign variety.

Once you've received the green light from the stakeholders, you can marshal the resources you need to build your interoperable environment, including representatives from the organizations whose systems you will be connecting. From there, the effort becomes largely technology-based, managed under whatever protocol the organization is most familiar and comfortable with. My own preference is Concurrent Engineering, but there are plenty of options.

What you will be attempting won’t be a walk in the park, and it won’t happen overnight, but it will work if you give it the time and attention it needs to be carefully designed, fully developed and phased in at a rate the organization can sustain.

As any farmer can tell you, in the right setting, silos aren't all that bad.

Creative Commons Creative Commons Attribution 2.0 Generic License Title image  by  anthony arrigo