statue with head in hand
Enterprise search comes with a whole raft of different issues than public facing search PHOTO: threephin

When you need to find a corporate policy, or information about your health plan or the latest contract for one of your clients, is your enterprise search box the first place you go to look? 

Or do you count on someone in your department or on your team who knows where everything is?

Unfortunately, the latter is much more common.

The good news is search can be fixed. It will take some initial effort and ongoing discipline, but great search is not rocket science.

The First Problem With Search: Staffing

Like most other enterprise software, search simply takes good ongoing operational methodology. Enterprise search is not a "fire and forget" technology — it takes effort to ace. Why? And what can you do about it?

Let’s start with staffing.

If your organization is like most, you have a database management system (DBMS) in-house. How many developers and operations folks are on the DBMS team? 

Most sources report staffing as a function of how many databases are in use, but in general, having five staff members responsible for the ongoing DBMS operations is the minimum — which doesn’t include IT tasks such as keeping the hardware running.

You probably have an SEO team, responsible for checking public web page views, Google keywords and traffic, and optimizing results either by ‘pay per click’ or by tweaking your content with custom keywords. 

Yet in many organizations, search has no real owner. Some organizations have one IT person whose responsibilities include keeping search software running, which may or may not include insuring that index updates run. Very rarely does the role involve confirming all content was indexed, and it most certainly doesn’t include responsibility for tuning any relevance.

Larger organizations may have a slightly larger staff that includes a "business owner," an "operations/production" person and possibly a "reporting" owner. 

But most of these roles are part time. Search gets attention when something breaks, and even then, only when everything else is running smoothly. Rarely do I see organizations staff the role of improving search relevancy, or coaching content authors on how to best use keyword fields in office and PDF documents to improve findability.

Well-run search needs at least a small team who is responsible for day to day operations, analytics and updates, as well as evangelizing search to the user community and proving training and support: basically, an internal search center of excellence. 

The Complications of Enterprise Search

Search within the organization is tough. Employees will complain that your search doesn't work as Google does, and they're right, it doesn't — for a number of reasons.

Versioning and Near Duplicates

Consider the file shares you personally use that are included in the internal search index. You probably have several copies of almost every document and even more near duplicates.

For example, as I write this article, I have saved three versions so far: articleV1, articleV2 and articleV3. I try to save the final copy in a different completed directory, and to remove or at least copy the incremental versions to an external drive that does not get indexed — but to be honest, that’s hard to do regularly.

File Formats

Another issue is the variety of file formats in the enterprise. They far exceed the document types we normally see on the internet. Enterprises have the usual HTML, PDF and Office document formats. But they also have database content from call and problem tracking databases as well as older file formats, such as WordStar and WordPerfect, and even email servers. 

Enterprises also use several specialized search clients such as bug tracking and customer contact databases as well.

Metadata

Another problem with enterprise content is metadata.

When adding content to your public website, a team assigns keyword metadata to help Google and other internet search engines find and crawl it. 

When you create a new document with Microsoft Word, it assigns a default document name — Document1 — and it pre-populates the Author field with my name, which I provided when I installed Word. The bad news is, like me, most people don’t have the discipline to add meaningful content in the Properties.

I heard an anecdote once about a large company that wanted to reward support employees who were actively creating application notes. The management team decided to use their internal search technology to discover and reward the most prolific content creators simply by searching the Author field across all app notes.

They identified one employee who created over 95 percent of all app notes: the administrative assistant for the support team. Which was bad — or technically incorrect — metadata. 

Can you guess the problem?

The admin had created the template that all of the support staff used for their app notes. As a result, the admin’s name was in the Author field, and virtually none of the support staff updated the metadata. Similarly, Document1 is a top file name, as is Workbook1. Check your site to see if you see the same behavior.

Security

Enterprise search has another characteristic that pubic search engines typically do not — security.

And security is often critical to corporations: exposing employees’ evaluations or salaries can be embarrassing. Exposing decisions regarding an acquisition prior to its public announcement could put the company at financial risk.

Take a Deep Breath and Fix Before You Buy

If you’re using enterprise search now, and you and your users are happy with it, congratulations — you’re doing something right. Keep up the great work!

If you are responsible for enterprise search and it’s just not working, take a deep breath. 

Your first reaction — and that of your management — may be to throw it out and replace it. Fight that urge. My experience has been that bad search is often a result of bad methodologies. And changing your approach to improve search is much less expensive than evaluating, implementing and learning the quirks of a new platform.