Apache Lucene Connector Framework Officially in Incubation

By Dee-Ann LeBlanc (@deele) Jan 21, 2010

The Apache Lucene Connector Framework (LCF) has reached the next stage toward becoming an official Apache (news, site) project.

What Is A Connector Framework?

There's no point in having a content repository if you can't search and access it. The Lucene (news, site) Connector Framework will ultimately provide an API allowing any application to interact with repositories such as SharePoint, various content management systems and EMC Documentum. Once indexed, search access can be limited per user.

According to the initial proposal, the key design points for the framework are:

Extendability - As new repositories enter the picture, new connectors can be added, along with new authorities for each repository's security model.
Incrementality - For each repository, process only what's changed on each crawl.
Restartability - Use a database with ACID () properties so crawls are safe against interruption or machine shutdown.
Security - Establish model of security tokens allowing search engines to enforce the repository's security model.
Limited Footprint - Operate reliably within a fixed memory footprint, regardless of configuration.
Performance - Manage connector-specific resources for maximum overall throughput.
Transparency - Generate reports on all crawl and connection activity.

For how to get involved in this project, see the Lucid Imagination press release.

What Is The Apache Incubator?

Every open source entity that wants to become an official Apache Software Foundation project has to go through the Apache Incubator. The two primary goals of the incubator are to: