While most of North America was fast asleep, Lucid Imagination kicked-off the first sessions its annual European conference, Apache Lucence EuroCon. Hundreds of attendees have converged in Barcelona, Spain to learn from Solr experts, compare notes with peers and maybe find a little fun outside of the search conference hours.

Conference Day One

Apache Lucene EuroCon sessions began early in the day with keynotes by Lucid Imagination’s Chief Scientist, Grant Ingersoll and Hortonworks CEO, Eric Baldeschwieler (aka Eric 14 due to my inability to pronounce his name with my endearing Texas accent). The first day of sessions features three tracks, 18 sessions, 2 keynotes, fast paced lightening talks and the interactive and always fun “stump the chump” question and answer sessions. Attendees are free to attend any combination of sessions and activities they desire.

An entertaining, but likely not as educational, mixer on the beach will follow today’s activities. I will of course attend, not for the free drinks, but because it is my duty; oh, the sacrifices we make for our careers. That sounded sincere, right? Tomorrow is the second and final day of sessions. It contains just as much information, but has a slightly later start, and will include 15 sessions, 1 keynote and a panel discussing the roadmap for Lucene/Solr hosted by several members of the open source project’s committers.

Much of the discussion at the conference from sessions to keynotes to hallway conversations is about the popular and closely related topic of big data, which should probably just soon be “data” since large information repositories are increasingly common. Sessions such as:

and today’s keynote addresses focus on the topic. Like the majority of other technologists, attendees are exploring options for optimizing the storage, management and accessibility of rapidly accumulating data. If you have a lot of data, eventually you will have to find something; conversely, as content grows, the meta-data associated with search solutions can become “big data” in its own right (e.g. think Yahoo! search phrase recommendations).

Finding Value in Big Data

I’ve been able to attend several sessions like Grant Ingersoll’s keynote, Solr 4 Highlights by Mark Miller, Lucene/Solr committer and Engineer/Manager for Lucid Imagination and Solr + Greenplum = MPP Solr  with George Chitouras of R&D Director the Data Computing/Greenplum Division of EMC. People within corporations and vendors are doing a lot of interesting things with Lucene/Solr, and the product’s features are definitely competitive with options offered by commercial solutions.

Between sessions, one of the people I had an opportunity to speak with was Baldeschwieler who is at the helm of the recently formed Hortonworks, which Yahoo formed by to focus on Hadoop. In his keynote, “Architecting the Future of Big Data and Search,” Baldeschwieler’s said,

Hortonworks predicts that over half of all of the world’s data will be stored in Hadoop.”

That’s a bold prediction. NoSQL is a relatively new technology, Hadoop is the NoSQL option and analysts expect a data growth rate of nearly 60 percent over the next few years. Baldeschwieler was quick to clarify that he was not suggesting that Hadoop would negate or reduce the utilization of traditional relational databases, but he feels that unstructured data will be a primary area of growth and that relational databases don’t handle unstructured content in the most optimized way.

In addition to his predictions, Baldeschwieler discussed the future of Apache Hadoop, which heavily focused on ease of use to increase the enterprise appeal of the platform. Planned enhancements for the next couple of releases include:

  • easier installation
  • better tools for monitoring and operational support
  • improvements to the API

and eventually integration of Lucene into Hadoop.

Of course I had to ask Baldeschwieler about the recent dispute regarding contributions to Hadoop between Hortonworks and competitor Cloudera. He of course tactfully avoided saying anything salacious, but he is clearly passionate about the topic.

I will be sitting in several of the sessions, so be sure to say hello if you see me around.