Yes, the Central Intelligence Agency is among 18 U.S. Intelligence organizations to benefit from the Apache Lucene/Solr enterprise search technology.
In-Q-Tel, the CIA’s VC arm, has announced its investment into Lucid Imagination, a commercial provider of open source search technologies.
This may mean a lot of things (just scratching the surface here): great exposure for Apache (not bad of a present for the ASF’s 10th birthday), promotion of open source adoption, better data mining opportunities for the U.S. intelligence -- you name it.
Facebook and Google, interestingly, are also (indirectly) in the mix.
A Bit About In-Q-Tel
What’s in a name? The In-Tel part is quite clear. “The Q is just the play on the character Q who provides all of the cool gadgets in the James Bond movies,” according to a former In-Q-Tel employee.
In-Q-Tel was formed in 1999 specifically to cater to the Central Intelligence Agency. According to the CIA, IQT’s origins can be traced to one of the CIA’s own -- to Dr. Ruth David, a former CIA deputy director for science and technology.
In-Q-Tel is “charged with accessing information technology (IT) expertise and technology wherever it exists and bringing it to bear on the information management challenges facing the Agency.”
Facing the IT industry, the company can do pretty much any business transaction, including venture funding, establishing joint ventures, fund grants, sponsor open competitions, award sole source contracts, etc.
The Agency has granted In-Q-Tel “many degrees of freedom,” and no approval is required for any business deal.
The company focuses its efforts around the following technologies useful for the CIA:
- data mining
- profiling search agents
- geographic information systems
- imagery analysis and pattern recognition
- language translation
- mobile computing
- secure computing
among others.
Although In-Q-Tel is openly affiliated with the CIA, they say that the questions about its business nature and the Agency having “other motives” have no grounds. In-Q-Tel is not “a front company for the Agency.” It is a nonprofit -- 501(c)3 -- corporation.
Why the Interest in Open Source Search?
As In-Q-Tel focuses largely on innovative technology for the CIA and other U.S. Intelligence efforts, search tech has been a long-time focus for In-Q-Tel. And now, Lucid Imagination will support the U.S. Intelligence Community by providing advanced access to Lucene and Solr search solutions.
In an interview with KMWorld back in 2004 (at the time, In-Q-Tel’s senior director of federal and intelligence community strategy) Greg Pepus, talked about In-Q-Tel’s multiple investments in search vendors. He talked about Convera, Endeca, Google and how keyword search is just one of the approaches to search. Kofax was mentioned for its strong multilingual focus. All of these vendors representing In-Q-Tel’s main areas of focus in intelligence technology.
Learning Opportunities
Webinar
Mar
29
The CX Commute vs. the CX Joy Ride
Is your CX traffic holding up your customer's journey?
Webinar
Mar
29
Is Your Microsoft 365 Data Recoverable? How to Survive the IT Hotseat When Data is Lost
See why MIcrosoft recommends third party backup
Webinar
Mar
30
An Optimized Customer Contact Strategy Combines Transparency and Intelligence: The State of Outbound Communications
Learn from the Neustar-commissioned Forrester Consulting study on challenges and solutions for contact centers
Webinar
Apr
5
Adopt a Modern Digital-First Marketing Strategy
Learn how to use a composable DXP to your advantage
Apache’s Lucene/Solr technology was just the next logical step, really, for expanding the company’s portfolio. The Lucene search library is in the top 5 Apache projects with more than 4,000 installations globally.
Lucene/Solr-based search has become one of the fastest-growing segments in the enterprise search market over the past three years, with more than 9,000 downloads per day and over 4,000 organizations using the open source software for some of industry’s most intensive search needs.
Investing in, say, Google search would be a bit senseless (but who knows). There is a connection though between Google and In-Q-Tel: Keyhole.
Keyhole, Inc. used to be one of In-Q-Tel’s assets before it was bought out by Google in 2004. Keyhole is now used in what we know as Google Earth, and it still remains in In-Q-Tel’s portfolio due to the 2003 investment.
In general, proprietary commercial vendors are more closed and more expensive. And with Apache’s liberal license and the oh-so-accessible nature of open source, the choice was obvious. Or, as Troy M. Pearsall, executive VP for architecture and engineering at In-Q-Tel, puts it: “While we evaluate many technologies as part of our investment process, we only invest in the most promising capabilities.”
What Does It Have to do With Facebook?
Just a tidbit of related trivia information. Coincidentally enough, folks connected to In-Q-Tel (former employees and present board members) apparently stood behind the second round of VC funding for Facebook, according to some sources.
Not to go into the conspiracy theory mode, but what can be a better playground for a data mining/intelligence project than Facebook with its gazillion users and feeds coming in from multiple other sources such as blogs, IMs, microblogging tools.
Hence, the note in Facebook’s privacy policy saying: "Facebook may also collect information about you from other sources, such as newspapers, blogs, instant messaging services, and other users of the Facebook service through the operation of the service (e.g., photo tags) in order to provide you with more useful information and a more personalized experience. By using Facebook, you are consenting to have your personal data transferred to and processed in the United States."
The CIA and the NCS are also said to use Facebook for recruiting.
An interesting mix, isn't it? Perhaps, we'll learn even more from the next James Bond movie.
