Man, technology is COOL. Particularly new stuff like image recognition-based search engines, which operate using audio-visual content rather than tags.
IBM (news, site) announced that they are currently working in collaboration with the European Union consortium on such a tool. Search in Audio-Visual Content Using Peer-to-peer Information Retrieval, (SAPIR) analyzes photos, sound files and even video queries. Once you feed the engine any of these file types, it scours the ‘net and hands back similar pieces.
How SAPIR Ticks
SAPIR has two major tools operating under the hood. The CoPhIR (Content-based Photo Image Retrieval) Test-Collection and IBM's MUFIN (Multi-Feature Indexing Network) both extract data from Flickr’s ginormous archive and index features such as color structure, color layout, shape edges and texture.
As SAPIR’s demo video shows, such combined technology can be used to help a tourist find information on their location simply by taking out their phone, snapping a photo and inserting it into the engine. Furthermore, they can combine text with media for more defined results:
Why Should You Be Interested?
Fair enough question. While at the onset such technology might seem like nothing more than frivolous and fun, it is, on the contrary, quite useful.
"SAPIR is a potential 'game-changer' when it comes to scalability in search and analyses," explained Yosi Mass, a research scientist at IBM Research-Haifa, and project leader for SAPIR. "It approaches the problem from a fundamentally different perspective, and opens up a universe of new possibilities for using multimedia to analyze the vast visual and auditory world in which we now live."
Picture one day being able to search for which storefronts have the t-shirt you want on display. Have an album full of photos you took on vacation but can’t remember the names of any of the places you went to? No longer a problem. That uh, oozy rash you have? Snap a photo of it, find other people who’ve been in the same gross boat and ask them how they got rid of it. Need to know the name of the mysterious vermin infesting your house so you can figure out the best way to kill? Easy! And, on a more serious note, think of how it could help branches of the Department of Justice and criminal investigations.
Get familiar. SAPIR is already available for testing, just head over to its homepage here.