Tuesday, April 21, 2009
Google similar image search
The future is closer than I thought.
Google has recently published in its lab section a similar image search service.
No details on recipe to build such system have been released, but from the first impression my guess is that some of the ingredients are SIFT, textual context, and inverted lists (in the sense that they are used to index visual similarity properties).
Google labs site indicates Chuck/Charles Rosenberg as one of the engineers that worked on similar image search, his Google bibliography has an interesting article (by Rosenberg, Ting Liu and Henry A. Rowley), Clustering Billions of Images with Large Scale Nearest Neighbor Search, which probably describes another ingredient of the recipe (large scale image clustering).
Two other interesting, and probably related, papers from Googlers (Shumeet Baluja and Yushi Jing) are PageRank for Product Image Search, and VisualRank: Applying PageRank to Large-Scale Image Search.
What’s the future for MiPai? Become the ingredient of a better recipe!
Why SIFT?
Look at the following search results for the Bill Gates’ mugshot:
The last results show a number partial matches of a cropped, scaled, and moved parts of the original image. This is a kind of match that SIFT is very good at spotting it, while other typical similarity measures (e.g. MPEG-7 visual descriptor) are not able to spot because they handle the whole image as a single entity.
Why textual context?
Look at the following search results for the St. Louis Gateway of the West, which has been previously retrieved by searching for the word “arch”:
The last result show a completely non-similar image, but its source page contains the word “arch”.