Your browser is not supported. Please update it.

24 November 2011

VideoSurf takes Microsoft into a whole new way of solving cloud problems

Microsoft this week has bought perhaps the most compelling video search technology company around, in the form of California-based VideoSurf, paying $70 million. The company has only one single record of investment and that was this April for $16 million, so it looks like the founders who put in $5.5 million, and recent investors have made something close to a $50 million killing, in what otherwise has been a buyers’ market.

The startup was founded by computer vision experts from academia and a CEO who sold FareChase to Yahoo, plus funding from Al Gore (it must use Al Gore- ithms) and Joel Hyatt. VideoSurf was founded in 2006 and began showing product two years later in 2008 and its most famous application is the use of a mobile application to identify video that its camera is pointing at.

The idea is for Microsoft to integrate this videoSurf technology into the Xbox 360 to improve search and discovery on Xbox LIVE. Presumably the fact that it works on a handset suggests that Microsoft will open up access to the Live! network to portable devices at some point.

VideoSurf’s technology is something described as content analytics and as far as we can see it comes down to pretty advanced face recognition technology courtesy of its chief scientist Israeli Professor, Achi Brandt,

Brandt is perhaps one of the world’s experts in the idea that a really, really complex problem is best solved by thinking about it not just mathematically, but by in layers. It’s about hierarchical processing of large data set problems called multiscale or multigrid problems, so mathematically working simultaneously on all layers at once, and cutting out impossible answers as rapidly as you can and getting down to a position where conventional math can solve what’s left.

The same multiscale/multigrid techniques are used for weather forecasting and the complex field of fluid dynamics (the math of how fluid will flow).

Brandt has now used these techniques to solve face recognition in seconds, and his system can work out what scene you are in what piece of film by simply comparing that face against a database. We’re fairly sure these problems have to go to the cloud to be worked on by servers, at the very least the look up has to go to the cloud.

VideoSurf has its own web destination, which is likely to either be scrapped, or radially improved and dropped into Xbox Live, as well as indexed video files of all Hulu, CNN, TMZ, Metacafe, Fancast, Comedy Central and Dailymotion to begin with, to search across, but it could certainly be made to work with the entire video contents of Xbox Live!

The system first came out on a Motorola Droid as an App and we don’t think anyone at the time had any idea of how much new pure science had gone into the App ‘ it was just seen as cool.

It perhaps has wider application than even its founders realize. Back in 2009
Google launched Google Goggles, as a visual search engine based on image recognition, where a picture in the camera is the search criteria instead of a word.
The VideoSurf software could perhaps be tweaked to open up the same kind of App capability for a broader range of WP7 Apps.

At the time we said of this it was a genuine new paradigm. Most video or image search tools are accessed using words ‘ and Google is perhaps not so good as specialists such as Blinkx for video search, because these approaches depend upon extracting words from the image, using things like closed captioning, scripts and voice recognition. But Google Goggles works on pattern recognition, comparing an image to a vast Google database of images. VideoSurf does the same with faces and can match up precise movie scenes if you start with enough data.