Intellifusion Technologies, founded in 2014 and based in Shenzhen, China, has made a name for itself as provider of facial recognition technology to the city’s police. This has now been extended to naming and shaming jaywalkers in a move that would be highly controversial in most countries.
Not to be confused with the New York based developer of automotive software with the same name, Intellifusion Technologies developed its first facial recognition product called DeepEye on the back of a 40 million yuan ($6.15 million) capital support grant from the Shenzhen local government’s “peacock campaign” – to attract high tech talent to the city.
This quickly led to the company’s big break in having DeepEye adopted for Shenzhen’s public security system in 2015. The system had been developed for China’s fast growing public surveillance market, featuring non-cooperative facial recognition, meaning identification of people from their faces as they move around rather than in stationary poses for cameras as in biometric applications like automated passport checking.
This ideally requires high resolution cameras combined with deep machine-learning to extract and amplify key distinguishing features of the face. Early non-cooperative facial recognition systems around 10 years ago suffered from very high error rates, when attempting identification of people from side angles rather than frontal images. The latest systems such as DeepEye begin by determining the precise location and orientation of the subject’s head and then crop the region of interest before constructing in effect a virtual 3D model which can then be viewed logically from any angle by rotating it.
The 3D model in turn is enhanced by analyzing the video image and allowing for movement, which provides additional information about each point of the face. Here the system exploits stereo-motion associated with the natural movement to gain more information about each point down to pixel size than would be possible from a still image frozen in time. That is critical for non-cooperative facial recognition.
The 3D face model then allows a set of unique shapes defined by geometry and color distribution to be extracted for recognition against 3D galleries. When the face is not front on, the 3D model assists in matching against 2D image galleries by rotating and rendering the 3D reconstruction into the right format for comparison.
Intellifusion’s DeepEye was used initially for identifying criminals, but was then extended more controversially to naming and shaming jaywalkers, which is widespread in China. Initially photos and family names of offenders were almost instantly displayed on large LED screens, for some light public shaming, but now Intellifusion is working with mobile phone carriers and social media platforms such as WeChat and Sine Weibo to send text messages to people as soon as they have been detected jaywalking – which would be cheaper and more practical than erecting large numbers of LED screens just for that purpose.
Meanwhile, Intellifusion’s apparent success in Shenzhen is leading to adoption in other cities and becoming part of the national CCTV network currently expanding rapidly. It is already by some distance the world’s largest monitoring network, with around 200 million CCTV cameras now deployed, set to reach 400 million upon completion by 2020.
Even then, accurate coverage will not be ubiquitous, which is why Chinese police have been experimenting with facial recognition goggles for surveillance in high profile locations to augment the security provided by the CCTV network.
Data on DeepEye’s accuracy is not available, but if it is comparable with the best systems that have been calibrated it will be quite effective. The most accurate non-cooperative facial recognition according to the 2017 Face in Video Evaluation (FIVE) testing performed by the U.S. National Institute of Standards and Technology (NIST) was NEC’s face recognition technology which achieved accuracy of 99.2%. The next best system came in at just under 97% so NEC had a significant lead, although the latest system Intellifusion has deployed in Shenzhen was not evaluated then.
FIVE’s conclusion was significant though in stating that accuracy of non-cooperative facial recognition systems was now approaching that of full frontal systems where subjects posed for the camera, but with the important qualification that this is only the case if the database has high quality still images taken face on. That has not happened yet even in China and would be the equivalent of requiring all citizens to have their fingerprints taken or DNA sampled whether or not they had a criminal record.