AI-based recognition processing is expanding its applications from text and voice to image data.
Sony has developed the Intelligent Vision Sensor, which combines the hardware engine for AI processing and streamlined AI algorithm, realizing the world‘s first image sensor equipped with AI processing functionality.
The Intelligent Vision Sensor is a revolutionary sensor that enables high-speed edge AI processing within the sensor unit, on top of the image processing. Leveraging Sony‘s stacking technology, we have successfully equipped with an AI engine and dedicated data memory onto a circuit layer, making it possible to integrate AI-enabled function into the tiny sensor.
This has enabled to replace the conventional system of transferring image data to a cloud service for AI processing, enabling to remove the data-transfer time and address the privacy concern , and also economize on power consumption and telecommunication cost.
* Among image sensors. According to Sony research.(as of the announcement of May 14, 2020)
The AI-processing-enabled image sensor has countless future application prospects, such as the mobile appliances, where it can improve camera functions and roll out to new app-based services.
Night views are always difficult to capture from behind window glass because of the reflection. This problem can be solved by AI that integrates the data from the color sensor and polarization image sensor, enabling to remove glare from the image while retaining a photogenic nightscape scenery.
Due to the physical constraints, optical zoom cannot be realized in a smartphone. It is therefore conventional to apply digital zoom, which functions by enlarging the captured image. However, enlarging compromises the image resolution and loses some details.
Applying AI to digital zoom can achieve highly accurate super-resolution processing based on the attributes of a photographed object. It has a potential for realizing clear zoomed images just like the optical zoom.
You aim a camera lens at an object, and AI identifies it, say “a dog,” or analyzes its details and gives you more information, such as “a Corgi” and more of relevant knowledge. There are possibilities to enhance its accuracy by leveraging the cloud technology. AI can learn the images of your pet dog, so it will follow and focus on your dog automatically when you point a camera to it. It would be also possible that AI cuts out the best-angle shots and save them. The images are automatically tagged with keywords such as “dog,” “park,” and “lawns,” which will possibly enable you to easily search for the images you want among a large number of saved photos.
AI will be able to analyze a selfie picture and read a pulse, diagnose dermatological conditions, etc. by means of the image data such as brightness and color elements as well as the multi-wavelength sensing and polarization sensor data. In the future, AI processing may be able to provide a detailed health check based on diverse biometric information.
AR is a technology to superimpose various information on images captured by a camera. Leveraging AI, 3D information obtained through a distance sensor and image data can be integrated and metadata, such as names of places and buildings, as well as CG images are overlaid. This will enable useful and enjoyable AR experiences through visually enriched images, such as arrows appearing on a captured image to indicate the route direction and anime characters popping out behind a tree for entertainment. It may be possible in the future to share the AR space and play a game with others.
This is a small-size, energy-efficient hardware engine designed specifically for the deep neural network (DNN) processing.
It enables a logic chip, which comes with spatial constraints, to be equipped with a hardware engine for AI processing.
Sony‘s original AI algorithm with its streamlined capability reduces the computation load compared to the conventional algorithm with the same functionality. This enables both energy-efficiency and high-speed processing, which are essential for the edge AI integration.