Image Sensor

Sony’s Intelligent Vision Sensor supports people
by capturing information and meanings which were unrecognizable before

What‘s Intelligent Vision Sensor? - the world‘s first* image sensor equipped with AI processing functionality

AI-based recognition processing is expanding its applications from text and voice to image data.
Sony has developed the Intelligent Vision Sensor, which combines the hardware engine for AI processing and streamlined AI algorithm, realizing the world‘s first image sensor equipped with AI processing functionality.
The Intelligent Vision Sensor is a revolutionary sensor that enables high-speed edge AI processing within the sensor unit, on top of the image processing. Leveraging Sony‘s stacking technology, we have successfully equipped with an AI engine and dedicated data memory onto a circuit layer, making it possible to integrate AI-enabled function into the tiny sensor.
This has enabled to replace the conventional system of transferring image data to a cloud service for AI processing, enabling to remove the data-transfer time and address the privacy concern , and also economize on power consumption and telecommunication cost.

* Among image sensors. According to Sony research.(as of the announcement of May 14, 2020)

The AI-processing-enabled image sensor has countless future application prospects, such as the mobile appliances, where it can improve camera functions and roll out to new app-based services.

AI feature opens up new possibilities to smartphones

Glass reflection removal makes a true-to-life night view shot through a glass window a piece of cake

Night views are always difficult to capture from behind window glass because of the reflection. This problem can be solved by AI that integrates the data from the color sensor and polarization image sensor, enabling to remove glare from the image while retaining a photogenic nightscape scenery.

The digital zoom comparable to the clear and crisp image quality of an optical camera

Due to the physical constraints, optical zoom cannot be realized in a smartphone. It is therefore conventional to apply digital zoom, which functions by enlarging the captured image. However, enlarging compromises the image resolution and loses some details.
Applying AI to digital zoom can achieve highly accurate super-resolution processing based on the attributes of a photographed object. It has a potential for realizing clear zoomed images just like the optical zoom.

AI camera encyclopedia powered by mobile object recognition and learning feature

  • Phone camera will be a tool for knowledge

    You aim a camera lens at an object, and AI identifies it, say “a dog,” or analyzes its details and gives you more information, such as “a Corgi” and more of relevant knowledge. There are possibilities to enhance its accuracy by leveraging the cloud technology. AI can learn the images of your pet dog, so it will follow and focus on your dog automatically when you point a camera to it. It would be also possible that AI cuts out the best-angle shots and save them. The images are automatically tagged with keywords such as “dog,” “park,” and “lawns,” which will possibly enable you to easily search for the images you want among a large number of saved photos.

  • Personal doctor in your smartphone

    AI will be able to analyze a selfie picture and read a pulse, diagnose dermatological conditions, etc. by means of the image data such as brightness and color elements as well as the multi-wavelength sensing and polarization sensor data. In the future, AI processing may be able to provide a detailed health check based on diverse biometric information.

  • Useful and highly realistic Augmented Reality (AR) experience through AI-powered visualization

    AR is a technology to superimpose various information on images captured by a camera. Leveraging AI, 3D information obtained through a distance sensor and image data can be integrated and metadata, such as names of places and buildings, as well as CG images are overlaid. This will enable useful and enjoyable AR experiences through visually enriched images, such as arrows appearing on a captured image to indicate the route direction and anime characters popping out behind a tree for entertainment. It may be possible in the future to share the AR space and play a game with others.

Technology - Sony‘s AI technology that enhances edge AI

Sony‘s unique small hardware engine for AI processing

This is a small-size, energy-efficient hardware engine designed specifically for the deep neural network (DNN) processing.
It enables a logic chip, which comes with spatial constraints, to be equipped with a hardware engine for AI processing.

Sony‘s original AI algorithm

Sony‘s original AI algorithm with its streamlined capability reduces the computation load compared to the conventional algorithm with the same functionality. This enables both energy-efficiency and high-speed processing, which are essential for the edge AI integration.

to PageTop