Imaging and Sensing Technology

Imaging and Sensing Technology

CMOS image sensor development at Sony began in 1996 and led to the launch of our first CMOS image sensor (IMX001) in 2000. At the time, CMOS image sensors produced noisy images under low light and were also inferior to CCD image sensors in the number of pixels. However, the lower readout speed of CCD image sensors convinced us that they would be unable to support high-resolution data as the industry moved from SD to HD video. In 2004, we therefore changed course greatly by shifting our focus from CCD to CMOS image sensor development. It was a bold decision. Instead of holding the world’s number one share in CCD image sensors, we would be building on a negligible market share in CMOS image sensors.
Later in 2007 we commercialized CMOS image sensors with an original column A/D conversion circuit for fast, low-noise performance, followed in 2009 by back-illuminated CMOS image sensors with twice the sensitivity of conventional image sensors – beyond the human eye.
Further examples of technical innovation that has enabled us to constantly lead the industry include stacked CMOS image sensors in 2012 – with higher image quality and multiple functions in a smaller package, thanks to layering of the pixel and signal-processing sections – and, in 2015, the world’s first image sensors with a Cu-Cu connection, enabling smaller packages, higher performance, and greater productivity in manufacturing.

Image sensor Advance in key technology

The CMOS image sensor with a column A/D converter achieves fast and low-noise performance by the column A/D converter (advances of peripheral circuit technology). The back-illuminated CMOS image sensor achieves higher sensitivity due to the back-illuminated structure (advances in pixel technology). The stacked CMOS image sensor achieves high image quality and multiple functions in a smaller package (advances in layering technology). The stacked CMOS image sensor with Cu-Cu connections realizes smaller packages, higher performance, and greater productivity by direct connection of copper pads (advances in chip-to-chip connection).

Column-Parallel A/D conversion circuit-equipped CMOS image sensors (commercialized in 2007)

These sensors use original Column-Parallel A/D conversion technology, with an A/D converter for each vertical row of pixels, arranged in a parallel configuration.
In this arrangement, analog signals read from the vertical signal lines can be conveyed directly to each row’s ADC across a minimal distance, which reduces loss of image quality from noise entering the signal during analog transmission and accelerates signal readout. Noise is also reduced through dual noise cancellation, with high-precision cancellation applied to both analog and digital circuits.

Back-illuminated CMOS image sensors (commercialized in 2009)

These image sensors adopt an innovative back-illuminated structure that offers lower noise and nearly double the sensitivity of conventional front-illuminated CMOS image sensors.
Without interference from wiring or transistors, light is received from the back side of silicon substrate, which increases the amount of light entering each pixel and reduces loss of sensitivity relative to light entering at various angles. This enables smooth, clear images even at night or in other low-light conditions.

Stacked CMOS image sensors (commercialized in 2012)

In the stacked structure adopted by these image sensors, the pixel section where back-illuminated pixels are formed is layered over a chip (instead of the supporting substrate used in conventional back-illuminated CMOS image sensors) where signal processing circuits are formed.
One advantage is that large-scale circuits can be mounted on a small chip. Since each section is formed on a separate chip, specialized manufacturing processes can be used to produce a pixel section with high image quality and a circuit section with high performance, enabling higher resolution, multi functionality, and a compact size.

Stacked CMOS image sensor with Cu-Cu connections (commercialized in 2015)

Cu-cu connections involve direct connections between the copper pads formed on the layering surfaces of pixel chips and logic circuit chips. Without the need to provide electrical connections through pixel chips or special areas for connections, manufacturers can make smaller image sensors at a higher rate of productivity. Offering greater freedom in pin layout and higher density, the technology will help enable stacked CMOS image sensors with expanded functionality.

Sony’s technologies, expanding into sensing applications

In the image sensor market, we are also focusing on the sensing field, where applications are expected to expand. Besides the imaging technology we have honed for viewing captured images, we will be combining this technology with sensing technology for acquiring and using a various information. In this way, we are cultivating new applications and markets for image sensors.

Sensing technology expanded by image quality, information, and higher speed

ToF Image Sensors: Expanding Possibilities in 3D Object Recognition

Time-of-flight (ToF) image sensors determine the distance to objects from the time it takes emitted light to reflect off the objects and reach the sensor. Pixel technology in Sony’s back-illuminated structure produces depth maps as accurate as those of conventional sensors even at 1.5 times the distance. Gesture, object recognition, and obstacle detection are expanding applications for these sensors, which are used in augmented reality (AR) and virtual reality (VR) scenarios, and in robots and drones that require autonomous operations.

  • Image captured with a CMOS image sensor
  • A back-illuminated ToF image sensor
  • 3D model image combining a depth map and monochrome image

Automotive image sensors: Better recognition in many environments, for safety and security

Sony has commercialized the IMX490 CMOS image sensor for automotive cameras with 5.4 effective megapixels, the industry’s highest *1 in a sensor that both mitigates LED flicker (from LED signs and traffic signals) and offers a wide dynamic range. Also, IMX324 was released, which is a stacked CMOS image sensor with 7.42 effective megapixels, the industry’s highest resolution *2 for forward-sensing cameras. We expect these sensors to be used in more cars than ever, in applications including advanced driver-assistance systems (ADAS) and camera monitoring systems (CMS) that replace rearview mirrors.

  • *1As of announcement on December 18, 2018.
  • *2As of announcement on October 23, 2017.

Sample comparison movies of image sensors

  1. Industry’s highest image sensor resolution for automotive cameras

    Comparison of distant sample images

    IMX324 shooting example IMX324 (7.42 million pixels) enlarged image IMX224 (1.27 million pixels) enlarged image
  2. High sensitivity of 2,666 mV (standard value F5.6, in pixel binning mode)

    Comparison under low-light (0.1 lux) in sample images

    IMX324 (pixel addition mode) shooting example IMX224 shooting example

Automotive initiatives: Sensor Fusion

One forward-looking initiative at Sony is called Sensor Fusion. The technology under development integrates raw data from camera feeds, LiDAR, and milliwave radar to identify vehicles and other objects. As a sensor manufacturer, Sony is uniquely positioned to combine the signal processing, noise reduction, and data optimization in this technology. These examples show how effective Sensor Fusion can be. Even under challenging conditions for object recognition – such as fog, glare, or rain at night – Sensor Fusion enables accurate identification sooner than other systems.

Explanatory drawing showing sensor fusion that recognizes vehicles, etc. by fusing camera images with LiDAR data and Raw data of millimeter wave radar
  • Integrates large amounts of raw data from camera images, LiDAR, and milliwave radar
  • Highly accurate object identification even in difficult conditions
  • Scenario 1:

    Current Technology

    Sony's Sensor Fusion

    I dentifies automobiles ahead obscured
    by fog
  • Scenario 2:

    Current Technology

    Sony's Sensor Fusion

    I dentifies automobiles ahead obscured
    by backlight
  • Scenario 3:
    Night & Rain

    Current Technology

    Sony's Sensor Fusion

    I dentifies automobiles ahead obscured
    by headlights and rain

Automotive initiatives: Solid State LiDAR

Leveraging Sony’s semiconductor technologies, Solid State LiDAR uses highly accurate distance measurement to gain a precise 3D grasp of real-life spaces. This also leads to improved recognition capability for long-distance objects.

(This point cloud image is for illustrative purpose.)

Polarization image sensors: Revealing the unseen

Polarizers were separate in conventional polarization cameras, but innovative CMOS image sensors from Sony incorporate the polarizer into a back-illuminated CMOS image sensor. As a one-chip solution with the polarizer on the photodiode, it enables more compact polarization cameras. Potential applications are not limited to automotive field but include a variety of other applications that involve capturing subjects obscured by glare, or capturing details of surface unevenness.

When taking a picture of a pond with a polarization image sensors, the waves are removed and the underwater koi-fish appear clear
  • Tablet filling inspection

    An ordinary image sensor

    Image of tablet and aluminum sheet

    A polarization image sensor

    Image of tablet and aluminum sheet
  • Lens distortion inspection

    An ordinary image sensor

    Image of glasses

    A polarization image sensor

    Image of glasses
  • Eliminating glare from sunlight

    An ordinary image sensor

    Image of a car

    A polarization image sensor

    Image of a car
to PageTop