The IMAGINATION PENTA-EURIPIDES² project aims to develop technologies and improved building blocks for the next generation of CMOS image sensors, signal processing, and transmission. In particular, it will develop ultra-high resolution and low noise image sensors – the building blocks for the visible spectrum, outside visible spectrum, and time-of-flight (ToF) image sensors. In addition, the project will research technologies such as ‘stitching’ for small global shutter pixels and stacking, new application algorithms to further improve image quality and functionality, compression, and transmission standards. The technical results will be demonstrated in prototype silicon, in machine vision and broadcast applications, and in technology demonstrators.
Key applications and their requirements
Among the many applications that call for everhigher resolutions and frame rates, and better image quality, are:
— Advanced production processes – these need an ever-more accurate and faster automated visual inspection. The most demanding applications (Display and Component Inspection) require cameras with resolutions of up to 250 megapixels (MP) and sustained data rates of 10 gigapixels per second (GP/s). Other new factory automation tasks need highresolution images complemented with a point cloud to supply depth information.
— Robotics in Industry 4.0 and Smart Factory environments such as collaborative, assembly, pick & place robots, and autonomous guided vehicles which require accurate, reliable depth information.
— Consumer experiences and broadcasting – consumers nowadays expect immediacy, individualisation, interactivity, and immersion from media-content consumption for an “even better than being there” experience.
Before covid-19 the technology trends in broadcasting to move to even better picture quality (4K, HDR, wide colour gamut, and 8K) and the trends to optimize workflows to reduce costs and enable multi-platform content creation were equally important. Today workflow optimizations and cost reductions prevail over next video standards. During the pandemic years investments have been on hold and with that the introduction of 4K and 8K has been delayed. In the coming years the focus will be on finalizing the move towards 4K and optimizing workflows to reduce both operational costs as well as capital investments.
Future developments
To address the needs outlined above, the IMAGINATION project partners will cooperate on the development of the following:
— Building blocks for new CMOS image sensors: smaller pixels, increased dynamic range, increased data rates, ultra-high-speed architectures, high-speed serial interfaces, the use of stitching, and investigation of 3D stacking.
— Video processing functionality that can handle 10 Gpixel/sec.
— New optical-mechanical architectures to enable more cost effective 4K broadcast productions.
— New solutions for video transmission over fibre, both compressed and uncompressed (data rates of 100 gigabits per second).
— New low-bitrate RAW Bayer data compression for hardware-and software-based systems.
— Signal processing to generate 3D point clouds from ToF sensors.
— Machine vision and broadcast application demonstrations in cameras and frame grabbers. Other silicon-based results for display and component inspection will be demonstrated in evaluation set-ups.
— Standards for image compression and transmission (CoaXPress, SMPTE, AMWA-NMOS).
Project Consortium
Developing and optimizing these technologies for professional products requires a critical mass of worldclass engineering and research skills. The IMAGINATION consortium brings together a highly qualified and multi-disciplinary team covering all the necessary technologies including optics, image capture, video processing, and transmission. The partners are key European players who are world leaders in their respective markets, with proven track records in the imaging industry. They include an R&D institute, fabless design houses, a semiconductor manufacturer, and system integrators with advanced know-how in image sensor technologies and designs, signal processing, transmission, and camera integration.
Expected Impact
There are extensive market opportunities for the technologies being developed within the IMAGINATION project. In broadcasting, several factors are likely to drive innovation in the coming years. Firstly, while linear television revenues are predicted to remain flat, Internet and Over The Top revenues are expected to grow at a CAGR of 35%. To diversify and compete in this context, Pay TV, video on demand, cinema and sports broadcasters will continue moving to higher 4K resolution (premium quality) formats, but introduction of 8K is expected to be delayed significantly. In addition, the consumption of live events is expanding worldwide alongside the growth in ‘ondemand TV’. This is leading to ongoing investment in cameras for live event production and broadcasting, but at the same time with more focus on optimizing the operational costs and investments of live productions. Furthermore, slow motion is growing in popularity – this means broadcasters will use cameras with slow-motion features in a wider range of applications. Across the industry, the trend is towards multi-purpose rather than specialty cameras for individual applications – as well as the drive mentioned above to optimize remote production to reduce operational costs, provide a faster ROI and improve the sustainability (environmental impact) of live television.
In machine vision, the market is not as standardized as in broadcasting. There is greater variation in image sizes, number of pixels, pixel functionality, and frame rates, as well as in the adoption of 3D technologies such as ToF. This more diversified landscape leads to a wide range of opportunities for both small and large players. In particular, European companies are known for their innovation and quality, which gives them strong market potential in new inspection system developments, including in China.
One of the key target areas for IMAGINATION is future imaging needs in high-end area-scan camera and grabber systems for display quality and semiconductor back-end component inspection. Differentiation for EU camera builders will come from driving application innovation on throughput, image pre-processing, product reliability and camera calibration for best-in-class inspection/metrology accuracy.
Overall, IMAGINATION will enable European companies to maintain and grow their leading positions in global broadcasting and machine vision markets. This will result in benefits for consumers, broadcasters and content creators in the entertainment domain, as well as for manufacturers using machine vision to increase efficiency and productivity in their operations.
Read the full Project Profile here.
PENTA and EURIPIDES² are two EUREKA Network Clusters, operated by AENEAS.