The MANTIS project focuses on broadcasting and security systems and aims to take current imaging applications to the next level by further improving image quality (spatial and temporal resolution, higher dynamic range), combining imaging modalities, creating new functionality with Artificial Intelligence (AI) algorithms, and optimising manufacture and application workflows to reduce costs while increasing performance. In pursuit of cost and system architecture optimisation, it will also explore if, and how much, intelligence can be integrated into image sensors. MANTIS will focus on image capture systems for high-end security and live broadcasting, targeting lower costs and new functionality while maintaining ease of use.
New solutions
At the core of the project is the development of a multi-modal, multi micro camera system connected to an AI-capable edge processing box. The project will also design a new micro camera architecture that supports single- and dual modal capture and the combination of visual imaging with either NIR or LWIR (long wave infrared).
Key innovations include:
- A micro camera demonstrator with the ability to support various sensors matching 4k60 imaging capabilities in a Size Weight Power and Cost optimised implementation, targeting high-end security applications;
- A multi camera demonstrator, in which 2 or more of the micro cameras are connected to a newly defined processing box with inter-camera processing, combining visible and LWIR imaging. This processing will consist of applications such as of fusion enhanced imaging, real-time small object detection, tracking and/or re-identification for enhanced situational awareness in surveillance and high-end security and monocular depth-map generation in live sports (broadcast);
- A super 35mm, global shutter, HDR, CMOS image sensor for visual applications with minimal colour cross-talk for better wide colour gamut representation, and a colour 2/3-inch CMOS image sensor based on the developments in the SENSATION project, both for new single sensor cameras;
- A demonstrator IC that combines small 2um global shutter pixel with high QE at NIR wavelength in-pixel HDR, low power consumption and on-board intelligence for new consumer and industrial applications;
- A LWIR thermal image sensor with updated ROIC design, fabricated CMOS wafers with MEMS-based bolometers and cap wafers.
MANTIS will also deliver new image sensor correction algorithms, a mathematical model to describe the non-linearity and temperature dependence of the NIR sensitivity of the CMOS image sensors, and a simulation tool to optimise the electro-optical (EO) system design of the multi camera demonstrator including the new micro camera architecture.
The project’s outcomes have potential in other expanding markets as well, such as automotive and smartphones. Indeed, the project will identify and develop new market leadership opportunities, thereby supporting the emergence of future European champions through disruptive business and / or market approaches. Crucially, by improving quality and reducing costs, it will enable the growth of applications that make life safer and bring exciting new experiences to citizens in Europe and worldwide.
Read the full Preject Profile here.
PENTA and EURIPIDES² are two EUREKA Clusters operated by AENEAS.