According to AENEAS Internal Regulations, the AENEAS Technical Experts Group is an executive body, in charge of advising the AENEAS Management Committee in all technical matters relating to programmes and projects relevant to the Association, under the responsibility of the AENEAS Supervisory Board.
As Xecs, the Eureka Cluster in the field of electronic components and systems, is operated by AENEAS, one important role of the AENEAS Technical Experts Group is to monitor Xecs projects, in close cooperation with the public authorities of the different countries supporting the programme. In this endeavor, experts from other associations supporting the programme, namely Inside and EPoSS, are invited to join the AENEAS TEG, forming a specific group dedicated to Xecs, therefore named Xecs TEG.
Xecs TEG members act as evaluators, mentors, and reviewers of projects submitted and running in Xecs Cluster.
Patrick Cogez, AENEAS Technical Director, explains the composition of the Xecs TEG, the calls for new members, and the importance of its work to project consortia in the Xecs cluster.
Who are members of the Xecs TEG?
“The Xecs TEG consists of 36 experts, about half from academia and the other half from industry. They represent a wide spread of countries across Europe, with many from the big players in the European field.
In terms of competencies, the Xecs TEG members cover the whole spectrum of the ECS value chain. Indeed, we map their competencies over the chapters of the ECS Strategic Research and Innovation Agenda (ECS SRIA) to ensure that the content of every chapter is fully covered. This includes advanced technologies such as quantum technologies and photonics.
How are the members selected and when is the next call to join?
“There is an annual call for experts to join the Xecs TEG. In the past, we had calls for the Eureka Clusters PENTA and EURIPIDES² Expert Groups which were open to AENEAS members only. When we initiated the Xecs programme, which targets Research and Innovation projects along the whole ECS value chain, we extended the scope of the Expert Group to allow members of our fellow industry association communities – EPoSS and Inside – to also propose candidates. In these calls, we indicate if we are looking for some particular expertise. We consider both candidates’ scientific and technical competencies and their experience of managing and assessing collaborative research and innovation projects.”
What are the specific tasks of Xecs TEG members?
“The Xecs Cluster has a two-stage proposal process. The first stage is submission of a project outline (PO), followed by a Full Project Proposal (FPP). Within this process, members of the Xecs TEG act as evaluators or mentors. For the projects which are labelled and funded, Xecs TEG members can also act as reviewers.”
What does ‘evaluation’ consist of?
“Evaluation takes place both at the PO and the FPP stages. A key goal for the Xecs TEG experts during the PO evaluation is to help the consortium to significantly improve the quality of their proposal and to present their project well if invited to submit to the FPP stage. Mostly, we will point out where we see weak points or other issues such as where the consortium hasn’t sufficiently included the state of the art. We may indicate where they need to add partners to cover key parts of the project or to ensure the full chain is involved. We may also encourage them to focus on just one or two areas, rather than going in too many directions at once.
Impact is a crucial aspect. The projects have to be innovative and have a real impact in terms of society, business, and technology. So, while we look at the quality of the proposal, the project has to have high-quality impact as well.
Then when a project consortium submits its Full Project Proposal to the Eureka Xecs Cluster, the assigned Xecs TEG experts will provide feedback to the relevant national public authorities who decide on which projects will be labelled and funded.
Our evaluation for the public authorities is a recommendation. The public authorities have their own national criteria and funding priorities, and they ultimately decide on which projects will be funded. Nonetheless, most of the time our views converge.
We have regular meetings with the public authorities supporting Xecs and they constantly say that they appreciate our input and share it with their own experts. So, while we are judging on some different criteria – for instance, we look at the consortium and proposal as a whole, whereas they are focused on their national companies and universities – the public authorities trust and appreciate the Xecs TEG feedback.”
What is involved in the mentoring stage?
“Once a project consortium has been invited to submit a FPP, they are assigned a mentor. This is unique to the Eureka Clusters, it is an important differentiating factor from other funding programmes.
The role of the mentor is first to convey what was said in the meeting with the public authority experts who assessed the PO proposal. Of course, the project consortium receives a detailed written account of the feedback, but it can be very helpful to have someone explain what’s behind the words – the FPP phase is quite strict. We give the project leaders and consortium partners someone to discuss things with, to explore how to improve the project or meet concerns raised by the experts.
Although the public authorities make the final decision, as I said they have a lot of trust in the Xecs TEG experts. So, the recommendations given by the Xecs TEG mentor to the project consortium are extremely valuable in ensuring the FPP is of the right quality.
As a member of the Xecs Expert Group, the mentor attends the plenary evaluation of the FPP by the Xecs TEG. She/he doesn’t advocate for the project but can answer questions and give clarifications if required.
What does the reviewer role consist of?
“Once a project is up and running (after labelling and funding), some Xecs members, usually selected among the experts who evaluated the project in the first place, are appointed as reviewers. They will participate in the project reviews, along with the public authorities, and provide feedback and advice to the consortium. They will also assess the requests for changes to the project, if any. In addition, the mentor retains her/his role of supporting the project
Project reviews usually take place over a day or half a day, during which the project consortium presents their progress. The mentor will work with the project team to prepare, including holding a dry run of the presentation ahead of the actual review. As the Xecs TEG members are themselves, seasoned project managers, they can advise the project team on what kinds of questions to expect from the public authorities and give them feedback on aspects that might be missing or need to be covered.
Patrick Pype, Xecs TEG Chair, Director Strategic Partnerships at NXP, adds a further perspective with an in-depth look at the Xecs TEG tasks.
Can you give more details on how the Xecs TEG members evaluate projects?
As Patrick Cogez said, the Xecs TEG members play a very important role, amongst others in the technical evaluation of new proposals. Depending on the size of the proposed project, two or three Xecs TEG members will read the proposal in detail and evaluate it based on a number of criteria.
When all the members have completed their evaluation, one of them will consolidate all the inputs in order to arrive at a single commonly agreed evaluation sheet. In some cases, evaluators may have different opinions on certain criteria. These are discussed over email or via video conference so that finally there is full alignment between the evaluators. If, however, the evaluators cannot come to an agreement, then this is discussed in the plenary session of the Xecs TEG.
All the evaluation sheets are read and commented upon at a 2-day gathering of all the Xecs TEG members in Paris. At this meeting, questions are raised, and clarifications may be requested on some of the texts submitted by the evaluators. The points where some evaluators disagree are discussed and at the end of the discussion, a consensus is reached. This is a very thorough process in order to ensure that the proposers get valuable feedback which helps them to improve between the PO and FPP stages and also to learn how to make high-quality proposals for future calls.
Lastly, a summary is made of the technical quality and content of the proposals in four categories (low, sufficient, good, top). At the PO stage there is also a classification of the potential of the proposal to become a good project with high impact, based on four categories (no, low, good, top).
How does the labeling process work? How are the evaluations from the Xecs TEG combined with those of the Member States (the public authorities)?
The Member States’ public authorities conduct their own evaluations in addition to those of the Xecs TEG. As Patrick Cogez explained, they look at different criteria for the Xecs TEG members and also focus on the added value of the proposal for their specific country.
During a joint meeting, the AENEAS Management Committee (MC) and representatives of the Member States (Public Authorities Board or PAB) run through all the proposals and discuss comments from both the Xecs TEG community and the participating country representatives.
I represent the Xecs TEG community at that meeting. The PAB representatives value the contribution of the Xecs TEG community very highly and mostly follow the technical feedback and recommendations the community provides. Unfortunately, some proposals will not make it. Sometimes this is because of a lack of technical quality and innovation or because of other priorities in certain countries, or there may be a lack of good consortia in certain countries.
Based on discussions between the MC and the PAB at their joint meeting, a number of projects will be labeled. This means that they are considered capable of delivering high enough quality and impact to go ahead. However, this does not yet guarantee that they will receive funding. To receive funding, they still have to follow the specific process in their country and sometimes still have to apply with a specific document in order to get clearance for funding.
What does your role as Chair of the Xecs TEG consist of?
Firstly, I would remark that it is a pleasure to be Chair of the Xecs TEG. The role of Chair is mainly to coordinate the process of the technical evaluation of the proposals and to represent the Xecs TEG community at the MC meetings and at the joint MC-PAB meetings.
As we have a group of more than 30 technical experts who have strong opinions on certain proposals, it is always a challenge to come to a unified aligned position where all agree. Opinions between industry and academia in particular can be quite contradictory so they require a good debate at the Paris meeting. That also means time management is important because some representatives are very passionate about certain topics, but in the end, we have to come to a consensus on all the proposals submitted.
Interacting with all these different people coming from different countries, with different positions in industry and RTOs, is something I love about my job. And as a result, being Chair of the Xecs TEG is also frequently a learning curve for me, and I can often apply this in my current job at NXP.
How do the consortium partners benefit from the evaluation process?
For the consortium partners there is a lot of value in interacting with the Xecs TEG members. They have a long history of evaluating proposals and know what is important to cover in the text for each criterion on which proposals are evaluated. They can give advice on what will work well and what will not in composing a proposal.
I deliberately call it “composing”, because on one hand it is about achieving a good technical quality, but on the other, it is also an art to master. I consider it like composing a song, a symphony or an opera. The listener sometimes feels the magic when listening to it and sometimes is disappointed. Similarly, Xecs proposals sometimes make the evaluators happy and sometimes they make them disappointed. However, the Xecs TEG members master the art of how to compose a good proposal and can help a consortium to arrive at a high-quality proposal in the end.
Read and download this interview here.