TG relation: Benchmarking and Competitions
Project cluster relation: Yes: EUROBENCH, SciRoc, Metrics, Natural Intelligence, Coresense …
Areas mainly addressed: Agri-Food, Maintenance and Inspection, Agile Production, Robotics and AI/Big Data Value, Ethical, Legal, Social, Economic implications of robotics, Cybersecurity in Robotics, Regulation and Standardization for Robotics and AI, Control technology including virtualisation and edge/cloud
Main questions to be answered:
· Reproducibility, benchmarking, standards
· Objective measures of robustness, reliability and agility
· TRL roadmap for new products
WS Description: The reproducibility of research results and the objective and operational comparison of different methods is still not mainstream in Robotics, AI and Automation Research. However, in particular thanks to the EU Commission significant progress has been done, This will be a gathering of the TG on Benchmarking and Competions. The SotA will be presented and the road ahead discussed. Info about currently completed and starting projects and results willl be shared. The meeting is open to all.
Intended outcome: Workshop expected to create a roadmap or white paper, Workshops engaging non-roboticists to understand needs/goals (policy, commercial, technical), Workshop discussion topics of common interest, success stories, use cases, etc, Workshop on recent developments in technology or applications
Approach: Trustable and Dependable Intelligent Robots for Smart Societies
Subtitle: Research Competitions, Benchmarking, Societal and Ethical Issues
This is the first part of a two-parts workshop.
We propose the following:
• Before the workshop we gather opinions from the communities involved in benchmarking about how reproducibility and benchmarking can be realized and made useful to support European robotics
• We then gather opinions from that community, and communities involved in funding, legal standards, etc (as far as this is possible and reasonable), in answering these questions
• We will use POLIS (pol.is) or a similar tool to promote synthesis of knowledge from these opinions, and build an emerging consensus
• We will share this as a draft white paper with the community before the workshop
• The workshop will consist of a series of short provocations – briefings by panellists on topics linked to this. There will then be a round table discussion of the community consensus to propose a route map for future work. The pitchers will be the union on
• After the workshop, the final draft of the white paper will be published.
Before the workshop we will spread by mailing lists, social networks etc. a number of crucial thought provoking and outright provocative questions and challenges to the community with the objective of raising awareness and stimulating contributions on the topic. Thought provoking video from recent robotics competitions will be shared before and during the event.
Hybrid Event
PART I Agenda
– first 80 mn:
– intro + summary of the white paper (MS): 15 mn
– 10 pitches by invited speakers of 5 mn each + spare time: 65 mn
(presentations + Q&A)
Contributors: We have a long list: Matteo Matteucci, Enrica Zereik, Davide Scaramuzza, Agnes Delaborde, Isabel Ferreira, Tony Downs (NIST), Avrin Guillame, Ricardo Sanz, Antidio Viguria Jiménez, Alin Albu-Schäffer , Francesco Ferro
Further information: Will follow up and put on the web the WS as soon as approved.
Organisers:
· Fabio Bonsignorio, fabio.bonsignorio@gmail.com, Heron Robots and FER, UNIVERSITY OF ZAGREB
· DIEGO TORRICELLI, diego.torricelli@csic.es, CSIC
· PedroLima, pedro.lima@tecnico.ulisboa.pt, ISR/IST University of Lisboa
· Studley Matthew, Matthew2.Studley@uwe.ac.uk, BRL, University of the West of England
Event Timeslots (1)
Room 207 – Floor 2
-
14 March - Benchmarking and Competitions
Fabio Bonsignorio