Multimodal Extreme Scale Data Analytics for Smart Cities Environments

Challenge

The “Smart City” paradigm aims to support new forms of monitoring and managing of resources as well as to provide situational awareness in decision-making fulfilling the objective of servicing the citizen, while ensuring that it meets the needs of present and future generations with respect to economic, social and environmental aspects. Considering the city as a complex and dynamic system involving different interconnected spatial, social, economic, and physical processes subject to temporal changes and continually modified by human actions. Big Data, fog, and edge computing technologies have significant potential in various scenarios considering each city individual tactical strategy. However, one critical aspect is to encapsulate the complexity of a city and support accurate, cross-scale and in-time predictions based on the ubiquitous spatio-temporal data of high-volume, high-velocity and of high-variety.

Objectives

MARVEL delivers a disruptive Edge-to-Fog-to-Cloud ubiquitous computing framework that enables multi-modal perception and intelligence for audio-visual scene recognition, event detection in a smart city environment. MARVEL aims to collect, analyse and data-mine multi-modal audio-visual data streams of a Smart City and help decision makers to improve the quality of life and services to the citizens without violating ethical and privacy limits in an AI-responsible manner. This is achieved via: 1) fusing large scale distributed multi-modal audio-visual data in real-time; 2) achieving fast time-to-insights; 3) supporting automated decision making at all levels of the E2F2C stack; and 4) delivering a personalized federated learning approach, where joint multi modal representations and models are co-designed and improved continuously through privacy aware sharing of personalized fog and edge models of all interested parties.

Our Role

Besides ensuring compliance with relevant H2020 rules on ethics, privacy and data protection, Privanova will also address legal aspects of processing personal data for research purposes during the entire project life-cycle. In particular, by implementing the Privacy-by-Design approach to data protection, management within PN will 1) constantly safeguard privacy and other rights of individuals participating in our research activities, 2) provide monitoring and feedback to project partners responsible for technology development so they can implement relevant privacy safeguards from the outset, and 3) ensure compliance of project outcomes with legal requirements thus enhancing their commercial exploitability. Finally we manage the project’s Ethics Advisory Board and interact with its members.

This project has received funding from the European Union’s Horizon 2020 research and Innovation programme under grant agreement N°957337. All information on this website reflects only the authors’ view. The Agency and the Commission are not responsible for any use that may be made of the information this website contains.