In Proc. of Int. Conf. on Embedded Wireless Systems and Networks (EWSN 2024) - Workshop on Enabling Machine Learning Operations for next-Gen Embedded Wireless Networked Devices (EMERGE), Dec. 2024
This paper presents Federated Learning with Adaptive Monitoring and Elimination (FLAME), a novel solution capable of detecting and mitigating concept drift in Federated Learning (FL) Internet of Things (IoT) environments. Concept drift poses significant challenges for FL models deployed in dynamic and real-world settings. FLAME leverages an FL architecture, considers a real-world FL pipeline, and proves capable of maintaining model performance and accuracy while addressing bandwidth and privacy constraints. Introducing various features and extensions on previous works, FLAME offers a robust solution to concept drift, significantly reducing computational load and communication overhead. Compared to well-known lightweight mitigation methods, FLAME demonstrates superior performance in maintaining high F1 scores and reducing resource utilisation in large-scale IoT deployments, making it a promising approach for real-world applications.
Computing Within Limits: An Empirical Study of Energy Consumption in ML Training and Inference
In Proc. of International Scientific Conference on Information, Communication and Energy Systems and Technologies - Workshop on Artificial Intelligence for Sustainable Development, Jul. 2024
Machine learning (ML) has seen tremendous advancements, but its environmental footprint remains a concern. Acknowledging the growing environmental impact of ML this paper investigates Green ML, examining various model architectures and hyperparameters in both training and inference phases to identify energy-efficient practices. Our study leverages software-based power measurements for ease of replication across diverse configurations, models and datasets. In this paper, we examine multiple models and hardware configurations to identify correlations across the various measurements and metrics and key contributors to energy reduction. Our analysis offers practical guidelines for constructing sustainable ML operations, emphasising energy consumption and carbon footprint reductions while maintaining performance. As identified, short-lived profiling can quantify the long-term expected energy consumption. Moreover, model parameters can also be used to accurately estimate the expected total energy without the need for extensive experimentation.
Intelligent Routing as a Service (iRaaS): A Flexible Routing Framework for Knowledge-Defined Networks
In Proc. of IFIP/IEEE International Federation for Information Processing (IFIP) Networking, Jun. 2024
The scope of the Sixth-Generation Self-Organized Networks (6G-SON) advances its predecessor’s capability towards agility, flexibility, and adaptability. On-demand overlay network- ing technologies have shown a prominent maturity while coping with the rising complexity and scale of enterprise, service provider, and data centre networks. In the recent past, the Software-Defined Networking paradigm has offered Model Driven Programmability resulting in minimizing the network management complexity through automation and orchestration. However, leveraging Ma- chine Learning-driven network optimization, a.k.a. Knowledge- Defined Networking (KDN), has still been a domain of interest for the Network Softwarization research community. In this article, we propose Intelligent Routing as a Service (iRaaS) architecture as an application layer cognitive routing framework for KDNs. iRaaS offers routing logic customization (i.e., customizing metric function, path-discovery algorithm, etc.) and provides an option to include heuristic parameters from trained models as a part of the metric calculation. iRaaS sits on the application plane above the knowledge plane in a KDN stack, thus providing platform- and vendor-agnostic coupling with existing network infrastructures. This article covers the scope of iRaaS by using reliability as a heuristic for standard path-discovery algorithms e.g., Shortest Path First (SPF) and Diffusion Update algorithm (DUAL) along with the architectural specification. We validate our approach through a Proof-of-Concept deployment.
UMBRELLA: A One-stop Shop Bridging the Gap from Lab to Real-World IoT Experimentation
UMBRELLA is an open, large-scale IoT ecosystem deployed across South Gloucestershire, UK. It is intended to accelerate innovation across multiple technology domains. UMBRELLA is built to bridge the gap between existing specialised testbeds and address holistically real-world technological challenges in a System-of-Systems (SoS) fashion. UMBRELLA provides open access to real-world devices and infrastructure, enabling researchers and the industry to evaluate solutions for Smart Cities, Robotics, Wireless Communications, Edge Intelligence, and more. Key features include over 200 multi-sensor nodes installed on public infrastructure, a robotics arena with 20 mobile robots, a 5G network-in-a-box solution, and a unified backend platform for management, control and secure user access. The heterogeneity of hardware components, including diverse sensors, communication interfaces, and GPU-enabled edge devices, coupled with tools like digital twins, allows for comprehensive experimentation and benchmarking of innovative solutions not viable in lab environments. This paper provides a comprehensive overview of UMBRELLA’s multi-domain architecture and capabilities, making it an ideal playground for Internet of Things (IoT) and Industrial IoT (IIoT) innovation. It discusses the challenges in designing, developing and operating UMBRELLA as an open, sustainable testbed and shares lessons learned to guide similar future initiatives. With its unique openness, heterogeneity, realism and tools, UMBRELLA aims to continue accelerating cutting-edge technology research, development and translation into real-world progress.
Mitigating System Bias in Resource Constrained Asynchronous Federated Learning Systems
J. Gao, I. Mavromatis, P. Li, P. Carnelli, and A. Khan
In Proc. of IEEE PerCom Workshop on Pervasive and Resource-constrained AI (PeRConAI), Mar. 2024
Federated learning (FL) systems face performance challenges in dealing with heterogeneous devices and non-identically distributed data across clients. We propose a dynamic global model aggregation method within Asynchronous Federated Learning (AFL) deployments to address these issues. Our aggregation method scores and adjusts the weighting of client model updates based on their upload frequency to accommodate differences in device capabilities. Additionally, we also immediately provide an updated global model to clients after they upload their local models to reduce idle time and improve training efficiency. We evaluate our approach within an AFL deployment consisting of 10 simulated clients with heterogeneous compute constraints and non-IID data. The simulation results, using the FashionMNIST dataset, demonstrate over 10% and 19% improvement in global model accuracy compared to state-of-the-art methods PAPAYA and FedAsync, respectively. Our dynamic aggregation method allows reliable global model training despite limiting client resources and statistical data heterogeneity. This improves robustness and scalability for real-world FL deployments.
Past, Present, Future: A Comprehensive Exploration of AI Use Cases in the UMBRELLA IoT Testbed
In Proc. of IEEE PerCom Workshop on Pervasive Computing Challenges in Trustable Crowdsensing Systems (TrustSense), Mar. 2024
Invited Paper
UMBRELLA is a large-scale, open-access Internet of Things (IoT) ecosystem incorporating over 200 multi-sensor multi-wireless nodes, 20 collaborative robots, and edge-intelligence-enabled devices. This paper provides a guide to the implemented and prospective artificial intelligence (AI) capabilities of UMBRELLA in real-world IoT systems. Four existing UMBRELLA applications are presented in detail: 1) An automated streetlight monitoring for detecting issues and triggering maintenance alerts; 2) A Digital twin of building environments providing enhanced air quality sensing with reduced cost; 3) A large-scale Federated Learning framework for reducing communication overhead; and 4) An intrusion detection for containerised applications identifying malicious activities. Additionally, the potential of UMBRELLA is outlined for future smart city and multi-robot crowdsensing applications enhanced by semantic communications and multi-agent planning. Finally, to realise the above use-cases we discuss the need for a tailored MLOps platform to automate UMBRELLA’s model pipelines and establish trust.
Intrusion Detection at the IoT Edge using Federated Learning
Security and Privacy in Smart Environments, Springer, Jan. 2024
Invited Paper
With the proliferation of Internet of Things (IoT) technologies in urban environments, cities are increasingly deploying Edge processing nodes for urban sensing. This large-scale integration of Edge nodes and sensing endpoints raises significant security concerns. For instance, existing Intrusion Detection techniques cannot scale well and do not consider the privacy and energy consumption implications that emerge when applied to those systems. In addition, the use of containerised applications managed by container orchestration platforms in these environments, while enabling diverse applications and allowing scanning of the container images, can still introduce vulnerabilities. This Chapter addresses the challenge of effectively detecting such malicious activities in large-scale resource-constrained IoT systems. We introduce a semi-supervised distributed learning solution employing Federated Learning for real-time anomaly detection across the IoT infrastructure. Our approach involves analysing Linux system call data through a Federated Learning Framework, significantly reducing the need for central data processing. The Chapter presents a comprehensive architectural overview of the system, its core components, and the methodology for deploying and updating anomaly detection models. It also provides the performance evaluation of our approach. Our results demonstrate that the size of the clients datasets and the use of pre-trained models play a significant role in the performance of Federated Learning (FL) models in intrusion detection for large-scale IoT environments. The work presented in this chapter was supported by UK Research and Innovation, Innovate UK [grant number 53707]
2023
Cybersecurity in Motion: A Survey of Challenges and Requirements for Future Test Facilities of CAVs
I. Mavromatis, T. Spyridopoulos, P. Carnelli, W. H. Chin, A. Khalil, J. Chakravarty, L. Cipolina Kun, R. J. Piechocki, C. Robbins, D. Cunnington, L. Chase, L. Chiazor, C. Preston, Rahul, and A. Khan
EAI Endorsed Transactions on Industrial Networks and Intelligent Systems, Dec. 2023
The way we travel is changing rapidly and Cooperative Intelligent Transportation Systems (C-ITSs) are at the forefront of this evolution. However, the adoption of C-ITSs introduces new risks and challenges, making cybersecurity a top priority for ensuring safety and reliability. Building on this premise, this paper introduces an envisaged Cybersecurity Centre of Excellence (CSCE) designed to bolster researching, testing, and evaluating the cybersecurity of C-ITSs. We explore the design, functionality, and challenges of CSCE’s testing facilities, outlining the technological, security, and societal requirements. Through a thorough survey and analysis, we assess the effectiveness of these systems in detecting and mitigating potential threats, highlighting their flexibility to adapt to future C-ITSs. Finally, we identify current unresolved challenges in various C-ITS domains, with the aim of motivating further research into the cybersecurity of C-ITSs.
FROST: Towards Energy-efficient AI-on-5G Platforms - A GPU Power Capping Evaluation
In Proc. of IEEE Conference on Standards for Communications and Networking (CSCN), Nov. 2023
The Open Radio Access Network (O-RAN) is a burgeoning market with projected growth in the upcoming years. RAN has the highest CAPEX impact on the network and, most importantly, consumes 73% of its total energy. That makes it an ideal target for optimisation through the integration of Machine Learning (ML). However, the energy consumption of ML is frequently overlooked in such ecosystems. Our work addresses this critical aspect by presenting FROST - Flexible Reconfiguration method with Online System Tuning - a solution for energy-aware ML pipelines that adhere to O-RAN’s specifications and principles. FROST is capable of profiling the energy consumption of an ML pipeline and optimising the hardware accordingly, thereby limiting the power draw. Our findings indicate that FROST can achieve energy savings of up to 26.4% without compromising the model’s accuracy or introducing significant time delays.
Multi-stage Attack Detection and Prediction Using Graph Neural Networks: An IoT Feasibility Study
In Proc. of IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Nov. 2023
With the ever-increasing reliance on digital networks for various aspects of modern life, ensuring their security has become a critical challenge. Intrusion Detection Systems play a crucial role in ensuring network security, actively identifying and mitigating malicious behaviours. However, the relentless advancement of cyber-threats has rendered traditional/classical approaches insufficient in addressing the sophistication and complexity of attacks. This paper proposes a novel 3-stage intrusion detection system inspired by a simplified version of the Lockheed Martin cyber kill chain to detect advanced multi-step attacks. The proposed approach consists of three models, each responsible for detecting a group of attacks with common characteristics. The detection outcome of the first two stages is used to conduct a feasibility study on the possibility of predicting attacks in the third stage. Using the ToN IoT dataset, we achieved an average of 94% F1-Score among different stages, outperforming the benchmark approaches based on Random-forest model. Finally, we comment on the feasibility of this approach to be integrated in a real-world system and propose various possible future work.
Federated Deep Learning for Intrusion Detection in IoT Networks
In Proc. of IEEE Global Communications Conference (GLOBECOM), Nov. 2023
The vast increase of IoT technologies and the ever-evolving attack vectors and threat actors have increased cyber-security risks dramatically. Novel attacks can compromise IoT devices to gain access to sensitive data or control them to deploy further malicious activities. The detection of novel attacks often relies upon AI solutions. A common approach to implementing AI-based IDS in distributed IoT systems is in a centralised manner. However, this approach may violate data privacy and secrecy. In addition, centralised data collection prohibits the scale-up of IDSs. Therefore, intrusion detection solutions in IoT ecosystems need to move towards a decentralised direction. FL has attracted significant interest in recent years due to its ability to perform collaborative learning while preserving data confidentiality and locality. Nevertheless, most FL-based IDS for IoT systems are designed under unrealistic data distribution conditions. To that end, we design an experiment representative of the real world and evaluate the performance of two FL IDS implementations, one based on DNNs and another on our previous work on DBNs. For our experiments, we rely on TON-IoT, a realistic IoT network traffic dataset, associating each IP address with a single FL client. Additionally, we explore pre-training and investigate various aggregation methods to mitigate the impact of data heterogeneity. Lastly, we benchmark our approach against a centralised solution. The comparison shows that the heterogeneous nature of the data has a considerable negative impact on the model performance when trained in a distributed manner. However, in the case of a pre-trained initial global FL model, we demonstrate a performance improvement of over 20% (F1-score) when compared against a randomly initiated global model.
Evaluating Concept Drift Detectors on Real-World Data
In Proc. of ACM Embedded Wireless Systems and Networks (EWSN), Sep. 2023
Cloud-IoT deployments are ubiquitous and employed in various application domains, including smart buildings. Often employed in public spaces, IoT devices are exposed to various security threats. One such attack is "anomalous concept drift". It occurs when an attacker tampers with a device causing it to report realistic sensor data that slowly deviates from the correct value. Evaluating concept drift detectors on real-world data is ideal. Though many indoor datasets exist, our real-world dataset provides a natural, long-term collection of indoor environmental sensor readings over six months. The dataset consists of environmental sensor samples collected via eight IoT devices in a real office setting. The dataset is particularly useful for evaluating concept drift detection algorithms as spatial aspects can be used along with the signals. The dataset has been made openly available, and in this paper we use it to inject malicious concept drifts and to evaluate the performance of several drift detection techniques. The injection tool’s source code is also publicly available.
FLARE: Detection and Mitigation of Concept Drift for Federated Learning based IoT Deployments
In Proc. of International Wireless Communications & Mobile Computing Conference (IWCMC) 2023, Jun. 2023
Intelligent, large-scale IoT ecosystems have become possible due to recent advancements in sensing technologies, distributed learning, and low-power inference in embedded devices. In traditional cloud-centric approaches, raw data is transmitted to a central server for training and inference purposes. On the other hand, Federated Learning migrates both tasks closer to the edge nodes and endpoints. This allows for a significant reduction in data exchange while preserving the privacy of users. Trained models, though, may under-perform in dynamic environments due to changes in the data distribution, affecting the model’s ability to infer accurately; this is referred to as concept drift. Such drift may also be adversarial in nature. Therefore, it is of paramount importance to detect such behaviours promptly. In order to simultaneously reduce communication traffic and maintain the integrity of inference models, we introduce FLARE, a novel lightweight dual-scheduler FL framework that conditionally transfers training data, and deploys models between edge and sensor endpoints based on observing the model’s training behaviour and inference statistics, respectively. We show that FLARE can significantly reduce the amount of data exchanged between edge and sensor nodes compared to fixed-interval scheduling methods (over 5x reduction), is easily scalable to larger systems, and can successfully detect concept drift reactively with at least a 16x reduction in latency.
Multi-sensor, Multi-device Smart Building Indoor Environmental Dataset
A dataset of sensor measurements is presented. Our dataset contains discrete measurements of 8 IoT devices located in various places in a research lab at the University of Bristol. Nordic nRF52840 DK IoT devices periodically collects environmental data, such as temperature, humidity, pressure, gas, room light intensity, accelerometer; including also a measurement quality indicator. The measurements were taken every 10 seconds over a six-month period between February and September 2022. In addition, we provide Received Signal Strength Indicator (RSSI) of the IoT devices. The data files are formatted as CSV files. There are various software libraries available to access and read this file format. We provide “README.txt” file which explains the repository and how to use dataset. Each data file is named according to its creation date and, once it reaches a size of 1MB, it is compressed and archived. A new folder is created every week to store all the data files from that week automatically. The dataset can be used for drift detection such as malicious or anomaly detection algorithms. It can also be used for smart building applications like occupation detection. The dataset can be found at https://data.bris.ac.uk/data/dataset/fwlmb11wni392kodtyljkw4n2
A Dataset of Human Body Tracking of Walking Actions Captured using two Azure Kinect Sensors
A dataset of body tracking information is presented. The dataset consists of 315 captured walking sequences. Each sequence is simultaneously captured by two Azure Kinect devices. The two captures are interleaved to effectively double the frame rate. Fifteen participants partook in this experiment. Each experiment consists of seven walking actions, and having three predefined trajectories per experiment. That results in 21 sequences per participant. The data were collected using the Azure Kinect Sensor SDK. They were later processed using the official tools and libraries provided by Microsoft. For each sequence and trajectory, the positions and orientations of thirty-two tracked joints were obtained and saved. The dataset is structured as follows. The experiments from each subject are saved in a single directory. Each directory contains multiple JSON files of timestamped body tracking information to enable the fusion of the two device streams. A calibration file is also provided, enabling the mapping of the coordinates between the two Azure Kinect devices capturing the data (mapping the coordinates of the device known as the Subordinate device to the Master device coordinate system). This data can be used to train neural networks for human motion prediction tasks or test pre-existing algorithms on Azure Kinect data. This dataset could also aid in gait recognition and analysis, as well as in performing action recognition and other surveillance activities. The dataset can be found at https://zenodo.org/record/7997856.
Demo: LE3D: A Privacy-preserving Lightweight Data Drift Detection Framework
In Proc. of IEEE Consumer Communications & Networking Conference (CCNC), Jan. 2023
This paper presents LE3D; a novel data drift detection framework for preserving data integrity and confidentiality. LE3D is a generalisable platform for evaluating novel drift detection mechanisms within the Internet of Things (IoT) sensor deployments. Our framework operates in a distributed manner, preserving data privacy while still being adaptable to new sensors with minimal online reconfiguration. Our framework currently supports multiple drift estimators for time-series IoT data and can easily be extended to accommodate new data types and drift detection mechanisms. This demo will illustrate the functionality of LE3D under a real-world-like scenario.
LE3D: A Lightweight Ensemble Framework of Data Drift Detectors for Resource-Constrained Devices
In Proc. of IEEE Consumer Communications & Networking Conference (CCNC), Jan. 2023
Data integrity becomes paramount as the number of Internet of Things (IoT) sensor deployments increases. Sensor data can be altered by benign causes or malicious actions. Mechanisms that detect drifts and irregularities can prevent disruptions and data bias in the state of an IoT application. This paper presents LE3D, an ensemble framework of data drift estimators capable of detecting abnormal sensor behaviours. Working collaboratively with surrounding IoT devices, the type of drift (natural/abnormal) can also be identified and reported to the end-user. The proposed framework is a lightweight and unsupervised implementation able to run on resource-constrained IoT devices. Our framework is also generalisable, adapting to new sensor streams and environments with minimal online reconfiguration. We compare our method against state-of-the-art ensemble data drift detection frameworks, evaluating both the real-world detection accuracy as well as the resource utilisation of the implementation. Experimenting with real-world data and emulated drifts, we show the effectiveness of our method, which achieves up to 97% of detection accuracy while requiring minimal resources to run.
2022
Energy-Rate-Quality Tradeoffs of State-of-the-Art Video Codecs
In Proc. of Picture Coding Symposium (PCS), Dec. 2022
The adoption of video conferencing and video communication services, accelerated by COVID-19, has driven a rapid increase in video data traffic. The demand for higher resolutions and quality, the need for immersive video formats, and the newest, more complex video codecs increase the energy consumption in data centers and display devices. In this paper, we explore and compare the energy consumption across optimized state-of-the-art video codecs, SVT-AV1, VVenC/VVdeC, VP9, and x.265. Furthermore, we align the energy usage with various objective quality metrics and the compression performance for a set of video sequences across different resolutions. The results indicate that SVT-AV1 provides the best tradeoff between energy consumption and quality. The reported results aim to serve as a guide towards sustainable video streaming while not compromising the quality of experience of the end user.
Resource-Interaction Graph: Efficient Graph Representation for Anomaly Detection
Security research has concentrated on converting operating system audit logs into suitable graphs, such as provenance graphs, for analysis. However, provenance graphs can grow very large requiring significant computational resources beyond what is necessary for many security tasks and are not feasible for resource constrained environments, such as edge devices. To address this problem, we present the \textitresource-interaction graph that is built directly from the audit log. We show that the resource-interaction graph’s storage requirements are significantly lower than provenance graphs using an open-source data set with two container escape attacks captured from an edge device. We use a graph autoencoder and graph clustering technique to evaluate the representation for an anomaly detection task. Both approaches are unsupervised and are thus suitable for detecting zero-day attacks. The approaches can achieve f1 scores typically over 80% and in some cases over 90% for the selected data set and attacks.
A Dataset of Images of Public Streetlights with Operational Monitoring using Computer Vision Techniques
A dataset of street light images is presented. Our dataset consists of ∼350k images, taken from 140 UMBRELLA nodes installed in the South Gloucestershire region in the UK. Each UMBRELLA node is installed on the pole of a lamppost and is equipped with a Raspberry Pi Camera Module v1 facing upwards towards the sky and lamppost light bulb. Each node collects an image at hourly intervals for 24h every day. The data collection spans for a period of six months.
Each image taken is logged as a single entry in the dataset along with the Global Positioning System (GPS) coordinates of the lamppost. All entries in the dataset have been post-processed and labelled based on the operation of the lamppost, i.e., whether the lamppost is switched ON or OFF. The dataset can be used to train deep neural networks and generate pre-trained models providing feature representations for smart city CCTV applications, smart weather detection algorithms, or street infrastructure monitoring. The dataset can be found at https://doi.org/10.5281/zenodo.6046758.
A Federated Learning-enabled Smart Street Light Monitoring Application: Benefits and Future Challenges
D. Anand, I. Mavromatis, P. E. Carnelli, and A. Khan
In Proc. of ACM MobiCom - Workshop on Al Empowered Mobile and Wireless Sensing (MORSE), Oct. 2022
Data-enabled cities are recently accelerated and enhanced with automated learning for improved Smart Cities applications. In the context of an Internet of Things (IoT) ecosystem, the data communication is frequently costly, inefficient, not scalable and lacks security. Federated Learning (FL) plays a pivotal role in providing privacy-preserving and communication efficient Machine Learning (ML) frameworks. In this paper we evaluate the feasibility of FL in the context of a Smart Cities Street Light Monitoring application. FL is evaluated against benchmarks of centralised and (fully) personalised machine learning techniques for the classification task of the lampposts operation. Incorporating FL in such a scenario shows minimal performance reduction in terms of the classification task, but huge improvements in the communication cost and the privacy preserving. These outcomes strengthen FL’s viability and potential for IoT applications.
Securing Synchronous Flooding Communications: An Atomic-SDN Implementation
In Proc. of ACM Embedded Wireless Systems and Networks (EWSN), Oct. 2022
Synchronous Flooding (SF) protocols can enhance the wireless connectivity between Internet of Things (IoT) devices. However, existing SF solutions fail to introduce sufficient security measures due to strict time synchronisation requirements, making them vulnerable to malicious actions. Our paper presents a design paradigm for encrypted SF communications. We describe a mechanism for synchronising encryption parameters in a network-wide fashion. Our solution operates with minimal overhead and without compromising communication reliability. Evaluating our paradigm on a real-world, large-scale IoT testbed, we have proven that a communication layer impervious to a range of attacks is established without sacrificing the network performance.
Reliable IoT Firmware Updates: A Large-scale Mesh Network Performance Investigation
In Proc. of IEEE Wireless Communications and Networking Conference (WCNC), Apr. 2022
Internet of Things (IoT) networks require regular firmware updates to ensure enhanced security and stability. As we move towards methodologies of codifying security and policy decisions and exchanging them over IoT large-scale deployments (security-as-a-code), these demands should be considered a routine operation. However, rolling out firmware updates to large-scale networks presents a crucial challenge for constrained wireless environments with large numbers of IoT devices. This paper initially investigates how the current state-of-the-art protocols operate in such adverse conditions by measuring various Quality-of-Service (QoS) Key Performance Indicators (KPIs) of the shared wireless medium. We later discuss how Concurrent Transmissions (CT) can extend the scalability of IoT protocols and ensure reliable firmware roll-outs over large geographical areas. Measuring KPIs such as the mesh join time, the throughput, and the number of nodes forming a network, we provide great insight into how an IoT environment will behave under a large-scale firmware roll-out. Finally, we conducted our performance investigation over the UMBRELLA platform, a real-world IoT testbed deployed in Bristol, UK. This ensures our findings represent a realistic IoT scenario and meet the strict QoS requirements of today’s IoT applications.
System and Method for Detecting and Rectifying Concept Drift in Federated Learning
A. Khan, P. E. Carnelli, Timothy D. Farnham, I. Mavromatis, and A. Portelli
United States, US20220083916A1, Mar. 2022
A computer-implemented method for identifying and rectifying a machine learning drift in a federated learning deployment comprising a parameter server and a plurality of worker nodes, wherein a first worker node comprises: a first machine learning model trained using a first data source; and a second machine learning model trained using a second data source; wherein the first data source is generated by the first worker node and the second data source is generated by a second worker node; the method comprising calculating, by the first worker node, using a trusted data set, a first performance metric associated with the first machine learning model and a second performance metric associated with the second machine learning model and determining, by the first worker node, whether a potential drift has occurred in at least one of the first and the second machine learning models.
2021
Dataset: Container Escape Detection for Edge Devices
In Proc. of ACM Embedded Networked Sensor Systems (SenSys), Nov. 2021
Edge computing is rapidly changing the IoT-Cloud landscape. Various testbeds are now able to run multiple Docker-like containers developed and deployed by end-users on edge devices. However, this capability may allow an attacker to deploy a malicious container on the host and compromise it. This paper presents a dataset based on the Linux Auditing System, which contains malicious and benign container activity. We developed two malicious scenarios, a denial of service and a privilege escalation attack, where an adversary uses a container to compromise the edge device. Furthermore, we deployed benign user containers to run in parallel with the malicious containers. Container activity can be captured through the host system via system calls. Our time series auditd dataset contains partial labels for the benign and malicious related system calls. Generating the dataset is largely automated using a provided AutoCES framework. We also present a semi-supervised machine learning use case with the collected data to demonstrate its utility. The dataset and framework code are open-source and publicly available.
UMBRELLA Collaborative Robotics Testbed and IoT Platform
In Proc. of IEEE Workshop on Communication and Networking for Swarms Robotics (RoboCom), Jan. 2021
This paper provides details of the collaborative robotics testbed platform that has been developed within the UMBRELLA project. The testbed is part of a larger open Industrial IoT testbed which is currently being deployed in the UK. The aim of the testbed is to permit flexible experimentation using different end devices (including smart city sensing and robot nodes) to evaluate algorithms or new practical application scenarios. For the collaborative robotics testbed this relates to warehouse robotics, which can move pallets of different sizes and shapes. The testbed also includes simulator facilities for validation of algorithms prior to deployment on the robot nodes. The nodes support a rich set of sensors, actuators, and wireless communication technologies. The software architecture is based on Docker containers and ROS2 DDS middleware for flexible and extensible evolution to support future sensors or network technologies. This will be provided as an open testbed to support research, experimentation and evaluation of swarm robotics and other Industrial IoT use-cases.
2020
DRIVE: A Digital Network Oracle for Cooperative Intelligent Transportation Systems
In Proc. of IEEE Symposium on Computers and Communications (ISCC), Jul. 2020
In a world where Artificial Intelligence revolutionizes inference, prediction and decision-making tasks, Digital Twins emerge as game-changing tools. A case in point is the development and optimization of Cooperative Intelligent Transportation Systems (C-ITSs): a confluence of cyber-physical digital infrastructure and (semi)automated mobility. Herein we introduce Digital Twin for self-dRiving Intelligent VEhicles (DRIVE). The developed framework tackles shortcomings of traditional vehicular and network simulators. It provides a flexible, modular, and scalable implementation to ensure large-scale, city-wide experimentation with a moderate computational cost. The defining feature of our Digital Twin is a unique architecture allowing for submission of sequential queries, to which the Digital Twin provides instantaneous responses with the "state of the world", and hence is an Oracle. With such bidirectional interaction with external intelligent agents and realistic mobility traces, DRIVE provides the environment for development, training and optimization of Machine Learning based C-ITS solutions.
On Urban Traffic Flow Benefits of Connected and Automated Vehicles
In Proc. of IEEE Vehicular Technology Conference (VTC2020-Spring), May 2020
Automated Vehicles are an integral part of Intelligent Transportation Systems (ITSs) and are expected to play a crucial role in the future mobility services. This paper investigates two classes of self-driving vehicles: (i) Level 4&5 Automated Vehicles (AVs) that rely solely on their on-board sensors for environmental perception tasks, and (ii) Connected and Automated Vehicles (CAVs), leveraging connectivity to further enhance perception via driving intention and sensor information sharing. Our investigation considers and quantifies the impact of each vehicle group in large urban road networks in Europe and in the USA. The key performance metrics are the traffic congestion, average speed and average trip time. Specifically, the numerical studies show that the traffic congestion can be reduced by up to a factor of four, while the average flow speeds of CAV group remains closer to the speed limits and can be up to 300% greater than the human-driven vehicles. Finally, traffic situations are also studied, indicating that even a small market penetration of CAVs will have a substantial net positive effect on the traffic flows.
2019
Location Anomalies Detection for Connected and Autonomous Vehicles
In Proc. of IEEE Connected and Automated Vehicles Symposium (CAVS), Sep. 2019
Future Connected and Automated Vehicles (CAVs), and more generally ITS, will form a highly interconnected system. Such a paradigm is referred to as the Internet of Vehicles (herein Internet of CAVs) and is a prerequisite to orchestrate traffic flows in cities. For optimal decision making and supervision, traffic centres will have access to suitably anonymized CAV mobility information. Safe and secure operations will then be contingent on early detection of anomalies. In this paper, a novel unsupervised learning model based on deep autoencoder is proposed to detect the self-reported location anomaly in CAVs, using vehicle locations and the Received Signal Strength Indicator (RSSI) as features. Quantitative experiments on simulation datasets show that the proposed approach is effective and robust in detecting self-reported location anomalies.
Operating ITS-G5 DSRC over Unlicensed Bands: A City-Scale Performance Evaluation
In Proc. of IEEE Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Sep. 2019
Future Connected and Autonomous Vehicles (CAVs) will be equipped with a large set of sensors. The large amount of generated sensor data is expected to be exchanged with other CAVs and the road-side infrastructure. Both in Europe and the US, Dedicated Short Range Communications (DSRC) systems, based on the IEEE 802.11p Physical Layer, are key enabler for the communication among vehicles. Given the expected market penetration of connected vehicles, the licensed band of 75 MHz, dedicated to DSRC communications, is expected to become increasingly congested. In this paper, we investigate the performance of a vehicular communication system, operated over the unlicensed bands 2.4 GHz-2.5 GHz and 5.725 GHz-5.875 GHz. Our experimental evaluation was carried out in a testing track in the centre of Bristol, UK and our system is a full-stack ETSI ITS-G5 implementation. Our performance investigation compares key communication metrics (e.g., packet delivery rate, received signal strength indicator) measured by operating our system over the licensed DSRC an the considered unlicensed bands. In particular, when operated over the 2.4 GHz-2.5 GHz band, our system achieves comparable performance to the case when the DSRC band is used. On the other hand, as soon as the system, is operated over the 5.725 GHz-5.875 GHz band, the packet delivery rate is 30% smaller compared to the case when the DSRC band is employed. These findings prove that operating our system over unlicensed ISM bands is a viable option. During our experimental evaluation, we recorded all the generated network interactions and the complete data set has been publicly available.
A Dataset of Full-stack ITS-G5 DSRC Communications over Licensed and Unlicensed Bands using a Large-scale Urban Testbed
A dataset of measurements of ETSI ITS-G5 Dedicated Short Range Communications (DSRC) is presented. Our dataset consists of network interactions happening between two On-Board Units (OBUs) and four Road Side Units (RSUs). Each OBU was fitted onto a vehicle driven across the FLOURISH Test Track in Bristol, UK. Each RSU and OBU was equipped with two transceivers operating at different frequencies. During our experiments, each transceiver broadcasts Cooperative Awareness Messages (CAMs) over the licensed DSRC band, and over the unlicensed Industrial, Scientific, and Medical radio (ISM) bands 2.4 GHz-2.5 GHz and 5.725 GHz-5.875 GHz. Each transmitted and received CAM is logged along with its Received Signal Strength Indicator (RSSI) value and accurate positioning information. The Media Access Control layer (MAC) layer Packet Delivery Rates (PDRs) and RSSI values are also empirically calculated across the whole length of the track for any transceiver. The dataset can be used to derive realistic approximations of the PDR as a function of RSSI values under urban environments and for both the DSRC and ISM bands – thus, the dataset is suitable to calibrate (simplified) physical layers of full-stack vehicular simulators where the MAC layer PDR is a direct function of the RSSI. The dataset is not intended to be used for signal propagation modelling. The dataset can be found at https://doi.org/10.5523/bris.eupowp7h3jl525yxhm3521f57, and it has been analyzed in the following paper: I. Mavromatis, A. Tassi, and R. J. Piechocki, “Operating ITS-G5 DSRC over Unlicensed Bands: A City-Scale Performance Evaluation,” IEEE PIMRC 2019. [Online]. Available: https://arxiv.org/abs/1904.00464.
Secure Data Offloading Strategy for Connected and Autonomous Vehicles
In Proc. of IEEE Vehicular Technology Conference (VTC2019-Spring), Apr. 2019
Connected and Automated Vehicles (CAVs) are expected to constantly interact with a network of processing nodes installed in secure cabinets located at the side of the road - thus, forming Fog Computing-based infrastructure for Intelligent Transportation Systems (ITSs). Future city-scale ITS services will heavily rely upon the sensor data regularly off-loaded by each CAV on the Fog Computing network. Due to the broadcast nature of the medium, CAVs’ communications can be vulnerable to eavesdropping. This paper proposes a novel data offloading approach where the Random Linear Network Coding (RLNC) principle is used to ensure the probability of an eavesdropper to recover relevant portions of sensor data is minimized. Our preliminary results confirm the effectiveness of our approach when operated in a large-scale ITS networks.
Agile Data Offloading over Novel Fog Computing Infrastructure for CAVs
In Proc. of IEEE Vehicular Technology Conference (VTC2019-Spring), Apr. 2019
IEEE Best Paper Award
Future Connected and Automated Vehicles (CAVs) will be supervised by cloud-based systems overseeing the overall security and orchestrating traffic flows. Such systems rely on data collected from CAVs across the whole city operational area. This paper develops a Fog Computing-based infrastructure for future Intelligent Transportation Systems (ITSs) enabling an agile and reliable off-load of CAV data. Since CAVs are expected to generate large quantities of data, it is not feasible to assume data off-loading to be completed while a CAV is in the proximity of a single Road-Side Unit (RSU). CAVs are expected to be in the range of an RSU only for a limited amount of time, necessitating data reconciliation across different RSUs, if traditional approaches to data off-load were to be used. To this end, this paper proposes an agile Fog Computing infrastructure, which interconnects all the RSUs so that the data reconciliation is solved efficiently as a by-product of deploying the Random Linear Network Coding (RLNC) technique. Our numerical results confirm the feasibility of our solution and show its effectiveness when operated in a large-scale urban testbed.
Efficient Millimeter-Wave Infrastructure Placement for City-Scale ITS
In Proc. of IEEE Vehicular Technology Conference (VTC2019-Spring), Apr. 2019
Millimeter Waves (mmWaves) will play a pivotal role in the next-generation of Intelligent Transportation Systems (ITSs). However, in deep urban environments, sensitivity to blockages creates the need for more sophisticated network planning. In this paper, we present an agile strategy for deploying road-side nodes in a dense city scenario. In our system model, we consider strict Quality-of-Service (QoS) constraints (e.g. high throughput, low latency) that are typical of ITS applications. Our approach is scalable, insofar that takes into account the unique road and building shapes of each city, performing well for both regular and irregular city layouts. It allows us not only to achieve the required QoS constraints but it also provides up to 50% reduction in the number of nodes required, compared to existing deployment solutions.
2018
Poster: Parallel Implementation of the OMNeT++ INET Framework for V2X Communications
In Proc. of IEEE Vehicular Networking Conference (VNC), Dec. 2018
IEEE Popularity Award
The field of parallel network simulation frameworks is evolving at a great pace. That is also because of the growth of Intelligent Transportation Systems (ITS) and the necessity for cost-effective large-scale trials. In this contribution, we will focus on the INET Framework and how we re-factor its single-thread code to make it run in a multi-thread fashion. Our parallel version of the INET Framework can significantly reduce the computation time in city-scale scenarios, and it is completely transparent to the user. When tested in different configurations, our version of INET ensures a reduction in the computation time of up to 43%.
5G Communication Framework for Smarter Autonomous Vehicles
Mavromatis, I.
PhD Thesis, 2018
A City-Scale ITS-G5 Network for Next-Generation Intelligent Transportation Systems: Design Insights and Challenges
In Proc. of Ad-hoc, Mobile, and Wireless Networks, Aug. 2018
As we move towards autonomous vehicles, a reliable Vehicle-to-Everything (V2X) communication framework becomes of paramount importance. In this paper we present the development and the performance evaluation of a real-world vehicular networking testbed. Our testbed, deployed in the heart of the City of Bristol, UK, is able to exchange sensor data in a V2X manner. We will describe the testbed architecture and its operational modes. Then, we will provide some insight pertaining the firmware operating on the network devices. The system performance has been evaluated under a series of large-scale field trials, which have proven how our solution represents a low-cost high-quality framework for V2X communications. Our system managed to achieve high packet delivery ratios under different scenarios (urban, rural, highway) and for different locations around the city. We have also identified the instability of the packet transmission rate while using single-core devices, and we present some future directions that will address that.
Efficient V2V Communication Scheme for 5G mmWave Hyper-Connected CAVs
In Proc. of IEEE International Conference on Communications Workshops (ICC Workshops), May 2018
Connected and Autonomous Vehicles (CAVs) require continuous access to sensory data to perform complex high-speed maneuvers and advanced trajectory planning. High priority CAVs are particularly reliant on extended perception horizon facilitated by sensory data exchange between CAVs. Existing technologies such as the Dedicated Short Range Communications (DSRC) are ill-equipped to provide advanced cooperative perception service. This creates the need for more sophisticated technologies such as the 5G Millimetre-Waves (mmWaves). In this work, we propose a distributed Vehicle-to-Vehicle (V2V) mmWaves association scheme operating in a heterogeneous manner. Our system utilises the information exchanged within the DSRC frequency band to bootstrap the best CAV pairs formation. Using a Stable Fixtures Matching Game, we form V2V multipoint-to-multipoint links. Compared to more traditional point-to-point links, our system provides almost twice as much sensory data exchange capacity for high priority CAVs while doubling the mmWaves channel utilisation for all the vehicles in the network.
Multi-Radio 5G Architecture for Connected and Autonomous Vehicles: Application and Design Insights
EAI Endorsed Transactions on Industrial Networks and Intelligent Systems, Mar. 2018
Invited Paper
Connected and Autonomous Vehicles (CAVs) will play a crucial role in next-generation Cooperative Intelligent Transportation Systems (C-ITSs). Not only is the information exchange fundamental to improve road safety and efficiency, but it also paves the way to a wide spectrum of advanced ITS applications enhancing efficiency, mobility and accessibility. Highly dynamic network topologies and unpredictable wireless channel conditions entail numerous design challenges and open questions. In this paper, we address the beneficial interactions between CAVs and an ITS and propose a novel architecture design paradigm. Our solution can accommodate multi-layer applications over multiple Radio Access Technologies (RATs) and provide a smart configuration interface for enhancing the performance of each RAT.
2017
Agile Calibration Process of Full-stack Simulation Frameworks for V2X Communications
In Proc. of IEEE Vehicular Networking Conference (VNC), Nov. 2017
Computer simulations and real-world car trials are essential to investigate the performance of Vehicle-to-Everything (V2X) networks. However, simulations are imperfect models of the physical reality and can be trusted only when they indicate agreement with the real-world. On the other hand, trials lack reproducibility and are subject to uncertainties and errors. In this paper, we will illustrate a case study where the interrelationship between trials, simulation, and the reality-of-interest is presented. Results are then compared in a holistic fashion. Our study will describe the procedure followed to macroscopically calibrate a full-stack network simulator to conduct high-fidelity full-stack computer simulations.
MmWave System for Future ITS: A MAC-Layer Approach for V2X Beam Steering
In Proc. of IEEE Vehicular Technology Conference (VTC-Fall), Sep. 2017
Millimetre Waves (mmWave) systems have the potential of enabling multi-gigabit-per-second communications in future Intelligent Transportation Systems (ITSs). Unfortunately, because of the increased vehicular mobility, they require frequent antenna beam realignments - thus significantly increasing the in-band Beamforming (BF) overhead. In this paper, we propose Smart Motion-prediction Beam Alignment (SAMBA), a MAC-layer algorithm that exploits the information broadcast via DSRC beacons by all vehicles. Based on this information, overhead-free BF is achieved by estimating the position of the vehicle and predicting its motion. Moreover, adapting the beamwidth with respect to the estimated position can further enhance the performance. Our investigation shows that SAMBA outperforms the IEEE 802.11ad BF strategy, increasing the data rate by more than twice for sparse vehicle density while enhancing the network throughput proportionally to the number of vehicles. Furthermore, SAMBA was proven to be more efficient compared to legacy BF algorithm under highly dynamic vehicular environments and hence, a viable solution for future ITS services.
Beam Alignment for Millimetre Wave Links with Motion Prediction of Autonomous Vehicles
In Proc. of Antennas, Propagation RF Technology for Transport and Autonomous Platforms, Feb. 2017
Invited Paper
Intelligent Transportation Systems (ITSs) require ultra-low end-to-end delays and multi-gigabit-per-second data transmission. Millimetre Waves (mmWaves) communications can fulfil these requirements. However, the increased mobility of Connected and Autonomous Vehicles (CAVs), requires frequent beamforming - thus introducing increased overhead. In this paper, a new beamforming algorithm is proposed able to achieve overhead-free beamforming training. Leveraging from the CAVs sensory data, broadcast with Dedicated Short Range Communications (DSRC) beacons, the position and the motion of a CAV can be estimated and beamform accordingly. To minimise the position errors, an analysis of the distinct error components was presented. The network performance is further enhanced by adapting the antenna beamwidth with respect to the position error. Our algorithm outperforms the legacy IEEE 802.11ad approach proving it a viable solution for the future ITS applications and services.
2015
Studying Quality of Experience (QoE) over Wireless Networks
Encyclopedia of Information Science and Technology, Third Edition, IGI Global, Dec. 2015
As stated earlier, QoE is subjective, user-centric and content-dependent. Now is the time that QoS firstly, and more terms later come into foreground. Under QoS the idea is that transmission rates, error rates and other characteristics can be measured, improved and guaranteed in advance. By using parameters as bit rate, jitter, delay, packet loss, a service can be described and evaluated. Later in this article, the differences between QoS and QoE will be discussed in more detail. In addition with QoS, the term Class of Service (CoS) can also be found in the literature in order to describe the semantics and parameters of a specific type of QoS. Applications and services are categorized as symmetrical or asymmetrical judging by the equality of the bandwidth in both uplink and downlink directions. Also elastic and non-elastic characterizations where used, based on the minimum level of bandwidth needed and finally interactive and non-interactive depending on the human contact with a service Grade of Service (GoS) describes all the phenomena occurring during connection setup, release and maintenance. It appears mostly in circuit switched optical services and is the probability of the service to be blocked or delayed for more than a specific interval due to the high traffic intensity and the lack of available resources.Finally, Quality of Resilience (QoR) is perceived as one of the dimensions of QoS and covers the gray areas when a service is operational but not fully functional and is characterized as degraded. Approaches related to QoR are not well formalized as the ones in QoS, CoS, or GoS.
2014
Propagation Modelling using Iterative Physical Optics