IPSN23: The 22nd International Conference on Information Processing in Sensor Networks

Covert channel, which can break the logical protections of the computer system and leak confidential or sensitive information, has long been considered a security issue in the network research community. However, recent research has shown that cooperative agents can use the "covert" channel to augment the communication of legitimate applications, rather than by adversaries seeking to compromise computer security. This further broadens the potential applications of covert channels. Despite this, the design and implementation of covert channels in the context of Low Power Wide Area Networks (LPWANs) have not been widely discussed. Current state-of-the-art uses On-off keying (OOK) on LoRa PHY to create a covert channel, but this channel has limited transmission distance and capacity. In this paper, we propose LoPhy, a resilient and fast covert channel over LoRa physical layer (PHY). LoPhy uses the Chirp Spreading Spectrum (CSS) modulation scheme to increase its resilience and explore the trade-off between the covert channel’s capacity and the legitimate channel’s resilience. We implement the proposed covert channel on off-the-shelf devices and software-defined radios and show that LoPhy achieves a 0.57% bit error rate at a distance of 700 m without affecting the legitimate channel’s performance. Moreover, we present two applications enabled by LoPhy to demonstrate the potential of LoPhy. Compared with the state-of-the-art, LoPhy brings up to 18 × reduction of bit errors and 63 × gain on noise resilience.

LoRa has emerged as one of the promising long-range and low-power wireless communication technologies for Internet of Things (IoT). With the massive deployment of LoRa networks, the ability to perform Firmware Update Over-The-Air (FUOTA) is becoming a necessity for unattended LoRa devices. LoRa Alliance has recently dedicated the specification for FUOTA, but the existing solution has several drawbacks, such as low energy efficiency, poor transmission reliability, and biased multicast grouping. In this paper, we propose a novel energy-efficient, reliable, and beamforming-assisted FUOTA system for LoRa networks named FLoRa, which is featured with several techniques, including delta scripting, channel coding, and beamforming. In particular, we first propose a novel joint differencing and compression algorithm to generate the delta script for processing gain, which unlocks the potential of incremental FUOTA in LoRa networks. Afterward, we design a concatenated channel coding scheme to enable reliable transmission against dynamic link quality. The proposed scheme uses a rateless code as outer code and an error detection code as inner code to achieve coding gain. Finally, we design a beamforming strategy to avoid biased multicast and compromised throughput for power gain. Experimental results on a 20-node testbed demonstrate that FLoRa improves network transmission reliability by up to 1.51 × and energy efficiency by up to 2.65 × compared with the existing solution in LoRaWAN.

LoRa networks have been deployed in many orchards for environmental monitoring and crop management. An accurate propagation model is essential for efficiently deploying a LoRa network in orchards, e.g., determining gateway coverage and sensor placement. Although some propagation models have been studied for LoRa networks, they are not suitable for orchard environments, because they do not consider the shadowing effect on wireless propagation caused by the ground and tree canopies. This paper presents FLog, a propagation model for LoRa signals in orchard environments. FLog leverages a unique feature of orchards, i.e., all trees have similar shapes and are planted regularly in space. We develop a 3D model of the orchards. Once we have the location of a sensor and a gateway, we know the mediums that the wireless signal traverse. Based on this knowledge, we generate the First Fresnel Zone (FFZ) between the sender and the receiver. The intrinsic path loss exponents (PLE) of all mediums can be combined into a classic Log-Normal Shadowing model in the FFZ. Extensive experiments in almond orchards show that FLog reduces the link quality estimation error by 42.7% and improves gateway coverage estimation accuracy by 70.3%, compared with a widely-used propagation model.

Edge intelligent applications, such as autonomous driving usually deploy multiple inference models on resource-constrained edge devices to execute a diverse range of concurrent tasks, given large amounts of input data. One challenge is that these tasks need to produce reliable inference results simultaneously with millisecond-level latency to achieve real-time performance and high quality of service (QoS). However, most of the existing deep learning frameworks only focus on optimizing a single inference model on an edge device. To accelerate multi-model inference on a resource-constrained edge device, in this paper we propose POS, a novel operator-level scheduling framework that combines four operator scheduling strategies. The key to POS is a maximum entropy reinforcement learning-based operator scheduling algorithm MEOS, which generates an optimal schedule automatically. Extensive experiments show that POS outperforms five state-of-the-art inference frameworks: TensorFlow, PyTorch, TensorRT, TVM, and IOS, by up to 1.2 × ∼ 3.9 × inference speedup consistently, with 40% improvement on GPU utilization. Meanwhile, MEOS reduces the scheduling overhead by 37% on average, compared to five baseline methods including sequential execution, dynamic programming, greedy scheduling, actor-critic, and coordinate descent search algorithms.

Recent years have witnessed the emergence of a new class of cooperative edge systems in which a large number of edge nodes can collaborate through local peer-to-peer connectivity. In this paper, we propose CoEdge, a novel cooperative edge system that can support concurrent data/compute-intensive deep learning (DL) models for distributed real-time applications such as city-scale traffic monitoring and autonomous driving. First, CoEdge includes a hierarchical DL task scheduling framework that dispatches DL tasks to edge nodes based on their computational profiles, communication overhead, and real-time requirements. Second, CoEdge can dramatically increase the execution efficiency of DL models by batching sensor data and aggregating the inferences of the same model. Finally, we propose a new edge containerization approach that enables an edge node to execute concurrent DL tasks by partitioning the CPU and GPU workloads into different containers. We extensively evaluate CoEdge on a self-deployed smart lamppost testbed on a university campus. Our results show that CoEdge can achieve up to reduction on deadline missing rate compared to baselines.

Running deep learning models on resource-constrained edge devices has drawn significant attention due to its fast response, privacy preservation, and robust operation regardless of Internet connectivity. While these devices already cope with various intelligent tasks, the latest edge devices that are equipped with multiple types of low-power accelerators (i.e., both mobile GPU and NPU) can bring another opportunity; a task that used to be too heavy for an edge device in the single-accelerator world might become viable in the upcoming heterogeneous-accelerator world. To realize the potential in the context of 3D object detection, we identify several technical challenges and propose PointSplit, a novel 3D object detection framework for multi-accelerator edge devices that addresses the problems. Specifically, our PointSplit design includes (1) 2D semantics-aware biased point sampling, (2) parallelized 3D feature extraction, and (3) role-based group-wise quantization. We implement PointSplit on TensorFlow Lite and evaluate it on a customized hardware platform comprising both mobile GPU and EdgeTPU. Experimental results on representative RGB-D datasets, SUN RGB-D and Scannet V2, demonstrate that PointSplit  on a multi-accelerator device is 24.7 × faster with similar accuracy compared to the full-precision, 2D-3D fusion-based 3D detector on a GPU-only device.

Motivated by many potential applications that could be enabled by acoustic motion tracking, in this paper we systematically examine the factors that limit the accuracy of acoustic tracking in practical scenarios. We identify three main challenges: (i) high mobility, (ii) low SNR, and (iii) hardware frequency response. We further show that the last two issues may exacerbate the performance issue under high mobility. We develop effective approaches to address the issues. In particular, to address high mobility, we tackle phase wrap-around using the derivative of the phase; we further estimate the Doppler shift under diverse scenarios and compensate the Doppler in channel impulse response (CIR). To address low SNR, we use a novel approach to estimate the phase shift between consecutive time intervals to effectively support time-domain beamforming and increase SNR. To tackle the uneven frequency response, we show that it is important to estimate and compensate the phase as well as the amplitude of the frequency response. Our extensive evaluation shows that each of our techniques is effective and putting them together significantly enhances the accuracy of acoustic motion tracking in general scenarios.

Indoor occupant sensing enables many smart home applications, and various sensing systems have been explored. Based on their installation requirements, we consider two categories of sensors – on- and off-body – and we look into the combination of them for occupant sensing due to their spatial and temporal complementarity. We focus on an example modality pair of wearable IMU and structural vibration that demonstrate modality complementarity in prior work. However, current efforts are built upon the assumption that the knowledge of the signal segments from two modalities are known, which is challenged in a multiple occupants co-living scenario. Therefore, establishing accurate cross-modal signal segment associations is essential to ensure that a correct complementary relationship is learned.

We present CMA, a cross-modal signal segment association scheme between structural vibration and wearable sensors. We propose AD-TCN, a framework built upon a temporal convolutional network that calculates the amount of shared context between an structural vibration sensor and associated wearable sensor candidates from the parameters of the trained model. We evaluate CMA via a public multimodal dataset for systematic evaluation, and we collect a continuous uncontrolled dataset for robustness evaluation. CMA achieves up to AUC value, F1 score, and accuracy improvement compared to baselines.

This paper introduces Wireless IoT-based Noise Cancellation (WINC) which defines a framework for leveraging a wireless network of IoT microphones to enhance active noise cancellation in noise-canceling headphones. The IoT microphones forward ambient noise to the headphone over the wireless link which travels a million times faster than sound and gives the headphone a future lookahead into the incoming noise. While leveraging wireless lookahead has been explored in past work, prior systems are limited to a single noise source. WINC, however, can simultaneously cancel multiple noise sources by using a network of IoT nodes. Scaling wireless lookahead aware noise cancellation is non-trivial because the computational and protocol delays can defeat the purpose of leveraging wireless lookahead. WINC introduces a novel algorithm that operates in the frequency domain to efficiently cancel multiple noise sources. We implement and evaluate WINC to show that it can cancel three noise sources and outperforms past work and state-of-the-art headphones without requiring completely blocking the users’ ears.

Tracking interpersonal distances is essential for real-time social distancing management and ex-post contact tracing to prevent spreads of contagious diseases. Bluetooth neighbor discovery has been employed for such purposes in combating COVID-19, but does not provide satisfactory spatiotemporal resolutions. This paper presents ImmTrack, a system that uses a millimeter wave radar and exploits the inertial measurement data from user-carried smartphones or wearables to track interpersonal distances. By matching the movement traces reconstructed from the radar and inertial data, the pseudo identities of the inertial data can be transferred to the radar sensing results in the global coordinate system. The re-identified, radar-sensed movement trajectories are then used to track interpersonal distances. In a broader sense, ImmTrack is the first system that fuses data from millimeter wave radar and inertial measurement units for simultaneous user tracking and re-identification. Evaluation with up to 27 people in various indoor/outdoor environments shows ImmTrack’s decimeters-seconds spatiotemporal accuracy in contact tracing, which is similar to that of the privacy-intrusive camera surveillance and significantly outperforms the Bluetooth neighbor discovery approach.

Micro-displacement measurement is a crucial task in industrial systems such as structural health monitoring, where millimeter-level displacement of specific points on the structure or machinery displace can jeopardize the integrity of the structure and potentially leading to catastrophic damage or collapse. Traditionally, such displacements on large structures are measured using visual sensing platforms or advanced surveying equipment. However, they either fall short in varying weather and lighting conditions or require installation and maintenance of high-power sensing platforms that are expensive to deploy at scale, especially if continuous measurements are desired.

In this paper, we explore simultaneous tracking of quasi-static micro-displacements of multiple objects or multiple points from a single vantage point using millimeter-wave (mmWave) radars. We present Platypus, a micro-displacement sensing system that enables sub-millimeter level accuracy by using mmWave backscatter tags and their reflection as phase carriers to shift the phase changes due to tiny displacements to clean frequency bins for precise tracking. It then reconstructs the phase changes with sub-millimeter level accuracy even from extended ranges (over 100m) or in non-line-of-sight (NLoS) situations. While Platypus enables many different applications, we demonstrate the proof-of-concept in structural health monitoring, where mmWave tags are attached to building models and track the structural micro-displacements, achieving a median of 0.3 mm accuracy.

This paper presents the design and implementation of mmRipple, which empowers commodity mmWave radars with the communication capability through smartphone vibrations. In mmRipple, a smartphone (transmitter) sends messages by modulating smartphone vibrations, while a mmWave radar (receiver) receives the messages by detecting and decoding the smartphone vibrations with mmWave signals. By doing so, a smartphone user can not only be passively sensed by a mmWave radar, but also actively send messages to the radar using her smartphone without any hardware modifications to either the smartphone or the mmWave radar. mmRipple addresses a series of unique technical challenges, including vibration signal generation, tiny vibration sensing, multiple object separation, and movement interference mitigation. We implement and evaluate mmRipple using commodity mmWave radars and smartphones in different practical conditions. Experimental results show that mmRipple achieves an average vibration pattern recognition accuracy of 98.60% within a 2m communication range, and 97.74% within 3m on 11 different types of smartphones. The communication range can be further extended up to 5m with an accuracy of 91.67% with line-of-sight path. To our best knowledge, mmRipple is the first work that allows smartphones to send data to COTS mmWave radars via smartphone vibrations and will enable many new applications such as vibration-based near field communication and pedestrian-to-sensing-infrastructure communication.

Novel backscatter communication techniques enable battery-free sensor tags to interoperate with unmodified standard IoT devices, extending a sensor network’s capabilities in a scalable manner. Without requiring additional dedicated infrastructure, the battery-free tags harvest energy from the environment, while the IoT devices provide them with the unmodulated carrier they need to communicate. A schedule coordinates the provision of carriers for the communications of battery-free devices with IoT nodes. Optimal carrier scheduling is an NP-hard problem that limits the scalability of network deployments. Thus, existing solutions waste energy and other valuable resources by scheduling the carriers suboptimally. We present DeepGANTT, a deep learning scheduler that leverages graph neural networks to efficiently provide near-optimal carrier scheduling. We train our scheduler with optimal schedules of relatively small networks obtained from a constraint optimization solver, achieving a performance within 3% of the optimum. Without the need to retrain, our scheduler generalizes to networks 6 × larger in the number of nodes and 10 × larger in the number of tags than those used for training. DeepGANTT breaks the scalability limitations of the optimal scheduler and reduces carrier utilization by up to compared to the state-of-the-art heuristic. As a consequence, our scheduler efficiently reduces energy and spectrum utilization in backscatter networks.

A truly ubiquitous, planet-wide Internet of Things requires ultra-low-power, long-lasting sensor nodes at its core so that it can be practically utilized in real-world scenarios without prohibitively high maintenance efforts. Recent advances in energy harvesting and low-power electronics have provided a solid foundation for the design of such sensor nodes. However, the issue of reliable two-way communication among such devices is still an active research undertaking due to the high energy footprint of traditional wireless transceivers. Although approaches such as radio duty cycling have proved beneficial for reducing the overall energy consumption of wireless sensor nodes, they come with trade-offs such as increased communication latency and complex protocols.

To address these limitations, we propose ARISTOTLE, an ultra-low-power, wake-up receiver-based sensor node design employing a star network topology. We have deployed ARISTOTLE in two different venues for carrying out the task of weather data collection. In addition to reporting the results of the two deployments, we also evaluate several performance aspects of our proposed solution. ARISTOTLE has a mean power consumption of 236.67 uW while it is in sleep mode and monitoring the radio channel for incoming wake-up signals. Utilizing various sizes of supercapacitors, ARISTOTLE was able to reach system availabilities between 47.83% and 97.36% during our real-world deployments.

Physical layer cross-technology communication (PHY-CTC) opens new horizons for spectrum utilization and wireless cooperation in the crowded ISM band. Current PHY-CTC technologies can be divided into two categories: one aims for high communication reliability, but sacrifices compatibility with commodity devices and the other maintains compatibility but suffers dramatically limited reliability. The latter mainly leverages the WiFi OFDM signal to emulate the ZigBee signal, while the Cyclic Prefix (CP) in OFDM brings inevitable signal disturbance and errors. To address these issues, we present WibZig, the first WiFi-To-ZigBee PHY-CTC technology that achieves high reliability and is fully compatible with commodity devices. By carefully selecting a cluster of CCK codewords that exhibit similar phase characteristics to ZigBee chips, we can emulate any given ZigBee symbol with great accuracy. In addition, WibZig adaptively controls the first CCK codeword of each cluster to eliminate inter-cluster phase discontinuity when emulating a ZigBee packet with multiple clusters. WibZig requires no hardware modification and is compatible with most commodity devices. We implement WibZig on both USRP and commercial devices and conduct extensive evaluations under various settings, which demonstrate a 15x improvement in reliability and a 7x increase in range compared to the latest PHY-CTC work.

The stethoscope is one of the most important diagnostic tools used by healthcare professionals, through a process called auscultation, to screen patients for abnormalities of the heart and lungs. While there are digital stethoscopes on the market which ease this process, it still takes years of training to properly use these devices to listen for abnormal sounds within the body. We present ARSteth, an intelligent stethoscope platform that improves the accessibility of stethoscopes for the general population, allowing anyone to perform auscultation in the comfort of their own homes. Our platform utilizes a combination of augmented reality (AR), acoustic intelligence, and human-machine interaction to dynamically guide users on where to place the stethoscope on different parts of the body (auscultation points), through visual and audio cues. Through user studies, we show that ARSteth, on average, can guide users within 13.2 mm from optimal auscultation points marked by licensed physicians in 13.09 seconds for each auscultation point. By guiding users towards more effective auscultation points, make preventative health screening more accessible and effective for everyone we are able to achieve higher confidence on classifying heart murmurs.

Low-power wireless networks have the potential to enable applications that are of great importance to industry and society. However, existing network protocols do not meet the dependability requirements of many scenarios as the failure of a single node or link can completely disrupt communication and take significant time and energy to recover. This paper presents Hydra, a low-power wireless protocol that guarantees robust communication despite arbitrary node and link failures. Unlike most existing deterministic protocols, Hydra steers clear of centralized coordination to avoid a single point of failure. Instead, all nodes are equivalent in terms of protocol logic and configuration, performing coordination tasks such as synchronization and scheduling concurrently. This concept of concurrent coordination relies on a novel distributed consensus algorithm that yields provably unique decisions with low delay and energy overhead. In addition to a theoretical analysis, we evaluate Hydra in a multi-hop network of 23 nodes. Our experiments demonstrate that Hydra withstands random node failures without increasing coordination overhead and that it re-establishes efficient and reliable data exchange within seconds after a major disruption.

This paper introduces MicroDeblur, an on-device image motion deblur solution for resource-constrained microcontroller-based vision systems. Although motion blurs caused by the movement or shake of the device (camera) are pervasive in embedded, IoT, and mobile devices, it has been considered a hard nut to crack for many microcontrollers with extremely-limited resources (e.g., hundreds of KB of RAM). To tackle this problem, we combine the DNN (deep neural network) motion deblur method with the classical motion deblur approach and take the best of both worlds, i.e., 1) powerful pattern recognition ability of DNNs and 2) simplicity and stability of matrix-based classical algorithms. To deblur an image, MicroDeblur takes three steps: 1) blur kernel estimation, 2) blur image transformation, and 3) iterative clear image restoration. We propose 1) depth-independent convolution that efficiently estimates the blur kernel (pattern) and 2) Toeplitz-based motion blur modeling that enhances the time and space complexity of the deblurring process by and , respectively, compared to the existing methods. To the best of our knowledge, MicroDeblur is the first self-sufficient blind deconvolution solution for a stand-alone microcontroller that does not rely on extra hardware or external systems. We implement MicroDeblur on an ARM Cortex-M4F, achieving a competitive quality of deblurred images using 187x and 429x smaller memory and energy, respectively, compared to high-end GPU-based solutions.

Despite the potential of vision-based personal monitoring (e.g., healthcare), private data leakage concerns hinder its wide deployment in personal spaces (e.g., bedrooms). A body of data anonymization designs was proposed throughout image processing and federated learning. They commonly store high-quality images and videos locally, which are anonymized via post-processing before cloud upload. However, the recent IoT camera hacking and local data leakage call for anonymized data at the sensing stage. Also, continuous and pervasive monitoring without blind spots in complicated indoor spaces requires a scalable and economic system. This paper present Mosaic, a vision-based end-to-end action recognition framework that (i) intrinsically achieves data anonymity from the sensing stage and (ii) battery-free operation for blind spot-free continuous monitoring. Mosaic leverages an extremely low resolution (eLR) Near-Infrared (NIR) image sensor with 6 × 10 pixels for video anonymity and RFID-compliant fully-passive tag with four solar cells for real-time eLR video streaming under as low as 50 lux (e.g., deep in the shelf without direct light). This is accompanied by light-weight action recognition neural network for real-time inference (18.4ms on Intel(R) Core i7-8700). Mosaic achieves an average of 98% accuracy on 10 action classes, hitting the balance between data anonymity and high-precision action recognition. By taking advantage of NIR (non-visible) frequency, Mosaic also works in dark without disturbing sleep. Lastly, wildfire detection reaching 20m was demonstrated, showcasing the potential for outdoor monitoring.

In many low-power wireless systems, a condition occurring at some nodes (e.g., an anomalous sensor sample, an aperiodic packet to transmit, a new joining node) determines whether the entire network should be awake (e.g., to react to the anomaly, deliver the packet, update the node group) or enter sleep. State-of-the-art protocols exploit periodic network-wide floods based on concurrent transmissions (e.g., via Glossy) to establish the global decision quickly, reliably, and efficiently. Still, time is of the essence: the faster the network agrees, the faster it either reacts or enters sleep.

Flick achieves this global decision with order-of-magnitude latency improvements and 5-nines reliability by detecting and disseminating a binary on/off vote via the preamble of a packet instead of its full content. The actual realization of this simple idea entails several techniques on the ultra-wideband (UWB) radios we use. We evaluate Flick over a 78-node network in the Cloves testbed showing that, e.g., a single node can globally flick the switch to on across a 10-hop diameter in < 500&#U+03BC;s , i.e., roughly the time for a Glossy packet to go across a single hop, and with 4.4 × less energy. We demonstrate the potential of Flick by integrating it into staple protocols and evaluating the performance improvements it enables.

In recent years, the number of wireless applications has increased significantly, resulting in the radio bands becoming expensive and prone to interference. There is a new research area aiming at mitigating these issues by creating communication links using ambient light. This area, called passive-VLC, not only exploits the visible light frequencies, but does so with low-power transmitters. All the previous work in passive-VLC, however, forget about individual wavelength bands of light, and do not exploit its wide spectrum, reducing the potential channel capacity. In this paper, we propose a novel method to transmit and decode data, using liquid crystal cells that modulate and consider the full spectrum, and put it to the test by prototyping a multi-symbol communication link. The main contribution of our work is to show that passive-VLC can move from spectrum-agnostic to spectrum-aware modulation. We explore this new domain by making use of a novel type of receiver (i.e., a spectrometer) and uncovering the advantages and caveats of this spectrum-aware approach.

Side-channels have traditionally been exploited as a means of uncovering sensitive information such as cryptographic keys from a computing device. In particular, past work has shown that electromagnetic (EM) radiation from a device’s processor and memory during the execution of code and data can be used by attackers to extract private information. In contrast, instead of considering side-channels and electromagnetic radiation as vulnerabilities, we see them as opportunities for wireless communication on resource-limited IoT devices. We present SideComm, a side-channel-based communication system that leverages processors’ EM side-channels to enable resource-limited IoT devices to wirelessly send their data without having any radios. The main advantage of this approach is completely eliminating the need for a conventional radio and antenna, which offers energy savings, simplicity, and flexibility for IoT devices. Our evaluation demonstrates SideComm’s ability to achieve a communication range of more than 10m (enabling ≥ 3 dB SNR at 15m) and to work in non-line-of-sight scenarios, such as around corners and through walls. We believe SideComm can enable increased connectivity for many resource-constrained IoT devices in smart environments.

More processors are integrated into Industrial Internet-of-Thing gateways to perform increasing emerging applications. Network-on-chip (NoC) offers a scalable, high-throughput, and energy-efficient communicate infrastructure. However, existing NoC routers cannot guarantee differentiated quality-of-service (QoS) for diversified applications. Hence, we propose a novel NoC router architecture with the gate control mechanism to provide customized QoS.

Due to resource constraints, it is challenging to optimize the inference performance in terms of energy consumption and latency on edge devices. In this paper, we leverage both the dynamic voltage frequency scaling (DVFS) technique and edge-cloud collaborative inference to minimize the overall energy consumption. We propose a deep reinforcement learning (DRL)-based method called DVFO to jointly optimize 1) CPU, GPU and memory frequencies, and 2) the ratio of offloaded feature maps in edge-cloud collaboration. Preliminary experimental results show that DVFO reduces the average energy consumption by 33% compared to the baselines. Moreover, it reduces the inference latency by more than 54%.

Human-machine interface (HMI) of high degrees of freedom (DoF) is one of the most critical bases of the metaverse. The ideal HMI for the metaverse should be cheap, robust, customizable, and ergonomically friendly. In light of this, we propose a triboelectric nanogenerator (TENG)-based sensing system. We developed a low-cost, soft, light, and customizable TENG sensor to collect data from the human body. We then used an artificial neural network (ANN) to obtain the corresponding human motion from collected sensory data. The effectiveness of the proposed system is demonstrated with experiments of a working prototype.

Millimeter wave (mmWave) 3D imaging has been applied for point cloud data (PCD) generation due to its valuable attributes, such as working under low light, compact size, and low-cost. However, past works have focused on transforming millimeter wave reflection signals into other data structures, like polar images and coarse PCDs before applying neural network to produce dense PCDs. Those algorithms will filter some useful features. To address this issue, our paper proposes an innovative prototype: mmWaveNet, a deep learning model that directly uses reflection signals as input and generates high-quality PCDs. We have experimentally evaluated mmWaveNet in a large indoor environment.

Emergency Medical Services (EMS) providers use their hands extensively for the rescue operation and providing care to the patients in an EMS scene. Using smartwatch based sensor data, i.e., accelerometer, gyroscope, and magnetometer, we are developing SenseEMS, a system for hand operated EMS intervention detection and real-time monitoring. SenseEMS will use a hybrid deep neural network with appropriate real-time algorithms on the sensor data to detect multiple hand operated activities, i.e. CPR compressions, attaching defibrillation pads and breathing bags, and to provide quality assessment on different metrics of the activity, i.e., the rate and depth of CPR compressions. Our initial results for this ongoing research show promising accuracy. Preliminary survey with 31 anonymous EMS responders suggests that this automated system will be highly beneficial for real-scene application and EMS training.

With the continuing advancement of ubiquitous computing and various sensor technologies, we are observing a massive population of multimodal sensors at the edge which posts significant challenges in fusing the data. In this poster we propose MultimodalHD, a novel Hyperdimensional Computing (HD)-based design for learning from multimodal data on edge devices. We use HD to encode raw sensory data to high-dimensional low-precision hypervectors, after which the multimodal hypervectors are fed to an attentive fusion module for learning richer representations via inter-modality attention. Our experiments on multimodal time-series datasets show MultimodalHD to be highly efficient. MultimodalHD achieves 17x and 14x speedup in training time per epoch on HAR and MHEALTH datasets when comparing with state-of-the-art RNNs, while maintaining comparable accuracy performance.

Object classification is a vital technology that is widely used to track and identify misplaced and out-of-stock items in shopping centers. While there have been a number of studies utilizing various sensing modalities such as computer vision, RFID, and vibration sensors, these methods are limited in their use due to privacy concerns, scalability, and the inability to identify stationary objects. To overcome these limitations, we propose a novel active vibration-sensing approach for object classification by utilizing music as an excitation source. Different objects can induce different deformations of the surface and further change the surface structural response. Therefore, we leverage vibrations from music on a store shelf and measure the structural responses on the surface when different objects are placed. Our evaluation of a store shelf demonstrates that distinct object characteristics lead to unique vibration responses, enabling accurate classification of 98.6% accuracy in distinguishing five common store objects. This study provides a promising avenue for a reliable, privacy-preserving, and scalable object classification system in various settings beyond shopping centers.

One of the major shortcomings in IoT/sensor networks is the finite energy supply available for computation and communication. To circumvent this issue, energy harvesting has been proposed to enable embedded devices to mitigate their dependency on traditional battery-driven power source. However, energy supply due to energy harvesting often varies, leading to nodes crashing due to energy exhaustion, with application(s) losing their state. Efficient state checkpointing in non-volatile memory (NMV) has been proposed to enable forward progress, albeit at the expense of significant overhead (viz., energy and time). In this poster, we show preliminary results that, for a certain class of applications, state checkpointing may adversely affect the performance of the applications. This is different from checkpointing in traditional distributed systems, where the network topology is generally assumed to be stable.

Ensuring two battery-free devices discover each other to start communication is challenging due to intermittent and unpredictable energy availability. In this abstract, we exploit ultra-low power channel sensing to enable efficient neighbor discovery. Preliminary results show our method is promising in various ambient energy scenarios.

The standard way to keep track of inventory is through human counting or with a hand-held device and inputting the data into the warehouse management system (WMS). WMS can be quite expensive for small warehouses and human counting is susceptible to errors. Alternatively, we propose an automated way to keep track of inventory by leveraging the surveillance cameras readily available in the warehouses. In practice, there are challenges such as incorrect stock numbers, overfitting issues, and a mundane labeling process. To identify and mitigate those challenges, we present real-world deployment to various warehouses for a period of 4 months with up to 97 percent counting accuracy.

We propose a novel activity-based profiling framework to estimate IoT users’ harvested energy based on their daily activities. Energy is harvested from natural sources such as the kinetic movement of IoT users. The profiling framework captures the users’ physical activity data to define activity-based profiles. These profiles are utilized to estimate the harvested energy by IoT users. We train and evaluate our framework based on a real Fitbit dataset.

Distracted driving is a major cause of road accidents. Multiple targets, including the driver’s head and two hands, need to be monitored to detect unsafe driving behaviors. Previous driving monitoring systems rely on cameras or wearable sensors, yet these solutions have multiple limitations, including insufficient monitoring or requiring multiple wearable components. This paper proposes a new driving behavior monitoring system for improving road safety, which achieves unsafe driving behavior monitoring using only one pair of unobtrusive Commercial-Off-The-Shelf (COTS) smart glasses. The proposed system monitors the two hand movements using the front-facing camera, and detects head movements using the onboard sensors. A proof-of-concept system was implemented on a pair of Android-powered smart glasses, and a small-scale emulation study was conducted in the lab environment. Experiments show our system can monitor steering wheel control techniques and common head turn motions with f1 scores of 93.1% and 99%, respectively.

Federated learning (FL) is an increasingly popular form of distributed learning across devices such as sensors and smartphones. To amortize the effort and cost of setting up FL training in real world systems, in practice multiple machine learning tasks may be trained during one FL execution. However, given that the tasks have varying complexities, naïve methods of allocating resource-constrained devices to work on each task may lead to highly variable performance across the tasks. We instead propose an α -fair based allocation algorithm that dynamically allocates tasks to users during multi-model FL training, based on the prevailing loss levels.

Cyber-physical-human systems (CPHS) in AI-based driver assistance applications require the integration of data from diverse modalities. These in-cabin CPHSs offer rich sensing capabilities, encompassing the vehicle, its surroundings, and the driver. One of the primary challenges in CPHSs is the incorporation of human behavior modeling to steer interactions that bolster human performance. This study introduces an in-cabin vehicular sensing framework that merges a driving simulator with a CPHS, thereby enabling researchers to devise and test multi-sensor fusion methodologies in human-in-the-loop driving situations. Initial experiments reveal that a driver interruptibility model can be effectively trained using the data gathered from our system.

Satellite technology offers exciting new opportunities for IoT applications. This poster shows the performance one may expect using Kinéis – a leading operator – as a representative example. We use an implementation we provide as open-source for repeatability, and measure an end-to-end latency of 45 min, a battery lifetime of 1,000 packet when using a pair of AA batteries, and an end-to-end reliability of 23%/54%/99% when using 1/10/18 repetitions.

The Internet of Things (IoT) offers transformative potential when combined with Machine Learning (ML), but labeling diverse IoT data remains challenging. To address this, we introduce SenseScape Testbed, an IoT experimentation platform for indoor environments with wireless sensor nodes, robots, and location-tracking nodes. This testbed enables IoT applications such as human activity recognition and indoor mobility tracking, supporting energy efficiency, occupant comfort, and context representation while providing a versatile environment for labeling and testing to advance ML algorithms tailored for IoT applications.

Medication non-adherence is a major healthcare challenge globally, with over half of patients with chronic conditions in developed countries failing to follow their prescribed medication regimen. This can lead to poor disease outcomes, increased hospital visits, and a significant financial burden on healthcare systems [1]. These issues have driven a recent wave of research, including the development of smart adherence products [6] that can be incorporated into a patient’s daily life to monitor medication adherence. In this work, we present a radar-based system for user identification while taking medication, which extends our recent work [5]. we conducted preliminary experiments examining semi-medication-taking activities executed by 6 subjects. Our system achieved 80% accuracy in identifying who has taken the medication in a group of 3 subjects.

This short paper summarizes our recent/ongoing works [2, 3, 4] on detecting DDoS attacks in IoT systems. In our studies, we conducted a thorough examination of using machine learning to detect Distributed Denial of Service (DDoS) attacks in large-scale Internet of Things (IoT) systems. Unlike prior works and typical DDoS attacks that focus on individual nodes transmitting high volumes of packets, we explored the more sophisticated and advanced future attacks that use a large number of IoT devices while hiding the attack by having each node transmit at a volume that mimics benign traffic. We introduced innovative correlation-aware architectures that consider the correlation between the traffic of IoT nodes and compare the effectiveness of centralized and distributed detection models. Through extensive analysis, we evaluated the proposed architectures using five different neural network models trained on a real-world IoT dataset of 4060 nodes. Our results showed that the combination of long short-term memory (LSTM) and transformer-based models with the correlation-aware architectures offer superior performance, in terms of F1 score and binary accuracy, compared to the other models and architectures, especially when the attacker conceals its actions by following benign traffic distribution on each transmitting node. Furthermore, we investigated the performance of heuristics for selecting a subset of nodes to share their data in resource-constrained scenarios for correlation-aware architectures.

Hardware and software advances have paved the way for large-scale urban sensor networks, but there is no guidance on where to place nodes and how to assess network design. In this work, I first re-imagine coverage of urban sensor networks with a view toward cities as more than just physical spaces. I then propose data-driven and nature-based algorithms for urban sensor network design and compare these proposed designs to those used in prior and existing sensor networks. This work will enable the deployment of robust sensor networks that produce useful citywide data for numerous stakeholders despite the complexities of urban environments.

Along with the recent advancements of lightweight machine learning and powerful systems and hardware platforms, intelligence beyond the edge has become the next tide of IoT. However, multiple barriers exist from data, algorithm, network and hardware perspectives. In this abstract, I provide an overview of my PhD research which aims at closing the gap towards deploying edge intelligence for large-scale and real-world IoT applications. I further introduce our recent contributions and the work planned ahead.

Cooperative mobile robot systems have great potential to solve real-world problems but are, in practice, limited by their physical capabilities, the environment in which they operate, and the technology available to them. My research investigates how communication, localization, sensing, mobility, and computation constraints affect the performance of mobile agent systems. In this extended abstract, I focus on two fundamental problems: delivery and dispersed computing, and discuss recent contributions to the literature, as well as our ongoing work.

Recent advances in machine learning have inspired the development of deep neural network (DNN)-based smart sensing applications for the Artificial Internet of Things (AIoT). However, the effectiveness of DNNs relies on the availability of large, labeled data to uncover useful feature representations. The widespread use of DNN models in computer vision (CV), natural language processing (NLP), and voice sensing can be attributed to the massively available labeled training datasets. Despite the abundance of IoT sensing data, the human-uninterpretable property of AIoT data makes it difficult to construct labeled datasets for DNN model training. Additionally, variations in sensor hardware or DNN models’ deployment environments introduce domain shifts, making generalized machine learning algorithms even more difficult to develop. The scarcity of labeled training data and run-time domain shifts are two main challenges in developing effective machine learning algorithms for AIoT sensing. The goal of my research is to address the above challenges for AIoT sensing applications. Two main research methodologies are involved. The first is to leverage the latest state-of-the-art machine learning techniques to develop effective models for smart sensing. The second approach involves integrating known prior knowledge into machine learning algorithms to develop more accurate and reliable DNN models for AIoT sensing applications.

Today’s advancements in IoT devices and edge computing platforms have given rise to new scenarios enabling context-aware applications in extremely interconnected environments. To promote the standardization of these platforms the European Telecommunications Standards Institute (ETSI) proposed the Multi-access Edge Computing (MEC) standard, enabling the execution of cloud-like services at the network edge. In this work, we propose the design of a novel MEC-compliant architecture that leverages underutilized far-edge resources to enlarge MEC edge node computational capacity and enhance service availability in highly mobile networks. Our approach allows far-edge devices to participate in a negotiation process embodying a rewarding system while addressing resource volatility as these devices join and leave the edge node resource infrastructure. Furthermore, we developed an original simulation framework to replicate the proposed architecture by using vehicle resources as far-edge devices. Its primary purpose is to demonstrate the viability and flexibility of our proposal, as well as to investigate novel application scenarios using real-world datasets. Our preliminary results show the feasibility and effectiveness of our proposal when using vehicular-based virtual resources in realistically simulated 5G networks.

Battery-free devices eliminate the need for batteries, which are expensive, environmentally harmful, and require frequent replacement, thus reducing waste and making devices more cost-effective. We introduce Riotee, the next-generation platform for the battery-free Internet of Things. The platform comprises a base module, a debug probe that allows to conveniently update the firmware on the base module, and a number of expansion boards that extend the capabilities of the platform without the need to design a custom printed circuit board (PCB). We provide a brief overview of Riotee, and describe a demo setup that showcases the key functionality and how to get started with the platform in less than three minutes.

RF signals can be leveraged for many sensing and monitoring tasks in industrial, home, or robot applications. Despite the advantages of leveraging WiFi sensing modality, no versatile WiFi sensors are available. We develop WiROS to address this immediate need. We leverage the robot operating system (ROS) framework to expose real-time WiFi sensing information to an end-user. Specifically, we demonstrate a plug-and-play toolbox providing access to coarse-grained WiFi signal strength (RSSI), fine-grained WiFi channel state information (CSI), and other MAC-layer information (device address, packet id’s or frequency-channel information). Additionally, we opensource state-of-art algorithms to calibrate and process WiFi measurements to intuitively visualize/debug measurements and measure signal path parameters like the signal’s angles of arrival or departure.

The open-sourced repository is https://github.com/ucsdwcsng/WiROS.

We demonstrate Platypus, a sub-millimeter micro-displacement sensing system presented in [3]. Micro-displacement measurement is a crucial task in industrial systems such as structural health monitoring, where millimeter-level displacement of specific points on the structure or machinery parts can jeopardize the integrity of the structure and potentially leading to catastrophic damage or collapse. Platypus enables sub-millimeter level sensing accuracy by using mmWave backscatter tags and their reflection as phase carriers to shift the phase changes due to tiny displacements to clean frequency bins for precise tracking. It then reconstructs the tag phase changes with sub-millimeter level accuracy even from extended ranges (over 100m) or in non-line-of-sight (NLoS) situations where the tag is blocked by other objects. Here, we demonstrate Platypus’s performance by attaching a Platypus tag to a stepper motor-driven motion-stage and demonstrating the micro-displacement detection in real time, and the system robustness against multipath and occlusions.

Traditionally, side channels have been exploited to uncover sensitive information such as cryptographic keys from computing devices. An attacker can extract private information from a device’s processor and memory by using electromagnetic (EM) radiation while code and data are being executed. Rather than seeing side-channels and electromagnetic radiation as vulnerabilities, we consider them as potential wireless communication channels for resource-constrained devices. The main advantage of this approach is completely eliminating the need for a conventional radio and antenna, which offers energy savings, simplicity, and flexibility for resource-constrained devices.

This demo is based on our upcoming IPSN’23 paper which presents the details of this radio-less side-channel-based communication framework. In this demo abstract, we briefly explain the concept, its implementation, advantages, and limitations, and then describe our setup and plan for the live demo for this framework. Our demo shows how a radio-less MCU (Arduino UNO) could communicate with a nearby receiver (a cheap UHF TV antenna connected to a software-defined radio) by generating side-channel EM radiations while running specialized software. Our demo confirms the feasibility of this method in a realistic setting and shows how our framework can enable low overhead connectivity for resource-constrained devices.

LoRa has emerged as a novel Internet of Things (IoT) communication paradigm, featuring with long-range and low-power transmission capabilities. With the widespread deployment of LoRa networks, the demand to perform Firmware Update Over-The-Air (FUOTA) tasks has become increasingly critical for unattended LoRa devices. However, in practice, three fundamental problems that hinder the performance of FUOTA tasks are revealed, including low energy efficiency, poor transmission reliability, and biased multicast grouping. In this demo, we present a novel FUOTA system, the first work that offers an effective and sustainable solution to achieve energy-efficient and reliable over-the-air firmware updates in LoRa networks. In particular, this system incorporates threefold key modules: delta scripting, channel coding, and beamforming. The delta scripting algorithm unlocks the capability of incremental update, the channel coding scheme ensures the reliability and robustness of large-scale firmware image distribution, and the beamforming strategy as an optional module can further serve the unicast user. Thus, this demo presents a working example of functionality customization to show the efficacy and feasibility of our FUOTA system in LoRa networks.

Heart rate is a critical metric for human cardiovascular health. Most common methods for measuring human heart rate involve wearable devices (e.g., electrocardiography, smart watches). However, such devices can cause discomfort to some patients, especially the elderly or young children. This paper presents FreePulse, a heart rate monitoring system for seated subjects using ambient vibrations. FreePulse builds on our past work using vibrations in the building structures around us to measure human activities and health. As people’s hearts beat, they push on the surfaces the body is touching, creating vibrations in those structures. We combine structure response characterization with human pulse modelling to identify these pulse-induced vibrations from ambient vibration. In testing, FreePulse has shown up to 96% pulse rate accuracy on average, competitive with consumer-grade wearable devices.

In ophthalmology, retinal laser therapy is a treatment for retinopathy that requires the use of magnifying lens to treat damaged regions of retinal landmarks, hence creating challenges of inverted magnified images and requiring prolonged training. Augmented Reality (AR) can benefit clinicians during retinal laser therapy by guiding them with retinal landmark holograms and contextual information. Though recent developments in AR magnification show that a direct overlay of the magnified scenes can be achieved, retinal laser therapy requires high precision and visual acuity while maintaining the visual perception of the rest of the environment. Therefore, we demonstrate an AR-based selective magnification system that provides contextual and visualization-based guidance to clinicians. An edge-computing architecture is developed for detecting and matching the feature points between the magnified image and color fundus image of the retina to identify the magnified region of retinal landmarks. We showcase how our AR guidance system can assist clinicians during retinal laser therapy.

Real-time object detection (OD) is a key enabling technology for a wide range of emerging mobile system applications. However, deploying an OD model pre-trained on a public dataset (source domain) in a specific local environment (target domain) is known to lead to significant performance degradation because of the so-called domain gap between the dataset and the environment. Collecting local data and fine-tuning the OD model on this data is a commonly used approach for improving the robustness of OD models in real-world deployments. Yet, the question of how to collect this data is currently largely overlooked; unsupported data collection is likely to produce datasets that contain significant proportion of redundant or uninformative data for model training. In this demo, we present BiGuide, a bi-level image data acquisition guidance for OD tasks, to guide users to change their camera locations or angles to different extents (significantly or slightly) to obtain the data which benefits model training via image-level and object instance-level guidance. We showcase an interactive demonstration of collecting data for a lemur species detection application we are developing and deploying at the Duke Lemur Center. Demo participants will take pictures of lemur toys with the mobile phone under the real-time guidance and will observe the real-time display of the metrics that assess the importance of the captured data. They will develop an intuition for how real-time image importance assessment and bi-level guidance improve the quality of collected data.

We present a demo showcasing the capabilities of an infrared (IR)-based light communication system with a movable receiver. The system employs IR laser (VCSEL) together with scattering lens as the transmitter and an avalanche photo-diode (APD) with collimator as the receiver, using the reflection cross section existing in the environment (ceilings, walls, etc.) to spread the coverage of the communication system. The demonstration involves transmitting messages encoded as binary data through modulated IR light signals to the movable receiver. The receiver captures the signals using the APD, which are then decoded to retrieve the original message. The demonstration aims to showcase the robustness of the system against various sources of interference and the flexibility of the movable receiver to capture signals from different angles and positions. We believe that our demonstration will be useful for showcasing the potential of IR-based light communication in various applications (e.g., wireless VR/AR and high speed reliable data link) over more traditional communication methods that have limitations such as privacy and bandwidth.

A digital modulator plays a crucial role in converting symbols into signals in an IoT gateway. However, the ever-increasing modulation schemes pose practical challenges, such as flexibility for different schemes and portability with different hardware platforms. To address these challenges, we propose a new approach that employs a neural network as an abstraction layer for physical layer modulators, called the NN-defined modulator. We will demonstrate that the NN-defined modulator functions like traditional modulators and offers high portability and efficiency with example communication to ZigBee and WiFi.