Sensory Substitution: Situational Awareness and Resilience using Available Sensors

This thesis proposes 1) the concept of sensory substitution to provide data extraction from multiple sensors, 2) data analytics approaches in scenarios, and 3) response vectors where rapidly deployable fixed and mobile sensors (such as Unmanned Aerial Vehicles [UAVs] as a flying sensor platform) and emerging human-in-the-loop sensing are used. A "resilience feedback loop" is used throughout to improve each of these approaches. This data can provide actionable intelligence for public safety and critical infrastructure systems. This thesis presents an algorithm called sensory substitution and resilience feedback, which improves situational awareness by solving two key design challenges - limitations on deployment of new infrastructure (sensors), and limited response vectors - using sensors in-place as a source of new information. In Internet of Things (IoT) environments, numerous sensors may be available - although required sensors may not. Sensory substitution can be a solution. To make a system resilient, any smart environment or system should provide redundancies. Sometimes adding hardware/sensors is not possible, so software must simulate other sensors, creating a multi-sensory approach, with a single sensor type. A commonly found measurement system for an application is to use a sensor designed to measure quantity X. In many real-world applications, modification constraints may limit the ability to deploy new hardware. In many cases, a sensor for X is present, though the measurement need is quantity Y. How can a sensor for X act as a substitute for Sensor Y to provide some of the missing information? An agile IoT approach can be a solution. In an effort for additional improvement for increasingly deeper situational awareness, a system called sensory substitution is developed for multiple sensing systems and generalized as part of an Agile IoT approach. However, an Agile IoT system can present privacy and security concerns. With such ubiquitous sensing, seemingly innocuous data could actually "leak" information. Since additional data can be collected with sensors already in-situ, relevant privacy and security implications are discussed.