The transition to the digital era is lined with many challenges, including the broadband infrastructure. The need for high data rate communications and accurate indoor localization/identification/tracking makes the ultra-wide band (UWB) technology particularly attractive. However, the huge instantaneous frequency band to be processed also leads to strong constraints, beyond the specifications of the state-of-the-art digital circuits.
The EU-funded project Ultra Wide Band Integrated Optical-and-Digital Approach for Smart Factory and Perceptive Car (UWB-IODA SF-PC) introduces a multidisciplinary approach relying on impulse radio UWB (IR-UWB) and UWB-over-fiber (UWBoF) technologies. They associate UWB waveform optimization, cutting edge signal processing techniques and artificial intelligence (deep learning based neural networks) to design intelligent, robust and high performance indoor wireless systems, for smart factories and perceptive cars.
This project aims at increasing the safety, security and convenience in Industry 4.0 environments, by multi-humans discrimination/detection, localization and tracking, without the need for them to wear tags or any additional identification equipment. It also aims to contribute to the road safety by preventing the driver drowsiness related accidents and to facilitate the interaction with the perceptive car by hand gesture recognition.

Our project is organized around three work packages. The first work package is related to UWB waveform optimization for spectral efficiency and waveform compression to realize low cost UWBoF transceivers. The second work package is related to context awareness tasks, such as physiological parameters, i.e., breathing rate and heart rate measurement and gesture recognition inside a car. The third work package is related to people detection, localization and classification of human and machines intended for Industry 4.0 with a mixed human and robots’ environment.

Original results have been already obtained so far and significant progress can be reported on the three project work packages. Thus, in the framework of WP1, we have developed a general system architecture and proposed optimized UWB waveforms complying with the project requirements and optimizing the spectral efficiency. We have also progressed on WP2 (context awareness tasks inside a perceptive car) and WP3 (people counting, detection and classification). Initially, we have installed and configured X4-M03 radar sensors from Novelda, Norway. These UWB radar sensors were used for both work packages WP2 and WP3. For reliable detection, localization and classification of human and machines inside Industry 4.0, it was necessary to use multiple radar sensors. To this end, a setting consisting of 4 radar sensors was deployed. The experiments were performed in multiple cluttered locations such as indoor room environment and inside a university lab environment. The work on WP3 is almost finalized and the related publication is already accepted to a reputed, high impact factor, IEEE journal.

The next work package, WP2 is related to physiological signal monitoring and gesture recognition for a perceptive car. We have divided this work package into sub-tasks. Our initial goal is to find robust algorithms for physiological monitoring of a human in a bed with some natural motions. To this end, we have first deployed an IR-UWB radar sensor to extract the physiological signal in resting position without any natural motion of body. Next, we are working to compensate the effect of the natural motions. We are developing advanced signal processing algorithms and using sensor fusion of radar as well audio sensors for the motion artifacts removal. This work will then be extended to the perceptive car to monitor the physiological signal of the driver as well as other occupants of the car.

The most important results beyond the state of the art have been obtained so far in terms of multi-human detection, localization and classification in an indoor environment. Indeed, we achieved better results in multi-path reduction as compared to the state of the art work in UWB radar based people counting. We have worked on a combination of techniques including a novel threshold algorithm, multi-path reduction technique based on radar sensors geometry and deep learning based algorithms with the image of the radar backscattered signals as input. The proposed approach involves multi-disciplinary techniques and is able to classify human and machine in an indoor environment with an accuracy of 85-90 %.

The highlights of the progress achieved so far are given below:
• Successful deployment of UWB sensors to transmit the signal for sensing the environment, which is then received and processed with MatLab, Python and PyTorch software.
• Successful configuration of multiple UWB sensors in indoor as well as laboratory environments.
• People detection, counting and classification achieved with multiple UWB sensors.
• Multi-path removal to achieve reliable people detection and counting in indoor environment using advanced signal processing techniques such as MCA-CFAR and clutter removal algorithms.
• People counting, activity recognition and gesture recognition tasks in multiple scenarios using advanced deep learning techniques.
• Successful extraction of vital signs from a resting human with no external movements.
• Design of novel algorithms for compensation of motion during vital signs monitoring which will be very helpful in vital signs monitoring inside a perceptive car.

The impact of the research activities and collaborations in the framework of this project is given below.
• Collision avoidance in Industry 4.0 and beyond
Our work related to multi-human detection, positioning and classification may play a significant role in collision avoidance in a hybrid human, robot environment in the Industry 4.0. The robust radar-based solution is not sensitive to the environmental conditions such as lighting, humidity and temperature unlike sensors like camera and LIDAR. This is not only useful for safety of the human participants but it can also reduce the costs associated with the damage of the autonomous vehicles during collisions in industries. Although the main goal of this work is to detect and avoid collisions in the Industry 4.0, this solution may also be suitable for implementing social distancing during a pandemic situation such as COVID-19. As our proposed approach provides very accurate and reliable indoor detection and positioning, therefore it can detect the situation when two people are getting closer than a certain threshold distance. It can also help the authorities to monitor the number of people in a given indoor environment.
• Context awareness tasks inside a perceptive car
Recently, researchers have used radar for different context awareness tasks such as occupancy detection and people counting. However, the physiological monitoring is still an unexplored area in context of perceptive car through UWB radar. The main challenge is the sensitivity of radar signals to movements. Our main goal is to achieve robust vital signs inside a car. To this end, we are developing state-of-the-art algorithms which involve different disciplines of signal processing, deep learning and algorithms design. We are testing other sensors such as audio recording to enhance our results. We believe that this multi-sensor fusion techniques and multi-disciplinary research based algorithms will provide a new direction to the radar based physiological monitoring and it would help the solution to be deployed in real life applications such as in-car, hospital and home monitoring of patients and elderly people. The in-car physiological sensing will be helpful for driver drowsiness detection which may prevent accidents and thus saving millions of lives in the future. Moreover, since the increase of elderly population has a huge impact on the demand for hospital beds, our solution has the potential to provide continuous, non-invasive health monitoring at home and office, and help reduce the pressure on the medical facilities in hospitals.
• Low cost UWB transceivers design
The part of the project that focuses on the optimal waveform design to achieve high spectral efficiency, and compressed sensing to design the low speed A/D converter in the UWB transceiver may be extremely helpful to decrease the cost of the transceiver and it may also be helpful to reduce the power requirements of the sensors. It could be beneficial for the commercialization of the proposed sensing solutions developed in the framework of this project and also advantageous from the environmental perspective.
• Overall impact
The overall vision of our research group is to deploy our solutions in real life environments such as hospitals, offices, cars and homes. Until now, we are on the right track to develop state-of-the-art solutions and we are considering all possible real life scenarios unlike the previous researches which were performed in controlled environments. We believe that the smart sensing solutions developed in the framework of this project will benefit the overall society and contribute to the development of smart cities and smart lifestyle.