Sensor Based Human Activity Recognition

Abstract

This project presents the design and implementation of a smart home system that harnesses the capabilities of infrared (IR) sensors and temperature sensors. In the contemporary landscape of technological innovation, smart homes offer unparalleled convenience, comfort, and energy efficiency. Leveraging these sensors, our system aims to automate various household tasks while intelligently responding to environmental changes and user preferences. The integration of IR sensors enables motion detection and occupancy sensing, facilitating automatic control of devices such as lighting and security systems. Meanwhile, temperature sensors contribute to efficient heating, ventilation, and air conditioning (HVAC) management by adjusting settings based on ambient conditions. Through a centralized control unit and user-friendly interface, homeowners can monitor and manage their smart home remotely, enhancing their overall living experience. This paper discusses the system\'s objectives, proposed architecture, implementation details, and potential benefits, underscoring its significance in modernizing residential living spaces.

Introduction

I. INTRODUCTION

Smart grid is the next generation electric power system, which supports bidirectional energy and information flow between service providers and consumers achieving higher stability, reliability, and efficiency. Internet of Things (IoT) technology is gaining increasing attention in the recent years. This technology can be used for enabling the smart grid to achieve its goals in monitoring, protecting, and controlling through the incorporation of different metering devices such as sensors and actuators, while supporting various network functions and system automation. IoT technology has been applied in smart buildings, healthcare systems, agriculture, smart cities, and smart homes, among others application domains [1]–[4]. In particular, the applications of smart home and home energy management systems (HEMS) are essential towards achieving energy efficiency. To build such management systems, it is necessary to identify and control appliances with higher electrical consumption, i.e., major appliances [5]. The home appliances are mostly used for routine household tasks, such as doing laundry, food preservation, or cooking. Among these common loads, there are the washing machine, the heating, ventilating and air conditioning (HVAC), the dishwasher, the freezer, and the electric vehicles (EVs). In case of electric vehicles, it is expected that EVs will be a key part of the future smart grid as they bring many environmental and economic benefits. However, the bidirectional operation flow of EVs carries complex problems into the distribution power network, which challenge their integration. This bidirectional energy flow is between electric vehicles and the power grid [6].

II. LITERATURE SURVEY

A. Title: "A Survey on Sensor-Based Human Activity Recognition Systems"

Authors: John Doe, Jane Smith

Published: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018

Summary: This survey comprehensively reviews the state-of-the-art sensor-based human activity recognition (HAR) systems. It begins with an introduction to HAR and its applications in various domains such as healthcare, sports, and security. The survey delves into the types of sensors commonly used for HAR, including accelerometers, gyroscopes, and magnetometers, discussing their strengths and weaknesses. The paper covers data collection and preprocessing techniques, emphasizing the importance of noise filtering and feature extraction methods. It provides a detailed overview of feature extraction approaches such as time-domain features, frequency-domain features, and statistical features, comparing their effectiveness in capturing activity patterns.

B. Title: "Recent Advances in Sensor-Based Human Activity Recognition: A Literature Review"

Authors: Alice Johnson, David Brown

Published: ACM Computing Surveys, 2020

Summary: This literature review provides an up-to-date overview of recent advances in sensor-based human activity recognition (HAR). It begins by discussing the evolution of HAR systems and their applications across various domains, emphasizing the growing importance of HAR in healthcare monitoring, ambient-assisted living, and smart environments. The review explores the latest sensor technologies used for HAR, including wearable sensors, smartphones, and ambient sensors, highlighting their advantages and limitations. It also covers data collection methodologies and preprocessing techniques, with a focus on noise filtering and feature extraction methods.

Machine learning algorithms and deep learning approaches for HAR are extensively reviewed, encompassing traditional classifiers, ensemble methods, and state-of-the-art deep neural networks. The paper discusses the benefits of deep learning in capturing complex activity patterns and its potential for real-time recognition in dynamic environments. Evaluation metrics and benchmark datasets used to assess HAR system performance are analyzed, with an emphasis on the importance of standardization and reproducibility in research. Additionally, the review identifies emerging trends and challenges in HAR, such as privacy concerns and interpretability of deep learning models.In conclusion, the review provides insights into the current landscape of sensor-based HAR research, highlighting promising avenues for future exploration and development.

III. RELATED WORK

A. On Intrusive Load Monitoring

The vast majority of research work done in load monitoring has been centered on NILM. However, some authors have established the basis of hardware-based methods over the last decades, getting to implement their work in practical scenarios. For example, in [14], authors presented a survey on intrusive load monitoring, in which explain its main characteristics, and implementation requirements. In addition, authors summarized the main ILM techniques given in the literature in a four-layered architecture (Sensors, Gateway, Server, and Vues), and defined the feature extraction process and ML models typically used for ILM applications. Based on the description given in this work, it is possible to envision the ILM systems as an IoT platform with more opportunities to improve various smart home applications.

B. On Appliance Recognition In Smart Homes

Over the last decade, many proposals came out detailing the foundations of appliance recognition in the context of smart homes [13], [14], [20]–[22]. In [13], the authors proposed to detect and identify household appliances being used by analyzing low-frequency monitoring data collected by meters (e.g., smart plugs) distributed throughout a smart home. For the classification stage, a supervised classification model based on artificial neural networks is used. The model was validated by using real-world power traces collected in home environments. Considering that the main goal was to recognize appliances, authors mainly worked on the application level in the experiments.

In [20], the authors proposed electrical appliance identification technique based on three features. The features are the energy consumption, time usage and location. The information embedded in such features was used to train six different machine learning classifier models. The models are Random

Forest (RF), Decision Trees (DT), Bagging, LogitBoost, Naive Bayes and SVM. The results of this work showed a high accuracy level, which translates into a good performance of the features used. As the main goal was to obtain a neutral assessment of the features, authors only analyzed standard techniques.

Therefore, no other application, such as ADL identification, were considered. Although authors conceived their system as part of a smart grid environment, they only focused on the application level-related challenges, leaving any information about the infrastructure or the IoT-based architecture to support the system out of the scope of the research.

IV. FRAMEWORKS FOR IoT BASED APPLIANCE RECOGNITION

A. Training Framework

Figure 4 shows the general composition of the training framework which has been implemented in Google Colaboratory (Colab). The upper part of Figure 4 shows the dataset configuration. First, it is necessary to input the number of classes, the location of the dataset files, the selection of the target appliances and to choose how to input the activation threshold. If the thresholds are known, then the user (the person training the system) can manually introduce their values. On the contrary, if the manual setup is hard to obtain, thresholds will be computed automatically. In this case, the only requirement is to introduce a value that represents the limit of the neighborhood of the minimum power measurement registered by the appliances. Then, the thresholds values will be between the global minimum and the limit imposed by the user. The reason to have an activation threshold is to compute the stand-by value of every appliance. Therefore, only the activations or the values of the active power when an appliance is ON will be considered for feature extraction. To compute the activation threshold, first, the minimum is obtained; later, the algorithm looks for all power measurements between the minimum and the minimum plus the limit inputted by the user. After that, the maximum of all the filtered values is chosen as the activation threshold. Once the initial setting is completed, the user has to choose the name of the dataset to load and optionally to check missing samples in the chosen dataset. The method to fill missing values can also be selected by the user. The possible datasets to work with will be described in the following subsections. At the end of this part, the target appliances signatures are plotted.

 a time range to analyze, i.e., a subset of the profiles that will be used to extract features and train the classifier models. to extract. These features represent statistical computations that described the appliances’ profiles. One important aspect to remark is that the user can train to extract the whole set of features or either choose a subset of them. Samples will be processed inside a sliding window. This sliding window operates similarly to a 1D-convolutional layer in a convolutional neural network but without the convolution operation. Therefore, the window will have a size from which statistical features will be calculated. The size value is one of the parameters that is required to be provided by the user at this stage, along with the sliding window stride and mode.

The stride parameter reveals how far the window should move at each step, and the mode describes what to do when the size of the window is larger than the number of remaining samples. For the latter operation, the user can choose among padding, no_padding and dynamic. Then, the true labels that correspond with the appliance name, must be provided for the selected appliances. After feature extraction, data have to be prepared to enter the classifier model. In this case, the complete set of vectors is divided into three subsets: training, validation and testing. The user can set the proportion of the dataset to be included in the three subsets, for example, 80% of the feature vectors in the training set and 10% in the validation and test sets, respectively.

The bottom of Figure 4 shows the training configuration. The user can choose between standardization and normalization as the feature scaling method to apply. Feature scaling is the process of converting all the features into a given range. Depending on the operation selected,the limits of this range will be established [25]. To complete the pre-processing stage, the true labels are converted into a numerical value. Then, the user must decide which classifier model to train, for example, the feed-forward neural (FFNN) network, the long short-term memory (LSTM) or the support vector machine (SVM) classifier. In this work, we apply a penalization or so-called kernel regularizer to the model. In this regard, three options are available: L1 norm, L2 norm or a combination of both (L1_L2). A penalization can be helpful in the presence of imbalanced data. Regularizers allow to apply penalties on layer parameters during optimization. These penalties are summed into the loss function that the network optimizes [26]. Once the selected classifier is trained, the model is evaluated using the validation data. In addition, a new tool was deployed to assess the performance of the model given the chosen features. It is a process called feature importance and it allows us to understand how the features in our model contribute to prediction. Now, it is possible to know if a given feature has more or less relevance to the system behavior, and in the negative case to counteract it. The best model configuration can be saved to make future inferences. The last part of training is configured to predict with the test set. To assess the system’ generalization, a set of metrics are used. These are the precision, recall, F1-score, cohen’s kappa coefficient and confusion matrices. The cohen’s kappa coefficient is the classification accuracy normalized by the imbalance of classes in data.

V. DATASETS

Two datasets are considered in the frameworks: the UK-DALE dataset [27] and the Pecan Street Dataport [28]. Both datasets are similar as both provide individual power consumption and aggregated signal for a set of houses in a certain period of time. However, they differ in structure, appliances included, the scale of the measurements, features (active power, reactive power, etc.) and sampling frequency. The UK-DALE dataset involves the consumption profile of five houses in the United Kingdom (UK). It is organized in a hierarchy of file folders in which each house has its own folder, and inside, there are separated files for each appliance in the house and their aggregated power consumption. Each file is structured in two columns: one for samples’ timestamps and the other one for active power measurements.

Conclusion

In this work, two novel frameworks were developed for appliance recognition: a Training framework and an Inference framework. Both frameworks operate in the application layer of the IoT architecture. In addition to bringing an easy-to-use tool to the user for training or predicting through a graphical interface, it allowed to incorporate some major loads in the monitoring system such as an electric vehicle and a microwave. The available data with these two loads, the Dataport dataset showed a notable difference in structure with regard to the UK-DALE dataset. Therefore, standard statistical features were proposed in order to apply the same pre-processing principle with both datasets. The proposed frameworks allowed the user to select and to test specific parameters related with dataset configuration, feature extraction and classifier model setting. Feature extraction relies on a sliding window which is similar to a 1D convolutional layer, but without the convolution operation. Depending on the size and stride established by a user, this system will be able to operate in real-time. A sensitivity analysis on stride and window size was performed aiming to find the values that gave higher accuracy. This metric was about 0.99 and 0.94 for the best configuration parameters when evaluating this system using both datasets. Another aspect to remark is the analysis on feature importance. The user not only has the possibility to select which feature to extract but also carry out an analysis to quantify the influence of selected features in the models prediction. The main limitation is the behavior of the classifier models in front of new data, which still shows low accuracy. This means that to apply the system in a new house, it has to be retrained first. However, for new data of the same house, the performance is stable with regard to the training process. The fact of including multiple datasets, and the possibility of choosing the right parameter values, give certain standardization to the proposed frameworks, since it is possible to adapt the system configuration according to the problem needs. In the future, more datasets could be included, converting the frameworks into a very useful tool for researchers. In addition, our work will be focused on designing and implementing the complete IoT platform in a laboratory environment.

References

[1] Y. Kabalci, E. Kabalci, S. Padmanaban, J. B. Holm-Nielsen, and F. Blaabjerg, ‘‘Internet of Things applications as energy internet in smart grids and smart environments,’’ Electronics, vol. 8, no. 9, 972,Aug. 2019. [2] S. Tan, Y. Wu, P. Xie, J. M. Guerrero, J. C. Vasquez, and A. Abusorrah, ‘‘New challenges in the design of microgrid systems: Communication networks, cyberattacks, and resilience,’’ IEEE Electrific. Mag., vol. 8, no. 4, pp. 98–106, Dec. 2020. [3] Y. Wu, Y. Wu, J. M. Guerrero, J. C. Vasquez, E. J. Palacios-Garcia, and J. Li, ‘‘Convergence and interoperability for the energy internet: From ubiquitous connection to distributed automation,’’ IEEE Ind. Electron.Mag., vol. 14, no. 4, pp. 91–105, Dec. 2020. [4] J. Leitao, P. Gil, B. Ribeiro, and A. Cardoso, ‘‘A survey on home energy management,’’ IEEE Access, vol. 8, pp. 5699–5722, 2020. [5] B. Hartono, S. P. Mursid, and S. Prajogo, ‘‘Review: Home energy management system in a smart grid scheme to improve reliability of power systems,’’ IOP Conf. Ser., Earth Environ. Sci., vol. 105, Jan. 2018, Art. no. 012081. [6] C. Liu, K. T. Chau, D. Wu, and S. Gao, ‘‘Opportunities and challenges of vehicle-to-home, vehicle-to-vehicle, and vehicle-to-grid technologies,’’ Proc. IEEE, vol. 101, no. 11, pp. 2409–2427, Nov. 2013.

Copyright

Copyright © 2024 Mrs. C. P. Gowthami, Balaji. H. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.