Doctors Assistive System Using Augmented Reality Glass Critical Analysis

Abstract

Surgeons are constantly searching for new technologies that can improve their operating room. They frequently use technologies that enable their industry to provide a better surgery and patient experience early on. Numerous advancements have been recognized as potential disruptive technologies in the surgical workplace as a result of the ongoing improvement of the surgical environment in the digital age. Augmented reality (AR) are rapidly becoming increasingly available, accessible and importantly affordable, hence their application into healthcare to enhance the medical use of data is certain. Whether it relates to anatomy, intraoperative surgery, or post-operative rehabilitation, applications are already being investigated for their role in the surgeons. AR is the addition of artificial information to one or more of the senses that allows the user to perform tasks more efficiently. We propose a system in which important information for the doctors are displayed on semi-transparent glasses included in an AR-headset and therefore are mixed with the real-worldview. In this paper, the real time data of patients in hospital collected by the sensors attached to patients once the sensor measured the values then it is processed and send to doctors augmented reality glass through wireless and alert if abnormal condition occurs. The doctor can take appropriate action based on the patients current health condition.

Introduction

I. INTRODUCTION

Surgeons are constantly searching for new technologies that can improve their operating room. They frequently use technologies that enable their industry to provide a better surgery and patient experience early on. Numerous advancements have been recognized as potential disruptive technologies in the surgical workplace as a result of the ongoing improvement of the surgical environment in the digital age. Augmented reality (AR) are rapidly becoming increasingly available, accessible and importantly affordable, hence their application into healthcare to enhance the medical use of data is certain[1]. Whether it relates to anatomy, intraoperative surgery, or postoperative rehabilitation, applications are already being investigated for their role in the surgeons. AR is the addition of artificial information to one or more of the senses that allows the user to perform tasks more efficiently[2].

II. LITERATURE REVIEW

1. Filip Malawski, AGH University, “Driver Assistance System Using Augmented Reality Headset,” IEEE Transactions on Electronics and Telecommunications, vol. 3, pp. 978-1-5386, 2018.

Doctors are regularly on the lookout for technologies that will enhance their operating environment. The digital world, the continuing enhancement of the environment has led to number of innovative ideas being highlighted as potential disruptive technologies. Augmented reality (AR) application into healthcare helps to enhance the medical use of data. AR is the addition of artificial information that allows the user to perform the tasks more efficiently. Our system gives doctor a goggle which helps to identify patient’s details using augmented reality technology. In a hospital, the details such as patient’s temperature, pressure, heartbeat rate, activities of the body and respiration rate are measured for critical patients and alert if any abnormal condition occurs during surgery. Then the doctor can take appropriate action based on the patient’s current health condition.

2. Y. Xu, D. Xu, S. Lin, T. X. Han, X. Cao, and X. Li, “Detection of sudden pedestrian crossings for driving assistance systems,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 42, no. 3, pp. 729–739, 2012.

Recent advances in image processing and machine learning methods have greatly enhanced the ability of object classification from images and videos in different applications. Classification of human activities is one of the emerging research areas in the field of computer vision.

It can be used in several applications including medical informatics, surveillance, human computer interaction, and task monitoring. In the medical and healthcare field, the classification of patients’ activities is important for providing the required information to doctors and physicians for medication reactions and diagnosis. Nowadays, some research approaches to recognize human activity from videos and images have been proposed using machine learning (ML) and soft computational algorithms.

IV. COMPONENTS DESCRIPTION

A. Controller (ATMEGA328)

Controller is heart of our system. This controller following features: 32Kbytes of in-system programmable flash with read-while-write capabilities, two 8-bit Timer/Counters, 23 programmable I/O Lines, and operating Voltage is 1.8 - 5.5V, Temperature Range -40°C to 105°C, three flexible Timer/Counters. Pin configuration of ATmega328 IC consists of 28 pins. There is Port B, Port C & Port D an 8-bit bi-directional I/O port with internal pull-up resistors.

B. Heart Beat Sensor

The pulse rate sensor's main use is to monitor a person's heart rate. The pulse rate's maximum and minimum set points are provided during programming. If the pulse rate goes below or above the set point then the alert will be immediately issued by the m Sensor is the essential part of any instrumentation system. Sensing is the first stage of any process in the instrumentation system. Sensors are required to sense the variations in the physical quantities. According to the variations in the physical quantities sensors give the output, which is electrical in nature

V. CALIBRATION PROCESS

In this work, we use Microsoft HoloLens1 as the calibration tool. We implement our calibration process with Unity3D2. The first step is to calibrate the HoloLens’ coordinate system with the vehicle coordinate system. We use ICP algorithm to calculate the transformation between HoloLens spatial mapping and our reference model.

Next step is taking mappings by HoloLens’ front camera at different viewpoints to form the training data set. Normally 500 samples that approximately cover the feasible range are enough. Finally we train the model using Matlab3 nonlinear regression toolbox.

VI. OPERATIVE BENEFITS

As mentioned above, AR and VR have the potential to have a variety of unique effects on surgery, particularly in the area of surgical training in a virtual surgical environment (Figure 9). Real-time improvement of the surgical method, however, is yet a somewhat experimental use. It is not yet validated that surgery can be enhanced with AR and in some instances, it could be distracting. Some features may be useful of systems like GG where with voice activation the operator could communicate beyond the theatre environment, retrieve images and test results without breaking scrub. Real-time updates regarding the progress of the trauma list would reduce unnecessary fasting of patients in the event of a delay in theatre. Real-time augmentation of surgery usually involves the blending of acquired 3D imaging with surgical reference points. Novel applications of AR include use to project optimal port placement on the abdomen for laparoscopic surgery [4]; using AR to identify the position of sentinel nodes with 3D freehand single photon emission computed tomography; and using this with near infra-red spectroscopy to provide visual guidance in lymph node dissection in cancer surgery. Specialised near infrared (NIR) devices have been developed for the detection of tissue vascularity using indocyanine green (ICG) dye . The use of ICG in lymphatic surgery is already well developed to help identify vessels and check for their patency hence the move from microscope to HMD is a likely future development . AR technology would also be able to seamlessly project diagnostic images intraoperatively for surgical planning to guide surgeons with optimal incisions and approach [5] Several studies have demonstrated the use of AR to guide surgeons through intricate anatomy during minimally invasive surgery.

VII. RESULTS

The sensors and analog processing circuitry were put together on PCBs and put within the wrist strap. Fig 9 shows the prototype hardware. The prototype was powered with a 9 V battery. The RF transmission using Zigbee has been tested to operate successfully at 30 meters range through obstacles such as concrete walls. When in operation, the wrist unit consumes 20 mA of current at 3.3 V power supply.

Conclusion

Thus, the microcontroller atmega 328 is used to build and implement the zigbee-based wireless Heartbeat and Temperature monitoring system, in which all signals are directly measured from human bodies and all parameter values are presented on LCD on the transmitter side. Through ZigBee, this information is wirelessly delivered to the receiver. The patient\'s physiological parameters are shown on the computer by the received signal that is sent to it via AR Glass.

References

[1] Filip Malawski, AGH University, “Driver Assistance System Using Augmented Reality Headset,” IEEE Transactions on Electronics and Telecommunications, vol. 3, pp. 978-1-5386, 2018. [2] Nianchenn Deng, Yanqing Zhou, Jiannan Ye, Xubo Yang, “A Calibration Method for On-Vehicle AR-HUD System Using Mixed Reality Glasses,” IEEE Conference on Virtual Reality and 3D user interfaces, vol 6, pp. 978-1-3365, 2018. [3] Y. Xu, D. Xu, S. Lin, T. X. Han, X. Cao, and X. Li, “Detection of sudden pedestrian crossings for driving assistance systems,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 42, no. 3, pp. 729–739, 2012. [4] Q. Liu, J. Zhuang, and J. Ma, “Robust and fast pedestrian detection method for far-infrared automotive driving assistance systems,” Infrared Physics & Technology, vol. 60, pp. 288– 299, 2013. [5] S. P. Narote, P. N. Bhujbal, A. S. Narote, and D. M. Dha

Copyright

Copyright © 2023 Venkatapathi Pallam, Vasudev Biyyala, Chandra Shekar Jadapally, Ramsai Nalla, Dr. Sudhakar Alluri. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.