AI Based Computer Assistant using Python

Abstract

Our proposed system AVATAR (AI Virtual Assistant Technology for Automatic Response) is an innovative voice assistant system, combining Artificial Intelligence (AI) and Python for human-like interactions. It seamlessly executes a range of functions, from dispatching emails to conducting searches on Wikipedia. The system\'s design incorporates essential Python packages and ultrasonic sensors for object detection and face recognition. Python\'s extensive libraries and clean syntax make it the optimal language for this project. AVATAR\'s security measures include biometric authentication and password protection. While internet connectivity is crucial for optimal performance, the system reliably operates within its designated input range. This research represents a significant leap in AI-driven applications, enhancing efficiency and user experience. Additionally, individuals who are blind or have amputations can utilize AVATAR, as the system is entirely voice-controlled.

Introduction

I. INTRODUCTION

Artificial intelligence, when integrated with machines, showcases the capacity to emulate human thought processes [1][2]. This entails designing a computer system that typically requires interaction from humans. Python, being a rapidly evolving language [3][4][5][6], offers a conducive environment for crafting scripts for voice assistants [3][7][8][11]. The assistant's instructions can be customized to suit the user's needs. The speech recognition system operates on par with well-known counterparts like Alexa and Siri[10]. Python boasts an API known as 'speech recognition,' enabling the conversion of speech into text. This empowers users to seamlessly control their assistant without the need for manual keyboard input. In essence, users can fully automate their computer, incorporating features for home automation [9]. Through the execution of this project, we have come to appreciate how the integration of AI in various domains significantly enhances daily life, leading to increased efficiency and time savings.

A.   Functionalities of AVATAR

The capabilities of this system represent a convergence of cutting-edge technologies, redefining interactive experiences. It seamlessly executes a spectrum of functions, including but not limited to:

1. Effortlessly dispatching emails with precision and speed.
2. Proficiently parsing through PDF documents, extracting information with finesse.
3. Initiating conversations on WhatsApp, seamlessly integrating with modern communication platforms.
4. Swiftly launching a suite of applications, from command line utilities to robust development environments like Command Prompt, Visual Studio Code, and more.
5. Demonstrating exceptional acuity in facial recognition, ensuring accurate identifications.
6. Leveraging advanced algorithms to discern and measure distances to objects, opening avenues for a myriad of applications.
7. Showcasing emotional intelligence through engaging dialogues and insightful questioning, emulating human-like interactions.
8. Expertly categorizing an extensive array of objects, from living organisms to inanimate structures, enhancing its versatility.
9. Harmonizing music choices with the user's emotional state, curating tailored playlists for a personalized auditory experience.
10. Seamlessly connecting with Bluetooth devices, expanding its integration with the user's ecosystem.
11. Conducting exhaustive searches on Wikipedia, delivering in-depth knowledge at your fingertips.
12. Launching web browsers with precision, providing quick access to popular platforms like Google, YouTube, Facebook, and more.
13. Delivering real-time weather forecasts, keeping users informed and prepared.
14. Issuing desktop reminders, aligning with user preferences and ensuring timely notifications.
15. Automating tasks across applications, streamlining workflows in Chrome, YouTube, WhatsApp, and beyond.
16. Infusing humor into interactions, customizing jokes to align with the user's mood.
17. Setting alarms promptly, enhancing time management and productivity.
18. Offering essential information such as current date, time, and more, seamlessly integrating with daily routines.
19. Executing system commands through voice prompts, offering a hands-free, efficient approach to computer operations.
20. Managing core computer functions, from volume adjustments to window management, optimizing the user experience.
21. Governing electronic devices like TVs and household appliances, expanding its reach beyond the digital realm.

In the development process, Visual Studio Code stood as the cornerstone of our environment, coupled with Python (.py) files, driving the engine of innovation. The integration of a diverse array of modules and libraries, including Speech Recognition, Datetime, Wikipedia, pyttsx3, Smtplib, pywhatkit, pyjokes, pyPDF2, pyautogui, pyQt, and others, empowered our system with unparalleled capabilities. The user interface was meticulously crafted for seamless and intuitive operation, creating an immersive and engaging experience with the Assistant. Our voice assistant, aptly named AVATAR, is a testament to the convergence of language processing and voice recognition principles. It offers a suite of features comparable to its contemporaries. Powered by Artificial Intelligence, it ensures precision and efficiency in every interaction, streamlining tasks and eliminating the need for manual input. The algorithms underpinning this assistant are meticulously optimized, exemplifying efficiency in every operation.

II. LITERATURE REVIEW

1. AI technology is gradually advancing in various professional systems, including Natural Networks (NN), Natural Language Processing (NLP), and Speech Recognition. These advancements are made possible through coding languages like Python, C++, C, etc. This technology is primarily visible in fundamental instruments such as smartwatches, health bands, speakers, Bluetooth earphones, cell phones, computers, and desktops. While this system works efficiently, there is still room for improvement [1][2].
2. The current version of Assistant, as described by Shende et al.[7], possesses some commendable features and is responsive. However, there is still much room for enhancement in terms of understanding and reliability. The paper also suggests that incorporating technologies like merging NLP, Machine Learning[4], IoT, etc., can lead to even better results. The potential of virtual assistants extends far beyond their current capabilities.
3. Python, a highly versatile programming language, is invaluable for creating virtual assistants. Python greatly simplifies the coding process for virtual assistants, thanks to its user-friendly syntax. It excels in performing various tasks due to its lower complexity compared to other languages. All necessary modules can be easily installed and utilized with Python [3].
4. According to the home automation system outlined by Pal et al.[10], it essentially involves operating electrical appliances wirelessly through user interface devices using voice commands. In this system, high-range communication devices can be implemented instead of relying solely on Bluetooth modules.
5. This review outlines the creation of a user-friendly, Python-based voice-enabled personal assistant for PCs. It significantly improves time management and productivity, performing various tasks with a single voice command. This innovation marks a noteworthy shift in human-computer interaction as described by Geetha al [11].Top of Form

III. PROPOSED WORK

In this project, the chosen Integrated Development Environment (IDE) is Visual Studio Code, serving as the primary platform for crafting and managing all Python files. Its seamless package management system facilitates easy installations. The project leverages an array of essential modules and libraries, including but not limited to pyttsx3, Speech Recognition, Datetime, Wikipedia, Smtplib, pywhatkit, pyjokes, pyPDF2, pyautogui, pyQt. A vibrant Graphical User Interface (GUI) has been meticulously designed to imbue conversations with AVATAR with an aesthetically pleasing and functionally intuitive interface.

A. System Design

The system is designed using principles of Artificial Intelligence and relies on essential Python packages. Python provides a wide array of libraries and packages that facilitate various tasks; for example, pyPDF2 can be used to read PDF files. The system consistently employs an ultrasonic sensor to monitor and measure the distance of any object (user) in front of it. Upon detecting an object (user) within a specific range, it initiates the face recognition process. This involves capturing an image or a series of images of the user's face. The captured face image is then compared with a database of pre-registered faces to verify a match. This database can be either locally stored or accessed remotely, depending on the system's configuration. In the event of a matching face, the system prompts the user for a password, adding an extra layer of security to ensure authorized access. The entered password is subsequently compared with the stored password associated with the recognized face. If both the face and password match, access is granted, allowing the user to utilize the system. Conversely, if there is no match or the password is incorrect, the system reverts to object detection and measurement mode, continuing to monitor the area and waiting for another user to approach.

When our system allows any user to utilize it, it will first take voice commands through the microphone, recognize the voice, process the input voice, and then execute the requested task. At the core of this project lies a fundamental workflow where the assistant dynamically responds to user input, seamlessly executing tasks in accordance with the user's articulated desires expressed in the richness of the English language. The workflow of this AI system is shown in Fig. 1.Top of Form

B. Imported Python Libraries                       

In our AI Assistant following python libraries are used:

1. SpeechRecognition: It is a python module that converts speech to text.
2. pywhatkit: It is python library to send WhatsApp messages at a particular time with some additional features.
3. Datetime: This library provides us with the actual date and time.
4. Wikipedia: It is a Python module for conducting searches on Wikipedia.
5. Smtplib: Simple mail transfer protocol that allows us to send mail and route mail between mail servers.
6. pyPDF2: It is a Python module capable of reading, splitting, and merging any PDF.
7. Pyjokes: It is a Python library that contains a collection of humorous jokes.
8. Webbrowser: It provides an interface for displaying web-based documents to users.
9. Pyautogui: It is a python library for the graphical user interface.
10. OS: It represents Operating System-related functionality.
11. SYS: It allows for operation within the interpreter, providing access to the variables and functions that typically have a close interaction with it.
12. Requests: The 'requests' module enables you to send HTTP requests using Python. The HTTP request returns a response object containing all the response data (content, encoding, status, etc).
13. Random: The random module is a built-in module used to generate pseudo-random variables. It can be employed to perform various actions randomly, such as obtaining a random number, playing random music, selecting random elements from a list, or displaying random videos, among other things.
14. Keyboard: This module is highly useful for simulating key presses, registering hotkeys, and performing various other actions. By importing the keyboard module in a Python program, we can carry out these actions.
15. JSON: This specific module is primarily used to convert the Python dictionary above into a JSON string that can be written into a file. It is commonly employed to read and write directly from JSON files.
16. BeautifulSoup: Beautiful Soup is a python library that is used to pull the data out of XML and HTML files.
17. Wolfram Alpha: This is API. It can compute answers using Wolfram algorithms, knowledgebase, and AI technology.
18. Subprocess: This is a specific Python module used to execute applications and codes by creating new processes within a Python script.
19. pyttsx3: It is a python library that converts text to speech.
20. Face_Recognition: This particular module of Python helps to recognize faces.
21. Cv2: This module helps to capture video or images.
22. Keyboard: By using this module one can take full control of the keyboard (Desktop, computer)

C.  Functions

1. takeCommand(): This function is utilized to capture user commands through the microphone as input and then returns the output as a string.
2. speakFunction(): This function is used to convert written words into spoken English. With Python, we can make computers speak
3. wishMe(): This function greets the user according to the time like Good Morning, Good Afternoon, and Good Evening.
4. Tester(): The 'tester' function is employed for clap detection. It was designed to identify a clap, allowing our system to initiate with a single clap.
5. Pass(): This specific function serves a security purpose. It enables the setting of a password for enhanced security within the system.
6. Read_distance(): This function is employed to measure the distance and detect if a user is present in front of the system.
7. latestnews(): This function helps to tell news.
8. WhatsAppMSG(): Helps to automate WhatsApp.
9. taskExecution(): This function encompasses all the essential task execution definitions, such as sending an email, using the PDF reader, fetching news, and handling various conditional statements like 'open Google,' 'open Notepad,' 'search on Wikipedia,' 'play music,' and 'open Command Prompt,' among others.

The imported python libraries and functions are shown in Fig. 2 and Fig. 3.

D. Why Choose Python Language for AVATAR

Although python was created before machine learning and AI become so popular, we have used the python language because python has certain qualities that set it apart from other languages. The qualities are discussed below.

1. Extensive Collection of Packages and Libraries: Leveraging a diverse range of packages and libraries for working with various types of data—textual (NLTK, SciKit, and NumPy), images (Sci-Kit image and OpenCV), audio (Librosa), and implementing deep learning models (TensorFlow, Keras, PyTorch), as well as visualizing data effectively (Matplotlib, Sci-Kit, Seaborn). This resourceful toolkit significantly accelerates development and enhances productivity in programming.
2. Code Readability: Python's clean syntax enhances machine learning endeavors, making program composition more straightforward and easily comprehensible.
3. Flexibility: Python's inherent versatility allows for the seamless design of complex tasks with minimal effort.

The popularity of the Python language is depicted in Figure 4.

IV. RESULTS AND DISCUSSION

AVATAR, our advanced voice assistant, represents a pinnacle of technological innovation. Its multifaceted capabilities span from efficient email dispatch to precise facial recognition, showcasing a seamless fusion of language processing and Artificial Intelligence. This remarkable system excels in tasks as diverse as curating personalized playlists based on the user's emotional state to delivering real-time weather forecasts. Its integration with Bluetooth devices and control over electronic appliances further extend its utility beyond the digital realm. The meticulously optimized algorithms driving AVATAR ensure not only efficiency but also a human-like touch, evident in its engaging dialogues and insightful questioning. By executing system commands through voice prompts, it offers a hands-free and efficient approach to computer operations, exemplifying the future of interactive technology.

A. Specialities of AVATAR

1. Biometric Security: This system boasts top-tier security measures, allowing access only through a secure password or advanced face recognition technology. This ensures that unauthorized access is virtually impossible.
2. Autonomous Computing: Users can seamlessly automate their computer operations, eliminating the need for manual intervention. When within a specific range, the system intuitively activates, providing a seamless and responsive computing experience.
3. Cognitive Questioning: Driven by deep learning, our system's cognitive abilities enable it to learn from past interactions. This empowers it to pose questions with a human-like thoughtfulness and intuition, transcending traditional voice assistants.
4. Comprehensive Room Automation: Beyond the digital realm, users can orchestrate the automation of electronic components such as lighting, fans, TVs, speakers, and even wearables. This advanced integration elevates the user's environment to new levels of sophistication.
5. Clap Activation: With a single clap, users can effortlessly kickstart the system, showcasing its responsiveness to auditory cues and its seamless integration into the user's environment.
6. Seamless Multitasking: The system's forte lies in its multitasking prowess. Once activated, it executes tasks in succession, without any interruptions, until prompted to stop. This ensures a streamlined workflow and enhanced productivity.
7. Error Resilience: Even in the face of incorrect pronunciation or missing commands, the assistant maintains its accuracy. It continues running without executing incorrect tasks, demonstrating its adaptability and user-centric design.
8. Dynamic Voice Options: The assistant offers a dynamic range of voices, both male and female, further personalizing the user experience and enhancing interaction versatility.
9. Conversational Interaction: By leveraging essential Python modules and libraries in a conversational manner, the system simplifies complex tasks, making interaction intuitive and user-friendly.
10. Responsive Intelligence: With a profound understanding of human language and context, the desktop assistant provides responses in clear and comprehensible English. This enables users to engage with it in an informed and intelligent manner, elevating the interaction beyond mere commands.
11. Triggerless Execution: Dispensing with the need for trigger phrases, the system intuitively prompts for instructions, listens attentively to the user's response, and proceeds to execute tasks seamlessly. This frictionless interaction sets a new standard for user-centric design in voice assistants.

Some screenshots of task executions are shown in the Fig. 5, Fig. 6, Fig. 7, Fig. 8 and Fig. 9.

B. Limitation

Our system is stable, providing bounded output for a specific range of input. It functions effectively across all its intended capabilities. Given that our assistant relies on internet connectivity to receive input commands, a robust internet connection is imperative. With a faster internet connection, tasks are executed swiftly and with high accuracy.   

C.   Security     

Although this system is fully protected by a password and also includes a face lock feature to ensure that no unauthorized person can access it without the user's permission. The security testing primarily focuses on identifying vulnerabilities and potential risks. Since AVATAR is a local desktop application, there is no risk of a data breach through remote access. The software is dedicated to a specific system, and it will only be activated when the user logs in.

D.   Stability

Our system is stable, providing bounded output for a specific range of input. It functions effectively across all its intended capabilities. Given that our assistant relies on internet connectivity to receive input commands, a robust internet connection is imperative. With a faster internet connection, tasks are executed swiftly and with high accuracy.

Conclusion

AVATAR stands at the forefront of voice assistance technology, a groundbreaking system that seamlessly combines natural conversation with unmatched effectiveness and efficiency. Its cutting-edge features not only differentiate it from its predecessors but also set new standards in the field. It\'s worth noting that throughout the development process, we diligently recognized the importance of addressing accent-related challenges to ensure the highest level of accuracy and inclusivity. Looking ahead, there are promising avenues to explore. The development of a dedicated AVATAR Android app is on our agenda, extending its accessibility to a broader user base. Furthermore, the foundational principles of AVATAR provide a robust platform for the creation of a new generation of voice assistants, poised to redefine the landscape of human-machine interaction. While our system currently boasts robust password and biometric protections, we acknowledge the imperative need for voice command encryption to further bolster security measures. In our pursuit of excellence, we are actively researching and investing in state-of-the-art microphone technology to address the challenge of background noise interference, a critical step towards ensuring seamless functionality in diverse environments. Looking further into the future, the integration of machine learning, deep learning, and AI technologies holds the promise of propelling the system to unprecedented heights, endowing it with the capacity to learn and execute tasks with unparalleled precision. Through dedicated deep learning training regimens, we are committed to maximizing AVATAR\'s capabilities. Envisioning the horizon, the potential emergence of a humanoid robot inspired by our system is a tantalizing prospect. With boundless possibilities at our fingertips, we dare to imagine an entire edifice seamlessly controlled by this visionary system, ushering in a new era of intelligent infrastructure management.\"

References

[1] Ashutosh Sakharkar, Shidesh Tondawalkar, Pratik Thombara, Prof. Rajashri Sonawane, “Python Based Ai Assistant For Computer,” International Research Journal of Engineering and Technology(IRJET), Volume:08, Issue:04, Apr 2021. [2] R. Sathya, M. Pavithra, G. Girubaa, “Artificial Intelligence For Speech Recognition,” International Journal of Computer Science & Engineering Technology (IJCSET), Volume:08, No. 01, Jan 2017. [3] Pooja C.Goutam, Monika S. Jalpure, Akshata S. Gavade, Pranjali Chaudhary, Prof.A.V.Gundavade, “Voice Assistant Using Python,” International Journal of Creative Research Thoughts(IJCRT), Volume:10, Issue: 06, June 2022. [4] Maghna Chandel, Sanjay Silakari, Rajeev Pandey, Smita Sharma, “A Study on Machine Learning and Python’s Frameworks,” International Journal of Computer Science and Engineering(JCSE), Volume:10, Issue:05, May 2022. [5] Anjali Fapal, Trupti Kanade, Bharati Janrao, Mrunalini Kamble, Megha Raule, “Personal Virtual Assistant For Windows Using Python,” International Research Journal of Modernization in Engineering Technology and Science (IRJMETS), Volume:03, Issue:07, July 2021. [6] Deepak Shende, Ria Umabiya, Monika Rathore, Aishwarya Bhisikar, Anup Bhange, “AI Based Voice Assistant Using Python,” International Journal of Emerging Technologies and Innovative Research (IJETIR), Volume: 06, Issue: 02, February 2019. [7] Divisha Pandey, Afra Ali, Shweta Dubey, Muskan Srivastava, Shyam Dwivedi, Md. Saif Raza, “Voice Assistant Using Python And AI,” International Research Journal of Engineering and Technology(IRJET), Volume: 09, Issue:05, May 2022. [8] Dr. Jaydeep Patil, Atharva Shewale, Ekta Bhushan, Alister Fernandes, Rucha Khartadkar, “A Voice-Based Assistant Using Google Dialogflow and Machine Learning,” International Journal of Scientific Research and Technology(IJSRT), Volume:08, Issue:03, May 2021. [9] Qzer Celik, Serthan Salih Altunaydin, “A Research on Machine Learning Methods and Its Applications,” Journal of Education Technology and Online Learning (JETOL), Volume:1, Issue: 03, August 2018. [10] Sitaram Pal, Ashish Chauhan, Sanjeev Kumar Gupta, “Voice Controlled Smart Home Automation System,” International Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume: 8, Issue, September 2019. [11] V. Geetha, C.K.Gomathy, Kottamasu Manasa Sri Vardhan, Nukala Pavan Kumar, “The Voice Enabled Personal Assistant for Pc using Python,” International Journal of Engineering and Advanced Technology (IJEAT) ISSN: 2249-8958, Volume-10 Issue-4, April 2021.

Copyright

Copyright © 2023 Indrajit Roy, Apurba Ghosh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.