Stock Market Prediction using Deep Learning

Abstract

Stock market is very uncertain and highly volatile as the prices of stocks keep fluctuating due to several factors that make prediction of stocks a very difficult and complicated task. In the finance and trading world stock analysis and trading is a method for investors and traders to make buying and selling decisions. Investors and traders try to gain an edge in the markets by taking informed decisions by studying and evaluating past and current data. Stock market prediction has always been an important research topic in the financial and trading field [2]. Prediction of stock market is the act of trying to determine the future value of a company stock (nifty & sensex) or other financial instrument traded on an exchange. Our project explains the prediction of a stock using Machine Learning, which itself employs different models to make prediction easier and authentic. The paper focuses on the use of Recurrent Neural Networks (RNN) called Long Short Term Memory (LSTM) to predict stock values. This will help us provide more accurate results when compared to existing stock price prediction algorithms. The eminent analysis of the stock will be an asset for the stock market investors and will provide real-life solutions to the problems and also yield significant profit.

Introduction

I. INTRODUCTION

Stock markets can be defined as dynamic, unpredictable, non-linear and highly volatile in nature. Stock price predictions are very important among many business people and retail investors. Predicting stock market prices is a difficult and challenging task as they are complex and diverse and it depends on various economic factors like economic uncertainty, company’s financial reports and performance and price indicator as well as non-economic factors such as political conditions, and investor’s expectations, etc. The prices of stocks are mainly governed by demand and supply, and the ultimate goal of buying shares is to make money by buying stocks in companies whose share price is expected to jump up [6]. Therefore to obtain higher trading profits and reduce unnecessary losses, the investors usually expect various techniques to predict and analyse the stock price movements and various trends. Stock market prediction therefore has been a major research topic among researchers in the financial area and captivates the attention of many investors. Interpreting the stock price pattern of a particular company by considering their past data and predicting their future growth and financial development will be highly beneficial.

There are two common methods of attempting to forecast stock prices of an organization. The first is fundamental analysis, which considers external factors like company profile, market situation, political and economic factors, textual-information in the form of financial news articles, social media and even blogs by economic investigators. The second is technical analysis, that attempts to find patterns in charts and use past price trends of stocks like closing and opening price, volume traded, adjacent close values and many more, to predict future price action. Nowadays, for predicting stock prices, advanced intelligent techniques based on either technical or fundamental analysis are used. 

Based on the data of historical stocks the stock price can be predicted. The most promising and prominent technique involves the use of Recurrent Neural Networks (RNN), that is basically the implementation of machine learning. Machine learning has been widely used in the capital market and plays a major role in predicting future stock prices based on historical data. Machine learning involves artificial intelligence which empowers the system to learn and improve from past experiences without being programmed time and again, thereby increasing the accuracy [2].

The proposed approach considers the available historic data of a particular share and it provides predictions on a particular feature. In order to predict a share price for a required time period, the proposed model uses the time series analysis. This model applies a type of recurrent neural network (RNN) capable of addressing linear problems and predicting time series- Long Short Term Memory (LSTM) networks. LSTM is a deep learning technique. Long-term Memory (LSTM) units execute very long sequences. LSTM evaluates the time series data by using both the historical and the present stock data accurately [2]. LSTM replaces the traditional artificial neurons in the network layer into the most useful memory cells. With these memory cells, networks are able to associate memory with remote input over time. Over the past few years, LSTM has been applied to stock market prediction in different stock markets around the world [2].

The most important aspect of machine learning is the dataset used. The dataset should be as solid and concrete as possible because a little change in the data can prolong massive changes in the results. This dataset comprises the following closing variables for companies like TATAGLOBAL, Facebook, Tesla, Microsoft and Apple. The model is then tested with the help of test data.

II. LITERATURE SURVEY

A. LSTM Based Stock Price Prediction

The proposed work by Pritam Ahire, Hanikumar Lad, Smit Parekh, Saurabh Kabrawala, D.Y Patil Institute of Engineering and Technology, Pune, India focuses on the use of recurrent neural network (RNN) based Machine learning techniques known as Long Short Term Memory (LSTM) to predict stock values and to provide efficient stock price prediction. The system predicts the stock prices for companies like Alcoa Corp Company, Carnival Corp, Tesla Corp. etc. The system is based on five factors such as: Date of stock price, Opening price, High, Low, Volume and the Close Interest for the respective companies.

B.  Stock Prediction Based on Optimized LSTM and GRU Models

The system presents a new model for optimizing stock forecasting. The authors Ya Gao, Rong Wang ,and Enmin Zou from School of Public Finance and Taxation, Central University of Finance and Economics, Beijing, China, School of Computer Science and Technology, Xidian University, Xi’an, China and School of Electronics and Information, Xi’an Jiaotong University, Xi’an, China include a range of technical indicators, including investor sentiment indicators and financial data, and perform dimension reduction on the many influencing factors of the retrieved stock price using deep learning  approaches as LASSO and PCA. In addition to this, a comparison of the performance of LSTM and GRU for stock market forecasting under various parameters is performed. The result of which show that both the approaches- LSTM and GRU can equally predict the stock prices efficiently [2].

C. Stock Market Prediction Using LSTM Recurrent Neural Network

The authors Adil Moghar and Mhamed Hamiche from University Abdelmalek Essaadi, Morocco have build a model to see in which precision a Machine learning algorithm can predict and how much the epochs can improve the  model. The system is build using Long-Short Term Memory model (LSTM) technique.

The data in this paper consist of the daily opening prices of two stocks in the New York Stock Exchange NYSE (GOOGLE and NKE) extracted from yahoo finance, for GOOGLE. For training the data the system uses mean squared error to optimize the model. Also, different Epochs for training data are used. This model is capable of tracing the evolution of opening prices for both assets used [3].

D. Stock Price Prediction Using LSTM

The proposed system is presented by Pramod B S and Mallikarjuna Shastry P. M., from REVA University, Bengaluru. As compared to today’s available prediction algorithms, this system gives accurate results.

The proposed algorithm uses machine learning techniques like recurrent neural networks named as Long Short Term Memory using the market data to predict the share price.

The network is trained and evaluated with multiple sizes of input data to urge the graphical outcomes. The various parameters used in this model comprise of Date, Open, Close, Volume/ trade quantity, High, Low and Turnover of TATAMOTORS share. The proposed model is able to predict the share price with very low loss and error rate, if there is an increase the epoch batch rates, the training will be more methodical.

E. Stock Market Prediction Using Machine Learning

This paper focuses on the usage of Regression and LSTM based Machine learning to predict stock values. Regression and LSTM models are engaged for this speculation separately. Regression involves minimizing the errors and LSTM remembers the data and results for the long run.

The authors in this project, Ishita Parmar, Navanshu Agarwal, Sheirsh Saxena, Ridam Arora, Shikhin Gupta, Himanshu Dhiman, Lokesh Chouhan, Department of Computer Science and Engineering, National Institute Of Technology, Hamirpur, India, uses supervised machine learning on a dataset obtained from Yahoo Finance. This dataset comprises the following five factors: open, close, low, high and volume [5].

F. Stock Price Prediction Using Long Short Term Memory

The proposed system by authors Raghav Nandakumar, Uttamraj K R, Vishal R, Y.V Lokeswari, Department of Computer Science and Engineering, SSN College of Engineering, Chennai, Tamil Nadu, India, propose an online learning algorithm which uses recurrent neural network (RNN) called Long Short Term Memory (LSTM), where the weights are adjusted for individual data points using stochastic gradient descent. This provides accurate outcomes when compared against existing stock price prediction algorithms. Benchmark stock market data was obtained from two primary sources: Yahoo Finance and Google Finance. The obtained data contained five features: Date, Opening price, High, Low, Volume, OpenInt. Also a comparison with respect to accuracy is done against an Artificial Neural Network. The accuracy of the prediction model is calculated using the RMSE (Root Mean Squared Error) metric [6].

III. PROPOSED SYSTEM

By proper use of machine learning techniques and algorithms, we can relate the previous data to the current data and train the machine to learn from it and make proper assumptions. Machine learning has many models but this paper focuses one of the most important and accurate of them which make the predictions efficient using it. The project will be a great asset for traders and investors for investing money in the stock market since it is trained on a huge collection of historical data. The project demonstrates the use of a machine learning model i.e. LSTM to predict the stock value with more accuracy as compared to other models. Analysis of stocks using deep learning will be useful for new investors to invest in the stock market. Stock market includes daily activities like nifty and sensex calculation, exchange of shares.

The proposed system measures the accuracy of stock prices by using the predictions for the test set and the actual values. The system also uses different areas of research including data pre-processing, LSTM and so on. In this proposed system, we will focus on predicting the future trends of the stock values using machine learning algorithms Long-Short Term Memory (LSTM) algorithm, a type of recurrent neural network. In this system, we train the machine by taking the various datasets from the past to make an accurate future prediction. Datasets from previous 10 years' stocks are used to train the model. We will majorly use five libraries like numpy, panda, DateTime, Matplotlib and scikit to solve the problem. 

Fig. 1: Proposed System architecture

The system works on a Comma Separated Variable (CSV) file, which has a record of all the dates and its raw data of the closing variable. From this raw data, knowledge is extracted by performing data pre-processing and refining to predict a close information for requested date of future. Once the knowledge is available, it will be fed to the LSTM algorithm to perform stock prediction and give a data visualization using python, this investment prediction will be subdivided into different time frames and a suitable advice from the prediction can be given to the consumer, as shown above.

IV. METHODOLOGY

A. Recurrent Neural Network(RNN)

A recurrent neural network (RNN) is a type of artificial neural network which uses chronological data or time series data. Recurrent neural networks utilize training data for knowledge. They are distinguished by their “memory” as they take information from previous inputs to influence the current input and output. On the other hand, traditional neural networks assume that inputs and outcomes are not related to each other, the output of the recurrent network depends on the prior attributes within the sequence. The main and most important feature of RNN is Hidden state, which can remember some data about a sequence.

B. Long-Short Term Memory(LSTM)

The LSTM is a special kind of RNNs due to its ability to memorize sequences of data. It contains information in memory, similar to computer memory. This memory can be seen as a closed group of cells, with a closed description, the cell decides to store or remove information. Each node in LSTM consists of a set of cells which are responsible for storing data streams that have passed, in each individual cell there is the upper line which connects the models as a transport line carrying the data from the past to the present ones, the independence of cells helps the model dispose of filters or add values of a cell to another. At last, the sigmoidal neural network (SNN) layer composing the gates drive the cell to an optimal value by disposing or letting data pass through.

Fig. 2: LSTM model

There are three gates in LSTM: input, forget and exit gate. These gates determine whether new input should be allowed, data deleted because it is not important, or allow it to affect output at current timeline.

1. Forget Gate: It determines when certain parts of the cell will be inserted with data that is most recent. It subtracts 1 in parts of the cell state to be kept, and 0 in values to be ignored.
2. Input Gate: Based on the input, this network category reads the conditions under which information should be stored or updated in the state cell.
3. Output Gate: Depending on the input mode and the cell, this gate determines which information is forwarded in the next location in the network.

C. Fundamentals

1. Data Collection:  Data collection is the basic and initial step. It deals with the collection of the right dataset. Based on various aspects, the dataset that is to be used in the stock market prediction has to be modified. Data collection also complements the dataset by adding more data that is external. The data used in this project mainly consists of the previous few years' stock prices. Initially, we will be analysing the Kaggle dataset and according to the accuracy, we will be using the model with the data to analyse the predictions accurately [3].
2. Pre Processing:  Data pre-processing is a part of data mining, which involves transforming raw data into a more reasoned format. Raw data is usually incomplete or inconsistent and usually contains many errors. The data pre-processing involves checking out for missing values, looking for categorical values, splitting the data-set into training and test sets and finally doing a feature scaling to limit the range of variables so that they can be compared on common technologies [3].
3. Training the Machine: Training the machine is similar to feeding the data to the algorithm to test the data. The models are tuned and fitted using Training sets. The training of the model comprises cross-validation where we get a well-grounded approximate performance of the model using the training data.
4. Data Scoring: Scoring the data is referred to as a process of applying a predictive model to a set of data. The technique used to process the dataset is the Long-Short Term Memory. We achieve interesting results, based on these learning models. Thus describes how the result of the model can help to predict the possibility of a stock to rise and fall based on certain parameters
5. Output Block Description: The stock selected by the user acts as an input to the system. The selected stock contains the time period for which the user needs the prediction. Analysis of the input data takes place resulting in generation of graph which acts as an output.

V. IMPLEMENTATION AND RESULTS

The implementation of proposed system using LSTM model using python shows the prediction of the future price of TATAGLOBAL share based on its historical data. The below visualization scatter plot figure shows the visualization of TATAGLOBAL prediction. In the result shown in the below graph is the plotted form our algorithm outcome by applying LSTM for achieving the accuracy [4].

The above image shows a graphical representation of the closing price of a particular user selected stock. This scattered plot shows the date and time along with the closing price.

This GUI consists of a drop down, which consists of stocks. The user is supposed to select single or multiple stocks from given stock list. It also shows a clear picture of highs and lows of selected stock.

This is stock analysis dashboard wherein the user can compare multiple stocks based on the past performance and he/ she can decide whether they want to invest in these stocks or no.

The proposed model is able to predict the share price with very low loss and error rate. From the implementation and results, we can thus infer that in general, the prediction accuracy of the LSTM model improves with increase in the size of the dataset and makes is more efficient [6].

VI. ACKNOWLEDGMENT

It is our privilege to express our sincerest regards to our project guide Prof. Suresh Babu K.S for giving his valuable time and suggestions. We deeply express our gratitude to our Head of the Information Technology Department Dr. Satishkumar Varma and our Principal Dr. Sandeep M. Joshi for encouraging us with whole-hearted cooperation and constructive criticism throughout the duration of this work and allowing us to showcase our work.

Conclusion

Predicting the stock market is a time-consuming and strenuous procedure. However, with the introduction to Machine Learning and its various algorithms, the Stock Market Prediction advancements have begun to include such approaches in analysing stock market data. By measuring the accuracy of the different algorithms, we found that the most suitable algorithm for predicting the market price of a stock based on various data points from the historical data is the Long-Short Term Memory (LSTM) algorithm. The algorithm will be a great asset for brokers and investors for investing money in the stock market since it is trained on a huge collection of historical data and has been chosen after being tested on a sample data. The project demonstrates the machine learning model to predict the stock value with more accuracy as compared to other machine learning models. In the future, the stock market prediction system can be further improved by utilizing a much bigger dataset having higher computing capacities than the one being utilized currently and number of training epochs that better suit our assets and maximize our predictions accuracy [3]. Furthermore, other models of Machine Learning could also be studied to check for the accuracy rate resulted by them. The sentiment analysis from social media can be linked with the LSTM to better train weights and further improve accuracy [6]. It has led to the conclusion that it is possible to predict the stock market with more accuracy and efficiency using machine learning techniques.

References

[1] Pritam Ahire, Hanikumar Lad, Smit Parekh, Saurabh Kabrawala, “LSTM Based Stock Prediction,” International Journal of Creative Research Thoughts(IJCRT), vol. 9, pp. 5118-5122, Feb. 2021. [2] Ya Gao, Rong Wang, and Enmin Zou, “Stock Prediction Based on Optimized LSTM and GRU Models,” Hindawi, vol. 2021, pp. 1-8, Sept. 2021. [3] Adil Moghar and Mhamed Hamiche, “Stock Market Prediction Using LSTM Recurrent Neural Network,” Sciencedirect, vol. 170, pp.1168-1173, Apr. 2020. [4] Mallikarjuna Shastry P. M. and Pramod B S, “Stock Price Prediction Using LSTM,” ResearchGate, vol. 83, pp. 5246-5251, May. 2020. [5] Ishita Parmar, Navanshu Agarwal, Sheirsh Saxena, Ridam Arora, Shikhin Gupta, Himanshu Dhiman, Lokesh Chouhan, “Stock Market Prediction Using Machine Learning,” in First Inter- national Conference on Secure Cyber Computing and Communications, 2018, doi: 10.1109/ICSCCC.2018.8703332 [6] Uttamraj K R, Raghav Nandakumar, Vishal R, Y.V Lokeswari, “Stock Price Prediction Using Long Short Term Memory,” International Research Journal of Engineering and Technology (IRJET), vol. 05, pp. 3342-3348, Mar. 2018.

Copyright

Copyright © 2022 Geeta Kolte, Varadraj Kini, Harikrishnan Nair, Prof. Suresh Babu K. S.. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.