A Review of Stock Prediction Using Various Machine Learning Techniques

Abstract

Stock market prediction is a complex task due huge volume of data that keeps on changing, and various uncertainties and interrelated local and global economic factors. The key to successful stock price forecasting is achieving best results with minimum amount of required input data. Thanks to the development of technology, in recent years more and more research in the field of the prediction is being done and it becomes easier for us to make stock price prediction by using various ways that include machine learning, deep learning and many other such technologies. Many exceptional innovations in the field of Machine Learning, have been enforced to develop a short-term prediction model. This paper surveys a number of resources from research papers that have studied on the topic.

Introduction

I. INTRODUCTION

The origin of stock market and stock trading dates back to around 12th century in Western Europe, particularly in Italy and France. But, the modern equivalent of the stock market started in Amsterdam, Netherlands around the 17th century. The Dutch East Indian Company was the first company that offered its shares to be traded in the stock market.

The share market is a prediction of future growth expectations of companies and the economy. Many factors attribute to fluctuation in stock price, which includes macroeconomic factors, market anticipation, and confidence in the company’s ability to manage and operate effectively and optimally. The Indian stock exchanges holds a place of global prominence. The Bombay Stock Exchange (BSE) is one of the oldest exchanges around the world, while the National Stock Exchange (NSE) is among the best in the field of technology. Investment in stock market is regarded as high gains with high risks, therefore attracts a large number of investors both professional and non-professional individuals. The advance technology of this age allows an individual to access a larger amount of information in a more effective manner. Even though the number of investors is very huge, the real amount of them who make profit from the stock market is considerably small. If you want to get the profit from the stock market, the analysis of the data is absolutely necessary. Continuous development in the field Artificial Intelligence leads to the wide use of machine learning and deep learning techniques in many research as well as practical scenarios. Such applications include natural language processing, image recognition, predictions in the field of medicine, etc. The neural networks used in these applications have also been developed thanks to the rise of deep learning technologies. Several studies have been the subject of using machine learning algorithms in the financial field, predicting prices, managing the portfolio of assets, as well as investments, and other operations. 

The rest of the paper discusses the research of stock prediction based on fundamental as well as technical analysis. It also expands upon the significant work done in the field of stock prediction by various researchers

II. STOCK MARKETING OVERVIEW

This section discusses the basics of the stock market which include various financial terms like stock market index, stock exchange, etc.

A. Basics of Stock Market

The stock market in general refers to the assortment of exchanges and other avenues, where buying and selling of shares of companies that are publicly held, take place. The place where these types of exchanges are held is called a stock exchange. In India the process of stock exchange is enforced and regulated by Securities and Exchange Board of India (SEBI). Some of the leading exchanges around the world are -- NYSE, NASDAQ in the United States of America, London Stock Exchange in the United Kingdom, etc. Most of the trading in the Indian stock market takes place on its two major stock exchanges of India: the Bombay Stock Exchange (BSE) and the National Stock Exchange (NSE). Once the shares of a publicly held company are listed in a stock exchange, the investors will buy or sell the shares of the said company from the exchange like listed above. The stock exchange work as an indirect market for trading of shares between investors. There are mainly two types of stocks:-

1. Common Stock: Common stock represents the ownership in a corporation.  This form of equity ownership typically yields higher rates of returns in long term. However, in case of liquidation of such corporation, common shareholders have rights to the corporation's assets only after bondholders, preferred shareholders, and other debtholders are fully paid.
2. Preferred Stock: Preference stock are shares of a corporation’s shares with dividends that are paid out to the shareholders before common stock dividends are issued. If the corporation enters bankruptcy or a case of liquidation arises, preferred stockholders are supposed to be paid from the corporation assets before common stockholders.

B.  Participants of Stock Market

There are various different participants in a stock exchange other than an individual investor.

1. Stockbrokers: A stockbroker is a financial professional who enact orders in the market on behalf of his/her clients. A stockbroker may also be known as a Registered Representative (RR) or an investment advisor to an individual.
2. Investment Banker: An investment banker is an individual who is a part of some financial institution and is primarily concerned with raising capital for corporations, governments, or other large entities which are considered the financial institution’s client.
3. Portfolio Manager: A portfolio manager is a person or group of responsible for investing a mutual, exchange traded or closed-end fund's assets, enforcing its investment plan, and managing day-to-day trading. A portfolio manager is one of the most important factors to consider for an individual when looking at fund investing. Historical performance indicates that only a minority of portfolio managers consistently beat the market to gain some significant profit.
4. Custodian: A custodian or custodian bank is a financial institution that holds the customers' securities for safekeeping to prevent them from being stolen or lost. The custodian can hold stocks or other assets in electronic or physical form.

C. Stock Market Index

A stock market index is used to evaluate the price of the stock market. It helps investors compare the current price with its historical performance. In case of BSE, its index is calculated using selected 30 best stocks; whereas, for NSE the calculation of index is done by selecting 50 best stocks. There are different types of indexes that exist according to the weighting method:-

1. Price Weighted Index: A price-weighted index is a stock index in which each company included in the index account for a fraction of the total index proportional to that company's share market price. In other words, adding the price of each stock in the index and dividing by the total number of companies in the index determines the value of price weighted index.
2. Equal Weighted Index: Equal weight is a method that gives equal importance to each stock in a portfolio, index, or an index fund. Therefore, stocks of the smallest companies are given equal statistical importance, or weightage, as the largest companies when evaluating the overall group's performance.
3. Capitalization Weighted Index: A capitalization-weighted index is a type of stock market index where individual components of the index are included in amounts that correspond to their total market capitalization (shortened as "market cap"). A company's total market capitalization is computed by multiplying its shares, owned by individual shareholders, institutional block holdings, and company insider holdings (also called outstanding share), by the current price of a single share. Thereby, market capitalization reflects the total market value of a company's outstanding shares.

III. TECHNIQUES USED FOR STOCK PREDICTION

A. Fundamental Analysis Based Stock Prediction

It is a method of measuring the company’s value by verifying economic and financial factors. It is a study of a company's infrastructure, company’s background, product portfolio, financial results of the company, Whether the company is having any debt, competitors, and the prospects of the company.

1. Assumptions of Fundamental Analysis

a. Stock price (current and future) depends on its intrinsic value and can anticipate return on investment.

b. Investors are 90% logical, examining their investments in detail.

2. Advantages of Fundamental Analysis

a. The advantages of fundamental analysis are its organised approach and it being able to predict changes even before they show up on the charts.

b. Fundamental analysis is a better method for long-term profit and growth with low risk.

3. Disadvantages of Fundamental Analysis

a. It becomes more difficult to formalize all this information for the purpose of automation, and interpretation of this information may be subjective to each individual.

b. It is hard to time the market using fundamental analysis.

B. Important Ratios for Fundamental Analysis

The stock valuation of a company depends mainly on some ratios. The share that does not reflect all these parameters as healthy are not considered to be a good buy.

1. The Price-to-Book Ratio (P/B): Book value is that value of a company, which the owner is likely to gather if they decide to liquidate (sell off in dire straits) the company.
2. Price-to-Earnings Ratio (P/E): P/E Ratio compares the market price with the EPS. Where, EPS (Earning per share) is the company’s total earnings by the numbers of remaining shares of the stock. Higher the value of P/E ratio, more people are likely to pay more for that share expecting better growth in coming future.
3. The PEG Ratio: The PEG ratio is the P/E ratio of a company by the year-by-year rate of growth of the company’s earnings. The lesser the value of the PEG ratio, the better the profit you will be getting for the stock's future estimated income.
4. Dividend Yield: The dividend yield shows how much pay-out you're getting for your money. It is the stock's annual dividend pay-out by the stock's price.
5. Debt to Equity Ratio: The value should not be more than 1, as the value less than 1 indicates that the company has very low amount of debt. This is highly crucial during a down trend in market as company has to pay large amount of interest beside low profitability. So, it is a good sign, if company has less debt-to-Equity Ratio.
6. Returns on Equity (ROE): It is used as a common indication of efficiency of a company, i.e., how much profit is made given the resources given by its stockholders. Investors usually look for companies with high ROE value.

No Element Stands Alone. P/B, P/E, PEG, Dividend Yield, Debt to Equity Ratio is too narrowly focused to be considered as a single measure for a stock. By joining these methods of valuation, you can get a better information on a stock's worth.

C. Technical Analysis Based Stock Prediction

It is a method of evaluating stocks by analysing statistical data generated by the companies’ market activity, past prices, and volume. This type of analysis looks for peaks, bottoms, trends, patterns, and other factors affecting a stock's price movement. Future of stock prices often depend on their past price of the stock and other correlated variables. It looks for patterns and indicators on stock charts that will determine a stocks future performance.

Despite its widespread use, technical analysis is criticized because it is highly subjective, i.e., different individuals can interpret charts in different manners.

1. Assumptions in Technical Analysis

Market moves in ways dictated by the ever-changing attitudes of investors in response to different scenarios. History repeats itself, i.e., under similar kinds of inputs the stock behaves in similar manner. Prices tend to go according to the trend rather than against it.

2. Advantages of Technical Analysis

a. It is used by approximately 90% of the major stock traders.

b. It is also used to analyse the stock for shorter period.

3. Disadvantages of Technical Analysis

1. Despite its widespread use, technical analysis is criticized because different individuals can interpret charts in different manners.
2. Different individuals can interpret charts in different manners.

IV.  ANALYSIS OF VARIOUS MACHINE LEARNING TECHNIQUES USED FOR STOCK PREDICTION

Jianguo Zheng et. al. (2021) applied the bat algorithm in the swarm optimization algorithm to optimize the free parameters of the Gaussian radial basis kernel function support vector regression model. The proposed BA-SVR model had better forecasting performance in the short, medium and long term. After comparing the BA-SVR model with the polynomial kernel function, support vector regression and the Sigmoid kernel function; without optimization of free parameters, the performance of the BA-SVR model in forecasting is still robust in different periods. The BA-SVR model has the best prediction performance when used in short-term scenario, followed by lower accuracy in mid-term, and the lowest accuracy for the long-term scenario, while the other two models have no such characteristics [1].

Rajendra N. Paramanik et. al. (2020) also performed text-based sentiment analysis on the stock market. They have proposed a Generalized Autoregressive Conditional Heteroskedasticity (GARCH) model of conditional volatility by incorporating the market sentiments for the period of 14 Years. They have found out that the impact of negative sentiment is more than positive sentiment, they also found that there are noise trades in Indian Stock market [2].

Mehar Vijh et. al. (2020) implemented Artificial Neural network & Random Forest Techniques for predicting the closing prices of stock price on the next day. Financial data like Stock price high, low, and current closing price of 5 companies which belong to different sectors had been taken for predicting the closing price of the stock. Comparative analysis based on RMSE, MAPE and MBE values clearly indicated that Artificial Neural Network gives better prediction of stock prices as compared to Random Forest Techniques [3].

Pratik Patil et.al. (2020) implemented a new network using graph theory and CNN, which leveraged spatiotemporal relationship information between different stocks by modelling the stock market as a complex network. the model used both stock indicators and financial news as input. Here the overall graph-based model that was used had performed much better than the classic ARIMA model. This was due to modelling and mapping the special structure of graph to the temporal features, which drastically helped in increasing the performance of the model. In this study two different ways of graph generation was used: correlation-based and causation-based graphs. This project considers 30 nodes graph owing to the complexity of the model and its training period. The GCN model proposed in this project is susceptible to exploding gradient problem as nodes with higher degree will have larger value in their convoluted feature representation, whereas nodes with smaller degree will have smaller value in feature representation[4].

Adil Moghar et. al. (2020) proposed RNN based on LSTM built to forecast future values for both GOOGL and NKE assets, the result of our model has shown some promising result. The testing result conform that the model is capable of tracing the  evolution  of  opening  prices  for  both  assets[5].

Haider Maqsood et. al. (2019) used deep learning-based techniques to forecast stock prices for top companies from four countries that were selected from developed, underdeveloped and emerging economies. A CNN model was proposed that made use of the price and sentiment analysis as input and compared the proposed model with linear regression and Support vector machine (SVM). It was concluded that not all significant events have a serious impact on stock exchange prediction. However, more important local events could affect the performance of prediction algorithms [6].

H.S. Sim et. al. (2019) proposed a CNN network that uses 9 technical indicators (close price, Simple moving average (SMA), Exponential moving average (EMA), etc.) to verify the applicability of the CNN method in the stock market. He concluded that the use of technical indicators in stock price forecasting by CNN has no positive effect [7].

Chang Li et. al. (2019) proposed a multi-task RNN model with Markov Random Fields (MRF). The multi-multilayer perceptron (MMLP) was introduced to automatically extract various different complementary features from individual stock price, which means that there is no need for the technical indicators. Features learned by MMPL were passed to a binary MRF with a weighted lower linear envelope energy function to utilize intra-clique higher-order consistency between stocks [8].

M. Nikou et. al. (2019) proposed an LSTM model and compared it with the ANN model, Support Vector Regression (SVR) model, and RF model. The results showed that the LSTM model performed better in the prediction of the closing prices of iShares MSCI United Kingdom than the other models mentioned in paper [9].

Sheng Chen et. al. (2018) introduced a CNN model to make the stock prediction and used a function to process the 1D data in the convolutional layer. They have pre-processed stock data which was taken as input into the model to improve the result of the model. The result has been evaluated by different stock data, and finally indicated that their CNN model is robust and can also be used to make the predictions even if the source data is 1D sequential[10].

Aparna Nayak et. al. (2016) developed a predictive machine learning model where stock prices of the next day were predicted by considering the historical price movement and traded volume of the stock. This was achieved using different predictive algorithm like decision boosted tree, logistic regression & support vector machine. The conclusion was made that decision boosted tree performed far better than logistic regression and support vector machine techniques [11].

Aditya Bhardwaj et. al. (2015) performed sentiment analysis using Python programming language for the stock market data using the live feed from TimesofIndia.com. They used the Sensex and Nifty Index data from different time intervals and found that for a particular time interval Sensex and Nifty data remained constant [12].

J. Patel et. al. (2015) conducted experiments in using 10 technical indicators’ signals as inputs, then they use prediction models to predict whether the stock will go up or down in the coming 10 days, technical analysis indicators include SMA, EMA, Momentum, Stochastic SK, MACD, RSI, etc. Experiments have shown random forest scored the highest performance with 83.56% accuracy with their inputs [13].

Y. Dai et. al. (2013) used Logistic Regression, Gaussian Discriminant Analysis, Quadratic Discriminant Analysis, and SVM. With the short-term model predicting the next day stock price, it has very low accuracy, the Quadratic Discriminant Analysis is the best among all models, it scored a 58.2% accuracy. SVM in had higher accuracy of 79.3% for long term prediction [14].

Arun Upadhyay et. al. (2012) implemented Multinomial Logistic Regression (MLR) model to predict the outperforming stock. They used financial ratios as usable selection criteria for determining performance in the stock market based on the stock return comparing with market return. The model was tested on 30 large market capitalization companies’ ratios over the period of four years. The classification results showed high accuracy in forecasting as 56.8%. The author concludes that the developed model can enhance an investor's stock price forecasting ability [15].

K.K. Sureshkumar et. al. (2011) used prediction algorithms and functions to predict future share prices and compares their performance. The results from analysis showed that used isotonic regression function offers the ability to predict the stock prices more accurately than the other techniques [16].

Mahdi Pakdaman Naeini et. al. (2010) predicted stock prices using different neural network algorithms. It was found that to predict the stock price for next day none of the methods MLP neural network or Elman recurrent network provided better results than regression model [17].

Mohsen Mehrara et. al. (2010) used Multi-Layered Feed Forward (MLFF) neural network with back-propagation learning algorithm and Group Method of Data Handling (GMDH) neural network with Genetic algorithm (GA) learning to predict TSE (Taiwan Stock Exchange) database. The paper used moving average crossover inputs based on technical analysis rules and the results showed the exponential moving average has better result than simple moving [18].

S. Agrawal et. al. (2010) presented a new approach for indicating stock market decisions by minimizing the risk involved in making investments. The system used Adaptive Neuro-Fuzzy Inference System (ANFIS) for taking decisions based on technical indicators. Among the various technical indicators available, the system used weighted moving averages, divergence and RSI (Relative Strength Index) [19].

Manna Majumder et. al. (2010) presented a neural network based computational approach for predicting the direction of movement of the S&P CNX Nifty 50 Index. The model proposed here has used the pre-processed dataset of closing value of S&P CNX Nifty 50 Index from 1st January, 2000 to 31st December, 2009. The model gave highest prediction accuracy of 89.65% in terms of accuracy in predicting the direction of the closing value of the index and an average accuracy of 69.72% over a period of 4 years [20].

Conclusion

Stock Markets provide a good platform for investment and trading. People have opportunity to invest in multiple options of stock market like mutual funds, index funds, shares etc. In this paper, we have reviewed various stock prediction techniques has been presented in different published and available literature. On this basis it can be safely concluded that the existing techniques while sometimes showing promising results to some degrees are not suitable for prediction of stock market trends as well as price of different socks on their own. We will have to a combination of these techniques to achieve greater accuracy. Even then, there exist a gap between technologies and user requirement for a safe and accurate stock prediction system. We have to take into account various real-time political & economic factors which affect the stock market are also taken into consideration other than the technical indicators as input variables, better results may be obtained. Also, including market specific domain knowledge into the system might help in generating more accurate results. This goal cannot be achieved by simply implementing technical analysis by using machine learning techniques, we also have to incorporate human input based on fundamental analysis. Thus, it is concluded that machine learning algorithms and models; and other technical analysis techniques alone are not enough to accurately predict the future of stock prices in the stock market. But they can be used as a quality-of-life tool to help an individual make decision about his/her investment strategies.

References

[1] Zheng, J.; Wang, Y.; Li, S.; Chen, H. The Stock Index Prediction Based on SVR Model with Bat Optimization Algorithm. Algorithms 2021, 14, 299. https://doi.org/ 10.3390/a14100299 [2] Rajendra N. Paramanik, Vatsal Singhal, “Sentiment Analysis of Indian Stock Market Volatility,” Procedia Computer Science, Volume 176, 2020,Pages 330-338, ISSN 1877-0509, https://doi.org/10.1016/j.procs.2020.08.035 [3] Mehar Vijh, Deeksha Chandola, Vinay Anand Tikkiwal, Arun Kumar, Stock Closing Price Prediction using Machine Learning Techniques, International Conference on Computational Intelligence and Data Science (ICCIDS 2019), Procedia Computer Science, Volume 167, 2020, Pages 599-606, ISSN 1877-0509, https://doi.org/10.1016/j.procs.2020.03.326. [4] Patil, P.; Wu, C.-S.; Potika, K.; Orang, M. “Stock market prediction using ensemble of graph theory, machine learning and deep learning models,” In Proceedings of the 3rd International Conference on Software Engineering and Information Management, Sydney, Australia, 12–15 January 2020. [5] Adil Moghar, Mhamed Hamiche, Stock Market Prediction Using LSTM Recurrent Neural Network, Procedia Computer Science, Volume 170, 2020, Pages 1168-1173, ISSN 1877-0509, https://doi.org/10.1016/j.procs.2020.03.049 [6] Maqsood, H., et al., “A local and global event sentiment based efficient stock exchange forecasting using deep learning,” International Journal of Information Management, 2020. 50: p. 432-451. [7] Sim, H.S.; Kim, H.I.; Ahn, J.J. “Is deep learning for image recognition applicable to stock market prediction? Complexity” 2019, 1–10. [8] Li, C.; Song, D.; Tao, D. Multi-task recurrent neural networks and higher-order Markov random fields for stock price movement prediction: Multi-task RNN and higher-order MRFs for stock price classification. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019; Association for Computing Machinery: Anchorage, AK, USA, 2019; pp. 1141–1151. [9] Nikou, M.; Mansourfar, G.; Bagherzadeh, J. Stock price prediction using DEEP learning algorithm and its comparison with machine learning algorithms. Intell. Syst. Account. Financ. Manag. 2019, 26, 164–174. [10] Sheng Chen and Hongxiang He 2018 IOP Conf. Ser.: Mater. Sci. Eng. 435 012026. [11] Nayak, A., Pai, M. M. M., & Pai, R. M. “Prediction Models for Indian Stock Market,” Procedia Computer Science, 89, 441449. [12] Aditya Bhardwaj, Yogendra Narayan, Vanraj, Pawan, Maitreyee Dutta, “Sentiment Analysis for Indian Stock Market Prediction Using Sensex and Nifty,” Procedia Computer Science, Volume 70, 2015, Pages 85-91, ISSN 1877-0509. [13] J. Patel, S. Shah, P. Thakkar, and K. Kotecha, “Predicting stock and stock price index movement using Trend Deterministic Data Preparation and machine learning techniques,” Expert Systems with Applications: An International Journal, Vol. 42, Jan. 2015, pp. 259-268. [14] http://cs229.stanford.edu/proj2013/DaiZhang-MachineLearningInStockPriceTrendForecasting.pdf [15] Upadhyay, A., Bandyopadhyay, G., Dutta, A., “Forecasting Stock Performance in Indian Market using Multinomial Logistic Regression”, Journal of Business Studies Quarterly, Vol. 3, No. 3, pp. 16-39, 2012. [16] Sureshkumar, K. K., Elango, N. M., “An Efficient Approach to Forecast Indian Stock Market Price and their Performance Analysis”, International Journal of Computer Application, Vol. 34, No.5, 2011. [17] Mahdi Pakdaman Naeini, Hamidreza Taremian, Homa Baradaran Hashemi(2010),” Stock Market Value Prediction Using Neural Networks”, International Conference on Computer Information Systems and Industrial Management Applications (CISIM), pp: 132-136, IEEE. [18] Mohsen Mehrara, Ali Moeini, Mehdi Ahrari and Alireza Ghafari, “Using Technical Analysis with Neural Network for Prediction Stock Price Index in Tehran Stock Exchange,” Middle Eastern Finance and Economics, EuroJournals Publishing, Inc. 2010. [19] Agrawal, S., Jindal, M., Pillai, G. N., “Momentum Analysis based Stock Market Prediction using Adaptive Neuro-Fuzzy Inference System (ANFIS)”, Proceedings of the International MultiConference of Engineers and Computer Scientists, Hong Kong, Vol. 1, March 17 -19, 2010. [20] Majumder, M., Hussain, A., “Forecasting of Indian Stock Market Index Using Artificial Neural Network”.

Copyright

Copyright © 2022 Shekhar Paliwal, Shivang Sharma. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.