Financial Crisis Prediction and Stock Market Analysis: A Systematic Review

Abstract

It is common known that attempting to foresee the movements of the stock market is a huge task that calls for a great lot of focus and concentration. Because of this, precisely predicting stock prices might potentially result in attractive rewards if the necessary decisions are made. This is the reason why this is the case. The non-stationary, noisy, and chaotic data that is accessible makes it difficult to generate correct forecasts about the stock market. This is because accurate predictions are difficult to produce. Because of this, it is difficult for investors to generate accurate forecasts in order to invest their money and earn a return from their investments. Various strategies are developed within the framework of the existing methodologies in order to forecast stock market movements. An overview of research articles arguing for different approaches is the goal of this study. In addition to being great academic publications, these research articles provide calculation methodologies, machine learning algorithms, performance factors, and other information. To address this recognized need, this study attempts to provide a comprehensive and current evaluation of the stock market forecasting approaches now in use. This research will include the categorization, analysis, and comparison of various approaches. In the end, the process of training and testing the model is evaluated using machine learning models. Based on the data, it is abundantly evident that logistic regression is better than any other approach that is presently being used.

Introduction

I. INTRODUCTION

Making predictions about the stock market has turned out to be a tough problem to solve. Fama's (1995) efficient-market theory says that in efficient information markets, it is hard to tell which way stock prices will move and how many times they will change because stock prices move randomly. The weak form says that past price changes can't be used to guess what will happen in the future. The semi-strong form says that neither past price changes nor any public information can be used to guess what will happen in the market. And the strong form says that none of the information, public or private, can be used to guess what will happen in the market as a whole. He came up with these three types of performance. One of the most interesting new ideas to come about in recent years is the creation of financial markets. Business, employment, and technology are just a few of the numerous industries that are significantly affected by these financial markets. When it came to the stock market, there were two primary options for investors in the past. These ways have helped them spend their money in ways that get them better returns with less risk. According to professional experts and buyers, the progress in [2] stock market prediction has become a lot more important. Because the market is so noisy, it is very hard to figure out how the stock market [3] is moving and what prices are doing. Because stock prices are so complicated, many things can change, like when quarterly gains are reported and when market news comes out. We figure out stock market indicators [4] by looking at how much money each company is worth on the market. Because of this, it is very hard to make correct predictions about the stock market [5] because the market is always changing.

Researchers and market experts [6, 7] have been very interested in coming up with and trying new ways to look at how the stock market moves. So, many math techniques, like autoregressive integrated moving average and clustering, are used to guess what will happen in the stock market. You should keep this in mind because this model gives you past facts and ideas that back regular postulates. Many studies on stock market predictions employed various forms of machine learning, such as support vector machines (SVMs), neural networks (NNs), and genetic adversarial networks (GANs) [8, 9]. The data analyst made a solid guess as to the stock market index using artificial neural networks (ANN) and support vector regression (SVR) [10]. Every machine learning algorithm has its own method for picking up on patterns. A number of writers [11] have proposed a combination approach to calculate the RSI. Combining a basic set with an ANN is what this approach is all about. One oscillator that compares the strength of prices currently to their strength in the past is the Relative Strength Index (RSI).  The results show that the mixed approach can handle big changes in stock market prices with a lot more accuracy than other machine learning methods.

As a rule, some specialists [12] predict stock market movements by analyzing the Korea stock price index using genetic algorithms (GAs) and long short-term memories (LSTMs). This is accomplished by making use of readily available, free information on money. In an effort to foretell future stock price movements, a variety of networks [13, 14] use the feed-forward network. Our primary objective in conducting this research was to survey the various approaches to stock market forecasting that have been developed so far in the hopes of providing some insight into the market's likely future actions. Consequently, this study examines the datasets used by various statistical and machine learning algorithms for stock market prediction. There should be a lot of different methods, performance models, datasets, and stock market forecast techniques used in the study. A lot of weight has been put on the progress that has been made in predicting stock prices by professional experts and buyers. It's hard to predict what will happen in the stock market and evaluate trends because the market is naturally noisy and trends change a lot. Stock prices are very complicated, and many things that change over time are affected by them. Quarterly earnings reports, market news, and a plethora of other items fall within this category. Expert traders utilize a variety of stock-based basic signals in their daily work.  You can use these signs to figure out how much money stocks are making, but it's still hard to tell what the market will do each day or each week [1]. In the same way that the business world is always changing, trying to correctly guess how stocks will move is both interesting and hard. In order to figure out how stocks will move, many economic and non-economic factors are considered. There are both economic and non-economic forces in these areas. Therefore, being able to accurately forecast the stock market is seen as a major obstacle to boosting production [2].

Using traditional methods, we can guess how much money the stock market will make by looking at how much money other stocks made, along with other financial and social factors. When buyers were asked to guess how much money the stock market would make, they were told to look into the things that make predictions possible. Stock trend predictions are hard to make because a lot of different things can change them. Some of these factors are traders' hopes, the market's financial state, the events that happen in the government, and some traits that are linked to the market's trends. Aside from that, stock price lists are notoriously nonlinear, complex, noisy, nonparametric, and prone to frequent changes [3]. It might be hard to get a good idea of a financial time series because of a number of complicated factors [4]. Some of these traits are instability, abnormalities, noise, and patterns that are always changing.

II. NEED FOR RESEARCH

The fact that buyers put some of their money into the stock market shows that they want to make money from it. The creation of new apps that could help people accurately predict the stock market may be one reason why investors are becoming more interested in it. Predicting the changes in the stock market [15] accurately depends on having access to information ahead of time. When tools are used for stock market forecasts [16,17], they can keep an eye on and control the market. Decisions are more likely to be well-informed when this is considered. There is a lot of data [18] on industrial stocks that the stock market needs to monitor. All of the financial markets are covered by this data. The requirements of the company's investors, who consider both purchasing and sales, dictate changes to these [19–21].  The market situation is affected by many things, such as predictions about future sales, news releases about income, and changes in management, to name a few. This being the case, investors would benefit from a precise stock market forecast [22] while making purchases. By using machine learning techniques, the trader might be able to make more money, but they will also be taking on more danger.

Starting with their price index, real-time data originates from a variety of sources, including websites and historical records like NASDAQ [23]. An investor may gauge their performance by comparing the current price flatten with prices that have occurred in the past using the price index, which is a component of the stock market. Data undergoes pre-processing [24] once data collection is complete to remove irrelevant information and noise. After then, individuals may be able to use pre-processed data to make educated stock market predictions. Feature selection algorithms are used to extract certain attributes [25] from massive datasets. The dataset is divided into two parts: current facts and future information. This is done using data scientist tools or simple applications. You will be able to use this information to your advantage while making stock market selections. Notifications are given to customers on the price index whenever a significant decision is made [26,27]. Investors may see the current profit or loss for the price index in this notification [28].

The notification is quite beneficial for investors because of this. The user stands to gain significantly from the shares if the state provided by the app [29] is "profit." Conversely, growth is paying closer attention to make better judgments if the costs index is low [30].

III. RESEARCH GAPS AND ISSUES

Although there are several methods for forecasting the stock market, good stock market prediction requires addressing certain limitations. Next, we'll go over the many methods of stock market prediction and talk about the problems and areas where research is lacking. When attempting to forecast the stock market, the NN encounters many obstacles, such as the following: Due to the big neurons included in the hidden layer and the appropriate weight adaptation, the built artificial neural network was not praised as an effective stock market forecasting approach [33].

The rationale for this is because neural models need very little computing power. Prediction accuracy suffered as a result of the slower testing and training completion rate of the Neural Networks (NN) constructed in [35]. Overfitting, being trapped in local minima, and the black box approach are some of the downsides that neural networks might help with. Due to untuned network parameters and the influence of misclassification of comparable patterns, the outcomes produced by the NN-based stock market prediction system developed in [51] were very misleading.

The primary research concern with the CNN-based stock prediction approach is that the convolutional neural network (CNN) trained inside the deep learning framework was inadequate for the really enlarged applications. In comparison to the other cutting-edge prediction method, CNN's recognition accuracy was much worse when used to stock prediction [45]. The intended stock investment decision assistance system [46] lacked the real-world data and methods that would have made it a good stock expert system. In addition to being poorly described on how the response is generated, artificial neural networks (ANN) need a lengthy training procedure to produce an optimal model [61].

It took longer to run the fuzzy time series model [5], and as the runtime increased, the forecast accuracy decreased. An further factor contributing to the computer process's complexity is the intricate development of splitting algorithms. Consequently, inaccurate predictions are provided by the fuzzy time series model. Stock market forecasts made use of a feature selection approach based on support vector machines (SVM). The system's accuracy was significantly diminished since the feature selection approach could not find the necessary number of optimum features. The value of the feature's correlation is crucial to the built NN-based prediction system and the support vector machine (SVM) [63].

The majority of the publications focused on predicting market trends using ANN and classifying stock data with fuzzy-based approaches. More than any other program, MATLAB is used by stock prediction algorithms to forecast market moves. When trying to forecast the stock market, the three most used datasets are the S&P 500, the TAIEX, and the BSE. Datasets from other sources are used for stock market forecasting and trend analysis; these include NASDAQ, the National Bank of Greece, the Nigeria Stock Exchange, Yahoo Finance, Citegroup and Motors, the Tehran Stock Exchange Index, and the Chinese Stock Market. The four most popular performance metrics are accuracy, precision, root-mean-square error (RMSE), and mean-absolute-error (MAE). Specificity, sensitivity, mean percentage error, median relative absolute error, average relative variance, relative difference in percentage, root mean squared, coefficient of multiple determination, information gain, F0-measure, and Mathew's correlation coefficient are additional metrics that a small number of studies further employ for analysis. One of the most common ways to measure the success of stock market tactics is by looking at their MAPE.

IV. RESULTS AND DISCUSSION

Examining the stock market involves looking at a few different statistical methodologies. Several statistical methods were used in the research that aimed to provide basic descriptive information for stock market analysis. Research that uses ARIMA, regression, and clustering to forecast stock market movements was among the studies selected. Detailed descriptions of each approach are provided below:

1. ARIMA: Time series data may also benefit from one more statistical tool for gaining insight into a dataset: the Arima [11].
2. Clustering: The clustering technique [13,14] is used to group things that have similar properties. There is a separation between the stocks with a high degree of correlation and those with a less degree of connection. This process is iterated until all stocks are included in all categories.       

The majority of the chosen topics use ML or in-depth learning techniques for stock market forecasting. To improve the accuracy of their stock market predictions, a few of research have chosen to use the hybrid technique. Almost every method for predicting the stock market is covered in further detail here. The following are examples of widely used prediction algorithms:

a. Support vector machine (SVM): If you're trying to predict a time series, SVM is a great option. Additional uses for support vector machines include classification and regression. Data points may be plotted in n-dimensional space using support vector machines (SVMs). Multiple coordinate planes display the stock market metrics, which are classified. When compared to other tools used by the financial industry, support vector machine (SVM) is clearly superior.

b. Neural Networks (NN): These algorithms [17] can learn from fresh datasets in a manner similar to the human brain. Simply said, it is a set of algorithms. The authors Chung and Shin [18] improved stock market forecasting using an outer break neural network (NN) approach. The information is culled directly from the livestock market as it happens. The creation of the deep LSTM dependent neural network was necessary for the use of the embedded layer. The neural LSTM network encoder allowed for the prediction of stock trends.

c. ANN: More than any other statistical tool, ANN can capture the structural link [19] of stock, particularly output and the variables that govern it. This makes ANN an excellent choice. Many researchers first used the ANN model before diving into data preparation. A plethora of performance factors are used in stock market prediction, as elucidated in RQ5.

d. Convolutional Neural Networks (CNNs): These networks feed data into the neural network [20]. In CNN, as compared to traditional neural network methods, the number of hidden layers is much more. When developing stock market predictions, convolutional neural networks (CNNs) are used.

e. Recurrent Neural Networks (RNNs): RNNs are a kind of artificial neural network (ANN) connected by a graph whose orientation follows the temporal chain of the nodes' connections with one another [20]. Thus, it enables the demonstration of complex dynamic behavior.

f. Support vector regression (SVR): The SVR [5] takes a few cues from the SVM, but other than that, they're almost twins. When predicting future stock market prices, the SVR is used. On the other hand, when predicting future stock market prices based on their time-series, the SVM is utilized.

g. Generative adversarial network (GAN): Newly taught utilizing a novel framework known as the GAN [23] are two game types. The antagonist cycle portrays the generator as a scam artist since it creates data that is very similar to genuine data in appearance and behavior, with the racists serving as judges who determine whether the data is processed or not.

Using a number of datasets, the study attempts to forecast the stock market. Some studies have shown that the general population has access to a number of databases. Using publicly available data sets, almost every one of the selected subjects has produced stock market forecasts. Classification and prediction are two of the many uses for these datasets. Several performance indicators are used to assess machine learning's potential to enhance its forecasting, stock market, and exchange anticipating abilities. An algorithm's performance may be evaluated using a variety of metrics, each tailored to a certain approach and dataset.

Below is a comprehensive overview of the many performance criteria that were used by the chosen studies to assess their success:

a. Accuracy: Accuracy is one of the metrics used to evaluate the categorization capabilities of the model [21]. In order for our model to work, there must be an assumption of informal correctness.

b. Root-mean-squared error (RMSE): This metric is used at the level where the discrepancy between the model's anticipated values and the retained values is calculated [22]. There is a tight relationship between the RMSE and the training and assessment database.

c. MAE: To calculate regression values, one uses the MAE [24,25]. In this case, we may calculate the error prediction by first tallying up all the variables' predicted and actual values that differ, and then dividing by the total number of data points that exceed all of them. The MAE is defined as the procedure for determining the difference between two continuous variables.

d. Mean squared error: It is common practice to utilize the square average error, or MSE, as a loss function when calculating the minimal square regression [26,27]. Similarly, this is the same as adding up all the differences between the predicted and actual values of the variable, then dividing that amount by the number of observations.

e. MAPE: KPI most often uses MAPE, or mean absolute percentage error, for calculating stock market estimates [28].

V. OBJECTIVES OF THE RESEARCH

Combining statistical approaches with machine learning models and knowledge of the financial industry is necessary when using data analysis to examine the stock market and forecast financial catastrophes. Here is a rundown of what this research project hopes to achieve:

We will undertake a comprehensive evaluation of the stock market taking into consideration the effects of the financial crisis scenario, and our goal is to develop a system that can reliably predict when financial crises will occur. Performance metrics are assessed using a combination of a hybrid feature selection mechanism and statistical measures derived from the relative strength index. Various techniques are also compared.

In order to investigate the accuracy of financial crisis prediction, many machine learning models were tested. Data was sourced from the Kaggle collection in particular.

Conclusion

Among the several methods discussed in this study of stock market splits are mathematical techniques and machine learning algorithms. This study endeavours to categorize the present techniques associated with updated methodology, the use of different datasets, performance matrices, and approaches by using thirty investigative pieces published in the most influential journals. The stock market prediction methods are classified using a multitude of machine learning techniques. A small number of the chosen studies use stock market hybrid methodologies to improve the accuracy of their predictions. The stock market is often forecasted using approaches that employ Artificial Neural Networks (ANN) and Neural Networks (NN). The complete stock market may be monitored and supervised using these technologies, which are designed specifically for that purpose. A major obstacle for stock market forecasts is that the majority of existing methods cannot be discovered by analyzing stock market data from the past. Thus, other variables, such as consumer attitude and policy choices made by the government, may affect stock markets. The current stock market system might need some work to make it more precise and reliable, but maybe that may be addressed down the road.

References

[1] B.o. Qian, K. Rasheed, Stock market prediction with multiple classifiers, Appl. Intell. 26 (1) (2007) 25–33. [2] N.A.A. Hussain, S.S.A. Ali, M.N.M. Saad, N. Nordin, 2016, December. Underactuated nonlinear adaptive control approach using U-model for multivariable underwater glider control parameters. In 2016 IEEE International Conference on Underwater System Technology: Theory and Applications (USYS) (pp. 19-25). IEEE. [3] Deepak Kumar, Pradeepta Kumar Sarangi, Rajit Verma, A systematic review of stock market prediction using machine learning and statistical techniques, Materials Today: Proceedings, Volume 49, Part 8, 2022, Pages 3187-3191, ISSN 2214-7853. [4] D. Shah, H. Isah, F. Zulkernine, Stock market analysis: A review and taxonomy of prediction techniques, Int. J. Financial Stud. 7 (2) (2019) 26. [5] Pathak, Ashish, Nisha P. Shetty., 2019. Indian stock market prediction using ML and sentiment analysis. In Computational Intelligence in Data Mining, pp. 595- 603. Springer, Singapore, pp. 595-603. [6] J. Patel, S. Shah, P. Thakkar, K. Kotecha, Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques, Expert Syst. Appl. (2015) 259–268. [7] Hernández-Álvarez, Myriam, Edgar A. Torres Hernández, Sang Guun Yoo., 2019. Stock Market Data Prediction Using ML Techniques.‘‘ In International Conference on Information Technology & Systems, Springer, Cham, pp. 539-547. [8] S. Banik, A.K. Khan, M. Anwer, 2012, December. Dhaka stock market timing decisions by hybrid machine learning technique. In 2012 15th International Conference on Computer and Information Technology (ICCIT) (pp. 384-389). IEEE. [9] Ahmad Waqas, Analyzing different ML techniques for stock market prediction, Int. J. Comput. Sci. Inform. Sec. 12 (2014) 12–17. [10] H. Yang, L. Chan, I. King. Support vector machine regression for volatile stock market prediction. In International Conference on Intelligent Data Engineering and Automated Learning 2002 Aug 12 (pp. 391-396). Springer, Berlin, Heidelberg. [11] H.R. Patel, S.M. Parikh, D.N. Darji. Prediction model for the stock market using news based different Classification, Regression, and Statistical Techniques: (PMSMN). In2016 International Conference on ICT in Business Industry & Government (ICTBIG) 2016 Nov 18 (pp. 1-5). IEEE. [12] K.J. Kim, W.B. Lee, Stock market prediction using artificial NN with optimal feature transformation, Neural Comput. Appl. 13 (3) (2004) 255–260. [13] P.D. Yoo, M.H. Kim, T. Jan. Financial forecasting: advanced machine learning techniques in stock market analysis. In2005 Pakistan Section Multitopic Conference 2005 Dec 24 (pp. 1-7). IEEE. [14] Andrés Alegría, Rodrigo Alfaro, Felipe Córdova, The effect of warnings published in a financial stability report on loan-to-value ratios, Latin American Journal of Central Banking, Volume 2, Issue 4, 2021, 100041, ISSN 2666-1438, [15] S.K. Chandar, M. Sumathi, S.N. Sivanandam, Prediction of the stock market price using a hybrid of wavelet transform and artificial neural network, Indian J. Sci. Technol. 9 (8) (2016) 1–5. [16] Jonathan L. Ticknor, A bayesian regularized artificial neural network for stock market forecasting, Expert Syst. Appl. 40 (14) (2013) 5501–5506. [17] A. Sharma, D. Bhuriya, U. Singh, 2017, April. Survey of stock market prediction using a machine learning approach. In 2017 International conference of Electronics, Communication and Aerospace Technology (ICECA) (Vol. 2, pp. 506-509). IEEE [18] Rao, S. Hule, H. Shaikh, E. Nirwan, Daflapurkar PM. Survey: stock market prediction using statistical computational methodologies and artificial neural networks. Int. Res. J. Eng. Technol. (08). 2015. [19] D. Enke, M. Grauer, N. Mehdiyev, Stock market prediction with multiple regression, fuzzy type-2 clustering, and neural networks, Procedia Comput. Sci. 1 (6) (2011) 201–206. [20] H. Chung, K.S. Shin, Genetic algorithm-optimized long short-term memory network for stock market prediction, Sustainability 10 (10) (2018) 3765. [21] X. Li, H. Xie, R. Wang, Y. Cai, J. Cao, F. Wang, X. Deng, Empirical analysis: stock market prediction via extreme learning machine, Neural Comput. Appl. 27 (1) (2016) 67–78. [22] E. Chong, C. Han, F.C. Park, Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies, Expert Syst. Appl. 83 (2017) 187–205. [23] X. Pang, Y. Zhou, P. Wang, W. Lin, V. Chang, An innovative neural network approach for stock market prediction, J. Supercomput. 76 (3) (2020) 2098– 2118. [24] M.M. Pai, A. Nayak, R.M. Pai, Prediction models for the Indian stock market, Procedia Comput. Sci. 89 (2016) 441–449. [25] K. Zhang, G. Zhong, J. Dong, S. Wang, Y. Wang, Stock market prediction based on the generative adversarial network, Procedia Comput. Sci. 147 (2019) 400– 406. [26] S. Selvin, R. Vinayakumar, E.A. Gopalakrishnan, V.K. Menon, K.P. Soman. Stock price prediction using LSTM, RNN, and CNN-sliding window model. In2017 international conference on advances in computing, communications, and informatics (icacci) 2017 Sep 13 (pp. 1643-1647). IEEE. [27] Chryssanthi Filippopoulou, Emilios Galariotis, Spyros Spyrou, An early warning system for predicting systemic banking crises in the Eurozone: A logit regression approach, Journal of Economic Behavior & Organization, Volume 172, 2020, Pages 344-363, ISSN 0167-2681 [28] Ritika Singh, Shashi Srivastava, Stock prediction using deep learning, Multimedia Tools Appl. 76 (18) (2017) 18569–18584. [29] M.R. Vargas, B.S. De Lima, A.G. Evsukoff. (2017, June). Deep learning for stock market prediction from financial news articles. In 2017 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA) (pp. 60-65). IEEE. [30] K.A. Althelaya, E.S.M. El-Alfy, S. Mohammed. (2018, April). Evaluation of bidirectional lstm for short-and long-term stock market prediction. In 2018 9th international conference on information and communication systems (ICICS) (pp. 151-156). IEEE. [31] R. Ray, P. Khandelwal, B. Baranidharan, 2018, December. A survey on stock market prediction using artificial intelligence techniques. In 2018 International Conference on Smart Systems and Inventive Technology (ICSSIT) (pp. 594-598). IEEE. [32] J. Li, H. Bu, J. Wu. (2017, June). Sentiment-aware stock market prediction: A deep learning method. In 2017 international conference on service systems and service management (pp. 1-6). IEEE. [33] E. Guresen, G. Kayakutlu, T.U. Daim, Using artificial neural network models in stock market index prediction, Expert Syst. Appl. 38 (8) (2011) 10389–10397.

Copyright

Copyright © 2024 Ramsheel M, Dr. M. Shanmugapriya. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.