A Simple Procedure To Test Neural Network Performance For Stock Trading Systems

An Artificial Neural Network-based Stock Trading System Using Technical Analysis and Big Data Framework Sezer, et al. ArXiv 2017

Neural networks have come along way in the last five years in their performance as stock prediction engines. Although this paper is a few years old, it provides a nice introduction into how a simple neural network can be used to predict buy and sell signals with common technical indicators. The authors from TOBB University of Economics and Technology in Ankara, Turkey have provided a reproducible stock trading neural network evaluation procedure.

By way of introduction, the authors discuss how previous models rely mainly on static rules and require manual interactions. They note that these models are often prone to human emotions and biases. A neural network on the other hand largely addresses these issues while providing promising performance and returns.

The premise of their neural network is quite simple: use three common technical indicators to predict buy, hold, and sell labels using a Multi-Layer Perceptron (MLP) neural network. The model performance is then measured against a buy-and-hold strategy benchmark.

Technical Details and Indicators

MLPs are one of the simplest neural networks. They consists of three layers of customizable depth: an input layer, hidden layers, and an output layer. If you’re unfamiliar with MLPs, a quick explanation can be found here from the team at edpresso. The authors chose to use this type of neural network because of this simplicity.

For the model inputs, three technical indicators are used. First is the Relative Strength Index (RSI) which calculates the historical strength or weakness of a stock. The closer the RSI is to 1 the more overbought the stock is thought to be (sell signal), and the closer to 0 the more oversold (buy signal). RSI is calculated as below:

The second indicator is the Moving Average Convergence and Divergence (MACD), which highlights the trend of the stocks price. This is calculated using exponential moving averages (EMA):

And lastly, they use the Williams %R indicator. This functions similar to the RSI with a range from 0 to -100. A reading above -20 is overbought, and a reading below -80 is oversold. This is calculated by the below formula:

The Data Preparation

The authors split the data preparation into two phases:

In our study, the daily stock prices for Dow 30 stocks, which are obtained from finance.yahoo.com, are used as training and test datasets. In the first phase, open and close prices, daily high and low price values for each stock are obtained. Then, for each day, all daily close prices are labelled as “Hold”, “Buy”, or “Sell” by automatically analyzing the peak and valley points indicating highest and lowest points for a specified period. The peak points are marked as “Sell”, the valley points are marked as “Buy” and the remaining points are marked as “Hold”.

The second phase consists of calculating the RSI, MACD, and Williams %R values for each daily stock price.

The paper also makes an interesting point to address the class imbalance issue arising from the data having more ‘Hold’ labels than either ‘Buy’ or ‘Sell’ labels.

In addition, in the second phase, the data imbalance problem is also solved. Normally, the occurrence of “Hold” labels is much greater than the number of the “Sell” and “Buy” labels in the training data. This effects the learning stage such that the model only learns the majority classes better (Hold), ignoring the smaller classes causing misclassification and misprediction of data. There are different solutions in literature for this problem. We preferred the approach of resampling the minority classes. In other words, we created multiple copies of “Buy” and “Sell” labeled data and introduced those to the training dataset. Thus, the number of three class labels are approximately equal solving the class imbalance problem.

Finally, the data is fed into the small MLP neural network consisting of four layers with 4 nodes in the input layer, 5 nodes in the second layer, 4 nodes in the third layer, and 3 nodes in the output layer (‘Buy’, ‘Sell’, ‘Hold’ signals).

The paper summarizes this entire process below.

Evaluation and Results

In our model, a stock is bought, sold, or held according to its predicted label result. For instance, if the predicted label equals to ”1” (buy), (the corresponding output neuron is activated) the stock is bought using the capital that exists at that particular point. We start with a total capital of $10,000. All available capital is used during each transaction. If the predicted label equals to ”2” (sell), the stock is sold and we get back to an all cash position. If predicted label equals to ”0” (hold), system does not do anything.

A ‘Buy and Hold’ (BaH) strategy is used as a benchmark over the same time period for the Dow 30 stocks to simulate a common investor strategy. The resulting performance metrics from the MLP neural network compared to the benchmark are mixed at best, with some stocks fairing better and others fairing worse.

Table 3 shows the comparative performance of our model against the BaH for all Dow 30 stocks. Our proposed framework’s average annualized return is 10.3%, and the average annualized return of BaH strategy is 13.83%. Our proposed strategy’s annualized return performed better than BaH strategy’s annualized return in only 9 out of 29. The average success percentage of all transactions (buy then sell) in our system is 67.33% indicating that every 2 out of 3 transactions resulted in a profit.

Final Word

I thought this particular paper was interesting because it lays out a simple, easy to follow procedure for experimenting with neural networks on stock price data to predict buy and sell labels. The performance of the neural network was quite poor, but it provides a nice foundation for further research in trading algorithms with more complex neural networks and datasets.

An outline of their full algorithm is provided below for reproducibility.

Back to Top