ANN based method for improving gold price forecasting accuracy through modified gradient descent methods

Received Sep 4, 2019 Revised Nov 19, 2019 Accepted Dec 2, 2019 Forecast of prices of financial assets including gold is of considerable importance for planning the economy. For centuries, people have been holding gold for many important reasons such as smoothening inflation fluctuations, protection from an economic crisis, sound investment etc.. Forecasting of gold prices is therefore an ever important exercise undertaken both by individuals and groups. Various local, global, political, psychological and economic factors make such a forecast a complex problem. Data analysts have been increasingly applying Artificial Intelligence (AI) techniques to make such forecasts. In the present work an inter comparison of gold price forecasting in Indian market is first done by employing a few classical Artificial Neural Network (ANN) techniques, namely Gradient Descent Method (GDM), Resilient Backpropagation method (RP), Scaled Conjugate Gradient method (SCG), Levenberg-Marquardt method (LM), Bayesian Regularization method (BR), One Step Secant method (OSS) and BroydenFletcher-Goldfarb-Shanno Quasi Newton method (BFGS). Improvement in forecasting accuracy is achieved by proposing and developing a few modified GDM algorithms that incorporate different optimization functions by replacing the standard quadratic error function of classical GDM. Various optimization functions investigated in the present work are Mean median error function (MMD), Cauchy error function (CCY), Minkowski error function (MKW), Log cosh error function (LCH) and Negative logarithmic likelihood function (NLG). Modified algorithms incorporating these optimization functions are referred to here by GDM_MMD, GDM_CCY, GDM_KWK, GDM_LCH and GDM_NLG respectively. Gold price forecasting is then done by employing these algorithms and the results are analyzed. The results of our study suggest that the forecasting efficiency improves considerably on applying the modified methods proposed by us.


INTRODUCTION
Gold price plays an important role in any economic system. Historically, gold has been considered a 'frontier-less currency' that may be traded at any time and under virtually all circumstances. Gold has conventionally been the most effective way to collect cash during the world wide stock exchange crash in 1987, and again in 1997 and 1998 during the Asian crises. The prices of gold and other financial assets like equities and other commodities are often closely correlated. For instance, the correlation between gold and 47 equities may generally be negative since during the stock market booms funds are frequently transferred from gold to stocks and vice versa at times of crises. The correlation between gold and oil is generally positive and international disputes may increase the prices of oil and gold simultaneously. Accurate gold price forecasting is therefore very important to have a prudent investment policy which also helps mitigating financial risks. However, there is no mathematical formulation or model available to describe movement of gold price. In such a scenario, methods based on application of AI are inherently capable of making suitable forecasts. The advantages of AI methods lie in their ability to model complex and nonlinear relationships without any prior knowledge of the processes generating them [1]. AI methods are traditionally classified in three categories: Artificial Neural Network (ANN), Fuzzy logic (FL), Genetic algorithm (GA) and their hybrids which are combinations of these methods. ANNs have become preferred tools for forecasting of the financial markets in general. This is because of their ability to extract trends and patterns in a given data set. Trained ANNs can be considered as expert systems in their domain [2]. However, within various ANNs, choosing the best one for a given problem is a very important task [3]. The best choice is naturally the one giving most efficient forecast for the problem at hand. A large amount of work has been done by researchers demonstrating the forecasting power of AI/ANN based methods in various disciplines. A brief review of it is given here. A crisp review of the efficiency of AI techniques for diagnostic systems [4]. Various ANN methods like GDM, Conjugate Gradient, Quasi Newton and LM method to predict and classify the patients with heart disease [5]. Employed a backpropagation method with modified learning rate and momentum factors and applied the method to breast cancer and iris data sets [6]. Their study shows an impressive decrease in the training times of backpropagation which is otherwise inherently plagued by slow learning and local minima problems. A feed forward ANN model with the application of LM learning algorithm for short-term load forecasting of daily peak load and demonstrated that the proposed ANN model gives more accurate predictions with optimal number of neurons in the hidden layer [7]. ANN and Random Forest (RF) approaches for short-term photovoltaic (PV) output current forecasting (STPCF) for the next 24-hours. The results have shown that both the proposed techniques are able to perform forecasting of future hourly PV output current efficiently [8]. A merged Long Short-term Memory (LSTM) for forecasting ground visibility at the airport by combining time series of predictor variable with another moderating variable. Forecasting accuracy thus improved over the traditional LSTM method [9]. ANNs to diagnose melanoma skin cancer at an early stage with a high degree of accuracy [10]. Applied various neurocomputing methods for rainfall forecasting giving highly accurate results [11]. Proposed and tested a new hybrid model, i.e. Generalized Space-Time Autoregressive with Exogenous Variable and a Neural Network (GSTARX-NN) model for forecasting spacetime data with calendar variation effect. They found that the hybrid GSTARX-NN model gave more accurate forecasts than the traditional GSTARX models [12]. Compared a few variants of exponential and backpropagation ANN models to forecast rice production. The results showed that neural network method is preferable to the statistical method since it results in lower Mean Square Error (MSE) and Mean Absolute Percentage Error (MAPE) [13]. AI has also been used in the financial sector for various kinds of forecasts, analyses and decision making. Used of machine learning algorithms is explored to analyse effect of financial news on stock market prices. Support Vector Machine (SVM) and RF algorithms were used and it is concluded that the RF algorithm gives better accuracy in comparison with SVM algorithm [14]. Proposed a procedure to construct Triangular Fuzzy Number from single point data and then used an Autoregressive model to forecast currency exchange rates of Association of South East Asian Nation (ASEAN) countries [15]. A Fuzzy Neural System (FNS) to forecast the inflation rate obtaining better results in terms of RMSE than traditional methods [16]. A fuzzy sets method predicting the Russia Trading System (RTS) index with a confidence level of about 90% though having only an incomplete data set at hand [17]. A hybrid model, wavelet radial bases function neural networks (WRBFNN), for forecasting of non-stationary time series and found that, in terms of MAPE and MSE, the proposed model is superior to the traditional wavelet feed forward neural networks (WFFN) model [18].
In the present work forecast of gold prices in the Indian Multi Commodity Exchange market, MCX, has been taken up. A comparative analysis of gold with other investment options helps one to prioritize capital investments [19]. Traditional mathematical models such as Moving averages (MA), Auto-Regressive Moving Averages (ARMA), Auto-Regressive Integrated Moving Averages (ARIMA) etc. as well as methods based on AI such as the ANN, Adaptive Neuro Fuzzy Inference Systems (ANFIS) etc. have been deployed by many researchers, the world around, for gold price forecasting achieving varying degree of success. After making a brief review of these results we present our work on gold price forecasting.
In our work we have proposed a few novel methods based on the classical GDM approach belonging to the ANN class and developed algorithms based on them on MATLAB platform. For this purpose the classical optimization function, the quadratic error function, of the classical GDM method is replaced in turn by several other optimization functions. These newly developed algorithms, along with many other popular ANN algorithms, are then deployed to forecast gold prices in India. This is done by systematically deploying all the above mentioned methods on the same data set of gold prices and computing various efficacy parameters of forecast including the most significant one the mean percent accuracy (MPA). An intercomparison of the results obtained with various methods shows that the hereby proposed methods result in significant improvement in price forecasts. The most efficient optimization function is thus identified for the given data set and the ANN method employed. The manuscript is organized as follows: Section 2 presents a brief description of the research methodology adopted. Section 3 discusses the results obtained and finally Section 4 concludes the findings of the present work.

RESEARCH METHODOLOGY
A few ANN methods, classical and proposed ones, have been employed here in the present work. A brief description of these methods follows. The backpropagation (BP) algorithm that was introduced by Rumelhart is a well-known method for training multilayer feed-forward artificial neural networks [20]. It makes use of the classical GDM technique. The Multi-Layer Perceptron (MLP) networks trained by BP algorithm have been a popular choice in financial applications [21]. The BP algorithm suffers from slow convergence and traps in local minima of error function. Several modifications have been introduced by researchers for improving the local minima problem. These are: gradient descent with momentum, gradient descent with adaptive learning rate, gradient descent with both momentum and adaptive learning rate and resilient backpropagation [22]. Some other methods are based on variations of numerical optimization technique itself resulting in advanced ANN methods like the LM, Quasi Newton and Conjugate Gradient methods. Conjugate Gradient algorithms perform searches along conjugate directions requiring calculation of second derivatives. Nevertheless, its convergence is generally faster than that of the steepest descent direction method [23]. Quasi Newton method may converge still faster than conjugate gradient methods since it avoids calculation of second derivatives. It approximates the Hessian matrix and updates it at each iteration. The Broyden-Fletcher-Goldfarb-Shanno (BFGS) method has been one of the most successful methods in the class of Quasi Newton methods [24][25]. The one step secant (OSS) is formulated in such a way that it attempts to overcome the limitations of the techniques like Quasi Newton and conjugate gradient methods. The method has an inherent computational advantage since Hessian matrix storage at every iteration is not required [26]. The LM method interpolates between the Gauss-Newton and the gradient descent methods. It also avoids calculation of the Hessian Matrix by replacing it by the Jacobian needing lesser computation [27]. Some details of these methods can be found in the work done by [28]. Bayesian regularization minimizes a linear combination of squared errors and weights and it exhibits good generalization qualities [29].
In all the methods mentioned above the optimization function to be minimized, E, is the quadratic sum of the errors, the difference between the actual (yd) and forecasted (yf) values of the variable of interest and is given by (1).
It is envisaged in the present work that variants of the objective function E may improve the forecasting efficiency. A total of seven classical methods mentioned earlier and five modified GDM algorithms proposed and developed by us, using different optimization functions, were chosen for the present study. Various optimization functions are discussed in next section.

Modified GDM methods
In the present work, five modified GDM methods/algorithms have been proposed and developed by us which incorporate different optimization functions instead of the standard quadratic error function in the classical GDM to forecast the gold price [30][31]. These five variants of the optimization function are discussed.

1.
Mean median (MMD) error function: Mean Median error function is given in (2). This function has the advantage of both the Mean error function and Median error function. It reduces the influence of large errors.
Minkowski (MKW) error function: Minkowski error function is given in (3). MSE error can be visualized as a special case of it with r =2. Variations of r around 2 have been observed to have significant influence on accuracy of forecast. Here r has been chosen to be 0.4.
3. Logcosh (LCH) error function: Log cosh error function is given in (4). It approximates x 2 /2 and |x| for small and large values of x respectively.
4. Cauchy (CCY) error function: Cauchy error function is given in (5). Cauchy error function is known to show robustness against outliers.

5.
Negative Logarithmic Likelihood (NLG) error function: Negative Logarithmic Likelihood error function is given in (6) Negative logarithmic likelihood error function is a measure of accuracy of a classifier and is defined as follows.
The modified algorithms incorporating the above functions are here referred to as GDM_MMD, GDM_MKW, GDM_LCH, GDM_CCY and GDM_NLG respectively. Forecasting efficiency is judged here by examining various efficacy parameters described briefly next.

Forecasting efficacy parameters
Many efficacy parameters can be defined to measure degree of success in forecasting. Let yd denote the actual stock price and yf the forecasted one. The error in forecast is then given by e= (ydyf). Let 'n' denote the total number of input-output sets, constructed from the time series of stock prices. Let the mean of all the actual stock prices being forecasted (the desired outputs) be denoted by ymean. The set of efficacy parameters of forecasting considered here to investigate efficiency of AI methods are discussed [32]. 1.
Mean Error (ME) Mean error (ME) is a measure of the average difference between the actual (desired) prices and the forecasted ones and is given in (7). As it shows the direction of error it may also be called forecast bias. 'ME' may turn out to be deceptively small as positive and negative values of individual e t may cancel each other.
RMSE contains all the properties of MSE and is given in (10). It is widely computed and reported in literature.  (11) MPE is independent of the magnitude of the prices as it is the average of percentage of forecasting errors and is defined in (11). It indicates the direction of error. Individual errors of opposite signs nullify each other making it deceptively small. Like 'ME' its value should also be kept as small as possible for a good forecast.
'MPA' depends on 'MAPE' directly. It is defined in (13). As it gives mean percent accuracy and not the error, it may sound more pleasing to some users. 8. Coefficient of Determination (R 2 ) R 2 is a measure that represents the goodness of fit of a model and is given in (14). Large values of R 2 may suggest a good fit to historical data. It may however not be so while forecasting out of sample data [32].

Data collection and preprocessing
Weekly time horizon has been considered in the present work. The gold price data is downloaded from a reliable site on investing [33]. A data window of 4 years of gold prices, from Jan 2015 to Dec 2018, has been considered here. The data consists of the prices of gold for the first trading day of every week in the entire time period. Prices are normalised between 0.1 and 0.9. Time series formation is done by making sets of weekly prices of 5 weeks as input and the price of the 6 th week as the corresponding output. Time series so formed are divided in three categories: training, testing and validation. Training and testing series constitute about initial 80% of the total. About 70% of these series are randomly selected for training the network and the remaining 30% are used for testing. Validation series correspond to chronologically the latest data and constitute about 20% of the total.The validation data is thus not contained in the training and testing data and is referred to, here, as out of sample data. Forecasting accuracy will be examined for varying validation periods.

RESULTS AND DISCUSSION
In the present study the one week ahead price of gold is forecasted from the previous 5 weeks' prices using the classical ANN methods and the modified GDM methods for various ANN architectures. The work done is categorized as below and discussed in detail in sections 3.1 to 3.3.

1.
Forecast of weekly prices of gold using classical ANN algorithms namely GDM, RP, SCG, LM, BR, BFGS, and OSS.

2.
Implementation of the proposed modified GDM methods to increase forecasting efficiency and intercomparison of the results obtained with those obtained by classical GDM.

3.
Comparison with the work done by other researchers for gold forecasting using traditional ANN methods.  Tables 1 and 2. Important observations are summarized below each table. Figure 1 plots MPA, MSE and R 2 obtained in the testing phase for all the algorithms. It can be observed from Table 1 and Figure 1 that, in the testing phase, the maximum MPA (90.35%) is obtained by deploying RP algorithm and the least MSE (2.68E-03) and the maximum R 2 (0.9318) are obtained by BR algorithm.  Further, the results were validated on an out of sample data (validation phase) also for the chronologically latest 46 weeks. All the efficacy parameters except R 2 are shown in Table 2. R 2 is not a reliable performance indicator for the out of sample data.   As the validation results are more significant than the test results for forecasting applications, BR can be considered as the best model for the data in consideration. Graphs in Figure 3, plot desired versus forecasted normalized prices of the testing phase and validation phase for BR algorithm with the ANN configuration 05-03-01. Figure 3. Desired v/s forecasted output for gold price test data and validation data using BR with ANN 05-03-01

Forecasting using the proposed GDM algorithms and inter-comparison with classical GDM
Forecasting has also been done using the 5 novel modified GDM algorithms, GDM_MMD, GDM_MKW, GDM_LCH, GDM_CCY and GDM_NLG, discussed in section 2.1 with various architectures. These results are compared with those of the classical GDM results given in Section 3.1. Efficacy parameters obtained for testing and validation phases are summarized in Tables 3 and 4. It can be observed from the Table 3 that in the testing phase the modified GDM methods give, in general, better accuracy than the classical GDM method for all configurations of the network. A comparison of all the MPAs obtained for various ANN architectures by using different algorithms, is shown graphically in Figure 4. It can be concluded from Figure 4, that in the testing phase, the best MPA obtained is given by the GDM_LCH method with the architecture 05-05-01 and it equals 92.4%. The least MSE obtained is 2.80E-03 and is given by GDM_MMD and GDM_LCH with the architecture 05-07-01. The maximum R 2 obtained is given by GDM_NLG with the architecture 05-07-01 and it equals 0.9229. Once the network is trained and tested it is expected to make good forecasts during the validation period. An important question can be raised here. How does the forecasting accuracy compare for various GDM methods, the classical and the modified ones and also how does it change as the validation time period is changed? In order to answer this question forecasts are made with all the GDM methods for varying lengths of validation period, namely, 4 weeks, 12 weeks, 24 weeks and 46 weeks. The results are tabulated in Table 4. It can be observed from the above table that all the modified GDM methods give better MPAs than the classical GDM method for all configurations of the network for the 46 weeks and 24 weeks validation periods. For 12 and 4 weeks validation periods also the modified methods give, in general, better MPA than the classical GDM. GDM_CCY outperforms all the other modified GDM algorithms as well as the classical GDM in terms of forecasting accuracy. It gives around 96% MPA across all validation time periods. For a better visualisation of our results a consolidated comparison of all the MPAs and MSEs obtained for various ANN architectures by using different algorithms for validation period of entire 46 weeks, is shown graphically in Figure 5. It can be observed from the above figure that for this validation period, the GDM_CCY algorithm with the architecture 05-03-01, outperforms all other modified and classical GDM algorithms in terms of forecasting MPA and MSE. Graphs in Figure 6 plot the desired versus forecasted normalized prices in the testing phase and validation phase respectively for GDM and GDM_CCY algorithms with the ANN configuration 05-03-01. Table 5 shows all the forecasting efficacy parameters of these two methods for the testing phase.  Figure 6. Desired v/s Forecasted output for gold price test data and validation data using Classical GDM and GDM_CCY It can be also be observed that deployment of the modified algorithm, GDM_CCY, improves the MPA by ~ 4% over the classical GDM in the testing phase. Other parameters are also significantly better. Table 6 shows all the forecasting efficacy parameters (except R 2 ) of the two methods for the validation period of 46 weeks. R 2 is not a reliable performance indicator for the out of sample data. It can be seen from the above table that deployment of the modified algorithm, GDM_CCY, improves the MPA by about 1.6% over the classical GDM in the validation phase. Other parameters are also significantly better in the modified method. It can also be observed from Table 2 and Table 6 that in the validation phase, the modified algorithm GDM_CCY outperforms even the best performing classical algorithm i.e. the BR algorithm.

Comparison with work undertaken by other researchers
Various researchers have attempted to forecast gold price in the past usually employing the conventional ANN and other methods. Used recursive and rolling neural network models to forecast onestep-ahead sign variations in gold prices getting an average accuracy of 60.68% [34]. Forecasted gold prices with ANN and ARIMA and concluded that, based on three statistical criteria namely coefficient of determination (R 2 ), mean absolute error (MAE), and root mean square error (RMSE), ANN outperformed ARIMA significantly [35]. Used ARIMA to forecast gold prices and obtained MAPE ranging between 3% and 5% [36]. Compared the gold price forecasting performance of an algorithm BAT-NN with that of ARIMA, ANN, ANFIS, MLP, RBF and GRNN. BAT-ANN produced smallest RMSE while ARIMA (1, 1, 3) gave the smallest value of MAPE 3.135 [37]. Observed that the PCA-GA-BP model gave better results than GA-BP and BP models, average relative errors of gold price prediction being 1.637%, 3.124% and 5.018% [38]. Considered monthly gold prices and compared forecasting performance of a classical ANN method with a hybrid model of ANN and GA. The hybrid model produced a lower MAPE (about 6.3) than that of the ANN model (about 7.6) [39].
In our study the algorithm GDM_CCY forecasts the weekly gold price with MPE about 96% (MAPE 4%) which compares well with the accuracies obtained by others summarized above. One should however note that a direct comparison cannot be done as the data set of the prices is not the same in all the above studies. The moot point here is that the modifications suggested by us result in significant improvements in forecasting efficiency. This is apparent when various efficacy parameters of forecast are compared with and without the proposed modifications.

CONCLUSION
In the present work we proposed and developed a few novel ANN algorithms to enhance forecasting accuracy of gold prices in a popular Indian commodity market. For this purpose we proposed to employ several alternate optimization functions instead of the prevalent quadratic error function based on the mean square error. Weekly gold price forecasts were made using various classical ANN algorithms and the hereby developed modified ANN algorithms. Several efficacy parameters of forecast so obtained have thus been inter compared. It was thus demonstrated that significant improvement in several efficacy parameters including the most important one, the mean percent accuracy (MPA), can in general be achieved when the proposed alternate optimization functions were used. The inter comparison shows that the Cauchy error function based algorithm produced the best results among all the classical as well as other proposed algorithms for the data under consideration. It is therefore, hereby, suggested that the methods proposed by us like GDM_CCY and others should also be deployed before taking a final call on the price forecasts.