Abstract:Feature engineering is an essential step in the pipelines used for many machine learning tasks, including time-series forecasting. Although existing AutoML approaches partly automate feature engineering, they do not support specialised approaches for applications on time-series data such as multi-step forecasting. Multi-step forecasting is the task of predicting a sequence of values in a time-series. Two kinds of approaches are commonly used for multi-step forecasting. A typical approach is to apply one model to predict the value for the next time step. Then the model uses this predicted value as an input to forecast the value for the next time step. Another approach is to use multi-output models to make the predictions for multiple time steps of each time-series directly. In this work, we demonstrate how automated machine learning can be enhanced with feature engineering techniques for multi-step time-series forecasting. Specifically, we combine a state-of-the-art automated machine learning system, auto-sklearn, with tsfresh, a library for feature extraction from time-series. In addition to optimising machine learning pipelines, we propose to optimise the size of the window over which time-series data are used for predicting future time-steps. This is an essential hyperparameter in time-series forecasting. We propose and compare (i) auto-sklearn with automated window size selection, (ii) auto-sklearn with tsfresh features, and (iii) auto-sklearn with automated window size selection and tsfresh features. We evaluate these approaches with statistical techniques, machine learning techniques and state-of-the-art automated machine learning techniques, on a diverse set of benchmarks for multi-step time-series forecasting, covering 20 synthetic and real-world problems. Our empirical results indicate a significant potential for improving the accuracy of multi-step time-series forecasting by using automated machine learning in combination with automatically optimised feature extraction techniques.Keywords: automated machine learning; machine learning; time-series forecasting
LSTM with a direct sequence-to-sequence multivariate approach performs the best for few-step-ahead forecasting in the limited data size. It has promising application for longer-step-ahead forecasting provided that the data size is sufficiently large.
Steps Ahead Trains Pdf 17
DOWNLOAD: https://vittuv.com/2vCccq
In a time-series sequence, prediction can be classified into two categories, known as single-step ahead and multi-step ahead (Saroha & Aggarwal 2014). Predicting multiple time steps into the future is called multi-step time-series forecasting. It also includes the forecasting of some variables for some future time steps, given over a significant time span of data (Dabrowski et al. 2020). Moreover, the extent of future forecasting is known as the forecasting horizon. Highly chaotic time series and those with missing data pose important issues in multi-step-ahead prediction, which have been addressed using non-linear filters and neural networks (Chandra et al. 2021). According to Guo et al. (2020), the recursive strategy has been normally applied in several fields among five strategies to address a multi-step-ahead prediction task that has been suggested in the literature. Besides the recursive strategy, a sequence-to-value and sequence-to-sequence forecasting approach has also been used in multi-step-ahead forecasting (Wunsch et al. 2021).
Based on the literature review presented, it can be observed that LSTM and NARX forecasting models are commonly used in multi-step-ahead prediction involving nonlinear multivariate data. The challenge in multi-step-ahead modelling is that normally the model achieves high accuracy for one or a few steps ahead. In other words, most of the studies with high accuracy conducted using NARX and LSTM are applied for one or a few steps ahead of forecasting. This study aims to build a model for multi-step-ahead forecasting of RF based on NARX Neural Networks and LSTM in univariate and multivariate approaches. In NARX, we will use direct autoregression and a recursive model. Unlike the recursive NARX approach, the direct autoregression NARX approach tries to train a multi-step-ahead prediction model directly without requiring future exogenous inputs to make multi-step prediction. In LSTM, we will use a direct sequence-to-sequence multivariate model and also a recursive univariate model, both of which will be discussed in Section 2.
The stage of preliminary data analysis and pre-processing is crucial in the initial stage of the machine learning model building. This process can significantly affect the prediction accuracy in any type of data (Hayder et al. 2020). The purpose of this study is to build a multi-step-ahead predictive model for RF using a machine learning/deep learning-based approach.
There are basically two ways of performing multi-step-ahead forecasts. It can be produced recursively by iterating a one-step-ahead model, or directly using a specific model for each span (Ben Taieb et al. 2012). In addition, the recorded RF data, together with other climate variables, are basically time-series data. Thus, the multi-step-ahead time-series prediction model can be developed based on two strategies, namely, univariate and multivariate time-series prediction. In the univariate time-series forecasting, it is assumed that RF has its seasonality and it depends on its past values only. On the other hand, in multivariate time-series forecasting, the RF will also depend on the other variables and their past values.
In this study, the multi-step-ahead prediction is performed using a nonlinear machine learning-based model with multivariate and univariate approaches. The two algorithms used are NARX (nonlinear autoregressive with exogenous inputs) Neural Networks and recurrent Neural Networks called LSTM, which fall under the category of deep learning.
There are two machine learning models used in this study as mentioned, namely NARX and LSTM. The architecture of NARX can be seen in Figure 4, where two types of connection are common (Thapa et al. 2020). NARX is used in multivariate mode that is applied to predict some step-ahead values of RF based on current and past values (lags) of WR and AE.
Figure 13 shows the recursive forecasting result of LSTM1 on the test data where RMSE = 191.42 is achieved, which is slightly lower than that achieved with NARX1. Figure 14 shows the observed vs predicted values, and NSE = 0.59 is achieved. This result indicates that LSTM capability for multi-step-ahead prediction in the univariate (NAR) approach is applied in this case.
The last part of this section is to highlight the comparison between this paper and other cited studies for multi-step-ahead prediction in hydrological systems. This comparison is viewed from some aspects of model algorithms, data size and model performance. This comparative summary is given in Table 7. We can see that in terms of data size, our study has a much lower number of samples as compared with other studies. This is certainly our major concern for improvement in the future. In terms of algorithms, NARX Neural Networks are commonly used, while LSTM is still rarely explored especially for multi-step-ahead forcasting, i.e. not considering studies in single-step-ahead forecasting. There is one interesting recent study by Guo et al. (2021) pointing out that gradient boosting machine, a new type of machine learning algorithm, performs favourably for some step-ahead prediction. This application could be explored in the future. In terms of accuracy, our proposed method in few-step-ahead forecasting using LSTM particularly is generally on a par with or slightly better than some results from other studies.
Four approaches for multi-step-ahead forecasting for the Kelantan RF in Malaysia using NARX neural networks and deep learning LSTM have been discussed. These approaches use different strategies to perform multi-step-ahead forecasting involving univariate and multivariate methods. The first approach is NARX neural networks using the recursive approach that gives acceptable performance with RMSE = 208.85 and NSE = 0.44 on test dataset. The second approach is NARX neural networks with direct multi-step-ahead prediction, which produces the highest NSE = 0.41 in four-step-ahead and nine-step-ahead forecasting in our experiment. The third approach is LSTM with a direct sequence-to-sequence prediction, which performs better for few-step-ahead forecasting, i.e. NSE = 0.78 for two-step-ahead forecasting. The performance, however, degrades quite significantly when longer-step-ahead is performed. The fourth approach is the LSTM univariate recursive method, which performs slightly better than the first approach. The third approach is promising for few-step-ahead prediction, but it needs larger data to build the model that is able to perform longer-step-ahead forecasting. Future work will involve collecting more data and also investigating optimization of hyperparameters of the machine learning/deep learning-based model such as using grid search or meta-heuristic optimization, which has been done earlier in another study (Hayder et al. 2020). Applications of ensemble and boosting machine learning such as random forest and gradient boosting algorithms can be explored as well. The application of this kind of study is important for water resource management and flood mitigation planning as Kelantan River has recorded some flooding events in the recent past.
With maglev technology, the train travels along a guideway of electromagnets which control the train's stability and speed. While the propulsion and levitation require no moving parts, the bogies can move in relation to the main body of the vehicle and some technologies require support by retractable wheels at low speeds under 150 kilometres per hour (93 mph). This compares with electric multiple units that may have several dozen parts per bogie. Maglev trains can therefore in some cases be quieter and smoother than conventional trains and have the potential for much higher speeds.[3] 2ff7e9595c
Comments