Time Series Prediction with LSTM Recurrent Neural Networks in Python with Keras
Time series prediction problems are a difficult type of predictive modeling problem.
Unlike regression predictive modeling, time series also adds the complexity of a sequence dependence among the input variables.
A powerful type of neural network designed to handle sequence dependence is called recurrent neural networks. The Long Short-Term Memory network or LSTM network is a type of recurrent neural network used in deep learning because very large architectures can be successfully trained.
In this post, you will discover how to develop LSTM networks in Python using the Keras deep learning library to address a demonstration time-series prediction problem.
After completing this tutorial you will know how to implement and develop LSTM networks for your own time series prediction problems and other more general sequence problems. You will know:
- About the International Airline Passengers time-series prediction problem.
- How to develop LSTM networks for regression, window and time-step based framing of time series prediction problems.
- How to develop and make predictions using LSTM networks that maintain state (memory) across very long sequences.
In this tutorial, we will develop a number of LSTMs for a standard time series prediction problem. The problem and the chosen configuration for the LSTM networks are for demonstration purposes only they are not optimized.
These examples will show you exactly how you can develop your own differently structured LSTM networks for time series predictive modeling problems.
Let’s get started.
- Update Oct/2016: There was an error in the way that RMSE was calculated in each example. Reported RMSEs were just plain wrong. Now, RMSE is calculated directly from predictions and both RMSE and graphs of predictions are in the units of the original dataset. Models were evaluated using Keras 1.1.0, TensorFlow 0.10.0 and scikit-learn v0.18. Thanks to all those that pointed out the issue, and to Philip O’Brien for helping to point out the fix.
- Update Mar/2017: Updated example for Keras 2.0.2, TensorFlow 1.0.1 and Theano 0.9.0.
- Update Apr/2017: For a more complete and better explained tutorial of LSTMs for time series forecasting see the post Time Series Forecasting with the Long Short-Term Memory Network in Python.
Updated LSTM Time Series Forecasting Posts:
The example in this post is quite dated, I have better examples available for using LSTMs on time series, see: