Written by: Avinash Kumar Pandey (Research Associate, ISB Hyderabad)
Introduction:
In this article, we will learn how to apply deep convolutional networks for predicting 1D time-series/sequences in python. This model could be easily applied to the stock-price prediction problem. Deep CNNs have been quite popular in areas such as Image Processing, Computer Vision, etc. Recently, the research community has been showing a growing interest in using CNNs for time-series forecasting problems. This article will be useful for a wide range of readers including deep learning enthusiasts, finance professionals, academicians, and data-science hobbyists.
Convolutional Neural Networks: Analogy between Computer Vision & Time Series Forecasting
In this section, we will start with an Image processing example to understand intuitively the similarity between Computer Vision and Time Series Analysis using CNNs. Let’s get started with the example involving this puppy (because it’s very cute).
CNN’s interpretation of Image matrices
Every image that is digitally available is actually a matrix of pixel values. Each pixel value can range from 0 to 255 depending on the intensity of the pixel. Each image also comprises channels depending on the color composition of the image. A grey image has one channel since each channel corresponds to the colors it contains. A color image has three channels comprising red, blue, and green colors. A convolutional neural network perceives each image as a matrix of pixel values in the dimension of image width, length, and the number of channels.
For example, let us assume that the puppy’s image is 2000 pixels wide and 2500 pixels in height. Also, since it is a color image, it would have 3 channels. Hence, the image would consist of 3 matrices of the size of dimensions [2500,2000]. Overall, we have a 3D matrix of dimension [2500,2000,3]. This 3D matrix then goes in as an input to the CNN model. Therefore, for a CNN model the above image is interpreted (in an over-simplified example) as a matrix as shown below :
Now that we have an understanding of how an image is perceived by the Machine Learning Model, we will proceed to understand the primary components of a Deep CNN model and understand how these components are also used for time series forecasting. Following are the primary layers of an ordinary CNN model.
1. Convolutional Layer
2. Pooling Layer
3. Fully Connected Layer
These layers are explained in brief for better understanding.
Convolutional Layer
In the convolutional layer, the image input matrix is multiplied by a feature matrix to extract important features of the image. The filter matrix slides through the input matrix and basically extracts the information from smaller sub-sections of the image. There are various filter matrices available to extract different types of features from the image. The figure below shows a basic filter operation on our input image matrix.
Image Source: provided by the Author
Pooling Layer
In the pooling layer, the resultant matrix is then multiplied to pooling matrix which extracts the maximum or average values from the small subsections. The pooled image is significantly compressed in size than the main image matrix. A basic illustration is shown below.
Fully Connected Layer
The pooled matrix is then flattened and then fed to the fully connected layer which learns the images through its neural networks. In our time series stock price forecasting example, the 1D time series is converted to a 3D matrix using the methodology below and the neural network analogy remains the same. A full flow chart for the application of CNN to time series prediction is shown below. The only difference between computer vision problems and time series ones is the input we give to the model (image matrix for computer vision and 1D array for time series forecast ).
Hence, if we are able to convert the 1D time-series sequence to an input image matrix shape, we could apply a CNN model for the forecasting problem. We will now discuss the methodology and a simple test sequence on which we will apply our model.
Methodology for CNN model:
We will be following the below-mentioned pathway for applying CNNs to a univariate 1D time series :
1) Import Keras libraries and dependencies
2) Define a function that extracts features and outputs from the sequence.
3) Reshape the input X in a format that is acceptable to CNN models
4) Design the CNN model architecture of convolutional layers(Conv-1D), pooling(max-pooling in our case ), flattening layer, and the fully connected neural layers.
5) Train the model and test it on our univariate sequence.
Data
Important Note regarding the data sequence: In this article, I will be showing you how to apply a CNN model to a 1D time series sequence. For illustration purposes, we will be using a very simple sequence from [100 to 190 ] with a common difference of 10 and see if our CNN model is able to pick up on that. You can always use stock price time-series data from open sources such as yahoo finance by using python library yfinance and I would leave that exercise on the reader.
Code Section :
Step 1: Firstly, import all the libraries from Keras for neural network architectures.
Step 2: Next, we will define a function that extracts features (lagged values) and outputs from the sequence.
As an example, let’s suppose the sub-sequence is [ 10, 20, 30, 40 ] and we decide to predict the sequence using the last two lagged values(defined as “steps” in the function). Then in our small example, the model will train and learn to predict the 3rd value (in our case “30”) using first and second values [10,20] as features. Similarly, the last value (“40”) will be predicted using two preceding terms ([20, 30]). Our function split_sequence creates such lists of features and outputs from the sequence and feeds the output to the model for training.
Step 3: Initializing Sequence, steps, and reshaping the output to input it to our CNN model.
As explained in step 2, we have created a function that converts a 1D sequence into sub-sequences (features and predicted values). In this step, we will define an artificial sequence from 100 to 190 with a gap of 10. Obviously, we know that the next logical number in this sequence should be 200. Now, we would like to see if the CNN model is to learn the sequence pattern and predict the next terms.
Output for the above section of code :
After reshaping the X matrices we will get the output as shown below:
Output for the above section of the code :
Step 4: Defining CNN Model Architecture
In this step, we would define the model and put down the necessary setting for our model. We would be using “ReLu” as our activation function. We are adding Conv1D as our convolutional layer since we are dealing with a one-dimensional sequence. We then add the MaxPooling layer and Flatten layer for pooling and flattening of the input matrix to be served as an input to the fully connected neural networks (100 neurons) to learn the pattern in our sequence. Lastly, we set one neuron for our output (Dense = 1). We will be using mean squared error as our loss function with Adam (Adaptive Moment Estimator) as our optimizer in the gradient descent problem.
Step 5: CNN Model Fitting
Step 6: Running the Model:
We see that the CNN model is able to identify the pattern and is able to predict the next steps. Full code is available at this Github link and you are free to use it. Also, code for multiple steps in the future is also included which you can use with stock price data.
Conclusion:
We learned how to convert a 1D univariate time series to an input matrix suitable for CNN model input. We then designed a Deep CNN model which was able to learn the pattern in the sequence and predict the output accurately. CNN models are popular for detecting the patterns in the pixel matrix via their convolutional layers. Similarly, upon suitable treatment, patterns (cyclical and trend components) in the time series data could be learned effectively by the CNN model.
Advanced CNN architecture, combined with other Deep Learning models such as LSTM could yield better performance. These models are currently being researched by academicians and practitioners alike.
About the Author :
Hi, My name is Avinash Pandey. I am passionate about Machine Learning applications in Finance and Portfolio Management. I am a graduate of IIT-BHU and currently working as a Research Associate in Finance Dept. at ISB Hyderabad. You can reach out to me on Linkedin or email me at [email protected].
References :
1) Artificial Intelligence in Finance (A python-based guide): Yves Hilpisch (O’Reilly Publication)
2)Chollet, François. 2017. Deep Learning with Python. Shelter Island: Manning
4) Python for Finance Cookbook: Erik Lewinson (Packt Publishing)
