Deep learning for timeseries

• A timeseries can be any data obtained via measurements at regular intervals. working with timeseries involves understanding the dynamics of a system—its periodic cycles, how it trends over time, its regular regime and its sudden spikes.

Different kinds of timeseries tasks

• Classification
• Event detection
• Anomaly detection

A temperature-forecasting example

• target : predicting the temperature 24 hours in the future. we want to develop a deep learning model using Long Short-Term Memory (LSTM) networks for weather forecasting. The model is trained on the Jena Climate Dataset, which contains weather data collected from the Max Planck Institute for Biogeochemistry in Jena, Germany.

Dataset Information

• Source: Max Planck Institute for Biogeochemistry
• Location: Weather Station, Jena, Germany
• Time Frame Considered: January 10, 2009 - December 31, 2016
• Features: The dataset consists of 14 weather-related features which recorded every 10 minutes, including: (Temperature, Pressure, Humidity, Wind speed, Wind direction,…)

Plotting the temperature timeseries

Temperature over the first 30 days of the dataset (ºC)

Preparing the data

• Normalizing the data
• Instantiating datasets for training, validation, and testing
• train model
• evaluate model
• plotting model predictions

Model Architectur

The weather forecasting model is built using PyTorch and is based on an LSTM neural network to capture temporal dependencies in time-series weather data.

Advanced use of recurrent neural networks

• Recurrent dropout— This is a variant of dropout, used to fight overfitting in recurrent layers.
• Stacking recurrent layers—This increases the representational power of the model (at the cost of higher computational loads).
• Bidirectional recurrent layers—These present the same information to a recurrent network in different ways, increasing accuracy and mitigating forgetting issues.
• Adjust the number of units in each recurrent layer in the stacked setup, as well as the amount of dropout.
• Adjust the learning rate used by the RMSprop optimizer, or try a different optimizer.
• Try using a stack of Dense layers as the regressor on top of the recurrent layer,instead of a single Dense layer.
• Improve the input to the model: try using longer or shorter sequences or a different sampling rate, or start doing feature engineering.

model summary

Plotting Model Predictions for different training scheme