Saltar al contenido principal
CargarIdioma (ES)Atención al cliente
EmpresarialesMóvilRedes socialesMarketingTecnologíaArte y fotografíaEmpleoDiseñoEducaciónPresentaciones y charlas públicasGobierno y org. sin ánimo de lucroAtención sanitariaInternetDerechoLiderazgo y gestiónMotorIngenieríaSoftwareReclutamiento y RR. HH.Venta al por menorVentasServiciosCienciasPYMES y liderazgoAlimentaciónMedio ambienteEconomía y finanzasDatos y análisisRelaciones con inversoresDeportesMeditaciónNoticias y políticaViajesDesarrollo personalSector inmobiliarioEntretenimiento y humorSalud y medicinaDispositivos y hardwareEstilo de vida
Setu Chokshi, profile picture
Subido porSetu Chokshi
PPTX, PDF2,660 vistas

Time series predictions using LSTMs

The document discusses the use of Long Short-Term Memory (LSTM) networks for time series predictions, detailing their architecture and functionality. It highlights methods to incorporate memory into neural networks to effectively model time series data and provides examples using sine wave prediction and power consumption datasets. Additionally, it includes several references for further understanding of LSTM networks and their application.

Tecnología◦
Temas relacionados:
Deep Learning•
Descargado 99 veces
Time Series Predictions using Long Short-Term Memory Setu Chokshi IoT Asia 2017 – 30th March
LSTMs are mainstream
What are Neural Network Y1 Y2 X1 X2 X3 h1 h2 h3 h4 Information Transfer Input Values Calculator – Activations Output (Activation) 1 2 3 4 Input Hidden Output X1 X2 X3 W11 W21 W31   h1 Example of  = REctified Linear Unit = max(0,input) Strengthen weak signals; Leave strong signals alone Weight is the strength of the connection between nodes 1
Challenges  Only fixed sized inputs & outputs  Performs mapping of features from input to output  No memory and hence difficult to model time series
Lets add some memory Input Hidden Output Input + prevInput Hidden Output Input+ prevHidden Hidden Output Input + prevInput Hidden Output Input + prevInput Hidden Output Input + prevInput Hidden Output Input + prevInput Hidden Output Input + prevHidden Hidden Output Input + prevHidden Hidden Output Input + prevHidden Hidden Output Input + prevHidden Hidden Output Approach 1: Add previous inputs Approach 2: Add previous hidden Lets do 4 time series steps Lets do 4 time series steps
Lets add some memory…and color Input Hidden Output Input + prevInput Hidden Output Input+ prevHidden Hidden Output Input + prevInput Hidden Output Input + prevInput Hidden Output Input + prevInput Hidden Output Input + prevInput Hidden Output Input + prevHidden Hidden Output Input + prevHidden Hidden Output Input + prevHidden Hidden Output Input + prevHidden Hidden Output Approach 1: Add previous inputs Approach 2: Add previous hidden Lets do 4 time series steps Lets do 4 time series steps
Lets build an LSTM Xt     0 1 2 3 ht-1 Ct-1 Ct✖ ✖  ✖σ σ σtanh tanh ht ht Ct-1
Now lets build an LSTM Element- wise Summation / Concatenati on Element- wise multiplicatio n X t ht- 1 Ct- 1 h t C t 0 σ  ✖ tanh Inputs: Outputs: Input vector Memory from previous blockOutput of previous block Memory from current blockOutput of current block Nonlinearities: Sigmoid Hyperbolic tangent Vector operations: Bias:
Lets skip the math, ok? Element- wise summation Element- wise multiplicatio n ✖ = =
Memory Pipeline Xt     0 1 2 3 ht-1 Ct-1 C t ht ✖ ✖  ✖σ σ σtanh tanh ht ✖
Forget Layer Xt     0 1 2 3 ht-1 Ct-1 Ct ht ✖ ✖  ✖σ σ σtanh tanh ht
Generate new memories: Input Xt   +  0 1 2 3 ht-1 Ct-1 Ct ht ✖ ✖  ✖σ σ σtanh tanh ht  2 tanh
Generate new memories: Candidate Xt     0 1 2 3 ht-1 Ct-1 Ct ht ✖ ✖  ✖σ σ σtanh tanh ht
Memory Pipeline Xt     0 1 2 3 ht-1 Ct-1 Ct ht ✖ ✖  ✖σ σ σtanh tanh ht
Generate the output Xt     0 1 2 3 ht-1 Ct-1 Ct ht ✖ ✖  ✖σ σ σtanh tanh ht
EXAMPLES
Sin wave predictor  Generate sin curve  Load 5000 X 50 sequences  90:10 split on train/test sets
Power Consumption Dataset  Power Consumption Dataset  47 Months of data  2075259 measurements  Active energy consumed per min  Load 4567 X 50 sequences  90:10 split on train/test sets
References  Understanding LSTM Networks http://colah.github.io/posts/2015-08-Understanding- LSTMs/  General Sequence Learning using Recurrent Neural Networks https://www.youtube.com/watch?v=VINCQghQRuM  Recurrent Neural Networks Part 1: Theory https://www.slideshare.net/gakhov  Facebook Prophet https://github.com/facebookincubator/prophet  Images adapted from Shi Yan https://medium.com/@shiyan/understanding-lstm-and-its- diagrams-37e2f46f1714  Anyone Can Learn To Code https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/
THANK YOU @setuc www.linkedin.com/in/setuchoks hi/ github.com/setuc/iotAsia2017