CS231n Assignment3--Q2

Q2: Image Captioning with LSTMs (35 points)

作业代码已上传至我github: https://github.com/jingshuangliu22/cs231n,欢迎参考、讨论、指正。

LSTM_Captioning.ipynb

Load MS-COCO data

idx_to_word <type 'list'> 1004train_captions <type 'numpy.ndarray'> (400135, 17) int32val_captions <type 'numpy.ndarray'> (195954, 17) int32train_image_idxs <type 'numpy.ndarray'> (400135,) int32val_features <type 'numpy.ndarray'> (40504, 512) float32val_image_idxs <type 'numpy.ndarray'> (195954,) int32train_features <type 'numpy.ndarray'> (82783, 512) float32train_urls <type 'numpy.ndarray'> (82783,) |S63val_urls <type 'numpy.ndarray'> (40504,) |S63word_to_idx <type 'dict'> 1004

LSTM: step forward

next_h error: 5.70541304045e-09
next_c error: 5.81431230888e-09

dx error: 6.6214710385e-09
dh error: 5.10364159425e-10
dc error: 1.46461864645e-08
dWx error: 9.92319895433e-09
dWh error: 6.53036889306e-08
db error: 2.05724967677e-09

LSTM: forward

h error: 8.61053745211e-08

dx error: 6.01572477862e-10
dh0 error: 6.01572477862e-10
dWx error: 6.01572477862e-10
dWh error: 6.01572477862e-10
db error: 6.01572477862e-10

loss: 9.82445935443
expected loss: 9.82445935443
difference: 2.26485497024e-12

Overfit LSTM captioning model

(Iteration 1 / 100) loss: 78.372126
(Iteration 11 / 100) loss: 44.292632
(Iteration 21 / 100) loss: 27.702264
(Iteration 31 / 100) loss: 10.905141
(Iteration 41 / 100) loss: 5.647874
(Iteration 51 / 100) loss: 1.872165
(Iteration 61 / 100) loss: 0.460800
(Iteration 71 / 100) loss: 0.196382
(Iteration 81 / 100) loss: 0.195097
(Iteration 91 / 100) loss: 0.120261

LSTM test-time sampling