CS231n Assignment3--Q1

Q1: Image Captioning with Vanilla RNNs

作业代码已上传至我github: https://github.com/jingshuangliu22/cs231n,欢迎参考、讨论、指正。

LSTM_Captioning.ipynb

Microsoft COCO

idx_to_word <type 'list'> 1004train_captions <type 'numpy.ndarray'> (400135, 17) int32val_captions <type 'numpy.ndarray'> (195954, 17) int32train_image_idxs <type 'numpy.ndarray'> (400135,) int32val_features <type 'numpy.ndarray'> (40504, 512) float32val_image_idxs <type 'numpy.ndarray'> (195954,) int32train_features <type 'numpy.ndarray'> (82783, 512) float32train_urls <type 'numpy.ndarray'> (82783,) |S63val_urls <type 'numpy.ndarray'> (40504,) |S63word_to_idx <type 'dict'> 1004

Look at the data

Vanilla RNN: step forward

next_h error: 6.29242142647e-09

Vanilla RNN: step backward

dx error: 6.88735954327e-11
dprev_h error: 5.28932394133e-10
dWx error: 1.12554920911e-10
dWh error: 4.84496557569e-10
db error: 2.72330774095e-11

Vanilla RNN: forward

h error: 7.72846618019e-08

Vanilla RNN: backward

dx error: 2.70104774724e-08
dh0 error: 1.7454525052e-09
dWx error: 3.40035760677e-10
dWh error: 2.01095678956e-09
db error: 3.23168709094e-10

Word embedding: forward

out error: 1.00000000947e-08

Word embedding: backward

Word embedding: backward

Temporal Affine layer

dx error: 4.98623200795e-11
dw error: 7.54622091734e-11
db error: 5.76987410469e-12

Temporal Softmax loss

2.30256439876
23.025705242
2.32606402665
dx error: 2.45647211476e-08

RNN for image captioning

loss: 9.83235591003
expected loss: 9.83235591003
difference: 2.61124455392e-12

W_embed relative error: 2.006287e-09
W_proj relative error: 2.435961e-09
W_vocab relative error: 2.411310e-09
Wh relative error: 2.055948e-08
Wx relative error: 3.195020e-07
b relative error: 1.777874e-09
b_proj relative error: 1.159276e-09
b_vocab relative error: 1.960674e-10

Overfit small data

(Iteration 1 / 100) loss: 82.463010
(Iteration 11 / 100) loss: 27.939999
(Iteration 21 / 100) loss: 8.880015
(Iteration 31 / 100) loss: 1.921411
(Iteration 41 / 100) loss: 0.639671
(Iteration 51 / 100) loss: 0.340682
(Iteration 61 / 100) loss: 0.287836
(Iteration 71 / 100) loss: 0.180632
(Iteration 81 / 100) loss: 0.187963
(Iteration 91 / 100) loss: 0.179619

Test-time sampling