提交 516dced0 authored 作者: Razvan Pascanu's avatar Razvan Pascanu

Fix to the provided memory bug with truncated BPTT.

If scan receives memory in the output storage map it uses it. Now if you run truncated BPTT, you do not loop over the entire sequence, and you should return 0 for all the entries you don not visit. I used to assume that the data I allocate does that ( because I used to allocate it using zeros), but now if data is provided I did not check if those values are actually 0s.
上级 f09f515b
......@@ -593,6 +593,12 @@ class Scan(Op):
outs[idx][0][store_steps[idx]-pdx:] = outs[idx][0][:pdx]
outs[idx][0][:store_steps[idx]-pdx] = tmp
del tmp
# This would normally happen only when doing truncated
# backpropagation through time. In such a scenarion Scan is
# expected to return 0 for all entries for which the gradient is
# not actually computed
elif store_steps[idx] > n_steps - self.mintaps[idx]:
outs[idx][0][n_steps-self.mintaps[idx]:] = 0
t_call = time.time() - t0_call
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论