提交 5fe09585 authored 作者: Sina Honari's avatar Sina Honari

correcting the code part of faq.txt

上级 77e4a15b
...@@ -2,9 +2,9 @@ ...@@ -2,9 +2,9 @@
=========================== ===========================
Frequently Asked Questions Frequently Asked Questions
=========================== ==================================
Updating a Subset of Weights How to update a subset of weights?
============================= ==================================
If you want to update only a subset of a weight matrix (such as If you want to update only a subset of a weight matrix (such as
some rows or some columns) that are used in the forward propogation some rows or some columns) that are used in the forward propogation
of this iteration, then the cost function should be defined in a way of this iteration, then the cost function should be defined in a way
...@@ -18,36 +18,36 @@ iteration only the rows of the matrix should get updated that their ...@@ -18,36 +18,36 @@ iteration only the rows of the matrix should get updated that their
corresponding words were used in the forward propogation. Here is corresponding words were used in the forward propogation. Here is
how the theano function should be written: how the theano function should be written:
# defining a shared variable for the lookup table >>> # defining a shared variable for the lookup table
>>> lookup_table = theano.shared(matrix_ndarray). >>> lookup_table = theano.shared(matrix_ndarray).
>>>
# getting a subset of the table (some rows >>> # getting a subset of the table (some rows
# or some columns) by passing an integer vector of >>> # or some columns) by passing an integer vector of
# indices corresponding to those rows or columns. >>> # indices corresponding to those rows or columns.
>>> slice = lookup_table[vector_of_indeces] >>> slice = lookup_table[vector_of_indeces]
>>>
# From now on, use only 'slice'. >>> # From now on, use only 'slice'.
# Do not call lookup_table[vector_of_indeces] again. >>> # Do not call lookup_table[vector_of_indeces] again.
# This causes problems with grad as this will create new variables. >>> # This causes problems with grad as this will create new variables.
>>>
# defining cost which depends only on slice >>> # defining cost which depends only on slice
# and not the entire lookup_table >>> # and not the entire lookup_table
>>> cost = something that depends on slice >>> cost = something that depends on slice
>>> g = theano.grad(cost, slice) >>> g = theano.grad(cost, slice)
>>>
# There are two ways for updating the parameters: >>> # There are two ways for updating the parameters:
# either use inc_subtensor or set_subtensor. >>> # either use inc_subtensor or set_subtensor.
# It is recommended to use inc_subtensor. >>> # It is recommended to use inc_subtensor.
# Some theano optimizations do the conversion between >>> # Some theano optimizations do the conversion between
# the two functions, but not in all cases. >>> # the two functions, but not in all cases.
>>> updates = inc_subtensor(slice, g*lr) >>> updates = inc_subtensor(slice, g*lr)
# OR >>> # OR
>>> updates = set_subtensor(slice, slice + g*lr) >>> updates = set_subtensor(slice, slice + g*lr)
>>>
# Note that currently we just cover the case here, >>> # Note that currently we just cover the case here,
# not if you use inc_subtensor or set_subtensor with other types of indexing. >>> # not if you use inc_subtensor or set_subtensor with other types of indexing.
>>>
#defining the theano function >>> #defining the theano function
>>> f=theano.function(..., update=updates) >>> f=theano.function(..., update=updates)
Note that you can compute the gradient of the cost function w.r.t. Note that you can compute the gradient of the cost function w.r.t.
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论