提交 5fe09585 authored 作者: Sina Honari's avatar Sina Honari

correcting the code part of faq.txt

上级 77e4a15b
......@@ -2,9 +2,9 @@
===========================
Frequently Asked Questions
===========================
Updating a Subset of Weights
=============================
==================================
How to update a subset of weights?
==================================
If you want to update only a subset of a weight matrix (such as
some rows or some columns) that are used in the forward propogation
of this iteration, then the cost function should be defined in a way
......@@ -18,36 +18,36 @@ iteration only the rows of the matrix should get updated that their
corresponding words were used in the forward propogation. Here is
how the theano function should be written:
# defining a shared variable for the lookup table
>>> # defining a shared variable for the lookup table
>>> lookup_table = theano.shared(matrix_ndarray).
# getting a subset of the table (some rows
# or some columns) by passing an integer vector of
# indices corresponding to those rows or columns.
>>>
>>> # getting a subset of the table (some rows
>>> # or some columns) by passing an integer vector of
>>> # indices corresponding to those rows or columns.
>>> slice = lookup_table[vector_of_indeces]
# From now on, use only 'slice'.
# Do not call lookup_table[vector_of_indeces] again.
# This causes problems with grad as this will create new variables.
# defining cost which depends only on slice
# and not the entire lookup_table
>>>
>>> # From now on, use only 'slice'.
>>> # Do not call lookup_table[vector_of_indeces] again.
>>> # This causes problems with grad as this will create new variables.
>>>
>>> # defining cost which depends only on slice
>>> # and not the entire lookup_table
>>> cost = something that depends on slice
>>> g = theano.grad(cost, slice)
# There are two ways for updating the parameters:
# either use inc_subtensor or set_subtensor.
# It is recommended to use inc_subtensor.
# Some theano optimizations do the conversion between
# the two functions, but not in all cases.
>>>
>>> # There are two ways for updating the parameters:
>>> # either use inc_subtensor or set_subtensor.
>>> # It is recommended to use inc_subtensor.
>>> # Some theano optimizations do the conversion between
>>> # the two functions, but not in all cases.
>>> updates = inc_subtensor(slice, g*lr)
# OR
>>> # OR
>>> updates = set_subtensor(slice, slice + g*lr)
# Note that currently we just cover the case here,
# not if you use inc_subtensor or set_subtensor with other types of indexing.
#defining the theano function
>>>
>>> # Note that currently we just cover the case here,
>>> # not if you use inc_subtensor or set_subtensor with other types of indexing.
>>>
>>> #defining the theano function
>>> f=theano.function(..., update=updates)
Note that you can compute the gradient of the cost function w.r.t.
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论