What's the difference of name scope and a variable scope in tensorflow?

点击打开链接

https://stackoverflow.com/questions/35919020/whats-the-difference-of-name-scope-and-a-variable-scope-in-tensorflow

Let's begin by a short introduction to variable sharing. It is a mechanism in TensorFlow that allows for sharing variables accessed in different parts of the code without passing references to the variable around. The method tf.get_variable can be used with the name of the variable as the argument to either create a new variable with such name or retrieve the one that was created before. This is different from using the tf.Variable constructor which will create a new variable every time it is called (and potentially add a suffix to the variable name if a variable with such name already exists). It is for the purpose of the variable sharing mechanism that a separate type of scope (variable scope) was introduced.

As a result, we end up having two different types of scopes:

• name scope, created using tf.name_scope
• variable scope, created using tf.variable_scope

Both scopes have the same effect on all operations as well as variables created using tf.Variable, i.e., the scope will be added as a prefix to the operation or variable name.

However, name scope is ignored by tf.get_variable. We can see that in the following example:

with tf.name_scope("my_scope"):    v1 = tf.get_variable("var1", [1], dtype=tf.float32)    v2 = tf.Variable(1, name="var2", dtype=tf.float32)    a = tf.add(v1, v2)print(v1.name)  # var1:0print(v2.name)  # my_scope/var2:0print(a.name)   # my_scope/Add:0

The only way to place a variable accessed using tf.get_variable in a scope is to use a variable scope, as in the following example:

with tf.variable_scope("my_scope"):    v1 = tf.get_variable("var1", [1], dtype=tf.float32)    v2 = tf.Variable(1, name="var2", dtype=tf.float32)    a = tf.add(v1, v2)print(v1.name)  # my_scope/var1:0print(v2.name)  # my_scope/var2:0print(a.name)   # my_scope/Add:0

This allows us to easily share variables across different parts of the program, even within different name scopes:

with tf.name_scope("foo"):    with tf.variable_scope("var_scope"):        v = tf.get_variable("var", [1])with tf.name_scope("bar"):    with tf.variable_scope("var_scope", reuse=True):        v1 = tf.get_variable("var", [1])assert v1 == vprint(v.name)   # var_scope/var:0print(v1.name)  # var_scope/var:0

UPDATE: op_scope/variable_op_scope is deprecated!

As of version r0.11, op_scope and variable_op_scope are both deprecated and replaced by name_scope and variable_scope.

shareimprove this answer

edited May 30 at 5:05

Bishwajit Purkaystha

150110

answered May 30 '16 at 23:09

Andrzej Pronobis

10.7k63458

 

7 

Thanks for the clear explanation. Naturally, a follow up question would be "Why does Tensorflow have both of these confusingly similar mechanisms? Why not replace them with just one scope method which effectively does a variable_scope?" – John Feb 25 at 18:02

 

I don't think I understand conceptually why the distinction between variable_scope vs name_scopeis even needed. If one creates a variable (in any way with tf.Variable or tf.get_variable), it seems more natural to me that we should always be able to get it if we specify the scope or its full name. I don't understand why one ignores the scope name thing while the other doesn't. Do you understand the rational for this weird behaviour? – Charlie Parker Mar 20 at 19:31

7 

The reason is that with variable scope, one can define separate scopes for re-usable variables that are not affected by the current name scope used to define operations. – Andrzej Pronobis Mar 21 at 5:23 

 

What I understand is name_scope is mainly useful when you name two variables with same name but in different operations. – DINESHKUMAR MURUGAN Jul 14 at 17:45

 

Hello， can you explain why the variable name in a variable_scope always ends with a :0? Does this means there may be variable names ends with :1, :2, etc, so how can this happens? – James Fan Aug 31 at 8:49