Tensor and Tensor Flow Graph

Neural Network Tutorials - Herong's Tutorial Examples

∟Tensor and Tensor Flow Graph

This section provides a tutorial example on how to use 'tensorflow' functions to create a simple tensor flow graph.

In the last tutorial, we have described the key concept, Tensor Flow Graph, used in TensorFlow. Now let's have a loser look at it.

A Tensor Flow Graph, also called a Computational Graph, a Dataflow Graph or TensorFlow Graph, is a graphical representation of an expression of multiple tensor operations. Here is the same tensor flow graph we have looked at before. It represents the tensor operation of [a] = ([b]+[c])*([c]+[2]), if we use [.] to as the tensor notation.

Tensor Flow Graph Example: [a] = ([b]+[c])*([c]+[2])

As you can see from the picture, a node in a tensor flow graph represents a single tensor operation, and an edge represents a single tensor flowing from one operation into another operation.

From a programming point of view, a node is really a structured data object (an instance of the "tensorflow" class) that holds the following primary properties:

The operation that produces an output tensor.
The dimensional shape of the output tensor.
The data type of the output tensor elements.
An ordered list of links to other nodes whose output tensors are used as input tensors to the operation.

If you connect all nodes (or tensor operations) through their links, you will get the final tensor flow graph.

Since an output tensor and the operation that generates the tensor has a one-to-one relation, a tensor operation (or node in a tensor flow graph) is also called a tensor. So the term tensor and the term operation are used interchangeably. If you call an operation as a tensor, you are referring to the tensor generated by the operation. If you call a tensor as an operation, you are referring to the operation that generates the tensor.

There are 3 main "tensorflow" functions that create 3 different flavors of tensor operations (or 3 different flavors of tensors if you refer to their output tensors) with no input tensors.

tf.Variable() - Creates a tensor operation whose output tensor can be initialized and/or updated later.
tf.constant() - Creates a tensor operation whose output tensor is static, not allowed to change.
tf.placeholder() - Creates a tensor operation whose output tensor in unknown, to be provided later.

Note that the above functions can only create individual tensor operations. They are not able to connect tensor operations into tensor flow graphs, because they do not take any input tensors.

To connect tensor operations into tensor flow graphs, we need to create tensor operations that take input tensors. "tensorflow" library does have a large number of functions to create a wide range of tensor operations that takes one or more input tensors. Here are some examples:

tf.sqrt(a) - Creates a tensor operation whose output tensor is calculated by taking the square root on each element of the input tensor.
tf.add(a, b) - Creates a tensor operation whose output tensor is calculated by taking the sum of elements from both input tensors at the same position. This is function is the same as expression (a + b), because the "+" operator is overloaded as tf.add().
tf.multiply(a, b) - Creates a tensor operation whose output tensor is calculated by taking the multiplication of elements from both input tensors at the same position. This is function is the same as expression (a * b), because the "*" operator is overloaded as tf.multiply().
tf.matmul(a, b) - Creates a tensor operation whose output tensor is calculated by taking the dot product (matrix multiplication) of two input tensors.
tf.transpose(a) - Creates a tensor operation whose output tensor is the transposed version of the input tensor.

Now we are ready to create a tensor flow graph with the above two categories of "tensorflow" functions.

First, let's try to implement the tensor flow graph presented earlier in this tutorial using "tensorflow" functions. Here is my version:

import tensorflow as tf

b = tf.Variable([[4,4,4],[4,4,4],[4,4,4]])
c = tf.Variable([[3,3,3],[3,3,3],[3,3,3]])
s = tf.constant([[2,2,2],[2,2,2],[2,2,2]])

t1 = tf.add(b,c)
t2 = tf.add(c,s)

a = tf.multiply(t1,t2)