**Neural Network Tutorials - Herong's Tutorial Examples** - 1.20, by Dr. Herong Yang

Tensor and Tensor Flow Graph

This section provides a tutorial example on how to use 'tensorflow' functions to create a simple tensor flow graph.

In the last tutorial, we have described the key concept, Tensor Flow Graph, used in TensorFlow. Now let's have a loser look at it.

A Tensor Flow Graph, also called a Computational Graph, a Dataflow Graph or TensorFlow Graph, is a graphical representation of an expression of multiple tensor operations. Here is the same tensor flow graph we have looked at before. It represents the tensor operation of [a] = ([b]+[c])*([c]+[2]), if we use [.] to as the tensor notation.

As you can see from the picture, a node in a tensor flow graph represents a single tensor operation, and an edge represents a single tensor flowing from one operation into another operation.

From a programming point of view, a node is really a structured data object (an instance of the "tensorflow" class) that holds the following primary properties:

- The operation that produces an output tensor.
- The dimensional shape of the output tensor.
- The data type of the output tensor elements.
- An ordered list of links to other nodes whose output tensors are used as input tensors to the operation.

If you connect all nodes (or tensor operations) through their links, you will get the final tensor flow graph.

Since an output tensor and the operation that generates the tensor has a one-to-one relation, a tensor operation (or node in a tensor flow graph) is also called a tensor. So the term tensor and the term operation are used interchangeably. If you call an operation as a tensor, you are referring to the tensor generated by the operation. If you call a tensor as an operation, you are referring to the operation that generates the tensor.

There are 3 main "tensorflow" functions that create 3 different flavors of tensor operations (or 3 different flavors of tensors if you refer to their output tensors) with no input tensors.

- tf.Variable() - Creates a tensor operation whose output tensor can be initialized and/or updated later.
- tf.constant() - Creates a tensor operation whose output tensor is static, not allowed to change.
- tf.placeholder() - Creates a tensor operation whose output tensor in unknown, to be provided later.

Note that the above functions can only create individual tensor operations. They are not able to connect tensor operations into tensor flow graphs, because they do not take any input tensors.

To connect tensor operations into tensor flow graphs, we need to create tensor operations that take input tensors. "tensorflow" library does have a large number of functions to create a wide range of tensor operations that takes one or more input tensors. Here are some examples:

- tf.sqrt(a) - Creates a tensor operation whose output tensor is calculated by taking the square root on each element of the input tensor.
- tf.add(a, b) - Creates a tensor operation whose output tensor is calculated by taking the sum of elements from both input tensors at the same position. This is function is the same as expression (a + b), because the "+" operator is overloaded as tf.add().
- tf.multiply(a, b) - Creates a tensor operation whose output tensor is calculated by taking the multiplication of elements from both input tensors at the same position. This is function is the same as expression (a * b), because the "*" operator is overloaded as tf.multiply().
- tf.matmul(a, b) - Creates a tensor operation whose output tensor is calculated by taking the dot product (matrix multiplication) of two input tensors.
- tf.transpose(a) - Creates a tensor operation whose output tensor is the transposed version of the input tensor.

Now we are ready to create a tensor flow graph with the above two categories of "tensorflow" functions.

First, let's try to implement the tensor flow graph presented earlier in this tutorial using "tensorflow" functions. Here is my version:

import tensorflow as tf b = tf.Variable([[4,4,4],[4,4,4],[4,4,4]]) c = tf.Variable([[3,3,3],[3,3,3],[3,3,3]]) s = tf.constant([[2,2,2],[2,2,2],[2,2,2]]) t1 = tf.add(b,c) t2 = tf.add(c,s) a = tf.multiply(t1,t2)

So easy, right? We can even update the graph diagram with tensor operation nodes labeled by their "tensorflow" function names.

Now, you should be able to create your own version of the same tensor flow graph, or other simple graphs.

Table of Contents

Deep Playground for Classical Neural Networks

Building Neural Networks with Python

Simple Example of Neural Networks

►TensorFlow - Machine Learning Platform

"tensorflow" - TensorFlow Python Library

"tensorflow" Interactive Test Web Page

TensorFlow Session Class and run() Function

TensorFlow Variable Class and load() Function

Linear Regression with TensorFlow

tensorflow.examples.tutorials.mnist Module

Simple TensorFlow Model on MNIST Database

Commonly Used TensorFlow Funcitons

PyTorch - Machine Learning Platform

CNN (Convolutional Neural Network)