**Neural Network Tutorials - Herong's Tutorial Examples** - 1.20, by Dr. Herong Yang

Commonly Used TensorFlow Funcitons

This section describes some commonly used TensorFlow functions, including add(), multiply(), matmul(), tf.reduce_sum(), tf.nn.softmax(), etc.

As a quick reference, here is a list of commonly used TensorFlow functions provided by different modules and classes

1. Functions provided by the "tensorflow" module itself:

import tensorflow as tf

tf.add(a, b) - Creates a tensor operation whose output tensor is calculated by taking the sum of elements from both input tensors at the same position. This is function is the same as expression (a + b), because the "+" operator is overloaded as tf.add().

tf.cast(a, t) - Creates a tensor operation whose output tensor is the input tensor, a, with data type changed to the given type, t.

tf.constant(cons) - Creates a tensor operation with an output tensor fixed to the given value.

tf.equal(a, b) - Creates a tensor operation with an output tensor of scalar Boolean true or false, if both input tensors are equal.

tf.exp(a) - Creates a tensor operation whose output tensor is calculated by applying Exp() on each element of the input tensor.

tf.matmul(a, b) - Creates a tensor operation whose output tensor is calculated by taking the dot product (matrix multiplication) of two input tensors.

tf.multiply(a, b) - Creates a tensor operation whose output tensor is calculated by taking the multiplication of elements from both input tensors at the same position. This is function is the same as expression (a * b), because the "*" operator is overloaded as tf.multiply().

tf.ones(s) - Creates a "constant" tensor operation whose output tensor is filled with 1 in the given dimension shape, s.

tf.one_hot(i, l, dtype=t) - Creates a "constant" tensor operation whose output tensor is filled with one-hot values of 1 on a list of given index locations, i, Each index is index location is expanded into a vector of length, l.

tf.reduce_mean(a, l) - Creates a tensor operation whose output tensor is the input tensor, a, with the given list of dimensions, l, reduced to its means (average values). If l is not specified, all dimensions are reduced and the output is a scalar.

tf.reduce_sum(a, l) - Creates a tensor operation whose output tensor is the input tensor, a, with the given list of dimensions, l, reduced to their sums. If l is not specified, all dimensions are reduced and the output is a scalar.

tf.sqrt(a) - Creates a tensor operation whose output tensor is calculated by taking the square root on each element of the input tensor.

tf.transpose(a) - Creates a tensor operation whose output tensor is the transposed version of the input tensor.

tf.variables_initializer(l) - Creates a special tensor operation whose job is to load initial values on the given list of "variable" tensors.

tf.zeros(s) - Creates a "constant" tensor operation whose output tensor is filled with 0 in the given dimension shape, s.

tf.nn.relu(a) - Creates a tensor operation whose output is calculated by applying the ReLU() activation function on each element in the input tensor

tf.nn.softmax(a, l) - Creates a tensor operation whose output is calculated by applying the Softmax() activation function on the input tensor, a, in the given list of dimensions, l. If l is not specified, the last dimension is used.

session = tf.Session() - Returns a tensor session object to be used to run (evaluate) any tensor expression represented as a tensor operation.

session.run(o, feed_dict=l) - Runs (evaluates) the given list of tensor operations, o, with the list of placeholders, l, to be loaded with specified values.

optimizer = tf.train.GradientDescentOptimizer(r) - Returns an optimizer object representing the Gradient Descent algorithm with the given learning rate, r.

optimizer.minimize(cost, var_list=l) - Creates a special tensor operation whose job is to update the given list of "variable" tensors using "optimizer" algorithm.

saver = tf.train.Saver() - Returns a saver object which can be used to save the trained model in a session into files.

saver.save(session, name) - Saves the trained model in the given session into files.

Table of Contents

Deep Playground for Classical Neural Networks

Building Neural Networks with Python

Simple Example of Neural Networks

►TensorFlow - Machine Learning Platform

"tensorflow" - TensorFlow Python Library

"tensorflow" Interactive Test Web Page

TensorFlow Session Class and run() Function

TensorFlow Variable Class and load() Function

Linear Regression with TensorFlow

tensorflow.examples.tutorials.mnist Module

Simple TensorFlow Model on MNIST Database

►Commonly Used TensorFlow Funcitons

PyTorch - Machine Learning Platform

CNN (Convolutional Neural Network)