**Neural Network Tutorials - Herong's Tutorial Examples** - 1.20, by Dr. Herong Yang

Impact of Extra Input Features

This section provides a tutorial example to show the impact of one extra input feature on a simple neural network model created on Deep Playground.

After learned how to use Deep Playground by building a simple neural network model in the previous tutorial, let's look the impact of an extra input feature in this tutorial.

1. Continue withe previous tutorial of a 1-layer model with linear activation function on the linear classification problem.

2. Click "X_{1}X_{2}" in the FEATURES list to add
x_{1}*x_{2} as the third input component in the model.

3. Click "reset" icon repeatedly until you see a poor initial weight matrix
by looking at the output area as shown below.
The initial weight matrix has a large positive value on the
x_{1}*x_{2} component, so the output
is heavily impacted by the X_{1}X_{2} input feature.

4. Click the "play one epoch" icon, the model improves quickly by
updating the weight matrix with a smaller value
on the x_{1}*x_{2} component to reduce its impact
on the output. Obviously, this extra input feature is bad for
solving this linear problem.

5. Click the "play one epoch" icon repeatedly, until the model is stabilized (when "training loss" become 0.000). After this point, playing more training epochs will not improve the model any more, since there is almost no training loss to generate changes on the weight matrix.

The picture below shows a case, in which 15 training epoch is enough to stablize the model providing an arc-like solution to our linear problem. This is acceptable because our samples are concentrated in 2 local areas only. Any line, curved or straight, is a valid solution as long as it can separate the 2 sample areas.

If you reset the model with different initial weight matrices
and train it again, the stabilized state will be close to the
one shown above.
It has two equal positive weights on X_{1} and X_{2} input features.
The extra input feature, X_{1}X_{2},
is almost wiped out by a very small weight, represented by the
the thin and grey link.

This suggests that the model with X_{1}, X_{2}
and X_{1}X_{2} input features has only 1 solution.

You can continue to play with different extra input features or add all input features to see how the model behaves.

Table of Contents

►Deep Playground for Classical Neural Networks

►Impact of Extra Input Features

Impact of Additional Hidden Layers and Neurons

Impact of Neural Network Configuration

Impact of Activation Functions

Building Neural Networks with Python

Simple Example of Neural Networks

TensorFlow - Machine Learning Platform

PyTorch - Machine Learning Platform

CNN (Convolutional Neural Network)