Artificial Neural Networks

Artificial Neural Networks (ANNs) are algorithmic models inspired by brain cells, designed to transform input data into output data.

One of the most popular ANN algorithms, Backpropagation, consists of an input layer, one or more hidden layers, and an output layer. Each node, or artificial neuron, connects to others with associated weights and thresholds. If a node’s output exceeds its threshold, it activates and transmits data to the next layer; otherwise, it does not.

The input layer receives the data fed into the network. In the hidden layers, the weights of the connections encode the information learned from the training data. The output layer gathers the predictions from the hidden layers and computes the model’s final prediction.

Initially, connections between nodes have random weights. Training involves iteratively adjusting these weights by processing a set of training examples and comparing the network’s predictions to the correct labels. When discrepancies arise, the weights are adjusted. This adjustment process occurs backward, from the output layer through each hidden layer to the input layer, hence the name “backpropagation.” Although convergence is not guaranteed, the weights typically stabilize, marking the end of the learning process. These adjusted weights, which range between 0 and 1 or -1 and 1, represent the knowledge the neural network has acquired. This encapsulates the essence of artificial neural networks.

The input to individual neural network nodes must be numeric and fall within the closed interval range of [0,1] or [-1,1]. This requires normalizing the inputs for each attribute from the training examples. Discrete-valued attributes can be encoded with one input unit per domain value. For numerical data, the normalization formula is:

Normalized Value = (Value – MIN) / (MAX – MIN)

where MIN is the smallest value in the dataset and MAX is the highest.

The Backpropagation algorithm is a widely used method for training supervised artificial neural networks. Initially, the network’s connections are established with randomly generated weights, typically between 0 and 1. The training process is iterative, involving the presentation of training examples to the network. During each iteration, a labeled example is fed into the input layer, and the network’s output is computed through forward propagation, which involves calculations through the hidden layers to produce the final output. The algorithm then compares this output to the expected results or target values. When discrepancies occur, the backpropagation algorithm applies an error-correction procedure by tracing back through the hidden layers to the input layer, adjusting the network’s weights to minimize the error.

This iterative process continues until the network’s performance improves and the desired accuracy is achieved. Although success is not guaranteed, the process is repeated through numerous cycles until either the weights converge, allowing the neural network to correctly evaluate all test samples, or the error falls within an acceptable threshold. Essentially, what a neural network “learns” is a collection of numeric values between 0 and 1 (the adjusted weights), encapsulating the essence of an artificial neural network.