Parallel implementation of backpropagation neural networks : a study of network-based parallelism

Abstract

Artificial neural networks have applications in many fields ranging from medicine to image processing. One of the most popular neural network architecture and learning algorithm is the multi-layer feedforward architecture where the Backpropagation (BP) learning scheme is used. Although the BP algorithm is popular, training takes a very long time for large neural networks with a large training set. Training can be sped by parallelising the BP algorithm on a parallel machine. This thesis presents a detailed study of network-based parallelisation of the BP algorithm on message passing multi-computers. In this scheme, the neural network is vertically sliced and distributed among the processing elements connected in a ring topology.