Interacting neural networks: an architecture for modelling distributed parameter dynamical systems

Abstract

The development of models of dynamical systems behaviour is a fundamental activity in science and engineering disciplines. This thesis examines the problem of modelling a class of dynamical systems using neural networks. Existing research reveals that neural network models have been developed for lumped parameter dynamical systems; that is, systems where the variables of interest vary only over the timedomain. However, there are no adequate neural network models for distributed parameter dynamical systems; that is, systems where the variables of interest vary over some other domain, e.g. the spatial
domain, in addition to the time domaln. The main goal of this research is to develop a neural network architecture for modelling distributed dynamical systems where one has limited and incomplete knowledge about the underlying behaviour of the system. The result of this research is a generic neural network architecture - the Interacting Neural Network (INN) architecture - that is capable of modelling a wide range of distributed dynamical systems. The fundamental problem associated with distributed systems which the INN architecture addresses is that of scaling. The scaling problem manifests itself when the complexity of a model increases in a manner which is unmanageable as the problem size increases. The INN architecture solves the scaling problem by using the philosophy of interacting subsystems which is a general methodology for managing complexity. The underlying principle of this methodology is to view the system as a combination of many
small subsystems and to focus the modelling effort at the subsystem level rather than at the system level. The resulting models are relatively simpler, but when allowed to interact, the complex behaviour of the original system can be retrieved.
The capabilities of the INN architecture are investigated by comparing its performance with other architectures on two distributed systems. First, investigations are carried out in modelling non-linear heat flow which serves as a case study to expound the capabilities of the INN architecture. Secondly, the architecture is applied to an aquifer problem to illustrate its capabilities on modelling practical problems. It is shown that the INN architecture captures the underlying behaviour of both systems, and more significantly, that the trained network can generalise spatially, wherein the same trained network can be
applied to different instances of a given system. The spatial generalisation capabilities of the INN architecture is a unique and powerful result, which when used appropriately can significantly extend the usefulness of neural network models.
Finally, two major factors that affect the generalisation ability of the INN architecture are investigated: (i) the effect of changing the geometry of a given system and (ii) the effect of the amount of training data available. New relationships are deduced for both factors.