16.9 Use
of Domain-Dependent Prior Information
The methods considered so far are mostly numerical
techniques that make no use of problem-specific information. Another powerful
way of favoring good generalization is through the use of domain-dependent prior
information.
As noted earlier, samples alone are not enough to uniquely specify
the target function in the absence of other constraints. In many applications
where neural nets are considered, there is significant human knowledge that
could be useful even though it is incomplete or only partially reliable. There
may be existing techniques that give reasonable but imperfect solutions or we
may know certain rules that should be satisfied by any correct solution. When
the goal is to develop a working application, it makes sense to use as much of
this information as possible.
The following sections review some ways of using
domain-dependent prior information in a neural network. Some are based on the
idea of adapting a good non-neural solution to provide the starting point for
further fine tuning in a neural network structure. It should be noted that
whether or not this leads to good generalization depends on many factors;in some
cases it may merely accelerate learning by giving the network a good headstart,
without really improving generalization.