16.10
Hint Functions
One way to provide additional constraints is through the use
of "hints" [361], [416]. In addition to outputs for the function of interest,
extra output nodes are added to the network and trained to learn certain hint
functions. The hint functions should be related to the function of interest and
are usually designed to be easier to learn. The extra functions may speed
convergence by generating nonzero derivatives in regions where the original
function has plateaued. They may also aid generalization by providing additional
constraints and removing certain local minima of the original function. They
discourage the choice of a solution that somehow matches the original function
on the training samples but does not include intermediate concepts embedded in
the hints. After training, the hint output nodes can be removed because they
usually are not of interest in the overall system.
The term hints is usually used to refer to augmented outputs,
but hint information can also be provided in the form of targets for the
(normally) hidden nodes. Hints can also be provided by shaping the target
function dynamically [193]. The initial target function is an easy to learn, coarse
approximation of the desired function which is gradually made more similar to
the desired function as the learner masters each stage. This is a standard
technique in animal training.