Jordan (1986):

Jordan's paper, ``Serial Order: A Parallel Distributed Processing Approach'', describes how a particular recurrent neural network architecture, described in page 3.2.1, which almost everyone calls a Jordan net now, may be trained to generate sequences of vectors corresponding to various tasks. Inputs to Jordan's nets are constant during the generation of a whole sequence and are aptly called the ``plan'' for that sequence: the net is taught to produce a different sequence for each plan.

State in Jordan nets is not hidden but fully observable (what Jordan (1986) calls hidden units are used as an intermediate step for the computation of the output, see eq. 3.12). Therefore, it is possible to train them using teacher forcing (see section 3.4.3). Jordan finds that teacher forcing favors fast learning.

Jordan also studies the behavior of trained DTRNN when their state is perturbed and observes that learned sequences are attractors for the DTRNN. For example, if the DTRNN has been trained to produce an oscillating output pattern, it is attracted towards the corresponding limit cycle of the network. When the vectors in the desired output sequence are not completely specified but only some of their components have target values, Jordan shows that the use of ``don't care'' conditions for the remaining coordinates favors the learning of ``anticipation'' behaviors: components not specified during learning tend to adopt values which approach the specified value for the component in the closest future vector of the sequence. The results are discussed in the context of the simulation of coarticulatory features of speech, that is, inter-phoneme influences. Finally, Jordan also studies the modelling of dual tasks, that is, tasks that are learned separately but have to be performed simultaneously.