This chapter collects a number of early papers in which neural networks are trained to be sequence processors. The notions of sequence or time are consubstantial to the concept of computation as a sequential behavior, and indeed, the question explored by papers in this chapter is the second ``main question'' mentioned in the introduction, that is, ``what can a neural network learn to compute?'', in terms of sequence processing and recognition.
The introductory material discusses briefly what is sequence processing (section 3.1); then, the following sections illustrate how neural networks may be used as sequence processors. The use of neural networks for sequence processing tasks has a very important advantage: neural networks are adaptive devices that may be trained to perform sequence processing tasks from examples. Section 3.2 gives a general introduction to a kind of neural networks which is very relevant to sequence processing, namely, discrete-time recurrent neural networks, under the paradigm of ``neural state machines''; section 3.3 briefly reviews some applications of recurrent neural networks to some real-world sequence processing tasks; section 3.4 gives an outline of the main learning (also training) algorithms; and the learning problems that may be observed are discussed in section 3.5. Finally, a brief introduction to each one of the featured papers is given in section 3.6