Long-term dependencies:

The problem of long-term dependencies, or the problems when training DTRNN to perform tasks in which a late output depends on a very early input which has to be remembered, is more specific to DTRNN, because it is a sequence-processing problem; one of the most exhaustive studies of this problem has been done by Bengio et al. (1994) (see (Haykin, 1998, 773)). See section 3.6 for a discussion.