Not all DTRNN architectures are capable
of representing all FSMFor example, as discussed in
section 4.2.1, Elman (1990) neural nets
can emulate any FSM using
state neurons
using a step function (see Kremer (1995)). As will be seen, Elman
nets using
sigmoids over rational numbers (Alquézar and Sanfeliu, 1995) may also be used to
implement FSM (see
the paper by Alquézar and Sanfeliu (1995)) ; more recently, Carrasco et al. (2000) have shown that
this is also the case for Elman nets using real
sigmoids.
On the other hand,
first-order DTRNN without an output
layer which use as output a
projection of the state vector, that is,
![]() |
(5.8) |
Finally, it is easy to show (using a
suitable odd-parity counterexample
in the way described by Goudreau et al. (1994))
that the networks by Robinson and Fallside (1991) networks (see section 3.2.1)
cannot represent the output function of all Mealy machines unless
a two-layer scheme as the following is used:
Another interesting example is that of Fahlman's recurrent cascade correlation networks, which are constructed by an algorithm having the same name (see section 3.4.3). Kremer (1996b) --see the following section-- has shown these networks cannot represent all FSM by defining suitable classes of nonrepresentable machines.