International Encyclopedia of Systems and Cybernetics

Already in 1957, J.von NEUMANN distinguished clearly sequential and parallel computers (at a time nobody had yet dreamed of connection machines), His model of a parallel "computer" was the brain, In his own words: "& the natural componentry favors automata with more, but slower, organs, while the artificial one favors the reverse arrangement of fewer, but faster, organs. Hence it is to be expected that an efficient organized large natural automaton (like the human nervous system) will tend to pick up as many logical (or informational) items as possible simultaneously, while an efficiently organized large artificial automaton (like a large modern computing machine) will be more likely to do things successively – one thing at a time, or at any rate not so many things at a time. That is, large and efficient natural automata are likely to be highly parallel, while large and efficient artificial automata will tend to be less so, and rather to be serial.

"… it should be noted, however, that parallel and serial operation are not unrestrictly substitutable for each other". More specifically, not everything serial can be immediately paralleled – certain operations can only be performed after certain others, and not simultaneously with them (i.e. they must use the results of the latter). In such a case, the transition from a serial scheme to a parallel one may be impossible, or it may be possible but only concurrently with a change in the logical approach and organization of the procedure. Conversely, the desire to serialize a parallel procedure may impose new requirements, since the results of the operations that are performed first must be stored while the operations that come after these are performed. Hence the logical approach and structure in natural automata may be expected to differ widely from those in artificial automata. Also, it is likely that the memory requirements of the latter will turn out to be systematically more severe than those of the former" (1958, p.50-52).

In effect, the parallel automaton – natural or artificial – does progressively construct algoritms through the application of a set of rules in its network, while the serial, i.e. sequential one is based on a previously introduced deterministic algorithm, which may of course be quite complex.

The treatment of complex networks models by sequential computers is inevitably distorted, as for example in the case of FORRESTER's systems dynamics.

R.W. FULLER and P. PUTNAM clarified still more this matter: "The parallel calculator functions to pick up correlations, to sift for and to isolate significant new correlations, while the series calculator is designed to help establish the consequences of correlations already well defined (Like NEWTON's laws)" (1967, p.99).