We generalize a class of neural network models that extend the Kohonen Self-Organising Map (SOM) algorithm into the sequential and temporal domain using recurrent connections. Behaviour of the class of Activation-based Recursive Self-Organising Maps (ARSOM) is discussed with respect to the choice of transfer function and parameter settings. By comparing performances to existing benchmarks we demonstrate the robustness and systematicity of the ARSOM models, thus opening the door to practical applications.