Comparing Models

By what criteria do we evaluate which of a set of competing models is more likely to be correct? Is it possible for two different models of the same phenomenon to both be correct, and what does this mean for model building efforts?


Parameters of comparison

There are a number of possible ways of evaluating models:


The Word Superiority Effect: A comparison of two models

Richman & Simon (1989) undertook to compare the performance of two models on modeling the word superiority effect: EPAM and the Interactive Activation model (IAM).

IAM is a connectionist model based on the pandemonium structure, with top-down feedback between the word and letter levels of the model.

EPAM is a symbolic processing model that assumes high-level cognitive processes are serial in nature.


EPAM

Uses a discrimination net (d-net) to organize the chunks (lists) of information it knows about.

A d-net is a tree structure where a particular node in the tree represents a test for some feature of the input list. Leaves of the tree contain the chunks of information already stored.

In EPAM, a list of information is used as the input to the d-net. The net performs tests on the input list that cause it to be sorted down a particular branch of the tree.

Once the point is reached where the list can no longer be sorted, one of two things happens. Either the sorting process has reached a leaf node containing an already known chunk of information, in which case the concept represented by the list has been identified, or else you have a new concept that needs to be added to the list.


EPAM and the WSE


The Data

Ineffectiveness of letter constraints:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Serial position effect:

Word frequency effects:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Word superiority effect: