Does the “signed-max” model prediction for emergence of high-confidence errors in visual clutter justify the results of Graziano & Sigman study?

In the ‘visual clutter paper’, Stefano Baldassi is trying to verify his “signed-max” model by testing the model’s prediction about the distribution of confidence reports in near-threshold multiple-interval discrimination tasks. In a task where the observer is supposed to monitor multiple stimuli and make a decision about a signal that is embedded in one [undetermined] stimuli, in addition to the noisy representation of signal itself, all stimuli contribute to the noise in the evidence-representation axis. The mechanism by which the observer accumulates these noisy evidences is controversial. The Baldassi’s “signed-max” model beat the simplistic “sum” model in simulation analyses (check this paper). The “signed-max” model assumes that all stimuli are treated equally and each produce a piece of evidence (=signal) and visual system chooses the most extreme of noisy signals as the target. This model predicts a binominal probability density function for evidence representations. And as the number of intervals increases the probability that a distractor would generate larger representation than the signal increases, hence performance decreases.

Unlike the Gaussian distribution of noise in the familiar one-interval 2AFC task, the binominal noise of “signed-max” model predicts that in higher set-sizes most errors are due to evidence representations far away from the 0[= neutral] on evidence axis. Hence in multiple interval tasks more errors would be high-confidence errors (also there will be more high confidence corrects). This is just what they confirmed behaviorally in the “clutter” paper.

In the “construction of confidence” paper Graziano & Sigman report the confidence ratings in a ‘partial reporting’ experiment. What they find is that a large portion of errors were high-confidence errors. They investigate the rationale behind these high-confidence errors and discuss that it is due to location-misattribution of correctly identified targets (a process reminiscent of ‘Binding problem’ studies where in short presentation times separate perceptual attributes of stimuli [e.g. shape, color, location] are perceived but does not integrate to form a unified percept of each object).

Assessment of current results based on “signed-max” model is impeded by several challenges:

1. The evidence in ‘partial report’ task is two-dimensional (letter identity and location) while the “signed-max” is based on one evidence axis (the direction of tilt).
2. Performance in ‘partial report’ is believe to rely on working memory while orientation discrimination is [hypothetically] based on primordial physiologic processes (V1 cells). The memory mechanisms is located down-stream to sensory perception circuits and are amenable to much more confounding factors.
3. The signature pattern of binominal evidence in “signed-max” is defined for the near-threshold signal strength (P(c) = 75%) where the signal is just noticeable and noise perturbations largely shape the accumulative pattern of decision-maker’s behavior. While the performance in ‘partial report’ is much lower (P(c) = 30%).
4. And a big contradiction: “signed-max” assumes that all stimuli are monitored independently, while “partial report” results clearly show that observers monitor some spatial locations at the sake of others.

All these obstacles make it unreasonably hard to do a mathematically plausible generalization of model. However, by subjective inspection an uncanny resemblance is seen between the bimodal distribution of confidence ratings in Graziano & Sigman paper and the bimodal distribution of evidence probability that “signed-max” model proposes for large set sizes.

But, and this is a big but, these graph have different x-axes, and we know that the confidence distribution is a derivative of the evidence distribution. Monte Carlo simulations (see above) show that error and correct representations peak at somewhat similar distances from 0, and this suggests that the confidence distribution from this evidence set would be a unimodal one. Moreover “signed-max” model predict that the ratio of high-confidence corrects and high-confidence errors both increase at larger set-sizes. In fact the increase in set size induces a bias in confidence reports rather than affecting the metacognition in any substantial way. This interpretation is also evident in this excerpt from the paper itself:

If the decision-maker has to monitor many events […] there will be a high probability that the decision, whether right or wrong, will be made with high confidence. […] as American presidential candidate John Kerry mentioned in the first 2004 debate: ‘‘It’s one thing to be certain, but you can be certain, and you can be wrong. Certainty sometimes can get you into trouble.’’

At the end let me state that I believe that the two-dimensional nature of the ‘partial report’ task underlies the formation of high-confidence errors. Low ‘feature-binding’ efficiency at short time-scales in combination with ‘retinotopic spatial bias’ can lead to many high-confidence errors but not as much high-confidence corrects. I’m not confident though!