Methodology

The survey begins with four colour-vision screening plates and one unrecorded practice question. It then presents 29 measured figure questions covering the design comparisons shown below.

Factorial cluster — Channel Family Fit × Channel Accuracy

Crosses Channel Family Fit (are the six arms kept as separate categories, or forced into a part-to-whole stack?) with Channel Accuracy (is the rate read off position -- the most accurate channel -- or area, one of the least?). Caution when interpreting the interaction: the two “fit = baseline” cells below use different violations of expressiveness -- a false part-to-whole stack (top-left) vs. a false trend implied by a connected line (top-right) -- to keep each cell's own accuracy channel consistent. A significant interaction term may reflect that these are different kinds of mistakes, not a clean fit×accuracy effect.

Channel Accuracy: baseline Channel Accuracy: treatment
Channel Family Fit: baseline

Stacked bar: forced part-to-whole framing (fit violated). Segment length is still a fairly good channel, but stacking undermines the comparison.

tokens
channel_accuracy = baseline, channel_family_fit = baseline
viewing
normal
prompt
Which arm has the LOWEST adverse-event rate?
options
Control, A, B, C, D, E, Skip
answer
E
pool
factorial_cluster / arms / normal / channel_accuracy=baseline, channel_family_fit=baseline
draw
factorial_cluster:arms:normal:channel_accuracy=baseline,channel_family_fit=baseline · variant 0

Connected line across unordered arms: implies a false trend between categories (fit violated), even though position is the most accurate channel (accuracy correct).

tokens
channel_accuracy = treatment, channel_family_fit = baseline
viewing
normal
prompt
Which arm has the LOWEST adverse-event rate?
options
Control, A, B, C, D, E, Skip
answer
C
pool
factorial_cluster / arms / normal / channel_accuracy=treatment, channel_family_fit=baseline
draw
factorial_cluster:arms:normal:channel_accuracy=treatment,channel_family_fit=baseline · variant 0
Channel Family Fit: treatment

Bubble chart: categories kept separate (fit correct), but circle area is a low-accuracy channel (accuracy violated).

tokens
channel_accuracy = baseline, channel_family_fit = treatment
viewing
normal
prompt
Which arm has the LOWEST adverse-event rate?
options
Control, A, B, C, D, E, Skip
answer
Control
pool
factorial_cluster / arms / normal / channel_accuracy=baseline, channel_family_fit=treatment
draw
factorial_cluster:arms:normal:channel_accuracy=baseline,channel_family_fit=treatment · variant 0

Sorted, separated bars: categories kept separate and position/length used to encode the rate (both correct).

tokens
channel_accuracy = treatment, channel_family_fit = treatment
viewing
normal
prompt
Which arm has the LOWEST adverse-event rate?
options
Control, A, B, C, D, E, Skip
answer
B
pool
factorial_cluster / arms / normal / channel_accuracy=treatment, channel_family_fit=treatment
draw
factorial_cluster:arms:normal:channel_accuracy=treatment,channel_family_fit=treatment · variant 0