Cognitive Paradigm Design

For detailed methodology guidance, see the research-literacy skill.

⚠️ Verification Notice

This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please open an issue.

Core Workflow

When given a research question, follow this sequence:

Step 1: Identify the Cognitive Construct

Map the research question to one or more core cognitive domains:

Step 2: Select a Paradigm

Use this decision tree to narrow paradigm choices:

Attention:

• Conflict/interference between dimensions -> Stroop task or Flanker task
• If response-level conflict is key -> Flanker (separates search from conflict)
• If word-reading automaticity is key -> Stroop
• Spatial orienting -> Posner cueing
• Exogenous (reflexive) vs. endogenous (voluntary) -> vary cue type and SOA
• Search efficiency / feature binding -> Visual search
• Temporal limits of attention -> Attentional blink

Memory:

• STM scanning speed -> Sternberg task
• False memory production -> DRM paradigm
• Recollection vs. familiarity -> Remember-Know
• VWM capacity -> Change detection
• Serial position effects -> Serial position curve

Decision Making:

• Decision making under ambiguity with learning -> Iowa Gambling Task
• Impulsivity / temporal discounting -> Delay discounting
• Sensitivity vs. bias decomposition -> Signal Detection Theory (Yes/No or 2AFC)
• Perceptual/cognitive discrimination -> 2AFC

Perception:

• Threshold estimation -> Psychophysical staircase (1-up/2-down or QUEST)
• Few trials available -> QUEST (converges in ~30-50 trials; Watson & Pelli, 1983)
• Simple implementation needed -> 1-up/2-down (converges in ~50-80 trials; Levitt, 1971)
• Subliminal processing / visibility control -> Masking paradigms
• Vary visibility continuously -> backward masking (SOA manipulation)
• Prevent conscious identification -> sandwich masking (forward + backward)

Language:

• Single-word recognition / lexical access -> Lexical decision
• Spreading activation / semantic networks -> Priming (with lexical decision or naming)
• Incremental sentence comprehension -> Self-paced reading or eye-tracking
• Budget-friendly, no specialized equipment -> Self-paced reading
• Maximum ecological validity and rich temporal data -> Eye-tracking

Executive Function:

• Simple response inhibition (withholding) -> Go/No-Go
• Action cancellation (stopping initiated response) -> Stop-signal task
• Need a latent measure of inhibition speed -> Stop-signal (yields SSRT)
• Cognitive flexibility / set shifting -> Task switching
• Working memory updating under continuous load -> N-back

Step 3: Configure Parameters

For each selected paradigm, consult references/classic-paradigms.md for the full parameter reference. Apply these general rules:

Timing Parameters

Trial Counts

Proportion Manipulations

• Congruency proportion (Stroop, Flanker): 50/50 is the unbiased standard. Deviating introduces list-wide proportion congruency effects that modulate conflict (Logan & Zbrodoff, 1979; Bugg & Crump, 2012). Only deviate if proportion effects are the research question.
• Cue validity (Posner): 80% valid for endogenous orienting (Posner, 1980); 50% (uninformative) for pure exogenous effects.
• Stop-signal proportion: 25% is standard. Higher rates induce proactive slowing (Verbruggen et al., 2019).
• Target prevalence (search, detection): 50% unless studying prevalence effects (Wolfe et al., 2005).
• Relatedness proportion (priming): Keep at ~25-50% to minimize strategic expectancy; lower RP isolates automatic priming (Neely et al., 1989).

Step 4: Design Controls

Apply these control procedures:

4.1 Condition Assignment

• Within-subjects preferred for most cognitive paradigms (maximizes power by eliminating between-subject variance; Maxwell & Delaney, 2004)
• Between-subjects required when conditions produce carry-over (e.g., training studies, deception manipulations, proportion manipulations)
• See references/design-principles.md, Section 1 for the full decision framework

4.2 Counterbalancing

• 2-3 conditions: Full counterbalancing (all k! orders)
• 4+ conditions: Balanced Latin Square (Williams, 1949); ensures each condition precedes every other condition equally often
• Always counterbalance: Stimulus-response mappings, response hand assignments
• Pseudo-randomize within blocks: No more than 3-4 consecutive same-condition trials; equal condition transitions (see references/design-principles.md, Section 5.4)

4.3 Practice Trials

• Simple RT tasks: 10-20 practice trials (Luce, 1986)
• Complex tasks (task switching, N-back): 20-40 practice trials with feedback
• Adaptive tasks (staircase): 50-100 familiarization trials before data collection (Watson & Pelli, 1983)
• Require >80% accuracy in practice before advancing
• Use different stimuli from experimental trials

4.4 Catch Trials and Comprehension Checks

• Detection tasks: Include 10-20% no-target catch trials (Posner, 1980)
• Reading tasks: Comprehension probes after 30-50% of sentences (Just et al., 1982)
• Masked priming: Post-experiment visibility check or 5-10% awareness probes (Forster & Davis, 1984)

Step 5: Specify Dependent Variables and Analysis

Primary DVs by Paradigm Type

Recommended Statistical Approaches

• Repeated-measures ANOVA: Standard for factorial designs; check sphericity (Girden, 1992)
• Linear mixed-effects models: Preferred for unbalanced designs, missing data, item-level analysis; include random intercepts for subjects and items ("by-subject and by-item" approach; Baayen et al., 2008)
• Bayesian analysis: Report Bayes factors for key comparisons when sample size is limited or null effects are informative (Rouder et al., 2009)
• Drift-diffusion modeling: For decomposing RT and accuracy into drift rate, boundary separation, and non-decision time (Ratcliff & McKoon, 2008)

Quick Reference: Paradigm Selection Matrix

Domain-Specific Warnings

These are non-obvious pitfalls that require domain expertise:

1. Stroop: Using fewer than 4 color-response mappings introduces item-specific contingency learning that mimics Stroop effects but is not conflict-based (Schmidt & Besner, 2008). Always use >= 4 colors.

2. Stop-signal: Never estimate SSRT from mean Go RT alone. The integration method accounts for the Go RT distribution shape. Failed-stop RTs must be faster than Go RTs (independence assumption check; Logan & Cowan, 1984). Use the consensus guide (Verbruggen et al., 2019).

3. Attentional blink: T1 must be masked (by a trailing distractor). Removing the T1+1 item eliminates the AB entirely (Raymond et al., 1992). Always include T1+1 distractor.

4. Change detection (VWM): Retention intervals shorter than ~900 ms may allow iconic memory to contribute, inflating K estimates. Use >=900 ms retention interval, and consider articulatory suppression to prevent verbal recoding (Luck & Vogel, 1997; Vogel et al., 2001).

5. DRM: False recall varies dramatically across lists (10-60%). Always report which word lists were used and their BAS values (Stadler et al., 1999). Roediger et al. (2001) normed 55 lists.

6. Iowa Gambling Task: Apparent "learning" may reflect frequency-of-loss avoidance rather than long-term value sensitivity. Consider deck-by-deck analysis, not just (C+D)-(A+B) (Steingroever et al., 2013).

7. Priming: High relatedness proportions (>50%) inflate priming through strategic expectancy, not automatic spreading activation. Use RP <= 25% to isolate automatic priming (Neely et al., 1989).

8. Task switching: In alternating-runs designs (AABB), the response-stimulus interval (RSI) is confounded with cue-stimulus interval (CSI). Use cued-switching designs to separate preparation time from passive decay (Monsell, 2003; Meiran, 1996).

9. Psychophysical staircases: Step sizes of <5% lead to staircases that fail to generate enough reversals. Use initial step sizes of at least 10-20% of the expected threshold range, then halve after the first 2-4 reversals (Garcia-Perez, 1998).

10. N-back: Omission errors are more informative than commission errors (unlike Go/No-Go). Always report d' rather than raw accuracy, as d' separates sensitivity from bias (Haatveit et al., 2010). Include lure trials (n-1 or n+1 matches) to assess interference susceptibility (Gray et al., 2003).

References

• Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390-412.
• Baker, D. H., Vilidaite, G., Lygo, F. A., et al. (2021). Power contours: Optimising sample size and precision in experimental psychology and human neuroscience. Psychological Methods, 26, 295-314.
• Brysbaert, M., & Stevens, M. (2018). Power analysis and effect size in mixed effects models. Journal of Cognition, 1(1), 9.
• Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Erlbaum.
• Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50, 1166-1186.
• Macmillan, N. A., & Creelman, C. D. (2005). Detection Theory: A User's Guide (2nd ed.). Erlbaum.
• Verbruggen, F., Aron, A. R., Band, G. P., et al. (2019). A consensus guide to capturing the ability to inhibit actions and impulsive behaviors in the stop-signal task. eLife, 8, e46323.