Ideation

Use them individually or in combination: cognitive frameworks to generate raw insight, structured brainstorming to evaluate and rank, interactive exploration for real-time dialogue.

Mode 1: Structured Brainstorming

Ten complementary ideation lenses. Each targets a different cognitive mode.

1. Problem-First vs. Solution-First Thinking

Research ideas originate from two distinct modes. Knowing which mode you are in prevents a common failure: building solutions that lack real problems, or chasing problems without feasible approaches.

Problem-First (pain point -> method):

• Start with a concrete failure, bottleneck, or unmet need
• Naturally yields impactful work because the motivation is intrinsic
• Risk: may converge on incremental fixes rather than paradigm shifts

Solution-First (new capability -> application):

• Start with a new tool, insight, or technique seeking application
• Often drives breakthroughs by unlocking previously impossible approaches
• Risk: "hammer looking for a nail" -- solution may lack genuine demand

Workflow:

1. Write down your idea in one sentence
2. Classify it: Is this problem-first or solution-first?
3. If problem-first -> verify the problem matters (who suffers? how much?)
4. If solution-first -> identify at least two genuine problems it addresses
5. For either mode, articulate the gap: what cannot be done today that this enables?

Self-Check:

• Can I name a specific person or community who needs this?
• Is the problem I am solving actually unsolved (not just under-marketed)?
• If solution-first, does the solution create new capability or just replicate existing ones?

2. The Abstraction Ladder

Every research problem sits at a particular level of abstraction. Deliberately moving up or down the ladder reveals ideas invisible at your current level.

Workflow:

1. State your current research focus in one sentence
2. Move UP: What is the general principle behind this? What class of problems does this belong to?
3. Move DOWN: What is the most specific, constrained instance? What happens at the extreme?
4. Move SIDEWAYS: Where else does this pattern appear in a different field?
5. For each new level, ask: Is this a publishable contribution on its own?

Example:

• Current: "Improving retrieval accuracy for RAG systems"
• Up: "What makes context selection effective for any augmented generation system?"
• Down: "How does retrieval accuracy degrade when documents are adversarially perturbed?"
• Sideways: "Database query optimization uses similar relevance ranking -- what can we borrow?"

3. Tension and Contradiction Hunting

Breakthroughs often come from resolving tensions between widely accepted but seemingly conflicting goals.

Common Research Tensions:

Workflow:

1. Pick your research area
2. List the top 3-5 desiderata (things everyone wants)
3. Identify pairs that are commonly treated as trade-offs
4. For each pair, ask: Is this trade-off fundamental or an artifact of current methods?
5. If artifact -> the reconciliation IS your research contribution
6. If fundamental -> characterizing the Pareto frontier is itself valuable

4. Cross-Pollination (Analogy Transfer)

Borrowing structural ideas from other disciplines is one of the most generative research heuristics.

Requirements for a Valid Analogy:

• Structural fidelity: The mapping must hold at the level of underlying mechanisms, not just surface similarity
• Non-obvious connection: If the link is well-known, the novelty is gone
• Testable predictions: The analogy should generate concrete hypotheses

High-Yield Source Fields for ML Research:

Workflow:

1. Describe your problem in domain-agnostic language (strip the jargon)
2. Ask: What other field solves a structurally similar problem?
3. Study that field's solution at the mechanism level
4. Map the solution back to your domain, preserving structural relationships
5. Generate testable predictions from the analogy
6. Validate: Does the borrowed idea actually improve outcomes?

5. The "What Changed?" Principle

Strong ideas often come from revisiting old problems under new conditions.

Categories of Change to Monitor:

Workflow:

1. Pick a well-known negative result or abandoned approach (3-10 years old)
2. List the assumptions that led to its rejection
3. For each assumption, ask: Is this still true today?
4. If any assumption has been invalidated -> re-run the idea under new conditions
5. Frame the contribution: "X was previously impractical because Y, but Z has changed"

6. Failure Analysis and Boundary Probing

Understanding where a method breaks is often as valuable as showing where it works.

Types of Boundaries to Probe:

• Distributional: What happens with out-of-distribution inputs?
• Scale: Does the method degrade at 10x or 0.1x the typical scale?
• Adversarial: Can the method be deliberately broken?
• Compositional: Does performance hold when combining multiple capabilities?
• Temporal: Does the method degrade over time (concept drift)?

Workflow:

1. Select a widely-used method with strong reported results
2. Identify the implicit assumptions in its evaluation (dataset, scale, domain)
3. Systematically violate each assumption
4. Document where and how the method breaks
5. Diagnose the root cause of each failure
6. Propose a fix or explain why the failure is fundamental

7. The Simplicity Test

Before accepting complexity, ask whether a simpler approach suffices.

Warning Signs of Unnecessary Complexity:

• The method has many hyperparameters with narrow optimal ranges
• Ablations show most components contribute marginally
• A simple baseline was never properly tuned or evaluated
• The improvement over baselines is within noise on most benchmarks

Workflow:

1. Identify the current SOTA method for your problem
2. Strip it to its simplest possible core (what is the one key idea?)
3. Build that minimal version with careful engineering
4. Compare fairly: same compute budget, same tuning effort
5. If the gap is small -> the contribution is the simplicity itself
6. If the gap is large -> you now understand what the complexity buys

8. Stakeholder Rotation

Viewing a system from multiple perspectives reveals distinct classes of research questions.

Workflow:

1. Describe your system or method in one paragraph
2. Assume each stakeholder perspective in turn (spend 5 minutes per role)
3. For each perspective, list the top 3 concerns or questions
4. Identify which concerns are unaddressed by existing work
5. The unaddressed concern with the broadest impact is your research question

9. Composition and Decomposition

Novelty often emerges from recombination or modularization.

Composition (combining existing techniques):

• Identify two methods that solve complementary subproblems
• Ask: What emergent capability arises from combining them?
• Example: RAG + Chain-of-Thought -> retrieval-augmented reasoning

Decomposition (breaking apart monolithic systems):

• Identify a complex system with entangled components
• Ask: Which component is the actual bottleneck?
• Example: Decomposing "fine-tuning" into data selection, optimization, and regularization reveals that data selection often matters most

10. The "Explain It to Someone" Test

A strong research idea should be defensible in two sentences to a smart non-expert.

The Two-Sentence Template:

Sentence 1 (Problem): "[Domain] currently struggles with [specific problem], which matters because [concrete consequence]." Sentence 2 (Insight): "We [approach] by [key mechanism], which works because [reason]."

If You Cannot Fill This Template:

• The problem may not be well-defined yet -> return to Lens 1
• The insight may not be clear yet -> return to Lens 7 (simplify)
• The significance may not be established -> return to Lens 3 (find the tension)

Integrated Brainstorming Workflow

Phase 1: Diverge (Generate Candidates)

Goal: Produce 10-20 candidate ideas without filtering.

1. Scan for tensions (Lens 3): List 5 trade-offs in your field
2. Check what changed (Lens 5): List 3 recent shifts (compute, data, regulation)
3. Probe boundaries (Lens 6): Pick 2 popular methods and find where they break
4. Cross-pollinate (Lens 4): Pick 1 idea from an adjacent field
5. Compose/decompose (Lens 9): Combine 2 existing techniques or split 1 apart
6. Climb the abstraction ladder (Lens 2): For each candidate, generate up/down/sideways variants

Phase 2: Converge (Filter and Rank)

Goal: Narrow to 3-5 strongest ideas.

Phase 3: Refine (Sharpen the Winner)

Goal: Turn the top idea into a concrete research plan.

1. Write the two-sentence pitch (Lens 10)
2. Identify the core tension being resolved (Lens 3)
3. Specify the abstraction level (Lens 2)
4. List 3 concrete experiments that would validate the idea
5. Anticipate the strongest objection and prepare a response
6. Define a 2-week pilot that would provide signal on feasibility

Framework Selection Guide

Common Pitfalls in Research Ideation

Mode 2: Cognitive Frameworks

Eight empirically grounded frameworks from cognitive science, applied to computer science and AI research. Each framework targets a distinct cognitive operation: combining, reformulating, analogizing, constraining, inverting, abstracting, exploring boundaries, and holding contradictions.

Framework 1: Combinatorial Creativity (Bisociation)

Novel ideas arise from combining existing concepts in unexpected ways. Arthur Koestler called this bisociation -- connecting two previously unrelated frames of reference.

Why it works: Meta-research consistently shows that breadth of knowledge is a precursor to creative output. The combination itself is the creative act.

Systematic Bisociation Workflow:

1. Select two domains you have at least passing familiarity with
2. List core primitives in each domain (5-10 fundamental concepts per domain)
3. Create a cross-product matrix: row = concepts from Domain A, column = concepts from Domain B
4. For each cell, ask: "What would it mean to apply A's concept to B's problem?"
5. Filter: Which combinations produce a non-trivial, testable research question?
6. Validate structural depth: Is the connection mechanistic or merely metaphorical?

Quality Test: A strong bisociation is not a surface metaphor ("the network is like a brain") but a structural mapping where the mechanism transfers.

Framework 2: Problem Reformulation (Representational Change)

Breakthroughs often come not from solving the problem as stated, but from re-representing the problem itself.

Reformulation Strategies:

Classic CS Examples:

• PageRank: Reformulated "find important web pages" from content analysis to graph eigenvalue problem
• Dropout: Reformulated "prevent overfitting" from regularization to approximate ensemble
• Attention: Reformulated "handle long sequences" from remembering everything to selectively querying

Framework 3: Analogical Reasoning (Structure-Mapping)

Dedre Gentner's structure-mapping theory: surface-level analogies are common but weak; structural or relational analogies produce the most powerful insights.

Levels of Analogical Depth:

Structure-Mapping Workflow:

1. Describe your problem using only relational/causal language (strip domain-specific nouns)
2. Search for structural matches: What other systems solve a structurally similar problem?
3. Pick the most distant match with genuine structural fidelity
4. Map the solution mechanism: How does the source domain solve this?
5. Transfer and adapt: What changes when you bring that mechanism into your domain?
6. Generate predictions: The analogy should tell you something you didn't already know

Framework 4: Constraint Manipulation (Boden's Framework)

Margaret Boden's framework distinguishes three forms of creativity:

Constraint Analysis Workflow:

1. List the constraints of your current approach (5-10 constraints)
2. Classify each constraint: Hard (physically necessary), Soft (convention), Hidden (implicitly assumed)
3. For each soft/hidden constraint, ask:
   • What if we relaxed it?
   • What if we tightened it?
   • What if we replaced it with a different constraint entirely?
4. The most productive move is often exposing and dropping a hidden constraint

Framework 5: Negation and Inversion

Take a core assumption in your field and negate it.

Negation Hall of Fame in CS:

TRIZ-Inspired Principles for CS:

Framework 6: Abstraction and Generalization Laddering

Three moves:

When to Generalize vs. Specialize:

• Generalize when you have results but no explanation
• Specialize when you have theory but no grounding
• Analogize when you are stuck in either direction

Framework 7: The Adjacent Possible (Kauffman / Johnson)

Innovation happens at the boundary of what is currently reachable. New ideas become thinkable once their prerequisites exist.

Adjacent Possible Mapping Workflow:

1. List recent enablers (last 1-3 years): new hardware, datasets, tools, theoretical results, regulations
2. For each enabler, ask: "What was previously impossible or impractical that this now permits?"
3. Combine enablers: The most powerful adjacent possibles arise from the intersection of multiple new enablers
4. Check for competition: If many people can see the same adjacent possible, speed or a unique angle matters

Timing Signal: If your idea requires technology that doesn't exist yet, it's beyond the adjacent possible -- park it. If your idea could have been done 5 years ago, someone probably did -- check the literature. The sweet spot is ideas that became feasible in the last 6-18 months.

Framework 8: Janusian and Dialectical Thinking

Albert Rothenberg's studies found that holding two contradictory ideas simultaneously is a hallmark of creative thinking. This mode doesn't resolve contradictions by choosing a side -- it generates new frameworks that transcend the opposition.

Dialectical Thinking Workflow:

1. Identify a binary in your field: A vs. B (two approaches treated as opposites)
2. Resist choosing a side. Instead ask:
   • "What would a system look like that achieves both A and B?"
   • "Under what conditions is the A-B trade-off not fundamental?"
   • "Is the opposition an artifact of how we formalized the problem?"
3. Seek synthesis: The resolution often requires a new abstraction
4. Test the synthesis: Can you demonstrate empirically that both goals are achievable?

Combining Cognitive Frameworks: A Protocol

Phase 1: Map the Space (15 min)

1. Constraint Manipulation (F4): List all constraints. Mark which are hard, soft, hidden.
2. Adjacent Possible (F7): List recent enablers that change the feasibility landscape.

Phase 2: Generate Disruptions (30 min)

3. Negation (F5): Negate 3 soft/hidden constraints. What systems emerge?
4. Bisociation (F1): Pick a distant field and create a cross-product matrix.
5. Problem Reformulation (F2): Restate your problem 3 different ways.

Phase 3: Deepen Promising Leads (30 min)

6. Analogical Reasoning (F3): For each promising idea, find a structural analogy.
7. Abstraction Laddering (F6): Move each idea up (generalize) and down (specialize).
8. Janusian Thinking (F8): Identify any tensions. Can you synthesize rather than choose?

Phase 4: Evaluate (15 min)

Apply the two-sentence test:

"[Domain] currently struggles with [problem] because [reason]. We [approach] by [mechanism], which works because [insight]."

Common Creative Blocks and Unblocking Strategies

Mode 3: Interactive Exploration

A conversational approach to brainstorming. Act as an equal thought partner -- the researcher should be doing at least half the talking.

Workflow

Phase 1: Understanding the Context

• Ask open-ended questions about current research, interests, or challenges
• Understand the field, methodology, and constraints
• Listen for implicit assumptions or unexplored angles

Example questions:

• "What aspect of your research are you most excited about right now?"
• "What problem keeps you up at night?"
• "What assumptions are you making that might be worth questioning?"
• "Are there any unexpected findings that don't fit your current model?"

Phase 2: Divergent Exploration

Techniques to employ:

1. Cross-Domain Analogies -- Draw parallels from other scientific fields
2. Assumption Reversal -- Identify core assumptions and flip them
3. Scale Shifting -- Explore the problem at different scales (spatial, temporal)
4. Constraint Removal/Addition -- "What if you could measure anything?" / "What if you had to solve this with 1800s technology?"
5. Interdisciplinary Fusion -- Suggest combining methodologies from different fields
6. Technology Speculation -- "What becomes possible with [emerging tech]?"

Interaction style:

• Rapid-fire idea generation with the researcher
• Build on suggestions with "Yes, and..."
• Encourage wild ideas explicitly
• Consult references/brainstorming_methods.md for additional structured techniques (SCAMPER, Six Thinking Hats, Morphological Analysis, TRIZ, Biomimicry)

Phase 3: Connection Making

• Look for common threads across different ideas
• Identify which ideas complement or enhance each other
• Find surprising connections between seemingly unrelated concepts

Phase 4: Critical Evaluation

• Be critical but not dismissive
• Identify both strengths and challenges
• Consider feasibility while preserving innovative elements
• "What would it take to actually test this?"
• "What's the first small experiment to run?"

Phase 5: Synthesis and Next Steps

• Summarize the most promising directions identified
• Highlight novel connections or perspectives discovered
• Suggest immediate next steps (literature search, pilot experiments, collaborations)
• Capture key questions for future exploration

Adaptive Techniques

When the researcher is stuck: Break the problem into smaller pieces, change the framing entirely, suggest exploring tangential ideas.

When ideas are too safe: Encourage risk-taking, play devil's advocate, ask about failed or abandoned approaches and why they might actually work.

When energy lags: Inject enthusiasm about interesting ideas, share genuine curiosity about a particular direction, take a brief tangent.

Usage Instructions for Agents

When a researcher asks for help with ideation:

1. Identify their starting point: Are they exploring a new area, stuck on a current project, or evaluating an existing idea?
2. Select the appropriate mode: Use the mode table above to pick the best fit
3. Select frameworks/lenses: Within the chosen mode, pick 2-3 relevant frameworks
4. Walk through interactively: Apply each framework step-by-step, asking for domain-specific inputs
5. Generate candidates: Aim for 10-20 raw ideas across frameworks
6. Filter and rank: Apply the Converge phase filters to narrow to top 3-5
7. Refine the winner: Help articulate the two-sentence pitch and define concrete next steps

Key Principles:

• Push for specificity -- vague ideas ("improve efficiency") are not actionable
• Challenge assumptions -- ask "why?" at least three times
• Maintain a written list of all candidates, even rejected ones (they may recombine later)
• Generative mode first, evaluative mode second -- do not filter prematurely
• Distant analogies are more valuable than nearby ones, but require more validation
• The researcher's domain expertise is essential -- the agent provides cognitive scaffolding
• The researcher makes the final call on which ideas to pursue