Research Literature Kg

Workflow

Phase 1: Literature Collection

1. Define research scope:

   • Primary topic (daily focus)
   • Secondary topic (weekly theme)
   • Keywords for search

2. Search arxiv:

   query = f'cat:{category}+AND+all:{keywords}'
   url = f'http://export.arxiv.org/api/query?search_query={query}&max_results=10&sortBy=submittedDate&sortOrder=descending'
   

3. Parse results: Extract title, authors, abstract, arxiv_id, category, published_date

Phase 2: KG Construction

1. Initialize KG (if needed):

   kg_tool init <db_path>
   

2. Add entities:

   kg_tool add-entity <db_path> paper <arxiv_id>
   

3. Store metadata: JSON properties with title, authors, abstract, category

Phase 3: Vector Embeddings

1. Generate embeddings:

   • Model: all-MiniLM-L6-v2 (384 dimensions)
   • Text: {title}. {abstract} (max 500 chars)

2. Store vectors:

   INSERT INTO kg_vectors (entity_id, vector, dimension, created_at)
   VALUES (?, ?, 384, ?)
   

Phase 4: Graph Analysis

1. PageRank: Find important papers

   kg_tool pagerank <db_path>
   

2. Louvain: Detect research clusters

   kg_tool louvain <db_path>
   

3. Semantic search: Find related papers

   kg_tool search <db_path> <query>
   

Phase 5: Pattern Extraction

1. Identify patterns: Look for recurring methods, frameworks, approaches
2. Extract skills: Use skill-extractor skill
3. Create new skill: Use skill-creator skill

Output Format

# Research Literature KG Report

**Date**: YYYY-MM-DD
**Topics**: Primary + Secondary

## Statistics
- Papers collected: N
- Entities in KG: M
- Vectors generated: K

## Top Papers (PageRank)
1. [ID] Title - score
2. ...

## Semantic Search Results
### Query: "quantum computing"
- Top matches with similarity scores

## Research Clusters (Louvain)
- Community 0: Topic A papers
- Community 1: Topic B papers

## Extracted Patterns
- Pattern 1: [Description]
- Pattern 2: [Description]

## New Skills Created
- skill-name: Description

Database Schema

-- kg_entities
CREATE TABLE kg_entities (
    id INTEGER PRIMARY KEY,
    entity_type TEXT,  -- 'paper', 'author', 'keyword', 'topic'
    name TEXT,
    properties TEXT,   -- JSON: {title, authors, abstract, category}
    created_at INTEGER
);

-- kg_vectors
CREATE TABLE kg_vectors (
    entity_id INTEGER,
    vector BLOB,       -- float32 array
    dimension INTEGER, -- 384 for all-MiniLM-L6-v2
    created_at INTEGER
);

-- kg_relations
CREATE TABLE kg_relations (
    id INTEGER PRIMARY KEY,
    source_id INTEGER,
    target_id INTEGER,
    rel_type TEXT,     -- 'cites', 'related', 'same_author'
    weight REAL
);

Error Handling

arxiv API Timeout

• Retry after 5 seconds
• Use backup search via web_search

Embedding Generation Failure

• Check sentence-transformers installed
• Fall back to simpler model if memory limited

Dimension Mismatch

• Check vector dimensions match (384)
• Re-generate mismatched vectors

Instructions for Agents

Step 1: Define Research Scope

Identify primary topic, secondary topic, and search keywords from user request.

Step 2: Collect Literature

Search arxiv API with category and keyword filters; parse title, authors, abstract, arxiv_id.

Step 3: Build Knowledge Graph

Initialize kg.db if needed; add papers as entities; store JSON metadata.

Step 4: Generate Embeddings and Analyze

Generate 384-dim embeddings via all-MiniLM-L6-v2; run PageRank and Louvain algorithms.

Step 5: Extract Patterns and Report

Use skill-extractor to identify reusable patterns; output Research Literature KG Report.

Examples

Example 1: Build Brain Research KG

User: "Build a knowledge graph from recent brain connectivity papers"

Agent:
1. Search arxiv: cat:q-bio.NC AND all:brain connectivity, max 10 results
2. Add papers to kg.db with metadata
3. Generate vector embeddings
4. Run PageRank to find most influential papers
5. Run Louvain to detect research clusters
6. Output report with top papers and patterns

Example 2: Semantic Search in Research

User: "Find quantum finance papers related to portfolio optimization"

Agent:
1. Initialize search query: "quantum portfolio optimization"
2. Fetch papers from arxiv quant-ph category
3. Add to kg.db and generate embeddings
4. Run similarity search for related papers
5. Report top matches with similarity scores

Related Skills

• arxiv-search: Paper search details
• skill-extractor: Pattern extraction
• skill-creator: Skill creation
• feishu-bitable: Alternative storage

Notes

• KG persists across sessions via SQLite
• Vectors enable semantic search
• Weekly topics rotate through domains
• Daily quantum mechanics focus39:["$","$L42",null,{"content":"$43","frontMatter":{"name":"research-literature-kg","description":"Build and analyze knowledge graphs from research literature. Automated pipeline: arxiv search → entity extraction → KG construction → vector embeddings → semantic search → skill pattern extraction. Use when user asks to analyze papers, build research knowledge bases, find related work, or extract reusable patterns from academic literature."}}]