Protein Encoding | Skills Pool

技能檔案

Protein Encoding

ESM-3 protein embeddings, encoder integration, pooling strategies

Jinyeop31100 星標2026年2月25日

職業: 數據科學家
分類: 計算化學

技能內容

Protein Encoding Skill

Critical Rules

NEVER modify ESM-3 encoder weights - always keep frozen during training
Use attention pooling (32 output tokens), NOT mean pooling
LoRA on k/v matrices only for protein tasks (differs from NLP)

ESM-3 Model

Model	Parameters	Embedding Dim	Recommended
esm3-sm-open-v1	1.4B	1,536	Default

Key Files

src/models/protein_encoder.py - Encoder implementations
src/models/pooling.py - Pooling strategies
configs/encoder/ - Encoder configurations

Integration Pattern

相關技能

ESM-3 (frozen) → Per-residue [L, 1536]
    ↓
Attention Pooling → [32, 1536]
    ↓
MLP Projector → [32, LLM_dim]
    ↓
LLM (with LoRA)

from src.models.protein_encoder import ESM3Encoder

encoder = ESM3Encoder(
    model_name="esm3-sm-open-v1",
    freeze=True  # ALWAYS True
)

# Get embeddings
embeddings = encoder(sequences)  # [batch, seq_len, 1536]