Systematic workflow for researching and validating blockchain RPC providers. Use when evaluating RPC providers for historical data collection, rate limits, archive access, compute unit costs, or timeline estimation for large-scale blockchain data backfills.
This skill provides a systematic, empirically-validated workflow for researching blockchain RPC providers before committing to large-scale data collection projects. Use when selecting an RPC provider for historical blockchain data backfill, evaluating rate limits, comparing free tier options, or estimating collection timelines.
Key principle: Never trust documented rate limits—always validate empirically with POC testing.
This skill follows a 5-step workflow. Each step builds on the previous:
: See for complete step-by-step guide with code examples, questions to answer, and success criteria.
references/workflow-steps.mdQuick start: For immediate testing, jump to Step 3 (Empirical Validation) using scripts/test_rpc_rate_limits.py.
When implementing the selected provider, use conservative targeting (80-90% of empirically validated rate) with monitoring and fallback strategy.
Full guide: See references/rate-limiting-guide.md for detailed monitoring requirements, fallback strategies, and safety margins.
Critical mistakes to avoid: Trusting documented burst limits (always validate empirically), testing with <50 blocks, parallel fetching on free tiers, ignoring compute unit costs, and forgetting archive access restrictions.
Full guide: See references/common-pitfalls.md for detailed anti-patterns with real-world examples (e.g., LlamaRPC 50 RPS → 1.37 RPS case).
calculate_timeline.py - Calculate collection timeline from rate limits (RPS or compute units)test_rpc_rate_limits.py - Empirical rate limit testing templateUsage guide: See scripts/README.md for detailed usage examples, configuration options, and success criteria.
references/workflow-steps.md - Complete 5-step workflow with detailed guidance for each stepreferences/rate-limiting-guide.md - Best practices for conservative rate targeting and monitoringreferences/common-pitfalls.md - Anti-patterns to avoid with real-world examplesreferences/example-workflow.md - Complete case study: 13M Ethereum blocks RPC selectionreferences/validated-providers.md - Alchemy vs LlamaRPC vs Infura vs QuickNode empirical comparisonreferences/rpc-comparison-template.md - Template for creating provider comparison matricesscripts/README.md - Complete usage guide for all scriptsscripts/calculate_timeline.py - Timeline calculator (RPS and compute unit modes)scripts/test_rpc_rate_limits.py - Empirical rate limit testing templateCase study: Selecting RPC provider for 13M Ethereum blocks → Alchemy chosen at 5.79 RPS (26 days timeline, 4.2x faster than LlamaRPC).
Full walkthrough: See references/example-workflow.md for complete step-by-step case study showing research, calculation, validation, comparison, and final recommendation.
Invoke this skill when:
This skill pairs well with:
blockchain-data-collection-validation - For validating the complete data pipeline after provider selectionscratch/ethereum-collector-poc/ and scratch/rpc-provider-comparison/