Name: Polars Bio
Author: K-Dense-AI

스킬 검색.../

Polars Bio | Skills Pool

pip install polars-bio
# or
uv pip install polars-bio

pip install polars-bio[pandas]

import polars as pl
import polars_bio as pb

# Create two interval DataFrames
df1 = pl.DataFrame({
    "chrom": ["chr1", "chr1", "chr1"],
    "start": [1, 5, 22],
    "end":   [6, 9, 30],
})

df2 = pl.DataFrame({
    "chrom": ["chr1", "chr1"],
    "start": [3, 25],
    "end":   [8, 28],
})

# Functional API (returns LazyFrame by default)
result = pb.overlap(df1, df2)
result_df = result.collect()

# Get a DataFrame directly
result_df = pb.overlap(df1, df2, output_type="polars.DataFrame")

# Method-chaining API (via .pb accessor on LazyFrame)
result = df1.lazy().pb.overlap(df2)
result_df = result.collect()

import polars_bio as pb

# Eager read (loads entire file)
df = pb.read_bed("regions.bed")

# Lazy scan (streaming, for large files)
lf = pb.scan_bed("regions.bed")
result = lf.collect()

import polars_bio as pb

# Find overlapping intervals (returns LazyFrame)
result = pb.overlap(df1, df2, suffixes=("_1", "_2"))

# Count overlaps per interval
counts = pb.count_overlaps(df1, df2)

# Merge overlapping intervals
merged = pb.merge(df1)

# Find nearest intervals
nearest = pb.nearest(df1, df2)

# Collect any LazyFrame result to DataFrame
result_df = result.collect()

import polars_bio as pb

# Read VCF file
variants = pb.read_vcf("samples.vcf.gz")

# Lazy scan BAM file (streaming)
alignments = pb.scan_bam("aligned.bam")

# Read GFF annotations
genes = pb.read_gff("annotations.gff3")

# Cloud storage (individual params, not a dict)
df = pb.read_bed("s3://bucket/regions.bed",
                 allow_anonymous=True)

import polars as pl
import polars_bio as pb

# Register files as SQL tables (path first, name= keyword)
pb.register_vcf("samples.vcf.gz", name="variants")
pb.register_bed("target_regions.bed", name="regions")

# Query with SQL (returns LazyFrame)
result = pb.sql("SELECT chrom, start, end, ref, alt FROM variants WHERE qual > 30")
result_df = result.collect()

# Register a Polars DataFrame as a SQL table
pb.from_polars("my_intervals", df)
result = pb.sql("SELECT * FROM my_intervals WHERE chrom = 'chr1'").collect()

import polars_bio as pb

# Compute depth across a BAM file
depth_lf = pb.depth("aligned.bam")
depth_df = depth_lf.collect()

# With quality filter
depth_lf = pb.depth("aligned.bam", min_mapping_quality=20)

import polars_bio as pb

# Switch to 0-based coordinates
pb.set_option("coordinate_system", "0-based")

# Switch back to 1-based (default)
pb.set_option("coordinate_system", "1-based")

# Read BED with explicit 0-based metadata
df = pb.read_bed("regions.bed", use_zero_based=True)

result = pb.overlap(df1, df2, suffixes=("_1", "_2"))
merged = pb.merge(df)

result = df1.lazy().pb.overlap(df2)
merged = df.lazy().pb.merge()

# Chain interval operations (note: overlap outputs suffixed columns,
# so rename before merge which expects chrom/start/end)
result = (
    df1.lazy()
    .pb.overlap(df2)
    .filter(pl.col("start_2") > 1000)
    .select(
        pl.col("chrom_1").alias("chrom"),
        pl.col("start_1").alias("start"),
        pl.col("end_1").alias("end"),
    )
    .pb.merge()
    .collect()
)

result = pb.overlap(
    df1, df2,
    cols1=["chromosome", "begin", "finish"],
    cols2=["chr", "pos_start", "pos_end"],
)

# Lazy (default) - collect when needed
result_lf = pb.overlap(df1, df2)
result_df = result_lf.collect()

# Eager - get DataFrame directly
result_df = pb.overlap(df1, df2, output_type="polars.DataFrame")

# Scan files lazily
lf = pb.scan_bed("large_intervals.bed")

# Process with streaming
result = lf.collect(streaming=True)

.pb accessor on DataFrame vs LazyFrame: Interval operations (overlap, merge, etc.) are only on LazyFrame.pb. DataFrame.pb only has write methods. Use .lazy() to convert before chaining interval ops.
LazyFrame returns: All interval operations and pb.sql() return LazyFrame by default. Don't forget .collect() or use output_type="polars.DataFrame".
Column name mismatches: polars-bio expects chrom, start, end by default. Use cols1/cols2 parameters (as lists) if your columns have different names.
Coordinate system metadata: When constructing DataFrames manually (not via read_*/scan_*), polars-bio warns about missing coordinate metadata. Use pb.set_option("coordinate_system", "0-based") globally, or use I/O functions that set metadata automatically.
Probe-build order matters: For overlap, nearest, and coverage, the first DataFrame is probed against the second. Swapping arguments changes which intervals appear in the left vs right output columns, and can affect performance.
INT32 position limit: Genomic positions are stored as 32-bit integers, limiting coordinates to ~2.1 billion. This is sufficient for all known genomes but may be an issue with custom coordinate spaces.
BAM index requirements: read_bam and scan_bam require a .bai index file alongside the BAM. Create one with samtools index if missing.
Parallel execution disabled by default: DataFusion parallelism defaults to 1 partition. Enable for large datasets:
```
pb.set_option("datafusion.execution.target_partitions", 8)
```
CRAM has separate functions: Use read_cram/scan_cram/register_cram for CRAM files (not read_bam). CRAM functions require a reference_path parameter.

Use scan_* for large files: Prefer scan_bed, scan_vcf, etc. over read_* for files larger than available RAM. Scan functions enable streaming and predicate pushdown.

Configure parallelism for large datasets:

import os
pb.set_option("datafusion.execution.target_partitions", os.cpu_count())

Use BGZF compression: BGZF-compressed files (.bed.gz, .vcf.gz) support parallel block decompression, significantly faster than plain GZIP.
Select columns early: When only specific columns are needed, select them early to reduce memory usage:
```
df = pb.read_vcf("large.vcf.gz").select("chrom", "start", "end", "ref", "alt")
```
Use cloud paths directly: Pass S3/GCS/Azure URIs directly to read/scan functions instead of downloading files first:
```
df = pb.read_bed("s3://my-bucket/regions.bed", allow_anonymous=True)
```
Prefer functional API for single operations, method-chaining for pipelines: Use pb.overlap() for one-off operations and .lazy().pb.overlap() when building multi-step pipelines.

Polars Bio

Overview

When to Use This Skill

Polars Bio

Overview

When to Use This Skill

Quick Start

Installation

Basic Overlap Example

Reading a BED File

Core Capabilities

1. Genomic Interval Operations

2. Bioinformatics File I/O

3. SQL Data Processing

4. Pileup Operations

Key Concepts

Coordinate Systems

Two API Styles

Probe-Build Architecture

Column Conventions

Return Types and Collecting Results

Streaming and Out-of-Core Processing

Common Pitfalls

Best Practices

Resources

references/

Clickhouse Io

Clickhouse Io

Claude Devfleet

Clickhouse Io

Ai First Engineering

Postgres Patterns