Rules when working with ClickHouse database in Gram for analytics and telemetry features
The server/internal/telemetry package uses ClickHouse for high-performance analytics queries. Unlike PostgreSQL queries, ClickHouse queries are NOT auto-generated by SQLc (SQLc doesn't support ClickHouse). We use the squirrel query builder for dynamic query construction.
CRITICAL: The squirrel query builder is ONLY permitted for ClickHouse queries in the telemetry package. DO NOT use squirrel for PostgreSQL queries - all PostgreSQL queries MUST use SQLc-generated code. This is the only acceptable exception to our "no query builders" policy.
queries.sql.go: Query implementations using squirrelpagination.go: Cursor pagination helpers (withPagination, withOrdering, etc.)README.mdWhen asked to add a new ClickHouse query to the telemetry package:
Create a params struct for query inputs
Build the query using squirrel (the sq var in queries.sql.go is pre-configured for ClickHouse):
type GetMetricsParams struct {
ProjectID string
DeploymentID string // optional
Limit int
}
func (q *Queries) GetMetrics(ctx context.Context, arg GetMetricsParams) ([]Metric, error) {
sb := sq.Select("id", "value", "timestamp").
From("metrics").
Where("project_id = ?", arg.ProjectID)
// Optional filters - explicit conditionals for clarity
if arg.DeploymentID != "" {
sb = sb.Where(squirrel.Eq{"deployment_id": arg.DeploymentID})
}
sb = sb.Limit(uint64(arg.Limit))
query, args, err := sb.ToSql()
if err != nil {
return nil, fmt.Errorf("building query: %w", err)
}
rows, err := q.conn.Query(ctx, query, args...)
// ... handle rows
}
Use pagination helpers from pagination.go:
withPagination(sb, cursor, sortOrder) - cursor paginationwithOrdering(sb, sortOrder, primaryCol, secondaryCol) - ORDER BYtestenv.Launch() in TestMain for infrastructure setuptime.Sleep(100 * time.Millisecond) after inserts (ClickHouse eventual consistency)See server/internal/telemetry/README.md for comprehensive documentation.