Directory Structure

LLM-related code is organized in specific directories:

apps/web/utils/ai/ - Main LLM implementations
apps/web/utils/llms/ - Core LLM utilities and configurations
apps/web/__tests__/ - LLM-specific tests

Key Files

utils/llms/index.ts - Core LLM functionality
utils/llms/model.ts - Model definitions and configurations
utils/usage.ts - Usage tracking and monitoring

Implementation Pattern

Follow this standard structure for LLM-related functions:

import { z } from "zod";
import { createScopedLogger } from "@/utils/logger";
import { chatCompletionObject } from "@/utils/llms";
import type { EmailAccountWithAI } from "@/utils/llms/types";
import { createGenerateObject } from "@/utils/llms";

export async function featureFunction(options: {
  inputData: InputType;
  emailAccount: EmailAccountWithAI;
}) {
  const { inputData, user } = options;

  if (!inputData || [other validation conditions]) {
    logger.warn("Invalid input for feature function");
    return null;
  }

  const system = `[Detailed system prompt that defines the LLM's role and task]`;

  const prompt = `[User prompt with context and specific instructions]

<data>
...
</data>

${emailAccount.about ? `<user_info>${emailAccount.about}</user_info>` : ""}`;

  const modelOptions = getModel(emailAccount.user);

  const generateObject = createGenerateObject({
    userEmail: emailAccount.email,
    label: "Feature Name",
    modelOptions,
  });


  const result = await generateObject({
    ...modelOptions,
    system,
    prompt,
    schema: z.object({
      field1: z.string(),
      field2: z.number(),
      nested: z.object({
        subfield: z.string(),
      }),
      array_field: z.array(z.string()),
    }),
  });

  return result.object;
}

Implementation Pattern

Follow this standard structure for LLM-related functions:

import { z } from "zod"; import { createScopedLogger } from "@/utils/logger"; import { chatCompletionObject } from "@/utils/llms"; import type { EmailAccountWithAI } from "@/utils/llms/types"; import { createGenerateObject } from "@/utils/llms"; export async function featureFunction(options: { inputData: InputType; emailAccount: EmailAccountWithAI; }) { const { inputData, user } = options; if (!inputData || [other validation conditions]) { logger.warn("Invalid input for feature function"); return null; } const system = `[Detailed system prompt that defines the LLM's role and task]`; const prompt = `[User prompt with context and specific instructions] <data> ... </data> ${emailAccount.about ? `<user_info>${emailAccount.about}</user_info>` : ""}`; const modelOptions = getModel(emailAccount.user); const generateObject = createGenerateObject({ userEmail: emailAccount.email, label: "Feature Name", modelOptions, }); const result = await generateObject({ ...modelOptions, system, prompt, schema: z.object({ field1: z.string(), field2: z.number(), nested: z.object({ subfield: z.string(), }), array_field: z.array(z.string()), }), }); return result.object; }

Llm

Directory Structure

Key Files

Implementation Pattern

Llm

Directory Structure

Key Files

Implementation Pattern

Best Practices

Testing

Continuous Learning V2

Continuous Learning V2

Continuous Learning V2

Continuous Learning

Continuous Learning

Pytorch Patterns