OctoMesh Pipeline Expert

Where Node Source Code Lives

Pipeline nodes are split across two repositories in the monorepo. All paths below are relative to the monorepo root (the parent directory containing octo-sdk, octo-mesh-adapter, etc.):

SDK nodes (ForEach, If, Switch, SetPrimitiveValue, For, SelectByPath, etc.):

octo-sdk/src/Sdk.Common/EtlDataPipeline/Nodes/
├── Control/       (ForEach, If, Switch, For, SelectByPath)
├── Extracts/      (SetPrimitiveValue, SetArrayOfPrimitiveValues, WriteJson)
├── Transforms/    (Concat, FormatString, TransformString, Hash, Math, etc.)
├── Triggers/      (FromPolling, FromPipelineDataEvent)
├── Loads/         (ToPipelineDataEvent, ToWebhook)
└── Buffering/     (BufferData, BufferRetrievalNode)

Mesh Adapter nodes (GetRtEntitiesByType, CreateUpdateInfo, ApplyChanges, etc.):

• Configuration classes (properties, defaults, validation):

  octo-mesh-adapter/src/MeshNodes.Sdk/
  ├── Extract/     (GetRtEntitiesByType, GetRtEntitiesById, GetOrCreate, etc.)
  ├── Transform/   (CreateUpdateInfo, CreateAssociationUpdate, CheckDuplicate, etc.)
  ├── Load/        (ApplyChanges, SaveInTimeSeries, SendEMail, SftpUpload, etc.)
  └── Trigger/     (FromWatchRtEntity, FromHttpRequest, FromEmail, etc.)
  

• Handler classes (execution logic — what the node actually does):

  octo-mesh-adapter/src/MeshAdapter.Sdk/Nodes/
  ├── Extract/
  ├── Transform/
  ├── Load/
  └── Trigger/
  

Finding the monorepo root: The monorepo root is the parent directory of this plugin's repository. Use ../ relative to the plugin working directory, or search upward for a directory containing both octo-sdk/ and octo-mesh-adapter/.

Naming Conventions

How to Research a Node

Step 1 — Find the source file (use Grep and Glob from the monorepo root):

# Search by node name (e.g., "CheckDuplicate")
Grep for: NodeName\("CheckDuplicate"
  in: octo-mesh-adapter/src/   (mesh adapter nodes)
  or: octo-sdk/src/            (SDK nodes)

# Or find the file by naming convention
Glob for: **/CheckDuplicate*Configuration*.cs
  in: octo-mesh-adapter/src/MeshNodes.Sdk/

Step 2 — Read the configuration class to learn:

• All available properties and their C# types
• Default values
• Required vs optional properties
• XML doc comments explaining each property

Step 3 — Read the handler class to learn:

• What the node actually does at runtime
• How it reads from and writes to the DataContext
• Error conditions and edge cases
• How properties interact with each other

Quick Lookup Examples

All paths below are relative to the monorepo root:

# Find where a specific node is defined
Grep pattern: NodeName\("CreateUpdateInfo"
Path: octo-mesh-adapter/src/

# If not found in mesh-adapter, search the SDK
Grep pattern: NodeName\("CreateUpdateInfo"
Path: octo-sdk/src/

# Find all available nodes for a category
Glob pattern: **/*NodeConfiguration*.cs
Path: octo-mesh-adapter/src/MeshNodes.Sdk/Transform/

# Find the handler to understand runtime behavior
Glob pattern: **/CheckDuplicateNode.cs
Path: octo-mesh-adapter/src/MeshAdapter.Sdk/

DataFlows and Pipeline Triggers

A DataFlow (System.Communication/DataFlow) is a logical grouping of related pipelines that work together as part of a single data processing workflow. It serves as the parent container for Pipeline and PipelineTrigger instances.

Migration note: CK migration 3.1.0→3.1.1 unified the old EdgeAdapter/MeshAdapter into a single Adapter type, and EdgePipeline/MeshPipeline into a single Pipeline type. Earlier migrations renamed DataPipeline → DataFlow and DataPipelineTrigger → PipelineTrigger. These are handled automatically by ChangeCkType transforms.

Key concepts:

• Pipelines belong to a DataFlow via System/ParentChild association
• A DataFlow establishes a shared topic exchange in the event hub, enabling inter-pipeline communication
• Pipelines within the same DataFlow can send data to each other using ToPipelineDataEvent@1 (with targetPipelineRtId) and FromPipelineDataEvent@1

A PipelineTrigger (System.Communication/PipelineTrigger) is a child of a DataFlow that triggers pipeline execution on a cron schedule via the Bot Service:

• Has Enabled and CronExpression attributes
• Has a Triggers association linking to one or more Pipeline entities
• The target pipelines must use FromPipelineTriggerEvent@1 as their trigger node
• Cron format: minute hour dayOfMonth month dayOfWeek year (6 fields)

Entity relationships:

DataFlow
  ├── Pipeline (child, via ParentChild) ── executes on ── Adapter
  └── PipelineTrigger (child, via ParentChild) ── triggers ── Pipeline(s)

Pipeline YAML Structure