Generate optimized tool call parsers for dynamo from HuggingFace model chat templates. Use this when you need to add support for a new model's tool calling format. Takes a HuggingFace model name, analyzes its chat template, compares with existing parsers, and either maps to existing parser or generates new Rust code with tests for the dynamo tool_calling library.
Add support for new models' tool calling formats by analyzing their chat templates and generating appropriate parser implementations for dynamo.
Follow this systematic workflow when the user provides a HuggingFace model name.
Fetch tokenizer config from HuggingFace Hub:
URL: https://huggingface.co/{model_id}/resolve/main/tokenizer_config.json
Extract chat template:
chat_template fieldname and template fields
tool_use template if availabledefault templateExtract special tokens (if relevant):
bos_token, eos_token, unk_tokenadditional_special_tokensThe chat template is a Jinja template. Analyze it to identify tool call patterns:
Find tool-related sections:
tools, tool_call, function, available_tools{% if tools %}...{% endif %} blocks{% for tool in tools %} loopsIdentify markers and format:
<tool_call>, [TOOL_CALLS], <|python_tag|>, <|tool▁call▁begin|></tool_call>, [/TOOL_CALLS], <|tool▁call▁end|>tojson filter, { } brackets<function=, <parameter= patternsfunction(arg=val) patterns<|DSML| tokensIdentify JSON structure (if JSON format):
name or functionarguments or parametersRead existing parser implementations in /lib/parsers/src/tool_calling/:
Check JSON parsers (json/ directory):
base_json_parser.rs - Generic JSON with markersdeepseek_v3_parser.rs - DeepSeek V3 formatdeepseek_v3_1_parser.rs - DeepSeek V3.1 formatCheck XML parsers (xml/ directory):
parser.rs - Qwen3 Coder XML formatCheck other formats:
pythonic/pythonic_parser.rs - Python syntaxharmony/harmony_parser.rs - Harmony protocoldsml/parser.rs - DeepSeek V3.2 DSMLReview config presets in config.rs:
ToolCallConfig::hermes(), mistral(), llama3_json(), etc.Check parser registry in parsers.rs:
get_tool_parser_map()ParserType enum and routing logicMatch the analyzed format:
If a match is found, create a configuration preset:
Add a new preset function to /lib/parsers/src/tool_calling/config.rs:
impl ToolCallConfig {
pub fn new_model_name() -> Self {
Self {
config: ParserConfig::Json(JsonParserConfig {
start_token: Some("<marker>".to_string()),
end_token: Some("</marker>".to_string()),
function_name_key: Some("name".to_string()),
function_arguments_key: Some("arguments".to_string()),
parser_type: JsonParserType::Basic,
}),
}
}
}
Register in parser map in /lib/parsers/src/tool_calling/parsers.rs
Create tests to verify the configuration works
If no existing parser fits, generate new parser code:
Choose parser template based on format:
base_json_parser.rs as templatexml/parser.rs as templateImplement required functions:
// Detection
pub fn detect_tool_call_start_<name>(chunk: &str, config: &Config) -> bool
// Parsing
pub fn try_tool_call_parse_<name>(
message: &str,
config: &Config,
tools: Option<&[ToolDefinition]>,
) -> Result<(Vec<ToolCallResponse>, Option<String>)>
// End detection (for streaming)
pub fn find_tool_call_end_position_<name>(chunk: &str, config: &Config) -> usize
Use regex for token matching:
OnceLock<Regex> for compiled regexesParse JSON/XML content:
serde_json for JSON parsingToolCallResponse structsAdd to appropriate directory:
json/ directoryxml/ directoryFor any new parser or configuration, generate comprehensive tests:
Basic tests:
Edge cases:
Integration tests:
Add tests to appropriate location:
#[cfg(test)] module)/lib/parsers/src/tool_calling/tests.rsUpdate module exports:
mod declaration in parent mod.rsRegister parser in parsers.rs if new parser:
get_tool_parser_map() functiontest_get_available_tool_parsers() testavailable_parsers array in the testDocument the parser:
Run tests:
cd lib/parsers
cargo test tool_calling
Verify with dynamo:
Dynamo Codebase:
/lib/parsers/src/tool_calling/ - All tool call parsers/lib/parsers/src/tool_calling/config.rs - Configuration presets/lib/parsers/src/tool_calling/parsers.rs - Parser registry/lib/llm/src/preprocessor/prompt/template/tokcfg.rs - Chat template structures/lib/llm/src/preprocessor/prompt/template.rs - Template loadingReference Implementations:
User: "Add tool calling support for Qwen/Qwen2.5-72B-Instruct"
Step 1: Fetch tokenizer config
https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/resolve/main/tokenizer_config.jsonStep 2: Analyze chat template
chat_template field{% if tools %} block<tool_call> and </tool_call>tojson filterStep 3: Compare with existing parsers
/lib/parsers/src/tool_calling/config.rsToolCallConfig::hermes() - uses <tool_call> markersStep 4: Use or adapt existing parser
qwen2_5() config presetStep 5: Generate tests
Step 6: Integrate
config.rsget_tool_parser_map())test_get_available_tool_parsers() test[TOOL_CALLS] [{"name": "func", "arguments": {}}]
→ Use base_json_parser with bracket markers
<tool_call>
{"name": "func", "arguments": {}}
</tool_call>
→ Use base_json_parser with XML-style markers
<tool_call>
<function=name>
<parameter=key>value</parameter>
</function>
</tool_call>
→ Use xml/parser.rs or create variant
<|tool▁call▁begin|>name<|tool▁sep|>args<|tool▁call▁end|>
→ Create specialized parser (see DeepSeek parsers)
Most models (>80%) can use existing parsers with appropriate configuration.