Build an internal data agent on certified metrics

An internal data agent should answer business questions from certified metrics - not from improvised SQL. This tutorial walks through a minimal path: publish a model, connect an agent over MCP, and eval that humans and the agent get the same number.

Prerequisites

• A Query Runner next to the warehouse with a read-only role (nine engines, including Snowflake, BigQuery, Postgres, and MySQL), registered as a source in datahashi
• A small YAML semantic model with at least one metric and a few dimensions
• An MCP-capable agent runtime (Claude Desktop, your own MCP host, or similar)

Keep transforms in the warehouse. The semantic model should stay thin: entities, relationships, measures, metrics, and access rules.

1. Author and publish

Define entities and a metric you already trust in a dashboard. Example shape:

entities:
  - name: Order
    table: public.orders
    primary_key: id
    measures:
      - name: revenue
        expr: amount
        agg: sum
      - name: order_count
        expr: id
        agg: count

metrics:
  - name: avg_order_value
    expr: revenue / order_count
    unit: USD
    format: currency

Validate in the app, then publish. Publishing freezes the model and its policy bundle as one immutable version. Downstream consumers pin that version so mid-flight edits cannot silently change answers.

2. Issue a governed credential

Grant the agent a role that exposes named metrics and the dimensions they need. Bind the same row-level rules you already use for humans. Do not grant raw measures unless you explicitly want that analysis, and even then prefer a separate credential.

Prefer OAuth for the agent so you are not pasting a long-lived warehouse secret into a config file.

3. Connect over MCP

Point the agent at the datahashi MCP server. The tool surface is fixed:

• list_catalog
• describe_metric
• run_semantic_query

None accept or return SQL. System instructions should say: answer quantitative questions only through these tools; if the catalog lacks a metric, say you cannot answer rather than guessing.

4. Same request shape as the console

Have the agent ask for avg_order_value by region for a fixed quarter. In parallel, run the identical Semantic Query in the console or via REST against the same published version. Compare the rows.

That comparison is the product truth you are buying: same metric for humans and AI agents, without hallucinated SQL. Joins are resolved by the model. Filters that define the metric cannot be dropped by the caller. Governance is the request shape, not a post-filter.

5. Write a small same-number eval

Do not stop at a demo chat. Codify trust:

1. Pick 5-10 golden questions with expected metric names, dimensions, filters, and pinned model version.
2. Run each through the agent (tool calls only) and through REST/console.
3. Assert equal numeric results within an agreed tolerance (usually exact for additive measures; document any known float cases).
4. Assert the agent never attempted a SQL-shaped tool call (there should be none to attempt).
5. Record metering for those principals so cost and scan volume are visible.

Fail the eval if the agent answers from memory or paraphrases a number it did not just fetch.

6. Ship behind clear refusal behavior

Production agents should refuse out-of-catalog questions, ambiguous grain, and requests that would require widening the credential. A correct “I don’t have that certified metric” is better than a plausible invention.

Metering exists on every query; use it to see which teams and agents drive load. Rollups can serve the hottest shapes without the warehouse, but they are not a latency guarantee - do not design the agent UX around one.

Publish the model. Bind governed MCP tools. Eval same-number trust before you roll the agent out company-wide.