AI Data Assistant

The problem

Connecting a language model to company data is easy to demo and hard to do safely. The usual shortcuts are to give the model a database connection, let it write free-form SQL, or paste whole tables into the prompt. They create the real risks: data leaking across users or tenants, unbounded queries, prompt injection through data, and answers nobody can trace back to a source.

The AI Data Assistant is a reference application for the safer pattern described in Using LLMs with Your Data: Practical Patterns. The model can choose among approved tools, and the application decides what each tool may do, for whom, and with which data.

Architecture

1. User

   • Web UI or API client

2. Application

   • API / orchestrationOwns the conversation and tool loop

3. Auth

   • Authentication
   • Authorization
   • Tenant context

   Trust boundary: resolved before any tool runs.

4. Approved tools

   • Parameterized queries
   • Metadata lookup
   • Retrieval

5. Data

   • SQL / data platformRow-level security per tenant

6. LLM

   • LLMChooses tools, drafts the answer

7. Response

   • Validated responseSchema, sources and policy checks

Cross-cutting concerns

• Audit log
• Tracing
• Cost limits
• Evaluation
• Per-user caching

How it works

1. 1QuestionFrom an authenticated user
2. 2Resolve identityUser, roles and tenant
3. 3Model selects a toolFrom the approved list, with arguments
4. 4Validate the callSchema, allowed values, row limits
5. 5Run with user permissionsParameterized query, tenant filter enforced
6. 6Bounded result to the modelOnly the rows and columns needed
7. 7Validate the answerFormat, cited sources, policy
8. 8Respond and logAnswer plus an audit record

The model never builds SQL that runs directly. Each tool is a function with a typed signature, for example “sales by region for a date range”. It maps to a parameterized query that the application owns. Tenant and permission filters are added by the application, never by the model, so a cleverly phrased question cannot remove them.

Implementation walkthrough

Planned milestones:

1. Tool layer: a small set of typed tools over a sample database, with parameter validation.
2. Identity and tenancy: authentication, roles and a tenant filter applied in the data layer.
3. Model integration: the tool-calling loop, with a provider-neutral interface.
4. Validation: structured output checks and source references in every answer.
5. Observability: traces of each step, an audit log, token and cost tracking.
6. Evaluation: a test set of questions with expected tool calls and answers.

Student path

For students

Prerequisites

• Python, basic SQL, and a basic understanding of HTTP APIs
• Access to an LLM API that supports tool or function calling

Concepts to understand first

Read Using LLMs with Your Data, especially the sections on tools and on security being an application responsibility.

Guided steps

Start with the tools alone and call them from tests, without a model. Add the model only when the tools behave correctly on their own. Then add identity, and confirm that the same question gives different, correct results for users in different tenants.

Exercises

• Add a new tool and write the validation for its parameters.
• Try to make the assistant reveal another tenant’s data, and document why it cannot.
• Add a test question where the right answer is “I cannot answer that with the available tools”.

Expected outcomes

You will understand the moving parts of a real LLM application: identity, tools, data access, the model, validation and logging. You will also understand why most of the safety lives outside the model.

Professional considerations

For professionals

Security and tenant isolation

Authorization is enforced in the data layer, for example with row-level security keyed on the tenant, and again in the tool layer. The model’s context never contains data the user could not query directly.

Prompt and data boundaries

Retrieved data is passed as data, clearly separated from instructions, and treated as untrusted. Tool results are size-limited, which bounds both cost and exposure.

Observability

Each request produces a trace: tool calls, arguments, row counts, latency and token usage. Logs avoid storing full prompts or results by default, because they can contain sensitive data. Detailed logging is an explicit, time-limited debugging option.

Caching

Caching applies to tool results per user and tenant, never globally, so one user’s cached answer cannot be served to another.

Failure modes

Covered failure modes include tools returning nothing, model timeouts, invalid tool arguments, and answers that fail validation. Each has a defined user-facing response rather than a raw error.

Alternatives considered

• Free-form text-to-SQL: flexible, but hard to secure and validate; it may be added later for read-only, sandboxed exploration.
• Retrieval only: works for documents, not for questions that need aggregation over structured data.
• A semantic layer as the only tool: a strong option when one already exists; the tool interface is designed to sit on top of one.

Common mistakes

• Giving the model a database connection string or free-form SQL execution.
• Applying the tenant filter in the prompt instead of in the query.
• Logging full prompts and results containing personal data.
• Caching answers globally across users.
• Treating a plausible-sounding answer as a validated one.

Next improvements

• Publish the tool layer and sample database.
• Add an evaluation report to the repository.
• Document a version on top of a Fabric SQL analytics endpoint.