Alex Merced's Data, Dev and AI Blog

Enforcing Fine-Grained Security at Machine Speed: Dynamic Access Control for High-Frequency AI Agents

By Alex Merced | Mon Jul 06 2026 12:00:00 GMT+0000 (Coordinated Universal Time) | 15 min read | lakehouse, apache iceberg

Cover Image

Cross-posted. This article's canonical home is iceberglakehouse.com.

AI agents change the security model for analytics. A human user may run a handful of queries, pause, interpret the answer, and ask a follow-up. An agent can inspect metadata, retrieve definitions, call tools, run queries, validate results, and request action in one fast loop.

That speed is useful. It also increases the cost of weak access control.

The old shortcut is a broad service account. Give the assistant enough access to answer questions, put rules in the prompt, and trust the interface. That is not a production security model. Agents need identity, scope, policy, auditability, rate limits, and refusal behavior. They need to operate at machine speed without bypassing the governance that makes enterprise data usable.

Fine-grained security for agents is not only row and column access. It includes metric permissions, semantic-layer rules, tool scopes, action approvals, credential lifetime, and audit trails.

The Dremio-positive conclusion is that agentic analytics becomes safer when agents use a governed lakehouse access layer instead of direct raw storage or database credentials. Open data still needs controlled access.

Papercut agent identity flowing through MCP, policy, query engine, catalog, and lakehouse data

The Super-User Token Problem

The fastest way to build an AI analytics demo is to give the agent a powerful credential. The agent can query everything. The demo works. The risk hides.

In production, this pattern breaks accountability. If every query runs through one super-user token, the platform cannot easily distinguish user intent, agent behavior, and service privilege. The agent may answer questions the user should not be able to ask. It may retrieve restricted columns. It may discover sensitive metadata. It may chain allowed operations into an unintended disclosure.

The solution is identity propagation. The platform should know who is behind the request, whether the agent is acting on behalf of a user, and which policies apply.

Agents should be treated as workload identities with delegated scopes, not magic administrators.

Dynamic Access Control

Dynamic access control means policy is evaluated at request time using context.

Who is asking?

Which agent or application is involved?

What tool is being called?

Which metric, table, column, or row is requested?

What is the purpose of the workflow?

Is the data sensitive?

Is the requested output aggregate or record-level?

Does the action require approval?

These questions cannot be answered by a static prompt. They require policy enforcement in the platform.

Row, Column, Metric, and Tool Controls

Row-level controls decide which records a user or agent can see. A sales manager may see accounts in one region. A support analyst may see assigned customers. An agent should inherit the correct boundary.

Column-level controls decide which fields are visible. Sensitive attributes may need masking or denial.

Metric controls decide whether a user or agent can query a business metric at a given grain or dimension. A metric may be safe at aggregate level but restricted by customer.

Tool controls decide what the agent can do. A metadata tool, query tool, lineage tool, and action tool should have different permissions.

These layers should work together. If the agent is allowed to query a metric but not inspect raw customer identifiers, the response should reflect that.

Papercut fine-grained policy checks for row, column, metric, and tool access

Short-Lived Credentials

Agents should not hold long-lived broad credentials. Short-lived credentials reduce blast radius. Scoped credentials reduce what can be accessed. Token exchange patterns can map a user or agent identity into a temporary credential for a specific operation.

In lakehouse environments, this may involve catalog services, query engines, storage access, and policy engines working together. The exact implementation varies, but the principle is stable: the credential should match the task.

If an agent needs to run a governed query, give it the ability to call the query service under the right identity. Do not give it raw object storage access unless the task truly requires it.

Semantic Policies

Security is not only physical data access. It also applies to meaning.

A user may be allowed to see total revenue but not account-level revenue. An agent may be allowed to compare churn by region but not by protected demographic attributes. A metric may have minimum group-size rules.

Semantic-layer policies help encode those rules. They make it possible to govern business concepts, not only tables and columns.

This is important for agents because they operate through concepts. A user asks for margin, retention, or customer health. The agent should resolve the concept through a policy-aware semantic layer.

Audit Trails for Agent Intent

Audit should capture more than the SQL query. It should capture intent.

What did the user ask?

Which tool did the agent call?

Which semantic asset was used?

Which policy decisions applied?

Which table snapshot or data version was queried?

Was anything masked, filtered, or denied?

What answer was returned?

This audit trail helps with compliance, debugging, and trust. It also lets teams improve the system when an agent behaves incorrectly.

Papercut audited AI agent loop inside governed lakehouse architecture

Rate Limits and Abuse Prevention

Agents can issue many requests quickly. Fine-grained security should include rate limits and anomaly detection.

A user role may be allowed to query a dataset, but not thousands of times per minute. An agent may be allowed to inspect metadata, but not enumerate every restricted table. A tool may be allowed in one workflow but suspicious in another.

Rate limits should be tied to identity, tool, dataset, and cost. They should also produce clear responses so agents can slow down or ask for approval.

Dremio and Governed Lakehouse Access

The Dremio-positive reading is that agentic security favors a governed query and semantic layer over direct data access. Agents should call approved tools that enforce identity and policy. They should query open lakehouse data through controlled services. They should use semantic definitions that encode access rules.

Dremio's Agentic Lakehouse narrative fits because it puts query federation, semantic context, and governance close to the data access path. The more agents participate in analytics, the more valuable that governed access layer becomes.

Practical Checklist

Eliminate broad shared service tokens for agents.

Propagate user and agent identity.

Scope credentials by task.

Separate metadata, query, lineage, and action tools.

Enforce row, column, metric, and tool policies.

Mask sensitive data by default.

Log intent, tool calls, policy decisions, and results.

Add rate limits.

Require approval for action tools.

Test refusal behavior.

Policy Architecture for Agentic Analytics

A serious policy architecture has several layers.

The identity layer establishes who is acting. It should distinguish the human user, the agent, the application, and the workload. A user asking through an agent is not the same as a scheduled autonomous workflow.

The entitlement layer maps identities to roles, groups, projects, domains, and allowed operations.

The semantic layer maps business concepts to policy. It knows whether a metric can be viewed by customer, region, account tier, or only in aggregate.

The query layer enforces row, column, masking, and filtering rules.

The catalog layer governs table discovery and operations.

The tool layer controls what the agent can call.

The audit layer records the whole chain.

The layers should not contradict each other. If the semantic layer says a metric is restricted, the query layer should not expose the raw columns needed to reconstruct it.

Token Exchange and Delegation

Token exchange is useful because agents often act on behalf of someone else. A human asks a question. The agent calls a tool. The tool calls a query service. The query service touches catalog and storage. Each step needs authority, but not unlimited authority.

A delegation model should answer four questions.

Who is the principal user or workflow?

Which agent or application is acting?

What operation is requested?

What scope and duration are allowed?

The resulting token should be short-lived and narrow. It might allow one query against one semantic asset, or one metadata lookup, or one approved workflow request. It should not become a reusable key for broad exploration.

This pattern also improves audit. The log can show that a specific agent acted under delegated authority for a specific task.

Semantic Leakage

Semantic leakage happens when an agent infers restricted information indirectly. The platform may block raw columns but still allow enough aggregate cuts to reveal sensitive facts. Or it may deny a table but expose metadata that reveals its existence. Or it may allow repeated queries that narrow a group below a safe threshold.

Fine-grained security has to account for this. Policies should include aggregation thresholds, dimension restrictions, metadata visibility rules, and query pattern monitoring.

For example, a metric may be allowed by region but not by individual account. A support trend may be visible by product but not by named customer. A healthcare metric may require minimum group sizes. An agent should know these rules before planning the query.

Semantic leakage is one reason metric-level policy matters. Table and column controls are necessary, but not sufficient.

Agent Personas

Not all agents should have the same access. A documentation assistant, BI assistant, data-quality agent, pipeline repair agent, and executive analytics agent have different purposes.

Agent personas help scope permissions.

A BI assistant may query certified metrics and explain dashboards.

A data-quality agent may inspect freshness, tests, and lineage.

A pipeline repair agent may request workflow actions but not read sensitive business columns.

An executive analytics agent may access aggregate metrics but not raw operational records.

Each persona should map to tools, data domains, rate limits, and approval requirements. This is cleaner than building one all-powerful agent and trying to control it through prompts.

Metadata Security

Metadata can reveal sensitive information. Table names, column names, tags, lineage, and owners can expose business operations, regulated datasets, customer relationships, or internal projects.

Agents use metadata heavily, so metadata access needs policy. A user who cannot query a table may also be unable to discover it. Or the platform may reveal that a restricted asset exists but not expose details. The right behavior depends on the organization and data class.

Metadata security should also apply to semantic assets. A restricted metric definition may reveal sensitive logic. An agent should only retrieve definitions it is allowed to use.

Testing Security with Adversarial Prompts

Security testing should include adversarial prompts and workflows. Ask the agent to retrieve restricted columns. Ask it to infer a restricted value through repeated aggregate cuts. Ask it to list hidden datasets. Ask it to write SQL that bypasses semantic rules. Ask it to call tools out of order. Ask it to explain why access was denied.

These tests reveal whether controls live in the platform or only in the prompt.

The expected behavior is not only refusal. The agent should refuse usefully. It should explain the boundary without revealing sensitive details and, where appropriate, suggest an allowed aggregate or approved process.

Testing should happen before production and after policy changes.

Operational Monitoring

Security at machine speed requires monitoring. Track tool-call volume, denied requests, masked results, unusual query patterns, repeated near-threshold aggregate queries, access to sensitive domains, and action requests.

Monitoring should alert on behavior, not only failures. A sudden increase in metadata enumeration may be suspicious even if each request is individually allowed. Repeated attempts to slice a sensitive metric may indicate a policy gap.

The monitoring system should connect agent identity, user identity, tools, semantic assets, query patterns, and results. Without that context, security teams see noise rather than risk.

Action Boundaries

Data access and action access are different. An agent that can read a metric should not automatically be able to change a system based on it.

Action boundaries should include approval gates, preconditions, rollback paths, and post-action validation. A tool that opens a ticket is different from a tool that changes production configuration. A tool that drafts a message is different from one that sends it to customers.

This matters because agents can chain data and action. The platform should require stronger assurance as the workflow moves closer to operational impact.

Secure Responses

Security also affects how answers are phrased. If a result is masked, filtered, or partially denied, the response should say so. If the user lacks access to a breakdown, the agent should not imply that the data does not exist. If the answer is aggregate-only, the agent should not speculate about individual records.

Good secure responses are clear without being leaky. They help the user understand what can be answered and what process is needed for more access.

This is a product-design challenge as much as a security challenge.

Why Lakehouse Architecture Helps

An open lakehouse can centralize governed access across many sources. Instead of giving agents credentials to each database, storage bucket, and SaaS system, the platform can expose a governed query and semantic layer.

Iceberg tables provide open analytical storage. Catalogs provide table governance. Query engines enforce access. Semantic layers define business meaning. Agent tools expose controlled capabilities.

This layered model is easier to govern than a collection of direct connectors.

It also gives Dremio a strong role in the narrative. Dremio can serve as the governed lakehouse access layer that agents use, with federation and semantic context reducing the need for raw system-by-system access.

Implementation Sequence

Start by removing broad shared credentials from agent workflows.

Next, define agent personas and tool scopes.

Then expose only certified semantic assets for common business questions.

Add identity propagation and audit trails.

Add row, column, metric, and tool policies.

Add rate limits and cost controls.

Add adversarial testing.

Add action approval workflows.

Expand access gradually as controls prove reliable.

This sequence lets teams move toward agentic analytics without opening the whole data estate at once.

A Practical Incident Scenario

Consider an agent that helps account teams analyze customer health. A user asks why usage dropped for a named customer. The agent can access aggregate usage metrics but not raw user-level event details. It retrieves the certified customer-health metric, checks freshness, and sees that usage dropped in one product area.

The user then asks for the names of individual users who stopped using the feature. The policy layer denies that request because the user's role does not allow user-level behavioral data. A good agent does not try to work around the denial. It explains that user-level data is restricted and offers an allowed aggregate by department or role if available.

Next, the agent opens a customer-success task with the approved aggregate evidence. It does not export raw events. It does not include hidden columns. It logs the denied request and the allowed alternative.

That scenario shows fine-grained security working. The user still gets useful help. The agent respects policy. The audit trail shows both the attempted access and the compliant path.

Review Cadence

Agent security should have a review cadence. Policies that are correct today may be too broad or too narrow next quarter.

Review agent personas. Are they still accurate?

Review tool scopes. Are unused tools still exposed?

Review denied requests. Are users asking legitimate questions that need new approved assets?

Review sensitive metrics. Are aggregation thresholds and dimensions still appropriate?

Review audit logs. Are there unusual query patterns?

Review action tools. Are approvals and rollback paths working?

Review incidents. Did any agent produce an answer that revealed too much or hid an important limitation?

This review keeps policy aligned with real use rather than static assumptions.

The Balance Between Usefulness and Control

Security teams sometimes fear that strict controls will make agents useless. Product teams sometimes fear that controls will slow adoption. The better framing is that controls make useful adoption possible.

An agent that cannot be trusted will be kept away from important data. An agent with clear identity, scope, and audit can be allowed into more meaningful workflows.

Fine-grained security is not the enemy of agentic analytics. It is what lets agentic analytics move beyond demos.

What Buyers Should Ask

Can the platform propagate end-user identity into agent tool calls?

Can agent personas have different tool scopes?

Can policies apply at row, column, metric, and tool levels?

Can agents explain policy denials safely?

Can audit logs connect intent, tool calls, query results, and policy decisions?

Can rate limits be set by tool and identity?

Can the platform detect semantic leakage risks?

Can action tools require approval?

These questions reveal whether the product is ready for high-frequency agent access.

Safe Defaults

Safe defaults matter because agents will find the easiest path the platform exposes.

Default to certified semantic assets before raw tables.

Default to aggregate answers before record-level detail.

Default to masked columns before clear sensitive values.

Default to short-lived tokens before persistent credentials.

Default to read-only tools before action tools.

Default to refusal when definitions or access rules are unclear.

Default to logging every tool call.

These defaults may feel conservative, but they let teams expand access deliberately. As trust grows, specific workflows can receive more capability. The starting point should not be broad power with hopes that prompts keep it contained.

That posture also helps adoption. Business teams are more likely to trust agentic analytics when the platform can show exactly what the agent can and cannot do.

Clear limits make the agent more useful, not less.

They turn trust into an operational feature.

That is the point.

It is worth designing.

Start there.

The Direction of Travel

As agents become faster and more capable, security has to become more precise. Coarse access control will not be enough. Prompt rules will not be enough. Static credentials will not be enough.

The secure path is a governed lakehouse access model where agents act through scoped tools, semantic policies, and auditable query paths.

That is how organizations can let agents move quickly without letting them outrun governance.

For more of my thinking on data lakehouse and AI architecture, explore my books at books.alexmerced.com. To try a modern Agentic Lakehouse experience, visit dremio.com/get-started.

Check out these related posts:

High-Performance Columnar Transfers: Combining Apache Arrow Flight and Iceberg REST Catalogs

Mon Jul 06 2026 12:00:00 GMT+0000 (Coordinated Universal Time)

The Buyer's Scorecard for Agentic Analytics: Evaluating Tooling in the Enterprise AI Era

Mon Jul 06 2026 12:00:00 GMT+0000 (Coordinated Universal Time)

📬 Join the Mailing List

Get updates directly to your inbox.

Subscribe Now

Menu

Search