Text-to-SQL vs. Semantic Layer

Ask an AI tool for last quarter's revenue and you'll get a number in seconds. The harder question is whether it matches the number finance reports, and whether you'd get the same number if you asked again tomorrow.

There are two ways an AI turns a business question into a query. With text-to-SQL, the LLM reads your schema and writes SQL from scratch. With a semantic layer, the LLM picks from metrics, dimensions and joins your team already defined, and a query engine writes the SQL. (Some vendors call the second approach text-to-semantics.)

LLMs write much better SQL than they did two years ago. dbt Labs' 2026 benchmark found text-to-SQL accuracy on its test set nearly doubled since 2023. So it's fair to ask whether modeling your data is still worth the effort.

The most useful way to compare the two approaches is by how each one fails. Text-to-SQL fails with a plausible wrong number. A semantic layer fails by telling you it can't answer. This article explains how each approach works, what the published benchmarks show, and how to decide which failure your use case can afford.

Key takeaways #

• Text-to-SQL asks an LLM to write SQL from your schema every time a question comes in.

• A semantic layer limits the LLM to choosing governed metrics, dimensions and filters, then compiles SQL deterministically.

• Text-to-SQL fails with plausible wrong numbers, while a semantic layer fails by declining questions outside its scope.

• On dbt Labs' modeled 2026 benchmark, semantic-layer accuracy reached 98.2% to 100%, versus 84.1% to 90.0% for text-to-SQL.

• Omni routes AI questions through its semantic layer and keeps raw SQL available for questions the model doesn't cover.

TL;DR #

Short answer: Use a semantic layer for any number that goes into a dashboard, a board deck or a customer-facing product, and keep text-to-SQL for one-off exploration. A semantic layer returns the same correct answer every time within its scope. Text-to-SQL answers more kinds of questions but can return wrong numbers that look right. Omni supports both in one tool, with the semantic layer as the default path for AI.

What is the difference between text-to-SQL and a semantic layer? #

Short answer: Text-to-SQL and a semantic layer differ in who writes the SQL. With text-to-SQL, an LLM writes the query from table and column names. With a semantic layer, the LLM selects predefined metrics, dimensions and filters. A query engine then generates the SQL. The semantic layer trades coverage for consistency.

How does text-to-SQL work? #

Short answer: Text-to-SQL gives an LLM your schema and a question. The LLM writes a SQL query from scratch, inferring what each table and column means from its name. Any question about existing data is fair game. Nothing checks whether the generated joins, filters or metric logic match how your business defines them.

Text-to-SQL is the default for most AI tools that connect straight to a warehouse. Point an assistant at your database, ask a question, and it writes and runs a query. On a small, cleanly named schema, that works well.

Text-to-SQL breaks in four predictable places as schemas get bigger and messier:

• Joins. A join that fans out (one row matching many) inflates totals. In Omni's 2026 benchmark, a coding agent hand-wrote a join through a membership table and reported a top campaign worth $55.5 billion in pipeline.

• Definitions. The schema doesn't say which column is revenue, or whether "active customer" means 30 days or 90. The LLM picks one, and nobody sees which.

• Table selection. Large warehouses hold staging tables, demo schemas and old copies. The LLM has no reliable way to know which one is canonical.

• Consistency. The LLM can write different SQL for the same question on different runs, so the answer can change.

Checkr saw the join and consistency problems firsthand. Before choosing a BI tool, its data team connected an LLM directly to the warehouse and found the same question returned different, incorrect answers each time. "For a question that required a join, the LLM didn't even try," said Sarah Fischbach, Staff Analytics Engineer, in Checkr's case study.

Schema size makes all four problems worse. The Spider 2.0 benchmark (published March 2025) uses enterprise-style databases that often exceed 1,000 columns. The authors' best agent, built on o1-preview, solved 21.3% of tasks, against 91.2% on the older Spider 1.0 benchmark. Leaderboard scores have climbed since, but the gap between clean academic schemas and real warehouses remains the core challenge for text-to-SQL.

How does a semantic layer change what the AI generates? #

Short answer: A semantic layer shrinks the LLM's job from writing SQL to choosing from definitions your team already approved. The LLM picks a metric, dimensions and filters. A query engine compiles them into SQL using the joins and logic stored in the model. If the LLM picks the right fields, the SQL is correct by construction.

This is what people mean by text-to-semantics. Take the question "How many active customers did we have in EMEA last month?" With a semantic layer, the LLM's output is a short selection: the Active Customers measure, the Region dimension filtered to EMEA, and a date filter for last month. The definition of "active," the join from customers to regions and the fiscal calendar all come from the model.

That shift changes three things:

• Answers repeat. The same field selection compiles to the same SQL every time, so two people asking the same question get the same number.

• Failures are visible. When the model has no field for what was asked, the semantic layer returns nothing rather than inventing a column. dbt Labs put it bluntly: "Text-to-SQL will cheerfully give you a wrong number."

• Permissions carry through. Row-level security and field access defined in the semantic layer apply to AI queries the same way they apply to dashboards.

The semantic layer only helps if the AI is required to use it. In Omni's 2026 benchmark, a coding agent given Omni's full semantic model and a written playbook still hand-wrote its own SQL and trailed Omni's agent by 21 points. Handing an LLM a semantic model as reference material is text-to-SQL with better documentation. The accuracy gains come from forcing the query through the engine.

Is a semantic layer more accurate than text-to-SQL? #

Short answer: Yes, for questions the semantic layer covers. In dbt Labs' April 2026 benchmark, a modeled semantic layer scored 98.2% with Claude Sonnet 4.6 and 100% with GPT-5.3 Codex. Text-to-SQL on the same modeled data scored 90.0% and 84.1%. Where the semantic layer lacked coverage, it declined to answer rather than guess.

Three published benchmarks compare the approaches directly. Each was run by a vendor with a stake in the result (Omni included), so read them as directional.

Two findings hold across all of them. First, modeling helps both approaches: dbt's text-to-SQL score jumped once three new dbt models were added. Second, the semantic layer's advantage is largest on the questions that matter most, the multi-table questions where a wrong join produces a believable wrong number.

The gap is also shrinking from the text-to-SQL side. dbt Labs measured overall text-to-SQL accuracy on its full question set climbing from 32.7% with GPT-4 in 2023 to 64.5% with 2026 models. Better LLMs raise the floor. They don't make a wrong answer look wrong.

How much does text-to-SQL cost compared with a semantic layer? #

Short answer: Text-to-SQL costs more per correct answer on large warehouses, mainly because agents explore the schema before answering. In Omni's 2026 benchmark, the alternative setups used 1 million to 6 million tokens per correct answer, against about 670,000 for Omni's agent. A semantic layer moves cost upfront, into building and maintaining the model.

Token cost comes from exploration. Without a model to route it, a text-to-SQL agent lists databases, inspects tables, runs test queries and retries. Every step re-reads everything before it, so cost climbs with each loop. In Omni's benchmark, one question sent the setup with no semantic layer through 60 warehouse queries, 489 seconds and 6.1 million tokens, and it never found the answer. Omni's agent answered correctly with one query in 34 seconds on 466,000 tokens.

Exploration also costs warehouse compute. Each test query an agent runs is billed by the warehouse, on top of the LLM's tokens.

Measure cost per correct answer rather than cost per query, since a cheap wrong answer still costs whatever decision it feeds. Omni's agent averaged $1.04 per correct answer against $1.29 for the next-best setup. That dollar gap, about 20%, is much smaller than the token gap, so check both.

Text-to-SQL is cheap on small schemas. When the whole schema fits in one prompt, the LLM writes one query in one pass. dbt Labs ran its benchmark that way and noted that loading the entire schema isn't practical for larger datasets.

A semantic layer has costs of its own. Model metadata goes into every prompt, and someone has to build and maintain the model. dbt Labs also found that higher reasoning settings slowed semantic-layer queries without improving accuracy: GPT models averaged over 20 seconds per query at the highest setting, against 8 seconds one level down.

When is text-to-SQL good enough? #

Short answer: Text-to-SQL is good enough for one-off exploration on small, well-named schemas, where the person asking can check the SQL. Text-to-SQL also suits analysts prototyping a question before modeling it. Text-to-SQL is risky for recurring metrics, executive reporting and anything automated, where a plausible wrong number can travel far before anyone notices.

The deciding factor is who reads the answer and whether they can check it. An analyst who reads SQL will catch a bad join. A sales VP looking at a chart in Slack won't.

Text-to-SQL fits when:

• The person asking can read the generated SQL and verify it.

• The question is one-off and won't become a recurring report.

• The schema is small, with a handful of clearly named tables.

• You're exploring new data that nobody has modeled yet.

Text-to-SQL is the wrong choice when:

• The number goes into a board deck, a forecast or a customer-facing product.

• Several teams ask the same question and need the same answer.

• An automated workflow acts on the result without a person reviewing it.

• The warehouse has hundreds of tables, overlapping copies or unclear column names.

dbt Labs lands in the same place. Its 2026 recommendation is to use a semantic layer where accuracy matters and fall back to text-to-SQL for ad hoc questions the semantic layer can't answer.

What are the limits of a semantic layer? #

Short answer: A semantic layer only answers questions its model covers. Building that model takes work, and new questions often need a new metric, join or field before AI can answer them. A semantic layer also does not fix bad source data. Teams that skip modeling get little benefit from one.

Coverage is the real cost. In dbt Labs' benchmark, the semantic layer scored 0% on questions its model couldn't express until someone added three new dbt models. Every question outside the model is either a modeling task or a fallback to SQL.

That cost is dropping. dbt Labs had an LLM write the three models that closed its coverage gap, and Snowflake built agents that suggest improvements to Cortex Analyst semantic models. AI can now draft much of the model, though someone who knows the business still has to review it.

Where the semantic layer lives also matters. A semantic layer that sits apart from the BI tool can drift from the dashboards people already trust. Omni covers that tradeoff in Your Semantic Layer Should Be Universal, Not Just Headless.

How does Omni combine a semantic layer with SQL? #

Short answer: Omni sends AI questions through its semantic layer by default. Omni's agent picks fields and filters from a Topic, a curated slice of the model. Omni's query engine then compiles the SQL. When a question falls outside the model, analysts can write raw SQL in the same workbook and promote useful logic back into the model.

Omni enforces the semantic layer instead of offering it as reference. The agent chooses what it needs from a Topic, and the query engine writes the SQL, so the agent never hand-writes the query for a governed question. That design is why Omni's agent answered 95 of 100 questions correctly in Omni's 2026 benchmark, while a coding agent holding the same semantic model trailed by 21 points.

Omni's query engine also handles the join problem that trips up text-to-SQL. Symmetric aggregates keep totals correct when a join fans out, without the person asking needing to know the table relationships.

SQL stays available for everything else. Omni lets analysts write raw SQL in a SQL tab, and Omni's docs state that when using raw SQL, Omni won't generate or override any SQL logic. Omni's Workbook Agent can also create or edit SQL tabs, though that option isn't available through the API, MCP or the Slack agent, or to topic-restricted users. Useful logic written in SQL can be promoted into the shared model, so the next person's AI question hits a governed definition.

The same governed path extends to outside assistants. Omni's MCP Server lets tools like Claude, ChatGPT and Cursor query through Omni's model, scoped to the model, the Topic and the asking user's permissions.

The tradeoff with Omni is the tradeoff with any semantic layer. Omni's AI is only as good as the Topics behind it, so teams need to model their core metrics before AI answers on them reliably. Omni's Evals feature helps here: teams run prompt sets of up to 25 questions against a model branch and see whether a modeling change helped before it ships. Checkr built its AI rollout around that loop and cut an operations question from hours to minutes, according to Checkr's case study.

How do you test which approach works on your data? #

Short answer: Test both approaches on about 25 real questions from your own team, with expected answers written in advance. Run each question at least twice. Score wrong-but-plausible answers separately from refusals, since only the first kind reaches a decision. Then choose an approach for each use case.

1. Collect about 25 questions people actually asked, from Slack threads, data request queues or recurring reports. Include several that need joins across three or more tables.

2. Write the correct answer for each question with the person who owns that metric, before running anything.

3. Run every question through text-to-SQL and through the semantic layer, against the same warehouse and data.

4. Run the full set at least twice and count how many answers change between runs.

5. Sort every result into correct, wrong but plausible, and refused or errored.

6. Record cost and time for each correct answer, not just accuracy.

7. Re-run the same set after every modeling change, so you can see whether coverage and accuracy improved.

Omni's benchmark article covers this method in more depth in How to run this benchmark on your own stack.

FAQ #

What is text-to-SQL? #

Text-to-SQL is an AI approach where an LLM turns a plain-language question into a SQL query by reading the database schema. The LLM writes the query from scratch each time. Text-to-SQL works on any data the LLM can see, but nothing checks whether its joins and metric logic match the business's definitions.

What is text-to-semantics? #

Text-to-semantics is an AI approach where an LLM answers a data question by choosing metrics, dimensions and filters from a semantic layer instead of writing SQL. A query engine then generates the SQL from those definitions. Text-to-semantics gives consistent answers for questions the semantic layer covers and declines questions it doesn't.

Is text-to-SQL accurate enough for business reporting? #

Text-to-SQL is usually not accurate enough for recurring business reporting on a large warehouse. In dbt Labs' 2026 benchmark, the best text-to-SQL result was 90.0% on a modeled project, which means roughly one wrong answer in ten. Those wrong answers looked like normal numbers, so a reader had no way to spot them.

How much more accurate is a semantic layer than text-to-SQL? #

A semantic layer is more accurate than text-to-SQL for questions it covers. dbt Labs' April 2026 benchmark found a modeled semantic layer reached 98.2% to 100% accuracy, versus 84.1% to 90.0% for text-to-SQL on the same data. Questions outside the semantic layer's scope get no answer instead of a wrong one.

Does a semantic layer guarantee correct AI answers? #

A semantic layer does not guarantee correct AI answers. The LLM can still pick the wrong metric, and the semantic layer is only as accurate as its definitions and source data. A semantic layer also only helps when the AI is required to query through it, which is why Omni compiles AI queries through its query engine.

Can you use text-to-SQL and a semantic layer together? #

Most teams should use text-to-SQL and a semantic layer together. The semantic layer handles governed metrics and recurring questions, and text-to-SQL handles one-off exploration outside the model. Omni supports both in one workbook, and useful SQL logic can be promoted into Omni's semantic model.

How does Omni answer a question its semantic layer can't cover? #

Omni's agent works from the Topics in Omni's semantic layer, so a question outside those Topics needs either a modeling change or SQL. Analysts can write raw SQL in an Omni SQL tab, and Omni's Workbook Agent can create SQL tabs in the workbook. The logic can then be promoted into the shared model for the next AI question.

Is text-to-SQL more expensive than a semantic layer? #

Text-to-SQL is usually more expensive per correct answer on large warehouses, because agents spend tokens and warehouse queries exploring the schema. In Omni's 2026 benchmark, alternative setups used 1 million to 6 million tokens per correct answer, against about 670,000 for Omni's agent with an enforced semantic layer. On small schemas that fit in one prompt, text-to-SQL can be cheap.

Methodology #

This article compares the two approaches on how each generates SQL, how each fails, and what published benchmarks show. Feature lists were left out on purpose. Benchmark figures come from the primary write-ups linked below and are quoted with the date and models tested. Every vendor benchmark cited was run by a company with a stake in the result, including Omni's, so the figures are directional rather than universal.

Sources

• Semantic Layer vs. Text-to-SQL: 2026 Benchmark Update, dbt Labs, April 7, 2026

• Agentic Semantic Model Improvement: Elevating Text-to-SQL Performance, Snowflake, March 31, 2025

• Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows, Lei et al., March 17, 2025

• How Accurate Are AI Analytics Agents? A 2026 Benchmark, Omni, October 22, 2026

• Checkr case study, Omni

• Omni docs: SQL generation, Symmetric aggregates, Building queries with AI, MCP Server, Evals