AI2SQL: The SQL runs, so why is the order total wrong?
Test duplicate amounts and an order with no line items before using generated PostgreSQL for reporting.
AI2SQL is worth comparing if analysts want a first draft of a PostgreSQL query, not if people unfamiliar with the metric need an unchecked report. Its official PostgreSQL guide describes generating queries from natural language. Correct dialect cannot decide whether a row is an order or a product line, or define revenue. For a stable metric, an approved query or semantic model usually costs less effort than regeneration.
Test duplicate amounts and a missing line
In a blank local test database, invent three orders: A and B are 100 each and C is 80. A has two item rows, B has one, C has none yet. Ask for all three orders, counting each order amount once and retaining the order with no items. The answer is three orders and 280. Use only invented schema and field descriptions for the PostgreSQL draft. Do not connect a real database or share real data dictionaries, customer rows or credentials. Compare with the team’s existing approach using read-only queries locally.

Original verification diagram. The inner join keeps the expected row count while changing which orders are counted.
Three plausible wrong answers
A left join to item rows then a sum counts A twice: four rows and 380. An inner join also loses C, giving three rows but 300. Summing distinct amounts does not fix it: even with all orders retained, two orders have the same 100 so only 180 remains. Fix the grain: if no item condition is needed, use the orders table. If filtering by product, start with one matching order ID per order and then take the order amounts.
Measure the reviewed result
Compare both paths on time to a reviewed result: drafting, checking grain, running the known-answer test and human review. Both the order count and total must be right, and the reviewer must explain why the empty order remains. Test again after giving A and B different amounts, without dropping the original duplicate-amount case. Consider AI2SQL for ad hoc drafting only if common variants pass, a reviewer owns it and total savings recur. Otherwise use the approved template.
Content note:This article is based on publicly available information and is intended as a reference for evaluating tools. Features, pricing, and related policies may change at any time; please refer to the latest official information.
Original source:ai2sql.io