built this after watching sql agents silently return wrong data. queries run fine, no errors - but the results are just wrong. syntax checkers don't catch it.
turns out db engine devs have the same problem testing query planners. they use metamorphic testing - generate equivalent queries that must return the same results, then check if they do. sqlancer found 400+ bugs in postgres/mysql/sqlite this way.
so i applied the same techniques (TLP, NoREC) to verify llm-generated sql at runtime. also analyzes EXPLAIN plans for perf issues.
trying to give sql agents a real feedback loop instead of just generating and hoping. curious if anyone else is working on this or tried different approaches.
yudduy•37m ago
turns out db engine devs have the same problem testing query planners. they use metamorphic testing - generate equivalent queries that must return the same results, then check if they do. sqlancer found 400+ bugs in postgres/mysql/sqlite this way.
so i applied the same techniques (TLP, NoREC) to verify llm-generated sql at runtime. also analyzes EXPLAIN plans for perf issues.
trying to give sql agents a real feedback loop instead of just generating and hoping. curious if anyone else is working on this or tried different approaches.