Show HN: Microsoft releases Flint, a visualization language for AI agents

https://microsoft.github.io/flint-chart/#/

57•chenglong-hn•1h ago

Data visualizations are the bridge between user and data.

But building AI agents that can generate visualizations reliably can be very tricky:

- simple chart specs can be reliable, but generated charts are often of low quality due to reliance on system defaults; - complex chart specs with explicit details can produce good-looking charts, but they are verbose and agents can struggle with reliability

We figured out it is a limitation on the language issue (not just AI capability thing) -- current visualization languages are a bit too low-level for AI agents, requiring them to explicitly make visual decisions that are supposed to be handled by a good compiler. Flint is a visualization intermediate language to address this issue, allow AI agents to solve this last-mile human-agent interaction problem. It provides a simple semantic-type based specification, and contains a layout optimization engine that can produce good-looking charts (filled with derived low-level details) from simple high-level specs. The result is also very human understandable and adaptable. Flint powers data formulator for generating visualizations (another open source project from microsoft https://data-formulator.ai/).

Flint is available open source, and we built a MCP server that you can directly plug flint in your favorite agent app to play with data.

Comments

chenglong-hn•1h ago

Project page: https://microsoft.github.io/flint-chart/

MCP setup: https://microsoft.github.io/flint-chart/#/mcp

ietcd•1h ago

https://github.com/scicloj/kindly

but enterprise

theK•1h ago

> requiring them to explicitly make visual decisions that are supposed to be handled by a good compiler

Isnt graphviz there for the same reason?

Edit: I see it is using JSON as the declaration language, I am OK with llms being "good at json" but a syntax also consumable by humans it is not!

chenglong-hn•55m ago

In fact, Json as a common language for human in visualization has been around for a while! The benefit of declarative grammar is that users can effective manipulate specs through UI (drag and drop, clicks).

Btw, Flint is intentionally designed to allow agent skip low-level params like scale, axe, zero, step size etc (which are extremely crucial for "GOOD-looking") and they are dynamically optimized by the compiler. So AI agents can have a easier time.

theK•36m ago

> Json as a common language for human in visualization has been around for a while

Plant, Mermaid, Graphviz are all declarative textual representations designed for human authoring, JSON is made for tools. Its not a criticism just a statement that if interop across agent and human was intended this is not the simplest option.

chenglong-hn•20m ago

right, in fact many small models still struggle with following json, some new forms are also needed

animal_spirits•1h ago

It compiles into Echarts, but echarts already has a JSON co figuration spec

chenglong-hn•59m ago

It's more like a simple high-level spec to make it easier. The idea is that you don't have to fill position / axes details just to make the chart work. The compiler has a bit of magic of using semantic types to optimize what parameters will be set in ECharts.

In some composite chart examples, the good-looking echart spec is like 5x longer than the simple Flint one!

FailMore•1h ago

The charts are very nice, and I think the visualisation layer for LLMs is a very interesting problem.

I’ve been building https://smalldocs.org for this exact reason. It’s an office suite for AI agents - but my main use case is giving a cli based LLM the canvas to express itself - charts, mermaid diagrams, etc. I’ve extended it a bit further to be a format for all types of work so the agent can embed slides and spreadsheets in a document.

Sample document: https://smalldocs.org/blogs/what-is-a-smalldoc

Source: https://github.com/espressoplease/smalldocs

giancarlostoro•1h ago

> mermaid diagrams

I'm terrible at diagrams, so I gave GPT very generic descriptions of one of our project, to convert in to that mermaid style, then for Lucid I pasted it in there, and had a visualization of what I needed. Worked out nicely.

FailMore•28m ago

Yep I find them to be very useful for explaining a system.

They can do a lot of cool things! Mermaid gallery here: https://smalldocs.org/s/xZrc-lNW1kbXpoIuU3l_ky#k=-0ehGe2B-hR...

altmanaltman•22m ago

interesting how you don't discuss literally anything about the project actually posted and spam your thing. Not pointing you out, seen many other comments like this on HN but always felt a bit weird about them

neomantra•52m ago

This is cool to see from a research team. A few weeks ago I was exploring a similar idea with ntcharts, where a user or LLM can specify a chart in a Golang or JSON object...

and then that spec would be rendered either to a Bubble TUI via NTCharts or to HTML/SVG via ECharts. That Echarts HTML could be naturally served by a Golang http service.

But Flint goes much deeper with semantic layers and settings optimizations. Perhaps a NTChart, or whatever terminal chart, could be a rendering target? I'll add it to the list to explore...

https://github.com/NimbleMarkets/ntcharts/blob/spec/spec/REA...

chenglong-hn•34m ago

This is fun! We started thinking it would just be an engineering task in the beginning, but doing a solid intermediate language turned out to be a research project (the paper will be out soon).

Also, I find NTChart very fun, maybe we should add NT chart to the list of compilation backend for Flint so it works in the library. Putting a reminder here: https://github.com/microsoft/flint-chart/issues/45

santiagobasulto•42m ago

Forget AI agents, this DSL is better even for humans. Cool project!

chenglong-hn•24m ago

For AI agents and Humans :)

kveykva•39m ago

Is there a specific explanation about how this is better or different than vega itself? https://vega.github.io/vega/docs/specification/

My understanding is that Vega was already an expressive DSL for visualizations and its probably already well spread through LLM training data.

chenglong-hn•31m ago

Vega was a high-level language in the past for human, but now they can be a bit too low-level for AI agents! AI agents have to write a lot of low-level params just to make charts looking good, and the result is that programs are hard to write reliably for AI agents.

Flint is a higher-level abstraction, with simpler much shorter spec, and the compiler derives low-level decisions so that charts are looking good.

So: flint lets agent write short program that achieving good looking charts that had to be done with lengthy program in the past.

dvt•35m ago

This is pretty crazy, literally built something almost exactly like this for a project I'm working on (a local-first AI agent that does work on folders while you sleep). Basically going from JSON "Lego blocks" to full reports (including charting, though a subset of what Flint offers). And with post-generation validation and retry steps.

Functions extremely well and the result is a very clear (and consitent) human-readable "output layer." Cool idea, fun to see people converging on similar concepts in the space.

chenglong-hn•30m ago

That's awesome!

I find that besides training better models, designing new language for agents is also a super viable paths to improve their performance!

nrub•10m ago

> simple chart specs can be reliable, but generated charts are often of low quality due to reliance on system defaults; - complex chart specs with explicit details can produce good-looking charts, but they are verbose and agents can struggle with reliability

N of only a few of us working on an analytics agent, I don't think we've been finding this to be the case. We've been impressed with just how good LLMs (even smaller open weight models) are at using Python and R for visualization. Often any shortcomings go away if we iterate a bit to about ambiguity. Are there any threads of research that could better support this claim or highlight where issues might be?

What Do We Know About the Microplastics Inside Us?

Chatto is now Open Source

Mistral's Robostral Navigate: a state of the art robotics navigation model

SWE-1.7 Reach Near GPT 5.5 and Opus Intelligence

Show HN: Microsoft releases Flint, a visualization language for AI agents

Decoding the obfuscated bash script on a Uniqlo t-shirt

GPT‑Live

Grok 4.5

OpenBSD has a use-after-free allowing local privilege escalation to root

Cloudflare Meerkat - Globally distributed consensus

EU now one step away from reviving private message scanning rules

A bug which affected only left handed users

GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

Understanding B-Tree Indexes in PostgreSQL: A Comprehensive Guide– Part 1

TypeScript 7

Show HN: Follow London Trains in 3D

EVE Online's Carbon engine is now open source: Fenris Creations explains why

I Built a Telegram Client for Pi

Show HN: Agent Draw: An agent draws while you talk, built on TLDraw

TabFont – guitar tabs rendered as you type

PlayStation can delete all your digital games after 3 years of inactivity (EU)

Apple to increase spend with Broadcom to produce billions more U.S. chips

Japan's Hayabusa2 probe to conduct flyby of Torifune asteroid

How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

OpenMandriva: Statement regarding attempted distribution sabotage

NoiseLang: Where N = 5 is a Dirac delta

Geosql: A Claude/Codex skill for geospatial data

Tenda firmware (multiple versions) contains hidden authentication backdoor

Structure and Interpretation of Computer Programs Video Lectures (1986)

Copy That Floppy – Cambridge guide for preserving data from fragile floppy disks

What Do We Know About the Microplastics Inside Us?

Chatto is now Open Source

Mistral's Robostral Navigate: a state of the art robotics navigation model

SWE-1.7 Reach Near GPT 5.5 and Opus Intelligence

Show HN: Microsoft releases Flint, a visualization language for AI agents

Decoding the obfuscated bash script on a Uniqlo t-shirt

GPT‑Live

Grok 4.5

OpenBSD has a use-after-free allowing local privilege escalation to root

Cloudflare Meerkat - Globally distributed consensus

EU now one step away from reviving private message scanning rules

A bug which affected only left handed users

GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

Understanding B-Tree Indexes in PostgreSQL: A Comprehensive Guide– Part 1

TypeScript 7

Show HN: Follow London Trains in 3D

EVE Online's Carbon engine is now open source: Fenris Creations explains why

I Built a Telegram Client for Pi

Show HN: Agent Draw: An agent draws while you talk, built on TLDraw

TabFont – guitar tabs rendered as you type

PlayStation can delete all your digital games after 3 years of inactivity (EU)

Apple to increase spend with Broadcom to produce billions more U.S. chips

Japan's Hayabusa2 probe to conduct flyby of Torifune asteroid

How to Build a Minimal ZFS NAS Without Synology, QNAP, TrueNAS (2024)

OpenMandriva: Statement regarding attempted distribution sabotage

NoiseLang: Where N = 5 is a Dirac delta

Geosql: A Claude/Codex skill for geospatial data

Tenda firmware (multiple versions) contains hidden authentication backdoor

Structure and Interpretation of Computer Programs Video Lectures (1986)

Copy That Floppy – Cambridge guide for preserving data from fragile floppy disks

Show HN: Microsoft releases Flint, a visualization language for AI agents

Comments