Closer to production quality Python notebooks with `marimo check`

31•dmadisetti•4mo ago

Comments

dmadisetti•4mo ago

marimo notebooks double as python Python programs, which means they can be lint-ed like any other code. `marimo check` catches notebook-specific issues (variable redefinition across cells, circular dependencies) with Rust-inspired (and uv; uv is amazing) error messages. It outputs JSON for AI agents to self-correct, integrates into CI pipelines, and includes `--fix` flags for automatic repairs. The team is already using it in their own CI and seeing Claude Code iterate on notebooks without human intervention.

aitchnyu•3mo ago

IME Agents work fine on human readable error messages.

sixo•3mo ago

Unfortunately the code in Marimo notebooks implements cross-variable references as untyped function arguments, meaning that nothing passed between cells can be type-safe. This makes it very hard to use tooling directly on the Python representation of notebooks.

dmadisetti•3mo ago

Arguments are typed if explicitly specified in definition, e.g.

    ```python
    @app.cell
    def my_cell():
        x : int = 1
        return (x,)

    @app.cell
    def consumer(x : int):
        y = x
        return
    ```

We've talked about building out implicit typing into our serialization- but that may be a larger can of worms.

sixo•3mo ago

That would be wonderful. A lot of the arguments are just imports, and these would have probably have the largest upside. It would be great if there was a construct like

    import numpy

    @app.cell
    def my_cell(np: types.ModuleType[numpy]):
        ... editor treats np like `import numpy as np` ...

I do not use Python enough to know if something like this can be hacked together. But if not, I imagine could be PIPed into the standard library if a convincing case were made for it.

dmadisetti•3mo ago

We've addressed this too by allowing a setup cell:

    ...

    with app.setup:
        import numpy as np
    
    @app.cell
    def cell_uses_np():
        # no np needed in signature
        result = np....

The best part about this pattern is that it enables "pure" (only dependent on setup cell) functions to be addressable:

    with app.setup:
        from typing import Optional, Any

    @app.function
    def my_fn(arg: Optional[Any]) -> Any:
        ...

but nice convergent design :) glad to see to see we're addressing pain points

sixo•3mo ago

ah, I recall seeing that feature but had not taken a look at it for imports.

Imports ARE the case where untyped cell arguments are most annoying, but of course it would be nice to get it for free in all cases.

wodenokoto•3mo ago

I’m very interested in moving beyond Jupyter notebooks so I have my eye on marimo.

My understanding is it’s just python code with a bit of notebook hints and an idea that gives the notebook experience.

So I didn’t quite catch from the article why ruff and pylons aren’t enough.

akshayka•3mo ago

Hi! Thanks for your interest. marimo is much more than that — unlike traditional notebooks, marimo is "reactive", meaning it models notebooks as dataflow graphs and keeps code an outputs in sync. Moreover, marimo notebooks are not "just notebooks". They can be seamlessly run as interactive web apps or as Python scripts.

Here is our original ShowHN post that explains what marimo is all about: https://news.ycombinator.com/item?id=38971966

Blog that goes deeper: https://marimo.io/blog/lessons-learned

daxfohl•3mo ago

I was just thinking about something like this a couple hours ago.

Jeremy Howard (fast.ai, etc) posted a few weeks ago about his new AI class [1] and I was pumped. I went through the fast.ai course a couple years ago and had been wondering if another would ever come out, and the overview of this sounded really interesting. But the first thing the announcement went into was Jupyter Notebooks, and it kept on and on about them and I rolled my eyes. I remember they were my least favorite part of the fast.ai class. They've always felt awkward to me, being used to a code editor and all the functionality it provides, and I'm sure I'm not alone.

I'm sure Jupyter is great for fast iteration if that's what you do all day, but if you spend most of your life in a standard code editor, you really want something that starts from there and adds a bit of interactivity on top, rather than jumping to a completely different concept. I haven't tried marimo yet, but I might give it a shot for my next project.

[1] https://news.ycombinator.com/item?id=45455810

spwa4•3mo ago

Just use VSCode, which will give you full language server editing in ipynb files?

daxfohl•3mo ago

Maybe. At least that way it doesn't feel like some separate half-baked proprietary code editor from the 90s, which is the vibe I get from Jupyter. But a regular .py file with a couple attributes that can generate notebooks still sounds far nicer, assuming it works fairly seamlessly.

ravila4•3mo ago

I haven’t used it, but from looking at Marimo’s examples and docs, I’m not convinced by some of its design choices. The idea that you can run notebook cells out of order is supposed to be a strength, but I actually see it as a weakness. The order of cells is what makes a notebook readable and self-documenting. The discipline of keeping cells in order may be painful, but it’s what makes the flow of analysis understandable to others.

Also, I find the way Marimo uses decorators and functions for defining cells pretty awkward (Although it’s nicely abstracted away in the UI). It looks like normal Python, but the functions don’t behave like real functions, and decorators are a fairly advanced feature that most beginners don’t use.

For me, Quarto notebooks strike a better balance when it comes to generating sharable documents, prototypes, and reports. They’re git-friendly, use simple markup for defining cells, and still keep the clear, linear structure.

However, Marimo might be the best tool for replacing Streamlit apps and “production notebooks” (Although I’d also argue that notebooks should not be in production).

dmadisetti•3mo ago

marimo has a quarto extension and a markdown fileformat [1] (marimo check works on this too!). The python fileformat was chosen such that "notebooks" are still valid python, but yes- the format itself is almost an implementation detail to most "notebook" users. Cells _are_ actually callable and importable functions though (you can give them a name), but the return signature is a bit different from what's serialized.

> The discipline of keeping cells in order may be painful, but it’s what makes the flow of analysis understandable to others.

We might have to agree to disagree here, you can still chose to have your notebook in order and something you can be disciplined about. The difference is that a marimo notebook can't become unreproducible the same way a jupyter notebook can, _because_ the order doesn't matter.

But thanks for the feedback!

[1]: https://github.com/marimo-team/quarto-marimo

loveparade•3mo ago

I wish this didn't have AI in it. I've been looking for a Jupyter alternative that is pure python and can be modified from a regular text editor. Jupytext works okay, but I miss the advanced Jupyter features. But I really don't want to deal with yet another AI assistant, especially not a custom one when I'm already using Claude/etc from the CLI and I want those agents to help me edit the notebooks.

Take out all the AI stuff and I'd give it a try. I use AI coding agents as my daily driver, but I really don't need this AI enshittification in every tool/library I'm using.

3eb7988a1663•3mo ago

I hate how much I lean into VSCode, but the Python interactive mode gets you a really good live coding environment. Instead of Jupyter cells, you have a regular .py file with chunks of code prefixed with a `# %%`. VSCode gives you a similar experience to a notebook, with the same controls (Run Above Cells, Restart and Run All, etc). So something like

  # %%
  import polars as pl

  # %%
  df = pl.DataFrame()
  df.shape

  # %%
  def foobar():
    return 1

Since it is a regular .py file all of your existing tooling will work with it. The one thing you lose vs a Jupyter notebook is saved output. I mostly use these .py files, but have a few .ipynb notebook files for when I want to commit the output from some important task.

akshayka•3mo ago

You can not use the AI features, nothing is enabled by default (you have to bring your own keys)

daxfohl•3mo ago

Reading the article, I don't think it has AI. They've just made the tools in a way that AI assistants can also use them, and so fix linting errors without anyone needing to fine-tune the LLM on the syntax.

That's actually pretty slick. I've been wondering how we could avoid blocking innovation in programming languages because of the death cycle of "no training data on language -> LLM can't learn language -> Assistant can't code language -> nobody uses language -> no training data on language".

loveparade•3mo ago

Yeah, reading the docs it seems you are right. The landing page mentions AI-native at the very top and all over the place, so I got the wrong impression that it's somehow tightly coupled to an AI integration. But looks like it's optional.

dmadisetti•3mo ago

> But I really don't want to deal with yet another AI assistant, especially not a custom one when I'm already using Claude/etc from the CLI and I want those agents to help me edit the notebooks.

So funny story- you can use exactly the same CLI tools in your notebook. Zed built out the ACP spec [1] which lets Claude Code go anywhere that implements it (as of Oct 2nd; emacs, vim, zed and marimo [2])

[1]: https://github.com/agentclientprotocol/agent-client-protocol

[2]: https://zed.dev/blog/acp-progress-report

ZeroCool2u•3mo ago

Any updates on the revamped VS Code extension? That's kind of a deal breaker for me.

dmadisetti•3mo ago

Soft launched in pre-release, but a full version is coming!

Haleshot•3mo ago

v.07 is live now on both VS Code and Open VSX marketplaces. https://github.com/marimo-team/marimo-lsp/

Al Lowe on model trains, funny deaths and working with Disney

Hoot: Scheme on WebAssembly

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

Stories from 25 Years of Software Development

The AI boom is causing shortages everywhere else

Reinforcement Learning from Human Feedback

The Waymo World Model

Start all of your commands with a comma (2009)

U.S. Jobs Disappear at Fastest January Pace Since Great Recession

Vocal Guide – belt sing without killing yourself

Selection Rather Than Prediction

Speed up responses with fast mode

France's homegrown open source online office suite

Coding agents have replaced every framework I used

A Fresh Look at IBM 3270 Information Display System

72M Points of Interest

Unseen Footage of Atari Battlezone Arcade Cabinet Production

Where did all the starships go?

Software factories and the agentic moment

Show HN: Kappal – CLI to Run Docker Compose YML on Kubernetes for Local Dev

Show HN: Look Ma, No Linux: Shell, App Installer, Vi, Cc on ESP32-S3 / BreezyBox

Learning from context is harder than we thought

Monty: A minimal, secure Python interpreter written in Rust for use by AI

Making geo joins faster with H3 indexes

Hackers (1995) Animated Experience

Sheldon Brown's Bicycle Technical Info

An Update on Heroku

Ga68, a GNU Algol 68 Compiler

Show HN: If you lose your memory, how to regain access to your computer?

Show HN: I spent 4 years building a UI design tool with only the features I use

Al Lowe on model trains, funny deaths and working with Disney

Hoot: Scheme on WebAssembly

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

Stories from 25 Years of Software Development

The AI boom is causing shortages everywhere else

Reinforcement Learning from Human Feedback

The Waymo World Model

Start all of your commands with a comma (2009)

U.S. Jobs Disappear at Fastest January Pace Since Great Recession

Vocal Guide – belt sing without killing yourself

Selection Rather Than Prediction

Speed up responses with fast mode

France's homegrown open source online office suite

Coding agents have replaced every framework I used

A Fresh Look at IBM 3270 Information Display System

72M Points of Interest

Unseen Footage of Atari Battlezone Arcade Cabinet Production

Where did all the starships go?

Software factories and the agentic moment

Show HN: Kappal – CLI to Run Docker Compose YML on Kubernetes for Local Dev

Show HN: Look Ma, No Linux: Shell, App Installer, Vi, Cc on ESP32-S3 / BreezyBox

Learning from context is harder than we thought

Monty: A minimal, secure Python interpreter written in Rust for use by AI

Making geo joins faster with H3 indexes

Hackers (1995) Animated Experience

Sheldon Brown's Bicycle Technical Info

An Update on Heroku

Ga68, a GNU Algol 68 Compiler

Show HN: If you lose your memory, how to regain access to your computer?

Show HN: I spent 4 years building a UI design tool with only the features I use

Closer to production quality Python notebooks with `marimo check`

Comments