uvx ty check uv tool install ty
ty check uv tool run ty
cd /tmp
git clone https://github.com/simonw/sqlite-utils
cd sqlite-utils
uvx ty check
Adding "time uvx ty check" shows it took:
uvx ty check 0.18s user 0.07s system 228% cpu 0.109 totalTy: 2.5 seconds, 1599 diagnostics, almost all of which are false positives
Pyright: 13.6 seconds, 10 errors, all of which are actually real errors
There's plenty of potential here, but Ty's type inference is just not as sophisticated as Pyright's at this time. That's not surprising given it hasn't even been released yet.
Whether Ty will still perform so much faster once all of Pyright's type inference abilities have been matched or implemented - well, that remains to be seen.
Pyright runs on Node, so I would expect it to be a little slower than Ty, but perhaps not by very much, since modern JS engines are already quite fast and perform within a factor of ~2-3x of Rust. That said, I'm rooting for Ty here, since even a 2-3x performance boost would be useful.
time uvx mypy .
uvx mypy . 0.46s user 0.09s system 74% cpu 0.740 total
0.56s user 0.14s system 302% cpu 0.231 total
Damn that's fast. And zero issues in this project with pyright so curious that these are...
Carl, you did not tell me that the errors we emit need to be correct!!
I think I need to go revisit some of my PRs...
"An age" is probably an attempted cleaning-up of "a coon's age."
Not making any sort of ethical statement, just interesting that rust keeps eating the python and JS tooling worlds.
https://github.com/facebook/pyrefly/blob/a8626110da034f8e513...
Astral announced they were building a typechecker back in January: https://x.com/charliermarsh/status/1884651482009477368
(However, vc-backed astral probably need control over theirs to keep monetization options open, and Facebook probably need control over theirs so it can be targeted at Facebook's internal cool-but-non-standard python habits... Sigh.
Why do we have nice things? Money. Why can't we have nice things? Also money.)
Ruff's linting and formatting is more likely to get plugin/extension support at some point in the future.
Ruff is a linter which (intentionally) does close to no type checking.
So you pretty much have to pair it up with a type check to get any even just half way decent static code analysis.
The reason we're stuck on mypy at work is because it's the only type checker that has a plugin for Django that properly manages to type check its crazy runtime generated methods.
I wish more python tooling took the TS approach of "what's in the wild IS the language", as opposed to a "we only typecheck the constructs we think you SHOULD be using".
Or in this case, writing it in Rust...
mypy is written in Python. People have forgotten that Python is really, really slow for CPU-intensive operations. Python's performance may not matter when you're writing web service code and the bottlenecks are database I/O and network calls, but for a tool that's loading up files, parsing into an AST, etc, it's no surprise that Rust/C/even Go would be an order of magnitude or two faster than Python.
uv and ruff have been fantastic for me. ty is definitely not production ready (I see several bizarre issues on a test codebase, such as claiming `datetime.UTC` doesn't exist) but I trust that Astral will match the "crazy reality" of real Python (which I agree, is very crazy).
Currently we default to our oldest supported Python version, in which `datetime.UTC` really doesn't exist! Use `--python-version 3.12` on the CLI, or add a `ty.toml` with e.g.
``` [environment] python-version = "3.12" ```
And we'll find `datetime.UTC`.
We've discussed that this is probably the wrong default, and plan to change it.
If I saw something like "datetime.UTC doesn't exist", I'd immediately think "wait, was that datetime.utc", not "ooh it got added in 3.11, I need to change my Python version"
Is there any big downside to do it the boring way, hardcode a list and compare the error to the list?
This is a known issue — we're currently defaulting to a conservative Python version, and `datetime.UTC` really doesn't exist until Python 3.11!
https://docs.python.org/3/library/datetime.html#datetime.UTC
We will probably change the default to "most recent supported Python version", but as mentioned elsewhere, this is very early and we're still working out these kinds of kinks!
There are some extremely CPU-intensive low-level operations that you can easily write in C and expose as a Python API, like what Numpy and Pandas do. You can then write really efficient algorithms in pure Python. As long as those low-level operations are fast, those Python-only algorithms will also be fast.
I don't think this is necessarily "cheating" or "just calling disguised C functions." As an example, you can write an efficient linear regression algorithm with Numpy, even though there's nothing in Numpy that supports linear regression specifically, it's just one of the ways a Python programmer can arrange Numpy's low-level primitives. If you invent some new numerical algorithm to solve some esoteric problem in chemistry, you may be able to implement it efficiently in Python too, even if you're literally the first person ever writing it in any language.
The actual problem is that it's hard for people to get an intuition of which Python operations can be made fast and which can't, AST and file manipulation are sadly in the latter group.
This gives somewhat counterintuitive results where declaring and summing a whole list of integers in memory can be faster than a simple for loop with an iterator.
But yeah writing stuff in a different (compiled) language is often better if that means the python interpreter doesn't need to go through as many steps.
and how you use the type annotations to indicate a type system is inconsistent and incomplete (e.g. NoneType vs. None for inconsistency and a lot of mess related to mataclasses (e.g. Enum) and supporting type annotations for them for incomplete)
the fact that even today something as fundamental as enums have issues with type checking _which are not just type checker incompetence_ is I think a good way to highlight what mess it is
or that `Annotated[]` was only added in 3.9 and has a ton of visual overhead even through its essential for a lot of clean definitions in modern python code (where for backwards compatibility there is often some other way, which can be de-facto wrongly typed but shouldn't be type linted, have fun type checkers).
TS might transpile to JS and can always be split into a js and type annotation file but is it's own language developed in tandem with the type check based on a holistisch approach to find how to type check then and then put it into the syntax and type checker.
Thats not true for python at all.
Python types where added as annotations to the language many years ago, but not in a holistic approach but in simplistic approach only adding some fundamental support and then extended bit by bit over the years (and not always consistently).
Furthermore this annotations are not limited to type checking which can confuse a type checker (through Annotated helps a lot, but is also verbose, wonder how long until there is a "Annotated" short syntax e.g. by impl @ on type or similar).
Basically what the type annotation feature was initially intended to be and what it is now differ quite a bit (dump example `list` vs. `List`, `Annotated` etc.).
This is made worse that a bunch of "magic" is deeply rooted in python, e.g. sub-classing `Enum`. Sure you have that in JS too, and it also doesn't work that well in TS (if you don't add annotation on the dynamically produced type).
Lastly TS is structurally typed, which allows handling a bunch of dynamic typing edge cases, while Python is, well, in-between. Duck-typing is (simplified) structural typing but `isinstance` is a common thing in Python and is nominal typing...
So yeah it's a mess in python and to make it worse there bunch of annoyances related to ambiguity or to many ways how to do a thing (e.g. re-exports+private modules, you can do that common coding pattern, but it sucks badly).
It doesn't actually preserve typing on the protocol's methods though
Rust devs in particular are on a bend to replace all other languages by stealth, which is both obviously visible and annoying, because they ignore what they don't know about the ecosystem they choose to target. As cool as some of the tools written for Python in Rust are (ruff, uv) they are not a replacement for Python. They don't even solve some annoying problems that we have workarounds for. Sometimes they create new ones. Case in point is uv, which offers custom Docker images. Hello? A package manager is not supposed to determine the base Docker image or Python version for the project. It's a tool, not even an essential one since we have others, so know your place. As much as I appreciate some of the performance gains I do not appreciate the false narratives spread by some Rust devs about the end of Python/JavaScript/Golang based on the fact that Rust allowed them to introduce faster build tools into other programming languages' build chains. Rust community is quickly evolving into the friends you are embarrassed to have, a bit like any JVM-based language that suddenly has a bunch of Enterprise Java guys showing up to a Kotlin party and telling everyone "we can be like Python too...".
Pydantic being so fast because it's written in Rust is a good thing, you can do crazy dynamic (de-)serializations everywhere with very little performance penalty.
Or how make a wrapper function with args and kwargs to pass through?
[1]: https://docs.python.org/3/library/typing.html#typing.datacla...
Nah, that's just part of the parade of excuses that comes out any time existing software solutions get smoked by a newcomer in performance, or when existing software gets more slow and bloated.
Here's one of many examples:
https://m.youtube.com/watch?v=GC-0tCy4P1U&pp=0gcJCdgAo7VqN5t...
not because they don't want to or because it's to slow
but because it's not really viable without fully executing module loading in a sandbox, which might seem viable until you realize that you still need to type check `__main__` modules etc. and that its a common trend in python to do configs by loading python modules and grabbing the module locals as keys of the config or loading some things might actually idk. initialize a GPU driver :sob: So it's kinda 100% guaranteed not possible to do fully correct type checking for all project :smh:
But also python is one of the slowest popular languages (and with a large margin to any not also "one of slowest" languages). Only by moving hot code into C++/Rust is it fast, which often is good enough, but a type checker is exactly this kind of software where this approach stops working.
And not directly related, but I wish more python modules did proper checks with Valgrind before shipping.
1. use rr for debugging your binary wheel, you can set up watchpoints, and reverse step/continue from the segfault.
2. compile and run your wheel with sanitizers (ASAN, UBSAN).
I rarely use valgrind so I can't comment on that.
(And uv and ruff have basically proved that at this point)
which brings us to another python issue, python is quite bad at such huge refactoring even with type checkers
but yeah python is by far the slowest widely used language, and for some use cases you can side step it by placing most hot code in C++/Rust extension modules, (or don't care because you are much much more network latency bound) but a type checker probably doesn't belong into that category
With that being said, the worst case scenario is that they go caput, but that still leaves the python community with a set of incredible new rust-based tools. So definitely a net win for the python community either way!
Talk is cheap, and people talk a lot about supporting projects.
Maybe if we could make some kind of statistics over the number of projects that were abandoned because maintainers didn't feel like working and dealing with random people for free anymore. Make the consequences of freeloading visible somehow.
Make like, a week long "holiday" where you either verify that your company has made an adequate donation to the OSS maintainers that make their products possible, or we all just go on strike for that week. Or... something. I'm sure somebody has a better idea than mine, lets get creative.
> I'm sure somebody has a better idea than mine, lets get creative.
Every creative scheme I've seen someone try to come up with fails to do what charging money for a product can. Charge money for stuff, have a free tier, enjoy sustainable software.
There's also a bunch of cases where adding tiers and payment flows blows the complexity budget and now what used to be a good idea is no longer worth it.
But I wouldn't mind being compensated for sharing what I create by those who find it useful.
Looks like you found the not-so-secret repository we're using to prepare for a broader announcement :)
Please be aware this is pre-alpha software. The current version is 0.0.0a6 and the releases so far are all in service of validating our release process. We're excited to get this in people's hands, but want to set the expectation that we still have a lot of work left to do before this is production ready.
Stay tuned for more for news in the near future!
(... I work at Astral)
For years I pushed black for formatting code. Once formatting was baked into ruff I ditched black. Having fewer dependencies to track and update simplifies my life and shortens my dependabot queue.
(where "not otherwise typecheckable" means types that can't be expressed with stubs - e.g., Django, dataclasses pre-PEP-681, pytest fixtures, etc.)
(It's also more difficult to support plugins effectively in a type checker like ty, than in a linter like ruff, since any non-trivial use case would likely require deep changes to how we represent types and how we implement type inference. That's not something that lends itself to a couple of simple hook extension points.)
Considering how fast uv and ruff took off, I am sure you are aware of the impact your project could have. I understand that supporting plugins is hard. However, if you are considering adding support for some popular libraries, IMHO, it would be really beneficial for the community if you could evaluate the feasibility of implementing things in a somewhat generic way, which could be then maybe leveraged by third-party authors.
In any case, thanks for all the amazing work.
I don’t have any such experience (short of a macro system, which requires code generation or runtime support) and it always makes me curious when people ask for type system plugins whether this is a standard feature in a type system I’ve never used.
Is this all based off a spec that python provides? If so, what does that look like?
Or do you "recode" the python language in rust, then use rust features to parse the python files?
Regardless of how it's done - This is a really fascinating project, and I'm really glad you guys are doing it!
https://github.com/python/cpython/blob/main/Parser/Python.as...
ty uses the same AST and parser as ruff. We don't use the ASDL grammar directly, because we store a few syntax nodes differently internally than how they're represented upstream. Our parser is hand-written in Rust. At first, our AST was also entirely hand-written, though we're moving in the direction of auto-generating more of it from a declarative grammar.
https://github.com/astral-sh/ruff/issues/15655
https://github.com/astral-sh/ruff/tree/main/crates/ruff_pyth...
https://github.com/astral-sh/ruff/blob/main/crates/ruff_pyth...
CPython uses a generated parser. The grammar is defined in https://github.com/python/cpython/blob/main/Grammar/python.g... which is used to generate the specification at https://docs.python.org/3/reference/grammar.html#full-gramma...
We use a hand-written parser, in Rust, based on the specification. We've written that previously at https://astral.sh/blog/ruff-v0.4.0#a-hand-written-parser
No, they are correctly using semantic versioning to indicate pre-alpha releases. https://github.com/astral-sh/ty/releases https://semver.org/
What am I missing here?
For example, as my recent struggles showed, SQLAlchemy breaks `pyright` in all kinds of ways. Compared with how other 'dynamic' ORMs like Prisma interact with types, it's just a disaster and makes type checking applications that use it almost pointless.
How does Ty play with SQLAlchemy?
I don’t know about SQLAlchemy, but for libraries like pandas I just don’t see how it can be done, and so people are actively replacing them with modern typed alternatives
I love how Python makes me so much faster due to its dynamic nature! Move fast, break things!
Ty could solve this if they rebel and decide to ignore the Python typing standards, which I honestly would appreciate, but if they take the sensible approach and follow the standards, it won't change anything.
I've been typing them with TypedDict for a while now and it's been fine. What can't you do?
I loathe the Python convention of just using kwargs instead of clearly annotated parameters; most libraries don't even have doc comments in the code, so you're really required to look up the documentation, hope that it actually describes the method you're interested in and contains more than stuff like "foo: the foo to use"—or fall back to rummaging in the library intestines to figure out how it works.
It's pathetic.
On a more serious note, I can't even blame library devs as long as they try. Type "hints" often are anything but _just_ hints. Some are expected to be statically checked; some may alter runtime behavior (e.g. the @overload decorator). It's like the anti-pattern of TypeScript's enums laid out here and there, and it's even harder to notice such side-effects in Python.
Maybe you're in a niche spot, or using scientist-based code. I've seen plenty of trainwrecks in 'conda-only' ""libraries"" done by scientists. Maybe that's the niche you're at?
If you know the SQL you want it's just a matter of writing it in SQLAlchemy's query language which is quite close to SQL. Should just be a matter of practice to become fluent in it. "Complex queries" usually turn up when you're doing something like rendering a table or report or something. You don't need the ORM for this kind of thing, just write a query.
An ORM is useful when you want to write domain logic to do read/write operations against domain entities and persist them back to a database. IMO people get hung up on ORMs and think if they're using one then they have to use it for everything then do the most horrible contortions that should have just been db queries. SQLAlchemy allows you to use the ORM judiciously.
Good documentation should absolutely provide a usable reference to quickly look up common ways to solve common problems. Even the PHP docs got that right twenty years ago.
Also, I disagree: A library should be as self-evident and incrementally understandable as possible, not require reading a full tome and grow a grey beard before being accessible.
> "Complex queries" usually turn up when you're doing something like rendering a table or report or something. You don't need the ORM for this kind of thing, just write a query.
Or, when building generic filtering/sorting/pagination logic for a bog-standard CRUD app. Or to do full-text search. Or when doing lateral joins to minimize queries. Or to iterate over a huge table. There's lots of cases where I want the ergonomics and malleability of ORM query instances even when working with complex queries.
in python eco system you have linters like ruff which do hardly any type checking and type checkers like mypy which do try to approach complete type checking, but still are absurdly full of holes
2. speed
any of the "established" type checkers either are supper slow (e.g. mypy) so you only run it like once before commit instead of "life" or do fail to properly type check so many things that if you have a requirement for "adequate static code analysis" they reliably fail that requirement (which might result in a legal liability, but even if not is supper bad for reliable code and long term maintenance)
also probably priorities are switched with 1st speed then closing holes as the later part is really hard due to how a mess python typing is (but in many code bases most code won't run into this holes so it's okay, well except if you idk. use pyalchemy as "ORM" subclassing base model (just don't terrible idea)).
Thanks for all your great work! Love ruff, rye/uv.
Pylance is borked on these forked distributions, so having a new solid alternative here that doesn't involve adopting yet another forked Pyright implementation (BasedPyright, Cursor Pyright, Windsurf Pyright, ...) sounds great to me.
> basedpyright re-implements many features exclusive to pylance - microsoft's closed-source extension that can't be used outside of vscode.
This is especially important when working in a team setting.
It doesn't feel great to use a forked type checker/LSP that's not enforced in your org's CI/CD. And it also doesn't feel great to force the forked type checker onto the entire organization when only a subset of folks may be using a forked vscode editor.
So a typical HN post might be “A new widget that saves times rendering Python code”. Whereas we get this constant barrage of “A new widget that saves times rendering Python code in Rust” with Rust appended to it.
Also currently the Python IDE support (autocompletion, refactoring, etc.) in VSCode is provided by Pylance which is closed source, so this would provide an open source alternative to that.
lint:unresolved-import: Cannot resolve imported module `pydantic` --> vartia/usr_id.py:4:6 | 2 | from typing import Optional, Any 3 | from enum import Enum 4 | from pydantic import BaseModel, ConfigDict
looking forward to the release version.
When it was released it might have been one of the easiest to use languages.
The focus on tooling and making the tooling fast has been sharp. Seeing people recommend using non-astral tooling seems nuts at this point.
astral have now replaced the awful pip with the fantastic uv
various awful linters with with the fantastic ruff
and now hopefully replacing the terrible type checkers (e.g. mypy) with a good one!
I hope they have the pypi backend on their list too, my kingdom for Maven Central in python!
IIRC they have floated the idea of private registries as a commercial offering in the past.
I would concur with you if you said Go, Rust, Ruby, or even heck, PHP, but Java is probably the only language that I know that is in a situation even as bad as Python or even worse (at least for me definitely worse, because at least I understand Python tooling enough even when using it only for hobby projects, while I still don't understand Java tooling enough even after working professionally with JVM languages for 7+ years).
Java is the only language that I know except Python that has multiple project/package managers (Maven, Gradle, probably even more). It also has no concept of lock files in at least Maven/Gradle, and while resolution dependency in Maven/Gradle is supposed to be deterministic, from my experience it is anything but: just a few weeks ago we had a deployment that failed but worked locally/CI because of dependency resolution somehow pulled different versions of the same library.
Fighting dependency hell because different dependencies pull different version constraints is a pain (all Java/JVM projects that I ever worked had some manually pinned dependencies to either fix security issues or to fix broken dependency resolution), and don't even get me in the concept of Uber JARs (that we had to use in previous job because it was the only way to ensure that the dependency tree would be solved correctly; yes maybe it was by incompetence of the team that maintained our shared libraries, but the fact that we even got at that situation is unacceptable).
Oh, and also Gradle is "so fun": it is a DSL that has zero discovery (I had IntelliJ IDEA Ultimate and I could still not get it to auto-complete 60% of the time), so I would just blindly try to discover what where the inputs of the functions. The documentation didn't help because the DSL was so dynamic and every project would use it slightly different, so it was really difficult to discover a way to make it work for that specific project (the examples that I would find would be enough different from my current project that 90% of time it wouldn't work without changing something). Python at least has `pyproject.toml` nowadays, and the documentation from PyPA is good enough that you can understand what you want to do after reading it for 10 minutes.
Modular.ai raised $100 million to solve tangentially similar problems with python. Astral has already had a much larger impact, while providing better integration with less than 10% of that money.
you could even say that astral and modular focus on two extreme ends of the developer experience spectrum - just making python tooling faster, vs making python-ish code faster.
Either way, Python's irritants are the 'stick' that motivates devs to try alternatives. Astral is patching python's problems at lightning speed. Soon, there may not be enough incentive left to migrate off python. I'm assuming Mojo's target customer is an application dev who uses python and not a seasoned system dev looking for a more aesthetic language.
fwiw, I hope Mojo succeeds.
[bring on the downvotes]
Not every situation calls for type safe languages, you're projecting a preference.
For now, I have some false negative warnings :
'global' variables are flagged as undefined `int:unresolved-reference: Name ... used when not defined` (yeah, it's bad, I know)
f(*args) flagged as missing arguments `lint:missing-argument: No arguments provided for required parameters ...`
On the modest codebase I tried it on (14k LOC across 126 files), it runs in 149ms compared to 1.66s in pyright (both run via uvx <tool>). I couldn't get it to play nicely with a poetry project, but it works fine (obviously) in a uv project.
Definitely some false-positives, as expected. Interestingly, it seems to hate the `dict()` initializer (e.g. `dict(foo="bar")`).
But I like that they’re focussing on creating something useful before chasing revenue. Once they’ve got a single tool that provides a consistent dev experience for Python developers and it’s widely adopted they should be able to pursue monetisation easily.
tmvphil•18h ago
- mypy (warm cache) 18s
- ty: 0.5s (and found 3500 errors)
They've done it again.
mil22•18h ago
https://github.com/microsoft/pyright
nine_k•18h ago
bjourne•15h ago
dathinab•14h ago
In python it's pretty common to have LSP separate from type checking separate from linting (e.g. ruff+mypy+ide_specific_lsp).
Which to be fair sucks (as it limits what the LSP can do, can lead to confusing mismatches in error/no-error and on one recent project I had issues with the default LSP run by vscode starting to fall apart and failing to propose auto imports for some trivial things for part of the project....)
But it's the stack where pyright fits in.
geekraver•14h ago
insane_dreamer•14h ago
lemontheme•5h ago
The PyCharm checker seems to miss really, really obvious things, e.g. allowing a call site to expect a string while the function returns bytes or none.
Maybe my colleagues just have it configured wrong but there’s several of them and the config isn’t shared.
0xFF0123•14h ago
mil22•12h ago
Ty: 2.5 seconds, 1599 diagnostics, almost all of which are false positives
Pyright: 13.6 seconds, 10 errors, all of which are actually real errors
There's plenty of potential here, but Ty's type inference is just not as sophisticated as Pyright's at this time. That's not surprising given it hasn't even been released yet.
Whether Ty will still perform so much faster once all of Pyright's type inference abilities have been matched or implemented - well, that remains to be seen.
Pyright runs on Node, so I would expect it to be a little slower than Ty, but perhaps not by very much, since modern JS engines are already quite fast and perform within a factor of ~2-3x of Rust. That said, I'm rooting for Ty here, since even a 2-3x performance boost would be useful.
the_duke•6h ago
There is a reason Typescript moved to a typed language.
rtpg•10h ago
I would like to just not use it, but the existence of pyright as a _barely_ functional alternative really sucks the air out of other attempts' continued existence. Real "extend/extinguish" behavior from MSFT.
lemontheme•6h ago
Can’t recommend it enough
js2•18h ago
> This project is still in development and is not ready for production use.
rybosome•17h ago
Indeed they have. Similar improvement in performance on my side.
It is so fast that I thought it must have failed and not actually checked my whole project.
_carljm•17h ago
This is an early preview of a pre-alpha tool, so I would expect a good chunk of those 3500 errors to be wrong at at this point :) Bug reports welcome!
joshdavham•15h ago
I was also one of those people who, when first trying Ruff, assumed that it didn't work the first time I ran it because of how fast it executed!
_carljm•15h ago
It will certainly be slower than Ruff, just because multi-file type analysis more complex and less embarrassingly parallel than single-file linting.