frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Bringing NumPy's type-completeness score to nearly 90%

https://pyrefly.org/blog/numpy-type-completeness/
43•todsacerdoti•1w ago

Comments

tecoholic•2h ago
> That's it! CanIndex was an unknown symbol and was probably mistyped, and replacing it with the correct SupportsIndex one brought NumPy's overall type-completeness to over 80%!

Let’s take a pause here for a second - the ‘CanIndex’ and ‘SupportsIndex’ from the looks are just “int”. I have a hard time dealing with these custom types because they are so obscure.

Does anyone find these useful? How should I be reading them? I understand custom types that package data like classes. But feel completely confused for this yadayava is string, floorpglooep is int style types.

jkingsman•2h ago
I find it especially helpful during refactors — understanding the significance/meaning of the variable and what its values can be (and are constrained to) and not just the type is great, but if typing is complete, I can also go and see everywhere that type can be ingested, knowing every surface that uses it and that might need to be refactored.
CaliforniaKarl•2h ago
> Let’s take a pause here for a second - the ‘CanIndex’ and ‘SupportsIndex’ from the looks are just “int”.

The PR for the change is https://github.com/numpy/numpy/pull/28913 - The details of files changed[0] shows the change was made in 'numpy/__init__.pyi'. Looking at the whole file[1] shows SupportsIndex is being imported from the standard library's typing module[2].

Where are you seeing SupportsIndex being defined as an int?

> I have a hard time dealing with these custom types because they are so obscure.

SupportsIndex is obscure, I agree, but it's not a custom type. It's defined in stdlib's typing module[2], and was added in Python 3.8.

[0]: https://github.com/numpy/numpy/pull/28913/files

[1]: https://github.com/charris/numpy/blob/c906f847f8ebfe0adec896...

[2]: https://docs.python.org/3/library/typing.html#typing.Support...

tecoholic•29m ago
Thanks for pointing those out. Your comment and the others who responded tell me I don’t typically work with complex software. I am a web dev who also had to wrangle data at times, the data part is where I face the issues.

I looked at the function in the article taking in an int and I stopped there. And then I read the docs for SupportsIndex, I just can’t think of an occasion I would have used it.

masspro•2h ago
* “which of the 3 big data structures in this part of the program/function/etc is this int/string key an index into?”

* some arithmetic/geometry problems for example 2D layout where there are several different things that are “an integer number of pixels” but mean wildly different things

In either case it can help pick apart dense code or help stay on track while writing it. It can instead become anti-helpful by causing distraction or increasing the friction to make changes.

CaliforniaKarl•2h ago
I've recently been writing a Python SDK for an internal API that's maintained by a different group. I've been making sure to include typing for everything in the SDK, as well as attempting to maximize unit test coverage (using mocked API responses).

It's a heck of a lot of work (easily as much work as writing the actual code!), but typing has already paid dividends as I've been starting to use the SDK for one-off scripts.

strbean•2h ago
Typing related NumPy whine:

The type annotations say `float32`, `float64`, etc. are type aliases for `Floating[32]` and `Floating[64]`. In reality, they are concrete classes.

This means that `isinstance(foo, float32)` works as expected, but pisses off static analysis tools.

refactor_master•1h ago
I hate when libraries do that, or create a `def ClassLike`.

I’ve found the library `beartype` to be very good for runtime checks built into partially annotated `Annotated` types with custom validators, so instead of instance checks you rely directly on signatures and typeguards.

jofer•2h ago
One of my biggest gripes around typing in python actually revolves around things like numpy arrays and other scientific data structures. Typing in python is great if you're only using builtins or things that the typing system was designed for. But it wasn't designed with scientific data structures particularly in mind. Being able to denote dtype (e.g. uint8 array vs int array) is certainly helpful, but only one aspect.

There's not a good way to say "Expects a 3D array-like" (i.e. something convertible into an array with at least 3 dimensions). Similarly, things like "At least 2 dimensional" or similar just aren't expressible in the type system and potentially could be. You wind up relying on docstrings. Personally, I think typing in docstrings is great. At least for me, IDE (vim) hinting/autocompletion/etc all work already with standard docstrings and strictly typed interpreters are a completely moot point for most scientific computing. What happens in practice is that you have the real info in the docstring and a type "stub" for typing. However, at the point that all of the relevant information about the expected type is going to have to be the docstring, is the additional typing really adding anything?

In short, I'd love to see the ability to indicate expected dimensionality or dimensionality of operation in typing of numpy arrays.

But with that said, I worry that typing for these use cases adds relatively little functionality at the significant expense of readability.

efavdb•2h ago
Would a custom decorator work for you?
jofer•2h ago
Unless I'm missing something entirely, what would that add? You still can't express the core information you need in the type system.
dwohnitmok•2h ago
This isn't static, but jaxtyping gives you at least runtime checks and also a standardized form of documenting those types. https://github.com/patrick-kidger/jaxtyping
jofer•2h ago
It actually doesn't, as far as I know :) It does get close, though. I should give it a deeper look than I have previously, though.

"array-like" has real meaning in the python world and lots of things operate in that world. A very common need in libraries is indicating that things expect something that's either a numpy array or a subclass of one or something that's _convertible_ into a numpy array. That last part is key. E.g. nested lists. Or something with the __array__ interface.

In addition to dimensionality that part doesn't translate well.

And regardless, if the type representation is not standardized across multiple libraries (i.e. in core numpy), there's little value to it.

nerdponx•1h ago
I wonder if we should standardize on __array__ like how Iterable is standardized on the presence of __iter__, which can just return self if the Iterable is already an Iterator.
HPsquared•1h ago
Have you looked at nptyping? Type hints for ndarray.

https://github.com/ramonhagenaars/nptyping/blob/master/USERD...

jofer•1h ago
That one's new to me. Thanks! (With that said, I worry that 3rd party libs are a bad place for types for numpy.)
nerdponx•1h ago
Numpy ships built-in type hints as well as a type for hinting arrays in your own code (numpy.typing.NDArray).

The real challenge is denoting what you can accept as input. `NDArray[np.floating] | pd.Series[float] | float` is a start but doesn't cover everything especially if you are a library author trying to provide a good type-hinted API.

stared•1h ago
Some time ago I created a demo of named dimensions for Pytorch, https://github.com/stared/pytorch-named-dims

In the same line, I would love to have more Pandas-Pydantic interoperability at the type level.

hamasho•1h ago
I also had a very hard time to annotate types in python few years ago. A lot of popular python libraries like pandas, SQLAlchemy, django, and requests, are so flexible it's almost impossible to infer types automatically without parsing the entire code base. I tried several libraries for typing, often created by other people and not maintained well, but after painful experience it was clear our development was much faster without them while the type safety was not improved much at all.
t1129437123•1h ago
"ArrayLike" is not a type. The entire Python "type system" is really hackish compared to Julia or typed array languages.

Does this "type system" ensure that you can't get a runtime type error? I don't think so.

IRS Open Sources its Fact Graph

https://github.com/IRS-Public/fact-graph
135•ronbenton•2h ago•36 comments

I'm recomming my customers switch to Linux rather that Upgrade to Windows 11

https://www.scottrlarson.com/publications/publication-windows-move-towards-surveillance/
42•trinsic2•35m ago•16 comments

Apple M5 chip

https://www.apple.com/newsroom/2025/10/apple-unleashes-m5-the-next-big-leap-in-ai-performance-for...
940•mihau•12h ago•1037 comments

Claude Haiku 4.5

https://www.anthropic.com/news/claude-haiku-4-5
429•adocomplete•8h ago•179 comments

Writing an LLM from scratch, part 22 – training our LLM

https://www.gilesthomas.com/2025/10/llm-from-scratch-22-finally-training-our-llm
55•gpjt•1h ago•1 comments

Next Steps for the Caddy Project Maintainership

https://caddy.community/t/next-steps-for-the-caddy-project-maintainership/33076
97•francislavoie•4h ago•16 comments

ImapGoose

https://whynothugo.nl/journal/2025/10/15/introducing-imapgoose/
30•xarvatium•3h ago•6 comments

Bringing NumPy's type-completeness score to nearly 90%

https://pyrefly.org/blog/numpy-type-completeness/
43•todsacerdoti•1w ago•20 comments

I almost got hacked by a 'job interview'

https://blog.daviddodda.com/how-i-almost-got-hacked-by-a-job-interview
721•DavidDodda•12h ago•386 comments

Gerald Sussman - An Electrical Engineering View of a Mechanical Watch (2003)

https://techtv.mit.edu/videos/15895-an-electrical-engineering-view-of-a-mechanical-watch
39•o4c•1w ago•7 comments

Zed is now available on Windows

https://zed.dev/blog/zed-for-windows-is-here
176•meetpateltech•9h ago•60 comments

Are hard drives getting better?

https://www.backblaze.com/blog/are-hard-drives-getting-better-lets-revisit-the-bathtub-curve/
125•HieronymusBosch•8h ago•55 comments

Show HN: Halloy – Modern IRC client

https://github.com/squidowl/halloy
280•culinary-robot•13h ago•77 comments

Pwning the Nix ecosystem

https://ptrpa.ws/nixpkgs-actions-abuse
236•SuperShibe•11h ago•42 comments

Blood-Sharing Drug Trend Fuels Global HIV Surge

https://www.nytimes.com/2025/10/08/world/asia/bluetoothing-drug-blood-sharing.html
37•zahlman•4d ago•23 comments

Leaving serverless led to performance improvement and a simplified architecture

https://www.unkey.com/blog/serverless-exit
292•vednig•14h ago•181 comments

F5 says hackers stole undisclosed BIG-IP flaws, source code

https://www.bleepingcomputer.com/news/security/f5-says-hackers-stole-undisclosed-big-ip-flaws-sou...
133•WalterSobchak•12h ago•64 comments

Recursive Language Models (RLMs)

https://alexzhang13.github.io/blog/2025/rlm/
68•talhof8•7h ago•20 comments

More About Jumps Than You Wanted to Know

https://gpfault.net/posts/asm-tut-4.html
7•nice_byte•6d ago•0 comments

US Dept of Interior denies canceling largest solar project after axing review

https://www.utilitydive.com/news/department-interior-cancels-review-nevada-solar-project-trump/80...
51•toomuchtodo•2h ago•24 comments

A kernel stack use-after-free: Exploiting Nvidia's GPU Linux drivers

https://blog.quarkslab.com/./nvidia_gpu_kernel_vmalloc_exploit.html
132•mustache_kimono•11h ago•15 comments

A Gemma model helped discover a new potential cancer therapy pathway

https://blog.google/technology/ai/google-gemma-ai-cancer-therapy-discovery/
37•alexcos•6h ago•5 comments

Princeton Engineering Anomalies Research

https://pearlab.icrl.org/
32•walterbell•1w ago•6 comments

Recreating the Canon Cat document interface

https://lab.alexanderobenauer.com/updates/the-jasper-report
88•tonyg•10h ago•7 comments

How First Wap tracks phones around the world

https://www.lighthousereports.com/methodology/surveillance-secrets-explainer/
62•mattboulos•3h ago•11 comments

Garbage collection for Rust: The finalizer frontier

https://soft-dev.org/pubs/html/hughes_tratt__garbage_collection_for_rust_the_finalizer_frontier/
112•ltratt•13h ago•111 comments

The brain navigates new spaces by 'darting' between reality and mental maps

https://medicine.yale.edu/news-article/brain-navigates-new-spaces-by-flickering-between-reality-a...
140•XzetaU8•1w ago•54 comments

FSF announces Librephone project

https://www.fsf.org/news/librephone-project
1384•g-b-r•1d ago•566 comments

Americans' love of billiards paved the way for synthetic plastics

https://invention.si.edu/invention-stories/imitation-ivory-and-power-play
61•geox•1w ago•34 comments

Helpcare AI (YC F24) Is Hiring

1•hsial•13h ago