frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Lazy iteration vs. array chaining on 500k rows – benchmark results

1•gvsh_maths•1h ago
I built a TypeScript iterator library (iterflow) and wanted to measure the actual heap difference between lazy and eager pipelines. This is the benchmark writeup.

The pipelines

Eager - standard array chaining:

const data = Array.from(generateRows(500_000));

const results = data .filter(r => r.active && r.value > threshold) .map(r => ({ id: r.id, score: r.value * 1.5 })) .slice(0, 10_000);

Each step produces a new intermediate array. .filter() allocates one, .map() allocates another, .slice() then discards most of both.

Lazy - same pipeline via iterflow:

import { iter } from '@mathscapes/iterflow';

const results = iter(generateRows(500_000)) .filter(r => r.active && r.value > threshold) .map(r => ({ id: r.id, score: r.value * 1.5 })) .take(10_000) .toArray(); generateRows is a generator, yields one row at a time. Nothing is materialized until .toArray() pulls values through the chain. No intermediate arrays.

Results

Dataset: 500,000 rows Pipeline: filter(active && value > 5000) → map(score) → take(10,000)

native array (.filter → .map → .slice) 15.4 MB (min 15.2 MB, max 16.2 MB) iterflow (.filter → .map → .take) 5.8 MB (min 5.8 MB, max 5.8 MB)

Methodology

- Metric: heapUsed delta before and after the pipeline, not total process memory - Both pipelines start from the same generator source — the delta measures pipeline allocations only, not source data --expose-gc with explicit gc() calls forced between every run - One warm-up run discarded before measurement - Median of 5 runs reported - The native array run materializes the full 500k dataset into data before the pipeline runs. That allocation is not included in the delta - both approaches are measured on the same footing.

A few notes on the library

- iter() is a wrapper around ES2015 generators and the iterator protocol - no magic, just a fluent API so the call site looks identical to array chaining - .sum() and .mean() are typed to Iterflow<number> only - calling them on a non-numeric iterator is a compile error - Has some streaming statistical operations (.streamingMean(), .ewma(), .windowedMin()) for running aggregations without a separate accumulator - Zero runtime dependencies

https://www.npmjs.com/package/@mathscapes/iterflow

Comments

keni34•1h ago
Good to know

Phlex: Ruby gem for building HTML and SVG view components

https://www.phlex.fun/
1•thunderbong•3m ago•0 comments

EQT eyes potential $6B sale of Linux pioneer SUSE, sources say

https://www.reuters.com/business/eqt-eyes-potential-6-billion-sale-linux-pioneer-suse-sources-say...
1•shscs911•5m ago•0 comments

Nextvi 4.0 – A small, hackable vi/ex editor with an optional patch system

https://github.com/kyx0r/nextvi
1•todsacerdoti•7m ago•0 comments

Personal MCP server on every Claude platform without Auth0

https://github.com/crumrine/fastmcp-personal-auth
1•brian200•9m ago•1 comments

SSH Secret Menu

https://twitter.com/rebane2001/status/2031037389347406054
1•piccirello•12m ago•0 comments

Strict Monospace Font with Nerd Fonts and Emoji for AI and Human, AI-CLIs Users

https://www.npmjs.com/package/@monolex/codexmono
1•monokist•14m ago•0 comments

Show HN: SAT Protocol – static social networking

https://github.com/remysucre/satproto
2•remywang•15m ago•1 comments

Context Hub: give coding agents curated, versioned docs

https://github.com/andrewyng/context-hub
1•saikatsg•15m ago•0 comments

German publishers reject Apple's revised app tracking rules, urge antitrust fine

https://www.reuters.com/legal/litigation/german-publishers-reject-apples-revised-app-tracking-rul...
1•1659447091•16m ago•0 comments

JadeGate – A deterministic safety proxy for MCP servers (no LLMs)

1•coolcoder9520•25m ago•1 comments

Spatial Programming Without Escape [video]

https://www.youtube.com/watch?v=eQgxFuw8f1U
1•bcjordan•26m ago•0 comments

January 6: A Date Which Will Live in Infamy – The White House

https://www.whitehouse.gov/j6/
2•rasengan0•26m ago•2 comments

Ask HN: What are your favorite podcast episodes?

1•nbaksalyar•27m ago•0 comments

Agentic Search: When Retrieval Stops Being Enough

https://medium.com/@calufa/agentic-search-when-retrieval-stops-being-enough-c62652fb94f9
1•_hfqa•29m ago•0 comments

Moonforge

https://moonforgelinux.org/
1•OuterVale•34m ago•0 comments

Do the Illegible

https://ashwinsundar.com/posts/legibility-and-programming/
1•todsacerdoti•37m ago•0 comments

First (?) Hacked Emacs Package

https://old.reddit.com/r/emacs/comments/1rowm5i/first_hacked_emacs_package/
3•todsacerdoti•37m ago•0 comments

Trump considering taking over Strait of Hormuz

https://www.cbsnews.com/news/trump-iran-cbs-news-the-war-is-very-complete-strait-hormuz/
5•geox•39m ago•5 comments

In Claude: Start Thinking Like a Product Manager

https://medium.com/@ggonweb/in-the-world-of-claude-start-thinking-like-a-product-manager-f6505b43...
1•ggonweb•40m ago•0 comments

Trump more popular than Harris, Newsom: Survey

https://thehill.com/homenews/administration/5775642-trump-2028-presidential-race-poll/
6•donsupreme•42m ago•0 comments

Production MCP Server Starter Kit – Auth, Rate Limiting, AWS CDK, Docker

https://github.com/tysoncung/mcp-server-starter
2•tysoncung•43m ago•1 comments

Claude Banged My Module

https://dcmc.github.io/blog/claude-banged-my-module/
3•DarrisMackelroy•49m ago•1 comments

Wdquery

https://github.com/altilunium/wdquery
1•altilunium•54m ago•0 comments

An iPhone-hacking toolkit used by Russian spies likely came from U.S. contractor

https://techcrunch.com/2026/03/09/an-iphone-hacking-toolkit-used-by-russian-spies-likely-came-fro...
4•cdrnsf•55m ago•0 comments

Hyperspace Agent

https://agents.hyper.space/research-report
1•vinhnx•57m ago•0 comments

Secure Secrets Management for Cursor Cloud Agents

https://infisical.com/blog/secure-secrets-management-for-cursor-cloud-agents
1•vmatsiiako•57m ago•0 comments

Built a lightweight AI summary plugin for Salesforce (no Einstein required)

https://dscopeai.com
1•pavelkds•57m ago•1 comments

The Great Silicon Brain Robbery: A Chronicle of Our Artificial Demise

https://laughingmachines.substack.com/p/the-great-silicon-brain-robbery-a
1•kmer007•1h ago•0 comments

Valedex – Extract financial metrics from CIM PDFs for private market deals

https://www.valedex.com/
1•marcelk123•1h ago•3 comments

Apple: The First 50 Years

https://www.applefirst50.com
1•pqvst•1h ago•0 comments