This is what UDP is for. Everything actually has to be async all the way down and since it’s not, we’ll just completely reimplement the OS and network on top of itself and hey maybe when we’re done with that we can do it a third time to have the cloud of clouds.

The entire stack we’re using right down to the hardware is not fit for purpose and we’re burning our talent and money building these ever more brittle towering abstractions.

In an ideal world you could just ask the host to stream 100MB of stuff into a byte array or slice of the wasm heap. Alas.

> I'm not here to disparage the work that came before — I'm here to start a conversation about what can potentially come next.

Terrible LLM-slop style. Is Mr Snell letting an LLM write the article for him or has he just appropriated the style?

Sadly it will never happen. WebAssembly failed to keep some of its promises here.

They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there.

They propose this:

  type Stream<T> = {
    next(): Promise<{ done, value: UInt8Array<T> }>
  }

  type Stream<T> = {
    next(): { done, value: T } | Promise<{ done, value: T }>
  }

- I can easily make mine from theirs

- In theirs the conceptual "stream" is defined by an iterator of iterators, meaning you need a for loop of for loops to step through it. In mine it's just one iterator and it can be consumed with one for loop.

- I'm not limited to having only streams of integers, they are

- My way, if I define a sync transform over a sync input, the whole iteration can be sync making it possible to get and use the result in sync functions. This is huge as otherwise you have to write all the code twice: once with sync iterator and for loops and once with async iterators and for await loops.

- The problem with thrashing Promises when splitting input up into words goes away. With async iterators, creating two words means creating two promises. With stream iterators if you have the data available there's no need for promises at all, you just yield it.

- Stream iterators can help you manage concurrency, which is a huge thing that async iterators cannot do. Async iterators can't do this because if they see a promise they will always wait for it. That's the same as saying "if there is any concurrency, it will always be eliminated."

https://github.com/ralusek/streamie

allows you to do things like

    infiniteRecords
    .map(item => doSomeAsyncThing(item), { concurrency: 5 });

    infiniteRecords
    .map(item => doSomeAsyncSingularThing(item), { concurrency: 5 })
    .map(groupOf10 => doSomeBatchThing(groupsOf10), { batchSize: 10 })
    // Can flatten back to single items
    .map(item => backToSingleItem(item), { flatten: true });

  import { Repeater } from "@repeaterjs/repeater";
  
  const keys = new Repeater(async (push, stop) => {
    const listener = (ev) => {
      if (ev.key === "Escape") {
        stop();
      } else {
        push(ev.key);
      }
    };
    window.addEventListener("keyup", listener);
    await stop;
    window.removeEventListener("keyup", listener);
  });
  const konami = ["ArrowUp", "ArrowUp", "ArrowDown", "ArrowDown", "ArrowLeft", "ArrowRight", "ArrowLeft", "ArrowRight", "b", "a"];
  (async function() {
    let i = 0;
    for await (const key of keys) {
      if (key === konami[i]) {
        i++;
      } else {
        i = 0;
      }
      if (i >= konami.length) {
        console.log("KONAMI!!!");
        break; // removes the keyup listener
      }
    }
  })();

It’s one of those abstractions that’s feature complete and stable, and looking at NPM it’s apparently getting 6.5mil+ downloads a week for some reason.

Lately I’ve just taken the opposite view of the author, which is that we should just use streams, especially with how embedded they are in the `fetch` proposals and whatever. But the tee critique is devastating, so maybe the author is right. It’s exciting to see people are still thinking about this. I do think async iterables as the default abstraction is the way to go.

The objection is

> The Web streams spec requires promise creation at numerous points — often in hot paths and often invisible to users. Each read() call doesn't just return a promise; internally, the implementation creates additional promises for queue management, pull() coordination, and backpressure signaling.

But that's 95% manageable by altering buffer sizes.

And as for that last 5%....what are you doing with JS to begin with?

I'm working on a db driver that uses it by convention as part of connection/pool usage cleanup.