This is my biggest complaint about Python.
Pipelining can become hard to debug when chains get very long. The author doesn't address how hard it can be to identify which step in a long chain caused an error.
They do make fun of Python, however. But don't say much about why they don't like it other than showing a low-res photo of a rock with a pipe routed around it.
Ambiguity about what constitutes "pipelining" is the real issue here. The definition keeps shifting throughout the article. Is it method chaining? Operator overloading? First-class functions? The author uses examples that function very differently.
It's not like you lose that much readability from
foo(bar(baz(c)))
c |> baz |> bar |> foo
c.baz().bar().foo()
t = c.baz()
t = t.bar()
t = t.foo() fn get_ids(data: Vec<Widget>) -> Vec<Id> {
data.iter()
.filter(|w| w.alive)
.map(|w| w.id)
.collect()
}
(-> c baz bar foo)
(-> c
baz
bar
foo)Yeah, I agree that this can be problem when you lean heavily into monadic handling (i.e. you have fallible operations and then pipe the error or null all the way through, losing the information of where it came from).
But that doesn't have much to do with the article: You have the same problem with non-pipelined functional code. (And in either case, I think that it's not that big of a problem in practice.)
> The author uses examples that function very differently.
Yeah, this is addressed in one of the later sections. Imo, having a unified word for such a convenience feature (no matter how it's implemented) is better than thinking of these features as completely separate.
>Let me make it very clear: This is [not an] article it's a hot take about syntax. In practice, semantics beat syntax every day of the week. In other words, don’t take it too seriously.
Just before that statement, he says that it is an article/hot take about syntax. He acknowledges your point.
So I think when he says "semantics beat syntax every day of the week", that's him acknowledging that while he prefers certain syntax, it may not be the best for a given situation.
The |> operator is really cool.
It is an exemplar of expressions [0] more than anything else, which have little to do with the idea of passing results from one method to another.
[0]: https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...
[1]: https://learn.microsoft.com/en-us/dotnet/csharp/linq/get-sta...
In this case I would say extension methods are what he's really referring to, of which Linq to objects is built on top of.
1) The method chaining extension methods on IEnumerable<T> like Select, Where, GroupBy, etc. This is identical to the rust example in the article.
2) The weird / bad (in my opinion) language keywords analogous to the above such as "from", "where", "select" etc.
fn get_ids(data: Vec<Widget>) -> Vec<Id> { data.iter() // get iterator over elements of the list .filter(|w| w.alive) // use lambda to ignore tombstoned widgets .map(|w| w.id) // extract ids from widgets .collect() // assemble iterator into data structure (Vec) }
Same thing in 15 year old C# code.
List<Guid> GetIds(List<Widget> data)
{
return data
.Where(w => w.IsAlive())
.Select(w => w.Id)
.ToList();
}Pipelining can guide one to write a bit cleaner code, viewing steps of computation as such, and not as modifications of global state. It forces one to make each step return a result, write proper functions. I like proper pipelining a lot.
i mean this sounds fun
but tbh it also sounds like it'd result in my colleague Carl defining an utterly bespoke DSL in the language, and using it to write the worst spaghetti code the world has ever seen, leaving the code base an unreadable mess full of sharp edges and implicit behavior
data.iter()
.filter(|w| w.alive)
.map(|w| w.id)
.collect()
collect(map(filter(iter(data), |w| w.alive), |w| w.id))
> Quick challenge for the curious Rustacean, can you explain why we cannot rewrite the above code like this, even if we import all of the symbols?
Probably for lack of
> weird operators like <$>, <*>, $, or >>=
Examples:
https://kotlinlang.org/docs/extensions.html
https://docs.scala-lang.org/scala3/reference/contextual/exte...
See also: https://en.wikipedia.org/wiki/Uniform_function_call_syntax
fun main() {
val s: String? = null
println(s.isS()) // false
}
fun String?.isS() = "s" == thisI wrote a little pipeline macro in https://nim-lang.org/ for Advent of Code years ago and as far as I know it worked okay.
``` import macros
macro `|>`\* (left, right : expr): expr =
result = newNimNode(nnkCall)
case right.kind
of nnkCall:
result.add(right[0])
result.add(left)
for i in 1..<right.len:
result.add(right[i])
else:
error("Unsupported node type")Makes me want to go write more nim.
I prefer to just generalize the function (make it generic, leverage traits/typeclasses) tbh.
> Probably for lack of > weird operators like <$>, <*>, $, or >>=
Nope btw. I mean, maybe? I don't know Haskell well enough to say. The answer that I was looking for here is a specific Rust idiosyncrasy. It doesn't allow you to import `std::iter::Iterator::collect` on its own. It's an associated function, and needs to be qualified. (So you need to write `Iterator::collect` at the very least.)
You probably noticed, but it should become a thing in RFC 3591: https://github.com/rust-lang/rust/issues/134691
So it does kind of work on current nightly:
#![feature(import_trait_associated_functions)]
use std::iter::Iterator::{filter, map, collect};
fn get_ids2(data: Vec<Widget>) -> Vec<Id> {
collect(map(filter(Vec::into_iter(data), |w| w.alive), |w| w.id))
}
fn get_ids3(data: impl Iterator<Item = Widget>) -> Vec<Id> {
collect(map(filter(data, |w| w.alive), |w| w.id))
}Building pipelines:
https://effect.website/docs/getting-started/building-pipelin...
Using generators:
https://effect.website/docs/getting-started/using-generators...
Having both options is great (at the beginning effect had only pipe-based pipelines), after years of writing effect I'm convinced that most of the time you'd rather write and read imperative code than pipelines which definitely have their place in code bases.
In fact most of the community, at large, converged at using imperative-style generators over pipelines and having onboarded many devs and having seen many long-time pipeliners converging to classical imperative control flow seems to confirm both debugging and maintenance seem easier.
However.
I would be lying if I didn't secretly wish that all languages adopted the `|>` syntax from Elixir.
```
params
|> Map.get("user")
|> create_user()
|> notify_admin()
```
``` params.get("user") |> create_user |> notify_admin ```
Even more concise and it doesn't even require a special language feature, it's just regular syntax of the language ( |> is a method like .get(...) so you could even write `params.get("user").|>(create_user) if you wanted to)
Also, what if the function you want to use is returned by some nullary function? You couldn't just do |> getfunc(), as presumably the pipeline operator will interfere with the usual meaning of the parentheses and will try to pass something to getfunc. Would |> ( getfunc() ) work? This is the kind of problem that can arise when one language feature is permitted to change the ordinary behaviour of an existing feature in the name of convenience. (Unless of course I'm just missing something.)
https://github.com/tc39/proposal-pipeline-operator
I'm very excited for it.
I haven't looked at the member properties bits but I suspect the pipeline syntax just needs the transform to be supported in build tools, rather than adding yet another polyfill.
It seems like most people are just asking for the simple function piping everyone expects from the |> syntax, but that doesn't look likely to happen.
I'm not a JS dev so idk what member property support is.
"bar"
|> await getFuture(%);
|> baz(await %);
|> bat(await %);
This also gets weird because if the `|>` is a special function that sends in a magic `%` parameter, it'd have to be context sensitive to whether or not an `async` thing happens within the bounds. Whether or not it does will determine if the subsequent pipes are dealing with a future of % or just % directly.
In reality it would look like:
"bar"
|> await getFuture()
|> baz()
|> await bat()
await bat(baz(await getFuture("bar")));
"bar"
|> await getFuture()
"bar"
|> await getFutureAsyncFactory()
Typically in JS you do this with parens like so:
(await getFutureAsyncFactory())("input")
But the use of parens doesn't transpose to the pipeline setting well IMO
Given this example:
(await getFutureAsyncFactory("bar"))("input")
("bar" |> await getFutureAsyncFactory())("input")
The whole pipeline takes on the value of the last function specified.
Our only hope is if TypeScript finally gives up on the broken TC39 process and starts to implement its own syntax enhancements again.
[1] https://2024.stateofjs.com/en-US/usage/#top_currently_missin...
They list this as a con of F# (also Elixir) pipes:
value |> x=> x.foo()
value |> x.foo()
users
& map validate
& catMaybes
& mapM persistIn my programming language, I added `.>` as a reverse-compose operator, so pipelines of function compositions can also be read uniformly left-to-right, e.g.
process = map validate .> catMaybes .> mapM persistThere is also https://hackage.haskell.org/package/flow which uses .> and <. for function composition.
EDIT: in no way do I want to claim the originality of these things in Elm or the Haskell package inspired by it. AFAIK |> came from F# but it could be miles earlier.
Is there any language with a single feature that gives the best of both worlds?
```
params
|> Map.get("user")
|> create_user()
|> (¬ify_admin("signup", &1)).() ```
or
```
params
|> Map.get("user")
|> create_user()
|> (fn user -> notify_admin("signup", user) end).() ```
params
|> Map.get("user")
|> create_user()
|> then(¬ify_admin("signup", &1))
params
|> Map.get("user")
|> create_user()
|> then(fn user -> notify_admin("signup", user) end)
"World"
|> then(&concat("Hello ", &1))
"World"
|> concat("Hello ", &1)
My dislike does improve my test coverage though, since I tend to pop out a real method instead.
Instead of:
```
fetch_data()
|> (fn
{:ok, val, _meta} -> val
:error -> "default value"
|> String.upcase()
```
Something like this:
```
fetch_data()
|>? {:ok, val, _meta} -> val
|>? :error -> "default value"
|> String.upcase()
```
Instead of writing: a().b().c().d(), it's much nicer to write: d(c(b(a()))), or perhaps (d ∘ c ∘ b ∘ a)().
1. do the first step in the process
2. then do the next thing
3. followed by a third action
I struggle to think of any context outside of programming, retrosynthesis in chemistry, and some aspects of reverse-Polish notation calculators, where you conceive of the operations/arguments last-to-first. All of which are things typically encountered pretty late in one's educational journey.
a(b())
then you're already breaking your left-to-right/first-to-last rule.
draw(line.start, line.end);
print(i, round(a / b));
line.start, line.end |> draw;
i, a, b |> div |> round |> print;Julia's multiple dispatch means that all arguments to a function are treated equally. The syntax `b(a, c)` makes this clear, whereas `a.b(c)` makes it look like `a` is in some way special.
https://dspace.mit.edu/handle/1721.1/6035
https://dspace.mit.edu/handle/1721.1/6031
https://dapperdrake.neocities.org/faster-loops-javascript.ht...
imap(f) = x -> Iterators.map(f, x)
ifilter(f) = x -> Iterators.filter(f, x)
v = things |>
ifilter(isodd) |>
imap(do_process) |>
collectThe APL family is similarly consistent, except RTL.
10→A
A+10→C a().let{ b(it) }.let{ c(it) }And it's already idiomatic unlike bolting a pipeline operator onto a language that didn't start with it.
// extension function on Foo.Companion (similar to static class function in Java)
fun Foo.Companion.create(block: FooBuilder.() -> Unit): Foo =
FooBuilder().apply(block).build()
// example usage
val myFoo = Foo.create {
setSomeproperty("foo")
setAnotherProperty("bar")
}
A
.B
.C
|| D
|| EI have no idea what this is trying to say, or what it has to do with the rest of the article.
auto get_ids(std::span<const Widget> data)
{
return data
| filter(&Widget::alive)
| transform(&Widget::id)
| to<std::vector>();
}I'm really want to start playing with some C++23 in the future.
fn get_ids(data: Vec<Widget>) -> Vec<Id> {
collect(map(filter(map(iter(data), |w| w.toWingding()), |w| w.alive), |w| w.id))
}
fn get_ids(data: Vec<Widget>) -> Vec<Id> {
data.iter()
.map(|w| w.toWingding())
.filter(|w| w.alive)
.map(|w| w.id)
.collect()
}
fn get_ids(data: Vec<Widget>) -> Vec<Id> {
collect(
map(
filter(
map(iter(data), |w| w.toWingding()), |w| w.alive), |w| w.id))
}
Of course this really only matters when you're 25 minutes into critical downtime and a bug is hiding somewhere in these method chains. Anything that is surprising needs to go.
IMHO it would be better to set intermediate variables with dead simple names instead of newlines.
fn get_ids(data: Vec<Widget>) -> Vec<Id> {
let iter = iter(data);
let wingdings = map(iter, |w| w.toWingding());
let alive_wingdings = filter(wingdings, |w| w.alive);
let ids = map(alive_wingdings, |w| w.id);
let collected = collect(ids);
collected
}> You might think that this issue is just about trying to cram everything onto a single line, but frankly, trying to move away from that doesn’t help much. It will still mess up your git diffs and the blame layer.
Diff will still be terrible because adding a step will change the indentation of everything 'before it' (which, somewhat confusingly, are below it syntactically) in the chain.
See how adding line breaks still keeps the `|w| w.alive` very far from the `filter` call? And the `|w| w.id` very far from the `map` call?
If you don't have the pipeline operator, please at least format it something like this:
fn get_ids(data: Vec<Widget>) -> Vec<Id> {
collect(
map(
filter(
map(
iter(data),
|w| w.toWingding()
),
|w| w.alive
),
|w| w.id
)
)
}
Also see how this still reads fine despite being one line:
fn get_ids(data: Vec<Widget>) -> Vec<Id> {
data.iter().map(|w| w.toWingding()).filter(|w| w.alive).map(|w| w.id).collect()
}
For me, it's both. Honestly, I find it much less readable the way you're split it up. The way I had it makes it very easy for me to read it in reverse; map, filter, map, collect
> Also see how this still reads fine despite being one line
It doesn't read fine, to me. I have to spend mental effort figuring out what the various "steps" are. Effort that I don't need to spend when they're split across lines.
For me, it's a "forest for the trees" kind of thing. I like being able to look at the code casually and see what it's doing at a high level. Then, if I want to see the details, I can look more closely at the code.
One day, we'll (re)discover that partial application is actually incredibly useful for writing programs and (non-Haskell) languages will start with it as the primitive for composing programs instead of finding out that it would be nice later, and bolting on a restricted subset of the feature.
This is far different than the pattern described in the article, though. Small shame they have come to have the same name. I can see how both work with the metaphor; such that I can't really complain. The "pass a single parameter" along is far less attractive to me, though.
... looking at you R and tidyverse hell.
For example, we can write: (foo (bar (baz x))) as (-> x baz bar foo)
If there are additional arguments, we can accommodate those too: (sin (* x pi) as (-> x (* pi) sin)
Where expression so far gets inserted as the first argument to any form. If you want it inserted as the last argument, you can use ->> instead:
(filter positive? (map sin x)) as (->> x (map sin) (filter positive?))
You can also get full control of where to place the previous expression using as->.
Full details at https://clojure.org/guides/threading_macros
The tone of this (and the entire Haskell section of the article, tbh) is rather strange. Operators aren't special syntax and they aren't "added" to the language. Operators are just functions that by default use infix position. (In fact, any function can be called in infix position. And operators can be called in prefix position.)
The commit in question added & to the prelude. But if you wanted & (or any other character) to represent pipelining you have always been able to define that yourself.
Some people find this horrifying, which is a perfectly valid opinion (though in practice, when working in Haskell it isn't much of a big deal if you aren't foolish with it). But at least get the facts correct.
Being able to inspect the results of each step right at the point you’ve written it is pretty convenient. It’s readable. And the compiler will optimize it out.
And for exceptions, why not solve it in the data model, and reify failures? Push it further downstream, let your pipeline's nodes handle "monadic" result values.
Point being, it's always a tradeoff, but you can usually lessen the pain more than you think.
And that's without mentioning that a lot of "pipelining" is pure sugar over the same code we're already writing.
BTW. For people complaining about debug-ability of it: https://doc.rust-lang.org/std/iter/trait.Iterator.html#metho... etc.
https://datapad.readthedocs.io/en/latest/quickstart.html#ove...
SimonDorfman•8h ago
flobosg•7h ago
tylermw•1h ago
madcaptenor•7h ago
thom•7h ago