frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Incremental – A library for incremental computations

https://github.com/janestreet/incremental
157•handfuloflight•4h ago•23 comments

Who's afraid of Chinese models?

https://stratechery.com/2026/whos-afraid-of-chinese-models/
593•mfiguiere•21h ago•400 comments

Running Doom on Our Custom CPU and Going Viral

https://www.armaangomes.com/blogs/doom/
53•arghunter•4h ago•10 comments

Kimi Work

https://www.kimi.com/products/kimi-work
522•ms7892•15h ago•226 comments

Jelly UI: Soft-body physics for native HTML form controls

https://jelly-ui.com/
463•baldvinmar•15h ago•150 comments

Five US tech giants' hidden debts soar to $1.65T on opaque AI funding

https://asia.nikkei.com/business/technology/five-us-tech-giants-hidden-debts-soar-to-1.65tn-on-op...
251•NordStreamYacht•4h ago•122 comments

Human mathematicians are being outcounterexampled

https://xenaproject.wordpress.com/2026/07/20/human-mathematicians-are-being-outcounterexampled/
324•artninja1988•13h ago•127 comments

A Koi Pond Mosaic Made from 10 Pounds of 3D Printer Waste

https://www.instructables.com/A-Koi-Pond-Mosaic-Made-From-10-Pounds-of-3D-Printe/
29•sudo_cowsay•4h ago•22 comments

Linux kernel will support $ORIGIN, sort of

https://fzakaria.com/2026/07/20/linux-kernel-will-support-origin-sort-of
21•ingve•1h ago•9 comments

How to pack ternary numbers in 8-bit bytes

https://compilade.net/blog/ternary-packing
6•JoshTriplett•6d ago•1 comments

Nativ: Run frontier open models locally on your Mac

https://blaizzy.github.io/nativ/
266•aratahikaru5•14h ago•87 comments

Flock Credibility Lost as It Repeatedly Lies to City Councils, Police, & Public

https://www.aclu.org/news/privacy-technology/tracking-alpr-cameras/flock-safety-credibility-lost-...
376•StatsAreFun•7h ago•104 comments

Show HN: Ex Situ – Open-source spatial index of displaced cultural artifacts

https://exsitu.app/map
27•hbyel•3h ago•13 comments

Show HN: Immersive Gaussian Splat tour of grace cathedral, San Francisco

https://vincentwoo.com/3d/grace_cathedral/
155•akanet•12h ago•33 comments

VTubing: How a Japanese Phenomenon Is Going Worldwide

https://www.tokyodev.com/articles/vtubing-how-a-japanese-phenomenon-is-going-worldwide
17•pwim•4h ago•1 comments

Agent swarms and the new model economics

https://cursor.com/blog/agent-swarm-model-economics
184•jlaneve•14h ago•83 comments

Launch HN: Bloomy (YC S26) – AI-powered mastery learning for K-12

85•alexsouthmayd•15h ago•86 comments

China’s open-weights AI strategy is winning

https://werd.io/american-ai-is-locked-down-and-proprietary-its-losing/
1093•benwerd•18h ago•831 comments

Tensor Notation in Mathup

https://mathup.xyz/#tensors
11•runarberg•6d ago•1 comments

The Psychology of Software Teams

https://www.routledge.com/The-Psychology-of-Software-Teams/Hicks/p/book/9781032963389
85•dcre•5d ago•22 comments

You only need the frontier model for one single edit

https://stencil.so/blog/prewalk
131•jxmorris12•6d ago•39 comments

The Power of Awareness: Overcoming Surveillance Capitalism

https://www.scottrlarson.com/presentations/overcoming-surveillance-capitalism-with-awareness/
101•trinsic2•12h ago•17 comments

My two year old taught me constraint solving

https://thecomputersciencebook.com/posts/how-my-2yo-taught-me-constraint-solving/
70•bambataa•1w ago•26 comments

Shinjuku Station in 3D

https://satoshi7190.github.io/Shinjuku-indoor-threejs-demo/
211•Gecko4072•18h ago•46 comments

I wrote an bash enumerator because I was sick of xargs

https://numerlab.org/2025/07/20/bashumerate-enumerator/
121•wallach-game•12h ago•101 comments

How we measured AI writing across arXiv, and where the measurement breaks

https://unslop.run/blog/measuring-ai-writing-on-arxiv
219•dopamine_daddy•15h ago•153 comments

Claude Fable produced a counterexample to the Jacobian Conjecture

https://xcancel.com/__alpoge__/status/2079028340955197566
746•loubbrad•1d ago•471 comments

Perfection is not over-engineering

https://var0.xyz/posts/perfection-is-not-over-engineering.html
236•var0xyz•18h ago•101 comments

Corners Don't Look Like That: Regarding Screenspace Ambient Occlusion (2012)

https://nothings.org/gamedev/ssao/
165•firephox•17h ago•70 comments

Hacker wipes Romania's land registry database

https://news.risky.biz/risky-bulletin-hacker-wipes-romanias-entire-land-registry-database/
637•speckx•18h ago•348 comments
Open in hackernews

I wrote an bash enumerator because I was sick of xargs

https://numerlab.org/2025/07/20/bashumerate-enumerator/
121•wallach-game•12h ago

Comments

wallach-game•12h ago
bashumerate — iterate over files, lines, ranges, or lists with a consistent {} syntax. No for loops, no find -exec, no xargs flags to remember. enumerate -f '*.sh' -- wc -l {} enumerate -L a b c -- 'echo {}' Under 150 lines of bash, pluable sources, NUL-safe. https://github.com/wallach-game/bashumerate
figmert•10h ago
Have you seen parallel?
ggm•10h ago

  find -print0 | xargs -0 -I {} "the {} iterated command"
koolba•10h ago
And for most of those commands add a dash dash so that nothing with a dash prefix turns into options.
somat•9h ago
Xargs is fine, but openbsd has a -J option and every time I read the man page to figure out how to use it I read the -I and -J options and my brain glazes over.

https://man.openbsd.org/xargs

Other brain glazing obsd wierdness is it's two argument cd command, a cryptid I am unable to wrap my head around.

https://man.openbsd.org/ksh#cd~2

0xbadcafebee•9h ago
I highly recommend memorizing the POSIX version of every *NIX command and sticking to them. 60% of the time, they work every time. (https://pubs.opengroup.org/onlinepubs/9799919799/idx/utiliti...)
ggm•9h ago
BSD vs linux with a standards committee inbetween.
fiddlerwoaroof•8h ago
In a world with nix and where nearly every system has zsh, restricting yourself to the POSIX version of these tools is masochistic.
JoshTriplett•8h ago
I highly recommend using whatever capabilities are convenient of any utility you're running, and not caring about what other systems do unless you're actually trying to write a portable shell script.
em-bee•9h ago
why would you ever pipe find into xargs instead of calling -exec?

    find -exec the '{}' iterated command ';'
gumby•9h ago
Because xargs is faster. Exec will invoke the command once per matching file (which is sometimes what you want, of course)!

While xargs will accumulate a bunch of file names, then when it as n names will invoke the command with those names, while continuing to accumulate names until n is reached or the pipe closes.

The size of n depends on the system, but is usually at least a thousand.

em-bee•8h ago
the example as given does not accumulate, so that is what i worked with.

    find -exec command '{}' '+' 
accumulates file arguments too. the only advantage of xargs is that you can tell it how many arguments to accumulate,̶ ̶t̶h̶e̶ ̶d̶o̶w̶n̶s̶i̶d̶e̶ ̶o̶f̶ ̶x̶a̶r̶g̶s̶ ̶i̶s̶ ̶t̶h̶a̶t̶ ̶y̶o̶u̶ ̶h̶a̶v̶e̶ ̶t̶o̶ ̶s̶p̶e̶c̶i̶f̶y̶ ̶t̶h̶e̶ ̶n̶u̶m̶b̶e̶r̶,̶ ̶w̶h̶e̶r̶e̶a̶s̶ ̶f̶i̶n̶d̶ ̶j̶u̶s̶t̶ ̶f̶i̶t̶s̶ ̶a̶s̶ ̶m̶a̶n̶y̶ ̶a̶s̶ ̶i̶t̶ ̶c̶a̶n̶.̶
unsnap_biceps•8h ago
xargs will fit as many as it can in 128 KiB or the system limit, which is smaller, so in practice, it's almost always pretty similar default packing
em-bee•7h ago
you are right, i forgot to check the default. fixed my comment.
caminanteblanco•10h ago
Obligatory XKCD: https://xkcd.com/927/
butvacuum•8h ago
along with: I love standards- Theres soany to choose from!
eqvinox•10h ago
Not sure I see the point of this. Remembering a bunch of options on one tool is no better than a bunch of tools/constructs? Especially if the latter are useful elsewhere. And it can't be used for scripts unless you start shipping it alongside, which… nah.

Just go with

  foo | while read X; do bar "$X"; done
fiddlerwoaroof•8h ago
I just use while’s default variable name, $REPLY most of the time. But `while` is my preferred tool for the xargs problem in most cases.
JoshTriplett•8h ago
I use "| while read" as well, because it works well in a pipeline and handles embedded spaces. It doesn't handle embedded newlines, but in practice, real files have embedded spaces, while embedded newlines only happen in test cases and exploits. (You can do actual NUL-delimited reads with `-d ''`, but for a quick command-line operation that's generally not necessary, and if you're going to be that careful you probably also need `-r`.)
laughing_man•7h ago
Depending on the actual command, this can be far slower and less efficient than xargs. You're creating a separate process for each invocation of bar when a lot of commands will take many targets for a single invocation.

Try this with find and grep vs xargs. There's a big difference.

eqvinox•3h ago
3 reasons against that:

* in a lot of cases the performance just doesn't matter

* xargs gets you the spaces in filenames landmine

* some commands don't even support multiple target filename arguments

For scripts in long term use, yeah, sure, figure out xargs maybe. Any other situation with a "| while read" solution, especially on an interactive session, is an oddball and simplicity wins.

tester457•9h ago
On the topic of xargs replacements, I love gnu parallel.

The --dry-run flag of parallel made me confident to do more batch processing than I ever did with xargs.

Parallel has an option for almost everything, it's almost too much.

But I have shopped around for alternatives. The creator Ole Tange maintains a painstakingly long article of the alternatives and their differences. [0]

The gnu parallel book and reading materials [1] are excellent too.

[0] https://www.gnu.org/software/parallel/parallel_alternatives....

[1] https://www.gnu.org/software/parallel/#Tutorial

quotemstr•9h ago
I love GNU Parallel too. I love how it's free software and how I can patch out the obnoxious citation notice.
somat•9h ago
I have used echo as a sort of poor mans equivalent for a safe check of a pipeline, removing the echo when I felt the rest of the pipeline was working correctly.

  shell stuff | xargs -n 1 -I % echo real command and % args
I have also been known to write scripts where instead of executing the critical parts it prints them. Then a dry run is

  script
and the real run is

  script | sh
bryancoxwell•7h ago
Love that idea, thanks for sharing
krackers
raggi•9h ago
in zsh you can just write:

   for x (*.sh); echo "before $x after"
or

   for x in *.sh; echo "before $x after"
or

   for x (*.sh) { echo -n "before "; echo -n $x; echo " after" }
PunchyHamster•8h ago
missed the point. reason to use xargs or parallel are generally two: list of arguments is too long, or list of arguments that is kept in memory would take too much

for example

    for a in `find / ` ; do echo $a ; done
will take A LOT of memory, while using find's -exec, xargs, or parallel will not
foobarqux•8h ago
But bashenumerate doesn't do that? The parent is saying you should zsh shortloops instead of bashenumerate not zsh shortloops instead of xargs.
unsnap_biceps•8h ago
I looked into the source and the underlaying source uses a glob and suffers from the issue you brought up.

https://github.com/wallach-game/bashumerate/blob/master/lib/...

raggi•2h ago
In my experience once you get to the point where you run out of space in the glob you’re often suffering with poor performance from spawning children as well and it’s time to move to a more formal program even if it’s a script, writing out work plans and completions to list files to avoid wasted time. It’s often a great guardrail to remind you to do this
samtheprogram•9h ago
Literally just use xargs with -I {} and quotation marks?
laughing_man•7h ago
That won't save you from weird file names, but null termination will.
samtheprogram•6h ago
Passing "{}" handles most sane cases including spaces. If I'm doing bash that needs to be robust (rare and/or dotfiles) or know the folder/dataset has weird filenames, sure.
0xbadcafebee•9h ago
I wanted something simpler — one consistent way to iterate over anything.

That's not actually simpler though. Simple is removing everything unnecessary. You took commands which could already do what you wanted, and added an extra program which calls them in specific ways. This will add bugs and maintenance headaches, not be portable, etc. This is added complexity.

The reason you made this script is not because you wanted simpler, you wanted easier. There's nothing wrong with that, and I'll grant you it probably is, especially for those unaccustomed to these commands. But easier != simpler. Often you'll find that simple is hard and easy is complexity deferred.

montag•3h ago
Hm. Underrated comment
loremm•8h ago
I have found that the most reliable way I like is to just construct the command externally and then pass to gnu parallel (mostly for --eta and --tmuxpane). And the great thing is, as others say, xargs -I. I prefer for shortness (and few collisoins), '@'

seq 1 10|xargs -I@ echo 'bash run.py @'|parallel -j 10

I know the echo is a little silly but then I can remove the |parallel and see if it's right. And if I don't want parallelism, I just pass to bash

PunchyHamster•8h ago
I feel like it could be just alias to some particular GNU Parallel set of options
chasil•8h ago
I use:

  find . -name '*.log' -print0 | xargs -0 rm
For this simple example (derived from the article), find also has a delete operator.

This null termination is now a POSIX standard.

ivlad•8h ago
Was exactly my thought: why not to use null termination?

Looks like a case where reading man page would have spared writing another copycat utility.

rmunn•5h ago
And null termination is guaranteed to work, because the only two characters forbidden in Unix filenames (for most varieties of Unix, I won't guarantee there aren't some weird variants out there) are / and null.

The only times I've needed something more than `find -print0 | xargs -0` has been when I need to apply logic to decide whether to process one of the files, in a way that's not easy to express in a `find` command. Then I write a small script with a for loop and if statements inside it.

But more people should know about `-print0`. It's the answer to 95% of the problems with `find | xargs`.

Walf•6h ago
Yeah, and I invariably add -r to xargs, so as not to execute the command unless results come through the pipeline.

If one is working with whole lines of text, setting the delimiter to newline is often desirable:

xargs -d \\n

teo_zero•5h ago
Good idea but strange syntax. It would be more idiomatic if the command came first and the list of globs last.

Additionally the use of "--" is not what everybody expects: here it is used to introduce one argument, the command, while it's usually meant to introduce multiple arguments without worrying if they have a leading "-".

A possible revised syntax with command as the first argument followed by a list of globs optionally introduced by "--" would allow to enumerate all files with a leading "-", which the current syntax cannot:

  enumerate 'whatever {}' -- '-*'
I'm assuming "-f" for simplicity, but the same reasoning holds for "-L" too.
montag•5h ago
I thought -- was typically a separator for passing a list of arguments directly to some subcommand
altairprime•2h ago
The historical meaning is, "do not interpret command line arguments following --"; e.g. the classical form shown by `echo > -f; rm -- -f`. Git has a more complex interpretation of it and no doubt there's others, but this root interpretation remains generally sound: It acts as a boundary between 'complex and intelligent processing of @ARGV elements' and 'every remaining element of @ARGV after -- is treated a string literal without further processing'.
dboreham•5h ago
55 comments and nobody mentioned the incorrect English in the title?
fhn•3h ago
I'm glad you here to save us. What would we do without you?
db48x•5h ago
> Or maybe you pipe into xargs and pray your filenames don’t have spaces…

Always use -0. Most gnu utilities support it. It makes them put a null byte after every filename instead of a newline. Completely eliminates the problem of dealing with whitespace in the filenames.

AdieuToLogic•4h ago
To support your recommendation and redress the strawman the article postulates, the post's author could have replaced:

  find . -name '*.log' | xargs rm
With:

  find . -name '*.log' -print0 | xargs -0 rm
jsrcout•43m ago
For tools that don't support -0, you can add the NULs yourself without much fuss. It's a one-liner in awk/perl/sed. Very handy.
gfalcao•5h ago
> Or maybe you pipe into xargs and pray your filenames don’t have spaces…

Most of this, if not all, is fixable by adding a `export IFS=$'\n'` to your bashrc. I'm not trying to disregard your project, just point out something that took me years to learn and I currently use extensively to solve this very problem. Perhaps you didn't know about it until now... :)

gfalcao•3h ago
Another trick that took me years to learn is to use `xargs -I` to split the results into "items"

For example

`ps aux | grep process-name | grep -v grep | awk '{ print $2 }' | xargs -Ieach kill -9 each`

ivlad•2h ago
Or, you use `xargs -0` for null termination instead of white space termination. `find` conveniently supports `-print0` that will use null character as separator.
AdieuToLogic•4h ago
> Ever found yourself writing this?

  for f in *.txt; do
    wc -l "$f"
  done
No, because `wc` accepts multiple files. And the example given is incorrect for any file having a `.txt` suffix and whitespaces.

> Or this?

  find . -name '*.sh' -exec wc -l {} +
No.

> These all work, but each has its own syntax, its own flags, its own quirks. I wanted something simpler — one consistent way to iterate over anything.

And therein lies the proverbial xkcd standards[0] proof.

0 - https://xkcd.com/927/

G_o_D•4h ago
Atleast follow standard practice in your example

Fails : enumerate -f '.txt' -- 'wc -l {}'

For all use cases

Correct : enumerate -f '

.txt' -- 'wc -l -- {}'

fhn•4h ago
calling it 'bashnumerate' and then have the command be 'enumerate' is confusing. find one and stick with it.
scrame•2h ago
find -print0...|xargs -0... works for me, and i don't always want to execute something and xargs can run parallel processes. i feel like this guy never bothered to read the man page.
zxexz•2h ago
I don’t want to be a downer here, but the structure of this repo and the verbosity+language of the docs feel 80% vibecoded. Not that that’s wrong; I just feel kinda gullible for even clicking on this.

One uses xargs or parallel only a few times before they remember some of the quirks that but them. And then they become cautious. And then it’s muscle memory. And if it’s not an often occurrence, they learn to check the man page.

Anyways, in a world of “vibecoding” why add another “tool” to the mix when the LLMs have been trained on all our stackoverflow-posted grievances to begin with?

dundarious•2h ago
If you want a shell to interact with the results, you can of course just use a (sub)shell.

    ls -1 ./*.sh | xargs -rd\\n sh -c 'for i in "$@" ; do ... ; done' sh
1. not strictly necessary to use -1 as I believe all common ls detect !isatty(stdout) and produce line-by-line output anyway.

2. xargs -r just doesn't run the command if there's no input, also not strictly necessary but I'm addicted to using it because it's the sensible default to me.

3. xargs -d\\n makes it collect fields as full lines, which is what you typically want, unless you're able to generate NULs.

4. use whatever shell you want of course, but I don't use bashisms, etc., by default, /bin/sh is fine for me, even if it's dash.

5. the trailing "sh" at the end is due to a quirk of `sh -c` usage, where $0 is the first non option argument, so `printf %s\\n 1 2 3 | xargs -rd\\n sh -c 'for i in "$@" ; do printf "%s " "$i" ; done ; echo'` (note the lack of trailing "sh") would only print "2 3 " as $0 is not included in "$@" ($0 is 1, $1 is 2, $2 is 3). It's very easy to just always give the shell name itself manually as $0 instead of trying to ingest "$0" into your logic.

Of course, you could `find . -maxdepth 1 -type f -name '*.sh' -print0 | xargs -r0 ...` instead, depending on what you're up to, that may be the easiest. It's definitely the simplest -- as long as your xargs has -0 support.

IdiotSavage•1h ago
In this thread: bash experts with arcane knowledge, unintentionally demonstrating how awful bash is.

The obvious solution would be to use something more sane, like PowerShell or nushell, but instead old experts will always defend the skills they have honed for years, while criticizing anything that's different.

lobofta•1h ago
Switched to Nushell and I am not looking back. I don't see any major reason why we should keep dragging Bash into the twenty first century. Nushell is the first time I feel like I can write complex systems operations in a shell without having to spend either a ton of time in the docs or being at the mercy of an LLM. It is godsend.
kfsone•1h ago
I was the kind of shell guru whose teeth itched when they saw someone doing `grep | awk`. Then I had to try and bring Windows into a Mac+Linux CI system under Jenkins groovy files that were full of `"""sh` fragments, shell scriptlets wrapping python, etc, etc, and I don't bat (I don't groovy either).

Option 1: Learn to bat and try to translate. Hmm. No. Just no Option 2: ?

Pwsh core had just come out. My immediate thought was that it would be great material for an anti-MS-ragging blog post, but then a line leapt out at me from one article I was glossing over: "... POSIX Terminal Shell Spec ...".

I still wanted that anti-MS-ragging blog material, so I decided to try and use Pwsh as a Rosetta stone until I got to a point I could convert to a real language.

But things just began to click for me. It was like going from Perl to Python - suddenly everything is an object and you can interact with

everything* that way.

There's no need for grep or awk or sed in pwsh, because the output of a shell command is an object -- a string (or []byte). It has methods.

(netstat -an).replace("192.168.86.", "10.0.100.")

6 years later, pwsh is the default shell on my Mac, Ubuntu boxes, lab vms, ... everything but my docker containers unless I'm feeling feisty.

pjmlp•1h ago
One thing with classical UNIX commands, is that you can expect to find them into random computers besides one's own laptop.

Not everyone has the luxury to only work with their own computer, or run random software on IT/customer managed systems.

kfsone•1h ago
Careful - you start from the POSIX terminal spec, you think maybe it would be interesting having objects instead of raw text streams, and next thing you've reinvented powershell...

(*35 years living and loving sh/ksh/bash/dash etc, only tried pwsh so I could write some comparisons and slag off MS a bit; now it's my default shell on everything)

account42•1h ago
> Consistent syntax — same {} placeholder for files, lines, ranges, or lists

Inconsistent syntax native bash methods so its an additional syntax to learn.

> Template mode — single-quoted commands work as shell templates: enumerate -f '*' -- 'cat {} | head -5'

Passing commands a a single string is BAD. Now you have to think about escaping and quoting. What does {} get replaced with if the enumerant contains unsafe characters? Can it be used as part of a larger argument or only on its own? Who knows, it's not bash. Compared to a bash loop its also always a subshell with all the implications that has - even find can be piped into a normal bash loop.

> Filters — --include and --exclude with glob patterns

A fraction of what find or native loops provide. And since this can't replace them in general enumerate is an additional thing to learn on top.

> Extensible — drop a file in lib/enumerators/ to add custom sources

To extend bash loops you don't even need root access, you just add the code to the loop.

iririririr•22m ago
being consistent in a sea of inconsistency is a good thing, no matter how hard you try to paint otherwise. the solution have to start at some point. But the rest, yeah, scary.
vladde•26m ago
i often find xargs ends up biting me, and i have wanted some alternative for a while... but i don't think this is the one for me.

it feel like the syntax here is odd. it still requires me to write quote my command i want to run? unfortunately i'll have to pass on this.

personally i'd want some variant where i can still auto-complete commands and have just have {} as a placeholder. (maybe time to learn how to use xargs for real?)

jllyhill•17m ago
Did you write it or did Claude code slopcoded it for you? Claude is the contributor to all your other repositories. There is a world of difference between "here's a problem that I'm really concerned with and poured all my expertise to solve it" and "I told Claude to fix it for me and now I'm gonna abandon it as soon as I'm done with the HN advertising".
cryo32•10m ago
https://imgur.com/a/K9TPYgf
fiddlerwoaroof•8h ago
The two arg cd is pretty useful (and, zsh implements it too because cd is a shell feature): if you have parallel directory structure (e.g. the way rails does tests) you can switch from app/b/c/d to spec/b/c/f by doing `cd app spec`
xorcist•7h ago
The problem with find is that you can't specify the maximum tolerated command line lengths, and find hasn't historically been very smart about it (it just had a compiled-in value). Apparently this is fixed in GNU find, but for older systems and other platforms that may still be an issue.

Another feature missing in find that xargs has is the maximum processes to start at a time. find will run the commands in sequence, but in many situations you really want to run a bunch in parallel.

em-bee•6h ago
good points that i wasn't aware of, thank you. though personally i rarely start huge operations that need to be parallelized so i prefer simplicity over speed.
ggm•8h ago
The prior has a point because 98.89% of the time I type xargs -n 1 -0 and we are deep in the useless pipe argument which Rob Pike amongst others has rehearsed well.

I do it because I do it, just like why I use egrep and sed in pipes along with awk.

t-3•8h ago
Leah Neukirchen's lr and xe are very nice as a find and xargs replacement (although lr's test flag is way too complex). lr *.file | xe -s ' ... ' is a really great pattern for iterative scripting and hard to get wrong.
laughing_man•2h ago
xargs supports null termination.

While it's true not every command supports multiple targets, presumably you know if you're using one of those commands.

xorcist•7h ago
Bash while loops are pretty readable and the above would be a nice to iterate over lines if it wasn't for that gnarly pipe, which is a common source of errors in this construct. Remember that a pipe starts a new shell. So:

  grep stuff file.txt | while read key value ; do [ "$key" = "target" ] && found="$value" ; done
where you might expect $found to end up with the value for the line that has "target" in the first column. Then you notice that darn pipe symbol. The variable found is set in a subshell that terminates and the value is lost. This is a problem every time you need to keep some sort of state when looping. If you can tolate a bash-ism then you could do:

  while read key value ; do [ "$key" = "target" ] && found="$value" ; done < <(grep stuff file.txt)
but that doesn't read as nice and isn't compatible. It does avoid a common source of problems though, and might be worth getting into muscle memory for the times it is needed.
dylan604•6h ago
> and isn't compatible

is that a GNU v POSIX type of compatibility issue?

eqvinox•3h ago
bash vs. POSIX sh (or some other shells)

zsh should be bash compatible on this AFAIR

eqvinox•3h ago
You can put the "< <( foo )" before the "while" btw, for pipe-like ordering of things. All redirections can be anywhere on the command line.
t43562•1h ago
While FTW! I just wondered whether it could be simplified because I get a bit tired of typing out my own belt and braces version of it:

  { # code that generates one item/filename per line of output }  | { while read ITEM; do # do something with "$ITEM"; done; }
So I just tried this and it seems to work for a trivial case although you need 1 escape:

  enum() {
    local source=$1; shift; local action=$1; shift  
    { eval "$source" ; } | { while read ITEM; do eval "$action"; done; }
  }

  $ enum "find /tmp" "echo found: \$ITEM"

  found: /tmp/.XIM-unix
  found: /tmp/.ICE-unix
•
7h ago
Shouldn't you use echo instead of cat?
somat•7h ago
yes echo, total brain fart there. corrected.
dataflow•9h ago
Oh man, parallel is awful. I have to rant about this because I literally tried it again today morning.

Every single damn time I try parallel and decide to give it another chance, something ends up not working or causing a problem. I can never get it to just do what I want and get out of the way.

Today I foolishly thought maybe I was the one who was holding it wrong every single time in the past, so I copy pasted another command that was supposed to work, and thought surely this would be straightforward. Boy was I wrong. I got some manifesto about academic citations and plagiarism, which confused the hell out of me. After I wasted time trying to figure out how to turn off that nonsense, the app just hung there trying to figure out how long its command line can be? Literally doing nothing? What the hell? I killed it but then my terminal didn't close because every time I did this apparently some perl command was spawned in the background blocked on nothing. Why the hell was perl even relevant? Nothing I wrote used Perl. Just run the darn commands I asked in parallel, is that so hard?

nvme0n1p1•8h ago
> the app just hung there trying to figure out how long its command line can be?

That's common on linux. Many tools read from stdin if a file path isn't given: cat, xargs, base64, cksum, etc.

The citation thing is a little silly, I'll give you that one.

dataflow•2h ago
> That's common on linux. Many tools read from stdin if a file path isn't given: cat, xargs, base64, cksum, etc.

No. I did pipe to stdin. It's not my first time using Linux...

Here's a command line I ran right now, and the output I see:

  $ echo "http://www.example.com" | parallel -k -j 8 curl -s "{}"
  Academic tradition requires you to cite works you base your article on.
  [...more nonsense...]
  To silence this citation notice: run 'parallel --citation' once.
  
  parallel: Warning: Finding the maximal command line length. This may take up to 1 minute.
So I wait a few seconds... until I get fed up and look at my process list, and I see perl is just... seemingly sitting there, doing seemingly absolutely nothing. I'm not going to waste a whole minute of my life waiting for this; I see no reason competently written software should take that long just to accomplish such a simple task where nothing is remotely close to reaching any limits.

So I Ctrl+C. And then the parent perl process gets killed, but the child apparently keeps running.

I press Ctrl+D to exit the terminal, and then:

  $ # (Ctrl+D pressed)
  logout
...it just sits there waiting. Ctrl+C and Ctrl+\ do nothing. I have to kill the lingering perl process manually.

xargs Just Works without any of this nonsense, yet somehow I'm the one holding GNU parallel wrong?

darrenf•2h ago
> parallel: Warning: Finding the maximal command line length. This may take up to 1 minute.

> So I wait a few seconds [snip]

This warning is only ever printed if running in Cygwin, not Linux or macOS or elsewhere. Cygwin is notoriously slow.

    # This is slow on Cygwin, so give Cygwin users a warning
    if($^O eq "cygwin" or $^O eq "msys") {
    ::warning("Finding the maximal command line length. ".
          "This may take up to 1 minute.")
    }
Also note that it only figures this out first time, after which it’s cached on disk.
dataflow•41m ago
This is on MSYS2, yes, and that excuses absolutely nothing, because this shouldn't be happening in the first place for speed to be even relevant. At the risk of repeating myself thrice: the messages are confusing, the Ctrl+C handling is just utterly broken, the citation message adds to the confusion while being frankly obnoxious, and all of the delays and outputs are unnecessary in the first place as proven by literally every other program that doesn't make me wait a minute before I can use it the first time, including xargs. If Microsoft's own Windows tools did this, everybody would bash them (no pun intended) till the end of time. But since it's GNU Parallel and not Microsoft Parallel, it's Windows's fault for being slow and also my fault for having the audacity to expect better, apparently.
fn-mote•7h ago
> Today I foolishly thought maybe I was the one who was holding it wrong […]

Apparently goes on to describe being confused about parallel reading from the standard input?

dataflow•2h ago
No: https://news.ycombinator.com/item?id=48988369
shawn_w•5h ago
>Why the hell was perl even relevant? Nothing I wrote used Perl.

parallel is written in perl.

tadfisher•3h ago
This is pretty funny. You almost had me until the Perl part. Not going to get baited this time!
brokenmachine•1h ago
I'm not new to command line tools, but I somehow managed to delete ~10k pictures while trying to use parallel to resize them.

I forget the details but it was some kind of surprisingly weird foot-gun behavior.

Luckily I had a backup, but it really has made me scared to try using parallel again.

jeffbee•8h ago
Parallel is ok but rush is nicer and much faster.

https://github.com/shenwei356/rush

foobarqux•8h ago
The rush project page on Github itself says it is only "slightly" faster.
jeffbee•5h ago
Modesty, I suppose. The larger your machine is, the more obviously faster it is.
metadat•8h ago
Every time I’ve used Gnu Parallel on a new system it required accepting a Eula, which is annoying. I stick to xargs -P99 these days and am happier.

Utility software which has non-essential different first-run behavior is hostile to users.

sidewndr46•5h ago
isn't parallel that tool that dumps some kind of begging message into your output each time you invoke it?
shawn_w•5h ago
No? You can tell it to stop printing the citation notice.
saghm•41m ago
That sounds a lot more like "yes, but..." than "no"
no-name-here
•
53m ago
I also love pwsh - I’m curious why it’s your default on Mac, Ubuntu boxes, and lab vms, but not docker containers?
iririririr•19m ago
ideally containers run a single process. if you're spawning shells you're doing something wrong. and idealier, you should use just bare processes with namespaces and abandon docker.
amelius•49m ago
Yes, Bash has no place in modern software engineering.

If it was invented today, professionals would cringe at it.