Do you see the invisible BOMb in the error message? Neither did I the first time. (And, in fact, Ghostty apparently stripped it out when I copied and pasted, so it's not actually there in this comment). But if I were to load that doit.sh script I created for this example into VS Code, I'd see the telltale "UTF-8 with BOM" file format.
Most people already know this, but maybe this will help someone out there. If you see a "No such file or directory" error and the program being executed apparently starts with #!, it probably actually starts with U+FFEF#! and you need to re-save the script in UTF-8 without a BOM(b) at the start.
Not my editor of choice but some swear by it and are prone to work cross platform across NAS's and SSH terminals with either windows or some *nix as 'primary' work space.
I'm sure other editors have this as an option, the time I ran into BOM issues I traced it back to the use of Notepad++ by a third party.
Irritatingly, you'll find BOMs to not be uncommon in CSV files because of Excel, which interprets files as CP1252 (a superset of the printable characters of ISO 8859-1, sometimes known as Win1252 or Windows-1252) if the BOM is not present, causing anything beyond ASCII to be misinterpreted (accented characters are usually the first thing people in Europe notice getting garbled, currently symbols other than $ too).
Though that would have been a far worse problem a decade ago. Thankfully, these days any random document you encounter on the Web is 99% likely to be UTF-8, so the correct heuristic these days is "if you don't know the encoding, try decoding with UTF-8 first. If that fails, try a different encoding." Decoding UTF-8 is practically guaranteed to fail on a document that wasn't actually UTF-8, so trying UTF-8 first is a very safe bet. (Though there's one failure mode: ASCII encoded as UTF-16. So you might also try a heuristic that says "if the UTF-8 parses successfully but has a null byte every other character in the first 256 bytes, then try reparsing the document as UTF-16").
I'm rambling and getting off my point. The point is, I don't find BOMs in UTF-16 documents to be a problem, because UTF-16 actually needs them. (Well, probably needs them). UTF-8 doesn't need them at all.
When you involve Windows, which likes a random mix of UTF-16, UCS-2, CP1252, and I guess also UTF-8 with BOM, you're screwed.
Ferret7446•2mo ago
See also https://www.felesatra.moe/blog/2021/07/03/portable-bash-sheb...