$ echo -n Vm0 | base64
Vm0w
(Because the output is necessarily 8/6 the size of the input, the suffix always adds 33% to the length.)
#!/usr/bin/env python3
import base64
def len_common_prefix(a, b):
assert len(a) < len(b)
for i in range(len(a)):
if a[i] != b[i]:
return i
return len(a)
def calculate_quasi_fixed_point(start, length):
while True:
tmp = base64.b64encode(start)
l = len_common_prefix(start, tmp)
if l >= length:
return tmp[:length]
print(tmp[:l].decode('ascii'), tmp[l:].decode('ascii'), sep='\v')
# Slicing beyond end of buffer will safely truncate in Python.
start = tmp[:l*4//3+4] # TODO is this ideal?
if __name__ == '__main__':
final = calculate_quasi_fixed_point(b'\0', 80)
print(final.decode('ascii'))
Vm0wd2QyUXlVWGxWV0d4V1YwZDRWMVl3WkRSV01WbDNXa1JTVjAxV2JETlhhMUpUVmpBeFYySkVUbGhoProbably not a very useful trick outside of certain specific environments
JWT does it as well.
Even in this example, they are double base64 encoding strings (the salt).
It's really too bad that there's really nothing quite like json. Everything speaks it and can write it. It'd be nice if something like protobuf was easier to write and read in a schemeless fashion.
asn.1 is super nice -- everything speaks it and tooling is just great (runs away and hides)
The purpose of Base64 is to encode data—especially binary data—into a limited set of ASCII characters to allow transmission over text-based protocols.
It is not a cryptographic library nor an obfuscation tool.
Avoid encoding sensitive data using Base64 or include sensitive data in your JWT payload unless it is encrypted first.
And of course text-based things themselves are quite wasteful.
And before "space is cheap": JWT is used in contexts where space is generally not cheap, such as in HTTP headers.
You have to ask the question "why are we encoding this as base64 in the first place?"
The answer to that is generally that base64 plays nice with http headers. It has no newlines or special characters that need special handling. Then you ask "why encode json" And the answer is "because JSON is easy to handle". Then you ask the question "why embed a base64 field in the json?" And the answer is "Json doesn't handle binary data".
These are all choices that ultimately create a much larger text blob than needs be. And because this blob is being used for security purposes, it gets forwarded onto the request headers for every request. Now your simple "DELETE foo/bar" endpoint ends up requiring a 10kb header of security data just to make the request. Or if you are doing http2, then it means your LB will end up storing that 10kb blob for every connected client.
Just wasteful. Especially since it's a total of about 3 or 4 different fields with relatively fixed sizes. It could have been base64(key_length(1byte)|iterations(4bytes)|hash_function(1byte)|salt(32bytes)) Which would have produced something like a 51 byte base64 string. The example is 3x that size (156 characters). It gets much worse than that on real systems I've seen.
messagepack/cbor are very similar to json (schemaless, similar primitive types) but can support binary data. bson is another similar alternative. All three have implementations available in many languages, and have been used in big mature projects.
If you just want a generic, binary, hierarchical type-length-value encoding, have you considered https://en.wikipedia.org/wiki/Interchange_File_Format ?
It's not that there are widely-supported IFF libraries, per se; but rather that the format is so simple that as long as your language has a byte-array type, you can code a bug-free IFF encoder/decoder in said language about five minutes.
(And this is why there are no generic IFF metaformat libraries, ala JSON or XML libraries; it's "too simple to bother everyone depending on my library with a transitive dependency", so everyone just implements IFF encoding/decoding as part of the parser + generator for their IFF-based concrete file format.)
What's IFF used in? AIFF; RIFF (and therefore WAV, AVI, ANI, and — perhaps surprisingly — WebP); JPEG2000; PNG [with tweaks]...
• There's also a descendant metaformat, the ISO Base Media File Format ("BMFF"), which in turn means that MP4, MOV, and HEIF/HEIC can all be parsed by a generic IFF parser (though you'll miss breaking some per-leaf-chunk metadata fields out from the chunk body if you don't use a BMFF-specific parser.)
• And, as an alternative, there's https://en.wikipedia.org/wiki/Extensible_Binary_Meta_Languag... ("EBML"), which is basically IFF but with varint-encoding of the "type" and "length" parts of TLV (see https://matroska-org.github.io/libebml/specs.html). This is mostly currently used as the metaformat of the Matroska (MKV) format. It's also just complex enough to have a standalone generic codec library (https://github.com/Matroska-Org/libebml).
My personal recommendation, if you have some structured binary data to dump to disk, is to just hand-generate IFF chunks inline in your dump/export/send logic, the same way one would e.g. hand-emit CSV inline in a printf call. Just say "this is an IFF-based format" or put an .iff extension on it or send it as application/x-iff, and an ecosystem should be able to run with that. (And just like with JSON, if you give the IFF chunks descriptive names, people will probably be able to suss out what the chunks "mean" from context, without any kind of schema docs being necessary.)
"eeey bruh, open the the API it's me"
Actual RSA oid is somewhere in the middle.
`eY` could be any JSON, but it's most likely going to be a JWT.
Neither is a perfect signal, but contextually is more likely correct than not.
I work with this stuff often enough to recognize something that looks like a key or a hash. I don't work with it often enough to have picked up `ey` and `LS`.
The PEM format (that begins with `-----BEGIN [CERTIFICATE|CERTIFICATE REQUEST|PRIVATE KEY|X509 CRL|PUBLIC KEY]-----`) is already Base64 within the body.. the header and footer are ASCII, and shouldn't be encoded[0] (there's no link to the claim so perhaps there's another format similar to PEM?)
You can't spot private keys, unless they start with a repeating text sequence (or use the PEM format with header also encoded).
Spending hours wrangling sendmail.cf, and finally succeeding, felt like a genuine accomplishment.
Nowadays, things just work, mostly. How boring.
When one of my tests crashed one of those unprotected mainframes, two guys who were then close to my age now stared at an EBCDIC core dump, one of them slowly hitting page down, one Matrix-like screen after another, until they both jabbed at the screen and shouted "THERE!" simultaneously.
(One of them hand delivered the first WATFOR compiler to Yorktown, returning from Waterloo with a car full of tapes. I have thought of him - and this "THERE!" moment - every time I have come across the old saw about the bandwidth of a station wagon.)
It doesn’t even need to be much better than ROT13. Security by obscurity is good for this situation.
$ echo '{"' | base64
Vs
$ echo "{\"" | base64
{" is ASCII 01111011, 00100010
Base64 takes 3 bytes x 8 bits = 24 bits, groups that 24 bit-sequence into four parts of 6 bits each, and then converts each to a number between 0-63. If there aren't enough bits (we only have 2 bytes = 16 bits, we need 18 bits), pad them with 0. Of course in reality the last 2 bits would be taken from the 3rd character of the JSON string, which is variable.
The first 6 bits are 011110, which in decimal is 30.
The second 6 bits are 110010, which in decimal is 50.
The last 4 bits are 0010. Pad it with 00 and you get 001000, which is 8.
Using an encoding table (https://base64.guru/learn/base64-characters), 30 is e, 50 is y and 8 is I. There's your "ey".
Funny how CS people are so incurious now, this blog post touches the surface but didn't get into the explanation.
https://web.cs.ucdavis.edu/~rogaway/classes/188/materials/th...
I've been doing this a long time but until today the only one I'd noticed was "MII".
They could just as easily have felt the underlying reason was so obvious it wasn’t worth mentioning.
I know how base64 encoding works but had never noticed the pattern the author pointed out. As soon as read it, I ubderstood why. It didn’t occur to me that the author should have explained it at a deeper level.
I know eyJhbG by heart
These blocks can be considered independent of each other. So for example, with the string "Hello world", you can do the following base64 transformations:
* "Hel" -> "SGVs"
* "lo " -> "bG8g"
* "wor" -> "d29y"
* "ld" -> "bGQ="
These encoded blocks can then be concatenated together and you have your final encoded string: "SGVsbG8gd29ybGQ="
(Notice that the last one ends in an equals sign. This is because the input is less than 3 characters, and so in order to produce 4 characters of output, it has to apply padding - part of which is encoded in the third digit as well.)
It's important to note that this is simply a byproduct of the way that base64 works, not actually an intended thing. My understanding is that it's basically like how if you take an ASCII character - which could be considered a base 256 digit - and convert it to hexadecimal (base 16), the resulting hex number will always be two digits long - the same two digits, at that - even if the original was part of a larger string.
In this case, every three base 256 digits will convert to four base 64 digits, in the same way that it would convert to six base 16 digits.
Besides that, I just spent way too much time figuring out this is an encrypted OpenTofu state. It just looked way too much like a terraform state but not entirely. Tells ya what I spend a lot of time with at work.
This is probably another interesting situation in which you cannot read the state, but you can observe changes and growth by observing the ciphertext. It's probably fine, but remains interesting.
Is this the state of modern understanding of basic primitives?
Also, it seem like the really important point is kind of glossed over. Base64 is not a kind of encryption, it's an encoding that anybody can easily decode. Using it to hide secrets in a GitHub repo is a really really dumb thing to do.
delecti•2h ago
morkalork•2h ago