Kudos for that!
~/bin/cooper-db-set
===================
#! /bin/bash
key="$1"
value="$2"
echo "${key}:${value}" >> /dev/null
~/bin/cooper-db-get
===================
#! /bin/bash
key="$1"
</dev/null awk -F: -v key="$key" '$1 == key {result = $2} END {print result}'
Exercises like this also seem fun in general. It's a real test of how much you know to start anything from scratch.
Otherwise great article, thank you!
I would say without transactions it is not a database yet from a practical standpoint.
Not that I would aspire to implement a general-purpose database. But even smaller tasks can make my mind spin too much.
As a different example: I'm moving this week. I've known I'm moving for a while. Thinking about moving -- and all the little things I have to do -- is way more painful than doing them. Thinking about them keeps me up at night, getting through my list today is only fractionally painful.
I'm also leveling up a few aspects of my "way of living" in the midst of all this, and it'd be terribly boring to tell others about it, but when next Monday comes.. it'll be quite sweet indeed.
this sounds familiar... :)
Edit: the flush example (2nd one) in the recap section does the same thing, when the text says that the records are supposed to be written to the file in sorted order.
I wish this came out earlier, there are a few insights in there that took me me a while to understand :)
Maybe give credit?
This is a tad bit misleading, as the LSM is used for the node-level storage engine, but doesn’t explain how the overall distributed system scales to 80 million rps.
iirc the original Dynamo paper used BerkeleyDB (b-tree or LSM), but the 2012 paper shifted to a fully LSM-based engine.
4ndrewl•6h ago
>
> "How do we store data persistently and then efficiently look it up later?"
Isn't that two problems?
dayjaby•6h ago
SirFatty•6h ago
cjbgkagh•6h ago
stvltvs•6h ago
BetaDeltaAlpha•5h ago
stvltvs•4h ago
https://www.sciencenewstoday.org/do-black-holes-destroy-or-s...
SahAssar•6h ago
The "efficiently" part can be considered a separate problem though.
prerok•5h ago
So, if we consider that persistent storage is a solved problem, then we can say that the reason for databases was how to look up data efficiently. In fact, that is why they were invented, even if persistent storage is a prerequisite.
nonethewiser•5h ago
grokgrok•6h ago
Efficiency of storage or retrieval, reliability against loss or corruption, security against unwanted disclosure or modification are all common concerns, and the relative values assigned to these features and others motivate database design.
kiitos•5h ago
reconstructing past memory states is rarely, if ever, a requirement that needs to be accommodated in the database layer
nonethewiser•5h ago
In another context perhaps you're ingesting data to be used in analytics. Which seems to fit the "reconstruct past memory stat" less.
i_k_k•6h ago
elygre•6h ago
Good times.
thomasjudge•1h ago
pcdevils•5h ago
hxtk•5h ago
datadrivenangel•4h ago
0 - https://www.youtube.com/watch?v=3t6L-FlfeaI
archerx•5h ago
warkdarrior•5h ago
mewpmewp2•5h ago
Etheryte•4h ago
pratik661•5h ago
rzzzt•5h ago
theideaofcoffee•4h ago
nonethewiser•5h ago
I guess there is a rather fine line between philosophy and pedantry.
Maybe we can think about it from another angle. If they are 2 problems databases were designed to solve, then that means this is a problem databases were designed to solve: storing data persistently.
Is that really a problem database were designed to solve? Not really. We had that long before databases. It was already solved. It's a pretty fundamental computer operation. Isn't it fair to say this is one thing? "Storing data so it can be retrieved efficiently."
gingersnap•4h ago
mrighele•4h ago
didip•4h ago
whartung•4h ago
No, that would be regexes.
mamcx•2h ago
How, in ACID way, store data that will be efficiently look it up later by a unknown number of clients and unknown access patterns, concurrently, without blocking all the participants, in a fast way?
And then add SQL (ouch!)