It has a very different purpose than a post like this though! Also: there's probably more effort at exposition in this blog post than in all of Math Academy's coverage (that's not a dunk on Math Academy).
haha definitely agree. a lot of these blog posts are great if you want to read about math. mathacademy is pretty much all the exposition chopped out and you spend 90% of your time doing math. I can see how some wouldn't like it but I think the problem solving aspect makes it was more useful for bruteforcing your way towards building intuition
This is part 2 of a series, all under the same name; the first part is extensively illustrated (and I'm not sure the part 1 illustrations are all that helpful).
The author seems to be taking a different tack though, and maybe doesn't want to be too tied to this particular geometric picture
"An expert in university-level linear algebra, including solving systems of equations, matrices, determinants, eigenvalues and eigenvectors, symmetry calculations, etc. - is asked the following question by a student: "This is all great, professor, and linearity is also at the heart of calculus, eg the derivative as a linear transformation, but I would now like you to explain what distinguishes linear from non-linear algebra."
"What kind of trouble can the student of physics and engineering and computation get into if they start assuming that their linear models are exact representations of reality?"
"A student new to the machine learning field states confidently, 'machine learning is based on linear models' - but is that statement correct in general? Where do these models fail?"
The point is that even though it takes a lot of time and effort to grasp the inner workings of linear models and the tools and techniques of linear algebra used to build such models, understanding their failure modes and limits is even more important. Many historical engineering disasters (and economic collapses, ahem) were due to over-extrapolation of and excessive faith in linear models.
Many times I've been puzzled by a concept just to go there and find it made simple and obvious. It's a real golden nuggett... Plus if you then want to go further into groups, rings, fields, and Galois theory, that's also there.
Nope. This is incorrect. The dot product is a weighted sum of a vector's elements, where the weights are the elements of the other vector. Weighted sum of two vectors would require a third entity to provide the weights.
I teach math by leading with examples. I try to show the intuition behind an idea, and why it is interesting. For this series, my reader is someone who knows algebra, and likes learning new things, especially when a teacher shows what is interesting about a topic.
## You didn't cover x about the dot product.
I try to only teach as much as is necessary to get the student to the next point, which is matrix multiplication. I usually end up cutting a lot of material out of my chapters to keep them simple. In this case, I cut out a whole section on the properties of a dot product, as well as a discussion about inner and outer products, because those weren't necessary to get to matrix multiplication. I think this context was lost while posting to HN.
## 3B1B already has a series on this.
I love 3B1B, but his style of teaching and mine are quite different. Even though we both teach visually, his videos are densely packed with information and his expectation is that you will watch the video a few times till you understand the topic. He also leads with math more than I do. My posts are written more like stories. My goal is they should be easy to get into, and by the time you have finished reading, you should understand more about the topic. I don't expect readers to read through multiple times. I personally learned linear algebra through Strang's videos and textbook, and those videos are awesome, but can be confusing. If you found the Strang or 3b1b videos confusing, hopefully my posts will make it easier for you to follow them. I think comment is spot on: https://news.ycombinator.com/item?id=45800657
If these ideas resonate with you, I think you'll like this post, and if not, there are plenty of guides that go the more traditional route. You can also read the first post in the series and see if you like it: https://www.ducktyped.org/p/an-illustrated-introduction-to-l...
For another example of my writing, see my series on AWS: https://www.ducktyped.org/p/a-mini-book-on-aws-networking-in...
seanhunter•4h ago
Firstly, the real illustrated guide to linear algebra is the youtube series "The Essence of linear algebra" by 3blue1brown[1]. It has fantastic visualisations for building intuition and in general is wildly superior to this, which seems fine but extremely superficial.
If you're done with 3b1b and want to take things further, then the go-to is the excellent 18.06SC course by the late and legendary Gilbert Strang. It's amazing, it's free. [2]
Still want more? OK now you're talking my language. If you are serious about linear algebra (Up to graduate level, after that you need something else) then you want the book "Linear Algebra Done Right" by Sheldon Axler. It's available for free from the author's website[3] and he has made a bunch of videos to supplement the book. There's also an RTD Math full lecture series[4] that follows the book and he explains each thing in a lot of detail (because Axler goes fast, so it's beneficial to unpack the concepts a bit sometimes).
[1] https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...
[2] https://ocw.mit.edu/courses/18-06sc-linear-algebra-fall-2011...
[3] https://linear.axler.net/ and https://www.youtube.com/watch?v=lkx2BJcnyxk&list=PLGAnmvB9m7...
[4] https://www.youtube.com/watch?v=7eggsIan2Y4&list=PLd-yyEHYtI...
sgdpk•3h ago
seanhunter•3h ago
incognito124•3h ago
http://linear.ups.edu/
barrenko•1h ago
selimthegrim•3h ago
tptacek•3h ago
tptacek•3h ago
But I'm not sure I've ever really learned anything from one of those videos. Appreciated something more? Absolutely. And maybe, sure, that's a kind of learning. But I cringe every time eigenvalues come up and people point to the 3B1B evector video.
In fact, if your goal is to actually get any kind of facility with the concept, this "weaksauce" blog post probably has a better didactic strategy than 3B1B. It strips the concept down, provides specific, minimized worked examples, and provides a useful framing for the concept (something basically at the core of 3B1B's process).
I learned linear algebra from Strang's 18.06. I later did a bunch of Axler helping my daughter through UIUC linear algebra. I like both. Strang is much closer to what the median HN person probably wants. In both cases though: don't do what I did at first, and just watch the videos and read the book. If you're not doing problems, you're probably not learning anything.
This blog post comes closer to "actually doing problems" than 3B1B. Ergo: its sauce is stronger, not weaker.
I came to the blog post expecting to roll my eyes. No discussion of inner product spaces? Not even a mention of conjugate symmetry? I was pleasantly surprised.
It's not easy to come up with a simple, accessible framing for a topic like this, and, maybe, the dot product is particularly tricky to give an intuition for (I'll go out on a limb and say that neither Strang nor Axler do a particularly great job at it --- "it" being, explaining the "why" of the dot product to someone who doesn't really even know what a vector is). The post doesn't purport to teach all of linear algebra. It's just an exercise in trying to explain one small part of it.
I'm not asking you to give the author a break, so much as suggesting that you're closing yourself off from appreciating different strategies for explaining complex topics, which is a valuable skill to have.
creata•1h ago
And maybe I'm being a pedant, but the dot product should be between two things that are in the same space. In the Minnesota lottery example, the probabilities should be a row vector instead. It's the exact same calculation, so again, maybe a bit too pedantic.
tptacek•59m ago
Also: sir, this is a blog post. It's wild seeing people say "if you really want to understand this topic, pick up Axler". I mean, yeah, also if you were serious you could just enroll in your local community college's Linear Algebra course.
My feeling is that a lot of the critique here is really signaling. For whatever reason, linear algebra is super high-status in this community, and people want to communicate that they've done something serious with it. (I'm sure I'm guilty of that too.)
creata•42m ago
I agree. I don't think "Linear Algebra Done Right" is a good fit for most people. It's way too dry, and I don't think his crusade against determinants helps the book. I don't know what a good book suggestion would be, though. Maybe just nab the course notes/slides/exercises off some university's website?
> this is a blog post
Blog posts can be and often are amazing.
> My feeling is that a lot of the critique here is really signaling.
Weird accusation, so just to be clear, I haven't done anything serious with anything, ever.
tptacek•33m ago
creata•17m ago
I haven't read Strang's book, so I can't comment on that. But yeah, if it never mentions the formula ||a||·||b||·cos(θ) or at least talks about how the dot product relates to parallel and orthogonal vectors, I would consider that a big hole in an introduction to the dot product.
> How would that have fit with the goals of this post?
Because the post is titled "an introduction to linear algebra... the dot product", and this is something that I believe should be in anything that considers itself an introduction to the dot product.
You seem to disagree, and I'd like to ask: why? I think this a fundamental aspect of the dot product, again, just as fundamental as the relationship between complex multiplication and rotation. I think my view is common.
> calling out a blog post
I didn't intend to do anything as strong as "call out" the blog post. I just wanted to express surprise at someone so strongly praising ("its sauce is stronger [than 3B1B's video series]") an alright post.
seanhunter•41m ago
tptacek•34m ago
photochemsyn•2h ago
https://news.ycombinator.com/item?id=38060159
For most people going into science and engineering as opposed to pure mathematics, Poole's "Linear Algebra: A Modern Introduction" is probably more suitable as it's heavy on applications, such as Markov chains, error-correcting codes, spatiel orientation in robotics, GPS calculations, etc.
https://www.physicsforums.com/threads/linear-algebra-a-moder...
beklein•1h ago