My first question was: why?
It also looks like it has some improvements for dealing with `null` from Java code. (When I last used it I rarely had to deal with null (mostly dealt with Nil, None, Nothing, and Unit) but I guess NPEs are still possible and the new system can help catch them.)
If anything is slowly down Scala 3 is that, including the tooling ecosystem that needs to be updated to deal with it.
val month = i match
case 1 => "January"
case 2 => "February"
// more months here ...
case 11 => "November"
case 12 => "December"
case _ => "Invalid month" // the default, catch-all
// used for a side effect:
i match
case 1 | 3 | 5 | 7 | 9 => println("odd")
case 2 | 4 | 6 | 8 | 10 => println("even")
// a function written with 'match':
def isTrueInPerl(a: Matchable): Boolean = a match
case false | 0 | "" => false
case _ => trueScala 3's optionally allows indentation based, brace-less syntax. Much closer to the ML family or Python, depending on how you look at it. It does indeed look better, but brings its share of issues.[1] Worse, a lot of people in the community, whether they like it or not, think this was an unnecessary distraction on top of the challenges for the entire ecosystem (libraries, tooling, ...) after Scala 3.0 was released.
Nothing to do with Haskell, even if it is also white space significant.
We had a similar experience moving Ruby 2->3, which has a ton of performance improvements. It was in fact faster in many ways but we had issues with RAM spiking in production where it didn't in the past. It turned out simply upgrading a couple old dependencies (gems) to latest versions fixed most of the issues as people spotted similar issues as OP.
It's never good enough just to get it running with old code/dependencies, always lots of small things that can turn into bigger issues. You'll always be upgrading the system, not just the language.
At the time Scala was on upswing because it had Spark as its killer app. It would have been a good time for the Scala maintainers to switch modes - from using Scala as a testbed for interesting programming-language theories and extensions to providing a usable platform as a general commercially usable programming language.
It missed the boat I feel. The window has passed (Spark moved to Python and Kotlin took over as the "modern" JVM language) and Scala is back to being an academic curiosity. But maybe the language curators never saw expanding mainstream usage as a goal.
For what it's worth, Spring has first tier Kotlin support, I haven't noticed this bias.
Put another way: Java only has access to a subset of the ecosystem
Almost all of the backend libraries I use are Java libs. Some of them have additional Kotlin extension libs that add syntax sugar for more idiomatic code.
Because in 5-10 years you'll have a Java project that people can still maintain as if it's any other Java project. If you pick Kotlin, that might at that point no longer be a popular language in whatever niche you are in. What used to be the cool Kotlin project is now seen as a burden. See: Groovy, Clojure, Scala. Of course, I recognize that not all projects work on these kinds of timelines, but many do, including most things that I work on.
Kotlin is Google's C#, with Android being Google's .NET, after Google being sued by coming up with Google's J++, Android Java dialect.
Since Google wasn't able to come up with a replacement themselves, Fuchsia/Dart lost the internal politics, they adopted the language of the JetBrains, thanks to internal JetBrains advocates.
Personally, I'm extremely glad to not have had to write .toStream().map(...).collect(Collectors.list()) or whatever in years for what could be a map. Similar with async code and exception handling.
For me one of the main advantages of Kotlin is that is decreases verbosity so much that the interesting business logic is actually much easier to follow. Even if you disregard all the things it has Java doesn't the syntax is just so much better.
This is true, but needs more context. Java 8 added Stream API, which (at this time) was a fantastic breath of fresh air. However, the whole thing felt overengineered at many points, aka - it made complex things possible (collector chaining is admittedly cool, parallel streams are useful for quick-and-dirty data processing), but simple everyday things cumbersome. I cannot emphasize how tiring it was to have to write this useless bolierplate
customers.stream().map(c -> c.getName()).collect(Collectors.joining(", "))
customers.map(c -> c.getName()).join(", ")
The choice was Kotlin. Scala is too "powerful" and can be written in a style that is difficult for others, and Java too verbose.
Kotlin is instantly familiar to modern TypeScript/Swift/Rust etc devs.
The only negative in my mind has been IntelliJ being the only decent IDE, but even this has changed recently with Jetbrains releasing `kotlin-lsp` for VS Code
Outside Android, I don't even care it exists.
If I remember correctly, latest InfoQ survey had it about 10% market share of JVM projects.
When inline is used on a parameter, it instructs the compiler to inline the expression at the call site. If the expression is substantial, this creates considerable work for the JIT compiler.
Requesting inlining at the compiler level (as opposed to letting the JIT handle it) is risky unless you can guarantee that a later compiler phase will simplify the inlined code.
There's an important behavioral difference between Scala 2 and 3: in 2, @inline was merely a suggestion to the compiler, whereas in 3, the compiler unconditionally applies the inline keyword. Consequently, directly replacing @inline with inline when migrating from 2 to 3 is a mistake.
In general it's a performance benefit and I never heard of performance problems like this. I wonder if combined with Scala's infamous macro system and libraries like quicklens it can generate huge expressions which create this problem.
And not all macros, but just the ones which expand to massive expressions
Think template expressions in C++ or proc macros in Rust
They should have made use of JVM bytecodes that allow to optimize lambdas away and make JIT aware of them, via invokedynamic and MethodHandle optimizations.
Naturally they cannot rely on them being there, because Kotlin also needs to target ART, JS runtimes, WebAssembly and its own native version.
Even then, they benchmarked it, and inlining was still faster* than invokedynamic and friends, so they aren't changing it now JVM 1.8+ is a requirement.
* at the expense of expanded bytecode size
Naturally it is a requirement, JetBrains and Google only care about the JVM as means to launch their Kotlin platform, pity that they aren't into making a KVM to show Kotlin greatness.
If it feels salty, I would have appreciated if Android team was honest about Java vs Kotlin, but they weren't and still aren't.
If they were, both languages would be supported and compete on merit, instead of sniffling one to push their own horse.
Even on their Podcast they reveal complete lack of knowledge where Java stands.
PS Yes, I know, there is some weird way to disable it. Somehow that way changes every version and is about as non-intuitive as possible. And trying to actually support the encapsulation is by a wide margin more work than it is worth.
Maybe Google could finally support latest Java versions on Android, instead of begrudgingly update when Kotlin lags behind Maven Central most used versions.
Which by the way is a Java 17 subset, not Java 8, when supporting Android versions below Android 12 isn't required.
Also not all Kotlin inlines are lambdas or even include method calls
The problem was overly-frequent inlining generating enormous expressions, causing a lot JIT phase and slow execution.
Look up the architecture of Catalyst + Tungsten
it's hard to buy it, considering that many of those "fatigued" moved on Kotlin, led by their managers' bs talking points.
Scala had/has a lot of promise. But how the language is marketed/managed/maintained really let a lot of people down and caused a lot of saltiness about it. And that is before we talk about the church of type-safety.
Scala is a more powerful language than Kotlin. But which do you want? A language with decent support that all your devs can use, or a language with more power but terrible support and only your very best devs can really take advantage of. And I say this as someone writing a compiler in Scala right now. Scala has its uses. But trying to get physicists used to Python to use it isn't one of them. Although that probably says more about the data science folks than Scala.
PS The GP is right, they should have focused on support and fixing the problems with the Scala compiler instead of changing the language. The original language spec is the best thing the Scala devs ever made.
The fundamental issue is that fixing Scala 2 warts warranted an entirely new compiler, TASTy, revamped macros... There was no way around most of the migration pains that we've witnessed. And at least the standard library got frozen for 6+ years.
However I agree that the syntax is a textbook case of trying to fix what ain't broke. Scala 3's syntax improvements should have stuck to the new given/using keywords, quiet if/then/else, and no more overloaded underscore abuse.
Checking the bug mentioned, it was fixed in 2022.
So, I’m wondering how one would upgrade to scala 3, while keeping old version of libraries?
Keeping updated libraries is a good practice (even mandatory if you get audits like PCI-DSS).
That part puzzled me more than the rest.
I was considerably less impressed by the reporting when I finally found out the culprit.
Sure it was “Scala 3” … but not really.
It was an interaction of factors and I don’t think it would take away from the story to acknowledge that up front.
> I did it as usual - updating dependencies
but later
> After upgrading the library, performance and CPU characteristics on Scala 3 became indistinguishable from Scala 2.13.
So... he didn't upgrade everything at first? Which IMO makes sense, generally you'd want to upgrade as little as possible with small steps. He just got unlucky.
Pinning specific versions of transitive deps is fairly common in large JVM projects due to either security reasons or ABI compatibility or bugs
(For scala-specific libs, there is a bit more nuance, because lib versions contain scala version + lib version, e.g. foolib:2.12_1.0.2 where 2.12 = scala version)
First, the "good practice" argument is just an attempt to shut down the discussion. God wanted it so.
Second, I rather keep my dependencies outdated. New features, new bugs. Why update, unless there's a specific reason to do so? By upgrading, you're opening yourself up to:
- Accidental new bugs that didn't have the time to be spotted yet.
- Subtly different runtime characteristics (see the original post).
- Maintainer going rogue or the dependency getting hijacked and introducing security issues, unless you audit the full code whenever upgrading (which you don't).
If performance is a feature it needs to be written in the code. Otherwise it implicitly regresses when you reorder a symbol and you have no recourse to fix it, other than fiddling to see if it likes another pattern.
The JVM is extremely mature and performant, and JVM-based languages often run 5x (or more) than non-JVM high-level languages like Python or Ruby.
> Turns out there was indeed a subtle bug making chained evaluations inefficient in Scala 3
I’m comparing with Haskell, Scheme, or even SQl which all promise to compile efficient code from high level descriptions.
I can thoroughly recommend it. Once of the best languages out there in terms of expressive power.
Scala is a great language and I really prefer its typesafe and easy way to write powerful programs: https://www.lihaoyi.com/post/comlihaoyiScalaExecutablePseudo... Its a great Python replacement, especially if your project is not tied to ML libraries where Python is defacto, like JS on web.
PS Perhaps they should make an actual unit test suite for their compiler. Instead they have a couple of dozen tests and have to guess if their compiler PR will break things.
It's really a shame because in many ways I do think it is a better language than anything else that is widely used in industry but it seems the world has moved on.
I used Scala for a bit around that period. My main recollection of it is getting Java compiler errors because Scala constructs were being implemented with deeply nested inner classes and the generated symbol names were too long.
spockz•7h ago
esafak•6h ago
noelwelsh•6h ago
cogman10•5h ago
For OOME problems I use a heap dump and eclipse memory analysis tool.
For microbenchmarks, I use JMH. But I tend to try and avoid doing those.
gavinray•3h ago
spockz•2h ago
Then we do benchmarking of the whole Java app in the container running async-profiler into pyroscope. We created a test harness for this that spins up and mocks any dependencies based on api subscription data and contracts and simulates performance.
This whole mechanism is generalised and only requires teams that create individual apps to work with contract driven testing for the test harness to function. During and after a benchmark we also verify whether other non functionals still work as required, i.e. whether tracing is still linked to the right requests etc. This works for almost any language that we use.
malkia•2h ago
We have continous benchmarking of one of our tools, it's written in C++, and to get "same" results everytime we launch it on the same machine. This is far from ideal, but otherwise there be either noisy neighbours, pesky host (if it's vm), etc. etc.
One idea that we thought was what if we can run the same test on the same machine several times, and check older/newer code (or ideally through switches), and this could work for some codepaths, but not for really continous checkins.
Just wondering what folks do. I can assume what, but there is always something hidden, not well known.
spockz•1h ago
In addition you can look at total cpu seconds used, memory allocation on kernel level, and specifically for the jvm at the GC metrics and allocation rate. If these numbers change significantly then you know you need to have a look.
We do run this benchmark comparison in most nightly builds and find regressions this way.