> The talk you're so excited about actually shows this asymptote.
Oh, I see the confusion. That asymptote is for the hypothetical case where the allocation rate grows to infinity (i.e. remains constant per core and we add more cores) while the heap remains constant. Yes, with an allocation rate growing to infinity, the cost of memory management (using any algorithm) also grows to infinity. That it's so obvious was his point showing why certain benchmarks don't make sense as they increase the allocation rate but keep the heap constant.
> So they don't go there, but it ought to have pulled you up short when you thought you could apply this understanding to an entirely unrelated paradigm.
I'm sorry, but I don't think you understand the theory of memory management. You obviously run into these problems even in C. If you haven't then you haven't been doing that kind of programming long enough. Some results are just true for any kind of memory management, and have nothing to do with a particular algorithm. It's like how in computational complexity theory, certain problems have a minimal cost regardless of the algorithm chosen.
> But at scale the other side matters - not the minimum but the maximum, and so whereas doubling from 16GB RAM to 32GB RAM was very cheap, doubling from 16 servers to 32 servers because they're full means paying twice as much, both as capital expenditure and in many cases operationally.
I've been working on servers in C++ for over 20 years, and I know the tradeoffs, and when doing this kind of programming seeing CPU exhausted before RAM is very common. I'm not saying there are never any other situations, but if you don't know how common this is, then it seems like you don't have much experience with low-level programming. Implying that the more common reason to need more servers is because what's exhausted first is the RAM is just not something you hear from people with experience in this industry. You think that the primary reason for horizontal scaling is dearth of RAM?? Seriously?! I remember that in the '90s or even early '00s we had some problems of not enough RAM on client workstations, but it's been a while.
In the talk he tells memory-management researchers about the economics in industry, as they reasonably might not be familiar with them, but decision-makers in industry - as a whole - are.
Now, don't get me wrong, low level languages do give you more control over resource tradeoffs. When we use those languages, sometimes we choose to sacrifice CPU for footprint and use a refcounting GC or a pool when it's appropriate, and sometimes we sacrifice footprint for less CPU usage and use an arena when it's appropriate. This control is the benefit of low level languages, but it also comes at a significant cost, which is why we use such languages primarily for software that doesn't deliver direct business value but is "pure overhead", like kernels, drivers, VMs, and browsers, or for software running on very constrained hardware.
> I didn't see any evidence.
The evidence is that in a highly competitive environment of great economic significance, where getting big payoffs is a way to get an edge over the competition, technologies that deliver high economic payoffs spread quickly. If they don't, it could be a case of some market failure, but then you'd have to explain why companies that can increase their profits significantly and/or lower their prices choose not to do so.
When you claim some technique would give a corporation a significant competitive edge, and yet most corporations don't take it (at least not for most projects), then that is evidence against that claim because usually companies are highly motivated to gain an advantage. I'm not saying it's a closed case, but it is evidence.
> When you claim some technique would give a corporation a significant competitive edge, and yet most corporations don't take it (at least not for most projects), then that is evidence against that claim because usually companies are highly motivated to gain an advantage.'
Corporations will generally want to optimize for ease of development and general ecosystem maturity. Rust is at a clear disadvantage at least wrt. the latter - the first generally usable version of the language was only released in late 2018 - and other safe languages are generally GC-based (compare Java/C# with C++). It's quite normal that a lot of performance would be left on the table wrt. both CPU and memory footprint.
But C++ is over 40 years old, and Java et al. displaced it in like five minutes. And that the footprint savings aren't worth much is pretty much the point. If you get 1GB/core and you use less, then you can't run more programs on the machine. The machine is exhausted when the first of RAM/CPU is, not when both are.
Nah. I'm an old man. I remember when Java 1.0 shipped. It got relatively little initial enterprise adoption considering it was from Sun who had a lot of enterprise contracts. Traction took years and was often aligned with adoption of Tim's crap hypermedia system, the "World Wide Web" which he'd created years prior but was just beginning to intrude into normal people's lives by the end of the 1990s.
A big factor is that Java was the entire ecosystem, you're getting a programming language (which is pretty good), a novel virtual machine (most of their ideas fell through but the basic thing is fine), dedicated hardware (mostly now forgotten), a component architecture, and a set of tools for them.
You're still on this 1GB/core thing which is the wrong end of the scale in two senses. Firstly, I've worked on systems where we'd want 1TB/core and so today that means you're buying a lot of CPU performance you don't need to get enough RAM because as you say, that machine is "exhausted" anyway.
But more importantly the scale for big products isn't dictated by RAM or CPU it's dictated by service provision, and at large scale that's just linear. Twice as much provision, twice the cost. Avoiding a GC can let you slash that cost. Cutting a $1M annual bill in half would justify hiring a Rust programmer though likely not a whole team. Cutting a $1Bn annual bill in half - which is much more like what Microsoft are spending on O365 - is obviously worth hiring a team.
It's not instant. GC tuning is basically instant. RIIR might take three, five, even ten years. So don't expect results tomorrow afternoon.
Rust is as old now as Java was when JDK 6 came out. But it's not just Java. Look at Fortran, C, C++, JS, C#, PHP, Ruby, Go, and even the late-bloomer Python - no popular language had an adoption rate as low as Rust at its rather advanced age. The trend just isn't there. It may well be that low-level programming will slowly shift to Rust an Zig, but there is no indication that low level programming as a whole isn't continuing its decline (in use, not importance).
> Avoiding a GC can let you slash that cost
But it doesn't, because RAM isn't the bottleneck in the vast majority of cases. It doesn't matter how linear costs are if RAM isn't the thing that's exhausted. That's why the use of manual memory management had been declining for decades.
At 1TB per core things still don't change because GC no longer has a high footprint cost in the old gen. You may use 15x RAM for the first 50 MB, but the overhead for the second GB is very small, thanks to the generational hypothesis: the older an object is, the less frequently it is allocated. The cost of a moving-tracing GC is propprtional to allocation-rate * live-set / heap-size per generation. When the live set is large, the allocation rate in that generation is low, which means that the heap size premium is also low.
> So don't expect results tomorrow afternoon.
Manual memory management is the older option, and it's been in decline for decades precisely because the savings in costs go in the other direction (not in all situations, but in most) due to the economics of RAM costs vs CPU costs. Without a marked change in the economics, the trend won't reverse even in another 50 years.
Oh, I see the confusion. That asymptote is for the hypothetical case where the allocation rate grows to infinity (i.e. remains constant per core and we add more cores) while the heap remains constant. Yes, with an allocation rate growing to infinity, the cost of memory management (using any algorithm) also grows to infinity. That it's so obvious was his point showing why certain benchmarks don't make sense as they increase the allocation rate but keep the heap constant.
> So they don't go there, but it ought to have pulled you up short when you thought you could apply this understanding to an entirely unrelated paradigm.
I'm sorry, but I don't think you understand the theory of memory management. You obviously run into these problems even in C. If you haven't then you haven't been doing that kind of programming long enough. Some results are just true for any kind of memory management, and have nothing to do with a particular algorithm. It's like how in computational complexity theory, certain problems have a minimal cost regardless of the algorithm chosen.
> But at scale the other side matters - not the minimum but the maximum, and so whereas doubling from 16GB RAM to 32GB RAM was very cheap, doubling from 16 servers to 32 servers because they're full means paying twice as much, both as capital expenditure and in many cases operationally.
I've been working on servers in C++ for over 20 years, and I know the tradeoffs, and when doing this kind of programming seeing CPU exhausted before RAM is very common. I'm not saying there are never any other situations, but if you don't know how common this is, then it seems like you don't have much experience with low-level programming. Implying that the more common reason to need more servers is because what's exhausted first is the RAM is just not something you hear from people with experience in this industry. You think that the primary reason for horizontal scaling is dearth of RAM?? Seriously?! I remember that in the '90s or even early '00s we had some problems of not enough RAM on client workstations, but it's been a while.
In the talk he tells memory-management researchers about the economics in industry, as they reasonably might not be familiar with them, but decision-makers in industry - as a whole - are.
Now, don't get me wrong, low level languages do give you more control over resource tradeoffs. When we use those languages, sometimes we choose to sacrifice CPU for footprint and use a refcounting GC or a pool when it's appropriate, and sometimes we sacrifice footprint for less CPU usage and use an arena when it's appropriate. This control is the benefit of low level languages, but it also comes at a significant cost, which is why we use such languages primarily for software that doesn't deliver direct business value but is "pure overhead", like kernels, drivers, VMs, and browsers, or for software running on very constrained hardware.
> I didn't see any evidence.
The evidence is that in a highly competitive environment of great economic significance, where getting big payoffs is a way to get an edge over the competition, technologies that deliver high economic payoffs spread quickly. If they don't, it could be a case of some market failure, but then you'd have to explain why companies that can increase their profits significantly and/or lower their prices choose not to do so.
When you claim some technique would give a corporation a significant competitive edge, and yet most corporations don't take it (at least not for most projects), then that is evidence against that claim because usually companies are highly motivated to gain an advantage. I'm not saying it's a closed case, but it is evidence.