Is there "benchmark data" on different memory techniques? I also wondered about this because I also memorized 50 digits on a lark back in high school, without the post's encoding technique and memory palace.
I added one block of 10 digits per night, spending maybe 10 minutes per night? Very roughly an hour total, ~same as the poster spent to get 100 digits, but without the prior investment they spent on the PAO (memorizing 300 key-value pairs, apparently).
Fair enough, but the abstract really is too opaque imo.
> The k-server conjecture states that a deterministic online algorithm can achieve competitive ratio k on every metric space.
I was like "algorithm for what?" when I knew what every one of those terms meant. Lots of things use k servers. Lots of problems involve metric spaces.
I agree with most of this, but you're misunderstanding "alignment" as coined. Yes, training powerful enough AI, any simple optimization target gets you malign behavior, because human values are not simple. If you insist on making powerful AI, you'd better instill respect for human values! That's "alignment".
How do you do that in the current paradigm other than creating yet another gameable metric? And something I didn't mention above is that there is no difference between "solving the task" and "optimizing the metric" for an ML model, even though there clearly is for us. So it's not clear to me how you "fix" something that is baked into the architecture. All I'm saying is "instilling respect for human values" is not something that can actually be done via a cost function. In no small part because we humans probably don't even agree on those values, let alone on a single metric with which to quantify and "optimize" them.
For example, we agree that "merit" is valuable and that we should reward "merit." But to reward it we have to quantify it, and what metric should we use? Raw SAT score to get into college? But that also captures socioeconomic factors that unfairly penalize some and reward others. We generally agree that those who provide more value should earn more money, but what does that look like? Do we all agree on what activities are or should be valuable, or on how they should be rewarded? Until recently, I thought we all agreed that "empathy" was a human value, but a lot of people in this space, who are making these decisions unilaterally for all of us, don't apparently share that belief.
Yes! There's both the daunting problem of technically how can we even do this, and the broader problems of what's good/acceptable and how do we resolve that among each other.
I believe this mismatch of rates of progress means we need to stop slamming the accelerator on capabilities for now even though as a libertarian I'm sure whatever governance process we manage to get to will be, uh... suboptimal.
> processes better regarded as generalization and prediction, which seem not only seem to have direct and obvious neural realizations
If the workings of those circuits are obvious to you, I'd really like to learn. Do you mean the level of analysis at https://transformer-circuits.pub/ ? (That looks like good work but not a deep understanding.)
The predictive nature of our cortex seems even more direct that that, although perhaps obvious is overstating it. To read about the 6-layer circuit of our cortex, google for "cortical microcircuit". Basically the repeating structure of our cortex (across all regions) is this vertical connectivity pattern ("circuit") between the 6 layers, and also the connections coming into and out of the cortex at the top and bottom of those layers. There are also horizontal connections within layers of the cortex, but these are short-distance/localized likely for coordinating activity with a local region (cortical column).
Our cortex is connected in a loop with our thalamus (cf recent talk of looped transformers!), which can be separated into thalamic core and matrix. Basically our senses feed into the thalamic core, and cortical outputs (predictions) feed into the thalamic matrix. The thalamic connections/loop seem to act a bit like a patch panel connecting different regions of the cortex.
The cortical microcircuit is a real physical circuit - a pattern of synaptic connections between layers and inputs/outputs. Those Transformer circuits are more functional than physical. We know how a Transformer is physically connected, with attention heads etc, and this "circuit" analysis then describes how it functions, e.g. with attention heads in adjacent transformer layers effectively combining to form "induction heads" that copy data across embeddings.
This reminded me of a story/vignette which my fallible memory tags as by someone like Twain or Poe, about an automatic rewriting machine. It was put to work on the same passage with an ever higher intensity setting, becoming more and more gothic to the point of parody -- iirc the final output was verse including the line "to atoms dash the doubly damned".
No memory, but searching says it was Lewis Carroll in Photography Extraordinary, published in 1885:
The Milk-and-Water School
Alas! she would not hear my prayer!
Yet it were rash to tear my hair;
Disfigured, I should be less fair.
She was unwise, I may say blind;
Once she was lovingly inclined;
Some circumstance has changed her mind.
The Strong-Minded or Matter-of-Fact School
Well! so my offer was no go!
She might do worse, I told her so;
She was a fool to answer "No".
However, things are as they stood;
Nor would I have her if I could,
For there are plenty more as good.
The Spasmodic or German School
Firebrands and Daggers! hope hath fled!
To atoms dash the doubly dead!
My brain is fire--my heart is lead!
Her soul is flint, and what am I?
Scorch'd by her fierce, relentless eye,
Nothingness is my destiny!
Sounds like a problem that'd hit anyone running LLMs. I haven't tried on mine so far, but people do talk about ordering a $10k Mac Studio just for that. Anyone else see this? Does the OS version matter?
I don't think I had the issue before macOS 26, but honestly I have no way to tell. I did run into one other HN commenter that suffers from the same slowness problem, but they didn't connect it to workloads, only uptime. I think it was the same issue though.
fwiw I also observed a persisting slowdown soon after upgrading my Studio to OSX 26. Since I haven't been using it very much I'm short on data bearing on the cause -- maybe it was GPU use though I don't recall doing anything like that.
I know that if I don't use LLMs, the machine lasts much longer without getting slow. If I do however, it gets slow very quickly and does not recover until a reboot. Super annoying.
My Intel Mac from 2015 could be up for months without interruption or slowdown. (mostly because I stopped updating after Mojave, but my point is it never needed a reboot)
> Gates said: "I'd skip out on athletics and go down to this computer center. We were moving ahead very rapidly: Basic, FORTRAN, LISP, PDP-10 machine language, digging out the operating system listings from the trash and studying those."
Is there more context clarifying this means they studied the source code of DEC Basic?
"The BASIC they did which we got ahold of the BASIC source code and enhanced t quite a bit was great."
Thanks! I think this is still talking about his high school volunteer hacking rather than the MS Basic era. (i.e. doesn't obviously support your "ported it".)
They got the source code of DEC Basic, studied it, "enhanced quite a bit" then wrote their own Basic which is similar to DEC Basic, and you don't think it is a derivative? Ok. It is a derivative in my book.
It is derivative as in it uses the same ideas not as in it uses the same code. The latter would run afoul of copyright law but the former is perfectly fine.
You can make your own reincarnation anime even if you formerly worked making cels for TenSura.
I added one block of 10 digits per night, spending maybe 10 minutes per night? Very roughly an hour total, ~same as the poster spent to get 100 digits, but without the prior investment they spent on the PAO (memorizing 300 key-value pairs, apparently).
You'd want to compare to Anki, too.
reply