By your argument, it would not be a problem if the RNG never generated 0. So, it must follow that it would also not be a problem if it never generated {1, 2, 3, ..., 253}.
That means that our RNG now only generates the values 254 and 255. Which of the values is generated is unpredictable on any given call. However, 7 of the 8 output bits are now always fixed and so completely predictable. Can you imagine how an attacker could exploit that?
Failing to generate only the number 0 is a weaker version of the same class of flaw.
We're talking about whether a modification of the expected probabilities changes the dynamics of the game. The example I gave was deliberately extreme, because that makes it easier to reason about.
If you want a casino example, then consider a roulette wheel that always lands on 36 but still pays out as usual. I think you'd want to play on it. Now consider one that always lands somewhere between 30 and 36. Still worth it, right? With careful bets and a good starting float you're still coming away from the table up (with a very high probability).
In fact for a roulette wheel you only need two dead pockets for the player to get an edge. Bias is exploitable.
Sure, but you’re pressing on the truth that a small modification taken to an extreme is a large modification.
When the original point was that a tiny fractional loss in an RNG is not going to make a practical difference. Which I believe is also true. And it is also true that a large loss in an RNG is catastrophic.
They can both be true.
And roulette is 2 out of 38, 5.2%. That’s 17 times more than the 1/256 here, which was already a simplification of the (I think) 1/65536 in question.
With respect, I think you're still missing my point, which is simply that bias introduces vulnerability. Without a real scenario to analyze, the scale doesn't really factor into the argument. A vulnerability is a vulnerability until proven mitigated.
> a tiny fractional loss in an RNG is not going to make a practical difference
I'm not so sure this is true. I don't think either of us is in a place to say whether this vulnerability has practical applications or not. A 1/65536 bias might seem like nothing important to you. It seems like potentially something to me, in a world where the attacker might control the volume of data generated.
It is absolutely untrue that a biased RNG has "zero practical impact." Modern cryptography has plenty of examples of relatively small biases leading to breaks. Check out Bleichenbacher's attack, for instance.
You could be correct that the very small bias here is not enough to be exploitable. But, given the history around this, it would be wrong to handwave it away as trivial.
Some cryptographic algorithms can be broken by this sort of bias (after 2^20, 2^35, or 2^70 operations), others are completely unaffected. There's certainly no excuse for not knowing, at least.
Most contemporary stories around AI include the implication that AI did something humans couldn't. This is because the big players have been shilling AGI hard for a while, and their valuations depend on maintaining the sentiment that serious progress in that direction is being made.
Washing machines can’t do anything humans do, they just remove labour. Trucks don’t do anything longboats can’t, it just need less labour and time/effort to build roads rather than canals. Computers can’t calculate anything humans can’t dry run by hand etc.
Everything in reality is about reducing time/effort/material/cost or achieving more with less resource.
Yes. I'd also argue there isn't anything humans couldn't do in theory, other than things strictly prohibited by known laws of physics. It's been pretty conclusively shown in the last 100 years, we're past tipping point of civilizational knowledge and scientific infrastructure.
What stops us from doing any specific thing is always allocation of resources - there's finite amount of time/effort/material/labor available, and past trivial amounts we need more and more people to agree on some allocation. Reduction of time/effort/material/labor costs of any thing is what moves it closer from "infeasible" to "feasible" for us to do. But again, short of violating laws of physics, it was never "impossible".
What I mean is that "can be done by humans" is ill defined. Are you allowed to use pencil and paper? That's not part of your body? Do you have to be strapped into a coffin for it to count as human-done? How about wheels, sticks and stones? Allowed to use those tools? How about an abacus? Or is electricity the line to draw?
No, they don't - that implication is in the heads of people who believe AI is magic, or believe AI is advertised as magic, or believe that humans are dumb.
The actual implication is, and always been, different: AI did something humans theoretically could, given enough time, motivation and budget, but they didn't, because it wasn't the best use of time or money. AI therefore demonstrates its value, by opening up problems that were previously uneconomical to solve.
> that implication is in the heads of people who believe AI is magic, or believe AI is advertised as magic, or believe that humans are dumb.
Yes! I believe that class of people are called "investors."
Your point is basically correct, I think, but the illusion of value that a corporate entity offers, and the reality of the value that the corporate entity is creating, are distinct things.
I'm the creator, and it is the former. I will consider spending some money on one of the available "AI detector" APIs and making a similar site for the latter. My main problem with them, though, is that I don't think they work very well and even if they work now, I imagine they'd be perpetually behind the newest models
In this domain, an apparent single unique piece of work is often composed of several breakthroughs. For example, when Andrew Wiles proved Fermat's Last Theorem, he had to develop multiple new pieces of mathematical technology to get there.
The claim here seems to be that the human mathematicians, working with AI, developed technology to go A->B->C. By training on those conversations, OpenAI was then able to encourage the model to go A->B->C->D.
In my opinion that situation should be acceptable, if openly disclosed, because it is in the public interest to make progress on these problems and because AI is clearly an amazing tool for making progress. But the human mathematicians are saying that OpenAI is presenting as if the model got from A->D entirely independently, without acknowledging their background contributions.
Also, wasn't their B+C research private at the time, with them only releasing those details publicly after this blew up?
If they had published B+C, I think that would lean more towards fair game, as that is how research works and is improved on over time. But it seems like unpublished/private B + C may have been used by the model to hint it into working out how to get from A->D.
That link is a helpful contribution to this discussion.
I'm not at all familiar with this area, but my reading is that he appears to call it out as a relatively obvious extension of his own work:
> It is a creative and at the same time elementary construction that uses not just property (T) for an application of my result with Kun, but also for the ambient group
G in order to overcome the problem, that the Γ-components might be of different size. Once this is achieved, the rest of the argument is straightforward.
Creative and at the same time elementary is where LLMs excel, generally speaking. It's why they are so good at writing code.
> On the other side, I was looking myself for such a mechanism ever since we wrote the paper in 2019 and admire the efficiency of this construction.
He seems to admit very clearly he does not see this as his own work. 'I was looking...' well why did he stop? Because the AI figured it out first.
It seems quite odd to me to 'admire the construction' of something, only for your opinion to sour once that something figures it out first.
I think a lot of the emotional reaction here is familiar to us non mathematicians: you spent years developing expertise, and then LLMs began producing competent work in areas that had previously required that expertise. That's understandably uncomfortable, but discomfort by itself isn't evidence of misappropriation.
Grossmann collaborated with Einstein on GR, supplying quite a bit of the mathematical capacity required (which initially didn't come easily to Einstein). They published jointly, until Einstein was competent enough to work independently [1]. That's not equivalent to the situation being claimed here.
In this particular case (i believe) the user actively turned on the feature which saves your entire location history and journeys on Google's servers so that you can go back and review later. It was opt-in.
Google has since redesigned this feature so that the history lives on the phone itself, not on their servers (removing the possibility for police officers to request this information from google - search warrant or not).
While I'd generally agree that Apple is the better of the two on the privacy side it's still a false choice because you pretty much have to pick one as a regular person as things currently stand.
It becomes harder over time to function in society without a smart phone (I know this because I barely use mine and it usually lives at home but my partner runs her life off hers).
The fact that most people in the western world have two terrible companies to choose between in order to have normal life conveniences is outrageous. (Here in Europe we actually have no choices that operate under our laws!)
We need laws mandating open and interoperable interfaces and standards for all the things a modern life takes for granted: public transit, banking, digital ID, etc.
Once that's in place, we can start talking about what people are "choosing". Because currently our "choice" is at best between a rock and a hard place (or as we say in my native tongue: between plague and cholera).
This style of argument is nonsense because deciding between Android and Apple has about a hundred unrelated but equally important consequences. There is no choice that aligns with all of your wishes (and rights).
Privacy? Just buy Apple (largely an urban legend at this point, but regardless).
Actual ownership? Just buy Android.
Security? Just buy Apple (or Pixel with GrapheneOS).
Repairability? Just buy one of these 4 Android models.
Well, nowadays, Google location history is stored on-device. Google Maps sends your position to their servers, both for geocoding and to power their traffic estimation. Even when location history was in the cloud, you could turn it off. It even asked when you opened Maps for the first time. Most people chose the default blue button to leave it on. You could also just turn off location on your phone entirely when you didn't want Google to know where you were.
The tricky bit is the cell tower location. That cannot be disabled unless you go into Airplane mode or turn your phone off. Broad location (which tower are you associated with) is needed for the network to route your calls, while precise location is used for E911.
One argument for LLMs is that although all information on topics X, Y and Z was already available somewhere, LLMs make that information more exploitable through collation, filtering and dynamic tailoring.
For a relatively narrow subject area (e.g. construction of pipe bombs) the collation is minimal, and so the filtering and tailoring probably isn't that important; a novice doesn't learn a lot more from the LLM than they would have done from a few Google searches.
For a broad subject (practical creation and exploitation of software vulnerabilities), the collation is very significant and the filtering means that LLMs can empower a novice to act at a similar level as an expert.
By your argument, it would not be a problem if the RNG never generated 0. So, it must follow that it would also not be a problem if it never generated {1, 2, 3, ..., 253}.
That means that our RNG now only generates the values 254 and 255. Which of the values is generated is unpredictable on any given call. However, 7 of the 8 output bits are now always fixed and so completely predictable. Can you imagine how an attacker could exploit that?
Failing to generate only the number 0 is a weaker version of the same class of flaw.
reply