Even if it did work, the privacy value of hiding the SNI is pretty minimal for an IP address that hosts only a couple domains, as there are plenty of databases that let you look up an IP address to determine what domain names point there - e.g. https://bgp.tools/prefix/18.220.0.0/14#dns
> GitHub's migration guide tells developers to treat the new IDs as opaque strings and treat them as references. However it was clear that there was some underlying structure to these IDs as we just saw with the bitmasking
Great, so now GitHub can't change the structure of their IDs without breaking this person's code. The lesson is that if you're designing an API and want an ID to be opaque you have to literally encrypt it. I find it really demoralizing as an API designer that I have to treat my API's consumers as adversaries who will knowingly and intentionally ignore guidance in the documentation like this.
> Great, so now GitHub can't change the structure of their IDs without breaking this person's code.
And that is all the fault of the person who treated a documented opaque value as if it has some specific structure.
> The lesson is that if you're designing an API and want an ID to be opaque you have to literally encrypt it.
The lesson is that you should stop caring about breaking people’s code who go against the documentation this way. When it breaks you shrug. Their code was always buggy and it just happened to be working for them until then. You are not their dad. You are not responsible for their misfortune.
> I find it really demoralizing as an API designer that I have to treat my API's consumers as adversaries who will knowingly and intentionally ignore guidance in the documentation like this.
Sounds like you’ve maybe never actually run a service or API library at scale. There’s so many factors that go into a decision like that at a company that it’s never so simple. Is the person impacted influential? You’ve got a reputation hit if they negatively blog about how you screwed them after something was working for years. Is a customer who’s worth 10% of your annual revenue impacted? Bet your ass your management chain won’t let you do a breaking change / revert any you made by declaring an incident.
Even in OSS land, you risk alienating the community you’ve built if they’re meaningfully impact. You only do this if the impact is minimal or you don’t care about alienating anyone using your software.
If you think that just because VSCode is free there's no money on the line, you're not thinking about things the way others do. As I said, reputation alone definitely has a cost and Microsoft has partnerships where VSCode is strategic. They probably just made a calculation that there's not enough users and/or the users using that private API were strategically misaligned with their direction.
> The lesson is that you should stop caring about breaking people’s code who go against the documentation this way. When it breaks you shrug. Their code was always buggy and it just happened to be working for them until then. You are not their dad. You are not responsible for their misfortune.
Sure, but good luck running a business with that mindset.
You could also say, if I tell you something is an opaque identifier, and you introspect it, it's your problem if your code breaks. I told you not to do that.
I think more important than worrying about people treating an opaque value as structured data, is wondering _why_ they're doing so. In the case of this blog post, all they wanted to do was construct a URL, which required the integer database ID. Just make sure you expose what people need, so they don't need to go digging.
Other than that, I agree with what others are saying. If people rely on some undocumented aspect of your IDs, it's on them if that breaks.
Exposing what people need doesn’t guarantee that they won’t go digging. It is surprisingly common to discover that someone has come up with a hack that depends on implementation details to do something which you exposed directly and they just didn’t know about it.
> Great, so now GitHub can't change the structure of their IDs without breaking this person's code
OP can put the decoded IDs into a new column and ignore the structure in the future. The problem was presumably mass querying the Github API to get those numbers needed for functional URLs.
Literally how I designed all the public facing R2 tokens like multipart uploads. It’s also a security barrier because forging and stealing of said tokens is harder and any vulnerability has to be done with cooperation of your servers and can be quickly shut down if needed.
You could but you would lose the performance benefits you were seeking by encoding information into the ID. But you could also use a randomized, proprietary base64 alphabet rather than properly encrypting the ID.
XOR encryption is cheap and effective. Make the key the static string "IfYouCanReadThisYourCodeWillBreak" or something akin to that. That way, the key itself will serve as a final warning when (not if) the key gets cracked.
Symmetric encryption is computationally ~free, but most of them are conceptually complex. The purpose of encryption here isn't security, it's obfuscation in the service of dissuading people from depending on something they shouldn't, so using the absolutely simplest thing that could possibly work is a positive.
XOR with fixed key is trivially figure-out-able, defeating the purpose. Speck is simple enough that a working implementation is included within the wikipedia article, and most LLMs can oneshot it.
Yes, XOR is a real and fundamental primitive in cryptography, but a cryptographer may view the scheme you described as violating Kerckhoffs's second principle of "secrecy in key only" (sometimes phrased, "if you don't pass in a key, it is encoding and not encryption"). You could view your obscure phrase as a key, or you could view it as a constant in a proprietary, obscure algorithm (which would make it an encoding). There's room for interpretation there.
Note that this is not a one-time pad because we are using the same key material many times.
But this is somewhat pedantic on my part, it's a distinction without a difference in this specific case where we don't actually need secrecy. (In most other cases there would be an important difference.)
Encoding a type name into an ID is never really something I've viewed as being about performance. Think of it more like an area code, it's an essential part of the identifier that tells you how to interpret the rest of it.
Can GitHub change their API response rate? Can they increase it? If they do, they’ll break my code ‘cause it expects to receive responses at least after 1200ms. Any faster than that and I get race conditions. I selected the 1200ms number by measuring response rates.
No, you would call me a moron and tell me to go pound sand.
It seems like Twilio has a conflict of interest that prevents them from offering WebAuthn, as that would be a tacit admission that their SMS and Authy products are not actually that secure.
rich irony that twilio numbers don't qualify to receive SMS codes when senders check if it's a virtual number (the regulated aka important ones do check)
It gets its data from Open Street Map, so it's only up-to-date if volunteers are keeping it up-to-date.
That said, https://nl.wikipedia.org/wiki/Centrale_Hemweg says the plant was converted to natural gas, not decommissioned, in 2019, and this is correctly reflected in Open Street Map / Open Infra Map. Do you have a citation that contradicts this?
> The Go module proxy doesn't make any guarantee that it will permanently store the checksum for any given module
Incorrect. Checksums are stored forever, in a Merkle Tree, meaning if the proxy were to ever delete a checksum, it would be detected (and yes, people like me are checking - https://sourcespotter.com/sumdb).
Like any code host, the proxy does not guarantee that the code for a module will be available forever, since code may have to be removed for legal reasons.
But you absolutely can rely on the checksum being preserved and thus you can be sure you'll never be given different code for a particular version.
Ah, my mistake. I had read in the FAQ that it does not guarantee that data is stored forever, but overlooked the part about preserving checksums specifically.
To be very pedantic, there are two separate services: The module proxy (proxy.golang.org) serves cached modules and makes no guarantees about how long cache entries are kept. The sum database (sum.golang.org) serves module checksums, which are kept forever in a Merkle tree/transparency log.
Ok. So to answer the question whether the code for v1.0.0 that I downloaded today is the same as I downloaded yesterday (or whether the code that I get is the same as the one my coworker is getting) you basically have to trust Google.
The checksums are published in a transparency log, which uses a Merkle Tree[1] to make the attack you describe detectable. Source Spotter, which is unaffiliated with Google, continuously verifies that the log contains only one checksum per module version.
If Google were to present you with a different view of the Merkle Tree with different checksums in it, they'd have to forever show you, and only you, that view. If they accidentally show someone else that view, or show you the real view, the go command would detect it. This will eventually be strengthened further with witnessing[2], which will ensure that everyone's view of the log is the same. In the meantime, you / your coworker can upload your view of the log (in $GOPATH/pkg/sumdb/sum.golang.org/latest) to Source Spotter and it will tell you if it's consistent with its view:
$ curl --data-binary "@$(go env GOPATH)/pkg/sumdb/sum.golang.org/latest" https://gossip.api.sourcespotter.com/sum.golang.org
consistent: this STH is consistent with other STHs that we've seen from sum.golang.org
For the question “is the data in the checksum database immutable” you can trust people like the parent, who double checks what Google is doing.
For the question “is it the same data that can be downloaded directly from the repos” you can skip the proxy to download dependencies, then do it again with the proxy, and compare.
So I'd say you don't need to trust Google at all in this case.
Any non-profit, or just charitable non-profits (aka 501(c)(3))? Unfortunately, the US does not consider producing open source software to be charitable activity.
If you add .git to the end of your module path and set $GOPRIVATE to the hostname of your Forgejo instance, then Go will not make any HTTPS requests itself and instead delegate to the git command, which can be configured to authenticate with client certificates. See https://go.dev/ref/mod#vcs-find
The simple task of following a bug requires you to:
1. Send an empty email to a special address for the bug.
2. Wait 15-30 minutes for Debian's graylisting mail server to accept your email and reply with a confirmation email.
3. Reply to the confirmation email.
The last time I tried to follow a bug, I never got the confirmation email.
In practically every other bug tracker, following a bug is just pressing a button.
Like most of Debian's developer tooling, the bug tracker gets the job done (most of the time) but it's many times more inconvenient than it needs to be.
Fair points. But without looking at it myself, and for the benefit of people reading along, do you have to do that if you already have an account on the tracker? For instance, it's easy to follow issues on GitHub, but that's after you've jumped through some similar hoops to create an account.
I really wish we could have both. An interactive web frontend, and the classic email-centric bug tracker, both serving the same data. I think both have its strengths. I suppose that the job is massive given how enormous and fast-moving the first have become.
I can't find it now but I recently saw a graph of new Debian Developers joining the project over time and it has sharply declined in recent years. I was on track to becoming a Debian Developer (attended a couple DebConfs, got some packages into the archive, became a Debian Maintainer) but I ultimately burned out in large part because of how painful Debian's tooling makes everything. Michael Stapelberg's post about leaving Debian really rings true: https://michael.stapelberg.ch/posts/2019-03-10-debian-windin...
Debian may still be "getting by" but if they don't make changes like this Git transition they will eventually stop getting by.
1. The monitoring client does not ensure that the checkpoint was created recently, so a malicious log can conceal malicious entries from monitors by serving an old checkpoint.
2. Though the age keyserver policy is not configured this way, the post suggests you could create a policy that requires only a minority of witnesses (e.g. 3 of 10) to cosign a checkpoint. If you do this, then monitors have to get checkpoints that are cosigned by at least 8 of the 10 witnesses. Otherwise, a malicious log could present one view to relying parties that is cosigned by one set of witnesses, and a different view to monitors that is cosigned by a different set of witnesses. There is currently no mechanism specified for monitors to get these extra cosignatures, so if you go with a minority policy you'll need to invent your own stuff in order for witnessing to actually accomplish anything.
Even if it did work, the privacy value of hiding the SNI is pretty minimal for an IP address that hosts only a couple domains, as there are plenty of databases that let you look up an IP address to determine what domain names point there - e.g. https://bgp.tools/prefix/18.220.0.0/14#dns
reply