“Cultural” technical issues are so frustrating to me. A certificate is fundamentally just a type of credential, like a password, but for historical reasons they’re treated like getting citizenship papers. There’s always this ceremony even in scenarios where it makes zero sense — such as internal-use certificates used for a gRPC API server behind a load balancer.
Why - for the love of God why - can’t I just obtain a cert like this directly out of a secret store such as an Azure Key Vault!?
These things are already full hardware security modules (HSMs) with all of the capabilities required to run just about anything short of a public Root CA and maybe even that too.
But no.
NO!
Script it yourself. Make a root cert, “upload” it, make a cert, sign it, upload it, link it, renew it, re-configure it, and on and on. Oh… you wanted a CRL too? A 1kb file? Ha-ha! No. Make it and host it yourself!!
It’s absurd.
So many services depend on a simple CA->cert setup: VPNs, API auth, load balancer back-ends, clusters, etc…
But my mark my words: no cloud will have a trivial turnkey solution this decade.
This is because running a CA is culturally accepted to be a hard problem. It is! If you’re DigiCert. It isn’t if you’re building a three-server internal use cluster. But the problem is hard, you see? That accepted fact! Everyone knows it! Ceremony is required. We can’t just hand your sever a 1kb credential file! That would be… unconventional!
It’s just not the way things are done, so stop asking.
It’s about damn time that we should be able to get intermediate signing certs that are domain limited from a public ca and issue out own “real” certs. This is fully supported in the standards. Anyone offering this product affordably yet?
Name constraints are an optional feature in the standards. A client can ignore the constraints and be completely standards compliant.
Should the CAs issue intermediate certs that are only secure if a client implements an optional feature?
And even if most web browsers support name constraints properly - who knows if that cheap network webcam does, or that old mail client, or that 20 year old retro PC game?
If you want to uphold the name constraints in your CA cert, mark the field as critical. At that point clients that don’t understand them should fail validation of the CA cert.
So it may have limited use-cases today if you require full compat for all clients. For example internal controlled networks like discussed in the article.
Just like you presumably already wouldn't issue LE certs when you need to support clients with ancient CA bundles.
Interesting idea. What use cases would this make easier or even possible? At first, it sounds like more work (you’re now your own CA but without full freedom of a truly self hosted one).
The main advantage is that this CA and all downstream certificates would be globally-trusted (limited to the domain), which is not the case for a custom CA.
Security-wise it shouldn't be any worse than wildcard certificates which are already a thing. It would actually improve things, because the user can now issue downstream certificates much more granularly without having to interact with the root CA (so you can issue fully offline for an intranet, or issue extremely-short-lived certificates that would otherwise run afoul of root CA's rate-limits).
It would improve things for everybody but the certificate authorities.
They're selling something with a marginal cost of zero for $50 each. A wildcard certificate costs more not because it is materially different or "harder" to issue, but because it replaces many $50 certificates. Thus, it "must" cost more, or everybody would just use wildcard certificates everywhere and reduce profits at the large public CAs.
It is actually possible to get a domain-specific CA like the one you're thinking of. I saw one at a large department of education. It allowed unlimited issuance of certificates such as HTTPS, mail-signing, document-signing, and some other types that could be locked to a DNS domain. However, there was still a cost per certificate and the up-front cost was huge, something like $100K.
I made a comment up-thread, but name-constrained CAs can issue for anything. It's enforced client-side, and not supported in far, far too many places to work. You'd be giving everyone the ability to issue for anything.
Not to mention that running any kind of public CA is harder to do properly than most people thing.
I get the negativity towards public CAs, but much of what you said isn't quite right, either.
The only "safe" way to introduce these would be to make a certificate format that's intentionally incompatible with existing implementations; that way only new implementations (which are aware of the domain constraint) will accept it where as old implementations would just reject the certificate as invalid/corrupt.
You just answered a rant about too much ceremony by linking to an entire service that costs a staggering USD 400 / month. That’s the cost of an FTE in like half the world… to issue what are essentially “fat passwords”.
Imagine with a straight face trying to sell someone a “password generator service” when in reality that’s just a one line script snippet.
The disconnect between the physical reality on the ground and how it is treated by industry is just absurd.
Ceremony, as in “a well established, rigorously implemented, meticulously documented, and diligently audited process” is required to establish and maintain trust. A CA promises that folks showing up bearing their certificate _are who they say they are_ - this doesn’t mean it is perfect (there are plenty of cases where public CAs went wrong) and some random person can place their trust in that assertion.
The whole _point_ of a PKI is trust, and why would this be any different for internal systems?
Trust can be established in many different ways. Through paperwork (literally the case for some public CAs!), via automation, or some other means.
The difficulty in running a CA at a very large scale (public or private) is that the root certificate revocation becomes prohibitively difficult. For this reason, its security is critical and much ceremony is warranted.
A CA doesn’t have to be a giant basket with millions of eggs in it. I have scenarios where I need a CA cert just so it can sign two (2) leaf certificates to make some load balancer happy, or to establish a site to site VPN or whatever. This is just "marking" the issued certs with a common attribute so that a load balancer can trust 'n' servers without having to have 'n' distinct certificates in its trust list.
That scenario does not need the same ceremony as DigiCert or the Microsoft kernel driver signing root certificate. It’s just a “shared secret” in a Cloudformation/Terraform/Bicep template.
If that CA gets compromised, I can just re-run the template to generate and deploy a new one! No ceremony. Just press play.
Not every gets married like Beyonce at an exotic island destination with 500 rich & famous guests. Some people just go down to the local marriage registrar office, get the paperwork done, and go home.
> The difficulty in running a CA at a very large scale (public or private) is that the root certificate revocation becomes prohibitively difficult. For this reason, its security is critical and much ceremony is warranted.
Unless the end-to-end, from the CA to the final certs, are managed by yourself or your immediate team, the scale would be irrelevant, and some ceremony would be required, as would be some processes that are internal to the team. My personal approach to avoid much of this is to use really short-lived certs to avoid the revocation theatre, but that doesn’t avoid the need to ensure that the CA is fundamentally secure.
> If that CA gets compromised, I can just re-run the template to generate and deploy a new one! No ceremony. Just press play.
Indeed! That, and re-distribute and implement trust for your CA Chain to all machines that need it. Your examples work fine (and I agree with) for small-scale use-cases, but if you run PKI for anything except yourself, you have to take care of the P for the KI.
Look, I am no fan of the rent-seeking bullshit the big public PKI vendors make us go through, and the incredibly high cost per certificate model. Charge me a reasonable fee for verification, and charge me a reasonable fee to set up my certificate vending endpoint, and be done. The same goes for internal use-cases. But the fact remains that verification and process are required. Fortunately, ACME for internal use and sane APIs and workflows such as Vault or smallstep provide make things very easy and cost effective. As for $400 per month for Azure certificates? Run your own instance, with your own (supposed) management processes, and see how it tallies up…
A password is a secret. A certificate is fully public (it’s useless if not). I don’t see how they’re similar in that dimension. Or how certificates are possible without a third party (a core reason for all the ceremony). Passwords get away with not needing a third party involved because they’re a prearranged process. Certificates aren’t, I need to be able to hit any website at any point, for the first time.
The key phrase here is internal use. Think authentication between two micro services, not HTTPS for someblog.com.
The third party in this case is just a file: the internal CA key.
Try to think of this in the following terms: the issued certs are just signed tokens, nothing more. They’re also conveniently a public-private key pair, but that’s not the point.
The point is that if you used a 1kb random password for service to service auth and someone tried to sell you a $400/mo service to generate them, you’d laugh in their face and then have security escort them from the building.
Sprinkle a tiny bit of cryptography on top and suddenly people think $4,800 annually is a bargain for 1kb passwords.
I can buy a decent used car for that kind of money.
Very few people still use TLS client authentication, but it does work. You can definitely get it up and running for less than 400 dollars per month. The magical keyword for Google here is "mutual TLS"
The thing is, credential management is very difficult, and people are willing to pay up to solve very difficult tasks. If you build a (non-Kubernetes} certificate management solution, you'll probably want to sell it for big bucks as well. Generating certificates is just an openssl command line on a timer, the difficulty is getting those certificates deployed in the right places automatically. It's all possible through some basic scripting, but to do it right you need to solve more than just the certificate part.
> The point is that if you used a 1kb random password for service to service auth and someone tried to sell you a $400/mo service to generate them, you’d laugh in their face and then have security escort them from the building.
Go ahead, run an internal secret store that issues properly trustworthy internal certificates. It isn’t hard, but neither is it cheap not. You oversimplify and gloss over a few _really_ important aspects of why and how certificates work. A certificate _acting like a password_ in some cases doesn’t mean it actually _is_ a password, and if you don’t really need the additional functionality to the point where the cost outweighs the delivered value, you are using the wrong tool for the job.
Running a CA is not at a technical challenge. Running a _trustworthy_ (there is that word again) CA is process heavy and it involves quite a few people doing work on a regular basis, and without those processes you may as well just something else - like an untrusted CA, or some kind of token issuing service (which, shockingly, also costs money to run properly).
A few really good options around DIY CAs were mentioned in this discussion, smallstep is pretty good for example. Nothing (except possibly your $org’s security policies) prevents you from throwing that up somewhere and start vending certs.
The fact that it is an internal CA to be used for internal purposes makes no difference, unless you believe that “everything internal is trusted because it is internal” in which case many other discussions on a very wide variety of subjects is due.
Internal here means I don’t need the certificates trusted by a third party.
Having built many Enterprise PKI systems — including Smart Card auth — I do know the complexity involved. I can prattle on for hours about how the Key Recovery Agents should be distributed and stored, and how “offline” means no network cables at all you dimwit.
I also know that there is virtually no difference between a root CA certificate and a signed leaf certificate.
They’re both just files.
The difference is the amount of ceremony.
DigiCert’s Root CA certificate files had a lot of ceremony — with good reason.
But the CA for “I need five devs in India to get VPN certs with a common root” is practically zero.
No, it does not take a “a lot of infrastructure” to host a 1kb file. It really doesn’t, and your persistent confusion is my point: you are simply unable to let go of your preconceptions.
Just last week I needed a pair of certs with a common root for a load balancer’s back end. Not for transmitting NSA secrets over intercontinental backhaul.
I already have access to a bone-fide HSM! For pennies!
Why can’t I be allowed to use that pre-engineered secure certificate storage system for its intended purpose!?
PKI can provide non-repudiation while signed tokens and API keys cannot. There's a big difference transmitting a bearer token vs establishing a TLS connection.
A public-private key pair is clever cryptography, yes, but INTERNALLY within a network they're not Magic(tm) that requires a multi-billion dollar market cap company to issue with some Indian call centre verifying my identity papers.
The same cryptographic algorithm has two wildly different uses: one that is $0.0000001 in value, and one that requires a third party organisation that needs to pay their employees and can maybe justify asking for tens of dollars. (Narrator: Let's Encrypt showed that they can't justify this either.)
People conflate the two and then try to charge $50 for the $0.0000001 use-case, which is a markup of five million percent. That's what's upsetting. It's just so absurd, and people blink slowly and then start suggesting $40 options as-if that discount somehow makes it okay. Or they start talking about "all the things you get" for that $50, when it just doesn't apply.
There should be a trivial set of commands along the lines of:
You can emulate the above with a 20-line script now, but it's fiddly, and doesn't cooperate with Bicep deployment templates. Similarly, there ought to be a built-in renewal mechanism (which is JUST 'cron' for the love of God!), but instead requires Azure Functions, layers of complex authorisations, and who knows what else...
You can't use Let's Encrypt for private DNS zones in the general case.
"Running your own CA" like it's a big ceremony is precisely what I'm saying ought not to be necessary.
Do you "run" your own random password generation service PaaS with custom Kubernetes controllers and everything? That's what someone else suggested, and not in jest!
A certificate is a key pair with both a public and private part. The private key is needed to sign data, while the public key can only validate that signature.
You can think of the private key as equivalent to a password, in the context the parent is talking about.
Thanks for the clarification. It’s a misuse of that term then. A certificate is not a key pair. A certificate is very clearly and narrowly defined: a statement of the form “this public key belongs to this name” (in this case a DNS name, but can be any name), signed by a (mutually) trusted authority. Once you trust that authority, you trust the connection of name to key.
Why - for the love of God why - can’t I just obtain a cert like this directly out of a secret store such as an Azure Key Vault!?
These things are already full hardware security modules (HSMs) with all of the capabilities required to run just about anything short of a public Root CA and maybe even that too.
But no.
NO!
Script it yourself. Make a root cert, “upload” it, make a cert, sign it, upload it, link it, renew it, re-configure it, and on and on. Oh… you wanted a CRL too? A 1kb file? Ha-ha! No. Make it and host it yourself!!
It’s absurd.
So many services depend on a simple CA->cert setup: VPNs, API auth, load balancer back-ends, clusters, etc…
But my mark my words: no cloud will have a trivial turnkey solution this decade.
This is because running a CA is culturally accepted to be a hard problem. It is! If you’re DigiCert. It isn’t if you’re building a three-server internal use cluster. But the problem is hard, you see? That accepted fact! Everyone knows it! Ceremony is required. We can’t just hand your sever a 1kb credential file! That would be… unconventional!
It’s just not the way things are done, so stop asking.