Even monorepos are not perfect for evolving to microservices. It’s incredibly hard to refactor generally. We still lack good monorepo tooling. Tests take huge amounts of time to run. Multiple languages becomes painful especially ones with slow compile speed like Rust.
Someone needs to make a programming language that encourages you to write code that can be easily split and recombined. Something that can do remoting well.
What is better than relational dbs (and for what purposes)?
I've been learning k8s fundamentals recently after using borg for a long time and then awhile kind of just poking at k8s trying to use it without learning it. Honestly I think it seems like about as simple as a solution to a broad set of real common problems could plausibly be.
Relational dbs are based on relational model. It’s a giant black box of magic. And it’s magic is based on a bunch of constraints to make the relational algebra work. It’s really quite flawed for modern web apps.
It ... really isn't a black box of magic? It's a transparent box of - as your comment points out - implementing a particular mathematical model.
You're gonna need to give me some more on why it's "quite flawed for modern web apps". I may even agree with you, but it's tough to know from what you've written so far.
Referring mostly to the query planner - you have no control over it, and you have to dive deep to figure out why it's making certain plans, and how it collects statistics to make these calls. Plus the planner essentially gives up at 6+ joins.
The whole industry is stupid at this point. I don’t think anyone knows what they are doing. Everyone just wants to play with the new shiny thing. Half the engineers probably have ADHD which leads to this behavior of seeking novelty.
Everyone is obsessed with their shitty stack they used at their last company, and this shit will keep going on and on.
The only things that matters is how easy is something to debug and to change. Think about how often you have heard that as any kind of priority. So many shitty decisions would be avoided if people started thinking about that.
Alan Kay once pointed out something along the line that when excitement exceeds knowledge you get fashion. For 10 years I’ve been in and around languages that were declared dead by the HN crowd because there wasn’t enough new hype around it. It’s not a perfectly inverse correlation, but honestly at this point when I see HN is excited about something new it raises red flags for me.
I see this more on twitter to be honest. HN, of late, has been a great source of info on the ‘boring’ stacks e.g Rails.
Twitter is where you get the JavaScript ‘influencers’ who appeal to a core demographic of engineers with FOMO who signal their ‘in status’ by waxing lyrical about the latest framework
Not to pat myself on the back but fashion was the exact analogy I came up while describing the situation. When everyone begins focusing on tech as an end in itself as opposed to it serving a bigger purpose we end up here.
And add to that the zero cost of capital of 2010s, it lead to engineers being given almost free reign which is generally a bad idea. “we are a technology company” would attract VC money, as if tech itself was a product of every darn startup.
>And add to that the zero cost of capital of 2010s
I suspect the timing of ChatGPT release just as interest rates started going up is a ploy to layer more obscurity into tech choices to drive the hype harder ad keep the gravy train going even as capex becomes more costly.
It's easy to sit back and say that, but I can almost guarantee that if you actually follow this up with a set of opinions about what stack / architecture / approach is easiest to debug and change, you will find yourself back in the argument with the rest of us stupid people.
I believe there's a lot more variables than "how easy is something to debug and change". The issue is that how easy something is to debug and change is very contextual to the point in the lifetime of an application, the industry the application will live in, and the changing requirements as the application grows in complexity.
Even the simplest interpretation of your statement leads to considerable argument. Often the ease of change and the ease of debugging are at odds with one another, where a highly dynamic approach might make change a breeze, but make debugging more difficult.
This very much. I find that often times we are looking at a narrow subset of the factors that lead to certain choices or technologies. Yes, there is a lot of fluff sometimes but there’s also the cycle of wanting to improve on existing pain points.
Things like serverless appeal to businesses due to the much smaller logistical implications it presents to a business. Some businesses enjoy that (at least as promised), It’s all taken care of and the team you need to manage it is likely to be smaller. It’s pay as you go for what would otherwise imply more human beings and contracts on your plate.
I imagine at some point those baked in costs and redundancies, the luxury of having huge scale on demand in a general sense, will be much higher than something thoughtfully designed when a company has understood their needs well enough after operating at scale.
For what it’s worth I prefer bare metal clusters :)
Serverless in the way we do it today is the best example of something that is impossible to debug.
Every choice people have to make in tech today comes with this crazy tradeoff of where you get some benefit like scalability, but then it becomes impossible to debug or change.
What's crazy about modern tech is that we've created a world where its so difficult to change things that its analogous to building pipes under asphalt roads. You have to spend all night with a construction crew digging up the road just to change a small thing, and then you have to put it all back again. Except that we work with digital stuff (any physical stuff is completely abtracted) where we can theoretically change anything instantly. The fact that we use shipping container analogies is ironic, if you've ever seen how long shipping loading/unloading and transport actually takes. We need to get back to the computer as the bicycle of the mind...at the moment it is not.
I think we will look back on this era of tech as we do to the era of the secretary on the typewriter...and laugh at the inefficiency.
Agreed on the trade offs, not sure if to the same degree though. It seems to me that it’s critical to know if you’re gonna end up going “against the grain” on a platform that makes as many decisions as a serverless one does.
I agree that "easy to debug" is subjective. But it's insanity to act like that doesn't mean that some architectures are objectively easier to debug than others. How can splitting something up into different services and introducing network calls make it easier? You're not reducing complexity at all. It makes debugging harder.
It would indeed be insanity to argue that, which is why I didn’t. The argument for splitting into micro services is generally that particular teams would own particular services and therefore make debugging/change easier for them, at the cost of increased overall complexity of the overall system. Anyone arguing it reduces complexity is being very hopeful.
I personally think an org should generally cling to a so called monolith as long as they possibly can. But it’s very hard to be prescriptive without knowing particulars of the problems an org is trying to solve.
The stack doesn't exist today. It could have if we didn't jump so quickly to new technology and all these different prog languages.
I've just never heard someone pitch new tech with: "easy to debug and change". It's always: look at this contrived example with some new esoteric feature.
I get your cynicism because I've made the same complaint. But I wouldn't universalize it, because I think it's more of a marketing and context issue than an issue fundamental to the whole ecosystem.
Startups often prioritize time-to-market for what is effectively a prototype. Startups get a lot of hype. Therefore, frameworks that focus on time to market over refactoring / debugging get a lot of hype. Sometimes those frameworks actually internally put a lot of effort into refactoring / debugging, but when it comes to marketing themselves, they don't.
But some frameworks are marketed more towards mature or large organizations, and they certainly do emphasize refactoring / debugging. The JVM and .NET runtimes are big examples here. Their debugging and operational monitoring capabilities are several classes above most of the competition, and their primary and tool-vendor ecosystems continually push further debugging capabilities as features.
Maybe I'm too caught up in startup-land, as I haven't touched these tools in a long time. The debugging tools and IDE support on these platforms have always been great.
The Visual Studio install time, the huge frameworks, slow start time, bad package managers, were such killers for productivity.
Half the engineers are made to believe they are God's gift to humanity and like so totally clever. But we all spend our time building banal and ultimately dumb shit nobody wants. But people construct their entire identity around tech anyway.
It is no surprise the place is littered with ivory towers.
There are a lot of financial incentives pushing engineers to seek novelty, even if they might not necessarily want to. Promotions and raises are heavily tied to projects introducing or inventing shiny new tech. Heck, even VC funding for starting a business is tied to this. One might also view this as the human condition: a bias to solve problems by adding rather than subtracting.
> even VC funding for starting a business is tied to this
This is the main problem.
Excluding VC, businesses exist to make profit. The purpose of IT and engineering is to enable that business or make it operate more efficiently - engineering and technology should never be a goal in and of itself.
Given this situation, the most efficient technology (that enables faster delivery of business software) would win. "Shiny" stuff would never win in terms of mindshare unless it's actually good and enables faster delivery than the status quo.
Now add VCs to the mix. Suddenly you have an infinite supply of free money that distorts the usual market dynamics, allowing businesses to keep operating with no regard to profitability. The downwards pressure of the market pushing for businesses to operate more profitably is no longer there. As an engineer, you no longer have the pressure of building technology for the purposes of making the business more efficient - you lost that valuable feedback loop.
Now run this for 10 years. Entire careers will be built on this - why even try to build a profitable business when you can be a "serial startup CEO" and merely use the right amount of shiny to solicit endless VC funding to continue the adventure? Imagine a well-meaning, budding engineer joining such a startup and learning such a corrupted idea of "engineering" with none of the usual objectives, pressures and feedback loops of engineering.
In the end, you end up with "senior" engineers (with over-inflated titles) whose entire careers are built on technology that wouldn't actually pass muster in an actual business under the normal market pressures. But they don't even know anything else or why that is wrong (because they've started their career during this charade and never had any taste of what real engineering is), so they keep pushing for these (now-inadequate) solutions, deluded that they are doing the right thing.
The software industry is filled to the brim with midwits who think their idea is the only right idea even though its nothing special and they’re ignoring actual problems
As op said - when’s the last time you heard someone talk about writing code that’s easy to debug? Never. Yet 90%+ of companies will make you pass arbitrary algorithms tests
The field is dominated by A types who confuse their neuroticism and ego for talent
But why to bring ADHD into this? This I don't understand.
I do understand the fact that everyone is following one or two bandwagons, but I don't have the need of saying that someone must have a mental condition for doing that.
There are MANY more reasons for adopting technologies. Marketing, corporate culture, hiring, trying things out, wrong people in leadership position (as the article shows!), etc etc. It's not just always the minions to do things wrong.
Why? It's a consequence of labelling a sizeable swathe of the IT workforce as "Disordered", having a "Deficit", and a "Mental condition" (which is NOT to be talked about in any context unapproved by the grand council of ADHD).
If you didn't slap derogatory labels on people, just let people who wanted to use amphetamines and methylphenidate, and addressed problems like organisation and hyperactivity independently people would end up with roughly the same quality of life but without the othering and stereotyping. Which is to say, this sort of stereotyping is an inevitable and continual iatrogenic consequence of the sociomedical status quo. When something is labelled a deficit and disorder - then people almost naturally compare that thing to other things they do not like.
My guess is ADHD people are drawn to the software industry. Open source is probably fueled by it. It’s a spectrum and I think a lot more people are on the spectrum than they realize.
It’s a constant pursuit for new stuff. Sometimes this can be good, but other times it’s just new for the sake of being different.
I think the minions are surprisingly to blame for a lot of the problems with tech. The tech debt excuse is of our own doing.
Does adopting serverless actually accomplish that? With servers you have a bunch of technologies to learn about that are abstracted away with serverless. Every person I know with ADHD in the IT field, not knowing many, is into servers.
> I don't have ADHD but I find your comment quite offensive.
I don't mean this in a bad way, but people not part of <group> need to dial it down a bit. Too many get offended by things that people in <group> don't see as a problem.
Young people with little knowledge (because they are young) give a try to every new piece of technology and start projects with them. I've been there.
Older people that eventually found good tools to solve problems have less time and enthusiasm to try new things. They still do but only when they think it's worth it. I'm there now.
> Older people that eventually found good tools to solve problems have less time and enthusiasm to try new things. They still do but only when they think it's worth it. I'm there now.
The best thing about this phase is you just get so much shit done instead of screwing around with 10 half-baked over-marketed frameworks.
> The only things that matters is how easy is something to debug and to change.
These are important, maybe most important, but software exists in an ecosystem. We still use some ASPX with .NET Framework at work and it's old tech but reliable - however, the world is moving on and it's becoming difficult to start new projects in it, not to mention end of life for patching and so on.
Likewise, at a prior role we used Perl and a lot of interfacing was made difficult by the fact that no one bothered making modules for Perl anymore. So while we can file these under easy to change, sometimes the world moves on without you and you have to think about the new shiny thing whether you want to or not. I personally started using Python at home not for any love of the language, but because everyone wrote modules and API examples for it.
I found a similar thing with Node.js. There are tons of under maintained packages while everyone moved to Go and Rust.
A lot of packages you never care what’s underneath if they are maintained and do the job. But when you have to manually patch something and it’s a pile of crap underneath then it sucks.
I much prefer to write things myself if I can now instead of using an existing package or framework. Having my own simple packages in my monorepo that I can instantly change is far better than new third party packages.
I start to wonder if checking in third party modules is the way to go.
I don’t think it’s the ADHD. I have it but I’m very conservative on the stack. And I know many developers who don’t have ADHD, some newbie and some who code longer than 20 years and are going after the latest shiny thing.
I think it’s more of what people assume it’s a status symbol among their peers. I noticed for example, in church, that some people think the more boring is more saint and don’t like more a church with a more contemporary dynamic approach and I think the same effect applies in the software development world.
You might enjoy Go. Lots of emphasis has been put into the development tooling and ecosystem. There is an impressive commitment to avoiding breaking changes.
We have a legacy Go microservices codebase that’s over a decade old and because the Go maintainers have paid so much attention to avoiding creating breaking changes and a guarantee to never create a “Go 2”, the codebase has aged much better than stuff written at the same time in Python (some of which is still in Python 2 because it is too risky to upgrade, and that functionality will be killed instead because we don’t know how to safely update it). Most of the time we don’t need to make any changes when bumping Go versions, just fixing up things our linter warns is deprecated.
Performance is absolutely a non-issue when modern "engineering" insists on running everything on underpowered VMs with high-latency network-based storage. Even if your language is truly, actually slow, you can get an instant performance boost just moving that workload to a modern bare-metal server (cloud providers basically ate all the performance improvements of modern hardware), or scale that (most likely stateless) workload horizontally if a single bare-metal machine isn't enough.
Initially I wanted Ukraine to kick Russia’s ass. Nothing else mattered. It was just so unprovoked and unfair. I wanted Europe to act sooner and faster.
But then I lost interest in the conflict and stopped following it daily like I think many others did and was able to view it from a realistic unemotive standpoint.
What is obvious like in all conflicts is that everyone will be friends again. Look at the atrocities Germany committed and Germans are now friends with everyone.
WW2 was against an evil racist ideology which had to be destroyed.
Russia is a kleptocracy run by one guy. When he dies it’s over. They will liberalize. It seems quite obvious.
A deal should have been struck to give Putin some territory. Instead we get bloodshed cheered on by the world which really has no skin in the game. It’s a spectator sport. And Ukrainian lives are being used to wear down Russia for everyone’s benefit.
Russia is a cancer that will spread. Letting Putin take over democratic countries is a great way to empower autocrats everywhere until the world is at war yet again.
Someone needs to make a programming language that encourages you to write code that can be easily split and recombined. Something that can do remoting well.