As an embedded engineer in a new position I always volunteer to take up an old crusty legacy product that nobody else wants to touch. It's a quick education into how the company approaches projects and all the ghosts they had to fight along the way. All the business logic is there, and sometimes it's even documented.
And once you have a semblance of knowledge about how the thing works and how to make fixes, you've built a small moat around your career there.
This happens to me all the time, but mostly with work code.
I'm under pressure, I have to deliver, things get rewritten many times, edge cases are found and fixed in production after they slipped through multiple qas but need to be fixed instantly or we lose money.
It happens.
But personal code? I am almost always proud of my work. It shows in general I was not under pressure and enjoying building it.
Even my code from the first year of programming, it shows it was made to make a point in writing a nice solution.
I may cringe at times, but still proud.
Context does matter, and code quality is more often than not a byproduct of our emotional state rather than skills.
I'm doing my best to prevent myself from being on thin ice with my future-me.
I posted recently about how I divorced not too long ago. I'm in an apartment now (for the first time in 16 years) while I stack up cash to build a new house.
I always think I'm not making any progress but then I see my next door neighbors ordering DoorDash 15+ times per week, and I realize that I'm doing pretty damn good for myself.
Or I walk by the mirror and think about how I've neglected my lifting for the past 6 months while I finished up this SWE degree. But I'll see those same neighbors struggle to walk up the stairs and I again realize that I'm making some awesome choices compared to those two folks.
I’ve found one of the best things I can do to help my jr devs feel more confident (and be willing to ask for help) is highlight my screwups and “basic” things I’ve just learned. It make me seem more approachable and less like everything I say is a pronouncement from on high, it makes them more willing to admit to their own mistakes, it clarifies that we’re all learning stuff all the time and it helps spread knowledge about things that folks might otherwise be too afraid to ask about for fear of looking silly.
A benefit of being a sr dev is getting the benefit of the doubt on mistakes, so I might as well use that to my advantage and let everyone learn from them, not just me.
I 100% agree with you, and have used the same technique. Showing juniors that you are also human and make mistakes allows them to be less anxious about learning and trying new things. It also helps to keep them from trying to hide their mistakes which can quickly lead to larger system issues.
Of course, you have to have a culture where making a mistake doesn't immediately lead to being let go or publicly reprimanded. I have definitely learned quite a few things about people and code by helping juniors work through bugs that may not have been so obvious (including occasionally finding bugs in 3rd party libraries).
I used to suffer from imposter syndrome significantly. It helped to see the mistakes others made which anyone could have made. It also helped to have my own good ideas and talent explicitly acknowledged. Learning to not take criticism personally and own your own mistakes also helps to rob imposter syndrome of its power because it demonstrates to others that you can be forthright and self aware, and that helps build trust.
Sure, but if there's anything I've learned from working on legacy software, it's that I don't know why the code was written the way it was. I'm not the only smart person to work on a codebase, and it's more common that another smart person did what they did for a reason, than that I'm a genius who knows better than they did.
Sometimes there were reasons, but they were not necessarily good ones. Like these:
- time pressure, leading to just making it work and not caring about edge cases, which were bolted on later. Or maybe never at all. And naturally, there is no useful testsuite.
- the prototype ended up in production
- a refactoring happened, but there was no follow-up into other areas of the code base
- features changed, but the code is still influenced by the necessities of the old way it worked. Similarly, for changing technical circumstances
- code was copy-pasted or was worked on by too many people
- code was herded too long by the same person and now, to put it nicely, bears traces of their bespoke coding style
Chesterton's Fence is still very relevant, therefore I'd still err on the side of writing a lot of high-level tests before touching anything.
Kind of light fighting against the human tendency [0] to blame the failings of others on their core character rather than the situations they are in, and vice-versa for oneself.
I don’t think there was any dig implied. And I will second that it will add to self esteem. Specially during discussions if you have more understanding about the code you will have both better contribution and take away
Sure, when you succeed in understanding someone else's legacy codebase, it builds self esteem. When you fail attempting to do that, it does the opposite.
I have inherited projects where I just could never understand what they were thinking well enough to make the service good (not running out of memory and crashing). It was very humbling; I still sometimes wake up in the middle of the night with things I wish I'd tried, but the assumptions of how it all worked was just scattered throughout a bunch of files and the assumptions of everything was just not how I'd do it.
Self-esteem that easily turns into hubris though. I think the real seniority shows when you are able to work on a legacy codebase full of the shittiest code and not have the slightest desire to rewrite it all.
I think that value comes from code that solves tough or complex problems, which can be humbling while improving your understanding. Feels good when you get a grasp of what’s going on and can understand that there may not be a simpler approach, just a different one.
There’s also legacy code that is not great and is not solving a complex problem but is just a product of the conditions at the time. It’s easy to look at that and assume you can do better given the same constraints, and this is neither educational nor esteem-building. Just an ego massage.
> Engineering isn't about working on the most interesting problems. It's about getting stuff done and management happy
Truth is harsh, however this seems to be 100% accurate for nearly all cases of employment. Rarely do you get to focus on simply interesting problems and good engineering as a primary concern
Boredom is in the mind, not the task. Things aren't boring, people are. An important type of intelligence is the capacity to find what's interesting about a task that others lack the imagination to see. One needs to be able to create their own interesting solutions rather than expecting them to be handed down on a plate.
> find what's interesting about a task that others lack the imagination to see.
word of warning from an old guy, don't create problems for yourself when you don't have to. Turning something boring into something interesting can have painful consequences down the road.
Boredom is lacking stimulation. Even the most cutting edge task can grow boring if you need to plumb with it dozens of times over a year. Just because you can find what's unique doesn't mean it stimulates you. That's the exact issue with why neuro divergent individuals are demonize: they don't take as much interest with people as "normal people" would.
>One needs to be able to create their own interesting solutions rather than expecting them to be handed down on a plate.
> You should tell that to every old manager I had.
They sound like boreful people. If you find you have become boreful, there's a good chance you may be experiencing burnout or depression, which are nasty diseases, but still ones that afflict the subject, not the object. Nothing is boring but for the person who perceives it that way.
They have a job and family at the end of the day too. Not everyone has the power nor passion to try and and move a billion dollar corporation to care for people over profit.
>If you find you have become boreful, there's a good chance you may be experiencing burnout or depression,
I haven't has a full time job in over 18 months, so I hope I'm not burnt out.
>Nothing is boring but for the person who perceives it that way.
That's like saying nothing is ugly except for what people think is ugly. Boredom is personal and shaped by each person's experience, so I don't think framing it as a person for not trying hard enough is going to go too far in practice (nor is it productive to judge people based on how they prioritize their lives). Of course some things will be boring to one person and a life passion for another.We don't have enough time nor energy to try and find appreciation for every object and concept on Earth.
You gotta go into R&D if you want to focus on the fun stuff without the annoying plumbing. But such positions require an entirely different pipeline from getting a SWE position out of college.
A good way to secure your position is to be the go-to expert for a product with many years of life ahead of it.
Fixing stuff on a legacy product may make management happy but if that product is discontinued next year then you haven't accrued technical expertise valuable to the company (but you may have built a reputation as a fixer and quick learner).
So, as usual, it is a balancing act.
Edit: this is my perspective from the embedded world. It probably applies generally, though.
When times get tight the new projects get shitcanned and the 10 year-old cash cow design gets the promised new features.
One crusty project I worked on was a legacy control board for a piece of restaurant equipment. The customer, the company that built the actual machine, had been building this product for 40 years. It had been through two PCB redesigns and two different microcontrollers, but the logic was tried and true and had to survive. A port of the project from 6800 assembly to C had completely gone off the rails and the contractor was dumped. All it took was a 20-opcode fix to a routine that the contractor just couldn't grok.
I wouldn't say that's the conclusion. If there's only one true thing about work is: management doesn't care about you. They can fire you for any reason, and thinking that by working on stuff nobody else wants to work on you are "safe", it's an illusion.
If any the conclusion is: work on what you want, life is short.
Engineers have a special place in society like doctors and lawyers. Working with management is part of the job, but engineers have a professional ethical obligation to say no if they are asked to something against the public good.
The split there isn’t in favor of doing stuff that’s fun and novel though; actually, the engineer should usually pick a boring proven solution if the public has a high stake in the outcome.
> Engineering isn't about working on the most interesting problems. It's about getting stuff done and management happy.
That's a perfectly reasonable thing to want out of engineering for yourself. I wouldn't state it as an absolute truth for all people though.
Personally, I'd like to be working on something that extends the state-of-the-art a little, even if only by a tiny fraction. It can be one for the other disciplines involved - it doesn't have to be the software I'm writing that is responsible for that (and it usually isn't), but that's what I derive satisfaction from.
Do you think that software becomes poorly understood and maintained because the company treats it as a prestigious job and rewards people for working on it?
This is how I got laid off. Working on legacy software, sole person on the team, eventually management decided that it could be replaced by AI or some such pixie dust.
Legacy software with a single dev can be on the fast track to getting shut down. If it was still a business priority, they'd be throwing more resources at it.
That's a very narrow definition of engineering. And while it's not wrong, it's absolutely more of a "management" POV. Like sure, for management, engineering is mostly about what you said, but that's it.
I find improving established code of far more interest and reward than greenfield. Coming to understand what a person was thinking while building something is fascinating. My updates are tangible life improvements to real human beings. That interests me, at least.
Greenfield, I have the incredibly stressful job of making all the major architectural mistakes people will complain about for years to come. I'm not interested in that. There's also a much higher chance with greenfield that the product will just fail and no one will enjoy your work.
I should say not prevent it. But minimize it in the maximum possible way.
I am sitting on a gold mine. But I can’t prove it. That’s the problem with this stuff. The industry moves horizontally with each design such that nobody knows if say the current design trend was better than the last. We go better or we go worse and nobody knows if it actually was.
Maybe I am sitting on a gold mine. The problem is you’d never know about it. You’d never believe it. Because a proof isn’t possible.
So because it’s not provable you have no clue about any design whether one is better or one is worse and thus you disbelieve everything.
The overall question is how do you create a design or pattern that has maximum adaptability to anticipate any possible requirement change with minimal rewrites? Where you only swap in and swap out modules without changing structure?
I think I found the best possible way. But you won’t believe me.
I'm not the OP, but I'll say that I don't believe you, because you're telling me you have The Perfect Solution, without providing even a hint of what that solution is. If I was skeptical at first, I'm now certain you're selling smoke & mirrors until you provide any evidence at all you actually have an idea here.
I literally said I don’t have a perfect solution and that you wouldn’t believe me. What I have is the best possible solution based on available solutions that we know of. I found the best way.
I understand what you are saying. Obviously the type of development you do is a factor, as well as the way you and/or your team does development. I think others are looking for the one true solution, and you are saying that you have found the solution that works for you, given the way you develop and the constraints you deal with. At least that was my understanding of it. If you are doing embedded development, being thrown into a large front-end development project isn't going to use the same solution.
I have a solution for custom content management systems that usually makes it fairly easy for me to incorporate customer requests without needing to toss out my existing solution and start over. Almost a type of framework built over an existing open source CMS product. I maintain about 90 websites as the only backend developer, and have spent about 12 years now working through various versions of the software to custom tailor a solution that works for nearly all situations I have come across. I don't do front-end development any longer, but I was involved heavily at the beginning stages to make sure both the front-end and backend were flexible and broken apart into modules.
No im saying i have a general solution for everyone. The general part only applies to the part where are software projects are similar. The part where they are dissimilar it can’t apply. It’s impossible to generalize over distinctly different things but I found a solution that generalizes over what’s similar in the overwhelming majority of projects.
If you find a templates solution works for you then that is not a general solution. It’s better for you but you are hardening and becoming opinionated on certain details and that works for what you’re working on. My solution is a lower level pattern that is more general and less opinionated. For what you do it likely is only effective if you are required to drastically change an existing project while maintaining as much existing code as possible. The solution prevents a complete rewrite and allows you to modularly pull out what you don’t need and replace with the things you do need.
If you don’t need 90 percent of your project this pattern won’t prevent that. What this pattern allows is for you to actually pull out that 90 percent. Many projects that only need you to pull out say 30 to 40 percent of it actually end up with a full rewrite because the modules aren’t decoupled enough.
Apologies for the misunderstanding. Sounds interesting, but saying you can't prove it, and nobody will believe it is a tough sell. I guess I find it hard to believe like others, but you are talking in very abstract terms, and not being able to (or refusing to) give any kind of details sounds like you are trolling or don't realize that your solution does only work for what you work on. I'd say I could be proven wrong, but it seems you don't wish to do anything other than generalize. Good luck to you if you've actually found what you describe.
No need to apologize. I assure you I'm not trolling. Yeah I am talking in abstract terms because I don't really want to get into a debate on the topic. The reason is because any design style in this area is basically unprovable and leads to endless debates. There's no point.
I do have something that explains my pattern and serves as support for it. But it's no proof because such a thing isn't really provable. Anyway it's easier to let AI explain it. In the following conversation I basically ask chatGPT about this topic with very general questions while manipulating it as little as possible.
chatGPT basically arrives at the same pattern I discovered. Where I cheated is when I started giving my own opinions on the several patterns chatGPT suggests and I hinted the LLM at the direction I wanted to go. Everything else is a very general question.
That being said, there are clearly other sets of general questions that can very well lead chatGPT to form alternate conclusions, but this one aligns with what I and many others have discovered.
Those are pretty well known FP oriented guidelines that many developers here on HN are familiar with. I don't think that your fear of not being believed is warranted.
HN is mostly not hardcore fp programmers. There’s a good amount of fp programmers but they are not the majority. Plenty of go and c++ lovers here and people who don’t understand the guidelines above.
The point is I don’t want to argue about it with someone who doesn’t get it.
I pulled this strategy at a new company and found that it opened up a million opportunities I could take my pick of. I basically got to shape my own role going forward.
Why? Because I very quickly made a name for myself as someone who learns quickly and then gets stuff done even in contexts where everyone else struggles. Solving problems that have long been considered intractable is a good way to show your value.
This is 100% true. Those problems you are solving probably don’t look all that important, but, in reality, they have been blocking sales, renewals, account expansions, or even access to markets for a long time. The business side of the house has probably given up or occasionally tries to wag the dog to get it fixed.
> Haven’t you also often made a moat between yourself and doing literally anything else more interesting at the company
Nope, assuming that code works and handles UTF8 (sorry, but there is a point where it is too far gone to save). You’ve made yourself essential. You know the business better. You are strategic. And often, you become the one person who knows the legacy code base. And you will quickly find yourself a part of super important decisions. You’ll be called out regularly for fixing things that have gone unfixed for years. You’ll also drive value by helping the company avoid spending vast amounts of money re-implementing functioning code. On top of that, new product is often a mix of new and old, so you’ll quickly find yourself in the middle of new product teams.
These insights are essential for playing the long game; it may sound distasteful for people early in their careers. That’s fine, but when you’re running out the clock to retirement, can’t go wrong.
The most interesting work is going to be 10% interesting and 90% not interesting. In most cases it will be less interesting than that.
Prioritizing "interesting" work means you'll always be exploited by the companies that can sell you on less pay, less benefits, less time off, less respect in exchange for some pittance of perceived novelty. Don't fall for this trap.
For what it's worth, I find it much easier for my ADHD to debug and improve existing code than to greenfield - it's actually interesting. Debugging is like a detective's work with small dopamine hit at each step of unraveling the situation and a huge reward once you finally grasp it. It's best to do on someone else's code, so you don't get too frustrated with your past self ;) Greenfielding provides some rewards too, but once I'd greenfielded anything remotely similar already there's a lot of forcing myself to do it around the beginning and end, where there's plenty of unrewarding work to do. For greenfielding to be fun and easy, it needs to be a completely new area that I had no previous experience in, but that doesn't tend to end up with excellent results.
On the other hand, the infrastructure variant of this is the exact opposite. Trying to figure out if something specific is in use and who uses it or if someone just clicked around in your cloud console and forgot about it is the worst. There's only so many ways I can be inventive in building proxies to measure usage.
We sometimes get incidents that things someone remembers should exist do not work, only to find out that that product was retired several years ago (but someone forgot to delete some DNS records, so there's an error message from... someone's server!). But only because we asked half the company on Teams.
... I literally had a multi month ticket open to track down an expense for something I either never used or only used once which ended up being a load balancer or some crap
As a senior engineer your job is to take on the unglamorous hard problems, both technical and organisational, that are holding the team back. Doing this, and being seen to do it, early-on in your time with an employer will be somewhere between appreciated and expected.
If you do this you should also be good at story telling. Not working on the new things, but making sure the core business running, is often times looked down upon in my experience. Most leaders will see the greenfield project move fast and wonder why the established and "complete" software moves slower.
Nope. If you're willing to take on the shitwork without a lot of handholding and show real progress (which actually impact earnings) you'll have a really good stab at new opportunities.
I found that working on legacy projects is often the most interesting.
There is the challenge of understating the original code base, which may look messy, but is full of history, you see the architectural mistakes and their consequences, how they managed to fix obscure bugs, tackle unexpected use cases, different code styles,... Then they ask you to implement a feature or fix a bug, you have to find what can be touched and what can't, what is a bug and what is a feature, see how much time you have, and decide on a strategy. This is production code and people are using it right now, and they are efficient with it, you have to help them and not force them to relearn everything for no good reason.
And there is technical debt management, your goal is to make the code better or at least not worse, but without introducing breaking changes and under time constraints. You have to manage your off time to make improvements that will help you in the future (chose wisely) as the next truckload of tickets arrive. It includes having a good relationship with your managers as no one wants you overworked, as it would just cause technical debt to accumulate with no time to repay it.
And good luck getting an AI to do that, if you want job security in the age of LLMs, that's your change. It doesn't mean you can't use AI, but you certainly won't be replaced by it.
Small, but not insurmountable. The time always comes when a new leader and their lieutenants come into the org and prioritize new features over maintenance, at which point your moat is null and void and you’re on borrowed time. I’ve yet to find a way to build a moat that withstands that level of hostility without actively harming the organization in the process.
I’d love to hear how others deal with this inevitability.
When you're on a reasonably large legacy project that's a pipe dream.
One of the old specialty C ERPs I've worked on has literally buried more than 3 (the ones I know of) "great Ruby rewrites". There are just _so many_ features, that the newcomers always ended up on the chopping block after 5-10 years and before they could achieve real customer traction. The old project just ended up getting some more interface fixes instead (aka even more features to catch up to) and to this day the compiler for the core hasn't even been upgraded to a version that can run on anything newer than XP SP1 (the improvements in SP2 break it).
Now they're doing another modernization project with some web services and stuff. We're happy to take their money for those as well. I'd be quite shocked if those do anything other than enlarge the moat of the core.
Yep. The only possible way to rewrite such projects is to do it gradually, piece by piece. It is also important that the refresh happens from within and is led by those who understand the legacy codebase and, especially, the business requirements it fulfills. No way can an external team do something like that. Being given project description "it must do everything the legacy project does, plus these new features" doesn't help either. If you get a ticket like that - run.
I just spent 5 years of my life building such a system. Then we were asked to do it again (a refresh of the refresh, if you will) and everyone marvels that suddenly we can do everything both fast and correct.
Well, we just spend 5 years training the team on how to build the old refresh, and now the new refresh can be done properly.
> The time always comes when a new leader and their lieutenants come into the org and prioritize new features over maintenance, at which point your moat is null and void and you’re on borrowed time.
The "on borrowed time" is an odd conclusion.
When someone takes over, it's typically a priority change ("We want everyone working on new projects, just freeze this old stuff") rather than some kind revenge path against anyone who worked successfully on the old stuff when it was a priority.
And a good engineer who is able to handle legacy code will typically be even better when working with greenfield code, because the latter is easier, all else equal. So unless you are refusing the priority change, why would you be on borrowed time?
It does happen that the new leader ends up saying out loud "everything we have is terrible," which is easily taken as "everyone who worked on it is terrible." And to be fair, sometimes that's true. And sometimes it's not true but the leader wants to bring over their crew of loyalists and needs to make room somehow.
Long-term ownership and careerist jockeying don't go well together.
My recent experience is that a lot of the system was not as good as it could have been because it's a hard problem. The company got bought out and all the new bosses come in and say all the existing code and way things are done is crap. Then, a year later, with everyone sacked and new people brought in, once again, these guys say the same thing and promise they are the top guns to fix it all up. I just checked out on the third round.
Ah, that's definitely true. But that feels like an orthogonal concern to you being an existing dev working on legacy stuff vs greenfield stuff... unless the new boss likes the greenfield stuff for some reason. What you are talking about is more of an ever-present possibility of "new management fallout" in almost any field.
if you've built up respect at the company and speak your mind regarding the maintenance vulnerability, the new crew should pay attention. if they don't, then you're screwed anyway, but so is the company.
source: endured this while working at a startup that was bought out by SFDC. Kept my principles, but lost the job. /shrug
Yup, you and Shermantanktop get what I was asking perfectly. Seeking confirmation I hadn’t overlooked anything that I could’ve done better, but you’re both dead-on.
The irony is that for those of us who enjoy maintaining, improving, and eventually replacing legacy stuff (code, systems, infrastructure, etc), what we really want is often just stability in career - hence the moat. It means it’s cheaper to keep us around for the long haul than rehiring us again at current market rates every few years, especially if we’re making ends meet with a high quality of life. But that’d mean focusing on the long term, which I don’t see many companies doing in general.
This can go wrong, as I found. Management and Tech leadership are focussed on the new projects with latest technology - they pay the biggest bucks for these and no one really cares about the devs maintaining the old stuff - even if important. The second problem is looking for new roles you resume is filled with old tech rather than new stuff that is in demand.
I’ve found it to be the opposite but it’s not really either/or.
Over here in the UK, if I constantly chase the new hotness I will have plenty of new startups to choose from but I’ll almost always be low-balled on pay with some practically meaningless equity package.
Gaining more in-depth experience opens up the better paying positions further up the ladder in more established companies.
Don’t want to keep your head in the sand for years but at the same time, there are a lot of skills that can’t be learned if you stay exclusively at the bleeding edge.
The moat is real. A shiny new project can always be cancelled, but withdrawing support for a suitably large legacy application that doesn't have a replacement is harder.
Also, some companies and teams do versions of this, officially or unofficially.
One pre-FAANG tech company I worked had a policy (known among the local schools) that each new hire has to "pay your dues" on grunt work, before you get to do fun/glamorous work. That kinda worked for them.
A problem they had was that they were doing desktop development in C and C++, which, among other purposes, had to ship software bundled with new brand-name PCs (so not necessarily able to bugfix over Web later). The real problem was that very few new grads are minimally competent at production-grade C or C++. But, in early Web years, the company couldn't hire enough experienced software engineers to relocate to a college town. So they had to try to train and retain local grads of a top CS program. That might've been part of the thinking behind "pay your dues"... before you get to make rookie mistakes with greenfield.
Similar here. First thing I do when joining a new project is to fix bugs in a legacy product. Forces me to read through the code, follow the data and execution. Nice thing about bugs is also they (mostly) have defined behavior.
It's all fun and games until eye̶s̶o̶m̶e̶o̶n̶e̶ ̶l̶o̶s̶e̶s̶ ̶a̶n̶ ̶e̶y̶e̶ management demands some new feature and refuses to understand why it would take so long to implement it on such a stable and well-established p̶i̶e̶c̶e̶ ̶o̶f̶ ̶o̶l̶d̶ ̶c̶r̶a̶p̶ project.
Agree. Also, not be judgemental about the state of things as in "which moron did this". Approach it from a point of understanding and respect rather than criticism. There are multiple reasons why the code base and tooling around it are in the crusty old state. They are usually decisions that engineers or middle management had no control over.
I've been with my company for a while now. Too often I find someone come in with the "this is trash and all of you are idiots" mentality. I've typically not been keen to help such folks. They usually soon ask for rescue from the very same dumpster fire they created.
... just don't make the mistake of documenting too well :-)
Developers need to realize that AI means that a lot of coddling our profession used to get is going away. Employment has always been a tenuous relationship with a fundamentally sociopathic entity, but things will be getting a lot worse in the future.
And once you have a semblance of knowledge about how the thing works and how to make fixes, you've built a small moat around your career there.