Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm glad to see more people sway to systemd. Systemd is 10 years in the making and it was met with skepticism right from the first day. Some of that is now slowly changing with systemd being accepted in more and more distributions. Service and runlevel management wasn't any better in the sysv era, nor were any of the multitude of custom start and boot scripts.

I remember when it took multiple days testing the configuration on different distributions, editions and versions to just get a single daemon process to start without failure. Then do the whole thing over again because the Debian based distros did not use the RedHat startup tools, different (or no) runlevel configurations, different service binders, different NTP daemon, different terminal deamon, etc.. And of course the French customers wanted the service to work with Mandriva, the German customers want SUSE support with extended security options like dropping permissions after boot.

Just like the article mentions you can define a portable service model with security and failure handling built in. There wasn't even anything that came close back in the day. Systemd may not have been built with the Unix philosophy in mind, but at some point that becomes a secondary concern.

Systemd unifies all systemd resources in units which work anywhere, its expandable and extendable, user friendly, allows for remote monitoring etc.



For people who had well working low maintenence environments, systemd came in and changed everything - breaking things, requiring changes to get things working again.

Its not just breaking init.d scripts, it’s ntp, dns, syslog. Systems throughout the OS fail to things that were no longer short commands with muscle memeory, there were now ridiculous convoluted commands like systemd-resolve --status instead of 30 years of typing cat /etc/resolv.conf

Even when you remember and type that in, you don’t get a simple list of nameserver and host, you get 100 lines of text you have to spend effort parsing to work out what’s going on.

When it’s less mental effort to run tcpdump port 53 to see where your DNS is going, there’s a problem.

For decades it was /etc/init.d/myserice restart

Now is it systemctl restart myservice or systemctl myservice restart? I have no idea as I’m not at a computer.

Or the restart fails it doesn’t tell you why, it gives you two locations to look for log files about why it might have broken. Init.d scripts didn’t do that. Even if there was something really wrong that log files don’t reveal, running init.d with bash -x allowed easy debugging

Systemd came in and changed working processes and systems and gave very little benefit to people with working processes and systems from a operator point of view.


I doubt answering rants is useful, but I'll try to give factual counter-arguments.

> there were now ridiculous convoluted commands like systemd-resolve --status instead of 30 years of typing cat /etc/resolv.conf

systemd-resolved is not enabled by default in Debian and many distributions, and it is not needed in any way by systemd. If you don't like it, don't use it!

Your rant does not sound very serious. Did you really have "ntp" or "syslog" in your muscle memory? That's strange, because most syslog daemon did not have a `syslog` command.

Anyway, systemd-resolved was created because it has uses. And for systems that used a dns cache (dnsmasq, etc), rejoice, because the config is now simpler than it was.

> For decades it was /etc/init.d/myserice restart

> Now is it systemctl restart myservice or systemctl myservice restart? I have no idea as I’m not at a computer.

Before systemd, at least on Debian, for a few years the recommended way was NOT calling `/etc/init.d/something`, but instead `service apache restart`. Since sysv was unsuitable for many uses, several alternatives emerged, like "runit", or "upstart" for Ubuntu. So, before systemd, the recommended way changed with the distribution.

Thanks to systemd, most linux installs now use `systemctl restart service1 service2`. Note that you can now act on multiple services at the same time. You can use this feature as a mnemonic.

> Or the restart fails it doesn’t tell you why, ... Init.d scripts didn’t do that.

In many cases, init.d scripts told you nothing when they failed. Each service has its own procedure. Nowdays you can always see what happened with the command systemctl prints on failure.

And `systemctl cat s1` display starting instructions that are rarely longer than a dozen of lines. I remember init.d scripts that were hundreds of lines long, and awfully hard to understand.


I tend to agree with most of the points you're making but I do want to point out that "If you don't like it, don't use it" isn't helpful advice for system administrators who weren't given a choice in the matter. It's fine if you're designing a system from the ground up, but most of the time, most of us have to work with what we're given.

I think most of the resentment against systemd is that it felt like it was forced down our throats. Systemd may be an improvement over the mess of shell scripts we had before but it isn't perfect. It's one thing to choose to give up decades of experience so one can voluntarily switch to a new system. It feels very difference when the choice isn't one's own.


If you can't opt out of systemd-resolved due to job policies, that's really not systemd's fault and there's nothing they can do to solve that situation. Why complain about it in that context?


What does this have to do with job policies? Systemd was forced down our throats by Red Hat adding hard dependencies on it to other software under their control, e.g., GNOME. Other distros then adopted systemd under duress, since many of the packages that Red Hat made depend on it are important to the Linux ecosystem, and the other distros didn't have the resources to fork them all.


> Systemd was forced down our throats by Red Hat […]

That's a very emotional take on the whole thing. As I saw it, systemd happened and the Ubuntu developers eventually concluded "well, that's better than upstart, let's use that." Plenty of other distros made the same rational decisions. Meanwhile, the GNOME developers thought "great, people are converging around a modern init system, we can actually integrate with it now," and so they did.

Also, GNOME is not under Red Hat's control. They contribute a lot, but the leadership rarely has a majority of Red Hat employees. While a large number of contributors work there (of course they would - Red Hat is big!), the majority are - again - from elsewhere. I can think of plenty of recent features that people assume are Red Hat driven and I can assure you they definitely are not.

What your take is doing is discounting a very large number of peoples' wisdom, time, and effort, by claiming their contributions are made as helpless victims of some conspiracy or as evil supporters of it. Both of these ideas are harmful.


> Meanwhile, the GNOME developers thought "great, people are converging around a modern init system, we can actually integrate with it now," and so they did.

> plenty of recent features that people assume are Red Hat driven and I can assure you they definitely are not.

My claim is specifically that the people who added the hard dependency on systemd to GNOME were Red Hat employees. I'm not talking at all on who wrote or merged any other code in it.


Your claims went further than that and it feels very disengenuous for you to try and walk them back now.


When was I ever talking about changes to GNOME other than the systemd dependency? What am I walking back?


> Systemd was forced down our throats by Red Hat adding hard dependencies on it to other software under their control, e.g., GNOME. Other distros then adopted systemd under duress

What utter drivel. I've participated in the discussions around systemd in various distributions. There was a huge amount of discussion, then one by one distributions switched. Some quickly, some took various years. Again, some distributions took various years to switch.

That you can only say things such as "forced down our throats" and "duress" says enough. Not capable to actually hold a discussion, let's be emotional and without any actual facts.


> That you can only say things such as "forced down our throats" and "duress" says enough. Not capable to actually hold a discussion, let's be emotional and without any actual facts.

Does this sound any better? Red Hat used their influence over GNOME (and other programs) to add a hard dependency on systemd to it. This forced other distros to either switch to systemd or drop support for GNOME. I suspect that had Red Hat employees not added hard dependencies on systemd to any other software, that no distributions other than Fedora and RHEL and its clones would require it.


Nope, it's still entirely incorrect. Again, the release team misjudged things. I was part of the GNOME release team at that time. We were actually warned about it, then misjudged it ("it'll be fine").

Further, it wasn't even a hard dependency. You're really not understanding components and APIs.

> This forced other distros to either switch to systemd or drop support for GNOME.

No, again entirely incorrect. GNOME runs without systemd. A few distributions worked on ensuring GNOME runs without it. It took a while to make that happen, so for a bit some distributions needed to keep some components back. But still: you're talking about systemd while it was an interaction of a few components. Systemd consists of loads of bits.

GNOME _runs_ on distributions without systemd! It took work to make that happen, we coordinated to ensure the problems would be solved.

> I suspect that had Red Hat employees not added hard dependencies on systemd to any other software, that no distributions other than Fedora and RHEL and its clones would require it.

Again, you're so incorrect it's not funny. Arch was really quick to switch to systemd. I help out with Mageia, they really wanted to switch as well, but it took (volunteer) time to make it happen. Opensuse took a while, but still, they would've switched.

The only unique ones were Ubuntu (political crap) and Debian (partly due to political influence by Ubuntu).

Systemd was selected on merit by loads of distributions, not this conspiracy thing you're pretending it to be.


Not GP.

The fact that it took _work_ to get GNOME to run without systemd is a bad thing in my book.

Also, to claim that systemd was selected on merit without anything backing up the claim of "merit" is disingenuous; plenty of worse solutions end up winning all the time.


It took work because systemd (more precisely logind) solved real problems and so the work had to be done again for the sake of non-systemd distros.


Can you tell me what problems logind solved? Because on my own machine, I don't have it, and I don't need it. In fact, it's a struggle to make sure it's not pulled in as a dependency of anything.


The point is that the replacements being talked about, systemd-resolved for dns, systemd-timesyncd for ntp, and systemd-networkd for whatever else you would prefer to use for network config, are not a mandatory part of systemd. You can use systemd without using these other components and it will work perfectly well with whatever other services you want to use for dns, ntp, and ip networking. These other services are not dependencies of GNOME, either.

People are conflating systemd itself with all the optional services it comes bundled with. I don't even believe resolved and networkd are enabled by default, at least not by the vendor. Whether a distro enables it depends on the distro.


You have to be an expert on systemd to figure out how not to use systemd.


This would be a comment with two parts. First one will be counterarguments, second will be a generalized response. Please bear.

Part One:

---------

> systemd-resolved is not enabled by default in Debian and many distributions, and it is not needed in any way by systemd. If you don't like it, don't use it!

It's not possible to ask systemd about which parts are enabled and up to which extent. It always adds a discovery phase before starting to make changes in a system. If you don't do this discovery, you're probably in a wrestling party with systemd. If you do this discovery, it costs you time. Systemd SHALL provide a way to see how much of its enabled up to what extent.

Network Manager got this right. If there's a distribution native configuration file for an interface, Network Manager ducks out and leaves system to its own. If systemd finds an equivalent service running both in its own ecosystem and from another package, it either overrides it or collides head-on with it, so either systemd equivalent works, or nothing works as expected. Neat(!).

> Before systemd, at least on Debian, for a few years the recommended way was NOT calling `/etc/init.d/something`, but instead `service apache restart`.

That'd be RedHat family of distributions. Debian doesn't have service command out of the box. Using both for over a decade, RH uses service, Debian uses /etc/init.d

> Thanks to systemd, most linux installs now use `systemctl restart service1 service2`. Note that you can now act on multiple services at the same time. You can use this feature as a mnemonic

There was a thousand ways to do that before systemd, systemd added yet another way. It's not bad, but it was not novel in any way.

> In many cases, init.d scripts told you nothing when they failed. Each service has its own procedure. Nowdays you can always see what happened with the command systemctl prints on failure.

systemd reports the failure in the lines of "service failed to start. go look to the logs. I also probably intercepted them, so journalctl may work".

Any good init script reports the error as "I cannot find my run file and/or process, please go get the logs. Something is probably borked". It's more/less the same thing. Better reporting in systemd requires systemd targeted service files, which are not mandatory, or most practical all the time.

Part 2:

-------

Whenever something comes up about systemd, this summarized conversation comes to life:

    U1: We were happy before systemd. It broke too many things, it also tried to hijack everything. Now everything is different.
    U2: *defend systemd in various and countless ways*
    U3/U1: You're wrong.
    *A flame war ensues for some time*
This is neither productive, nor beneficiary to anyone. I'm using this thing called linux for more that 15 years. It's nearing 20. I've used init.d, mudur, upstart, systemd, etc. They all have advantages and disadvantages

However, neither of these systems have this fierce defendant army of systemd. To be brutally honest, systemd has a lot of advantages, makes many things more practical, and faster in many ways.

OTOH, systemd is not radically fast. It's faster, but not blindingly. It's practical, but not always. It's hijacking of services makes some stuff very backwards. Replacement of NTP and resolved makes things hard to manage. Default binary logs makes some admins and external systems go blind.

I'm not against progress or systemd in general, but please be a little more accommodating for neutral and negative comments about systemd. Not all the commenters are bone headed caveman who love their flint stones and reject lighters with all their life!


>It's not possible to ask systemd about which parts are enabled and up to which extent. It always adds a discovery phase before starting to make changes in a system. If you don't do this discovery, you're probably in a wrestling party with systemd. If you do this discovery, it costs you time. Systemd SHALL provide a way to see how much of its enabled up to what extent.

It absolutely is possible to do this. It's the same as finding out if any service is enabled or not. Every view of a service includes it's state and whether or not it is enabled. So if you want to check an individual service, look at systemctl status $service, and if you want to look at all services: systemctl list-unit-files

And if you want to look at only the enabled ones, you do the Unix thing and "| grep enabled"

I don't see why systemd needs to hardcode a command for this.


Just want to add, when you systemd list-unit-files | grep enabled, you do need to know the first field is the actual state and the second field is the vendor default state.

But this comment is totally right. If you're a Linux sysadmin, how hard is it seriously to type into a search engine "systemd list enabled services." Exactly this command very helpfully comes up in DuckDuckGo's knowledge graph bubble so you don't even need to follow the link to askubuntu. I'm sure Google search does the same.


>There was a thousand ways to do that before systemd, systemd added yet another way. It's not bad, but it was not novel in any way.

The difference here is that systemd actually went to the individual distro maintainers and listened to their concerns, made the necessary changes, and convinced them all to adopt it. That's damn hard to do in the Linux world, I commend anyone who can do it successfully.

Regarding your part 2: For whatever reason, there is an absurd amount of misinformation posted whenever systemd comes up. If you posted something that was wrong about some other service manager, I would correct that too. You deserve to know the right answer to things, for your sake, not for the sake of systemd (or any other program). Please don't dismiss attempts to correct misinformation as being unproductive, it's the flame war which is the unproductive part.


"Systems throughout the OS fail to things that were no longer short commands with muscle memeory, there were now ridiculous convoluted commands like systemd-resolve --status instead of 30 years of typing cat /etc/resolv.conf"

As a sysadmin, for me things like that were very, very minor issues.

The main problem was that systemd had awful documentation, written by people who'd clearly never had to use systemd in anger and just assumed that everything would work swimmingly (and please don't say read the man pages.. those are barely adequate).

When things broke there were no simple and obvious ways to fix it, you had to dive in to its labyrinthine spaghetti architecture and hope and prayed you somehow got the Rube Goldberg machine to work.

Hopefully that's improved by now, and there's some canonical documentation that really shows you how it all fits together and how to fix it when it falls apart.


The documentation (man pages) is still pretty bad. It's always a guessing game for me where to look up the config options: systemd.exec, or systemd.service, or systemd.limits, etc.


I think you're missing a few things here:

* Services started with sysvinit would put logs where they want, which is fine if you know where they are but per-service you might be guessing. Having everything always in the same place is handy.

* sysvinit wasn't giving you any of these security benefits.

* If your system really was working fine before, why did you need to upgrade it to a newer distribution with systemd?


> * If your system really was working fine before, why did you need to upgrade it to a newer distribution with systemd?

Because the only two alternatives are 1) running mainstream distros from before 2014 or 2) running obscure distros that still don't use systemd.


yes, people forget that we are a village.

Contrary to “popular misconception”, Linux is not a settler's freehold where every holdout makes their own rules.

Admins have to work with a diversity of systems. They can choose how to setup new ones, but they have to work with a range of them. What everyone does has an effect on everyone else. If some distro introduces a new way of doing things, it has some chance of ending up affecting a lot of people - they might have to adapt to also support that way, or handle it in some way.

In that way we are a village, things are connected, and that’s why there is some degree of "social control" - looking across the neighbor’s fence and meddling with their way of solving the problem - it can in some sense become ours, if we are unlucky.

Fortunately, we can relentlessly copy good solutions from others in the village too.


So the admins should really not rant about systemd but complain about the distributions who switched to systemd or about their employers who force them to use such distributions.


Or just buckle down and view a new chance to learn something as an enjoyable opportunity. Dammit people we aren’t paid for being experts but the ability to become expert-like in new areas.


More than one thing can be bad.


That's the marketplace of ideas in action. Volunteer-run distributions like Debian and Arch have switched over. If the whole world where RHEL, you might have a point, but it's not.

You can contribute to your own distro, and the "veteran UNIX admins" made Devuan. If it still counts as obscure and you don't want it to be, you - yes, you - can do something about it.

The free/open-source community is a do-ocracy. The things that are worked on are the things the people doing the work want to work on. If you have a well-paid sysadmin job where you are providing your employer value by using the work they provide you for free - and, in particular, using their ongoing work which they continuously provide you for free, because you feel like a mainstream distro from 2014 doesn't suit your needs - then you can either be grateful for what you get for free or you can contribute back.

(Which doesn't necessarily have to be contributing your own work. I'm sure if you get your employer to donate one FTE's salary to Devuan, you can change its obscurity pretty quickly!)


Or you could've actually improved init.d and add all the features to is, and make easy to use/create.


I suspect many sysadmins, certainly the ones who complain about syslog, were quite happy with init.d

Traditionally developers would write code and packagers would package them into a distro specific rpm/deb -- including the init scripts (which may be pushed back upstream). Developers wanted to bypass this slow process, and there were far more developers than people willing to package the software.

Personally I've never had a problem writing an init.d script.


All I ever could do was add a line to rc.local that started a supervisor process to run my stuff. But I have many service unit’s as part of user-data.sh or cloud init and they work and restart on failure and are visible to other admins I haven’t talked to etc. it was surprisingly easy.


> For decades it was /etc/init.d/myserice restart

You should always use "service myservice start" instead of "/etc/init.d/myservice start". Running "/etc/init.d/myservice start" directly means the service ends up accidentally inheriting parts of your shell's state (environment variables, limits, current directory), which is a different environment from when the service is started at boot. The "service" command carefully cleans up most of the environment before running the script, making it much more similar to what will happen when it starts automatically on next boot.

And if you were used to "service myservice start", it now automatically forwards to "systemctl start myservice" when the /etc/init.d/myservice script does not exist, so it keeps working nearly the same after the transition to systemd.


> Or the restart fails it doesn’t tell you why, it gives you two locations to look for log files about why it might have broken. Init.d scripts didn’t do that.

Some things are a matter of preference, but this bit is just wrong. Init.d was hilariously worse. Some services had their own configuration locations, some had those exposed via /etc/defaults, some used syslog, some redirected stdout/stderr, some redirected one and discarded the other.

You're right that there are two places now - logs are either in the journal or in app-specific log location. And stdout/err go to the journal. Those 2 places mean fewer places to look through than we had before.


> For decades it was /etc/init.d/myserice restart

Thing is, it was never this command, it was always https://linux.die.net/man/8/service , but your command also worked in 99% of the times, until it didn't and restarted service misbehave. Systemd streamlined whole experience.


I only used Linux mostly during university, while at CERN and only after most UNIXes gave up fighting against it.

From what I remember, everyone else on UNIX world was already following up on systemd like systems before it got adopted on Linux world, and on Red-Hat/SuSE distributions daemon scripts existed for quite a long time as well, so no it wasn't * /etc/init.d/myserice restart* for decades.


> Now is it systemctl restart myservice or systemctl myservice restart

Actually long before that it was service restart myservice, which still works.


Unfortunately, it was `service myservice restart`, which is the wrong order if you want to be able to do multiple things at once (or even just align with any other `cmd subcmd args` program).


  service myserver restart

  /etc/init.d/myserver restart
Seems the traditional order to me

  $ service myserver status
  $ service myserver reload
  $ service myserver restart
makes perfect sense when tackling a single service


The traditional order is more like

    # /etc/init.d/myserver reload
    unknown command reload
    # /etc/init.d/myserver restart
    unknown command restart
    # /etc/init.d/myserver stop
    myserver: warning: frobulator did not fully unfurl
    # /etc/init.d/myserver start
    myserver: still running on pid 1234
    # kill -TERM 1234
    kill: (1234) - No such process
    # grep "running on pid" /etc/init.d/myserver
            echo "$0: still running on pid `cat /var/run/myserver.pid`"
    # rm -f /var/run/myserver.pid
    # /etc/init.d/myserver start
What you're calling "traditional" is the minor, purely UI-facade-level consistency we got after 10 years trying to clean that mess up. But actually we eventually gave up, and went on to invent something better.


I sometimes feel a bit weird when I see complaints like this. Because I have never experienced broken by systemd.

For logging, can't you write a systemd service in bash with -x flag?


Recently a systemd update that came with a flatcar linux update added/enabled a systemd-resolved stub that hogged port 53, preventing the actual DNS server on the machine from starting up. Does that count as broken by systemd?


No. Sounds like broken distribution.


Oh, you expect the error message when you restart? Oh, sweet summer child. You won't get that error message until someone connects to the unix socket and the daemon lazily gets started, and crashes because you had a typo in your config file. You didn't know that, because you were under the understanding that the "restart" succeeded, since it said it did and returned EXIT_SUCCESS.


I used Linux about a decade before systemd came along, and I welcome it. I think it made almost everything better, and not by a small margin.


If you are using UNIX like environments, you are forced to remember arcane commands - after almost 25 years of using Linux, I still have to open the man page for something as ubiquitous as grep - because there's so many options and you are always finding new use cases.

The best solution is to maintain infinite bash history and stop trying to remember arcane stuff.

Don't even get me started on tools like jq - so convoluted that I use python instead


It took like six attempts but I finally like jq. Very handy for one liners or quick bash scripts that read data from AWS cli output. It yeah in general prefer to remember the search terms for the working examples. I Google “Awk add total” at least one a month.


> Now is it systemctl restart myservice or systemctl myservice restart? I have no idea as I’m not at a computer.

I felt so alone in the world until this moment


Well, by the time the systemd gets as old as sysvinit currently is, everybody will have the correct one in their muscle memory.


I agree with the GP in that I can never remember the order of args for systemctl either. But I also accept that systemd is here to stay (and I like it, especially for writing services, and across distros!), and I'll get used to it eventually. For example, I managed to get over `ifconfig -> ip addr` fairly quicker, even if it was really annoying at the start!


You do know that resolving something involves other things than a simple udp query to port 53?


Here's a decent overview if anyone is interested in details.

https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-l...


The key Unix skill isn’t typing cat /etc/resolv.conf but strace -f -ff -o /tmp/1 -p 33333 to figure out how your DNS resolution is occurring. Maybe it is the traditional way, maybe some code is configured to use a network placed resolver. You can find out.


Tab tab...


I think it's also because those people that didn't want systemd have just moved on. I moved my servers to alpine and my desktop to FreeBSD. It's just not a thing in my thoughts anymore. I wouldn't write about it. So it seems the Linux community is more aligned now.

However alpine is working on a similar thing based on s6 but with modularity and lightweightness as design goals. This sounds great to me. I'm not against the idea of a service manager, but I think systemd is overreaching.


Indeed, they've either moved on or grudgingly accept it, even if they don't like it. Same thing happened with solaris, aix, hpux people who had to learn to accept linux as being the way of the world.

I'm sure many people - especially developers who use OSX all the time - love systemd. That's fine, but people won't love systemd, just like solaris people learnt to accept linux.

I'm also sure not everyone loves it. Some have moved on, some haven't but now tolerate it, they've spent the time to cope with it, maybe it's costing them more time every day than pre-systemd, but it's not big enough problem to move on. That's just life. I'm sure some people didn't like it when program manager was replaced by the start button in NT 4 either.

It seems that systemd fanboys just can't acknowlege some people don't like their new world order, which is rather sad in itself.


If anyone's interested here are the details: https://skarnet.com/projects/service-manager.html


There's a lot of poorly-understood incidental complexity in the systemd codebase, and this can bite users even when doing basic service and runlevel management. The systemd approach is to try and make it 100% declarative based on simple .ini files, but the semantics of this seemingly "declarative" configuration was never properly specified. Even many systemd fans seem to be quite aware of this, and there seems to be a common understanding that some ground-up reimplementation of these ideas based on a clearer underlying "philosophy" will be needed at some point. Systemd has been a successful experiment in many ways, but relying on throwaway experimental code for one's basic production needs is not a good idea.


> [...] but the semantics of this seemingly "declarative" configuration was never properly specified.

What's missing from docs like [0] that makes you say that?

[0] https://www.freedesktop.org/software/systemd/man/systemd.uni...


I know so very little of systemd as it doesn't touch anything I do.

I can pattern match that https://blog.darknedgy.net/technology/2015/10/11/0/ - "Structural and semantic deficiencies in the systemd architecture for real-world service management, a technical treatise" from six years ago (Discussion at https://news.ycombinator.com/item?id=10370348 ) might be relevant.

Are those points reasonable? Dunno. If so, have they been addressed? No clue.


https://blog.darknedgy.net/technology/2020/05/02/0/ is a 2020 post from the same blog, pointing out that these issues basically remain unaddressed. But again, this is not just some "anti-systemd" talking point; the pro-systemd side also acknowledges this! They just think sysv-init was even worse.


I honestly would like to see a better service manager than systemd, but just my opinion from following development of these things: a huge reason why it can't happen comes from underlying deficiencies in the kernel, and with Unix. The real core issues can't be addressed without a large amount of changes there, which are outside the control of a service manager.


I'd love to see a later development of a proper system manager that learns from systemd's faults and allows for compatible interfaces for transition.

Much like Pipewire exposes PulseAudio interfaces but is implemented differently.


To me it's the other way around -- pipewire is the sound server equivalent of systemd. For better or for worse.


What would you prefer then? Jack?


I just use systemd and pipewire, it's not a problem for me personally. They're the worst options... except for all the others :)


> semantics of this seemingly "declarative" configuration was never properly specified

Yeah, and it is not only underspecified, but too weak to be useful, which just pushes all the init.d logic somewhere else. What you've accomplished is moving it somewhere nonstandard, great job.

Also, the command line ergonomics suck. Systemd is deeply unfriendly to humans.

It was a power play in support of a long-term RH strategy, supported by a lot of bad faith arguments. Fascinating to watch as sociology, less nice as a forced-user.


It's interesting to see these allegations ("power play in support of a long-term RH strategy") without any proof or for that matter any explanation of what the strategy would be.


I believe that Red Hat employees were some of the main contributors to Gnome, which quickly made systemd a hard requirement, forcing distros which used it as the default desktop to make systemd the default init system.


> I believe that Red Hat employees were some of the main contributors to Gnome, which quickly made systemd a hard requirement, forcing distros which used it as the default desktop to make systemd the default init system.

Yet another person writing fantasy, not facts. Systemd wasn't "quickly" made a hard dependency. It was a soft dependency for ages, then eventually the release team made a mistake around systemd-logind. I was part of the GNOME release team at that time. Still, GNOME runs without systemd. Meanwhile we had loads of discussions with loads of distributions.

Yet the things you write: quickly done, apparently lots of people were secretly paid by Red Hat, forcing distributions? All devoid of any facts, just emotions that dismiss the amount of work me and loads of volunteers have done.


Perhaps my framing of the situation was a little simplistic. To provide more facts, let me point out that the question of which init system should be default in Debian was first asked[0] to the tech ctte in October of 2013, however it was already clear the previous month that the GNOME packagers were trying to make systemd the required init system[1] in Debian (which had GNOME as its default graphical environment already):

"Debian GNOME packagers are planning the same AFAIK; they rather just rely on systemd (as init system, not just some dependencies). In the end, the number of distributions not having systemd decreases."

That was written by Olav Vitters, of the GNOME Release Team, who later admitted[2] "Personally I’m totally biased and think the only realistic choice is systemd."

You could argue that any blame for this dependency therefore lies with the Debian packagers, rather than Red Hat employees, but actually, if you look into the history, there was already a push for making GNOME dependent on systemd three years earlier by none other than Lennart Poettering.[3]

Even at that time, Josselin Mouette, founder of the Debian GNOME team, obsequiously replied "I don’t have anything against requiring systemd, since it is definitely the best init system out there currently" and later acknowledged the influence Red Hat had over the direction of GNOME, saying "Red Hat being the company spending the most on GNOME, it is obvious that their employees work on making things work for their distribution" and "on the whole we don’t intend to diverge from the upstream design, on which a lot of good work has been done."[4]

So there was definitely pressure on Debian from GNOME to make systemd the default init, and pressure from Red Hat to make GNOME depend on systemd. Whether or not these decisions were all coordinated in advance in a smoke-filled room is beside the point, given that things worked out exactly the way such a conspiracy would have wanted.

[0] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=727708#5

[1] https://blogs.gnome.org/ovitters/2013/09/25/gnome-and-logind...

[2] https://blogs.gnome.org/ovitters/2014/02/03/my-thoughts-on-t...

[3] https://mail.gnome.org/archives/desktop-devel-list/2011-May/...

[4] https://raphaelhertzog.com/2012/01/27/people-behind-debian-j...


How about the two democratic votes in favor of debian made in a system that is more democratic than real world voting? Or that Arch independently made the switch?


You're missing the initial Debian vote, which was made in the Technical Committee, and which came down to a tie.

If you want to claim that process was democratic, you have to believe that the members of the committee perfectly represent the opinions of all Debian developers (to say nothing of Debian users), and therefore have to excuse the fact that the vote had to be settled by giving one of those people, Bdale Garbee (HP's CTO of Linux), effectively two votes.

It was only three days after the tech committee's decision that Mark Shuttleworth announced that Ubuntu would fall in line by abandoning Upstart[0], and not until many months later that the first General Resolution was put forward to try overturning the committee.[1] Of course by that time everyone was tired of the arguments and it would have soured relations with Canonical to force Ubuntu back away from systemd, so the GR was doomed from the start.

[0] https://www.admin-magazine.com/News/Ubuntu-Abandons-Upstart

[1] https://www.debian.org/vote/2014/vote_003


It's not a hard requirement, GNOME still runs on the BSDs.


Thankfully, sanity eventually did prevail in GNOME (just related to this matter, not in general), but there was a period of time where systemd absolutely was a hard dependency,


I'm not sure what period you're alluding to, but if that did happen, it seems it happened because things stalled on the BSD side, not because of any changes in GNOME. Can you please elaborate what you mean? I'm interested to get a BSD developer's view of the history, but I've only seen a few vague blog posts on this matter.


> Thankfully, sanity eventually did prevail in GNOME (just related to this matter, not in general)

You've rewritten history to pretend you're right. Then you follow up with more drivel? Sorry, aside from trolling, what is your point?


> You've rewritten history

This seems pretty clear that GNOME used to depend on systemd. How am I rewriting anything? https://wiki.gentoo.org/wiki/Hard_dependencies_on_systemd#Pa...


GNOME relies on some API that has been implemented by elogind. Nothing in this bit changed. Gentoo (I think with the help of some others) ensured it was implemented (forked) so the API could still be used without systemd.

I thought you were the same as the other person. You're continuing the same argument, so it really doesn't matter if you're the same or not: "quickly made systemd a hard requirement" is bullshit. Further, it could easily be worked around.

If you notice e.g. the history of Ubuntu it happens regularly that you hold back a component if there's a problem integrating it. This happens across multiple distributions. It isn't something unique, nor special.

GNOME nor Red Hat did NOT "quickly made systemd a hard requirement". Gentoo was great to ensure that the API that was depended upon was implemented separately. Aside from that, a distribution could also hold back the logind change that caused this change. Skipping over all of these details is great to make this into some big conspiracy story. However, it is rewriting history. It wasn't something unique. Yeah, GNOME release team misjudged one thing. But it actually took a few years before it became an issue. Not this drivel with "OMG they added a hard dependency". It wasn't like that.


That's not a full picture, apparently that was due to some specific problem in Gentoo: https://blogs.gnome.org/ovitters/2013/09/25/gnome-and-logind...

AFAIK the GNOME Wayland session still depends on logind, but that's more because there has been no interest in getting it to work on BSD yet.


I hope you're right and we get a better, simpler declarative format in the future. systemd isn't bad but i've found when writing service files that some stuff is not as obvious as it should be.


> semantics of this seemingly "declarative" configuration

What do you mean? Why the scare quotes? I'm not aware of anything in .service and related files which isn't declarative.


The question is, was it better before systemd?


No way in hell


I don't think it's that people are warming to systemd. It's more that there are two kinds of people now:

1. People to young to remember stable software. 2. People who have given up, and just accepted that Linux too "just needs a reboot everynow and then to kinda fix whatever got broken".

systemd has normalized the instability of shitty system software. And just like how you don't see front page news every day about 1.3M traffic deaths per year because it's not news, you don't see people up in arms about shitty Linux system software.

It's normal now. It didn't use to be.

Yes, ALSA is better than OSS, and then PulseAudio and now pipewire. It can do more. But when did it become acceptable to get shit, just because the shit could do more things?

Pipewire is not bug free (I have a bug that's preventing me from an important use case), but it's sure more reliable than PulseAudio, while still being more capable.

So maybe Pipewire is showing a trend towards coders actually giving a shit?


3. People who realized that the FOSS community they were originally so happy to have found - of people who actually care about writing software that works, rather than user-hostile malware designed around profit margins - effectively doesn't exist anymore, and refuse to update their software anymore at all because they know that recent versions are unusably broken, to the point of requiring "a reboot everynow and then to kinda fix whatever got broken".


I’ve since moved away from systemd for all my Linux boxes, work and home.

We still cannot block systemd from making a network socket connection so security model is shot right there by the virtue of systemd running as a root process.

In the old days of systemd, no network sockets were made.

Systemd has become a veritable octopus.

Now, I use openrc and am evaluating S6.


Selinux can block systemd from making network sockets.

What do you mean by "security model" in this case? What model is that?


By "cannot block systemd from making a network socket connection", I think GP meant that your system will break if you block systemd from making network socket connections, not that it's physically impossible to do so.


Yes. If you borked your systemd config file, system-wide failure. Non-bootable, non-useable.

Also if intransigent network connection occurs (fuZzing or not), untested errors occur … notably in PID 1 which isn’t easy to debug.


But the solution is for it to not try make the socket rather than needing another system to correct bad behaviour of the first system?


There ye go. Make system process small as efficient, not to mention easily auditable.


Systemd is already in root mode. Just need to do malicious buffer overruns … somewhere, somehow … at root level … someday.

And it’s codebase is too ginormously huge.


Service and runlevel management wasn't any better in the sysv era, nor were any of the multitude of custom start and boot scripts.

They might not have been better or more robust, but they where easier to understand and reason about. You could explain the entire thing to the most junior of sysadmins in a few minuets, tell them to read the boot scripts, and they would basically understand how everything worked.


Really? Have you ever written init scripts for a hand full of services for your typical saas application? It’s always been a mess, and I’m very happy systemd was copied from / inspired by Apple’s launchd.


I'm not saying it was necessarily easier to use, just easier to understand what was going on. Explaining step by step how a Unix systems started up used to be trivial and make sense. I don't hate systemd, I just don't understand it. But that could also be symptom of me being old.


Naa. I'm young and generally positive about systemd, but I'd happily admit that the complexity gap is huge. It's practically impossible to explain systemd to anyone not intimately familiar with linux and systems programming without a bunch of handwaving.


This. I can easily believe that systemd is an improvement in many ways for people who have time to understand it (especially in NixOS, as noted by another commenter). But I'm not happy that so many parts of Linux now have such a steep learning curve.


But system init is a hard, complex problem. You can’t create a simple solution for that, since there is an inherent complexity. I prefer systemd over having a bunch of bash scripts trying to do service restart, logging, dependency management and failing at it. You would still get the same complexity but at a different (worse) level.


> But system init is a hard, complex problem. [...] a bunch of bash scripts trying to do service restart, logging, dependency management and failing at it.

Playing devil's advocate: system init by itself is easy, just have a single script starting each daemon in sequence, like it was done in the distant past (IIRC, "init" started both the getty for each terminal, and ran a single startup script). It's the "service restart, logging, dependency management" part that's complicated. And unfortunately, since nowadays devices are often hot-pluggable, you can't really escape from the "dependency management" part.


Dependency management is not only due to hot-pluggability, but inherent dependencies between different services. This is the same problem as with package managers and I would not necessarily say it is easy.


> But system init is a hard, complex problem.

It's not. Read the shell scripts that openbsd uses to init. Simple, straightforward, easy to understand.


The complexity is buried in the huge work the OpenBSD devs make to keep the kernel and the base system small, elegant and consistent.

I read your comment more as a tribute to the excellent work of the OpenBSD team than a denial of the thesis of the complexity of the init process.


> The complexity is buried in the huge work the OpenBSD devs make to keep the kernel and the base system small, elegant and consistent.

>> You can’t create a simple solution for that, since there is an inherent complexity.

They didn't bury the complexity, they removed it. And I agree, that's hard to do. It'd be nice if the systemd folks put in the same effort to remove the complexity from their system.


As a DevOps, having to know a bunch of Linux and system programming is a job requirement. In the old days Unix system admins were very familiar with this also.


This mess is easy to solve with a library of functions to call, e.g. /etc/rc.d/init.d/functions, which can be imported and used. Unfortunately, there is no standard API for this.


Until service restarts and error recover, log storage sneaks. In the past I just call pm2/forever(or something like that, I write nodejs most) to do the rest. Because roll those yourself... is really a pain.

And a few months ago I retried that with systemd. It's really just about 10 line of configs. And you are done.

Besides that, it also has a build-in scheduler with a command for you to tell when was the task runs, did it success? And what about the outputs. Although you could say it is just a cron replacement with better ui. But why no? I don't really care about the unix philosophy, I just care what do solve the problem for me.


> Service and runlevel management wasn't any better in the sysv era, nor were any of the multitude of custom start and boot scripts.

Things would have been fine for a lot of people if they had stopped at SysV script replacing, and general start up.

At this point, with all the additional functionality continuously being added, I'm waiting for systemd to fulfil Zawinski's Law:

> Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

* http://www.catb.org/jargon/html/Z/Zawinskis-Law.html

* https://en.wikipedia.org/wiki/Jamie_Zawinski


> Systemd is 10 years in making

Systemd is 10 years in making, and still manages to brick production servers.

The problem is not with SystemD or its coding as such, but the ideology it came with, and bad developers who push it.

The last attempts to make it saner basically reverted it back to sysvinit. So, not much difference now.


> The last attempts to make it saner basically reverted it back to sysvinit

Uhm, how so?


How likely is it that the prolific systemd team and tech decision makers in Linux distributions don’t have a design instinct and came up with this ball of mud full of accidental and unneeded complexity? Have you considered that there may be teams and requirements outside your current sphere of experience?


It certainly does sometimes happen that difficult software projects end up being implemented by people who (at least at the start) don't understand the problems involved, for reasons related to the "winner's curse" [1].

That is, people who underestimate the difficulty of a project are more likely to attempt it, and people who don't understand the area well are more likely to underestimate its difficulty.

[1] https://en.wikipedia.org/wiki/Winner%27s_curse


It happens but it’s not clear what you intend to say with that, so maybe just say it? I don’t think the systemd team could have imagined the success and scope of the project from day 1. Another explanation for their success is that the team was onto something, and by using proper engineering practices (work incrementally on pieces that are individually useful) became successful. Think T S Kuhn’s progressive research program.


I'm saying that I don't think reasoning by considering questions like

« How likely is it that the prolific systemd team and tech decision makers in Linux distributions don’t have a design instinct and came up with this ball of mud full of accidental and unneeded complexity? »

is likely to be fruitful.

If people want to discuss whether systemd is well designed, it would be better to look at the design directly.


Agreed that this line of questioning is not likely to be fruitful. The alternative of discussing the design would have my preference normally, but i am not sure that it works any better for this hyper polarized topic.


Maybe there are, but that doesn't help me.

The systemd way seems to be "one tool of complexity 40 to 50 things" rather than "50 tools of complexity 1 each doing one thing".

When you only need 10 things, you only need 10 simple tools of complexity 10, rather than one tool of complexity 40.


That’s not how complexity works. Inherent complexity can’t really be outsourced meaningfully.


For SystemD, I see none.

Though, I see unfulfilled urge to give Linux "serious enterprise grade" twist, and bog everything in "serious enterprise frameworks of doom"


If you think that systemd is an ‘enterprise framework of doom’ then you must not have worked in Java enterprise software development.


Having suffered under both, I'd totally spend the rest of my life developing enterprise Java if it meant systemd went away forever.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: