Even 68k-based systems running in single digit megahertz could run full featured terminal emulation and have a lot of other stuff going too. There's legacy stuff in terminals, but compared to all the other stuff you've got going (Wayland, GTK, frickin' browser engine) it isn't bloated.
How is graphical vim even different from TUI vim? At least Emacs can render images.