Since people will bring up AccessKit, assuming it to be a potential panacea (… despite the obvious performance cost of doing absolutely everything twice):
The pure canvas approach used by things like egui (and it probably can’t change it) and current iced (it used to have iced_web which rendered to DOM, but that’s been abandoned for now for no obvious reason) is fundamentally and irredeemably unsuitable for general-purpose web content and apps, in ways that things like AccessKit cannot fix. I’ve written about this a few times; https://news.ycombinator.com/item?id=33861831 is probably the best thread. The two items I like to focus on are links and scrolling, both of which are unavoidably broken.
The performance point in your linked post is worth highlighting: I see a lot of developers complaining "if the web had an accessibility API for this, then it wouldn't be a problem."
The web does have an accessibility API. It's called the DOM.
What developers are upset about is that they have to make their apps performant in that accessibility layer. The DOM restricts you from developing a fast app for sighted users and a much laggier less-featured app for users using the accessibility API, because it forces you to always have the accessibility API turned on. It forces you to treat the accessibility API as your default rendering target (and allows you to occasionally make something inaccessible by embedding a canvas element inside of that UI for one or two parts of your app.
And if you can't develop a fast app using the accessibility API, then having a toggle to turn that layer off is not really a solution for that problem. I'm very glad AccessKit exists, it's better than nothing, but that doesn't mean the apps that are using it should be proud of themselves.
The crux of the web’s accessibility performance problem (minor in general, but major once you try to add accessibility to a canvas) is that it requires that everything be materialised at once and ahead of time, regardless of whether it’s being used. By contrast, the browser itself is able to lazily and (on most or all platforms, I believe) partially construct the accessibility tree. (mwcampbell can correct me if I’ve misunderstood how various platforms work.)
Browsers seem to have resisted adding fundamental alternatives of construct-on-query accessibility trees, or some kind of “I need AT stuff” switch so you can skip setting ARIA attributes or materialising other DOM if it’s not going to be used.
The end result is that the web doesn’t properly support accessible virtualised scrolling (where you have a hundred thousand emails in a mailbox, but only render the ones on screen, still providing a meaningful scrollbar) or infinite scrolling. There’s role=feed which improves matters in most screen reader configurations, but it’s still making some compromises. But then, virtualised scrolling is imperfect even for non-AT use—things like browser find-in-page won’t work properly. Still more of the “if you want it to work properly, it has to exist in the DOM all the time” stuff.
You're completely right about platform-native accessibility APIs.
To be honest, AccessKit itself doesn't support lazy accessibility trees for virtualized scrolling, at least not yet. It can avoid constructing the accessibility tree completely if the platform accessibility API isn't queried at all, but there isn't yet a way to only partially construct an accessibility tree while indicating that there's more available on demand. So to some extent, I guess AccessKit brings this web performance problem to native.
Okay, that is a fair point on virtualized scrolling, and while I can somewhat point to reasons why that's the case they're not strictly related to accessibility. They're more an API/extension concern than an accessibility concern, I guess.
That being said... I have also kind of soured on virtualized scrolling over time as I've spent more time working on UX projects, and I realize that sort of sounds like an excuse (we don't support this well, but you shouldn't be doing it anyway), but... I do kind of feel like infinite scrolling and giant lists of elements to the point where the list needs to be virtualized is bad UX for most apps.
----
And to be clear, I'm not necessarily saying that the DOM is perfect as an accessibility tree, just that it's good enough compared to the kinds of accessibility APIs that most native devs want when they complain about the DOM. In practice, a full-featured accessibility API with the same feature-set as the DOM and the same level of capabilities would likely have performance costs no matter how it was implemented in the browser.
Sure it could be better. But also all of those improvements could just be added to the DOM instead of making a new native web API. The biggest difference between adding those improvements to the DOM vs reconstructing something very similar to the DOM as a separate browser API is that devs could ignore the performance costs of that API and only benchmark their apps with the accessibility API turned off.
And I think it's good for devs not to have that option (or at least, for browsers themselves not to directly provide that option; again I'm not saying that I don't think AccessKit should exist).
Lead developer of AccessKit here. I know it won't be a panacea, particularly for the non-screen-reader-related problems you point out. And about the performance problem, there's probably no getting away from the despised "enable accessibility" button. But if these types of applications are going to exist despite our wishes to the contrary, then I have to do what I can to make them accessible. It could be the difference between someone being able to do a job, complete a course, or whatever, and not. I have to do what I can to improve the accessibility of the world as it is, not the world as we wish it was.
Good intentions, but this leads people to create even more apps that they then proudly proclaim are just as accessible as alternatives because of AccessKit. It's a bit of a catch-22.
It's far better that AccessKit exist than that it not.
That being said, yeah, there are people in this very comment section saying that of course egui is accessible, it supports AccessKit. So it's not really theoretical, any attempt to close the gap with AccessKit is going to be used by people as an excuse to build inaccessible applications. That's already happening in this comment section right now.
But that doesn't mean that AccessKit shouldn't exist it's just... one of the downsides to reducing the number of entirely inaccessible apps. It's a partial fix for a bad situation, and it's good for devs to use it if they have to, but it's far better for devs not to need to use it.
I think many people, especially SV devs whose job is finding solutions to problems, forget that there are no solutions to some problems. That only solution to these problems is to not pursue what causes the problem altogether.
Yes, it would be better if web applications that use canvas for their whole UI didn't exist. But they do, and I'm sure there will be more of them. I won't be able to persuade developers to stop developing apps this way. So I have to do what I can to help make them accessible, however imperfectly.
Right; to be clear, my issue is not with AccessKit or really any of the work you're doing here. I'm really happy that AccessKit exists, I'm glad that you're working on it.
AccessKit is trying to help fix a problem as best it can. My issue is with the people and frameworks that are causing the problem.
I think that's less bad than someone losing their job, or failing to get a particular job, because they can't access a canvas-based app. Edit: The former isn't entirely a hypothetical; I know of a blind person who lost his job in 2006 because use of an inaccessible app (a Java applet running on the Microsoft JVM, IIUC) became a requirement. I try to keep that bigger picture in mind with everything I do in AccessKit.
I just wish that the people that make canvas-based web stuff would realise how awful it is and tell people. I’ve interacted with a Flutter web thing once, for something that was only targeting the web. It was very painful. Flutter was completely the wrong tool for the job. But when they advertise their stuff, do they say “look, it kinda supports the web, but if you’re actually trying to target the web, please don’t use this because it’s just not good, and can’t be good unless we rip out the canvas stuff and render to real DOM elements”? Almost never.
this take is a little bit too simplistic. There are use-cases where you are bound to a web-based framework but JS solutions are just not fast enough, e.g. data visualisations via jupyter notebooks. So you're stuck with some horribly slow solutions or non-interactive ones giving you pngs. Since you're plotting data, the text is minimal and the needed dialogs etc. are also minimal (clicking on a point and getting a label) or having a few options to rotate/turn on some features/zoom. And pngs with rendered text is as inaccessible as it gets, although data-visualisation may be fundamentally incompatible with accessibility tools since it is all about seeing stuff.
So I usually produce pngs via matplotlib and only selectively use interactive, js based, visualisations but they are get so unbelievable slow quickly when that your whole tab freezes.
I believe you’ve misunderstood my intent. When I speak of the pure canvas approach, I mean throwing away almost all that the browser gives you, and rendering everything in a single canvas, with fake scrolling, fake links, &c.
For individual widgets like data visualisation, go ahead, use canvas if you want; I won’t complain—though make sure if you have any links in it that you have actual <a> links on top of the canvas for the user to interact with. And also consider if you can SVG. Anyway, individual canvases are fine; but don’t put everything in a single canvas.
Things like Google Docs and Figma are often cited as examples of how canvas-based rendering can be not bad, but neither are pure canvas: the user interface of both is full DOM, and it’s only the document area that’s canvas. (Also, I find Google Docs somewhat unpleasant: its rendering latency and throughput while you type is terrible, and keyboard caret navigation doesn’t match my platform’s behaviour in many places.)
The pure canvas approach used by things like egui (and it probably can’t change it) and current iced (it used to have iced_web which rendered to DOM, but that’s been abandoned for now for no obvious reason) is fundamentally and irredeemably unsuitable for general-purpose web content and apps, in ways that things like AccessKit cannot fix. I’ve written about this a few times; https://news.ycombinator.com/item?id=33861831 is probably the best thread. The two items I like to focus on are links and scrolling, both of which are unavoidably broken.