I can assure you loading times in terms of seconds has everything to do with the app making a network call or some other loading, and not SwiftUI itself.
It's spinning up a remote view for the preference pane, and then SwiftUI is taking a while to render it because the author did a poor job of optimizing it.