Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This post is completely and totally wrong. At least you got to ruin my day, I hope that's a consolation prize for you.

There is NO meaningful connection between the completion vs polling futures model and the epoll vs io-uring IO models. comex's comments regarding this fact are mostly accurate. The polling model that Rust chose is the only approach that has been able to achieve single allocation state machines in Rust. It was 100% the right choice.

After designing async/await, I went on to investigate io-uring and how it would be integrated into Rust's system. I have a whole blog series about it on my website: https://without.boats/tags/io-uring/. I assure you, the problems it present are not related to Rust's polling model AT ALL. They arise from the limits of Rust's borrow system to describe dynamic loans across the syscall boundary (i.e. that it cannot describe this). A completion model would not have made it possible to pass a lifetime-bound reference into the kernel and guarantee no aliasing. But all of them have fine solutions building on work that already exists.

Pin is not a hack any more than Box is. It is the only way to fit the desired ownership expression into the language that already exists, squaring these requirements with other desireable primitives we had already committed to shared ownership pointers, mem::swap, etc. It is simply FUD - frankly, a lie - to say that it will block "noalias," following that link shows Niko and Ralf having a fruitful discussion about how to incorporate self-referential types into our aliasing model. We were aware of this wrinkle before we stabilized Pin, I had conversations with Ralf about it, its just that now that we want to support self-referential types in some cases, we need to do more work to incorporate it into our memory model. None of this is unusual.

And none of this was rushed. Ignoring the long prehistory, a period of 3 and a half years stands between the development of futures 0.1 and the async/await release. The feature went through a grueling public design process that burned out everyone involved, including me. It's not finished yet, but we have an MVP that, contrary to this blog post, does work just fine, in production, at a great many companies you care about. Moreover, getting a usable async/await MVP was absolutely essential to getting Rust the escape velocity to survive the ejection from Mozilla - every other funder of the Rust Foundation finds async/await core to their adoption of Rust, as does every company that is now employing teams to work on Rust.

Async/await was, both technically and strategically, as well executed as possible under the circumstances of Rust when I took on the project in December 2017. I have no regrets about how it turned out.

Everyone who reads Hacker News should understand that the content your consuming is usually from one of these kinds of people: a) dilettantes, who don't have a deep understanding of the technology; b) cranks, who have some axe to grind regarding the technology; c) evangelists, who are here to promote some other technology. The people who actually drive the technologies that shape our industry don't usually have the time and energy to post on these kinds of things, unless they get so angry about how their work is being discussed, as I am here.



Thank you for this post. I have been interested in rust because of matrix, and although I found it a bit more intimidating than go to toy with, I was inclined to try it on a real project over go because it felt like the closest to the hardware while not having the memory risks of C. The co-routines/async was/is the most daunting aspect of Rust, and a post with a sensational title like the grand-parent could have swayed me the other way.

As an aside, It would be great to have some sort of federated cred(meritocratic in some way) in hackernews, instead of a flat democratic populist point system; it would lower the potential eternal September effect.

I would love to see a personal meta-pointing system, it could be on wrapping site: if I downvote a "waste of hackers daytime" article (say a long form article about what is life) in my "daytime" profile, I get a weighted downvoted feed by other users that also downvoted this item--basically using peers that vote like you as a pre-filter. I could have multiple filters, one for quick daytime hacker scan, and one for leisure factoid. One could even meta-meta-vote and give some other hackers' handle a heavier weight...


To hopefully make your day better.

I for one, amongst many people I am sure, am deeply grateful for the work of you and your peers in getting this out!


I second this. Withoutboats has done some incredible work for Rust.


For what it's worth, I agree 100% with the premise of withoutboats' post, based on the experience of having worked a little on Zig's event loop.

My recommendation to people that don't see how ridiculous the original post was, is to write more code and look more into things.


Please, calm down. I do appreciate your work on Rust, but people do make mistakes and I strongly belive that in the long term the async stabilization was one of them. It's debatable whether async was essential or not for Rust, I agree it gave Rust a noticeable boost in popularity, but personally I don't think it was worth the long term cost. I do not intend to change your opinion, but I will keep mine and reserve the right to speak about this opinion publicly.

In this thread [1] we have a more technicall discussion about those models, I suggest to continue that thread.

>I assure you, the problems it present are not related to Rust's polling model AT ALL.

I do not agree about all problems, but my OP was indeed worded somewhat poorly, as I've admitted here [2].

>Pin is not a hack any more than Box is. It is the only way to fit the desired ownership expression into the language that already exists

I can agree that it was the easiest solution, but I strongly disagree about the only one. And frankly it's quite disheartening to hear from a tech leader such absolutistic statements.

>It is simply FUD - frankly, a lie - to say that it will block "noalias,

Where did I say "will"? I think you will agree that it at the very least will it will cause a delay. Also the issue shows that Pin was not proprely thought out, especially in the light of other safety issues it has caused. And as uou can see by other comments, I am not the only one who thinks so.

>the content your consuming is usually from one of these kinds of people:

Please, satisfy my curiosity. To which category do I belong in your opinion?

[1]: https://news.ycombinator.com/item?id=26410359

[2]: https://news.ycombinator.com/item?id=26407565


> Please, calm down.

By the way, that will almost certainly be taken in a bad way. It's never a good idea to start a comment with something like "chill" or "calm down", as it feels incredibly dismissive.

> I do appreciate your work on Rust, but

There's a saying that anything before a "but" is meaningless.

This is not meant to critique the rest of the comment, just point out a couple parts that don't help in defusing the tense situation.


Thank you for the advice. Yes, I should've been more careful in my previous comment.

I have noticed this comment only after engaging with him in https://news.ycombinator.com/item?id=26410565 in which he wrote about me:

> You do not know what you are talking about.

> You are confused.

So my reaction was a bit too harsh partially due to that.


So why did you not present your own solutions to the issues that you criticized or better yet fix it with an RFC rather than declaring a working system as basically a failure (per your title). I think you wouldn't have 10% of the saltiness if you didn't have such an aggressive title to your article.


I have tried to raise those issues in the stabilization issue (I know, quite late in the game), but it simply got shut down by the linked comment with a clear message that further discussion be it in an issue on in a new RFC will be pointless.

Also please note that the article is not mine.


You know the F-35 is a disaster of a government project from looking at it, why not submit a better design? That isn't helpful. You might be interested in the discussion from here: https://news.ycombinator.com/item?id=26407770


Is it just me or you're supporting your parent's point of:

> ...the decision was effectively made on the ground of "we want to ship async support as soon as possible" [1].

When you write:

> Moreover, getting a usable async/await MVP was absolutely essential to getting Rust the escape velocity to survive the ejection from Mozilla...

This whole situation saddens me. I wish Mozilla could have given you guys more breathing room to work on such critical parts. Regardless, thank you for your dedication.


That is not a correct reading of the situation. async/await was not rushed, and does not have flaws that could have been solved with more time. async/await will continue to improve in a backward compatible way, as it already has since it was released in 2019.


Please keep going, Rust is awesome and one of the few language projects trying to push the efficient frontier and not just rolling a new permutation of the trade-off dice.


I've jumped on the Rust bandwagon as part of ZeroTier 2.0 (not rewriting its core, but rewriting some service stuff in Rust and considering the core eventually). I've used a bit of async and while it's not as easy as Go (nothing is!) it's pretty damn ingenious for language-native async in a systems programming language.

I personally would have just chickened out on language native async in Rust and told people to roll their own async with promise patterns or something.

Ownership semantics are hairy in Rust and require some forethought, but that's also true in C and C++ and in those languages if you get it wrong there you just blow your foot off. Rust instead tells you that the footgun is dangerously close to going off and more or less prohibits you from doing really dangerous things.

My opinion on Rust async is that it its warts are as much the fault of libraries as they are of the language itself. Async libraries are overly clever, falling into the trap of favoring code brevity over code clarity. I would rather have them force me to write just a little more boilerplate but have a clearer idea of what's going on than to rely on magic voodoo closure tricks like:

https://github.com/hyperium/hyper/issues/2446

Compare that (which was the result of hours of hacking) to their example:

https://hyper.rs/guides/server/hello-world/

WUT? I'm still not totally 100% sure why mine works and theirs works, and I don't blame Rust. I'd rather have seen this interface (in hyper) implemented with traits and interfaces. Yes it would force me to write something like a "factory," but I would have spent 30 minutes doing that instead of three hours figuring out how the fuck make_service_fn() and service_fn() are supposed to be used and how to get a f'ing Arc<> in there. It would also result in code that someone else could load up and easily understand what the hell it was doing without a half page of comments.

The rest of the Rust code in ZT 2.0 is much clearer than this. It only gets ugly when I have to interface with hyper. Tokio itself is even a lot better.

Oh, and Arc<> gets around a lot of issues in Rust. It's not as zero-cost as Rc<> and Box<> and friends but the cost is really low. While async workers are not threads, it can make things easier to treat them that way and use Arc<> with them (as long as you avoid cyclic structures). So if async ownership is really giving you headaches try chickening out and using Arc<>. It costs very very little CPU/RAM and if it saves you hours of coding it's worth it.

Oh, and to remind people: this is a systems language designed to replace C/C++, not a higher level language, and I don't expect it to ever be as simple and productive as Go or as YOLO as JavaScript. I love Go too but it's not a systems language and it imposes costs and constraints that are really problematic when trying to write (in my case) a network virtualization service that's shooting (in v2.0) for tens of gigabits performance on big machines.


I skimmed some of this, but are you asking why you need to clone in the closure? Because "async closures" don't exist at the moment, the closest you can get is a closure that returns a future, this usually has the form:

   <F, Fut> where F: Fn() -> Fut, Fut: Future
i.e. you call some closure f that returns a future that you can then await on. when writing that out it will look like:

   || {
    // closure
       async move {
          // returned future
       }
   }
`make_service_fn` likely takes something like this and puts it in a struct, then for every request it will call the closure to create the future to process the request. (edit: and indeed it does, it's definition literally takes your closure and uses it to implement the Service trait, which you are free to do also if you didn't want to write it this way https://docs.rs/hyper/0.14.4/src/hyper/service/make.rs.html#...)

The reason you need to clone in the closure is that is what 'closes over' the scope and is able to capture the Arc reference you need to pass to your future. Whenever make_service_fn uses the closure you pass to it, it will call the closure, which can create your Arc references, then create a future with those references "moved" in.

It's a little deceptive as this means the exact same thing as above, just with the first set of curly braces not needed

   || async move {}
This is still a closure which returns a Future. Does all of that make sense? Perhaps they could use a more explicit example, but it also helps to carefully read the type signature.


Wait so you're saying "|| async move {}" is equivalent to "|| move { async move {} }"? If so then mystery solved, but that is not obvious at all and should be documented somewhere more clearly.

In that case all I'm doing vs. their example is explicitly writing the function that returns the promise instead of letting it be "inferred?"


Well, no, that second one isn't valid rust, perhaps you mean:

   move || async move {} 
But this is not equivalent to:

  || async move {}
crucially the closure is not going to take ownership of anything. This is kind of besides the point though, what I'm getting at is that both of the above are a closure which returns a future. i.e. you can also write them in this style:

    || {
       return async move {};
    }
Maybe that's more clear with the explicit return?

I don't understand your second question about it begin "inferred", I never used that word. make_service_fn is a convenience function for implementing the Service trait.


Ohhh.... I think I get it. The root of my confusion is that BRACES ARE OPTIONAL in Rust closures.

This is apparently valid Rust:

let func = || println!("foo!");

I didn't know that, which is why I thought "|| async move ..." was some weird form of pseudo-async-closure instead of what it is: a function that returns an async function.

Most of the code I see always uses braces in closures for clarity, but I now see that a lot of async code does not.


> I didn't know that, which is why I thought "|| async move ..." was some weird form of pseudo-async-closure instead of what it is: a function that returns an async function.

It does not return an async function, it is a closure that returns a future. Carefully read the function signature I had posted:

    fn foo<F, Fut>(f: F) where F: Fn() -> Fut, Fut: Future
async move {} is just a future, there is no function call. || is a closure, put them both together and you have a closure that returns a future.

edit: I'm trying to think of how else to explain this. a future is just a state machine, an expression, there is no function call.

   let f = async move { };
Is a valid future, you can f.await just fine.


You are awesome. Thank you for clarifying these things.


Thank you for your tremendous work!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: