Try replacing "acting unexpectedly" in your thought process (which superficially I agree with) with the words "acting safely."
It remains to be seen if autonomous driving systems are actually safe. But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that.
It remains to be seen if autonomous driving systems are actually safe.
What a strange, leading expression. The phrase "[i]t remains to be seen if" is almost meaningless and highly editorial.
Alternative example:
It remains to be seen if microplastics will cause higher rates of cancer.
It remains to be seen if organic peanut butter will cause lower rates of cancer.
What is your reaction to this data published by Waymo? Do you agree that many would reasonably conclude that this particular AV system was safe in these cities? I say yes. I write this as someone who is cautiously optimistic about AV. Waymo seems genuinely lower on the hyper compared to other AV companies. I hope they continue this path of higher transparency to encourage other AV companies to do the same.
>But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that.
I don't disagree, but in order for the first driver to "account" for the actions of the second, they have to have some reasonable ability to predict what that driver will do. That gets us back to the theory-of-mind question.
Are we still talking about a car getting rear ended because it braked? Because you’re meant to leave enough room between you and the car in front to stop safely even if it unexpectedly brakes as hard as possible. Running into the back of a car in front of you (that didn’t just pull out) is always your fault.
I think people are often missing point. Yes, in a rear end collision the fault is almost always the following driver. Having a framework for assigning liability is not the same as having a safety framework. Consider an AV that is consistently brake-checking those behind them due to nuisance alarms. Now I have a harder time predicting what the car in front of me is going to do. Is that a safer or less-safe scenario? Sure, I can mitigate it by giving more trailing distance, but now we've traded traffic flow/congestion for an equal level of safety.
To add onto the sibling's point, the "safe following distance" has a rule of thumb of "3 seconds behind". At 65MPH, I'm assuming you're in the US, is approximately 300 feet.
I'm willing to bet that's around 10 times what you were considering as a safe following distance in your head, and probably still 5 times more than what you were picturing for the safe distance behind a brake checking AV.
The problem with the “safe distance” (my new car has it built into the cruise control) is that it’s more than large enough for a vehicle to merge into, which repeatedly happens.
As more and more cars default to this safely, however, it’ll start to equal out.
> now we've traded traffic flow/congestion for an equal level of safety.
If you don't maintain a safe following distance for your speed, you are the one creating the dangerous driving environment. Tailgating is worse for both traffic flow and safety.
This, again, gets to missing the point. If a disproportionate amount of cars are nuisance brake checking, it increases the level of uncertainty in driving behavior. I now have to overcompensate on average to maintain the same level of safety.
> I now have to overcompensate on average to maintain the same level of safety.
Tailgating is bad, regardless of if people brake check or not. If automous vehicles are what it takes to get you to stop tailgating and follow at a safe distance, then that is just an added bonus.
There is no single definition of tailgaiting other than being able to not being able to stop at a reasonable distance. So it is impossible to declare what constitutes tailing, especially in the mixed case of human drivers and robot drivers (who have a reputation for nuisance braking).
Why, for example, do you think trainers post "Student Driver" stickers on their cars? It's because it signals the driver may be more unpredictable and people (rightly) tend to give them wide berth. You're essentially advocating that everyone treat everyone else (and every robot) as a student driver. That's fine for a dichotomous safety mindset, but other people would prefer to recognize the tradeoffs with that approach.
Or maybe you're just deliberately bent on misunderstanding my point, I can't read your mind :)
> There is no single definition of tailgaiting other than being able to not being able to stop at a reasonable distance. So it is impossible to declare what constitutes tailing, especially in the mixed case of human drivers and robot drivers (who have a reputation for nuisance braking).
What is this nonsense? The safe following distance is determined by how fast you can stop, not by who is driving the vehicle you are following.
> Why, for example, do you think trainers post "Student Driver" stickers on their cars? It's because it signals the driver may be more unpredictable.
No, it signals they have less experience and are more dangerous drivers. When it comes to driver predictability, student drivers tend to be far more predictable than the adult, overconfident drivers. I've never seen a student driver roaring past me in stopped traffic on a shoulder, or floor the gas to pass me through a light because they didn't want to turn in a turn only lane, or any of the other unpredictable things I see on a regular basis from experienced drivers.
> You're essentially advocating that everyone treat everyone else (and every robot) as a student driver. That's fine for a dichotomous safety mindset, but other people would prefer to recognize the tradeoffs with that approach.
I think that if more driver treated the people around them as student drivers, our roads would be a lot safer.
I know that if people followed at a safe distance then we would have fewer traffic jams.
Edit: You also seem stuck on the idea that Waymo unsafely unexpectedly brakes more often than human drivers, yet that isn't clear to me from the data we have. Indeed it seems like the opposite is true from the data.
>The safe following distance is determined by how fast you can stop, not by who is driving the vehicle you are following.
So when you're driving, do you somehow know the braking distance of every car and reaction time of every driver around you? You don't, and since their braking distance is needed to know your own braking requirements, you have to use heuristics. Maybe your heuristic is "assume everyone will cram on the brake, full tilt, at any time." But, that is not a pragmatic solution given our current infrastructure. We don't have the road capacity for everyone to drive that way. So we make tradeoffs. Part of that tradeoff means anticipating what other drivers will do and adjusting accordingly. Naturally, this will trade some safety for other things we value. That is the reality of the world we live in. You seem to be advocating for something else. The OP was that we might struggle to apply such heuristics without a theory of mind to guide us.
We probably just disagree on the student driver vs. overconfident drivers. I feel like I'm pretty good at anticipating aggressive drivers, and I fear them must less than the super-tentative driver that tends to put other people at risk. But unless you have data, we're just talking about subjective opinion here so it's not really worth delving into further.
>I think that if more driver treated the people around them as student drivers, our roads would be a lot safer.
Sure. But again, it doesn't really fit with the world we live in. Should we all, in general, drive more defensively? Sure. But I doubt our infrastructure will allow for 25+ car lengths between vehicles that the NHTSA recommends, so we're stuck making some tradeoffs.
I agree on the data point. I'm not making strong claims about safety. I'm making claims about uncertainty. One thing that is clear (and I've advocated elsewhere) is that we don't have good data (in part, because companies get to share only what they want in many cases), which makes uncertainty greater.
> We don't have the road capacity for everyone to drive that way. So we make tradeoffs. Part of that tradeoff means anticipating what other drivers will do and adjusting accordingly. Naturally, this will trade some safety for other things we value.
You keep making the same argument over and over even though I've repeatedly explained that it does not match our understanding of how traffic jams form. Traffic jams are caused by braking, especially hard braking. Tailgating increases the need to brake hard if the person in front of you brakes or someone needs to merge. Following closely does not increase throughout, it is simply bad driving with no upside.
> I'm pretty good at anticipating aggressive drivers, and I fear them must less than the super-tentative driver that tends to put other people at risk.
Wow, if you read the behavior I descrived as "agressive driving" then we have completely different standards. I was describing reckless driving that blatantly violates the rules of the road.
Stop using excuses for your bad behavior and you might even be able to become a better driver.
What behavior of my own have I stated? Or are you inferring unwarranted conclusions to make ad hominem arguements?
>Traffic jams are caused by braking
Guess what one of the main issues has been in AV...nuisance braking. So much so that they suppress safety-critical actions to avoid unwarranted braking.
I also think you're confusing the proximate causes of conjestion for the root causes. Traffic conjestion is a load sharing problem. Tailgating is, in part, a symptom of inadequate capacity. You are advocating a solution that exacerbates it by reducing carrying capacity.
To reiterate (yet again) we aren't in disagreement about whether slowing down, or increasing following distance, will increase safety. It will, but also, that's not the point I was making. I'm just saying that is a superficial understanding of the problem and you aren't accounting for the tradeoffs. Those tradeoffs are the reason your proposal misses its mark.
> Tailgating is, in part, a symptom of inadequate capacity. You are advocating a solution that exacerbates it by reducing carrying capacity.
Why are we still debating something that has been answered by science? Stop assuming your intuitions are correct and look at what the science actually says about how follow distances affect the maximum capacity of a road.
Has it been answered by science? Because some of the studies I've come across list "heavy traffic" as a cause of tailgating. I think you might be speaking with unwarranted confidence.
25+ car lengths between vehicles that the NHTSA recommends
Is this hyperbole? I tried to Google for it, but I could not. The best I could find was the three second rule. Some back-of-the-envelope calcs: Length of Honda Accord (according to Google): 192 inches. 75 miles per hour / 60 minutes per hour / 60 seconds per minute * 5280 feet per mile * 12 inches per foot * 3 seconds / 192 inches = 20.625 car lengths. Even three seconds isn't enough to reach your recommendation of 25+ car lengths.
The actual calculation involves how fast the vehicle you’re following can decelerate, and your reaction time.
You can (usually) follow a large semi a bit closer because its braking distance is longer than yours.
But because of reduced visibility you can end up with “revealed brake checkmate” where the semi swerves into the next lane because a vehicle is stopped in the lane, which you then need to swerve or hit.
That's ridiculous. There's nothing "unrealistic" about 4.5 seconds of stopping distance - how could there be, are you thinking that the highways physically wouldn't fit all the cars spaced at that length?
It's unrealistic if you expect the existing infrastructure to meet both that requirement, the capacity requirements, and the time requirements of people travelling. In the real-world, there are tradeoffs. People who don't understand that tend to have superficial understandings of the problem because their mental models show little nuance.
I’ve entirely lost any semblance of a point you might have had initially. You’re doubling down on a weak stance making hyperbolic claims like “our infrastructure can’t handle cars leaving a safe following distance”. What nonsense!
It’s more effective to let your initial point stand and let the discussion run its course. You’re working against yourself now.
The claim seems reasonable. Most of the time infrastructure is fine, but that 20% of the time between 4pm and 6pm when you are actually using it, it often is not.
More following distance => less throughput. There is already a math problem at play for how to get millions of single occupancy to and from a given location in a day. Which is to say, gridlock occurs today - it is a fact. Increasing following distance by 2 to 5x would potentially be quite bad for otherwise heavy but flowing traffic. Eg: NYC, it is known for good throughput. Light turns green, everyone picks up speed and keeps tight following distances. Anything less and that city would lock up more so than it already does.
Part of the issue is just complexity and variety. 2x following distances on most highways would likely be only a good thing. But in each and every circumstance, not necessarily good, perhaps disaster.
There is pretty well established modeling that demonstrates that the stop and go rhythm that arises from tight following distances leads to worse throughput. To avoid slinking, everyone needs to accelerate and decelerate at the same rate. A safe stopping distance is the only way build in the micro buffers needed to achieve relatively consistent speed at human scale, or without large scale traffic orchestration. The naïve takes just sound like armchair hand waving. I guess that’s what I’m responding to.
It does seem on the road that many people lose the ability to predict 30 seconds in the future. That car in front of you has 10 feet to move? GO GO GO! Wait, it only moved just 10 feet - BRAKE BRAKE BRAKE!
I think it can fun to be the driver that smooths out the shock waves, so I do largely agree with your point. Though, all things being equal, double following distance and that will double the length of a traffic jam. Longer traffic jams are more impactful - it takes longer to get through them & the longer traffic jam distance is more likely to cause back-ups on on-ramps and in turn to the surface streets leading to those on-ramps. Which is to say, a higher prevalence of grid-lock.
The AI crowd is right to say that with 100% AI cars, you can have smooth and fast traffic with impossibly tight following distances. I do believe you need some impractical necessities to accomplish that though.
You seem very combative and seemingly deliberately missed the connections in other sub-threads, so why don’t you tell me what point you think I’m making and I can tell you how it’s accurate.
I’m not combatting anything… I am just offering my perspective that I think your OP is stronger on its own. The sub-thread digressions betray your unfamiliarity with the topic and your narcissistic nature rather than reinforce your OP. Take it or leave it, it’s just an internet opinion.
If you can't relay someone's point as you understand it, there's little reason to weigh in, and even less for that person to try and clarify it. It comes across as a need to argue, rather than a desire to understand.
But you aren't overcompensating. Instead you are driving safely. If sudden braking is so rare that you feel comfortable riding right behind a car in front of you then when they do suddenly brake (which will happen eventually), you are now in a very dangerous situation.
If it were doing it consistently then that would be less of an issue. The problem is that complex systems make decisions for complex reasons and are very difficult to predict as a result.
That said, if you routinely tailgate the driver in front on the assumption that nothing will go wrong then you've chosen to accept the consequences when an otter runs out in front of them (hey I didn't see that coming either) and they suddenly limit brake and you're sitting in their back seat. Or a steering tie breaks in a classic car coming the other way, which hits the car in front of you head on and now they're stationary.
The question of how much additional caution (in terms of lower speed limits, longer following distances etc.) is optimal in terms of overall QALYs is, I feel, vastly under-considered and under-discussed.
None of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do. Your objection becomes essentially hypothetical unless these cars are behaving in ways that are both outside of the norms established by the driving public, and dangerous.
That’s true, but also one of the selling points of some AI tasks. As a non hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many “best practices” it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road.
Modern AVs are not driving like they have a death wish by any stretch of the imagination (and these systems are not developed using raw RL). They are driving safer than humans. Any concern that they are not following established best practices is entirely unfounded and strictly grounded in FUD.
I don’t think you can make this strong of a claim with the available data. Likewise, someone can’t make strong claim in the opposite direction. The best data I’ve seen is from NTSB investigations, and it clearly shows some very dangerous behavior. But it’s just a snapshot of data.
And I think you’ve taken away the wrong point. It was about unexpected behavior, not “driving with a death wish.”
The ability of people to consistently miss/twist the point to fit their own predetermined viewpoint is tiresome.
> And I think you’ve taken away the wrong point. It was about unexpected behavior, not “driving with a death wish.”
Indeed - so how did systems with a "death wish" enter into this discussion? Well, it turns out it was by you, about which you said "...maybe not so good on a public road."
In this light, your complaint about others twisting the point seems rather ironic.
>so how did systems with a "death wish" enter into this discussion?
It was the way the pilots described the unexpected behavior of the RL model.
If I related a story about ChatGPT where someone said, "It wrote like it was drunk" would you insist that I'm saying ChatGPT actually imbibes in alcohol before coming up with a response? I think you might be missing the point of an analogy. This tends to correlate with people with dichotomous thinking, which is also part of the thread and the difficulty with people understanding the intended point.
The part you missed was that the "maybe not so good on a public road" was about breaking "best practices," not about acting with a death wish. The intent was to underscore, yet again, was that unexpected behavior is sometimes beneficial in a wartime environment where you want to keep the other party guessing and off-balance, but not beneficial in a public safety domain. Again, twisting an argument to make a preconceived point rather than reading it as it was actually written. It's hard to get someone to understand a point when their biases are hell-bent on deliberately not understanding it.
> It was the way the pilots described the unexpected behavior of the RL model.
And you chose to quote it, and to make the point that it is not the sort of thing we want on public roads - as if there's the slightest hint in the extensive testing of these cars that the apocalyptic scenario you chose to introduce to this discussion was anything but hypothetical and hyperbolic FUD.
And then you had the effrontery to chide dcow for responding to your claim, as if they were taking the discussion in this absurd direction. You've chosen to repeat yourself here, so I will say again that there is considerable irony in what you are doing - irony in the sense of statements being made that display a lack of self-awareness.
> > It was the way the pilots described the unexpected behavior of the RL model.
> And you chose to quote it, and to make the point that it is not the sort of thing we want on public roads - as if there's the slightest hint in the extensive testing of these cars that the apocalyptic scenario you chose to introduce to this discussion was anything but hypothetical and hyperbolic FUD.
At some point there were numerous reports of cars using driving assist steering into the dividers at highway offramps. I believe there were in fact some some real accidents that happened this way. That definitely was unexpected behaviour and I would argue one could even qualify it as the car acting like it had a death wish so I don't think the OP statements can be qualified as hyperbolic FUD.
Now you're going to argue that these were early incarnations of non AV systems (despite the name), but I think they do illustrate how the systems can behave in unpredictable (and dangerous) ways when they encounter novel situations. That's why I commend waymo for not following the hype and keeping the environment they operate very restricted.
> Now you're going to argue that these were early incarnations...
Well, yes I am, because, as you realize, it is a valid point! I will also add that, I have, myself, made the point that we have to be careful because complex systems have bizarre failure modes, so just because, in everyday circumstances, these systems may appear to function 'sensibly', we cannot simply trust them to be 'sensible' outside of the scope of testing.
I am, however, a believer in the relevance of empirical evidence, and I also agree (with some caveats) that human-driver performance is a valid basis for establishing whether, and to what extent, autonomous vehicles may be permitted on public roads. We are discussing the publication of some results of Waymo's extensive testing, and I stand by what I wrote in my first response here: none of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do.
I agree with you that Waymo is taking the right approach here (and, FWIW, I regard Tesla's two-faced stance as unethical.)
I doubt I would have objected to Bumby's invocation of a death wish in response to the past events of which you speak, but in response to Waymo's test results and the points I made about them, I think 'hyperbolic FUD' is justified. At some point, a rational person has to make accommodations when things have changed and arguments based on old data lose their relevance.
>I am, however, a believer in the relevance of empirical evidence, and I also agree (with some caveats) that human-driver performance is a valid basis for establishing whether, and to what extent, autonomous vehicles may be permitted on public roads.
This bypasses the entirety of the trust argument originally stated by leveraging the very point (statistics) it cautioned against. (As stated elsewhere: "So all the bleating about statistics may be necessary, but not sufficient, to get wide-scale adoption of AVs on public roadways.")
>none of these three cases
Regardless if you think the decision should be based on statistics alone, my further point is that an n=3 sample size is not adequate to make strong claims. Add to it that Waymo only reports the data they want to[1], there is a reason to be careful about making any claims.
Here's how that scenario plays out when we rely on organizations to self-report safety data in a competitive environment, from my experience. Even though it may start with the best of intentions, cost and schedule pressure builds. Now items that would have otherwise been reported as safety incidents are now classified with vague terms like "test anomalies" and essentially buried. They will still report safety metrics, but now it's, at best, incomplete and misleading. Until some event that's egregious enough forces the company to be more transparent.
It is absurd of you to suppose that Waymo's testing has yielded just three data points on the safety of its cars... Just suppose there had been no incidents incidents at all - then they would clearly have to be banned from the road, as we would have absolutely no data pertaining to their safety!
Again, missed the point. We shouldn't make strong claims about self-curated data when there's an incentive to make that data look safer than it actually was.
We know they aren't fully transparent. We also know that other well-known and well-funded AV developers have very bad practices that are highlighted when they are forced by regulators to be transparent. While that isn't a smoking gun against Waymo, it should give us pause and make one question a naive perspective in favor of a skeptical one.
There's a subtle nuance you're missing. I am not saying the data is biased, I'm saying we have good reason to believe the data may be biased. I have been careful not to say any strong claims about Waymo here, because my stance is we probably don't have enough data to make such claims. It's a small, but crucial difference. We would need more data (ie a larger sample size) to make a strong bias claim.
Given the protracted nature of this thread, I get why it's confusing. I have a couple sub-threads that make those two points separately, and was simply trying to show how both are related. If one isn't aware of the broader context of the discussion I was making, I understand why it may seem like goalpost moving. But in reality, I was deliberately trying to tie the two related posts together because (IMO at least) they are related.
If your absurd claim that Waymo's trials provide just three relevant data points is not part of "the point", then why did you make it? It does not give us any confidence in the proposition that "the point" has been well thought-out.
Furthermore, "the point" keeps shifting: recently it shifted to raising doubts about the provenance of the data whipped up from a six-year-old article. At this point, I feel that a quote is appropriate: "the ability of people to consistently miss/twist the point to fit their own predetermined viewpoint is tiresome."
As I've said elsewhere, my point was part of a larger context. My point was about how important trust is to adoption of AV tech. That goes well beyond the Waymo cases illustrated. The sample size and quality of the data illustrates the need for a broader context of information needed, in addition to the need to understand that humans don't build trust simply from statistical arguments.
And in the vein of trying to steel-man your position, I gave the comment to ChatGPT to see if it, too, considered the central point a claim about AV having a "death wish." Here's what it said:
"The statement highlights that some AI tasks, while effective, may deviate from conventional practices. An example is given where the Department of Defense (DoD) employed a company to train a dogfighting simulator using reinforcement learning (RL). Pilots were surprised by the simulator breaking established best practices and behaving recklessly, akin to a pilot with a disregard for safety. The implication is that while such behavior might be advantageous in a military context, it may pose risks or be unsuitable in civilian settings, such as public roads. The statement underscores the need to carefully consider and tailor AI applications to specific contexts and objectives."
So it seemed to recognize that the central point is that "the behavior" in question is "breaking established best practices" and that the "implication is that while such behavior might be advantageous in a military context, it may pose risks or be unsuitable in civilian settings". There's probably some irony in the fact that AI did better at a reading task.
Sounds like you’re arguing that “AI” is better at navigating complex human discussion than the multiple humans in this thread? I’ll take that conclusion, I guess (whether you ultimately believe yourself or someone else arguing against your own points doesn't really matter, does it).
Really the only thing left is for you to take a flight to SF and watch the Waymo cars drive. Or ride in one if you dare.
Impressive as LLMs are, they lack a theory of mind and are inferior to humans in parsing meaning from statements.
This dispute is not over a difficult or subtle issue: all the people who have responded in this thread see clearly the obvious and unequivocal reading - and, in your own example, ChatGPT also does! It has identified the tacit subject of the sentence "Possibly good in war, maybe not so good on a public road" as the specific military system, with behavior explicitly described as like having a death wish, that is the only topic of the preceding sentence. For one thing, the phrase 'possibly good in war' makes no sense in the reading you are trying to pass off: why, out of nowhere, did the needs of the military appear? And unexpected behavior is not something desirable in general in military systems, any more than elsewhere - it would take very special circumstances and a specific sort of behavior for that proposition to even be entertained. We can see, therefore, that dcow was right to respond 'your comparison is RL dogfighting?...' and 'Modern AVs are not driving like they have a death wish by any stretch of the imagination...'
Oh, and next time you invoke ChatGPT's response, include the prompt, verbatim, like this for example:
Prompt: In the statement 'That’s true, but also one of the selling points of some AI tasks. As a non-hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many "best practices" it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road.', what is being called ' not so good on a public road'?
Response: In the statement, the phrase "not so good on a public road" refers to the behavior of the software dogfighting simulator trained with reinforcement learning (RL). The implication is that the simulator, which exhibited behavior contrary to conventional "best practices" and behaved like a pilot with a "death wish," might not be suitable or safe for use in a public road scenario. This suggests a concern about the potentially risky or unpredictable behavior of the AI system in a real-world, civilian setting such as driving on public roads.
What we have in this discussion is a motte-and-bailey fallacy, as we can see in your response to my first post here, which was:
None of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do. Your objection becomes essentially hypothetical unless these cars are behaving in ways that are both outside of the norms established by the driving public, and dangerous." Your reply, in outline, goes like this:
> That's true...
Here we are in the motte, where you nominally accept that the relevance of your concern, which is not unreasonable in itself, is constrained by the extensive testing that has been performed by Waymo so far...
> ...but...
Here we enter The bailey, where we are supposed to turn our attention to an unrelated system, which was found, on testing, to have alarming unexpected behavior. The bailey has become a place where Waymo has been curating the data to the point where we simply don’t have the slightest idea whether there’s dangerous behavior outside of the human norms lurking in Waymo cars.
It has also become a place where all of Waymo’s extensive testing has produced just three data points against this view. I must say that it seems generous of you to concede even three, if Waymo is curating the data to the extent you imply.
>It has also become a place where all of Waymo’s extensive testing has produced just three data points against this view.
This is where you are bypassing the point about how much faith we can put in the data, because we don't have the full results of their extensive testing. We only have the results they are willing to share.
As an analogy, I have a relative who loves to talk about all the times he's won money gambling on the craps table. I almost never hear any information about his losses, unless they are couched to say how much more money he's won later. So would you say I can conclude he's an expert gambler who should quit his job to play craps full time, or do you think there might be some human bias in reporting going on that I should be skeptical about?
So you want to bring up your "three data points" claim again? OK. Those three data points are the three incidents listed in the top post of this thread [1]. About them, you wrote "regardless if you think the decision should be based on statistics alone, my further point is that an n=3 sample size is not adequate to make strong claims." [2] I don't just "bypass" this argument, I dismiss it as an absurd characterization of all the testing Waymo has been performing. As I pointed out at the time, if none of these incidents had occurred during these tests, then, by your logic, we would have no information at all about the safety of the cars! This position, applied consistently, amounts to a complete rejection of statistical methods, and I "bypass" that.
At least the idea that Waymo is hiding data that would reveal the vehicles as being too dangerous to be on public roads is not quite that wrongheaded, but it implies that Waymo is doing this on a massive scale that cannot possibly succeed in the long run. It is not clear, for example, how it could hide information about similar or worse incidents from the insurance companies, and a story about your gambling relative is not changing my mind.
>At least the idea that Waymo is hiding data that would reveal the vehicles as being too dangerous
This is your dichotomous thinking again. I am not making a point about it being "safe" or "unsafe" as a dichotomous choice. I'm saying we don't have good data to make a claim one way or another. That's also different from saying, "we have no information at all." When you combine that with the fact that we have other evidence that RL models can result in unpredictable behavior, it should give us pause.
Insufficient data + priors about unpredictable behavior = uncertainty of performance
The through-line of my entire point is that uncertainty erodes trust, and trust is necessary for wide adoption in the public sphere. It isn't a hard concept if you can lay your bias and dichotomous thinking aside to consider it. Waymo seems to agree; although the PR headline is about 7MM miles traveled, the report says:
>"the required ADS VMT to establish statistical significance ranges from tens to hundreds of millions of
miles, and the fatal outcome requires hundreds of millions to billions of miles of driving are needed."
So unless you have better data to show, you've demonstrated nothing to make me change my mind.
I'll help you out here, since there still appears to be some difficulty.
I've replaced your word "it" to make the point as clear as possible.
>"And you chose to quote it, and to make the point that" [unexpected behavior] "is not the sort of thing we want on public roads"
Or to put it different terms, the sometimes unpredictable behavior resulting from RL may be a feature on the battlefield but a bug on public roadways.
I still stand by that point. And, yes, that means dcow also missed that point. There's nothing hypothetical about bringing about real-world case studies of autonomous behavior based on RL models. We've been through this so many times now that I'm coming to the conclusion you may be arguing in bad faith or you get incredibly distracted by certain terms that it inhibits reading comprehension.
>I've replaced your word "it" to make the point as clear as possible.
>"And you chose to quote it, and to make the point that" [unexpected behavior] "is not the sort of thing we want on public roads"
Ah, so we're playing the "I didn't say what I just said" game now. Here's what you actually wrote:
"That’s true, but also one of the selling points of some AI tasks. As a non hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many “best practices” it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road."
Anyone with the slightest familiarity with language will find no credibility in your claim that in this comment, the thing that is being called "possibly good in war" but "not so good on a public road" is anything other than the one explicitly-mentioned aspect of the system that you have specifically chosen to present as an example. Furthermore it was completely reasonable for dcow to respond "modern AVs are not driving like they have a death wish by any stretch of the imagination" after you chose to make the above statement.
With your latest response, you continue to put more weight on an anecdote about this unrelated system than you do on the extensive empirical evidence from testing the actual system that is the subject of this article.
Ignoring the no true Scotsman-ism of your post, what's odd is that you are telling someone who actually wrote it what was intended. You've made it completely clear you misunderstood it. I've pointed out exactly where you made the mistake. Yet you can't seem to bring yourself to admit that just maybe your biases made you infer more than what was actually said. The "explicitly mentioned aspect" is the unpredictable behavior. You can tell that, not just from the wording, but from the fact that has been the consistent throughline of the entire sub-thread. And not to beat a dead horse, my secondary point has consistently been that we should not put too much emphasis on self-curated data when there is a bad incentive to embellish it. Yet here you are.
> what's odd is that you are telling someone who actually wrote it what was intended
They're telling you what you wrote, not what you intended to write.
They're correct.
> The "explicitly mentioned aspect" is the unpredictable behavior.
No, it was breaking best practices and behaving like it had a death wish. That's something that has some overlap with being unpredictable, but is not at all the same thing. You can have a predictable death wish, even.
> maybe your biases made you infer more than what was actually said
I don't think I have much bias here and I agree with them. Also consider that the person that writes something is biased to think the communication was clearer than it actually was.
> And not to beat a dead horse, my secondary point has consistently been
It's neat to have a correct secondary point, but it won't make your primary point correct, and people don't need to add a disclaimer of "while your secondary point is fine" every time they criticize your primary point.
Why do you think "best practices" are considered "best practices"? Is it because they lead to the most desired outcomes? If so, what do you think happens when best practices are broken?
Now I'll concede it's possible that RL can lead to finding new best practices. But that's not the case in the story relayed. It is only a good practice for the scenario because it was an unpiloted aircraft because it otherwise puts the pilot at too much risk (which is why the pilot say it was acting "as if" it had a death wish. He was anthropomorphizing it.) That still means it's bad practice when the goal is to save human lives.
Yes, I understand the concept of a best practice, and why breaking it is bad.
But that's not the same as being unpredictable. Those are orthogonal complaints, and nothing in your quote suggests that the "death wish" flying was more unpredictable.
And we already know that these cars act super cautious, the opposite of acting like they have a death with.
So that quote has no useful information that we could transfer to the car situation.
> What's odd is that you are telling someone who actually wrote it what was intended...
What I am saying here should be clear to someone such as yourself, who is invoking our theory of mind in his claims: I am making a distinction between what you are now saying you meant all along, and what other people will recognize as having been your intent when you first wrote the passage in question. We cannot present proof, but nevertheless we know, beyond reasonable doubt. Your explanation does not pass the sniff test.
I’m not sure this is the blanket case. Hot air balloons, for example, get right-of-way regardless, on the assumption they have less maneuverability. Weird edge case, I know, but just throwing it out there to underscore the danger of absolute statements.
I have to admit I have not piloted a ballon but even a sailing vessel over taking a power boat has to avoid the vessel in front. I would also question the rationality of operating an aircraft you cannot steer:)
But who gets the right of way between 2 hot air balloons?
Funny story, I actually "crashed" in a hot air balloon as a kid, when the hill we were landing on had a draft running up over it that caught the balloon after the basket had touched down, and dragged us along the ground sideways for a good quarter mile.
If I recall correctly, the lower one has right of way because visibility from lower to higher is blocked compared to higher to lower. It is easier for the one further up in the air column to spot and react to the lower one. Going up is also likely a less dangerous proposition than going down.
This is half remembered from a Snowmass balloon rally conversation.
From the article : "But it’s not enough to answer the most important safety question: whether Waymo’s technology makes fatal crashes less likely."
Wouldn't the most important safety question be "whether Waymo’s technology makes fatal crashes MORE likely?" Why assume Autonomous Vehicles have to perform better?
A significant proportion of the current fatal accident rate is based on incidents which involve driving so bad that humans implicated are severely punished for it, or at least banned from driving for a period.
Why would anyone set the benchmark for a commercial driving company lower?
Why do you assume "properly licensed" drivers will be better? My driving test in the United States was a joke. Many are the same. Nothing about it prepared me to be a good driver (which I am not). To me, the phrase "properly licensed" means nothing more than I hold a small card in my wallet that say I am allowed to drive this car. It says nothing about my skill level.
I disagree. Some people will drunk drive no matter the legal consequences, or in my case, I once rear-ended someone because I was drowsy but kept driving because I wanted to get home (fortunately no one was hurt, and yes, the accident was completely my fault). The comparison should be against the general population of human drivers because that's the reality on the road.
You really think the safety bar for running a commercial autonomous taxi service should be set lower than the average human legally entitled to drive!?!
because everyone thinks they're above average. If you're only better than the below average drivers, then everyone will think they're better off driving themselves even if that's not true.
Also, we've accepted and made legal frameworks around the concept that people sometimes kill each other with cars. Robots killing people with cars does not benefit from this carve out.
> hence I will not trust any system that does not perform at least as well as I do.
...which is a pretty big fallacy and, if implemented as law (which I am working to make sure it is not), will get a lot of people killed.
Literal drug addicts, people driving under the influence of alcohol, and people driving distracted are driving right now. Even AV that performs at the level of the median driver will massively reduce the number of those people actually making driving decisions, and therefore directly reduce road deaths.
Why? Because most accidents and deaths are caused by drivers performing at the low end of the spectrum. In the US, a full third of accidents are caused by drunk driving - which would be virtually nonexistent with AVs ("virtually" because the AVs aren't going to be perfect and are going to need manual intervention).
The bar is literally the average, because if everyone drove average, deaths would plummet.
And, unlike the American public, it's massively easier to improve AV driving behavior than average human driver behavior. A system that starts out eliminating the lower end of driving performance while improving over time is a deal too good to turn down if you care about human life.
I am a terrible driver -- so easily distracted. Almost zero men I have met in my life are willing to admit the same. (However, some women in my life do.) As a result, I try to avoid driving as much as possible and use mass transit.
It remains to be seen if autonomous driving systems are actually safe. But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that.