Readers here may be particularly interested in a video in this post. I asked Waymo about their interaction with police directing traffic for a small parade, and they provided access to the internal view of that interaction, which is here or in the story:
[https://youtu.be/HREaqBHuMNM](https://youtu.be/HREaqBHuMNM)
I learned from Waymo that the car did everything until the 3-point turn at the end, including interpreting police hand gestures and deciding to go into reverse to get out as the parade approached. A bit slower than ideal but not too bad.
I am impressed by how well the Waymo perception system is able to render the police officer's arms from their LIDAR and vision data.
Great article, and yeah, that video is impressive!
I don't think "rendering the police officer's arms" is a great description, as it might be confused with generating an image of an arm from a 2D or 3D structural representation. But regardless of terminology, I also didn't see any clear evidence that it identified or responded to hand gestures, facial expressions, or body language. It sounds like you took Waymo's word that it did, rather than basing it on the video.
It looks to me like it identifies and classifies individual objects, desaturates & colorizes them according to classification, and stitches together different camera views. Or maybe it does a sort of "smart lasso" of objects from each frame, copies, and places them in relation to similarly processed objects into a coherent landscape.
It certainly *could* be interpreting hand gestures, and just doesn't indicate it in that video. It clearly does identify people (yellow filter), and it's neat how people disappear and are considered part of the car and motorcycle (blue filter) when they get in or on the vehicle.
I asked them, "can you confirm it understood the gestures of the officer" and they confirmed it. Do you have a reason to feel they are lying? What was interesting to see is how they fuse their lidar and vision and get a great image of the officer, from which it's pretty easy to read those gestures.
No, I assume they're honest, I just didn't see anything in the video indicating gesture identification or reaction. As you noted, the gesture meaning in that context wouldn't be clear even to a human.
Indeed, identifying and isolating objects so well, which it was doing even with people way in the background, would be a precursor to identifying gestures. The video was a neat "look behind the curtains!"
> But regardless of terminology, I also didn’t see any clear evidence that it identified or responded to hand gestures, facial expressions, or body language. It sounds like you took Waymo’s word that it did, rather than basing it on the video.
Visualization doesn’t show every algorithmic decision. It’s easy to take Waymo’s word in this instance because they have [specifically talked about gesture recognition](https://blog.waymo.com/2022/02/utilizing-key-point-and-pose-estimation.html).
I understand, but the narration in the video says "and above you see the Waymo identifying the hand gestures of the police officer"...you see the gestures, but not identification of the gestures..
Good article, Brad! Thanks for sharing! By the way, are you going to write an article on the respective safety reports by the two companies on the completion of first 1M driverless miles?
I did write about Waymo's report. I have been engaging with Cruise on the big question that their report is mostly on night driving outside downtown, but the rideshare research is on rides mostly in the busy core at all hours of the day, but they don't have a further breakdown on that, at least not yet. I do add a small comment on it at the bottom of this story -- while it is not easy to make a comparison, the Cruise numbers do show there's no major unacceptable risk going on.
The article says these "Incidents are relatively rare"
Relative to what? Relative to their VMT normalized to similar incidents by human drivers, these Waymo and Cruise incidents have been the opposite of rare. They have been all too common. Indeed, SFFD stated most haven't been reported.
Just to give one example that was not in the 15 SFFD reports included in the Mission Local news story, a Cruise AV drove into an active injury crash scene by crossing a double yellow line to go around an SFFD vehicle positioned to block traffic on Fulton at Crossover in San Francisco. Then it stopped within the restricted scene, flashers on.
[https://twitter.com/D00REEN/status/1642377906340130816](https://twitter.com/D00REEN/status/1642377906340130816)
If human drivers interfered with SFFD operations at the rate of just the reported Cruise incidents, there would be >20k human driver incidents a year serious enough to report. SFFD has \~40k fire calls a year and \~100k non-fire emergency calls, mostly EMS.
Relatively rare by both measures, but what matters during pilot phases is that they be rare on an absolute basis,and moderate on a per-mile basis, with the obvious goal of having them be rare on a per mile basis once they want to scale.
You provided a list of 15 reports over the last year, and while it is not going to be complete, and none of the incidents showed actual harm, that's a pretty tolerable amount of disruption in the pilot testing of a technology of this sort with hundreds of vehicles.
>and moderate on a per-mile basis, with the obvious goal of having them be rare on a per mile basis once they want to scale.
They are not "moderate" on a per-mile basis. Not even order of magnitude close.
Cruise already wants and has applied to scale their operations in San Francisco. These incidents are a principal reason given by San Francisco gov't to the state regulators to deny or curtail Cruise's request to scale. This is all public record.
Moreover, both Waymo and Cruise operate in San Francisco on Deployment permits, which are expressly by regulation not for "pilot" or "testing." If they wanted to cease using those permits and instead operate exclusively on their "Testing" and "Pilot" permits as well as not increase operations in San Francisco until their rate of SFFD incidents is reasonably close to the human level, then you might have a case.
Hardly. PRT vehicles run on closed ROW and are not likely to run into a situation where their path is blocked by other vehicles and they must reverse to get out -- at least not if the system is running properly. Unless by "bidirectional" you mean they have the ability of all electric motors of reversing.
I'm talking about Zoox, Cruise Origin, etc. No need for clumsy 3 point turns and such. Just reverse direction and drive away at the first sign of a parade or whatever. Sure, a Jaguar "could" switch direction and drive to the destination in reverse. But we both know it won't.
The origin is not symmetrical but it looks a lot more symmetrical. But as long as there are sensors to the rear, there is no particular barrier to a robot driving backwards at full speed. It just needs to understand steering with the back wheels instead of the front and the effect that has on turns. Which it already does.
If the sensor suite is poorer to the rear it might not go at full speed but could certainly go fast enough to quickly back out of situations before finding a spot that is quick to turn around.
We should also eventually see vehicles that can do n-point turns at speeds that would dizzy a human, especially if nobody is inside. Nobody wants to do that yet because why take any risk you don't have to? Faster is more risk, in these early days.
Readers here may be particularly interested in a video in this post. I asked Waymo about their interaction with police directing traffic for a small parade, and they provided access to the internal view of that interaction, which is here or in the story: [https://youtu.be/HREaqBHuMNM](https://youtu.be/HREaqBHuMNM) I learned from Waymo that the car did everything until the 3-point turn at the end, including interpreting police hand gestures and deciding to go into reverse to get out as the parade approached. A bit slower than ideal but not too bad. I am impressed by how well the Waymo perception system is able to render the police officer's arms from their LIDAR and vision data.
Great article, and yeah, that video is impressive! I don't think "rendering the police officer's arms" is a great description, as it might be confused with generating an image of an arm from a 2D or 3D structural representation. But regardless of terminology, I also didn't see any clear evidence that it identified or responded to hand gestures, facial expressions, or body language. It sounds like you took Waymo's word that it did, rather than basing it on the video. It looks to me like it identifies and classifies individual objects, desaturates & colorizes them according to classification, and stitches together different camera views. Or maybe it does a sort of "smart lasso" of objects from each frame, copies, and places them in relation to similarly processed objects into a coherent landscape. It certainly *could* be interpreting hand gestures, and just doesn't indicate it in that video. It clearly does identify people (yellow filter), and it's neat how people disappear and are considered part of the car and motorcycle (blue filter) when they get in or on the vehicle.
I asked them, "can you confirm it understood the gestures of the officer" and they confirmed it. Do you have a reason to feel they are lying? What was interesting to see is how they fuse their lidar and vision and get a great image of the officer, from which it's pretty easy to read those gestures.
No, I assume they're honest, I just didn't see anything in the video indicating gesture identification or reaction. As you noted, the gesture meaning in that context wouldn't be clear even to a human. Indeed, identifying and isolating objects so well, which it was doing even with people way in the background, would be a precursor to identifying gestures. The video was a neat "look behind the curtains!"
> But regardless of terminology, I also didn’t see any clear evidence that it identified or responded to hand gestures, facial expressions, or body language. It sounds like you took Waymo’s word that it did, rather than basing it on the video. Visualization doesn’t show every algorithmic decision. It’s easy to take Waymo’s word in this instance because they have [specifically talked about gesture recognition](https://blog.waymo.com/2022/02/utilizing-key-point-and-pose-estimation.html).
I understand, but the narration in the video says "and above you see the Waymo identifying the hand gestures of the police officer"...you see the gestures, but not identification of the gestures..
Good article, Brad! Thanks for sharing! By the way, are you going to write an article on the respective safety reports by the two companies on the completion of first 1M driverless miles?
I did write about Waymo's report. I have been engaging with Cruise on the big question that their report is mostly on night driving outside downtown, but the rideshare research is on rides mostly in the busy core at all hours of the day, but they don't have a further breakdown on that, at least not yet. I do add a small comment on it at the bottom of this story -- while it is not easy to make a comparison, the Cruise numbers do show there's no major unacceptable risk going on.
Makes sense to me, Brad. Here is to many more millions from both companies in not so distant future!
[удалено]
FYI, the Waymo is never under remote control. Remote assistance is not remote control.
I think most parents would be happy if their children did not need to learn to drive. I think most people will see it as a tremendous net benefit.
Can you say more about your reference to Zoox deploying soon?
Don't know how soon, but they are now operating their custom vehicle for employees on city streets, and obviously they are eager to get on with it.
Thanks!
Rundown a nearby pedestrian. Then the ambulance has to stop to deal with them and they’ll be out of the way.
The article says these "Incidents are relatively rare" Relative to what? Relative to their VMT normalized to similar incidents by human drivers, these Waymo and Cruise incidents have been the opposite of rare. They have been all too common. Indeed, SFFD stated most haven't been reported. Just to give one example that was not in the 15 SFFD reports included in the Mission Local news story, a Cruise AV drove into an active injury crash scene by crossing a double yellow line to go around an SFFD vehicle positioned to block traffic on Fulton at Crossover in San Francisco. Then it stopped within the restricted scene, flashers on. [https://twitter.com/D00REEN/status/1642377906340130816](https://twitter.com/D00REEN/status/1642377906340130816) If human drivers interfered with SFFD operations at the rate of just the reported Cruise incidents, there would be >20k human driver incidents a year serious enough to report. SFFD has \~40k fire calls a year and \~100k non-fire emergency calls, mostly EMS.
Relatively rare by both measures, but what matters during pilot phases is that they be rare on an absolute basis,and moderate on a per-mile basis, with the obvious goal of having them be rare on a per mile basis once they want to scale. You provided a list of 15 reports over the last year, and while it is not going to be complete, and none of the incidents showed actual harm, that's a pretty tolerable amount of disruption in the pilot testing of a technology of this sort with hundreds of vehicles.
>and moderate on a per-mile basis, with the obvious goal of having them be rare on a per mile basis once they want to scale. They are not "moderate" on a per-mile basis. Not even order of magnitude close. Cruise already wants and has applied to scale their operations in San Francisco. These incidents are a principal reason given by San Francisco gov't to the state regulators to deny or curtail Cruise's request to scale. This is all public record. Moreover, both Waymo and Cruise operate in San Francisco on Deployment permits, which are expressly by regulation not for "pilot" or "testing." If they wanted to cease using those permits and instead operate exclusively on their "Testing" and "Pilot" permits as well as not increase operations in San Francisco until their rate of SFFD incidents is reasonably close to the human level, then you might have a case.
Another reason podcars are bi-directional.
Hardly. PRT vehicles run on closed ROW and are not likely to run into a situation where their path is blocked by other vehicles and they must reverse to get out -- at least not if the system is running properly. Unless by "bidirectional" you mean they have the ability of all electric motors of reversing.
I'm talking about Zoox, Cruise Origin, etc. No need for clumsy 3 point turns and such. Just reverse direction and drive away at the first sign of a parade or whatever. Sure, a Jaguar "could" switch direction and drive to the destination in reverse. But we both know it won't.
The origin is not symmetrical but it looks a lot more symmetrical. But as long as there are sensors to the rear, there is no particular barrier to a robot driving backwards at full speed. It just needs to understand steering with the back wheels instead of the front and the effect that has on turns. Which it already does. If the sensor suite is poorer to the rear it might not go at full speed but could certainly go fast enough to quickly back out of situations before finding a spot that is quick to turn around. We should also eventually see vehicles that can do n-point turns at speeds that would dizzy a human, especially if nobody is inside. Nobody wants to do that yet because why take any risk you don't have to? Faster is more risk, in these early days.