A person laying on the ground in a crosswalk was likely never considered by the team to include in their training data. Those outlier situations are exactly what real world data is needed for. And the only way to properly train for most of these situations is to drive in the real world. The real world isn’t perfect situations and nice lines on fresh asphalt so while base training in perfect situations is useful, it will still miss the exact same situation in a real world environment with crappy infrastructure.
Not sure what or how Cruise uses the data collected in real-time, but I can see camera visuals categorizing a person laying in the crosswalk as something like damage to painted lines, and small debris that can be ignored. Other sensors like radar and lidar might have categorized returns as something like echoes or false results that could be ignored, again because a person laying in the crosswalk is extremely unlikely. False data returns happen all the time with things like radar and lidar, millions of data points are ignored as outliers or info that can be safely ignored, and sometimes that categorization is incorrect.
These cars won’t even drive over a fire hose laid out across the road (this was literally a test we did regularly when I worked on Waymo’s SDCs). They definitely won’t drive over a person (intentionally).
Inertia is a factor here. Cars can’t stop on a dime. By the time the pedestrian was knocked in front of the SDC, there was likely not enough time to have possibly avoided hitting her, just due to physics.
A person laying on the ground in a crosswalk was likely never considered by the team to include in their training data
I didn’t bother reading any further than this. The person was on the crosswalk when both cars started moving. Neither car should have been moving while anyone was still on the crosswalk.
That was the exact moment I called bullshit as well. You’d damn well better plan for people tripping and falling. It happens all the time, but generally is pretty minor if not exacerbated by being run over. This is like saying they didn’t train it on people holding canes or in wheelchairs.
It’s not about the ability to recognise someone lying in the road (although they obviously do need to be able to recognise something like that).
She was still walking, upright, on the crosswalk when both cars started moving. No car, driverless or otherwise, should be moving forward just because the lights changed.
Thats the whole point of their comment. The car did not recognize anyone was on the crosswalk because it was never trained to look for people laying in the crosswalk.
And that’s fine. But if it’s unable to recognize any object in the road, it’s not fit for purpose. The fact that the object was a person just makes it so much worse.
Agreed. I’m not defending Cruise at all. They should have humans in the car if they are testing. Or at least a drone-style driver sitting in a room watching a camera feed. I wonder if the car thought there was just a speed bump ahead. Some speed bumps are striped similar to crosswalks. I can see situations where the autopilot can’t determine if something is a speed bump or genuine obstruction (either false positive or negative).
I actually work at one of these AV companies. We definitely have training data on adults and children laying down. I’d be very very very surprised if Cruise doesn’t due to all the people laying down on the sidewalks in SF. In addition, the clarity of the lidar/camera data on objects on the road is very clear. You can see the dips and potholes in the road as well as specifically see the raises of the painted lines. There’s no way they weren’t tracking the person.
I could see predictions on the pedestrian saying the coast is clear. Once the initial crash happens, there likely isn’t enough room to stop in time even with a max break.
A person laying on the ground in a crosswalk was likely never considered by the team to include in their training data. Those outlier situations are exactly what real world data is needed for. And the only way to properly train for most of these situations is to drive in the real world. The real world isn’t perfect situations and nice lines on fresh asphalt so while base training in perfect situations is useful, it will still miss the exact same situation in a real world environment with crappy infrastructure.
Not sure what or how Cruise uses the data collected in real-time, but I can see camera visuals categorizing a person laying in the crosswalk as something like damage to painted lines, and small debris that can be ignored. Other sensors like radar and lidar might have categorized returns as something like echoes or false results that could be ignored, again because a person laying in the crosswalk is extremely unlikely. False data returns happen all the time with things like radar and lidar, millions of data points are ignored as outliers or info that can be safely ignored, and sometimes that categorization is incorrect.
Well if you know you need backup with edge cases, why isnt there a human in the car with controls?
Not only that, but no matter whether it can identify a person as a person, cars shouldn’t be driving over objects that are child sized or larger.
These cars won’t even drive over a fire hose laid out across the road (this was literally a test we did regularly when I worked on Waymo’s SDCs). They definitely won’t drive over a person (intentionally).
Inertia is a factor here. Cars can’t stop on a dime. By the time the pedestrian was knocked in front of the SDC, there was likely not enough time to have possibly avoided hitting her, just due to physics.
I didn’t bother reading any further than this. The person was on the crosswalk when both cars started moving. Neither car should have been moving while anyone was still on the crosswalk.
That was the exact moment I called bullshit as well. You’d damn well better plan for people tripping and falling. It happens all the time, but generally is pretty minor if not exacerbated by being run over. This is like saying they didn’t train it on people holding canes or in wheelchairs.
It’s not about the ability to recognise someone lying in the road (although they obviously do need to be able to recognise something like that).
She was still walking, upright, on the crosswalk when both cars started moving. No car, driverless or otherwise, should be moving forward just because the lights changed.
Thats the whole point of their comment. The car did not recognize anyone was on the crosswalk because it was never trained to look for people laying in the crosswalk.
And that’s fine. But if it’s unable to recognize any object in the road, it’s not fit for purpose. The fact that the object was a person just makes it so much worse.
Agreed. I’m not defending Cruise at all. They should have humans in the car if they are testing. Or at least a drone-style driver sitting in a room watching a camera feed. I wonder if the car thought there was just a speed bump ahead. Some speed bumps are striped similar to crosswalks. I can see situations where the autopilot can’t determine if something is a speed bump or genuine obstruction (either false positive or negative).
They are 100% trained on bodies laying prone on the ground.
She was standing up when the cars started moving.
I actually work at one of these AV companies. We definitely have training data on adults and children laying down. I’d be very very very surprised if Cruise doesn’t due to all the people laying down on the sidewalks in SF. In addition, the clarity of the lidar/camera data on objects on the road is very clear. You can see the dips and potholes in the road as well as specifically see the raises of the painted lines. There’s no way they weren’t tracking the person.
I could see predictions on the pedestrian saying the coast is clear. Once the initial crash happens, there likely isn’t enough room to stop in time even with a max break.