Driving a rented Tesla Y, this became the most dangerous part of using the highway autopilot. The phantom braking seemed to trigger sometimes by going under an overpass, or when lighting contrast was high, but it definitely happened a couple times for no discernible reason at all in good conditions, and had no regard for traffic behind the vehicle when suddenly braking really hard out of nowhere.
This is nuts. I rented a Model 3 several years ago and had to stop using autopilot because the phantom breaking, which is really phantom _panic_ breaking, was so dangerous and terrifying. How has this not been solved? I've used lane keeping in many other cars since then and never once had an incorrect breaking event.
Overpass phantom braking has been getting reported by drivers for years. IIRC Tesla's solution was to mark those overpass locations in some special hexmap so that the cars would then just not brake there. I don't have a source for that, though. I would hope Tesla can solve phantom braking because that could certainly be causing accidents out there.
I've occasionally seen my Tesla on autopilot slow down for like a shadow or something. You definitely need to pay attention to it.
The article quotes: “With only one sensor type, it’s harder to be sure because you do not have the cross-check from a different type of sensor,” he said.
This is presented as though it's a clear win though. We know for example from the Uber fatality that you can also get false negatives because of failures of sensor fusion, which is a very hard problem. Requiring multiple sensors means that all of your sensor systems need to notice a danger, and you need to be able to integrate that data properly, before you can react.
So I can definitely see that there are major advantages to vision-only. But it's clear that the problem has not yet been solved by Tesla nonwithstanding Elon's inflated claims.
>Requiring multiple sensors means that all of your sensor systems need to notice a danger, and you need to be able to integrate that data properly, before you can react.
This isn't true. The principle in general is that you want to detect situations of danger, minimizing false positives and false negatives, with your pay off function probably valuing false negatives as much worse than false positives. With more sensors, the worst case scenario is that it doesn't improve the payoff function and you simply don't use the sensor information. But in the best case scenario the extra sensor can be used to massively increase your payoff - your camera gets confused by a shadow for example, but your radar doesn't detect shadows, so you can eliminate those false positives. Since the general principal to solve self-driving has been to train AI to process sensor input to make decisions, the worst case scenario is you end up with a neural net that literally just doesn't look at the other sensor data (assuming you're training your AI reasonably, if you aren't then maybe you shouldn't be working on an incredibly difficult problem with safety critical applications)
Tesla has essentially bet that they can get their optical systems so good that other sensors like Lidar do hit that situation where they offer nothing in terms of improving the pay off, but every piece of evidence we've seen so far has indicated that they haven't achieved that, and this is many years after Tesla had been confident that they would acheive it.
I don't know what point you're trying to make. It's logically clear that you don't need unanimity in what the sensors are detecting to make a decision.
> Requiring multiple sensors means that all of your sensor systems need to notice a danger
This is the opposite of true. In well-designed sensor fusion algorithms, every new piece of sensor data, however noisy, helps to overconstrain the estimate. Each sensor reading helps to inform the interpretation of the other readings. If your system is designed such that more information worsens your inference, you have designed a terrible system.
I massively disagree with your reasonning as to why vision only has advantages, or at least i disagree that they're really advantages.
What you are saying and I understand you're not saying this is the way to go but merely giving a point of view but hear me out, is that if you have two inputs instead of one, the two might disagree, which makes it harder to figure out which one is right. But if you have only one input, it can't disagree with it self so it's easier to decide.
But if the one you have is the one of the two that would be wrong, you haven't gained anything, beside being sure about it while being wrong. At least with two inputs you're able to know something is off. And usually you add more inputs so you can decide, and make a strategy depending on context (absolute majority, 50%+1, ...).
And sorry but if when having multiple input the system is confused, or needs time to correlate all the data, or whatever like that, this is still useful as again it lets you know something is wrong danger will robinson.
I think the field of engineering, with planes and spaceships and whatever, has proven how the "simplificity" advantage of single sensor is a scam that will cost lives to avoid fixing issues.
Maybe systems that use both vision and lidar or radar have this problem, but it seems unlikely that taking away the r/lidar will solve it. I don't think it is so much that multi-mode systems have problems because they have more information, but that they have problems despite having more information.
Are you saying that the lack of redundancy is as a "major advantage to vision-only?" Because integrating those redundant systems (sensor fusion) is hard?
> all of your sensor systems need to notice a danger...before you can react
I'd be really surprised if they implemented it this way. I thought the whole pitch behind lidar was that if vision doesn't see the person walking in front of you, lidar can still see the moving person shaped thing and you can still act.
You're still left with the two-sensor problem though: if lidar says "pedestrian" but vision says "clear," you don't know if it's a false positive in lidar or a false negative in vision.
If your safety reaction is consequence-free, no problem. If it isn't (like slamming on brakes in traffic), you really want a 3rd sensor so you can do 2oo3 voting.
Idk why people have the preconception of two (or more) sensors each deciding on an outcome indepently..
Suppose you have a color sensor and a form sensor and you have to identify an orange (fruit). Clearly in conjuction the two sensors will be way better, than assigning everything with color orange orange and everything round an orage.
It's for safety reasons so that the system doesn't enter an undefined state if one of those sensors fails, which it eventually will.
If the signal goes from "black nothing" to "orange round" you know you went from no object to an orange, but what if your form sensor breaks and it goes from "black nothing" to "orange nothing?"
You just made a point for redundancy ie more sensors...
Sensor failures are independent events and need to be correctly identified no matter what.
To take Lidar and Vision, both say something about distance and form of objects. Together they can achieve better performance. If one of them fails (and failure is identified) it defaults to the other sensor only and will be somewhat worse, but should at least allow it to pull over and warn you about sensor failure.
Also failures are much easier to identify when you have a baseline reference of one or more other sensors, in isolation much harder.
What if you have multiple cameras and multiple lidar sensors? You get into statistical modeling immediately and I imagine that’s how the production systems like these work. If 3 cameras and 2 lidars detect a person, the model says X, which translates to such and such FPR and FNR.
Would be cool to know what approaches they take. Stereo and then perspective scaling (smaller thing are farther away) and then probably a blend of spice of ML for recognizing things/knowing expected scale on the fly... idk I'm arm-chair skeptical of vision only but humans do it.
Actually on the small downtown streets of my home city they allow parking too close to the intersection such that the view of my civic doesn't allow me to see particularly far down a relatively busy one way street. To compensate for this I roll down the window to listen for tire noise. Maybe not strictly necessary but I'm not sure its quite true to say that people drive by vision 100 percent of the time. Of course they could just add a fisheye mirror on each intersections street or roll back the parking but still. Another example that comes to mind is those old one lane tunnels where you have to stop at the edge and listen for a honk for a few seconds, honk and then go.
Parking too close to the intersection is the bane of my existence in my home city. If someone decides to park their lifted 4runner anywhere on the block, say goodbye to seeing oncoming traffic -- it just becomes a game of russian roulette, trying to cross the street. And I drive a (small) SUV!
An increasingly complex set of if / else statements.
Is it a left turn? Is it a stationary emergency vehicle? Is it a small child? Don't worry, it's all software and we'll keep sending OTA updates until we catch every edge case.
Yeah it's just amazing to have this real time world system chewing through data. Particularly if say they're taking a picture and pulling points from it and doing that at some refresh rate/computing... idk just cool.
And the loops checking like "I accelerated, it has been this much time, did I actually move 10 meters" or whatever. Then you check if something stationary shrunk by that much in size from the camera perspective...
Side tangent but some drones use this cool technology called "visual inertial odometry"
I’ve driven 15000 miles in the past year on my Model Y, a large part of it shuttling between the Bay Area and Reno, so mostly I-80. And many miles of autopilot.
So for, not a single phantom break.
So I wonder what it is that makes it not happen for me? Probably half of those miles are in the evening, so that might be a partial explanation. Maybe the direction from which the sun is coming?
FWIW: I'm in the same boat (albeit lots of miles in Florida and Texas). I think I've had a total of _one_ "uncomfortable" issue from an overhang 2 years ago in my model 3 (which was corrected by me just hitting the accelerator).
It seems the right set of circumstances convince the navigate on autopilot system that jamming on the brakes is the right thing to do.
I've had a Rav4 for 3 years. I use the adaptive cruise control a lot on highway/freeway and I have never, ever had it exhibit phantom braking. It will sometimes brake a little longer than you expect for cars that are exiting on off ramps, or changing lanes, but it's always for a real car that's really going slower than I am.
I've had a Subaru with Eyesight for almost six years and had zero phantom brakes during that time.
I've had it disable itself momentarily (<30 seconds) due to driving against strong sunlight (it uses visible light cameras) but aside from that the system works flawlessly.
I don't understand why more sophisticated systems that are continuously improved are simply expected to "commonly" phantom brake, but a much dumber/simpler system based on parallaxing two cameras (like the human eyeball, per the name) works reliably.
Actually I don't understanding why nobody else is using parallax in their systems? It is basically depth information for FREE using the cameras you already have. Just angle them a little, so they overlap, and then calculate the depth.
Your video is specifically about pedestrian safety systems, whereas my comment is about crash avoidance systems, apples and oranges. You may have meant to reply elsewhere.
I had a Model 3 and the phantom breaking is horrible. Yes my VW ID.4 sometimes shows an orange line indicating I am too close to something, but it does not slam on the brakes.
I've a 2013 Dodge Charger that has adaptive cruise control (ACC) & forward collision warning (FCW), but no automatic breaking except when using the ACC.
I've never had any phantom breaking when on the ACC, but I also hardly use it (I can't even remember last time I turned it on). I do, however, get "phantom" warnings from the FCW. I use quotes because, to me, the false positives are understandable. They used to happen all the time approaching my old house, where there was an S-turn with a guard rail. There were times that yes, you are driving straight towards the guard rail and the FCW would warn on that, because it doesn't account for the road turning. It's a dumb smart feature; doesn't even have lane assist. For me, it's fine. I know the road ahead. I see the curve. The system doesn't. But, because it doesn't slam on the brakes for me, it's just annoying and not a safety issue to me. If the exact same FCW was capable of breaking for me all the time, yeah, I'd have issues with it and would likely have lodged complaints with Dodge & the NHTSB.
I drive ford fusion for 5 years now that has adaptive radar cruise control, I use adaptive cruise 2 hour hours a day 5/days a week for my commute. It has never once did phantom stop.
The "stop sign thing" was only when you had FSD set to assertive mode, were already traveling under 5.6 mph, no pedestrians were present, no other cars were taking the right of way, and the intersection's speed limit was < 30 mph.
There was no actual risk involved, and it was removed immediately with an OTA update.
Should we trust their software enough to accept that it can accurately make that judgement? It's a really hard problem and they are struggling enough with the fundamentals. I don't think their software is actually at the point where they can make that assertion with any believable certainty.
> A new car built by my company leaves somewhere traveling at 60 mph. The rear differential locks up. The car crashes and burns with everyone trapped inside. Now, should we initiate a recall? Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of-court settlement, C. A times B times C equals X. If X is less than the cost of a recall, we don't do one.
Humans don’t really use stereopsis beyond the reach of their arms (which makes sense if you think about it). Beyond that we use semantic cues, which is why we can also understand pictures.
Sadly most research in this area went out of fashion 30+ years ago.
> Humans don’t really use stereopsis beyond the reach of their arms
This is outright false. A person with acute vision can perceive stereopsis out to 1/4 mile. Trivially, 3D movies are projected onto screens which are 10m away.
> Humans don’t really use stereopsis beyond the reach of their arms (which makes sense if you think about it).
But of course we do. It's how we throw rocks and hit what we aim at. It's how we catch things. It's how we walk around anywhere that has obstacles. We use it beyond the reach of our arms really frequently.
I can still throw a rock fairly accurately with one eye closed. I can catch pretty much perfectly with one eye closed. I can walk around without hitting anything with one eye closed.
Depends. If I'm hunting with a bow or a throwing stick, I'd bet stereopsis is pretty important. Do I need it? Maybe not, but I'll be more accurate with it.
If I'm running through the woods, trying to escape from a bear, stereopsis is pretty useful, because dodging trees while at a dead run is life or death.
I don't do either of those, but some of my ancestors probably did. But I play sports. I may be able to catch a ball with one eye. Getting myself to where the ball is going? I couldn't do that as well with only one eye.
So: Do I need stereopsis? No. Did the human race? Yes, it was a competitive advantage to have.
On a long highway run I find cruise control helps with economy more than anything else. This is because it stops you unconsciously accelerating when you don't need to. It also gives you the chance to wriggle your feet and legs a bit which can be nice. It's only really of use in light traffic as it'll quickly put you in conflict with other drivers who aren't using it. The other benefit is cognitive, but not in the way you might expect: it's not relaxing to turn it on, you still have to steer and very much drive the car; it's more that when you get bored, you can turn it off and "hand fly" the car again, and that change of modality is refreshing in itself.
(This is purely an aside but I have a temporary soapbox so why not. I'm skeptical of more advanced forms of cruise control, even simple things like adaptive. Anything that reduces your need to concentrate tends to leave you open to daydreaming (never mind actual distractions like phones); an example of the "TSA agent problem" where staring at a screen of benign bags all day turns out to be a bad way to detect guns in hand luggage. Better I think to have nothing, absent sci-fi-years-in-the-future autonomous cars where you can sleep while they're underway. Humans are awful at passively waiting for something out of the ordinary to occur; computers are great at that. Computers suck at common sense and detecting intention; humans are amazing. The current state of self driving cars plays to both actors' worst weaknesses!)
The only use I’ve ever found for traditional cruise control is to reduce ankle strain on extremely long road trips. Not a big fan.
Adaptive cruise control is about my favorite “driver assist” technology though. Makes the constant speed changes of rush hour driving much less stressful when you use a large enough safety window. Also on open highways it is just wonderful as you basically never touch it (unlike traditional which constantly has to be disengaged for even a single other car in your lane).
Worth noting that my vehicle’s emergency braking system is distinct and active regardless of ACC status. Also I’m always ready to brake manually.
I use it constantly. It saves having to constantly adjust pressure on your gas pedal to maintain a speed. I also use the up/down buttons constantly to make small adjustments. It’s easier and more precise than the foot.
It’s interesting that many people are saying this. I’ve driven from Palo Alto to NYC and back 5 times and never had any leg issues going 10 hours a day.
It is not about "leg issues", it is also about you mind needs to control your leg. And about your fuel costs: software can regulate power much more precise, speed becomes virtually constant, no acceleration means no changes in a kinetic energy, so less fuel is needed.
Top bicycle athletes spend many hours learning how to pedal to minimize accelerations due to cyclic nature of pedaling. With a car it seems easier (all you need is to keep your foot steady, not to maintain constant torque while switching from one leg muscle to another) but I cannot do it. Cruise control beats me every time.
> Top bicycle athletes spend many hours learning how to pedal to minimize accelerations due to cyclic nature of pedaling.
To expand on this: it's about maintaining a specific cadence (usually in the 100-110 RPM range). You don't change your cadence for the terrain; you maintain your cadence and change gears to match the terrain.
Yes, you need to maintain cadence, because of reasons, like saving your knees from wearing off. But if you want to win Tour de France, then you need to learn how to maintain constant speed also. Not just visually constant, not just as your computer shows, but even more constant. Speed deviates, it depends on a phase of your pedaling, but it means excessive watts lost due to visually undetectable accelerations.
Constant cadence and constant speed are different goals, the first one is about biology (how to get your body to do more), the second one is pure physics. Maybe it is even "anti-biological" thing, because human's legs are designed to push, not to pull and not to create force perpendicular to a leg. I even heard something about using gears that are not exactly circular, to help an athlete to maintain constant tension of a chain.
Amateurs (me for example) do not train for constant speed, though to some extent it comes naturally with practice. But world class athletes measure energy conversion inefficiencies in tenths of a watt, so they get a special training for this.
Yes, indeed. I'm not aware of any hardship when I go uphill full throttle. Going downhill is harder because I have to keep my foot in the air not to depress the gas pedal. In those cases though, I either rest my foot on the break pedal or put it down floor depending on the conditions. When the driver ahead slows down, I slow down using breaks.
I've driven for hours with and without cruise control. I've never felt any discomfort in my lower leg muscles because of using pedals.
Drivers who don't maintain a constant speed on the expressway are one of my pet peeves. Using cruise control is a simple fix to this.
The worst drivers are "booger cars" - they maintain the same speed as you in the adjacent lane with their front bumper roughly adjacent to your rear bumper. I call them "booger cars" because, if you try to get them to stop sticking to you by speeding up or slowing down, they just match your speed. About the only way to get rid of them is to "rub them off" by passing a slower vehicle that is in their lane, breaking their fixation on your car.
Your "booger cars" (I like that term) are very annoying. I refer to the "rub them off" as using the third car to "set a pick" a la basketball.
The kind of driver that especially irritates me is the one that hangs out in the passing lane, driving slowly either at or under the limit, and then when you when you try and get around them, you can't without doing +20 over the limit.
I find cruise control (in a 20 year old vehicle) extremely handy. Means I'm maintaining a constant speed no matter the grade/slope of the road and on long journeys also means I don't get cramp in my right ankle.
I tend to despise most modern driving aids, but cruise control (just the very basic "maintain speed") is extremely handy and makes driving a lot easier. Maintaining a constant speed (to within 1mph) by foot over a 2 hour drive is almost impossible. It also seems to help my fuel economy.
do you mean cruise control as in autopilot and lanekeeping? or traditional cruise control that just keeps the same speed?
for the latter, long drives it helps relieve some tension in my legs to not have to be pressing the pedal to maintain speed. i cant speak for the former as i havent used them
I use OpenPilot, which is third party L2 driver assistance sw/hw. I started looking into it after getting into an accident on the freeway. It's amazing for highway driving. You do still have to pay attention, but long trips are much more relaxing. You don't have to do anything besides pay attention while you are staying in the same lane, and driving is much smoother with it.
During long distance highway driving, foot/ankle muscles get tired out and sore holding the gas pedal constantly. Also, some people are very bad at maintaining a constant speed, frequently accelerating and then releasing the pedal, so their speed looks like a sine wave, which absolutely destroys fuel economy. For those people, cruise control improves efficiency.
Driving on motorway in low or medium levels of traffic.
What traffic is there moves at a consistent pace and cruise control can give your foot a rest for a period of time.
Elon pissed off the company with good radar + camera integration: Mobileye.
In 2016, Mobileye got into a bit of a spat with Tesla, and Mobileye fired Tesla as a customer. Every other company continues to work with Mobileye's radar+optical+ultrasonic solution, but Tesla had to build its design from scratch independently.
Its no surprise to me that Mobileye, with a base of dozens-of-millions of cars, manages to do something better than Tesla (who has less than a million cars IIRC).
Considering they sold 930k cars in 2021 alone, I would guess they have a good bit more of cars on the road now, but less than Mobileye.
Radar, as we know from some Autopilot incidents, has problems detecting stationary obstacles.
I sincerely hope all the progress with the FSD package, which is different from the Autopilot software causing the phantom braking, quickly leads to an Autopilot update which avoids phantom braking and is good at avoiding any obstacles.
Which car manufacturer uses LIDAR in production vehicles? What they usually do is use radar. As Tesla did in the past. This is less likely to do phantom braking, but on the other side is bad at detecting stationary obstacles (remember all the complaints about Autopilot hitting stopped cars?)
While it is true just some chinese EV manufactures use LIDAR right now (Nio, Xpeng), Volvo[1] and Mercedes[2] are on board. I believe once the sensors become cheap enough, people will just use that rather than bet on cameras.
I think the point is not to rely on a single device, but to use an ensemble of sensors that work together to remove single points of failure (single point of confusion?).
Can't wait for the Tesla apologists to come over and tell everyone that "it doesn't bother them that much". Tesla's FSD is a bad joke that has gone on for too long.
