> As long as the resulting package (flawed self driving system + the average driver) isn't significantly more dangerous than the average unassisted human driver...

Shouldn't that compared to "average driver + myriad of modern little safety features" instead of "average unassisted driver"? The one who has the means to drive a Tesla with the "full driving" mode certain has the means to buy, say, a Toyota full of assistance/safety features (lane change assist, unwanted lane change warning and whatnots).

Why isn’t defeating the self-driving attention controls a crime like reckless driving? Isn’t that the obvious solution?

It almost certainly is, at least when combined with the intentional inattention that follows.

Making it a crime isn't an "obvious solution" to actually make it not happen. Drunk driving is a crime and yet people keep doing it. Same with texting and driving.

The problem is determining who is liable for damages, not prevention. Shifting the liability for willfully disabling a safety control puts them on notice.

Prevention as a goal is how we end up with dystopia.

Gonna be pretty difficult to enforce. Many US states don't even enforce a minimum roadworthiness of cars on the roads.

Does that even matter? If the state doesn’t care to enforce its laws against reckless driving, why should the manufacturer be encumbered with that responsibility?

> drivers using it as intended with close supervision

Doesn't this hide a paradox? Using a self-driving car as intended implies that the driver relinquishes a part of the human decision making process to the car. While close supervision implies that the driver can always take control back from the car, and therefore carries full personal responsibility of what happens.

The caveat here is that the car might make decisions in a rapidly changing, complex context which the driver might disagree with, but has no time to correct for through manual intervention. e.g. hitting a cyclist because the autonomous system made an erroneous assertion.

Here's another way of looking at this: if you're in a self-driving car, are you a passenger or a driver? Do you intend to drive the car yourself or let the car transport you to your destination?

In the unassisted scenario, it's clear that both intentions are one and the same. If you want to get to your location, you can't but drive the car yourself. Therefore you can't but assume full personal responsibility for your driving. Can the same be said about a vehicle that's specifically designed and marketed as "self-driving" and "autonomous"?

As a driver, you don't just relinquish part of the decision making process to the car, what essentially happens is that you put your trust in how the machine learning processes that steer the car were taught to perceive the world by their manufacturer. So, if both car and occupant disagree and the ensuing result is an accident, who's at fault? The car? The occupant? The manufacturer? Or the person seeking damages because their dog ended up wounded?

The issue here isn't that self-driving cars are inherently more dangerous then their "dumb" counter parts. It's that driving a self-driving car creates it's own separate class of liabilities and questions regarding responsible driving when accidents do happen.

The average driver breaks multiple laws on every trip. Most of the time no one gets hurt. But calibrating performance against folks violating traffic and criminal laws sets the bar too low for an automated system. We should be aiming for standards that either match European safety levels or the safety of modes of air travel or rail travel.

I disagree. Perfect is the enemy of good, and rejecting a better system because it isn't perfect seems like an absurd choice.

I'm not saying improvements should stop there, but once the system has reached parity, it's OK to deploy it and let it improve from there.

Except that doesn't work if you're trying to produce a safe product. Investigations into crashes in the airline industry have proven that removing pilots from active participation in the control loop of the airplane results in distraction and an increased response time when an abnormal situation occurs. Learning how to deal with this is part of pilots' training, plus they have a co-pilot to keep an eye on things and back them up.

An imperfect self driving vehicle is the worst of all worlds: they lull the driver into the perception that the vehicle is safe while not being able to handle abnormal situations. The fact that there are multiple crashes on the record where Telsas have driven into stationary trucks and obstacles on roads is pretty damning proof that drivers can't always react in the time required when an imperfect self driving system is in use. They're not intrinsically safe.

At the very least drivers should be required additional training to operate these systems. Like pilots, drivers need to be taught how to recognize when things go awry and react to possible failures. Anything less is not rooted in safety culture, and it's good to see there are at least a few people starting to shine the light on how these systems are being implemented from a safety perspective.

> Perfect is the enemy of good, and rejecting a better system because it isn't perfect seems like an absurd choice.

Nothing absurd about thinking a system which has parity with the average human driver is too risky to buy unless you consider yourself to be below average at driving. (As it is, most people consider themselves to be better than average drivers, and some of them are even right!) The accident statistics that comprise the "average human accident rate" are also disproportionately caused by humans you'd try to discourage from driving in those circumstances...

Another very obvious problem is that an automated system which kills at the same rate per mile as an average human drivers will tend to be driven a lot more because no effort (and probably replace better-than-average commercial drivers long before teenagers and occasional-but-disproportionately-deadly drivers can afford it).

Yes, I agree. We should hold automated systems to a higher standard. Unless you’re proposing we ban automated systems until they’re effectively perfect because that would perversely result in a worse outcome: being stuck with unassisted driving forever.