Why Self-Driving Cars Must Be Programmed to Kill
 

I had not given this any thought but damn, yeah, what do you do.

"How should the car be programmed to act in the event of an unavoidable accident? Should it minimize the loss of life, even if it means sacrificing the occupants, or should it protect the occupants at all costs? Should it choose between these extremes at random?"

Why Self-Driving Cars Must Be Programmed to Kill | MIT Technology Review

I find the very idea of self-driving cars deplorable, but I think they will be here shortly.

Pretty interesting to think about.

Implement the 3 Laws.

Read the threads about seniors who refuse to stop driving...

I believe he has undermined his basic argument by using a flawed premise. A self-driving car would not allow itself to get into a situation where it would be suddenly forced to make such a choice.
Unless you suddenly have people jumping off bridges, the vehicle would slow down where a person would just press on while trying to avoid.

Best
Les

When's the last time you drove through an Elementary School zone?

We average a kid a year tagged crossing the street.

That's with crossing guards and strict 20 mph speed limits.

Had one just this past Friday.

"When's the last time you drove through an Elementary School zone?"

Good point.
I had to think about this for a while. I suspect it was a rhetorical question, but it has certainly been over a year.
I know kids can be forgetful of their surroundings. A friend of mine had an elementary school student run headlong into the side of his car. What can you do, aside from drilling it into their heads to look both ways, etc.

OK, substitute a kid diving in front of the car instead of a person dropping out of the sky. If its sensors have not picked out the rapidly approaching threat, all the car can do is try to stop as quickly as possible. If there is a car in the opposing lane it won't go there. A person might though and that is the point of the article.

Best
Les

I predict that self-driving cars will be given their own strip of the road to separate them from 'driven' cars and trucks.

So this will sort of turn one lane of the roadway into a version of a scaled down railway track (&/or tramway track). Driven cars will not be allowed to use this reserved lane.

In effect, this means that this novel technology will be stealing space on the road that you have already paid for.
We already have this in our city for bicycles which have commandeered roadspace on busy throughways. The bike lanes have a lower level of utilization but cause major traffic jams in peak times.

The point of the article is we have to consider there will be situations that are simply not possible for the vehicle to avoid.

We can't assume those worst case scenarios will never happen.

If i've learned one thing in life it's there are always times when your choices are limited to bad or worse.

Yep I have posted in those threads. Losing car keys was a major blow for my father. Anything beyond a block or two is too far to walk when you're 80 and have issues. I am seriously hoping that self driving cars are mainstream in the next 25-30 years. That's probably about the time folks will be looking to lift my keys too.

OPEN THE POD BAY Door, HAL

I am looking forward to my self driving car. I can ride to and from the party, NP! :D

Hopefully the cars communicate and therefore collisions are going to be very few and far between. If they do collide, the rule should be minimize the loss of life but with a correction factor for occupant's age. I.e. a 90 year old vs. a 20 year old occupant - the 90 year old should become crumple zone for the 20 year old. ;)

G

That's an interesting perspective. How should we rate the value of age? I'd say 40-60 year olds should have the greatest value. They survived youth, they have education and experience behind them, and many good years to apply it ahead of them. 0-30 year olds should be of low value as they know next to nothing and have a high accident mortality. (you might sacrifice someone else to save them, only to have them do something stupid and get themselves killed anyway.) Over 60 you don't have a lot of productive years ahead, and over 70 you're just useless.
Please don't miss the sarcasm in my reply.

How will a self driving car differentiate between a shopping cart running into it's path ("OK to hit") and a pram running into it's path ("Not OK to hit")?
If a soccer ball suddenly appears on the road, will the car know it's OK to continue (good field of view, soccer players not nearby) or slam on it's brakes? (poor field of view, kid may be chasing the ball).
Will a car sacrifice itself? (Hit the kid, or miss the kid by swerving into the path of the truck?)
Can the car differentiate between animates? (Hit the dog, miss the truck vs miss the kid, hit the truck).

Self driving cars will need to make moral decisions.
We have enough trouble deciding which crisis human decisions are morally justified.

What about the necessary breaking of road rules?
Driving through a red light to make way for the ambulance to get through the intersection?

Selfdriving cars will only have a future on large roads/freeways/Autobahn.
In the cities and where people are walking around selfdriving cars will not/never work and there the driver will be assisted, as today.

Eventually they'll be ubiquitous, even in cities and residential areas. But I can't see their use in those more stressful cases happening in my lifetime.

KITT v KARR???

For myself, as someone who rides a motorcycle every day, I rather look forward to the predictability of said self-driven cars. It would beat wondering what maneuver the yo-yo next to me is going to do next.
Jim

I think they will be less predictable. Because a human does not think like a machine.
I think you will be surprised what maneuver those machines will be doing! Maybe it will not be allowed to mix self driving vehicles with human driven.

I fear that the insurance companies will remove the unpredictable human element from the equation and we will have no choice but to be driven.

I'll be glad knowing that the self-driven motorcycle behind me isn't going to try to scare me by passing me in a double yellow zone while simultaneously doing a wheelie.

Some of that is pretty simple, thermal imaging rules out the shopping cart.

Soccer ball has no thermal image but as you point out, clear field of view or did the ball appear from behind a large object like a parked car?

When is it appropriate to break the established rules of the road to avoid a collision?

Now we are getting into order of priority, a decision tree.

We can certainly create the if / and / or tree but that brings with it another thing to deal with, reaction time.

What is the processing delay in the car reaching a decision and then acting on it?

If the process starts to exceed human reaction time it's a no go.

A lot of philosophical questions so that we can sit back and do work on our laptops on the way to work. Great.
Can your car drive you home if you've been drinking?

Yes, but what happens if several things happen simultaneously?

How about accelerating through a red light from a dead stop...when the ABS in the car behind you has kicked in, increasing its stopping distance as it is careening down a snowy hill directly towards you?

I'm glad that my car still lets me make decisions.;)

Exactly.

The simplest way to think about it is a scoring system. Could be 1-10, 1-100, 1-1,000 etc...

Factors are weighted and combine for a cumulative score, some things increase the score, other things may have a negative value and decrease the score.

The more granular the scoring the less likely you end up with scenarios that tie although you still have to assume it may eventually happen.

Now the other thing is every action the car takes may introduce new parameters that need to be calculated.

The system could become so complex that it ends up being detrimental to the point that you resort to basic logic, action is taken that represents the least loss of life with no consideration for any other factors.

Again that's something that you can get lost in trying to set logic for.

The basics are simple, the car has range finders which can be used to calculate closing distance and speed to calculate if the oncoming car will stop in time.

But you are at a 4 way intersection with cars coming from the left and right. Can your car find a hole to get you through?

Your car needs to calculate the closing rates of those cars and whether or not it can provide the timed acceleration necessary.

Now it needs to make basic assumptions, what are the road conditions? Snow covered so a hard launch is out of the question. Tires, what condition are they in? How much weight (passengers, cargo) are in your vehicle adding to the gross weight that will affect traction / acceleration.

The AI has to have some behavioral modeling, what are the oncoming cars likely to do when they see you pull into the intersection and how will their actions affect the results of the initial go / no go decision.

If those other cars are also self driving then it's a bit easier because you have a predictive model to calculate against but what programming are they carrying? That opens a whole other can of worms.

There will need to be an industry standard for the logic, you can't let everyone develop their own. One base set of code that all the cars use.

There would certainly be periodic updates required. How do we ensure all cars are running the same version? Real time over the air updates or is it enough to flash the computer during annual vehicle inspections?

We would need an agency that oversees the whole thing, more bureaucracy.

I just hope cars are programmed to fulfill the stereotypes attached to them. You know: Mustangs rev their engines 3-4 times minimum at a stoplight, minivans ignore yield signs, and BMW cars self-park, taking up two spaces. :)

And for sure the BMWs don't use turn signals. :eek:

In the situations posed by the article, I think the cars should have a randomizer routine in the program so that you can never really predict the outcome. That way you can wager or turn it into a drinking game.

Which brings up another question... will self driving cars be allowed to fuel with ethanol? Or would that be drinking and driving?

I'm happy enough for them to give the bicycle lane to the self driving cars, as long as they program the cars to knock over the bicyclists and then honk and rev tauntingly at them.

Interesting. I thought it would just try not to crash. Calculating whose life to try to save is crazy

Liability.

Imagine the lawsuit when a mother pushing her baby stroller across an intersection gets taken out by a self driving car that was avoiding a head on crash with another vehicle.

The plaintiffs argument will be, "you were able to program it to avoid a head on so why not to avoid __________"

I don't think we'll ever see true self-driving cars. Most likely more driver aids. Automakers will never accept total liability for every accident that occurs. Lawmakers will never give automakers total immunity.

Sounds like a bountiful new revenue stream for trial lawyers.

I hope PP and I will be around in 20 years to revisit this thread, lol. Autonomous cars are inevitable, google (and soon apple) will prove it. Collisions will be avoided, and response will be faster and more reliable than human control.

In the 1980's, Rutgers University had less storage/processor power in their mainframe than my iphone has now. I remember a 'friendly debate' with a professor over real-time language translation, which seemed (to me) highly unlikely at the time. If you gave an engineer specs then for one of today's phones, they would have said impossible.

I can only think of 1 event, in my driving career (conservative estimate 900,000 miles) where I am not sure a computer would have been able to avoid the tractor trailer sliding down the hill sideways in front of me, in the snow&ice. But I'm pretty sure sensors and a computer would have found the 2 cars that almost hit me head-on sooner, and done a better job than I the countless times I have driven stupidly late, sick, tired, mad, or slightly inebriated.

If an autopilot can land a passenger jet, surely an autonomous car can avoid collisions safely.

Less than 20 years, more likely 10. I'd be interested in 1 today, for my mother.

I wonder from a liability stand point how this is any different than self-landing airplanes?

I wonder if insurance companies like the predictable risk better when a human is removed?

I wonder how upgrades will be controlled without making things accidentally worse?

Microsoft couldn't do it but Jobs could as the ultimate QA guy at Apple but since he is gone that Competetive advantage is now out the window

In theory this could all work but humans are behind it all anyway, and one mistake could now mean hundreds or millions of lives

An airplane with autoland capability requires both software (easy) and lots of hardware both in the airplane and on the ground, in the immediate area of the runway. Not just a simple GPS. It is not reasonable to outfit every road/intersection with the equipment necessary to compare a self-guided car to autoland capability.

Besides that, the airplane is not cleared to land until all other airplanes are out of the way, hence no risk of collision. Only one airplane uses the runway at a time.

While traffic alerts are common now in airplanes (even my little 4-seater has it), no airplane takes action in its own. Just an alert for the human at the controls.

......

Imagine a self-driving car cruising along the hiway at 60 mph or so. Suppose a mosquito full of 98 degree blood flies near the car, so that the approaching mosquito is quickly gaining in size compared to the small infrared sensor. Car goes into threshold braking, mosquito splats right on the sensor, so the car fires the airbags.

Now the car gets rear-ended, and causes a chain reaction pile-up.

I'm sure the engineers working on this have figured out what to do about wayward bugs.

I hope so......

To your point a more realistic parallel would be aircraft collision avoidance systems multiplied by the delta between planes and cars and even then it only assists with vehicle on vehicle avoidance, won't help if Bambi decides to cross the road at an inconvenient time.

https://en.wikipedia.org/wiki/Airborne_collision_avoidance_system

^this

Or self driving cars get own lanes and are only self driving on highways and other safe environments!


Regarding airplanes: I have never understood why there are no restrictions about what the pilots can do and override. How the hell can it be possible to crash a perfectly working plane into a mountain?! This should be avoided by the electronics in all scenarios! Cant be that hard!