Why isn't AI driving my car yet? - A brief exploration of autonomous vehicles and analogous life-vital ML systems

Optimally, once a self-driving car model can be developed and consistently achieve an F-score that outperforms the human equivalent we need all humans to stop driving in favor of the better alternative. Unfortunately, this would no doubt be an ethical and legal nightmare. People don’t want to give up their autonomy to a “machine” even if it is theoretically safer. Or not yet at least. In this post I want to explore a bit about why that might be, and what advantages a human driver could have over a purely digital one.

The most dangerous aspect of when applying machine learning to problems vital to life or organizational viability in the context of the reading is ML's inability to adapt to unfamiliar input. This is a well-founded concern considering the behavior of AI as pointed by Zhi Quan Zhou’s Case Against Mission-Critical Applications of Machine Learning and the subsequent author response. In the short term, it is wise to continue to rely on machine learning as a simple tool for analysis, and not to assign these models much in the way of responsibility or decision-making. Leave the final say in these sorts of problems to humans as we are much better at adapting to unfamiliar situations, and reacting in general.

At the very least, humans are good at avoiding catastrophe due to unexpected stimulus and even ambiguity. For example, our fight or flight response can be triggered by a doorbell when no guests were expected, similarly, the feeling of anxiety is our body’s reaction to a generalized fear of an unknown threat or abstracted internal conflict that could potentially require a heightened sense of awareness. ML does not have these responses to inconclusively. For most models, it can only guess at list of distinct pre-determined outputs.

A long-term solution for these vital problems would be to shift focus to training models with a response to ambiguity. For instance, if confidence values for a particular decision are under a "safety threshold", then apply some sort of contingency function i.e. pump the breaks.

As far as life-vital systems go I think that autonomous vehicles are a great place to start for a few reasons: the inputs are mostly predictable, there are redundant security measures can be implemented in the event of failure, and "distracted bags of chemicals" are historically really bad at driving anyway; the bar is pretty low.