Machine employees in government

In 2016, as Intelligentsia Research, we coined the term “machine employees” to describe AI in the workplace. Our goal was to differentiate traditional IT technologies from modern AI; where deep learning and other machine learning techniques were being deployed with the specific intent of taking on a decision making role.

The concept of machine employees is important because, while they can substitute for humans in processes where people once evaluated, decided and acted, they aren’t held to the same standards as people. They can’t be easily questioned, sued or taxed. In short, machine employees aren’t accountable, but the humans that employ them should be. A human should be able to explain, justify and override a poorly performing machine employee.

But everywhere you look, it seems there are more and more instances of machine employees that are poorly designed and deployed, with government services a particular concern. A recent essay in the Columbia Law Review by Kate Crawford and Jason Schultz from AI Now, outlines where AI systems used in government services have denied people their constitutional rights. They argue that, much like other private actors who perform core government functions, developers of AI systems that directly influence government decisions should be treated as state actors. This would mean that a “government machine employee” would be subject to regulation under the United States Bill of Rights, which would prohibit the federal and state governments from violating certain rights and freedoms, something that is a particular risk when services are provided to groups where people are already at a disadvantage.

Here is the key question — are AI vendors and their systems merely tools that government employees use or does the AI perform the functions itself? Are these systems the latest tech tool for human use or is there something fundamentally different about them? If the intent of a machine employee is to replace a human employee, or substitute a significant portion of a their decision making capability, then our intuitions tell us it’s the latter.