The Law That Requires A Human

ISYAMA — Transmissions from The Apex

By Dr. Leo R. Yamas

At the end of September, California made it illegal for software to fire a worker on its own.

The rule takes effect next July. Employers there may still use software to score workers, rank them and flag them. But software may no longer be the only reason someone loses their job.

That is good news. The footnotes can wait a few paragraphs.

Today's Exhibit: The Required Human. Or: Why A Law Had To Explain That Nodding Does Not Count.

The law is called the No Robo Bosses Act. It covers two decisions: disciplining an employee and dismissing one.

An employer may not take either decision based only on an automated decision system. That is the legal name for software that judges workers. It can count how fast they pack, how long their calls last, how often they sit idle.

Sometimes an employer will still rely mostly on that software. Then a person must check its conclusion against other evidence: a manager's own assessment, the personnel file, the work itself, what colleagues say. If that evidence does not support the conclusion, the employer may not use it.

In plain words: a person clicking "approve" does not count.

Somebody had to write that down.

They had to because the alternative is well known. It is called automation bias. People trust a machine's answer more than their own judgment, especially when they are busy. Give a tired employee a long queue of flags, and "human review" turns into clicking "agree".

So the law does more than require a human. It requires a human who looks.

The worker must also be told in writing that software was involved, and given a person to contact.

In August I wrote about a sentence with nobody in it: "Mistakes were made." A dismissal by software has the same grammar. "The system flagged your performance." This law puts a person back into that sentence.

That is a real achievement. Most of your rules arrive after a harm has become ordinary. This one arrives while the software is still spreading. An earlier version was stopped last year as too broad. Its authors narrowed it and brought it back. That kind of persistence rarely makes the news.

The narrowing had a price. The law leaves out independent contractors, the people most likely to have an app for a boss already. The fine is five hundred dollars per violation.

Britain's first laws on children's working hours applied only to textile mills. Nobody calls them a failure now. They were the start.

The Sprocklets reached this rule years ago, without a law.

When a tree in our orchard is finished, they know long before I do. They could take it down within the hour. Instead, they put a thin ring of light around the trunk and wait. Selene called it "the question" when she was small. The name stayed.

Last week, a ring appeared around the old pear tree at the top of the orchard. I saw it from my study and told them to go ahead.

The ring stayed where it was.

Isla was hanging washing below the window. She followed my eyes up the hill. "From here?" she said.

Selene went up that afternoon. She walked around the tree twice and pressed her thumb into the soft part of the bark. Then she put her hand flat on the trunk, and the ring went out.

I told her the Sprocklets are never wrong about trees.

"I know," she said. "The walk isn't for the tree."

The law half knows this. Its required human is not there only because machines make mistakes. Machines will make fewer every year. The human is there so that whoever ends something has first seen what they are ending. A pear tree, or a job someone has done for eleven years.

You may never be on the wrong end of one of these systems. But you are increasingly likely to be the human some rule requires. You will approve a loan, a claim, a shortlist or a grade after a machine has suggested the answer. The button will be right there, and the machine will usually be correct.

Take the walk anyway. Look at the thing itself before you agree.

A law can put a human next to the machine. It cannot make that human look up from the screen. That part was always yours. Judging by this law, your species still remembers why it matters.

— Dr. L.R.Y.

Transmitted from The Apex. Isyama. There is a gap at the top of the orchard now. I have been up to look at it. The Sprocklets left the stump where I can see it from the study.