How to tell whether machines can do your job

Machine learning will not spell the end of work, but will have profound effects on the economy.
Researchers developed a list of questions designed to evaluate whether a task or a job is suitable for automation.

by Robert Ferris, CNBC.com, 22 Dec 2017

There may be a way to predict which jobs are most vulnerable to being taken over by machines, says a new research paper.

Two researchers from the Massachusetts Institute of Technology and Carnegie Mellon University say machine learning will not spell the “end of work” for humans, but will have considerable impacts on the economy and the way people work. They have created a set of 21 questions that evaluate how suitable a task might be for machine learning based on what they know about machine learning systems’ current and future capabilities.

They published their paper Thursday in the journal Science.

Managers should be able to use the rubric to evaluate the tasks performed in every job in their organization, while policymakers can use the list to determine which occupations are most likely to be affected by automation, said the paper’s co-author, Erik Brynjolfsson, in an interview with CNBC. Brynjolfsson is a professor at the MIT Sloan School of Management. The report’s other author is Tom Mitchell, a professor of computer science at Carnegie Mellon University.

“It is very rare that an entire occupation is suitable or not suitable for machine learning, but within a job there may be several tasks that are,” Brynjolfsson said. One of the paper’s main premises is that we are not anywhere near what is called general artificial intelligence, meaning machines that can perform all tasks humans are capable of.

The challenge for entrepreneurs and managers will be to “unbundle” jobs, sort them into the tasks that can be automated and those that can’t, and then “rebundle” tasks into new jobs, Brynjolfsson said.

The list includes statements such as “Task does not require complex, abstract reasoning,” and “Task does not require detailed, wide-ranging conversational interaction with a customer or other person” and “Long term planning is not required to successfully complete the task.” Users rate how much they agree or disagree with the statement, and then add up the scores to arrive at a final rating.

Some common characteristics of tasks suitable for machine learning emerge. For example, machine learning is well suited for tasks that involve a clearly defined set of inputs paired with an equally clear set of outputs.

Machines are, for example, good at tasks like classifying things, such as breeds of dog. They can, also use data to evaluate whether someone is likely to default on a loan.

They also need access to large data sets for training, which may not be available or may be difficult to build.

Machines also have trouble with tasks that require long chains of reasoning based on background knowledge about the world, or the kind of “common sense” that humans have. They may be good at games like chess or Go, which do have several steps in them, but they fare less well in a game where they have to locate a newly introduced item in a room, for example.

They also tend to be bad at explaining their decisions. For example, machines can diagnose some diseases as well as or better than the best human doctors, the authors write, but they generally cannot explain how they arrived at those diagnoses. This partly comes down to the fact that artificial neural networks make use of different processes than human brains do.

Machines are also limited to jobs that have a tolerance for error, or no need of provably correct or optimal solutions. They also fare better in jobs where there is no change in what is being learned over time.

Finally, machines fare worse than people at lots of job that require physical dexterity. For all of the advances robots have made in motion, they still remain pretty clumsy compared with people.

The paper also considers the potential impact of several economic factors in how machine learning will affect labor and wages. For example, in some cases, computers will be substituted for people.

Automation may also lower prices for some tasks, which may affect demand, along with employment and total spending. The lowered cost of air travel over time, for example, drove greater total spending and employment in the travel industry.

The authors point out that machine learning may outpace the impact of previous inventions that are widely used such as electricity or the internal combustion engine. Those advances increased total productivity and unleashed waves of complementary innovations.

“Individuals, businesses, and societies that made the right complementary investments — for instance, in skills, resources, and infrastructure — thrived as a result,” the authors wrote, “whereas others not only failed to participate in the full benefits but in some cases were made worse off.”