Robots, Dexterity and Visual Recognition

Many lower wage jobs are currently protected from automation primarily because technology cannot yet replicate a person’s ability to recognize and manipulate objects. That is changing, and once affordable  robotic technologies begin to outperform people in tasks that require sophisticated hand-eye coordination combined with moderate-level decision making skills, millions of jobs will be at high risk.

Here are a couple of videos showing these technologies in action, along with articles at Singularity Hub:

Color Sorting Robot:

Robot to Pick Only Ripe Strawberries:

The thing to keep in mind is that the capabilities of technologies like these will certainly accelerate over the coming decades. The degree of progress we see over the next ten years will be dramatically more than what we saw over the last decade. Moore’s Law, for example, would imply a factor of 32 increase (5 doublings) in general capability.  There will also be dramatic cost reductions.

As soon as these technologies find their way into applications that offer a profitable economic trade-off vs. employing workers, they will be widely deployed. Competitive pressures will require this. The primary danger to the US economy will probably be when jobs in the service sector become heavily threatened.  In countries like China, these technologies are likely to accelerate manufacturing automation and ultimately make it difficult to maintain employment as workers continue to migrate from rural areas to manufacturing centers.

Note that both videos are Japanese. Japan is on the leading edge of robotic and automation technologies. So far the unemployment rate in Japan has remained relatively low (although there are many underemployed workers), but this is largely due to regulation and social/cultural attitudes.  The story in the United States— where businesses have complete freedom to slash their workforces—is likely to be quite different.

18 thoughts on “Robots, Dexterity and Visual Recognition

  1. Hey Martin,

    I’ve enjoyed your book and posts and think you definitely are accurate in much of your material as I’ve come to realize a similar outcome with automation from my research. However, I wonder just how big of a clash we will be in when (if we haven’t already) reach peak oil before automation truly takes effect. Obviously this problem will affect technology greatly as much of our growth and technical reality has been focused around oil. I think I read somewhere that something like 1500 nuclear power plants would be needed to replace the United States current oil consumption. Even if we were to pull that off we there would certainly be disruptions in technology at this point in time. I have no doubt that machines will affect the workforce but I’m curious to your thoughts on this process if oil interruptions occur?

  2. Robots just got a huge boost in capability with the release of the Kinect.

    Just Google Kinect robots.

    And Braden, with the new generations of solar cells, ultracapacitance batteries, ultra high efficiency fuel cells, and the likely emergence of superconducting wire in the next decade, we may never see an interruptions, especially if Dense Plasma Focus Fusion keeps improving at the same rate, or if Black Light Power gets verified (though both of those are less likely than the probable creation of cheap, efficient solar cells)

    1. Thanks for the reply Valkyrie. I can’t say I’ve heard of Plasma Focused Fusion or Black Light Power (will check them out) but many of the other forms of energy you mentioned won’t meet the demand of the next century without promptly implementing them now, rather then slowly (and later) like we are doing now. However if we do hit a peak in the next decade or so there will be disruptions no matter how fast we can churn these newer technologies out, even with geothermal energy and nuclear added into the mix. I think like Martin mentions, a very dramatic change in the economy is going to have to happen. The infinite growth paradigm in economics will largely be squandered from diminished resources and this will require a further restructuring of the economy, which I’m not sure Martin has really factored into his outlook. Hopefully we won’t all start killing each other in WWIII to get the necessary resources to fuel our lifestyles and instead will focus on a more efficient city, economy, and world in general to propel us into the next century with minimal disruptions. It took about 35 to 40 years to implement the structures and technology to erect oil so efficiently from the ground so I can only hope we get started before we hit the other side of the bell curve.

      1. I have to disagree on how fast these technologies can and likely will replace dependence on oil over the next decade. The efficiency of solar cells has been increasing by leaps and bounds, while simultaneously methods have been found to make them at fractions of the cost of previous methods. We are likely to be producing cells of 80 to 90% efficiency charging batteries capable of holding 10 to 100 times more electrical potential well within this decade, and be incorporated into nearly all electric devices by it’s end. You are basing objections on those technologies in their prior forms of last decade, probably due to being unaware of the current state of research.

        We are indeed going to see a lot of upheaval, because so many industries were built on a model dependent on scarcity. Martin is completely correct in his assessment that we are rapidly approaching the end of this economic model. What he does not recognize is that this is being accompanied by the rise of a different economic model based on abundance. While I haven’t been able to get a copy to read yet, Macrowikinomics appears to explain some of mechanisms by which this new economic model is rising.

        I don’t believe we will end up in a WW III,though there might be considerable unrest, and even major shake ups in balances of power that will likely result in a unified world government, but any warfare likely to result is probably going to be severely limited.

        We will indeed be losing jobs to automation. but this is only a short term bad thing, and a long term good thing.

  3. Valkyrie Ice :
    I have to disagree on how fast these technologies can and likely will replace dependence on oil over the next decade. The efficiency of solar cells has been increasing by leaps and bounds, while simultaneously methods have been found to make them at fractions of the cost of previous methods. We are likely to be producing cells of 80 to 90% efficiency charging batteries capable of holding 10 to 100 times more electrical potential well within this decade, and be incorporated into nearly all electric devices by it’s end. You are basing objections on those technologies in their prior forms of last decade, probably due to being unaware of the current state of research.

    I am not sure which is unlikelier, 80% efficient cells by 2020 or 100x energy density batteries by 2020. Even in the very recent past there is no historical trend to support this unbridled optimism. This is the territory of Popular Science magazine covers, not informed projection.

    At the same time, 80% efficient cells are neither necessary nor sufficient for widespread terrestrial deployment of solar power. Conversion efficiency becomes an important limit only for space or weight constrained applications, though efficiency does affect costs due to the need for (e.g.) more mounting hardware and labor for physically larger systems. 20% conversion efficiency can yield about 1 kilowatt hour per day per square meter over large swaths of the USA. At that conversion efficiency, in much of the USA roof-tops have enough area to provide (on average) enough electricity for all household needs. 20% efficient cells based on abundant silicon are already commercially available. The challenge is mostly to get prices down. If by some miracle 80% efficient cells became available, it’s still the price per kilowatt hour that is crucial.

    I hope for improved batteries, though I have little hope for order-of-magnitude improvement this decade. Again decreased prices per kilowatt hour are more important at this point than absolute performance. The batteries in the Tesla Roadster would be good enough for widespread use if their cost could be slashed by 80%. Electric vehicles are a natural match to household or car-park solar since the vehicle needs a lot of battery capacity anyway, and it can buffer solar output locally instead of having to push to the grid when supply exceeds building demand. I think this is a desirable outcome and far more likely than grid independence for existing buildings. It also has a far more direct impact on oil consumption, since grid-tied electrical applications contribute very little to oil demand in the first place.

    1. They recently used the tobacco mosaic virus to create nanopatterned batteries with 10x the capacity of current L-Ion. It has ALREADY been done. Your pessimism is misplaced. All that needs to happen is scaling up to industrial levels, which could be done inside five years

      Carbon Nanotubes have also been proven to function as ultracapacity batteries, and the cost for making them is dropping rapidly. So we might not even be using L-Ion in five years, which will dramatically lower costs as we will no longer need lithium, which is expensive because it is a rare earth.

      Solar Cells are also dropping in cost, and various technologies are improving their efficiency. I recently read an article about using Quantum Dots to fine tune layers of a cell to absorb a single frequency of light so that the same cell can convert far more of the light into electricity, and another about a backing material which can convert the otherwise wasted heat produced by the cell directly into electricity as well.

      I understand your pessimism, but I am constantly researching these technologies because I write about them.

      1. Super-efficient quantum dot photovoltaic devices have been talked about for at least 10 years. In theory they can go up to 65% efficiency, compared with the best demonstrated result of only 27.6% for silicon. But I have yet to read of a quantum dot device with conversion over 5% in the laboratory. The record as of 2008 was 3.6%. This is well short of even the cheapest silicon or thin film cells. Even if lab efficiency improves, commercial cells lag the lab by at least a few years. I have no doubt that as of 2020 the most economical (joules per dollar) commercial solar cells will have conversion efficiencies under 40%, less than half of your estimate. And I am further confident that even the most expensive and exotic laboratory cell will be under 70%.

        But as I said before, that doesn’t preclude a global solar boom. Cost per unit of electricity produced is much more important than conversion efficiency, at least for stationary terrestrial use.

  4. This is a fascinating place. To take this discussion back to robots (nothing against energy chemistry)…

    I read the book but was a bit nonplussed by its conclusion. Yes work will be automated increasingly. But human wants are limitless. Once robots take over jobs, we’ll fix robots for income. Then we’ll get paid to build more robots to do more work. Once robots build themselves, we’ll all get paid to design robots. When robots start designing themselves, we’ll be designing spaceships and planetary-scale machines that we live on (mini-planets). All of this is thousands of years away (or hundreds at least.) After all, if robots build us stuff cheaply and efficiently, won’t we just want more and more of it?

    In the meantime, we’ll continue to do things like advertising, painting, cooking, massage, greeting, consulting, coaching, and the myriad other services that human beings do for other human beings.

    It’s just manufacturing that is being automated away, not work itself.


    1. I agree. The KINDS of jobs are going to change drastically, but there will be those who create new kinds of jobs. Additionally Knowledge Based AI systems will not only replace numerous trained workers in the upper class (Doctors, Lawyers, Financial consultants, Corporate CEOS, etc) it will enable people to be able to competently use skills that have previously been limited to only a few.

      For example, an art program could be setup to “draw” in a certain artist’s style, say Da Vinci. The novice user of said art program could be drawing a stick figure, but the program could take that, apply all the rules of Da Vinci’s style, and create a finished work. It will allow those who previously lacked the various mental and physical skills to be an artist to express creativity.

      But more than that, think about the difficulty of programming an MMO. Now think about a program that will allow you to design EVERYTHING about an MMO, from landscape to monsters, using easy to use intuitive controls, and that will take your designs, follow your instructions, and write all the program code needed to make it a functional game.

      These things WILL be tools used for a radically different concept of “Work”.

      The problem is that PRIOR to that, they will lead to a destruction of the current economic system, which depending on the speed of that destruction, could lead to widespread suffering prior to the establishment of new economic models.

      However, since the next group of people who are likely to be replaced in the job market are the lower levels of the “Wealthy” (doctors, lawyers, Financial advisiors, CEOs, etc) the backlash caused by these people losing their privileged status as “wealthy” will likely lead to a universal “unemployment benefit” as they do a 180 about face and begin screaming for those benefits they are currently so eager to deny the unemployed now. This should help mitigate the worst suffering until the new economic paradigms are established.

    2. Jason: please re-read the section “‘Software’ Jobs and Artificial Intellignce” starting on page 67 of the book, this section makes it abundantly clear that it is NOT “just manufacturing that is being automated away”.

      Valkyrie Ice, here is the seed of the technology to replace doctors, lawyers, financial advisors and CEOs:

      Advertising, advertising is a complex mental skill and won’t disappear until we start to see AGI (artificial general intelligence) get better, but with A/B testing and more advertising moving to the web, we will see less skilled advertisers become much more effective and a lot of the money go out of advertising.
      Cooking, the robots above could be taught to cook if they can recognize color, they couldn’t do novel cooking but could replace all cooks at all chain restaurants.
      Painting, there are abundant painting robots now, with cheaper vision and object recognition this job will disappear.
      Greeting, like at wal-mart? this is barely a job, after delivery vehicles are automated, wal-mart will be under priced by a company that only delivers and doesn’t have stores.
      Consulting and coaching (coaching is a consulting specialty); these are similar to advertising, and will have to wait till AGI is better.

      So… no it is not just manufacturing that is going to be automated away, work itself is being automated away, but we’re going to see it first in manufacturing, then driving (look at the DARPA grand challenge results and Standford’s audi TT that drove up Pike’s Peak).

      1. While those jobs are indeed somewhat more complex, they are still amenable to rules based systems. While a low level AGI might not be able to compete with a good advertiser, a small business armed with one could still do far better with an automated advertising bot than it would without, which will impact lower tier advertising firms.

        As for the rest, well I just saw this article:

        Scott Kirsner’s Innovation Economy blog reports on Heartland Robotics

        Heartland Robotics targets a $5000 robot that is an alternative to low wage workers. It will be a hardware platform that will use an iPhone application model to customize its software.

        The one-armed Obero robot is the basis from which this two armed robot is being developed.

        Since the company isn’t talking, I spoke to three people who have visited Heartland and seen demos of its technology. All were impressed — “The road map is pretty compelling,’’ said one. They had questions, though, about how long it will take to get the product ready for customers, and how to market a general-purpose robotic technology. It is not focused on addressing a specific pain point that one particular industry faces (such as the robotic agricultural helpers being developed by Harvest Automation Inc., another Massachusetts robotics company that has raised significant funding).

        Visitors to Heartland describe a robot that looks like a human from the waist up, with a torso; either one or two arms with grippers; and a camera where you might expect the head to be. The robot is on a rolling base rather than legs; it can be moved around but doesn’t move autonomously. The arm and gripper can be quickly trained to do a repetitive task just by moving them, no software code required.

        And I’m told the robot has a sense for when people get close, so it doesn’t pose a safety hazard to humans working alongside it.

        The company is apparently targeting a $5,000 price point.


    Service with a smile also comes with an electronic voice at the Dalu Robot restaurant, where the hotpot meals are not as famous yet as the staff who never lose their patience and never take tips.

    The restaurant, which opened this month in Jinan in northern Shandong province, is touted as China’s first robot hotpot eatery where robots resembling Star Wars droids circle the room carrying trays of food in a conveyor belt-like system.
    More than a dozen robots operate in the restaurant as entertainers, servers, greeters and receptionists. Each robot has a motion sensor that tells it to stop when someone is in its path so customers can reach for dishes they want.

    The service industry in China has not always kept up with the country’s rapid economic growth, and can be quite basic in some restaurants, leading customers in the Dalu restaurant to praise the robots.

    “They have a better service attitude than humans,” said Li Xiaomei, 35, who was visiting the restaurant for the first time.

    “Humans can be temperamental or impatient, but they don’t feel tired, they just keep working and moving round and round the restaurant all night,” Li said.
    Inspired by space exploration, robot technology and global innovation, the restaurant’s owner, Zhang Yongpei, said he hopes his restaurant will show the world China is a serious competitor in developing technology.

    “I hope this new concept shows that China is forward-thinking and innovative,” Zhang said.

    As customers enter the dimly lit restaurant lined with blinking neon lights to simulate a futuristic environment, a female robot decorated with batting eyelashes greets people with an electronic “welcome.”

    During the meal, crowds of up to 100 customers, are entertained by a dancing and talking robot that looks more like a mannequin with a dress, flapping its arms around in a stiff motion.

    Zhang said he hopes to roll out 30 robots – which cost $6,000 each – in the coming months and eventually develop robots with human-like qualities that serve customers at their table and can walk up and down the stairs.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s