First think about the practical, legal and PR challenges of putting self-driving vehicles on public roads.

Putting autonomous robots in amongst humans is that problem, on steroids.

In my opinion the hardware isn't there yet for general purpose robotics and the cost of training the AIs to do the work is still too high. Maybe if the AI bubble pops we'll see GPUs going for cheap so robotic AI training will accelerate.