If the machines are permitted to make all their own decisions, we can’t make any conjectures as to the results, because it is impossible to guess how such machines might behave. We only point out that the fate of the human race would be at the mercy of the machines. It might be argued that the human race would never be foolish enough to hand over all power to the machines. But we are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power. What we do suggest is that the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines’ decisions. As society and the problems that face it become more and more complex and as machines become more and more intelligent, people will let machines make more and more of their decisions for them, simply because machine-made decisions will bring better results than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won’t be able to just turn the machines off, because they will be so dependent on them that turning them off would amount to suicide."
- Industrial Society And Its Future, Ted Kaczynski (1975)
I think it's meaningful to reflect on this today, when we have something man-made at least resembling human intelligence, to see where he was right or wrong.
See my comment for the rest, this hits the character limit.
Aerbil313•6h ago
But suppose now that the computer scientists do not succeed in developing artificial intelligence, so that human work remains necessary. Even so, machines will take care of more and more of the simpler tasks so that there will be an increasing surplus of human workers at the lower levels of ability. (We see this happening already. There are many people who find it difficult or impossible to get work, because for intellectual or psychological reasons they cannot acquire the level of training necessary to make themselves useful in the present system.) On those who are employed, ever-increasing demands will be placed: They will need more and more training, more and more ability, and will have to be ever more reliable, conforming and docile, because they will be more and more like cells of a giant organism. Their tasks will be increasingly specialized, so that their work will be, in a sense, out of touch with the real world, being concentrated on one tiny slice of reality. The system will have to use any means that it can, whether psychological or biological, to engineer people to be docile, to have the abilities that the system requires and to “sublimate” their drive for power into some specialized task. But the statement that the people of such a society will have to be docile may require qualification. The society may find competitiveness useful, provided that ways are found of directing competitiveness into channels that serve the needs of the system. We can imagine a future society in which there is endless competition for positions of prestige and power. But no more than a very few people will ever reach the top, where the only real power is (see end of paragraph 163). Very repellent is a society in which a person can satisfy his need for power only by pushing large numbers of other people out of the way and depriving them of THEIR opportunity for power.
One can envision scenarios that incorporate aspects of more than one of the possibilities that we have just discussed. For instance, it may be that machines will take over most of the work that is of real, practical importance, but that human beings will be kept busy by being given relatively unimportant work. It has been suggested, for example, that a great development of the service industries might provide work for human beings. Thus people would spent their time shining each other’s shoes, driving each other around in taxicabs, making handicrafts for one another, waiting on each other’s tables, etc. This seems to us a thoroughly contemptible way for the human race to end up, and we doubt that many people would find fulfilling lives in such pointless busy-work. They would seek other, dangerous outlets (drugs, crime, “cults,” hate groups) unless they were biologically or psychologically engineered to adapt them to such a way of life.
Needless to say, the scenarios outlined above do not exhaust all the possibilities. They only indicate the kinds of outcomes that seem to us most likely. But we can envision no plausible scenarios that are any more palatable than the ones we’ve just described. It is overwhelmingly probable that if the industrial- technological system survives the next 40 to 100 years, it will by that time have developed certain general characteristics: Individuals (at least those of the “bourgeois” type, who are integrated into the system and make it run, and who therefore have all the power) will be more dependent than ever on large organizations; they will be more “socialized” than ever and their physical and mental qualities to a significant extent (possibly to a very great extent) will be those that are engineered into them rather than being the results of chance (or of God’s will, or whatever); and whatever may be left of wild nature will be reduced to remnants preserved for scientific study and kept under the supervision and management of scientists (hence it will no longer be truly wild). In the long run (say a few centuries from now) it is likely that neither the human race nor any other important organisms will exist as we know them today, because once you start modifying organisms through genetic engineering there is no reason to stop at any particular point, so that the modifications will probably continue until man and other organisms have been utterly transformed.
Whatever else may be the case, it is certain that technology is creating for human beings a new physical and social environment radically different from the spectrum of environments to which natural selection has adapted the human race physically and psychologically. If man is not adjusted to this new environment by being artificially re-engineered, then he will be adapted to it through a long and painful process of natural selection. The former is far more likely than the latter."