Opinion/The difference between human and machine »intelligence«…

Humans have a curious property in how they go about puzzles; even formal ones, by »deriving a number of theorems, quite at random, just to see what kind of things turn up. Pretty soon, they begin to notice some properties of the theorems they have made; that is where human intelligence enters the picture.« ¹ Even for rule systems based on very simple generative procedures, it is often not obvious, until trying a few derivations, what patterns emerge, leading one not only to see the patterns, but a level of understanding by looking at the rules; ultimately, then, all theorems' can be traced back to their often very basic axioms.

Wherever this capability comes from, there is something significant about what happens when it operates. »It shows one difference between people and machines. It [is] certainly possible—in fact it would be very easy—to program a computer to generate theorem after theorem of an [axiomatic ]system; and we could include in the program a command to stop only upon generating« some derivation that we wonder if the system can deduce. However, we now know that a computer so programmed might never stop. That wouldn’t surprise anyone these days, in particular after Alan Turing proved in 1937 that the halting problem is undecidable. ² Asking a human on the other hand it would not be surprising »if he came back after a while, complaining that he can’t get rid of [some invariant loop condition], and therefore it is a wild goose chase,« and therefore it seems the program will never terminate. »Even if a person is not very bright, he still cannot help making some observations about what he is doing, and these observations give him good insight into the task—insight which the computer program […] lacks.«

With this notion, it is possible to be more explicit about what is meant by saying that this shows a difference between people and machines. »It is possible to program a machine to do a routine task in such a way that the machine will never notice even the most obvious facts about what it is doing; but it is inherent in human consciousness to notice some facts about the things one is doing. But you knew this all along. If you punch “1” into an adding machine, and then add 1 to it, and then add 1 again, and again, and again, and continue doing so for hours and hours, the machine will never learn to anticipate you and do it itself, although any person would pick up the repetitive behavior very quickly. Or, to take a silly example, a car will never pick up the idea, no matter how much or how well it is driven, that it is supposed to avoid other cars and obstacles on the road; and it will never learn even the most frequently traveled routes of its owner.«

»The difference, then, is that it is possible for a machine to act unobservant; it is impossible for a human to act unobservant. Notice [it is not being said] that all machines are necessarily incapable of making sophisticated observations; just that some machines are. Nor [is anyone claiming] that all people are always making sophisticated observations; people, in fact, are often very unobservant. But machines can be made to be totally unobservant; and people cannot. And in fact, most machines made so far are pretty close to being totally unobservant. Probably for this reason, the property of being unobservant seems to be the characteristic feature of machines, to most people. For example, if somebody says that some task is “mechanical”, it does not mean that people are incapable of doing the task; it implies, though, that only a machine could do it over and over without ever complaining, or feeling bored.«

»It is an inherent property of intelligence that it can jump out of the task which it is performing, and survey what it has done; it is always looking for, and often finding, patterns. [Whilst] an intelligence can jump out of its task, [it] doesn’t mean that it always will. However, a little prompting will often suffice. For example, a human being who is reading a book may grow sleepy. Instead of continuing to read until the book is finished, he is just as likely to put the book aside and turn off the light. He has stepped “out of the system” and yet it seems the most natural thing in the world to us. Or, suppose person A is watching television when person B comes in the room, and shows evident displeasure with the situation. Person A may think he understands the problem, and try to remedy it by exiting the present system (that television program), and flipping the channel now, looking for a better show. Person B may have a more radical concept of what it is to “exit the system”—namely to turn the television off! Of course, there are cases where only a rare individual will have the vision to perceive a system which governs many peoples' lives, a system which had never before even been recognized as a system; then such people often devote their lives to convincing other people that the system really is there, and that it ought to be exited from!«

»How well have computers been taught to jump out of the system? [...] »If you think of "the system" as being "whatever the computer had been programmed to do", then there is no doubt that the computer has no ability whatsoever to exit from that system.« [^1]