Skip to main content
November 1 (Wed) 13:30-14:30
ACCURATELY DETERMINING WHAT AI CAN DO AND CANNOT DO FOR ITS APPLICATION IN SOCIAL INNOVATION
Professor, National Institute of Informatics
Director, Research Center for Community Knowledge,
National Institute of Informatics
Noriko Arai, National Institute of Informatics
Noriko Arai explains the workings of AI
Professor Arai began her speech establishing the premise that 'computers are based on mathematics and mathematics comprise just three languages: the language of logic; the language of probability; and the language of statistics'.
The first half of her speech discussed the mechanisms of statistical machine learning, including deep learning. Expected loss requires thinking based on all data, but since it is impossible to acquire all data, a sample must be used. The reality of machine/deep learning is that it tends to assume that elements of the sample with the smallest loss empirically are the smallest in actuality. On the other hand, using deep learning often automatically brings the ability to the optimum levels, carrying out craftsman-like tuning.
When the Todai Robot Project began in 2011, another AI project had just won on a TV quiz show, in which answers were centered on numbers and proper nouns. However, the professor explained, the AI was not comprehending the questions. Rather, it picked up on important keywords in the question, searched using them, and then answered by selecting words with the highest probability of being correct.
National Center Test for University Admissions could not be solved with this method; there is too much variation in what constitutes a correct answer.
Professor Arai explained, "The difficulty with National Center Test is that they are 70% true-or-false judgment type questions. In the multiple-choice section, the examinee must choose which statement is true or false. We programmed Todai Robot so it can devise its own fill-in-the-blank question based on the provided true-or-false questions and search for the answer. Doing that made it possible to achieve a score of 76 out of 100 in just one year, exceeding the deviation score of 66.5. You see, results depend on how you use something."
If you ask your smart phone, 'is there a good Italian restaurant nearby?' it will come up with some good suggestions. However, if you ask the same question replacing 'good' with 'bad' it will still provide you with the same answer. Further, if you ask for 'a restaurant other than Italian' it won't suggest you Japanese or Chinese restaurants, but rather pizzerias. In other words, it does not understand the meaning of 'other than'. As she explained, this proves that AI still cannot understand natural language.
This would lead you to believe that Todai Robot would have a difficult time with a written test. However, it was able to write an essay that achieved a mid-level result in a mock entrance exam for the University of Tokyo (abbreviated as “Todai”). Todai Robot memorized massive amounts of data from textbooks and other sources, and used the keywords specified by Todai to infer which sentences to extract. It then organized them in chronological order and compiled the result into a 600-character essay.
So how about machine translation? The language model used in machine translation can guess the probability of what is going to be written next, but it cannot write a story by itself since it needs a topic to proceed. Still, it was effective for questions about word order. But when it came to fill-in-the-blank questions, it chose wrong answers.
Next, Professor Arai expressed concerns with people's overconfidence in AI. She cited a newspaper article that reported that neural networks imitated the human brain, pointing out that it was actually a mouse's brain that was used. She criticized this idea, stating that no matter how many mouse brains you amass, it's hard to imagine them engaging in politics or medical treatment.
She also criticized the opinion of robots having eyes. However, she explained that since efficiency can be achieved with the required precision, they do have a broad spectrum of applications such as image diagnosis and product checking.
Even so, a number of significant issues remain. For example, manufacturers are responsible for product liability and the cameras for this require a high accuracy. However, if standards for the installed camera change, it would no longer be able to utilize the training data, which was established at great cost.
She explained, "Human training data cannot be made automatically. Human training data is only useful for human society. This is because the expected correct answer is based on human common sense, human ethics, and human satisfaction."
Todai Robot has been able to solve mathematical problems with natural language processing. This has resulted in achieving the result within the top 20% of a mock entrance exam for the University of Tokyo. That being said, AI in its current form is not more intelligent than humans as AI cannot understand meaning and is not guaranteed to be correct. But there is an even larger problem: most of the students pursuing white-collar jobs in the future will score lower than AI.
She warns, "Many companies will be unable to gather people with extraordinary talent, which will impede innovation. The important takeaway is that we must question whether or not we can foster talent who have abilities that are differentiated from AI's."