This decade’s data is the driving force behind a new university experiment in artificial intelligence.
Dr. Finn and her team built a neural network, a mathematical system that can learn skills from large amounts of data. By correctly identifying patterns in thousands of pictures of cats, a neural network can learn to identify a cat. By analyzing hundreds of old phone calls, it can learn to recognize spoken words. Or, by examining how the tutor evaluates coding tests, it can learn to self-assess these tests.
The Stanford system spent hours analyzing examples from old midterms, learning from a decade of possibilities. Then it’s ready to learn more. Given just a few additional examples from the new exam offered this spring, it can quickly capture the task at hand.
“It has many kinds of problems,” said Mike Wu, another researcher working on the project. “It can then adapt to problems it hasn’t seen before.”
This spring, the system provided 16,000 feedback, and students agreed with it 97.9% of the time, according to a study by Stanford researchers. By comparison, students agreed with the instructor’s feedback 96.7% of the time.
Pham, an engineering student at Lund University, Sweden, is amazed how well the technology works. Although the automated tool was unable to evaluate one of his programs (perhaps because he wrote a piece of code unlike anything the AI had ever seen), both identified errors. specifically in his code, including what is known in computer programming and math as fence post errors, and suggesting ways to fix them. “Rarely do you get such thorough feedback,” Pham said.
This technology is effective because its role is well defined. When taking the test, Mr. Pham wrote code with a very specific purpose in mind, and there were only so many ways he and other students could go wrong.
But with the right data, neural networks can learn a wide range of tasks. It’s the same basic technology that recognizes faces in photos you post to Facebook, recognizes commands you bark at your iPhone, and translates from one language to another on services like Skype and Google Translate. For the Stanford team and other researchers, the hope is that these techniques can automate education in other ways.