A child can speak after hearing only a few million words in the first few years; training a large model requires trillions of words. Where this gap comes from is still not fully understood.
The Princeton research team asked this question differently. They installed cameras and microphones in several homes and recorded most of the children's daily lives from birth to nearly three years old. Then build a model for each of the 8 children. Each model only learns what the child heard that day, and moves forward day by day.
The model starts from scratch, has no idea where one word ends and the next begins, and has never seen text.
The results are poor at first: what you hear today you will barely remember tomorrow. By the time he was equivalent to two years old, he had not learned a single word.
The turning point was adding a sleep-like mechanism to it. At "evening", the model replays the most recent recording bit by bit. After that, my learning speed increased, I mastered the pronunciation system of English, and I could recognize thousands of words by the age of two.
All 8 families got through, and it didn’t matter that the conditions were different. New problems arise: real children develop at different speeds, but the model shows the same speed in every family. The researchers adjusted three parameters—how much to replay at night, how quickly to learn, and when to start memorizing words—and then each model kept up with its own child.
Project leader Uri Hasson means that for more than a hundred years psychology has been observing how children grow up from the outside. Now we can do the opposite: build a learning system, feed it the real experiences of children, and see if it will grow up in the same way.