When training a neural network programmers are always on the lookout for something called "overfitting" when the AI seems to stop generalizing and just memorizes the training data; typically that's the point where the training stops. However, when a researcher at OpenAI was working with a small neural network and clear signs of overfitting occurred, he forgot to turn it off and then went on vacation. When he got back from vacation, he found to his surprise that the network had very dramatically improved its performance. It's like it had obtained a profound understanding of the data, he called it "Grokking''. I've had the experience, and you probably have too, of studying something for a long time and not understanding it, and then suddenly bang there is an aha moment and everything becomes clear. It sounds sort of like that.
How Do Machines ‘Grok’ Data? <https://www.quantamagazine.org/how-do-machines-grok-data-20240412/?mc_cid=d55f4d909a&mc_eid=1b0caa9e8c> John K Clark See what's on my new list at Extropolis <https://groups.google.com/g/extropolis> spg -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to everything-list+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/everything-list/CAJPayv1bisUdiRrT0N%3DA3djscRe8K%2B9-8w6tM02gywhmzzHC6w%40mail.gmail.com.
