GPT-2 uses a transformer network, like NNCP, the top ranked program on my benchmark. It would not meet the hardware constraints for the Hutter prize. NNCP uses a GPU. The Hutter prize requires 50 hours in a single thread and under 10 GB memory.
On Sun, Jul 18, 2021, 8:15 PM <immortal.discover...@gmail.com> wrote: > I revisited a thought and am confused and may see a way now. If we try > throwing GPT-2 at Matt's contest, why wouldn't it get top rank? It's RAM > that is big would be deleted after training no? Then upon decompression > would start training again from scratch. Why is the RAM too big after > compression? All you store is the steers for predictions for its slight > inaccuracy per letter, not the model, to get the compressed file for all > letters in the dataset. > > Also if you say it would be too hard to run it, we could for 100MB. Though > this isn't even a problem because you can still get the compression score > even if expensive and timely. > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/Tcc6753a33ad48f78-M7732782b4d807bee11bb4fef> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tcc6753a33ad48f78-M4e84f7a32410246766d98e9b Delivery options: https://agi.topicbox.com/groups/agi/subscription
