Indeed, if someone removes a dozen char/word types in the training/test set if they use Perplexity as evaluation, they can get a higher score. To work/compare with enwiki8 you must either decompress it losslessly or train on it in full and in full on test set ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T2a0cd9d392f9ff94-Mad032fa4e019de2c87d66d51 Delivery options: https://agi.topicbox.com/groups/agi/subscription
- Re: [agi] The limitations of the validit... James Bowery
- Re: [agi] The limitations of the va... immortal . discoveries
- Re: [agi] The limitations of th... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... stefan.reich.maker.of.eye via AGI
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... Matt Mahoney
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... immortal . discoveries
- Re: [agi] The limitations o... James Bowery