Are you saying BPC = Lossless Compression? So openAI had GPT do lossless compression then to get a BPC score? But I don't think they did lossless compression...
Doesn't BPC need an arithmetic function? I mean you have a set of predicted probabilities for each letter (or word/token, which need to be divided down to letters), then....oh...wait I see something here....this is to make the file/dataset smaller to prove it has a good predictor, while: BPC seems to be way to show how good it predicts, without any compression... For example say we predict a=60% likely next, b=30%, c=10%. Now, instead of an arithmitic coder to give us a long d5D83nF#jsiv that encodes each letter, we instead say OK, next letter is 'c' (it sucks at predicting this one), so we got 10% here then.....then for each next letter: 30%, 20%, 40%, 10%, 10%....then we can add em all up or average to get let's see: we have 500% to eat at down to be perfect predictor, and ate down 110% in total, so: 0.22 score. Right??? This seems cool, then can't we do this instead of compression? It'd be much faster/simpler I think. ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Td13a829978c4c9f3-M9a6ec6d0a7a512a93976da73 Delivery options: https://agi.topicbox.com/groups/agi/subscription
