leerho commented on PR #191: URL: https://github.com/apache/datasketches-rust/pull/191#issuecomment-5351985821
What kind of tests did you run? There are valid cases were the table would be left empty after a purge. For example, if the table size is 1000 and all the updates are the same value. Or if all the updates are close to the same value and you have very few very large values. If the sampling process misses those large values the table will likely end up empty and the sketch will likely report zero most frequent values. Why? because the distribution is essentially flat and the very few heavy values aren't picked up by the random sample of the table and the values left after the purge may not enough to exceed the floor detection threshold of the sketch based on the given size of the table. So the sketch will report zero most frequent values. Which is correct. I think your question is more subtle. If the table is empty, and yet there is some residual history that is not in the table, should the sketch report isEmpty()= true? Well, the table is empty and the sketch will likely report 0 most-frequent values. So at that point in time, it is effectively empty. Having some residual history would allow you to continue updating the sketch from the stream. Empty in this case is not the same as a virgin sketch. It is just saying that it has no samples to compute anything from. Now, perhaps you found a bug and we should discuss it more. But I'd like more information about the exact tests you ran -- and perhaps, if I have time, I will try and repeat them and dig a litter deeper. Cheers< Lee. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
