Hi authors of fastbit, This is my first post. Thanks for your wonderful job!
Fastbit do help me solved a performance problem which need select users from a huge user database with so many characters quickly. After that, I still have some idea to make Fastbit better. My topic is about the complex condition support. In our use case, we need select from billions of users which have thousands of characters (chocolate lover for example), and we use a very long condition clause (more than 1000 and/or items). All character is a boolean value with yes/no In my first attempt of fastbit, I found that unfortunately the condition can't support more than 200 and/or items. After further study, I found that the reason is Stack overflow at ibis::qExpr::simplify. This function is a recursion function. A very long condition make too deep recursion which cause the stack exhaust. Then I expand the stack size to 20M bytes. That make my program support 10 thousands or/and items. The approach solved my problem but not elegant. The best solution is eliminate the recursion of ibis::qExpr::simplify. I think It's a realizable. The refactoring can significant expand the limit of Fastbit which make Fastbit more useful. Can you consider this idea? Another pity is no BOOLEAN data type support. Would you consider to support it in future version? cheers, Davey
_______________________________________________ FastBit-users mailing list [email protected] https://hpcrdm.lbl.gov/cgi-bin/mailman/listinfo/fastbit-users
