#general
@chundong.wang: I’m wondering if there’re way other than groovy to get filter like “past 7 days” to work? Found
@npawar: Kishore’s talk on Star-tree index, happening now! Please join if you’re available:
#troubleshooting
@mnk666yura: Hi, I was playing with Pinot and noticed something strange with group by on certain column (some results are missing).
@mnk666yura: ```select visitor_id, count(*) from visitor_ids group by visitor_id order by count(*) desc```
@mnk666yura: gives: ```visitor_id count(*) 78144a788f39b7075bb37758 44 979ba2f45c229ab908bdd829 26 db5e6a7b6b48b64d3df838a7 22 3402d34497b08b5d0172128f 21 4f212877c876191c7db40a11 20``` But if specify missing visitor_id, it is there ```select visitor_id, count(*) from visitor_ids where visitor_id = '6d797e84d6d19d5825ef7060' group by visitor_id order by count(*) desc``` gives: ```visitor_id count(*) 6d797e84d6d19d5825ef7060 66```
@g.kishore: whats the cardinality of visitorId
@mnk666yura: column.visitor_id.cardinality = 148594 column.visitor_id.totalDocs = 190000
@g.kishore: I think we limit 100k unique visitors within a segment
@g.kishore: if that visitor id is not found in first 100k unique ids we see in a segment it wont show up
@g.kishore: you can change this config
@g.kishore: ```num.groups.limit"```
@g.kishore: @mayanks can you confirm the above behaviour^^
@mayanks: Yes, default number of group limits to 100k
@mnk666yura: @g.kishore Thank you, works as expected.
@g.kishore: we plan to make this a bit smarter but still have a limit
@singalravi: Hi, How should we manage idealstate znode for a table having a large number of segments? One of our real time table has more than 18k segments and zookeeper node size is greater than 3.2 MB. zookeper is supposed to have nodes of size less than 1 MB. will it have a negative impact on pinot controller performance when it read or write (during segment completion) ideal state of the table?
@g.kishore: Not really, but we just running the minion tasks to merge the segments and possibly move to offline table
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
