siddharthteotia commented on pull request #6845: URL: https://github.com/apache/incubator-pinot/pull/6845#issuecomment-830272565
@mcvsubbu @yupeng9 , unlimited (however deep the hierarchy is) recursing should be allowed since that was one of the things we had discussed in the design doc. If the user knows what they are doing (storage explosion) and chooses to flatten everything outright, then they get tabular data with all leaves as individual columns thus allowing all SQL to work on nested data. No need to use JSON index in such cases. It is ok to have a limit (as eventually we might run out of stack memory and server might crash). User should go back and update the flatten config to may be choose a different sub-tree that needs to be flattened or specify the depth of hierarchy upto which we should flatten. In any case, our algorithm should be robust enough to handle any arbitrary level of flattening with the depth being configurable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
