#pinot-helix
@syedakram93: @syedakram93 has joined the channel
#troubleshooting
@syedakram93: enabled authorization in my setup, and successfully added schema, while trying to ingest data, i am getting below error. *bin/pinot-admin.sh LaunchDataIngestionJob -jobSpecFile /home/sas/apache-pinot-incubating-0.8.0-SNAPSHOT-bin/examples/batch/airlineStats/ingestionJobSpec.yaml* Got exception to kick off standalone data ingestion job - java.lang.RuntimeException: Failed to read from Schema URI - '
@syedakram93: can someone please help?
@nadeemsadim: is there some way to delete few rows in pinot table .. means i want to get rid of some garbage data consumed into pinot .. so can i run some delete query to delete only rows matching that query
@mayanks: Deletion is at segment level, not row level
@nadeemsadim: ok
@jmeyer: Hello :smile: Does Pinot have any support for hierarchical aggregations ? Say we have a tree like structure, with values at the leaves, is there an efficient way to get the values at the intermediates & root nodes ? (Tree size can be ~10 layer deep, with ~10000-100000 leaves)
@mayanks: Have you looked at Star tree index?
@jmeyer: I've read a bit about it, not sure it could fit that use case ? (looks like it could :slightly_smiling_face:)
@mayanks:
@jmeyer: Do you have pointers on how to configure it / how the data should look like to make it work in that scenario ? Maybe an important detail is that I should have said "_tree*S"*_ Ideally only known at runtime (not integration time) I'll look into it first
@shaileshjha061: Hi @mayanks @dlavoie Something wrong with GCS Integration. Pinot server Pods went into crashloopbackoff when I try to integrate with GCS Error Log: ```ERROR [PinotFSFactory] [Start a Pinot [SERVER]] Could not instantiate file system for class org.apache.pinot.plugin.filesystem.GcsPinotFS with scheme gs``` @mohamed.sultan
@dlavoie: A full stacktrace from the logs always help to troubleshoot.
@shaileshjha061: let me send
@mohamed.sultan: found the issue @dlavoie
@mayanks: @mohamed.sultan Could you please share the issue? Also, we are request help from community to help improve our docs. Would you be intrested in documenting what you found? If so, please join <#C023BNDT0N8|pinot-docsrus>
@jmeyer: Does any sort of query cache exist in Pinot ?
@mayanks: No, there isn't one right now. For realtime cases, the data is changing in realtime, so caching the query result might not make sense.
@jmeyer: Yes in the case of realtime table the case would be invalidated very often In case of offline tables, I guess the best way is to add an external cache at the REST API layer ? And invalidate it whenever new segments are pushed
@mayanks: Added to FAQ:
@jmeyer: Looks great and clear, thanks @mayanks!
@machhindra.nale: Trying to browse Pinot in Superset. Following query works fine in the Pinot Query console. BUT fails in Superset. Notice the special character in the metric. Is that something because of Pinot SQLAlchemy driver? ```SELECT DATETIMECONVERT(metricTime, '1:MILLISECONDS:EPOCH', '1:MILLISECONDS:EPOCH', '1:MINUTES'), AVG(metricValue) AS "AVG_1" FROM metric_v6.metric_v6 WHERE metricTime >= 1621555200000 AND metricTime < 1622160000000 AND metric = 'CECCP%' GROUP BY DATETIMECONVERT(metricTime, '1:MILLISECONDS:EPOCH', '1:MILLISECONDS:EPOCH', '1:MINUTES') LIMIT 10000``` Screenshot of the Superset SQL Editor -
@mayanks: Can you check broker log to see what query was sent to Pinot?
@fx19880617: Is it the % ?
@fx19880617: It could be the python client translation
#pinot-docs
@syedakram93: @syedakram93 has joined the channel
#presto-pinot-connector
@syedakram93: @syedakram93 has joined the channel
#minion-improvements
@laxman: Have a basic tuning related question. Whats the recommended size for REALTIME and OFFLINE tables?
@g.kishore: Segment size or table size
@laxman: segment size
@g.kishore: 100 to 500mb
@laxman: I had gone through this documentation
@laxman:
@laxman: 100 to 500 MB for REALTIME right. How about OFFLINE segments?
@laxman: is it same for both?
@npawar: yes, same for both
@laxman: okay. thank you
#pinot-docsrus
@patidar.rahul8392: @patidar.rahul8392 has joined the channel
@syedakram93: @syedakram93 has joined the channel
@jmeyer: @jmeyer has joined the channel
@mayanks: Thanks for joining folks. I recommend helping out questions being asked by checking if it could have been answered by doc or faq.
@mayanks: Don't by shy folks :grin:, let's get the party started. @jmeyer You asked a question where star tree could be the answer. Perhaps that could go into the FaQ?
@jmeyer: Haha I'm still trying to figure out how it would fit, but if I find a way to use star tree in my usecase, maybe we could add a similar example in the docs ? (as mine is quite different from the current one)
@mayanks: No worries.
@jmeyer: Anyway, this channel is a good move I think, I'll suggest improvements whenever I see some :slightly_smiling_face:
@mayanks: Here's a simple one for anyone to take. I already answered, but we can add it to FaQ:
@jmeyer: Yep, maybe (if not already present), have an introduction about the limitations & differences between OLTP & OLAP (in the context of Pinot of course) ?
@mayanks: Sure, go for it, file a PR.
@mayanks: I have added a link on how to contribute to the docs in the channel topic
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org For additional commands, e-mail: dev-h...@pinot.apache.org
