[
https://issues.apache.org/jira/browse/DRILL-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17608728#comment-17608728
]
ASF GitHub Bot commented on DRILL-2362:
---------------------------------------
jnturton commented on PR #1750:
URL: https://github.com/apache/drill/pull/1750#issuecomment-1256153990
Hi Kunal! Thank you for this contribution, I'd like to help to move it
forward.
> However, I am not sure if we should automatically index any new profile
that has been copy-pasted into the root dir. For e.g., we might get a profile
from a JIRA and would like to view it. Should we leave it there (and try to
render it) or should we index it ASAP ?
I share your concern here. I think we should consider having Drill only
write _new_ profiles to partitioned directories. Any partitioning of historical
profiles can be done externally by admins, in my opinion, and we can add
examples of "housekeeping" scripts for doing that to the Drill documentation.
Would you like to do any of the following?
- Resume the work on this PR, I volunteer to be a reviewer.
- Receive a PR from me to your fork here that's rebased and has some changes
I think we want.
- You're too busy now, so I'll pull your commits here into a new branch of
my own and open a new PR.
Thanks
James
> Drill should manage Query Profiling archiving
> ---------------------------------------------
>
> Key: DRILL-2362
> URL: https://issues.apache.org/jira/browse/DRILL-2362
> Project: Apache Drill
> Issue Type: New Feature
> Components: Storage - Other
> Affects Versions: 0.7.0
> Reporter: Chris Westin
> Priority: Major
>
> We collect query profile information for analysis purposes, but we keep it
> forever. At this time, for a few queries, it isn't a problem. But as users
> start putting Drill into production, automated use via other applications
> will make this grow quickly. We need to come up with a retention policy
> mechanism, with suitable settings administrators can use, and implement it so
> that this data can be cleaned up.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
