Re: [VOTE] Add Dependabot to Drill
+1 On Mon, May 17, 2021, 6:21 AM Martin Tzvetanov Grigorov < mgrigo...@apache.org> wrote: > Hi, > > +1 > > Regards, > Martin > > On 2021/05/16 14:40:46, Charles Givre wrote: > > Hello all, > > I'd like to propose adding Dependabot to our commit process. If you > aren't familiar with Dependabot, it scans dependencies and alerts you to > dependencies that have vulnerabilities. I ran dependabot on Drill's > source, and found several rather serious CVEs associated with dependencies, > hence the PRs to update Guava, JUnit, and a few others. > > > > I know that these automated code quality tests aren't always the best in > terms of producing false positives, but I do think it is in general a good > thing to at least be aware of these kinds of issues so that we can resolve > them if they are deemed worthy. > > > > So... I'd like to call a vote. Would you like to add dependabot to > Drill's github repo? Please vote yes or no by Thursday. > > > > Thanks and Keep on Drilling! > > -- C > > > > >
Re: [ANNOUNCE]: New committer: Ankush Kapur
Thank you, looking forward to it. On Thu, Aug 6, 2020, 11:06 AM Igor Guzenko wrote: > Congratulations Ankush! > > On Thu, Aug 6, 2020 at 8:12 AM weijie tong > wrote: > >> Congratulations Ankush! >> >> On Thu, Aug 6, 2020 at 2:37 AM Charles Givre wrote: >> >> > The Project Management Committee (PMC) for Apache [PROJECT] has invited >> > Ankush Kapur to become a committer and we are pleased to announce that >> he >> > has accepted. >> > >> > Being a committer enables easier contribution to the project since there >> > is no need to go via the patch submission process. This should enable >> > better productivity. Being a PMC member enables assistance with the >> > management and to guide the direction of the project. >> > >> > Welcome Ankush! >> > -- C >> > >> > >> > >> >
[jira] [Created] (DRILL-7758) Handle User Provided Druid Query Intervals
Ankush Kapur created DRILL-7758: --- Summary: Handle User Provided Druid Query Intervals Key: DRILL-7758 URL: https://issues.apache.org/jira/browse/DRILL-7758 Project: Apache Drill Issue Type: Sub-task Components: Storage - Other Affects Versions: 1.18.0 Reporter: Ankush Kapur Currently, when the query is converted to a druid select query, the query interval defaults to the `minTime` and `maxTime` from the segments metadata fetched from the endpoint, `v1/datasources?simple` There is functionality to parse the event intervals provided by the user but it seems to be not working. This needs to be fixed so that a user can provide custom query interval. Most likely, this will be by having a druid storage plugin specific reserved word in the SQL which will be interpreted by the druid storage plugin as the user input for the query interval -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7757) Druid Storage Plugin Enhancements
Ankush Kapur created DRILL-7757: --- Summary: Druid Storage Plugin Enhancements Key: DRILL-7757 URL: https://issues.apache.org/jira/browse/DRILL-7757 Project: Apache Drill Issue Type: Improvement Components: Storage - Other Affects Versions: 1.18.0 Reporter: Ankush Kapur Now that the Druid Storage Plugin MVP has been submitted, there are few improvements/fixes that need to be performed to the plugin for general hygiene of the plugin and well as scaling drill queries on druid. This Jira item is to track such changes until the next release. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Druid Storage Plugin for Drill
Hi Charles, That makes sense to me. It would be helpful to use a common HTTP module. I used the `DefaultHttpClient`, which i am not too stoked about. As long as there is an interface to mock for the new http module, it would work great. - Ankush On Tue, May 5, 2020 at 8:53 AM Charles Givre wrote: > Hi Ankush, > Thanks for all your work on the Druid plugin. I'm going to try to do a > review over the weekend. I had a question/suggestion. Recently we > committed a storage plugin for REST APIs that use the OkHTTP3 library as > the basis for HTTP requests. In the quest to get it committed there was > some discussion about the fact that Drill uses several different libraries > for making HTTP REST requests. > > I wanted to ask your thoughts about perhaps moving this class from the > HTTP plugin to a utility class and then having both the Druid and HTTP > plugin use this class for REST requests? > What do you think? > -- C > > > > On Apr 27, 2020, at 12:55 AM, Ankush Kapur wrote: > > Making good progress with the changes requested from the review(s). > > - Ankush > > On Sun, Apr 19, 2020 at 12:36 AM Charles Givre wrote: > >> I don't mean to be a pain in the you know where about this... I very >> much would like to see this as a part of Drill. >> Best, >> -- C >> >> >> >> On Apr 18, 2020, at 4:54 PM, Ankush Kapur wrote: >> >> Apologies for being MIA, got hung-up at work really bad. >> >> Yes, I did start working on the changes requested in the review. Will act >> on the other things you mentioned, as well. >> >> Thank you for following-up though, helps if someone is watching over :) >> >> - Ankush >> >> On Fri, Apr 17, 2020 at 12:56 AM Charles Givre wrote: >> >>> Hi Ankush, >>> How's it going? I hope you are doing ok with all that is happening >>> around us. I hate to be a pest but I was wondering if you've had the >>> chance to work on the Druid storage plugin review? I'm happy to help with >>> some of the changes, but I didn't want to step on your toes or re-do things >>> you're currently working on. >>> >>> Would it be possible for you to rebase on the current master as well? >>> There have been some improvements unrelated to this plugin which will be >>> helpful as we move towards committing this. >>> Stay safe! >>> >>> -- C >>> >>> >>> >>> >>> >>> On Mar 22, 2020, at 8:34 AM, Ankush Kapur >>> wrote: >>> >>> Yes, thank you. both for your time reviewing... >>> >>> I am working on changes based on it. >>> >>> - Ankush >>> >>> On Fri, Mar 20, 2020 at 11:25 AM Charles Givre wrote: >>> >>>> Hi Ankush. >>>> I hope all is well. Were you able to see Paul and my review comments >>>> for the Druid plugin? >>>> Thanks, >>>> -- C >>>> >>>> On Mar 13, 2020, at 8:17 AM, Charles Givre wrote: >>>> >>>> HI Ankush, >>>> I hope you're doing ok. I live in Maryland and all our schools are now >>>> closed for the next two weeks, so it will be interesting Anyway, I >>>> tagged you in a few comments, but it's weird that my comments didn't show >>>> up on the main page. Take a look here and you should see them: >>>> https://github.com/apache/drill/pull/1888/files >>>> >>>> General comments: >>>> 1. Please verify that all logger creations are in the correct format. >>>> Also, I'd strongly suggest avoiding info messages and use either debug, >>>> warn or error as the case may be. >>>> 2. I had a general question about security and passing credentials to >>>> Druid. I don't really know how Druid handles authentication, but it didn't >>>> seem like there was any way to pass creds to Druid. In any event, this >>>> will need to be documented in the README file. >>>> 3. Please go through and remove any commented out code. >>>> >>>> Writing a storage plugin is not easy, and I'm really impressed that >>>> your first contribution to Drill is something of this scale. This is >>>> really nice work and I'm really hoping we can get it committed for the next >>>> release. >>>> Thanks, >>>> -- C >>>> >>>> >>>> On Mar 13, 2020, at 5:56 AM, Ankush Kapur >>>> wrote: >>>> >&g
Re: Druid Storage Plugin for Drill
Making good progress with the changes requested from the review(s). - Ankush On Sun, Apr 19, 2020 at 12:36 AM Charles Givre wrote: > I don't mean to be a pain in the you know where about this... I very much > would like to see this as a part of Drill. > Best, > -- C > > > > On Apr 18, 2020, at 4:54 PM, Ankush Kapur wrote: > > Apologies for being MIA, got hung-up at work really bad. > > Yes, I did start working on the changes requested in the review. Will act > on the other things you mentioned, as well. > > Thank you for following-up though, helps if someone is watching over :) > > - Ankush > > On Fri, Apr 17, 2020 at 12:56 AM Charles Givre wrote: > >> Hi Ankush, >> How's it going? I hope you are doing ok with all that is happening >> around us. I hate to be a pest but I was wondering if you've had the >> chance to work on the Druid storage plugin review? I'm happy to help with >> some of the changes, but I didn't want to step on your toes or re-do things >> you're currently working on. >> >> Would it be possible for you to rebase on the current master as well? >> There have been some improvements unrelated to this plugin which will be >> helpful as we move towards committing this. >> Stay safe! >> >> -- C >> >> >> >> >> >> On Mar 22, 2020, at 8:34 AM, Ankush Kapur wrote: >> >> Yes, thank you. both for your time reviewing... >> >> I am working on changes based on it. >> >> - Ankush >> >> On Fri, Mar 20, 2020 at 11:25 AM Charles Givre wrote: >> >>> Hi Ankush. >>> I hope all is well. Were you able to see Paul and my review comments >>> for the Druid plugin? >>> Thanks, >>> -- C >>> >>> On Mar 13, 2020, at 8:17 AM, Charles Givre wrote: >>> >>> HI Ankush, >>> I hope you're doing ok. I live in Maryland and all our schools are now >>> closed for the next two weeks, so it will be interesting Anyway, I >>> tagged you in a few comments, but it's weird that my comments didn't show >>> up on the main page. Take a look here and you should see them: >>> https://github.com/apache/drill/pull/1888/files >>> >>> General comments: >>> 1. Please verify that all logger creations are in the correct format. >>> Also, I'd strongly suggest avoiding info messages and use either debug, >>> warn or error as the case may be. >>> 2. I had a general question about security and passing credentials to >>> Druid. I don't really know how Druid handles authentication, but it didn't >>> seem like there was any way to pass creds to Druid. In any event, this >>> will need to be documented in the README file. >>> 3. Please go through and remove any commented out code. >>> >>> Writing a storage plugin is not easy, and I'm really impressed that your >>> first contribution to Drill is something of this scale. This is really >>> nice work and I'm really hoping we can get it committed for the next >>> release. >>> Thanks, >>> -- C >>> >>> >>> On Mar 13, 2020, at 5:56 AM, Ankush Kapur >>> wrote: >>> >>> Hi Charles, >>> >>> Things are good here, and hope the same for you and your family. >>> >>> Pardon me, but I do not see your review comments on the four files >>> mentioned. >>> >>> - Ankush >>> >>> On Thu, Mar 12, 2020 at 8:36 PM Charles Givre wrote: >>> >>>> Hi Ankush, >>>> I hope all is well. I started the review on your plugin. I've >>>> reviewed about four files so far. When you have a chance, please take a >>>> look at the review comments and if you have any questions, please let me >>>> know. >>>> Best, >>>> -- C >>>> >>>> >>>> On Feb 28, 2020, at 8:33 AM, Charles Givre wrote: >>>> >>>> Hi Ankush, >>>> I hope all is well. I've been speaking with Paul about your plugin and >>>> I'm going to do the code review. I usually don't do big code reviews like >>>> this, so I was hoping someone else would pick it up, but that hasn't >>>> happened and I really want to see this get committed to the next version of >>>> Drill. >>>> >>>> Since this is a large PR, I'm going to do this in small reviews rather >>>> than one massive review so please expe
Re: Druid Storage Plugin for Drill
Apologies for being MIA, got hung-up at work really bad. Yes, I did start working on the changes requested in the review. Will act on the other things you mentioned, as well. Thank you for following-up though, helps if someone is watching over :) - Ankush On Fri, Apr 17, 2020 at 12:56 AM Charles Givre wrote: > Hi Ankush, > How's it going? I hope you are doing ok with all that is happening around > us. I hate to be a pest but I was wondering if you've had the chance to > work on the Druid storage plugin review? I'm happy to help with some of > the changes, but I didn't want to step on your toes or re-do things you're > currently working on. > > Would it be possible for you to rebase on the current master as well? > There have been some improvements unrelated to this plugin which will be > helpful as we move towards committing this. > Stay safe! > > -- C > > > > > > On Mar 22, 2020, at 8:34 AM, Ankush Kapur wrote: > > Yes, thank you. both for your time reviewing... > > I am working on changes based on it. > > - Ankush > > On Fri, Mar 20, 2020 at 11:25 AM Charles Givre wrote: > >> Hi Ankush. >> I hope all is well. Were you able to see Paul and my review comments for >> the Druid plugin? >> Thanks, >> -- C >> >> On Mar 13, 2020, at 8:17 AM, Charles Givre wrote: >> >> HI Ankush, >> I hope you're doing ok. I live in Maryland and all our schools are now >> closed for the next two weeks, so it will be interesting Anyway, I >> tagged you in a few comments, but it's weird that my comments didn't show >> up on the main page. Take a look here and you should see them: >> https://github.com/apache/drill/pull/1888/files >> >> General comments: >> 1. Please verify that all logger creations are in the correct format. >> Also, I'd strongly suggest avoiding info messages and use either debug, >> warn or error as the case may be. >> 2. I had a general question about security and passing credentials to >> Druid. I don't really know how Druid handles authentication, but it didn't >> seem like there was any way to pass creds to Druid. In any event, this >> will need to be documented in the README file. >> 3. Please go through and remove any commented out code. >> >> Writing a storage plugin is not easy, and I'm really impressed that your >> first contribution to Drill is something of this scale. This is really >> nice work and I'm really hoping we can get it committed for the next >> release. >> Thanks, >> -- C >> >> >> On Mar 13, 2020, at 5:56 AM, Ankush Kapur wrote: >> >> Hi Charles, >> >> Things are good here, and hope the same for you and your family. >> >> Pardon me, but I do not see your review comments on the four files >> mentioned. >> >> - Ankush >> >> On Thu, Mar 12, 2020 at 8:36 PM Charles Givre wrote: >> >>> Hi Ankush, >>> I hope all is well. I started the review on your plugin. I've reviewed >>> about four files so far. When you have a chance, please take a look at the >>> review comments and if you have any questions, please let me know. >>> Best, >>> -- C >>> >>> >>> On Feb 28, 2020, at 8:33 AM, Charles Givre wrote: >>> >>> Hi Ankush, >>> I hope all is well. I've been speaking with Paul about your plugin and >>> I'm going to do the code review. I usually don't do big code reviews like >>> this, so I was hoping someone else would pick it up, but that hasn't >>> happened and I really want to see this get committed to the next version of >>> Drill. >>> >>> Since this is a large PR, I'm going to do this in small reviews rather >>> than one massive review so please expect waves of review. I think it's >>> best that way it doesn't get overwhelming. My game plan will be to focus >>> on a few files at a time rather than trying to look at the entire plugin >>> and give a few comments here and there. I'll send future correspondence >>> via the dev alias in the interest of transparency as well. I'm going to >>> ask Paul to look at the pushdown code as he is a lot more familiar with >>> that than I am, but we're not there yet. >>> >>> Thanks for all your work and patience on this and I am excited about >>> getting it into Drill. I know it will be a benefit for the community. >>> Best, >>> -- C >>> >>> >>> >>> On Jan
Re: Druid Storage Plugin for Drill
Yes, thank you. both for your time reviewing... I am working on changes based on it. - Ankush On Fri, Mar 20, 2020 at 11:25 AM Charles Givre wrote: > Hi Ankush. > I hope all is well. Were you able to see Paul and my review comments for > the Druid plugin? > Thanks, > -- C > > On Mar 13, 2020, at 8:17 AM, Charles Givre wrote: > > HI Ankush, > I hope you're doing ok. I live in Maryland and all our schools are now > closed for the next two weeks, so it will be interesting Anyway, I > tagged you in a few comments, but it's weird that my comments didn't show > up on the main page. Take a look here and you should see them: > https://github.com/apache/drill/pull/1888/files > > General comments: > 1. Please verify that all logger creations are in the correct format. > Also, I'd strongly suggest avoiding info messages and use either debug, > warn or error as the case may be. > 2. I had a general question about security and passing credentials to > Druid. I don't really know how Druid handles authentication, but it didn't > seem like there was any way to pass creds to Druid. In any event, this > will need to be documented in the README file. > 3. Please go through and remove any commented out code. > > Writing a storage plugin is not easy, and I'm really impressed that your > first contribution to Drill is something of this scale. This is really > nice work and I'm really hoping we can get it committed for the next > release. > Thanks, > -- C > > > On Mar 13, 2020, at 5:56 AM, Ankush Kapur wrote: > > Hi Charles, > > Things are good here, and hope the same for you and your family. > > Pardon me, but I do not see your review comments on the four files > mentioned. > > - Ankush > > On Thu, Mar 12, 2020 at 8:36 PM Charles Givre wrote: > >> Hi Ankush, >> I hope all is well. I started the review on your plugin. I've reviewed >> about four files so far. When you have a chance, please take a look at the >> review comments and if you have any questions, please let me know. >> Best, >> -- C >> >> >> On Feb 28, 2020, at 8:33 AM, Charles Givre wrote: >> >> Hi Ankush, >> I hope all is well. I've been speaking with Paul about your plugin and >> I'm going to do the code review. I usually don't do big code reviews like >> this, so I was hoping someone else would pick it up, but that hasn't >> happened and I really want to see this get committed to the next version of >> Drill. >> >> Since this is a large PR, I'm going to do this in small reviews rather >> than one massive review so please expect waves of review. I think it's >> best that way it doesn't get overwhelming. My game plan will be to focus >> on a few files at a time rather than trying to look at the entire plugin >> and give a few comments here and there. I'll send future correspondence >> via the dev alias in the interest of transparency as well. I'm going to >> ask Paul to look at the pushdown code as he is a lot more familiar with >> that than I am, but we're not there yet. >> >> Thanks for all your work and patience on this and I am excited about >> getting it into Drill. I know it will be a benefit for the community. >> Best, >> -- C >> >> >> >> On Jan 21, 2020, at 8:43 AM, Charles Givre wrote: >> >> Hey Ankush, >> See responses inline. >> >> >> On Jan 21, 2020, at 8:12 AM, Ankush Kapur wrote: >> >> Hi Charles, >> >> Thanks for taking time to make the changes. >> >> >> No problem. Also FYI, Drill uses 2 space indents not 4. I just pushed >> an update with all the spacing on 2 space. Is there anything that I can >> assist with? >> >> >> Could you please point me to the "commented-out" code you are referring >> too. That's actually an issue i was trying to track down last weekend while >> writing tests. I noticed that the "where" clause was not being pushed down. >> This was keeping me from writing the tests I have been working on. >> >> >> The commented out code was in the DruidStoragePlugin, but it does not >> appear to be commented out anymore. Have you rebased on the most current >> branch? When I attempted to build your branch, I got an error in the >> RecordReader class. I'll take another look today or tomorrow. Regarding >> pushdowns, have you seen the Calcite adapter for Druid ( >> https://calcite.apache.org/docs/druid_adapter.html). I'm wondering if >> you can basically extend
