Fwd: Re Drill Issue
Folks, Apologise for overloading a Jira. I opened a new one https://issues.apache.org/jira/browse/DRILL-4653. Please use this. -- Forwarded message -- From: Subbu Srinivasan Date: Tue, May 3, 2016 at 5:29 PM Subject: Re Drill Issue To: dev@drill.apache.org Folks, Please see my notes. https://issues.apache.org/jira/browse/DRILL-4352 If the approach is acceptable I will look at adding this functionality for JSON, obviously more work needed for CVS type also. -- Pardon me for typos or if I do not start with a hi or address you by name. Want to make sure my carpel tunnel syndrome does not get worse. Subbu -- Pardon me for typos or if I do not start with a hi or address you by name. Want to make sure my carpel tunnel syndrome does not get worse. Subbu
Drill - 4641
Please look at https://issues.apache.org/jira/browse/DRILL-4641 -- Pardon me for typos or if I do not start with a hi or address you by name. Want to make sure my carpel tunnel syndrome does not get worse. Subbu
Re Drill Issue
Folks, Please see my notes. https://issues.apache.org/jira/browse/DRILL-4352 If the approach is acceptable I will look at adding this functionality for JSON, obviously more work needed for CVS type also. -- Pardon me for typos or if I do not start with a hi or address you by name. Want to make sure my carpel tunnel syndrome does not get worse. Subbu
Re: Time for a 1.7 release
Can https://issues.apache.org/jira/browse/DRILL-4653 be reviewed? On Wed, Jun 15, 2016 at 12:14 PM, Aman Sinha wrote: > Hello everyone, > > A 1.7 release is quite overdue ! I would like to get the ball rolling... > > In preparation, could developers please close the pull requests that have > already been merged into master ? This will give me a better sense of what > is in progress. > > Can all the folks working on open issues let me know if there are any > JIRAs you would like to get into the release? > >Tentatively, I am thinking beginning of next week to get a release > candidate. > > thanks, > -Aman >
Re: Time for a 1.7 release
Who can review https://issues.apache.org/jira/browse/DRILL-4653 ? On Wed, Jun 15, 2016 at 1:37 PM, Parth Chandra wrote: > +1 on the 1.7 release > > I'm reviewing the following and hope to get them in the release before > cutoff: > https://issues.apache.org/jira/browse/DRILL-2593 > https://issues.apache.org/jira/browse/DRILL-4309 > > > > On Wed, Jun 15, 2016 at 1:20 PM, Jinfeng Ni wrote: > > > I'm reviewing a follow-up PR [1] for DRILL-4573. I think we need get > > it merged in, since it's a regression in terms of query correctness > > from release 1.6. > > > > [1] https://github.com/apache/drill/pull/512 > > > > On Wed, Jun 15, 2016 at 12:21 PM, Dave Oshinsky > > > wrote: > > > This is a pretty basic bug affecting decimal values, with a simple fix: > > > https://issues.apache.org/jira/browse/DRILL-4704 > > > > > > It would be great if it could be reviewed. > > > > > > -Original Message- > > > From: Aman Sinha [mailto:amansi...@apache.org] > > > Sent: Wednesday, June 15, 2016 3:15 PM > > > To: dev > > > Subject: Time for a 1.7 release > > > > > > Hello everyone, > > > > > > A 1.7 release is quite overdue ! I would like to get the ball > > rolling... > > > > > > In preparation, could developers please close the pull requests that > > have already been merged into master ? This will give me a better sense > of > > what is in progress. > > > > > > Can all the folks working on open issues let me know if there are any > > JIRAs you would like to get into the release? > > > > > >Tentatively, I am thinking beginning of next week to get a release > > candidate. > > > > > > thanks, > > > -Aman > > > > > > > > > > > > ***Legal Disclaimer*** > > > "This communication may contain confidential and privileged material > for > > the > > > sole use of the intended recipient. Any unauthorized review, use or > > distribution > > > by others is strictly prohibited. If you have received the message by > > mistake, > > > please advise the sender by reply email and delete the message. Thank > > you." > > > ** > > >
Re: Dynamic UDFs support
Dev ops needs some control on when/how to deploy UDF's. From an operational perspective we need to provide some control on how these jars can be loaded into a running system. On Tue, Jun 21, 2016 at 9:46 AM, Neeraja Rentachintala < nrentachint...@maprtech.com> wrote: > While trying to figure out the design of where to load the jars from and > how to distribute across Drillbits, we need to keep one thing mind. > The primary goal of the Dynamic UDFs feature is that Central IT has > deployed a Drill cluster and users of the environment that are working with > the data on the cluster need to be able write their own UDFs and deploy > them onto the cluster without having to work with the IT/deployments teams > to restart Drill cluster. > > To this extent, one question I have is who is responsible to place the UDF > jar on the specific locations on Drillbits Are we expecting end users to > keep the jars accessible for Drill to load. Or does the user simply supply > a local directory of the jar which is taken by Drill and deployed on all > the Drillbits in the cluster either with YARN or without YARN. > > > > On Tue, Jun 21, 2016 at 9:34 AM, Arina Yelchiyeva < > arina.yelchiy...@gmail.com> wrote: > > > 1. DELETE command - I missed to indicate it document but had it in my > mind. > > When user issues DELETE command, all UDF associated with indicated jar is > > removed from DrillFunctionRegistry. And then binary and source files are > > also deleted from UDF classpath. > > > > 2. Distribution race condition described by Paul > > User issues CREATE command and gets confirmation that UDFs is registered > > only if all drilllbits have confirmed that registration was successful. > > I don't expect user to start using UDFs in queries prior to CREATE > command > > success / failure result, which is possible but strange. > > > > 3. DoY > > @Paul > > If instead of using $DRILL_HOME/jars/3rdparty/udf directly we use > > $DRILL_UDF environment variable which will be set during drillbit start > > (like $DRILL_LOG_DIR). Location stored in this variable will be added to > > Drill classpath during start. > > Will it ease DoY integration somehow? > > > > Kind regards > > Arina > > > > On Tue, Jun 21, 2016 at 7:15 PM yuliya Feldman > > > > > wrote: > > > > > Just thoughts: > > > You can try to reuse distributed cache Let Drill AM do the needful in > > > terms of orchestrating UDF jars distribution. > > > But > > > I would be inclined to have a common path that is independent of the > fact > > > that it is Drill on YARN or not, as maintaining two separate ways of > > > dealing with loading/unloading UDFs will be painful and error prone. > > > One more note (I left a comment in the doc) - not sure about > > authorization > > > model here - we need to have some. > > > Just my 2cThanks > > > > > > From: Paul Rogers > > > To: "dev@drill.apache.org" > > > Sent: Monday, June 20, 2016 7:32 PM > > > Subject: Re: Dynamic UDFs support > > > > > > Hi Neeraja, > > > > > > The proposal calls for the user to copy the jar file to each Drillbit > > > node. The jar would go into a new $DRILL_HOME/jars/3rdparty/udf > > directory. > > > > > > In Drill-on-YARN (DoY), YARN is responsible for copying Drill code to > > each > > > node (which is good.) YARN puts that code in a location known only to > > YARN. > > > Since the location is private to YARN, the user can’t easily hunt down > > the > > > location in order to add the udf jar. Even if the user did find the > > > location, the next Drillbit to start would create a new copy of the > Drill > > > software, without the udf jar. > > > > > > Second, in DoY we have separated user files from Drill software. This > > > makes it much easier to distribute the software to each node: we give > the > > > Drill distribution tar archive to YARN, and YARN copies it to each node > > and > > > untars the Drill files. We make a separate copy of the (far smaller) > set > > of > > > user config files. > > > > > > If the udf jar goes into a Drill folder > ($DRILL_HOME/jars/3rdparty/udf), > > > then the user would have to rebuild the Drill tar file each time they > > add a > > > udf jar. When I tried this myself when building DoY, I found it to be > > slow > > > and error-prone. > > > > > > So, the solution is to place the udf code in the new “site” directory: > > > $DRILL_SITE/jars. That’s what that is for. Then, let DoY automatically > > > distribute the code to every node. Perfect! Except that it does not > work > > to > > > dynamically distribute code after Drill starts. > > > > > > For DoY, the solution requirements are: > > > > > > 1. Distribute code using Drill itself, rather than manually copying > jars > > > to (unknown) Drill directories. > > > 2. Ensure the solution works even if another Drillbit is spun up later, > > > and uses the original Drill tar file. > > > > > > I’m thinking we want to leverage DFS: place udf files into a well-known > > > DFS directory. Register the udf into, say, ZK. Whe
Re: Drill Hangout Meeting Minutes - 6/28/16
First, Congrats to team for the 1.7 release? Can we revisit and look at DRILL-4653? I have made more changes as per feedback. On Tue, Jun 28, 2016 at 12:31 PM, Zelaine Fong wrote: > Attendees: Aman, Jinfeng, Sudheesh, Parth, Arina, Paul, John O, Padma, > Gautam, Khurram, Zelaine > > 1) JDBC Storage Plugin Issues > > We discussed DRILL-4696. Parth and Sudheesh suggested that even with the > change suggested in DRILL-4177, this particular problem may still result in > out of memory because the MySQL fetch size needs to be set at the statement > level instead of in the connect string. Parth added a comment with this > suggestion in DRILL-4177. > > Aside from the memory problem, the fact that the 4-way join isn't being > pushed completely to MySQL may be a limitation in the JDBC storage plugin. > The plugin doesn't seem to be taking into consideration the actual > underlying row count of the table. Aman suggested possibly tweaking one of > the optimizer parameters as a short-term workaround to force all plans to > be fully pushed down. > > 2) 1.7 has been released in open source Apache! >
Re: Drill Hangout Topics for 07/12/2016
Can drill-4653 be reviewed? On Mon, Jul 11, 2016 at 8:49 AM, Jinfeng Ni wrote: > Hey, > > Just a reminder that we will have Drill hangout at 10am-11am, 07/12/16 PST. > > If you have any topics to discuss on tomorrow's hangout, please reply > this thread and list the topics you want to discuss. I'll also collect > topics at the beginning of the hangout. > > Thank you, > > Jinfeng >
Re: [GitHub] drill pull request #518: DRILL-4653.json - Malformed JSON should not stop th...
Thanks chunhui-shi .
Is there a recommended way to test for count of results of a query?
On Thu, Jul 14, 2016 at 1:41 PM, chunhui-shi wrote:
> Github user chunhui-shi commented on a diff in the pull request:
>
> https://github.com/apache/drill/pull/518#discussion_r70879554
>
> --- Diff:
> exec/java-exec/src/test/java/org/apache/drill/exec/store/json/TestJsonRecordReader.java
> ---
> @@ -159,64 +164,91 @@ public void drill_3353() throws Exception {
>test("create table dfs_test.tmp.drill_3353 as select a from
> dfs.`${WORKING_PATH}/src/test/resources/jsoninput/drill_3353` where e =
> true");
>String query = "select t.a.d cnt from dfs_test.tmp.drill_3353 t
> where t.a.d is not null";
>test(query);
> - testBuilder()
> - .sqlQuery(query)
> - .unOrdered()
> - .baselineColumns("cnt")
> - .baselineValues("1")
> - .go();
> + testBuilder().sqlQuery(query).unOrdered().baselineColumns("cnt")
> + .baselineValues("1").go();
> } finally {
>testNoResult("alter session set `store.json.all_text_mode` =
> false");
> }
>}
>
> - @Test // See DRILL-3476
> + @Test
> + // See DRILL-3476
>public void testNestedFilter() throws Exception {
> String query = "select a from cp.`jsoninput/nestedFilter.json` t
> where t.a.b = 1";
> String baselineQuery = "select * from
> cp.`jsoninput/nestedFilter.json` t where t.a.b = 1";
> -testBuilder()
> -.sqlQuery(query)
> -.unOrdered()
> -.sqlBaselineQuery(baselineQuery)
> +
> testBuilder().sqlQuery(query).unOrdered().sqlBaselineQuery(baselineQuery)
> .go();
>}
>
> - @Test // See DRILL-4653
> -public void testSkippingInvalidJSONRecords() throws Exception {
> -try
> -{
> - String set = "alter session set `" +
> ExecConstants.JSON_READER_SKIP_INVALID_RECORDS_FLAG+ "` = true";
> - String query = "select count(*) from
> cp.`jsoninput/DRILL-4653.json`";
> + @Test
> + // See DRILL-4653
> + /* Test for CountingJSONReader */
> + public void testCountingQuerySkippingInvalidJSONRecords() throws
> Exception {
> +try {
> + String set = "alter session set `"
> + + ExecConstants.JSON_READER_SKIP_INVALID_RECORDS_FLAG + "`
> = true";
> + String set1 = "alter session set `"
> + +
> ExecConstants.JSON_READER_PRINT_INVALID_RECORDS_LINE_NOS_FLAG
> + + "` = true";
> + String query = "select count(*) from
> cp.`jsoninput/drill4653/file.json`";
> + testNoResult(set);
> + testNoResult(set1);
> +
> testBuilder().unOrdered().sqlQuery(query).sqlBaselineQuery(query).build()
> + .run();
> +} finally {
> + String set = "alter session set `"
> + + ExecConstants.JSON_READER_SKIP_INVALID_RECORDS_FLAG + "`
> = false";
>testNoResult(set);
> - testBuilder()
> - .unOrdered()
> - .sqlQuery(query)
> - .sqlBaselineQuery(query)
> - .build().run();
> }
> -finally
> -{
> - String set = "alter session set `" +
> ExecConstants.JSON_READER_SKIP_INVALID_RECORDS_FLAG+ "` = false";
> + }
> +
> + @Test
> + // See DRILL-4653
> + /* Test for CountingJSONReader */
> + public void testCountingQueryNotSkippingInvalidJSONRecords() throws
> Exception {
> +try {
> + String query = "select count(*) from
> cp.`jsoninput/drill4653/file.json`";
> +
> testBuilder().unOrdered().sqlQuery(query).sqlBaselineQuery(query).build()
> --- End diff --
>
> Should we compare with expected count here?
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
Re: [GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...
This mechanism falls in line with other JSON processing similar to serde's with Hive, UDF's enabled at global level will apply to all users and is outlined using documentation. What is your stance if we move to the JSONFormatPlugin? On Fri, Jul 15, 2016 at 2:08 PM, jaltekruse wrote: > Github user jaltekruse commented on the issue: > > https://github.com/apache/drill/pull/518 > > I don't think we should merge this without a mechanism to return a > warning to the user to tell them at least that some data was ignored, and > ideally some indication of how much data was discarded. While I do > understand this is not the default behavior, I think there is still too > high of a risk that an admin could set this at a global level and users > would be unaware of some of their data being discarded. > > I am willing to discuss the benefits of merging this before such a > system exists, but until this issue has been thoroughly evaluated I am -1 > on the change. > > One improvement you could make to the current implementation is moving > the option to the format plugin instead of the system/session list. This > enables users to include setting the option in there query with the "table > with options" syntax that was added last fall. We already have a JIRA open > for moving the all_text_mode and read_numbers_as_double options to this > location, because it doesn't really make sense to change query results > based on session state. Unfortunately this change does not completely > remove my initial concern, because not all users can modify or see the > storage plugins in the case when web UI security is enabled. Non-admin > users in these cases could be surprised by this behavior. > > For examples of how this is done, you can look at the text plugin > config, you would just need to add these options as properties to the json > config which is currently mostly empty. > > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java#L93 > > > https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/TextFormatPlugin.java#L135 > > Select with options: https://issues.apache.org/jira/browse/DRILL-4047 > Jira for moving the existing options: > https://issues.apache.org/jira/browse/DRILL-4206 > > > --- > If your project is set up for it, you can reply to this email and have your > reply appear on GitHub as well. If your project does not have this feature > enabled and wishes so, or if the feature is enabled but not working, please > contact infrastructure at infrastruct...@apache.org or file a JIRA ticket > with INFRA. > --- >
Re: [GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...
Hi Folks, When can we discuss this feature? Would next hangout be appropriate? Thanks Subbu On Mon, Jul 25, 2016 at 10:20 AM, Subbu Srinivasan wrote: > This mechanism falls in line with other JSON processing similar to serde's > with Hive, UDF's enabled at global level will apply to all users and is > outlined using documentation. > > > What is your stance if we move to the JSONFormatPlugin? > > On Fri, Jul 15, 2016 at 2:08 PM, jaltekruse wrote: > >> Github user jaltekruse commented on the issue: >> >> https://github.com/apache/drill/pull/518 >> >> I don't think we should merge this without a mechanism to return a >> warning to the user to tell them at least that some data was ignored, and >> ideally some indication of how much data was discarded. While I do >> understand this is not the default behavior, I think there is still too >> high of a risk that an admin could set this at a global level and users >> would be unaware of some of their data being discarded. >> >> I am willing to discuss the benefits of merging this before such a >> system exists, but until this issue has been thoroughly evaluated I am -1 >> on the change. >> >> One improvement you could make to the current implementation is >> moving the option to the format plugin instead of the system/session list. >> This enables users to include setting the option in there query with the >> "table with options" syntax that was added last fall. We already have a >> JIRA open for moving the all_text_mode and read_numbers_as_double options >> to this location, because it doesn't really make sense to change query >> results based on session state. Unfortunately this change does not >> completely remove my initial concern, because not all users can modify or >> see the storage plugins in the case when web UI security is enabled. >> Non-admin users in these cases could be surprised by this behavior. >> >> For examples of how this is done, you can look at the text plugin >> config, you would just need to add these options as properties to the json >> config which is currently mostly empty. >> >> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONFormatPlugin.java#L93 >> >> >> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/TextFormatPlugin.java#L135 >> >> Select with options: https://issues.apache.org/jira/browse/DRILL-4047 >> Jira for moving the existing options: >> https://issues.apache.org/jira/browse/DRILL-4206 >> >> >> --- >> If your project is set up for it, you can reply to this email and have >> your >> reply appear on GitHub as well. If your project does not have this feature >> enabled and wishes so, or if the feature is enabled but not working, >> please >> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket >> with INFRA. >> --- >> > >
Re: Suggestions for hangout topics for 08/09
What time is tomorrow's mtg scheduled for? On Mon, Aug 8, 2016 at 10:48 AM, Gautam Parai wrote: > If you have any suggestions for Drill hangout topics for tomorrow, you can > add it to this thread. We will also ask around at the beginning of the > hangout for any topics. We will try to cover whatever possible during the > 1 hr. > > Topics: > 1. DRILL-4653: Malformed JSON should not stop the entire query from > progressing. >Discussion about the PR. >
Re: [GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...
Thanks Jason. I am in the process of doing more unit testing, corner case testing etc before submitting for final review. Thanks Subbu On Mon, Aug 22, 2016 at 9:51 AM, jaltekruse wrote: > Github user jaltekruse commented on the issue: > > https://github.com/apache/drill/pull/518 > > After this was discussed at the Hangout a few weeks back, I had been > thinking about it more. > > My initial request was for warnings to be returned along with query > results. Some initial work was posted last fall to add warnings to the RPC > protocol, but unfortunately it was not brought to completion. These kinds > of warnings can be received by JDBC and ODBC sources and many tools show > them to users. > > https://github.com/apache/drill/pull/263 > > In the hangout we discussed that this changeset is currently logging > when part of a file is ignored. I don't believe that there is currently an > expectation that users should have to grep through a log file to find out > any additional information about the execution of a successful query, the > log files are there for admins to debug system issues. > > I still think it would be good to have a way to remove the need to > look through the log file to see if this behavior was used in a query, but > as there wasn't a lot of concern expressed by others when we discussed it, > I'm changing my vote to a +0. > > > --- > If your project is set up for it, you can reply to this email and have your > reply appear on GitHub as well. If your project does not have this feature > enabled and wishes so, or if the feature is enabled but not working, please > contact infrastructure at infrastruct...@apache.org or file a JIRA ticket > with INFRA. > --- >
Question on input file parsing
Folks,
What is the general thoughts of the team on how DRILL parses input data?
Is the philosophy that the input is typically delimited (Eg: new line in
case of JSON data)
or should we remain agnostic and let the underlying parser implementation
interpret start of
the next input record ?
Is it valid to have two json records in a single line?
{"json"}{"json"}
I am asking this from DRILL-4653 perspective?
Thanks
Subbu S
Re: [HANGOUT] Topics for 10/04/16
Can we close on https://github.com/apache/drill/pull/518 ? On Mon, Oct 3, 2016 at 10:27 AM, Sudheesh Katkam wrote: > Hi drillers, > > Our bi-weekly hangout is tomorrow (10/04/16, 10 AM PT). If you have any > suggestions for hangout topics, you can add them to this thread. We will > also ask around at the beginning of the hangout for topics. > > Thank you, > Sudheesh >
Re: Time for a 1.9 Release?
+1. On Sun, Oct 30, 2016 at 10:23 PM, Paul Rogers wrote: > For release numbers, 1.10 (then 1.11, 1.12, …) seems like a good idea. > > At first it may seem odd to go to 1.10 from 1.9. Might people get confused > between 1.10 and 1.1.0? But, there is precedence. Tomcat’s latest 7-series > release is 7.0.72. Java is on 8u112. And so on. > > I like the idea of moving to 2.0 later when the team introduces a major > change, rather than by default just because the numbers roll around. For > example, Hadoop when to 2.x when YARN was introduced. Impala appears to > have moved to 2.0 when they added Spill to disk for some (all?) operators. > > - Paul > > > On Oct 28, 2016, at 10:34 AM, Sudheesh Katkam > wrote: > > > > Hi Drillers, > > > > We have a reasonable number of fixes and features since the last release > > [1]. Releasing itself takes a while; so I propose we start the 1.9 > release > > process. > > > > I volunteer as the release manager, unless there are objections. > > > > We should also discuss what the release version number should be after > 1.9. > > > > Thank you, > > Sudheesh > > > > [1] https://issues.apache.org/jira/browse/DRILL/fixforversion/12337861 > >
Re: [ANNOUNCE] - New Apache Drill Committer - Neeraja Rentachintala
Congrats. Wish I had more time to contribute more(, perils of working in a startup) On Thu, Nov 17, 2016 at 11:18 AM, Jason Altekruse wrote: > Congratulations! Thanks for all your contributions to Drill! > > Jason Altekruse > Software Engineer at Dremio > Apache Drill Committer > > On Thu, Nov 17, 2016 at 11:12 AM, Abhishek Girish > wrote: > > > Congrats Neeraja! > > > > On Thu, Nov 17, 2016 at 11:10 AM, Parth Chandra > wrote: > > > > > On behalf of the Apache Drill PMC, I am very pleased to announce that > > > Neeraja Rentachintala has accepted the invitation to become a committer > > in > > > the project. > > > > > > > > > Welcome Neeraja ! > > > > > >
[jira] [Created] (DRILL-4641) Support for lzo compression
subbu srinivasan created DRILL-4641: --- Summary: Support for lzo compression Key: DRILL-4641 URL: https://issues.apache.org/jira/browse/DRILL-4641 Project: Apache Drill Issue Type: Improvement Components: Storage - Other Affects Versions: Future Environment: Not specific to platform Reporter: subbu srinivasan Would love support for quering lzo compressed files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4651) S3 connector and empty bucket issue
subbu srinivasan created DRILL-4651: --- Summary: S3 connector and empty bucket issue Key: DRILL-4651 URL: https://issues.apache.org/jira/browse/DRILL-4651 Project: Apache Drill Issue Type: Bug Components: Storage - Information Schema Affects Versions: 1.6.0 Reporter: subbu srinivasan show schemas will not list the information about a registered s3 plugin if the bucket is empty. This is in embedded mode. Steps to reproduce: - Go to http://localhost:8047/storage - Add s3 plugin (make sure bucket empty) - show schemas will not information about the s3 plugin/workspaces - Add a test file to bucket and show schemas will show the plugin/workspace -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4653) Malformed JSON should not stop the entire query from progressing
subbu srinivasan created DRILL-4653: --- Summary: Malformed JSON should not stop the entire query from progressing Key: DRILL-4653 URL: https://issues.apache.org/jira/browse/DRILL-4653 Project: Apache Drill Issue Type: Improvement Components: Storage - JSON Affects Versions: 1.6.0 Reporter: subbu srinivasan Currently Drill query terminates upon first encounter of a invalid JSON line. Drill has to continue progressing after ignoring the bad records. Something similar to a setting of (ignore.malformed.json) would help. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4719) Need To Support IAM role based access for supporting Amazon S3
subbu srinivasan created DRILL-4719: --- Summary: Need To Support IAM role based access for supporting Amazon S3 Key: DRILL-4719 URL: https://issues.apache.org/jira/browse/DRILL-4719 Project: Apache Drill Issue Type: Improvement Components: Storage - Other Affects Versions: 1.6.0 Reporter: subbu srinivasan We need amazon secret accessid/credentials as part of the core-site.xml. This is not ideal in many deployments, we would use IAM roles to accomplish access to s3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
