[jira] [Commented] (DRILL-4232) Support for EXCEPT set operator
[ https://issues.apache.org/jira/browse/DRILL-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612021#comment-17612021 ] ASF GitHub Bot commented on DRILL-4232: --- cgivre commented on PR #2599: URL: https://github.com/apache/drill/pull/2599#issuecomment-1264537373 @Leon-WTF I don't know if you saw this or not, but #2602 has been merged. We are actually running on Calcite 1.32 now! > Support for EXCEPT set operator > --- > > Key: DRILL-4232 > URL: https://issues.apache.org/jira/browse/DRILL-4232 > Project: Apache Drill > Issue Type: New Feature > Components: Query Planning & Optimization >Reporter: Victoria Markman >Assignee: Tengfei Wang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-6268) Drill-on-YARN client obtains HDFS URL incorrectly
[
https://issues.apache.org/jira/browse/DRILL-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612020#comment-17612020
]
ASF GitHub Bot commented on DRILL-6268:
---
cgivre closed pull request #2139: DRILL-6268: Drill-on-YARN client obtains HDFS
URL incorrectly
URL: https://github.com/apache/drill/pull/2139
> Drill-on-YARN client obtains HDFS URL incorrectly
> -
>
> Key: DRILL-6268
> URL: https://issues.apache.org/jira/browse/DRILL-6268
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: Paul Rogers
>Assignee: Charles Givre
>Priority: Major
>
> The Drill-on-YARN client must upload files to HDFS so that YARN can localize
> them. The code that does so is in {{DfsFacade}}. This code obtains the URL
> twice. The first time is correct:
> {code}
> private void loadYarnConfig() {
> ...
> URI fsUri = FileSystem.getDefaultUri( yarnConf );
> if(fsUri.toString().startsWith("file:/")) {
> System.err.println("Warning: Default DFS URI is for a local file
> system: " + fsUri);
> }
> }
> }
> {code}
> The {{fsUri}} returned is {{hdfs://localhost:9000}}, which is the correct
> value for an out-of-the-box Hadoop 2.9.0 install after following [these
> instructions|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html].
> The instructions have the reader explicitly set the port number to 9000:
> {code}
>
>
> fs.defaultFS
> hdfs://localhost:9000
>
>
> {code}
> The other place that gets the URL, this time or real, is
> {{DfsFacade.connect()}}:
> {code}
> String dfsConnection = config.getString(DrillOnYarnConfig.DFS_CONNECTION);
> {code}
> This value comes back as {{hdfs://localhost/}}, which causes HDFS to try to
> connect on port 8020 (the Hadoop default), resulting in the following error:
> {noformat}
> Connecting to DFS... Connected.
> Uploading /Users/paulrogers/bin/apache-drill-1.13.0.tar.gz to
> /users/drill/apache-drill-1.13.0.tar.gz ... Failed.
> Failed to upload Drill archive
> Caused by: Failed to create DFS directory: /users/drill
> Caused by: Call From Pauls-MBP/192.168.1.243 to localhost:8020 failed on
> connection exception: java.net.ConnectException: Connection refused;
> For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
> {noformat}
> (Shout out here to [~arjun-kr] for suggesting we include the extra exception
> details; very helpful here.)
> The workaround is to manually change the port to 8020 in the config setting
> shown above.
> The full fix is to change the code to use the following line in {{connect()}}:
> {code}
> String dfsConnection = FileSystem.getDefaultUri(yarnConf);
> {code}
> This bug is serious because it constrains the ability of users to select
> non-default HDFS ports.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8321) Change kafka_2.13 dependency scope to test
[
https://issues.apache.org/jira/browse/DRILL-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612019#comment-17612019
]
ASF GitHub Bot commented on DRILL-8321:
---
cgivre merged PR #2662:
URL: https://github.com/apache/drill/pull/2662
> Change kafka_2.13 dependency scope to test
> ---
>
> Key: DRILL-8321
> URL: https://issues.apache.org/jira/browse/DRILL-8321
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.20.2
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 1.20.3
>
>
> Drill has 2 scala dependencies:
> * {{org.apache.kafka.kafka_2.13}}
> * {{com.madhukaraphatak.java-sizeof_2.11}}
> which are targets on different scala versions 2.13 and 2.11. But Scala has no
> backward compatibility for major releases, so we can’t have 2 libraries
> compiled on various versions of scala.
> To solve the issue there are only 2 ways:
> # Compile both libraries on the same major Scala version.
> # Remove one of the libraries from Drill
> {{kafka_2.13}} is server side (kafka’s server side) dependency and is
> unnecessary on the client side (Drill). Probably, it was added carelessly to
> Drill to a compile scope, while it is necessary only in a test scope.
> So {{kafka_2.13}} can be removed from compile scope. It will reduce the Drill
> package size and the main – it will solve scala version conflict.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8327) GoogleSheets not Reporting Schemata to Info_Schema
[ https://issues.apache.org/jira/browse/DRILL-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612018#comment-17612018 ] ASF GitHub Bot commented on DRILL-8327: --- cgivre opened a new pull request, #2667: URL: https://github.com/apache/drill/pull/2667 # [DRILL-8327](https://issues.apache.org/jira/browse/DRILL-8327): GoogleSheets not Reporting Schemata to Info_Schema ## Description GoogleSheets (GS) was not reporting available documents to the information schema. This PR fixes that so that users are now able to query the information_schema to find a list of available documents. Note that it is not possible to populate the tabs into the information schema due to Google's rate limits. If this was populated, it would be very easy for a user with a few dozen files to exceed the rate limit. Users can obtain the list of available sheets by using the `_sheets` implicit column. ## Documentation Updated README. ## Testing Ran existing unit tests and added two additional tests to confirm values are being populated into the `information_schema`. > GoogleSheets not Reporting Schemata to Info_Schema > -- > > Key: DRILL-8327 > URL: https://issues.apache.org/jira/browse/DRILL-8327 > Project: Apache Drill > Issue Type: Bug > Components: Storage - GoogleSheets >Affects Versions: 2.0.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 2.0.0 > > > The GoogleSheets (GS) plugin was not reporting the available documents to the > info schema. This PR makes some modifications so that users can determine > which documents are available via the information schema. > The GS plugin does not report the tabs as tables to the information schema > because that can cause Drill to exceed Google's rate quota. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (DRILL-8327) GoogleSheets not Reporting Schemata to Info_Schema
Charles Givre created DRILL-8327: Summary: GoogleSheets not Reporting Schemata to Info_Schema Key: DRILL-8327 URL: https://issues.apache.org/jira/browse/DRILL-8327 Project: Apache Drill Issue Type: Bug Components: Storage - GoogleSheets Affects Versions: 2.0.0 Reporter: Charles Givre Assignee: Charles Givre Fix For: 2.0.0 The GoogleSheets (GS) plugin was not reporting the available documents to the info schema. This PR makes some modifications so that users can determine which documents are available via the information schema. The GS plugin does not report the tabs as tables to the information schema because that can cause Drill to exceed Google's rate quota. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8326) snakeyaml 1.33
[ https://issues.apache.org/jira/browse/DRILL-8326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611864#comment-17611864 ] ASF GitHub Bot commented on DRILL-8326: --- pjfanning opened a new pull request, #2666: URL: https://github.com/apache/drill/pull/2666 ## Description recent snakeyaml changes need extra fixes ## Documentation (Please describe user-visible changes similar to what should appear in the Drill documentation.) ## Testing (Please describe how this PR has been tested.) > snakeyaml 1.33 > -- > > Key: DRILL-8326 > URL: https://issues.apache.org/jira/browse/DRILL-8326 > Project: Apache Drill > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > > [https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes] – fixes bug in code > point limit added in 1.32 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (DRILL-8326) snakeyaml 1.33
PJ Fanning created DRILL-8326: - Summary: snakeyaml 1.33 Key: DRILL-8326 URL: https://issues.apache.org/jira/browse/DRILL-8326 Project: Apache Drill Issue Type: Improvement Reporter: PJ Fanning [https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes] – fixes bug in code point limit added in 1.32 -- This message was sent by Atlassian Jira (v8.20.10#820010)
