[jira] [Commented] (DRILL-8327) GoogleSheets not Reporting Schemata to Info_Schema
[ https://issues.apache.org/jira/browse/DRILL-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613461#comment-17613461 ] ASF GitHub Bot commented on DRILL-8327: --- cgivre merged PR #2667: URL: https://github.com/apache/drill/pull/2667 > GoogleSheets not Reporting Schemata to Info_Schema > -- > > Key: DRILL-8327 > URL: https://issues.apache.org/jira/browse/DRILL-8327 > Project: Apache Drill > Issue Type: Bug > Components: Storage - GoogleSheets >Affects Versions: 2.0.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 2.0.0 > > > The GoogleSheets (GS) plugin was not reporting the available documents to the > info schema. This PR makes some modifications so that users can determine > which documents are available via the information schema. > The GS plugin does not report the tabs as tables to the information schema > because that can cause Drill to exceed Google's rate quota. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8327) GoogleSheets not Reporting Schemata to Info_Schema
[
https://issues.apache.org/jira/browse/DRILL-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613279#comment-17613279
]
ASF GitHub Bot commented on DRILL-8327:
---
cgivre commented on code in PR #2667:
URL: https://github.com/apache/drill/pull/2667#discussion_r988476178
##
contrib/storage-googlesheets/src/main/java/org/apache/drill/exec/store/googlesheets/GoogleSheetsStoragePlugin.java:
##
@@ -61,6 +62,7 @@ public class GoogleSheetsStoragePlugin extends
AbstractStoragePlugin {
private final OAuthTokenProvider tokenProvider;
private DataStore dataStore;
private Sheets service;
+ private Drive driveService;
Review Comment:
Confirmed. The objects in question don't have a `close` method.
> GoogleSheets not Reporting Schemata to Info_Schema
> --
>
> Key: DRILL-8327
> URL: https://issues.apache.org/jira/browse/DRILL-8327
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - GoogleSheets
>Affects Versions: 2.0.0
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 2.0.0
>
>
> The GoogleSheets (GS) plugin was not reporting the available documents to the
> info schema. This PR makes some modifications so that users can determine
> which documents are available via the information schema.
> The GS plugin does not report the tabs as tables to the information schema
> because that can cause Drill to exceed Google's rate quota.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8327) GoogleSheets not Reporting Schemata to Info_Schema
[
https://issues.apache.org/jira/browse/DRILL-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613269#comment-17613269
]
ASF GitHub Bot commented on DRILL-8327:
---
cgivre commented on code in PR #2667:
URL: https://github.com/apache/drill/pull/2667#discussion_r988459145
##
contrib/storage-googlesheets/src/main/java/org/apache/drill/exec/store/googlesheets/GoogleSheetsStoragePlugin.java:
##
@@ -61,6 +62,7 @@ public class GoogleSheetsStoragePlugin extends
AbstractStoragePlugin {
private final OAuthTokenProvider tokenProvider;
private DataStore dataStore;
private Sheets service;
+ private Drive driveService;
Review Comment:
@jnturton I don't think they need to be closed. These objects are really
more like containers for the various bits that go into calling the Google APIs.
They don't actually open any input streams, at least I don't think they do.
You can see that when we actually pull data from Google, there is some
convoluted call which ends in `execute`. That makes the actual HTTP call.
> GoogleSheets not Reporting Schemata to Info_Schema
> --
>
> Key: DRILL-8327
> URL: https://issues.apache.org/jira/browse/DRILL-8327
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - GoogleSheets
>Affects Versions: 2.0.0
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 2.0.0
>
>
> The GoogleSheets (GS) plugin was not reporting the available documents to the
> info schema. This PR makes some modifications so that users can determine
> which documents are available via the information schema.
> The GS plugin does not report the tabs as tables to the information schema
> because that can cause Drill to exceed Google's rate quota.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8327) GoogleSheets not Reporting Schemata to Info_Schema
[
https://issues.apache.org/jira/browse/DRILL-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613270#comment-17613270
]
ASF GitHub Bot commented on DRILL-8327:
---
cgivre commented on code in PR #2667:
URL: https://github.com/apache/drill/pull/2667#discussion_r988459316
##
contrib/storage-googlesheets/src/main/java/org/apache/drill/exec/store/googlesheets/GoogleSheetsStoragePlugin.java:
##
@@ -61,6 +62,7 @@ public class GoogleSheetsStoragePlugin extends
AbstractStoragePlugin {
private final OAuthTokenProvider tokenProvider;
private DataStore dataStore;
private Sheets service;
+ private Drive driveService;
Review Comment:
Let me verify that before we merge this.
> GoogleSheets not Reporting Schemata to Info_Schema
> --
>
> Key: DRILL-8327
> URL: https://issues.apache.org/jira/browse/DRILL-8327
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - GoogleSheets
>Affects Versions: 2.0.0
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 2.0.0
>
>
> The GoogleSheets (GS) plugin was not reporting the available documents to the
> info schema. This PR makes some modifications so that users can determine
> which documents are available via the information schema.
> The GS plugin does not report the tabs as tables to the information schema
> because that can cause Drill to exceed Google's rate quota.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8327) GoogleSheets not Reporting Schemata to Info_Schema
[ https://issues.apache.org/jira/browse/DRILL-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612018#comment-17612018 ] ASF GitHub Bot commented on DRILL-8327: --- cgivre opened a new pull request, #2667: URL: https://github.com/apache/drill/pull/2667 # [DRILL-8327](https://issues.apache.org/jira/browse/DRILL-8327): GoogleSheets not Reporting Schemata to Info_Schema ## Description GoogleSheets (GS) was not reporting available documents to the information schema. This PR fixes that so that users are now able to query the information_schema to find a list of available documents. Note that it is not possible to populate the tabs into the information schema due to Google's rate limits. If this was populated, it would be very easy for a user with a few dozen files to exceed the rate limit. Users can obtain the list of available sheets by using the `_sheets` implicit column. ## Documentation Updated README. ## Testing Ran existing unit tests and added two additional tests to confirm values are being populated into the `information_schema`. > GoogleSheets not Reporting Schemata to Info_Schema > -- > > Key: DRILL-8327 > URL: https://issues.apache.org/jira/browse/DRILL-8327 > Project: Apache Drill > Issue Type: Bug > Components: Storage - GoogleSheets >Affects Versions: 2.0.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 2.0.0 > > > The GoogleSheets (GS) plugin was not reporting the available documents to the > info schema. This PR makes some modifications so that users can determine > which documents are available via the information schema. > The GS plugin does not report the tabs as tables to the information schema > because that can cause Drill to exceed Google's rate quota. -- This message was sent by Atlassian Jira (v8.20.10#820010)
