[jira] [Created] (DRILL-7261) Rollup of CSV V3 fixes
Paul Rogers created DRILL-7261: -- Summary: Rollup of CSV V3 fixes Key: DRILL-7261 URL: https://issues.apache.org/jira/browse/DRILL-7261 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.16.0 Reporter: Paul Rogers Assignee: Paul Rogers Fix For: 1.17.0 Rollup of related CSV V3 fixes along with supporting row set framework fixes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7260) Error on clause that should not be evaluated
benj created DRILL-7260:
---
Summary: Error on clause that should not be evaluated
Key: DRILL-7260
URL: https://issues.apache.org/jira/browse/DRILL-7260
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.16.0
Reporter: benj
Some requests failed when evaluate condition which should not be evaluated.
Example:
{code:java}
SELECT * FROM (
SELECT * FROM (
SELECT '1d28' a )
WHERE regexp_matches(a,'^[0-9]*$')) /* Keep only row on INT format */
WHERE CAST(a AS INT) < 2000; /* This CAST should't failed due */
/* to filtering in the inner queries */
=>
Error: SYSTEM ERROR: NumberFormatException: 1d28
{code}
Note that the problem doesn't appear in all situation.
Examples :
{code:java}
/* Just adding another row and it's work ! */
SELECT * FROM (
SELECT * FROM (
SELECT '1d28' a UNION SELECT '2a2')
WHERE regexp_matches(a,'^[0-9]*$'))
WHERE CAST(a AS INT) < 2000;
=>
+---+
| a |
+---+
+---+
{code}
{code:java}
// MyFile.csvh
a,b
1,2
a1a,3
// !MyFile.csvh
SELECT * FROM (
SELECT * FROM `MyFile.csvh ` WHERE regexp_matches(a,'^[0-9]*$')) AS x
WHERE CAST(a AS INT) < 200;
=>
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+
{code}
In this case, there is no error - the cast is never evaluated (as expected) for
the row with a='a1a'.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [drill] arina-ielchiieva commented on a change in pull request #1788: DRILL-7204: Add proper validation when creating plugin
arina-ielchiieva commented on a change in pull request #1788: DRILL-7204: Add
proper validation when creating plugin
URL: https://github.com/apache/drill/pull/1788#discussion_r284259374
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/StorageResources.java
##
@@ -204,6 +204,13 @@ public JsonResult
createOrUpdatePluginJSON(PluginConfigWrapper plugin) {
@Consumes(MediaType.APPLICATION_FORM_URLENCODED)
@Produces(MediaType.APPLICATION_JSON)
public JsonResult createOrUpdatePlugin(@FormParam("name") String name,
@FormParam("config") String storagePluginConfig) {
+name = name.trim();
+if (name.isEmpty()) {
+ return message("Error (a storage name cannot be empty)");
+}
+if (!name.matches("^[a-zA-Z0-9._-]+$")) {
Review comment:
We should also enforce such pattern when adding plugins from plugin override
file during Drill start up.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [drill] arina-ielchiieva commented on a change in pull request #1788: DRILL-7204: Add proper validation when creating plugin
arina-ielchiieva commented on a change in pull request #1788: DRILL-7204: Add
proper validation when creating plugin
URL: https://github.com/apache/drill/pull/1788#discussion_r284259821
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/StorageResources.java
##
@@ -204,6 +204,13 @@ public JsonResult
createOrUpdatePluginJSON(PluginConfigWrapper plugin) {
@Consumes(MediaType.APPLICATION_FORM_URLENCODED)
@Produces(MediaType.APPLICATION_JSON)
public JsonResult createOrUpdatePlugin(@FormParam("name") String name,
@FormParam("config") String storagePluginConfig) {
+name = name.trim();
+if (name.isEmpty()) {
+ return message("Error (a storage name cannot be empty)");
Review comment:
Could you please replace alerts with more user friendly message window?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [drill] arina-ielchiieva commented on a change in pull request #1788: DRILL-7204: Add proper validation when creating plugin
arina-ielchiieva commented on a change in pull request #1788: DRILL-7204: Add
proper validation when creating plugin
URL: https://github.com/apache/drill/pull/1788#discussion_r284259821
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/StorageResources.java
##
@@ -204,6 +204,13 @@ public JsonResult
createOrUpdatePluginJSON(PluginConfigWrapper plugin) {
@Consumes(MediaType.APPLICATION_FORM_URLENCODED)
@Produces(MediaType.APPLICATION_JSON)
public JsonResult createOrUpdatePlugin(@FormParam("name") String name,
@FormParam("config") String storagePluginConfig) {
+name = name.trim();
+if (name.isEmpty()) {
+ return message("Error (a storage name cannot be empty)");
Review comment:
Could you please replace alerts with more user friendly message window?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [drill] arina-ielchiieva commented on a change in pull request #1788: DRILL-7204: Add proper validation when creating plugin
arina-ielchiieva commented on a change in pull request #1788: DRILL-7204: Add
proper validation when creating plugin
URL: https://github.com/apache/drill/pull/1788#discussion_r284259374
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/StorageResources.java
##
@@ -204,6 +204,13 @@ public JsonResult
createOrUpdatePluginJSON(PluginConfigWrapper plugin) {
@Consumes(MediaType.APPLICATION_FORM_URLENCODED)
@Produces(MediaType.APPLICATION_JSON)
public JsonResult createOrUpdatePlugin(@FormParam("name") String name,
@FormParam("config") String storagePluginConfig) {
+name = name.trim();
+if (name.isEmpty()) {
+ return message("Error (a storage name cannot be empty)");
+}
+if (!name.matches("^[a-zA-Z0-9._-]+$")) {
Review comment:
Since we don't enforce this on the backend side, I don't think we should
enforce this on the frontend side as well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [drill] agozhiy opened a new pull request #1788: DRILL-7204: Add proper validation when creating plugin
agozhiy opened a new pull request #1788: DRILL-7204: Add proper validation when creating plugin URL: https://github.com/apache/drill/pull/1788 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [drill] arina-ielchiieva commented on issue #1749: DRILL-7177: Format Plugin for Excel Files
arina-ielchiieva commented on issue #1749: DRILL-7177: Format Plugin for Excel Files URL: https://github.com/apache/drill/pull/1749#issuecomment-492585600 @cgivre I suggest new storage or format plugins to be implemented using new row set-based framework implemented by @paul-rogers. Besides better memory management, usage of this framework will allow new plugin / format to use schema provisioning feature. Hopefully, existing plugins will be re-written to use this framework but new should be already implemented using this framework. Also please note that you need to add `bootstrap-format-plugins.json` as per changes in DRILL-7030. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [drill] arina-ielchiieva commented on issue #1778: Drill-7233: Format Plugin for HDF5
arina-ielchiieva commented on issue #1778: Drill-7233: Format Plugin for HDF5 URL: https://github.com/apache/drill/pull/1778#issuecomment-492585538 @cgivre I suggest new storage or format plugins to be implemented using new row set-based framework implemented by @paul-rogers. Besides better memory management, usage of this framework will allow new plugin / format to use schema provisioning feature. Hopefully, existing plugins will be re-written to use this framework but new should be already implemented using this framework. Also please note that you need to add `bootstrap-format-plugins.json` as per changes in DRILL-7030. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
