[jira] [Updated] (DRILL-8306) UserRemoteException : DATA_READ ERROR: Error parsing JSON - Unexpected character ('-' (code 45)): was expecting comma to separate Object entries
[
https://issues.apache.org/jira/browse/DRILL-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SHUBHAM KUMAR updated DRILL-8306:
-
Description:
While querying below Json we are getting the DATA_READ ERROR: Error parsing
JSON - Unexpected character ('-' (code 45)): was expecting comma to separate
Object entries.
json payload:
{
"_id": "US26781824744023100811",
"state": "SAP_PROCESSING",
"createdAt": 2022-08-27T15:07:42.162Z,
"customerId": "c5acee7c-35db-4741-a09f-20f8f76cc0bd",
"orderTotal": 50,
"shipmentInfo": [
{
"trackingNumber":"1ZWW2398YW72540758",
"URL":"https://ulta.narvar.com/tracking/ulta/UPS?service=SP_numbers=1ZWW2398YW72540758=60540_number=K124155501=false_date=2019-11-29T16:40:02_date=2019-12-05T00:00:00-0600=46143;
}
],
"updatedAt": 2022-08-28T15:07:42.162Z
}
Note: this issue coming mainly for createdAt and updatedAt field. if i make
value of these fields as "createdAt": "2022-08-27T15:07:42.162Z" kind of
string then able to query it successfully. Please suggest some solution
was:
I am trying to connect below mongo db instance using below URI in Apache
Drill, but not able to fetch all databases its just showing only *mongo.admin*
database but in actual it holds 4 databases.
URI: mongodb+srv://username:password@hostname/databasename
but same URI we are using in MongoDB compass and able to view all 4 databases.
> UserRemoteException : DATA_READ ERROR: Error parsing JSON -
> Unexpected character ('-' (code 45)): was expecting comma to separate Object
> entries
> -
>
> Key: DRILL-8306
> URL: https://issues.apache.org/jira/browse/DRILL-8306
> Project: Apache Drill
> Issue Type: Bug
>Reporter: SHUBHAM KUMAR
>Priority: Major
>
> While querying below Json we are getting the DATA_READ ERROR: Error parsing
> JSON - Unexpected character ('-' (code 45)): was expecting comma to separate
> Object entries.
> json payload:
> {
> "_id": "US26781824744023100811",
> "state": "SAP_PROCESSING",
> "createdAt": 2022-08-27T15:07:42.162Z,
> "customerId": "c5acee7c-35db-4741-a09f-20f8f76cc0bd",
> "orderTotal": 50,
> "shipmentInfo": [
> {
> "trackingNumber":"1ZWW2398YW72540758",
>
> "URL":"https://ulta.narvar.com/tracking/ulta/UPS?service=SP_numbers=1ZWW2398YW72540758=60540_number=K124155501=false_date=2019-11-29T16:40:02_date=2019-12-05T00:00:00-0600=46143;
> }
> ],
> "updatedAt": 2022-08-28T15:07:42.162Z
> }
>
>
> Note: this issue coming mainly for createdAt and updatedAt field. if i make
> value of these fields as "createdAt": "2022-08-27T15:07:42.162Z" kind of
> string then able to query it successfully. Please suggest some solution
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (DRILL-8306) UserRemoteException : DATA_READ ERROR: Error parsing JSON - Unexpected character ('-' (code 45)): was expecting comma to separate Object entries
SHUBHAM KUMAR created DRILL-8306:
Summary: UserRemoteException : DATA_READ ERROR: Error parsing
JSON - Unexpected character ('-' (code 45)): was expecting comma to separate
Object entries
Key: DRILL-8306
URL: https://issues.apache.org/jira/browse/DRILL-8306
Project: Apache Drill
Issue Type: Bug
Reporter: SHUBHAM KUMAR
I am trying to connect below mongo db instance using below URI in Apache
Drill, but not able to fetch all databases its just showing only *mongo.admin*
database but in actual it holds 4 databases.
URI: mongodb+srv://username:password@hostname/databasename
but same URI we are using in MongoDB compass and able to view all 4 databases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Clean up deprecated Apache code in Drill
[
https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603154#comment-17603154
]
ASF GitHub Bot commented on DRILL-8117:
---
kingswanwho commented on code in PR #2499:
URL: https://github.com/apache/drill/pull/2499#discussion_r968518305
##
exec/java-exec/src/test/java/org/apache/drill/exec/impersonation/TestInboundImpersonation.java:
##
@@ -156,22 +159,25 @@ public void unauthorizedTarget() throws Exception {
@Test
public void invalidPolicy() throws Exception {
-thrownException.expect(new
UserExceptionMatcher(UserBitShared.DrillPBError.ErrorType.VALIDATION,
-"Invalid impersonation policies."));
+String query = "ALTER SYSTEM SET `%s`='%s'";
Review Comment:
@jnturton Hi James, the **_client_** here inherits from BaseTestQuery, which
is DrillClient but not ClientFixture in TestLargeFileCompilation as Paul
suggested. DrillClient doesn't have method like client.alterSystem().
BaseTestQuery is deprecated actually, but not marked as deprecated because it
is still widely used. The best choice here I think is change base test class
from BaseTestQuery to ClusterTest so that we could use client.alterSystem() for
clean and compact code style. There are multiple Inheritance levels here, which
are TestInboundImpersonation -> BaseTestImpersonation -> PlanTestBase ->
BaseTestQuery. Due to PlanTestBase involves many classes. Maybe change
BaseTestImpersonation inheritance from PlanTestBase to ClusterTest directly is
an easier way. However, this PR mainly focus on the deprecated clean up, but
induces more modifications on UT. Is this an appropriate change here?
> Clean up deprecated Apache code in Drill
>
>
> Key: DRILL-8117
> URL: https://issues.apache.org/jira/browse/DRILL-8117
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.20.1
>Reporter: Jingchuan Hu
>Priority: Major
> Fix For: 2.0.0
>
>
> Clean up and upgrade deprecated Apache code like:
> Class PathChildrenCache in Class ZookeeperClient and Class StringEscapeUtils
> in Class PlanStringBuilder
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Clean up deprecated Apache code in Drill
[
https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603135#comment-17603135
]
ASF GitHub Bot commented on DRILL-8117:
---
jnturton commented on code in PR #2499:
URL: https://github.com/apache/drill/pull/2499#discussion_r968538795
##
exec/java-exec/src/test/java/org/apache/drill/exec/impersonation/TestInboundImpersonation.java:
##
@@ -156,22 +159,25 @@ public void unauthorizedTarget() throws Exception {
@Test
public void invalidPolicy() throws Exception {
-thrownException.expect(new
UserExceptionMatcher(UserBitShared.DrillPBError.ErrorType.VALIDATION,
-"Invalid impersonation policies."));
+String query = "ALTER SYSTEM SET `%s`='%s'";
Review Comment:
Ah, okay. In that case I think we can go two ways here, since upgrading this
test, or family of tests, to ClusterTest could be a significant expansion of
PR's scope.
1. We can merge this as it is and leave the ClusterTest work for another PR
or
2. we can expand the scope here and you can convert BaseTestImpersonation
from PlanTestBase to ClusterTest. I believe that ClusterTest offers the plan
string matching features in PlanTestBase.
I leave the choice for you to make based on which way you prefer to work.
> Clean up deprecated Apache code in Drill
>
>
> Key: DRILL-8117
> URL: https://issues.apache.org/jira/browse/DRILL-8117
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.20.1
>Reporter: Jingchuan Hu
>Priority: Major
> Fix For: 2.0.0
>
>
> Clean up and upgrade deprecated Apache code like:
> Class PathChildrenCache in Class ZookeeperClient and Class StringEscapeUtils
> in Class PlanStringBuilder
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Clean up deprecated Apache code in Drill
[
https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603127#comment-17603127
]
ASF GitHub Bot commented on DRILL-8117:
---
kingswanwho commented on code in PR #2499:
URL: https://github.com/apache/drill/pull/2499#discussion_r968518305
##
exec/java-exec/src/test/java/org/apache/drill/exec/impersonation/TestInboundImpersonation.java:
##
@@ -156,22 +159,25 @@ public void unauthorizedTarget() throws Exception {
@Test
public void invalidPolicy() throws Exception {
-thrownException.expect(new
UserExceptionMatcher(UserBitShared.DrillPBError.ErrorType.VALIDATION,
-"Invalid impersonation policies."));
+String query = "ALTER SYSTEM SET `%s`='%s'";
Review Comment:
Hi James, the **_client_** here inherits from BaseTestQuery, which is
DrillClient but not ClientFixture in TestLargeFileCompilation as Paul
suggested. DrillClient doesn't have method like client.alterSystem().
BaseTestQuery is deprecated actually, but not marked as deprecated because it
is still widely used. The best choice here I think is change base test class
from BaseTestQuery to ClusterTest so that we could use client.alterSystem() for
clean and compact code style. However there are multiple Inheritance levels
here, which are TestInboundImpersonation -> BaseTestImpersonation ->
PlanTestBase -> BaseTestQuery. Due to PlanTestBase involves many classes. Maybe
change BaseTestImpersonation inheritance from PlanTestBase to ClusterTest
directly is an easier way.
> Clean up deprecated Apache code in Drill
>
>
> Key: DRILL-8117
> URL: https://issues.apache.org/jira/browse/DRILL-8117
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.20.1
>Reporter: Jingchuan Hu
>Priority: Major
> Fix For: 2.0.0
>
>
> Clean up and upgrade deprecated Apache code like:
> Class PathChildrenCache in Class ZookeeperClient and Class StringEscapeUtils
> in Class PlanStringBuilder
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8117) Clean up deprecated Apache code in Drill
[
https://issues.apache.org/jira/browse/DRILL-8117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603126#comment-17603126
]
ASF GitHub Bot commented on DRILL-8117:
---
kingswanwho commented on code in PR #2499:
URL: https://github.com/apache/drill/pull/2499#discussion_r968518305
##
exec/java-exec/src/test/java/org/apache/drill/exec/impersonation/TestInboundImpersonation.java:
##
@@ -156,22 +159,25 @@ public void unauthorizedTarget() throws Exception {
@Test
public void invalidPolicy() throws Exception {
-thrownException.expect(new
UserExceptionMatcher(UserBitShared.DrillPBError.ErrorType.VALIDATION,
-"Invalid impersonation policies."));
+String query = "ALTER SYSTEM SET `%s`='%s'";
Review Comment:
Hi James, the **_client_** here inherits from BaseTestQuery, which is
DrillClient but not ClientFixture in TestLargeFileCompilation as Paul
suggested. DrillClient doesn't have method like client.alterSystem().
BaseTestQuery is deprecated actually, but not marked as deprecated because it
is still widely used. The best choice here I think is change base test class
from BaseTestQuery to ClusterTest so that we could use client.alterSystem() for
clean and compact code style. However there are multiple Inheritance levels
here, which are TestInboundImpersonation -> BaseTestImpersonation ->
PlanTestBase -> BaseTestQuery. Due to that PlanTestBase involves many classes.
Maybe change BaseTestImpersonation inheritance from PlanTestBase to ClusterTest
directly is an easier way.
> Clean up deprecated Apache code in Drill
>
>
> Key: DRILL-8117
> URL: https://issues.apache.org/jira/browse/DRILL-8117
> Project: Apache Drill
> Issue Type: Improvement
>Affects Versions: 1.20.1
>Reporter: Jingchuan Hu
>Priority: Major
> Fix For: 2.0.0
>
>
> Clean up and upgrade deprecated Apache code like:
> Class PathChildrenCache in Class ZookeeperClient and Class StringEscapeUtils
> in Class PlanStringBuilder
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8136) Overhaul implict type casting logic
[
https://issues.apache.org/jira/browse/DRILL-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603085#comment-17603085
]
ASF GitHub Bot commented on DRILL-8136:
---
cgivre commented on PR #2638:
URL: https://github.com/apache/drill/pull/2638#issuecomment-1243728471
> Turns out this particular one isn't possible without breaking other things
that need VARCHAR to go to INT before it goes to DATE.
>
> ```
> apache drill> select date_diff('2022-09-08', '1970-01-01');
> EXPR$0 19243 days 0:00:00
> ```
>
> So we're stuck with the following for date_diff. There do remain other
automatic casts from VARCHAR to DATE that will still succeed though.
>
> ```
> apache drill> select date_diff('2022-09-08', date '1970-01-01');
> EXPR$0 19243 days 0:00:00
> ```
Maybe in for this edge case we do go ahead and make versions of the
`DATE_DIFF()` function that accept strings and do the conversion manually.
(Not in this PR )
> Overhaul implict type casting logic
> ---
>
> Key: DRILL-8136
> URL: https://issues.apache.org/jira/browse/DRILL-8136
> Project: Apache Drill
> Issue Type: Improvement
>Reporter: Esther Buchwalter
>Assignee: James Turton
>Priority: Minor
>
> The existing implicit casting system is built on simplistic total ordering of
> data types[1] that yields oddities such as TINYINT being regarded as the
> closest numeric type to VARCHAR or DATE the closest type to FLOAT8. This, in
> turn, hurts the range of data types with which SQL functions can be used.
> E.g. `select sqrt('3.1415926')` works in many RDBMSes but not in Drill while,
> confusingly, `select '123' + 456` does work in Drill. In addition the
> limitations of the existing type precedence list mean that it has been
> supplmented with ad hoc secondary casting rules that go in the opposite
> direction.
> This Issue proposes a new, more flexible definition of casting distance based
> on a weighted directed graph built over the Drill data types.
> [1]
> [https://drill.apache.org/docs/supported-data-types/#implicit-casting-precedence-of-data-types]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (DRILL-8289) Add Threat Hunting Functions
[ https://issues.apache.org/jira/browse/DRILL-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Givre resolved DRILL-8289. -- Resolution: Done > Add Threat Hunting Functions > > > Key: DRILL-8289 > URL: https://issues.apache.org/jira/browse/DRILL-8289 > Project: Apache Drill > Issue Type: New Feature > Components: Functions - Drill >Affects Versions: 2.0.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 2.0.0 > > > # Threat Hunting Functions > These functions are useful for doing threat hunting with Apache Drill. These > were inspired by huntlib.[1] > The functions are: > * `punctuation_pattern()`: Extracts the pattern of punctuation in > text. > * `entropy()`: This function calculates the Shannon Entropy of a > given string of text. > * `entropyPerByte()`: This function calculates the Shannon Entropy of > a given string of text, normed for the string length. > [1]: https://github.com/target/huntlib -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8136) Overhaul implict type casting logic
[
https://issues.apache.org/jira/browse/DRILL-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603055#comment-17603055
]
ASF GitHub Bot commented on DRILL-8136:
---
jnturton commented on PR #2638:
URL: https://github.com/apache/drill/pull/2638#issuecomment-1243628238
Turns out this particular one isn't possible without breaking other things
that need VARCHAR to go to INT before it goes to DATE.
```
apache drill> select date_diff('2022-09-08', '1970-01-01');
EXPR$0 19243 days 0:00:00
```
So we're stuck with the following for date_diff. There do remain other
automatic casts from VARCHAR to DATE will still succeed though.
```
apache drill> select date_diff(date '2022-09-08', date '1970-01-01');
EXPR$0 19243 days 0:00:00
```
> Overhaul implict type casting logic
> ---
>
> Key: DRILL-8136
> URL: https://issues.apache.org/jira/browse/DRILL-8136
> Project: Apache Drill
> Issue Type: Improvement
>Reporter: Esther Buchwalter
>Assignee: James Turton
>Priority: Minor
>
> The existing implicit casting system is built on simplistic total ordering of
> data types[1] that yields oddities such as TINYINT being regarded as the
> closest numeric type to VARCHAR or DATE the closest type to FLOAT8. This, in
> turn, hurts the range of data types with which SQL functions can be used.
> E.g. `select sqrt('3.1415926')` works in many RDBMSes but not in Drill while,
> confusingly, `select '123' + 456` does work in Drill. In addition the
> limitations of the existing type precedence list mean that it has been
> supplmented with ad hoc secondary casting rules that go in the opposite
> direction.
> This Issue proposes a new, more flexible definition of casting distance based
> on a weighted directed graph built over the Drill data types.
> [1]
> [https://drill.apache.org/docs/supported-data-types/#implicit-casting-precedence-of-data-types]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
