[jira] [Commented] (DRILL-8354) Add IS_EMPTY Function.
[ https://issues.apache.org/jira/browse/DRILL-8354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631125#comment-17631125 ] ASF GitHub Bot commented on DRILL-8354: --- cgivre merged PR #2703: URL: https://github.com/apache/drill/pull/2703 > Add IS_EMPTY Function. > -- > > Key: DRILL-8354 > URL: https://issues.apache.org/jira/browse/DRILL-8354 > Project: Apache Drill > Issue Type: Improvement > Components: Functions - Drill >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 2.0.0 > > > When analyzing data, there is currently no single function to evaluate > whether a given field is empty. With scalar fields, this can be accomplished > with the `IS NOT NULL` operator, but with complex fields, this is more > challenging as complex fields are never null. > This PR adds a UDF called IS_EMPTY() which accepts any type of field and > returns true if the field does not contain data. > > In the case of scalar fields, if the field is `null` this returns true. In > the case of complex fields, which can never be `null`, in the case of lists, > the function returns true if the list is empty. In the case of maps, it > returns true if all of the map's fields are unpopulated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8354) Add IS_EMPTY Function.
[
https://issues.apache.org/jira/browse/DRILL-8354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631053#comment-17631053
]
ASF GitHub Bot commented on DRILL-8354:
---
cgivre commented on code in PR #2703:
URL: https://github.com/apache/drill/pull/2703#discussion_r1017891449
##
exec/java-exec/src/test/resources/jsoninput/is_empty_tests.json:
##
@@ -0,0 +1,27 @@
+[
+ {
+"numeric_col": 1.3,
+"text_col": "text",
+"list_col": [
+ "v1", "v2"
+],
+ "map_column": {
+"field1": "value1",
+"field2": "value2",
+"nested_map": {
+ "nested_field1": "nested_value1",
+ "nested_field2" : "nested_value2"
+ }
+}
+ },{
+"numeric_col": 2.3,
+"text_col": "",
+"list_col": [],
+"map_column": {}
+ },{
+"numeric_col": null,
+"text_col": null,
+"list_col": [],
+"map_column": {}
+ }
Review Comment:
@jnturton Done! I addressed your review comments. Thanks for the quick
review!
> Add IS_EMPTY Function.
> --
>
> Key: DRILL-8354
> URL: https://issues.apache.org/jira/browse/DRILL-8354
> Project: Apache Drill
> Issue Type: Improvement
> Components: Functions - Drill
>Affects Versions: 1.20.2
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 2.0.0
>
>
> When analyzing data, there is currently no single function to evaluate
> whether a given field is empty. With scalar fields, this can be accomplished
> with the `IS NOT NULL` operator, but with complex fields, this is more
> challenging as complex fields are never null.
> This PR adds a UDF called IS_EMPTY() which accepts any type of field and
> returns true if the field does not contain data.
>
> In the case of scalar fields, if the field is `null` this returns true. In
> the case of complex fields, which can never be `null`, in the case of lists,
> the function returns true if the list is empty. In the case of maps, it
> returns true if all of the map's fields are unpopulated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8354) Add IS_EMPTY Function.
[
https://issues.apache.org/jira/browse/DRILL-8354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631049#comment-17631049
]
ASF GitHub Bot commented on DRILL-8354:
---
cgivre commented on code in PR #2703:
URL: https://github.com/apache/drill/pull/2703#discussion_r1017888203
##
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/IsEmptyUtils.java:
##
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import org.apache.drill.common.types.TypeProtos.MinorType;
+import org.apache.drill.exec.vector.complex.impl.SingleMapReaderImpl;
+import org.apache.drill.exec.vector.complex.reader.FieldReader;
+
+public class IsEmptyUtils {
+
+ /**
+ * This function recursively traverses a Drill map to determine whether the
map is empty or not.
+ * @param reader A {@link FieldReader} containing the field in question
+ * @return True if the field contains no data, false if it does.
+ */
+ public static boolean mapIsEmpty(FieldReader reader) {
+
+if (reader.getType().getMinorType() == MinorType.MAP) {
+ SingleMapReaderImpl mapReader = (SingleMapReaderImpl) reader;
+
+ // If the map reader has no fields returns nothing return true
Review Comment:
Fixed.
> Add IS_EMPTY Function.
> --
>
> Key: DRILL-8354
> URL: https://issues.apache.org/jira/browse/DRILL-8354
> Project: Apache Drill
> Issue Type: Improvement
> Components: Functions - Drill
>Affects Versions: 1.20.2
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 2.0.0
>
>
> When analyzing data, there is currently no single function to evaluate
> whether a given field is empty. With scalar fields, this can be accomplished
> with the `IS NOT NULL` operator, but with complex fields, this is more
> challenging as complex fields are never null.
> This PR adds a UDF called IS_EMPTY() which accepts any type of field and
> returns true if the field does not contain data.
>
> In the case of scalar fields, if the field is `null` this returns true. In
> the case of complex fields, which can never be `null`, in the case of lists,
> the function returns true if the list is empty. In the case of maps, it
> returns true if all of the map's fields are unpopulated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8354) Add IS_EMPTY Function.
[
https://issues.apache.org/jira/browse/DRILL-8354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630886#comment-17630886
]
ASF GitHub Bot commented on DRILL-8354:
---
jnturton commented on code in PR #2703:
URL: https://github.com/apache/drill/pull/2703#discussion_r1017701372
##
exec/java-exec/src/test/resources/jsoninput/is_empty_tests.json:
##
@@ -0,0 +1,27 @@
+[
+ {
+"numeric_col": 1.3,
+"text_col": "text",
+"list_col": [
+ "v1", "v2"
+],
+ "map_column": {
+"field1": "value1",
+"field2": "value2",
+"nested_map": {
+ "nested_field1": "nested_value1",
+ "nested_field2" : "nested_value2"
+ }
+}
+ },{
+"numeric_col": 2.3,
+"text_col": "",
+"list_col": [],
+"map_column": {}
+ },{
+"numeric_col": null,
+"text_col": null,
+"list_col": [],
+"map_column": {}
+ }
Review Comment:
```suggestion
}, {
"numeric_col": 1
}
```
Can we add an object that is outright missing some queried properties and
check that the is_empty unit tests pick those up as being empty?
##
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/IsEmptyUtils.java:
##
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.expr.fn.impl;
+
+import org.apache.drill.common.types.TypeProtos.MinorType;
+import org.apache.drill.exec.vector.complex.impl.SingleMapReaderImpl;
+import org.apache.drill.exec.vector.complex.reader.FieldReader;
+
+public class IsEmptyUtils {
+
+ /**
+ * This function recursively traverses a Drill map to determine whether the
map is empty or not.
+ * @param reader A {@link FieldReader} containing the field in question
+ * @return True if the field contains no data, false if it does.
+ */
+ public static boolean mapIsEmpty(FieldReader reader) {
+
+if (reader.getType().getMinorType() == MinorType.MAP) {
+ SingleMapReaderImpl mapReader = (SingleMapReaderImpl) reader;
+
+ // If the map reader has no fields returns nothing return true
Review Comment:
```suggestion
// If the map reader has no fields return true
```
> Add IS_EMPTY Function.
> --
>
> Key: DRILL-8354
> URL: https://issues.apache.org/jira/browse/DRILL-8354
> Project: Apache Drill
> Issue Type: Improvement
> Components: Functions - Drill
>Affects Versions: 1.20.2
>Reporter: Charles Givre
>Assignee: Charles Givre
>Priority: Major
> Fix For: 2.0.0
>
>
> When analyzing data, there is currently no single function to evaluate
> whether a given field is empty. With scalar fields, this can be accomplished
> with the `IS NOT NULL` operator, but with complex fields, this is more
> challenging as complex fields are never null.
> This PR adds a UDF called IS_EMPTY() which accepts any type of field and
> returns true if the field does not contain data.
>
> In the case of scalar fields, if the field is `null` this returns true. In
> the case of complex fields, which can never be `null`, in the case of lists,
> the function returns true if the list is empty. In the case of maps, it
> returns true if all of the map's fields are unpopulated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8354) Add IS_EMPTY Function.
[ https://issues.apache.org/jira/browse/DRILL-8354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630639#comment-17630639 ] ASF GitHub Bot commented on DRILL-8354: --- cgivre opened a new pull request, #2703: URL: https://github.com/apache/drill/pull/2703 # [DRILL-8354](https://issues.apache.org/jira/browse/DRILL-8354): Add IS_EMPTY Function ## Description When analyzing data, there is currently no single function to evaluate whether a given field is empty. With scalar fields, this can be accomplished with the `IS NOT NULL` operator, but with complex fields, this is more challenging as complex fields are never null. This PR adds a UDF called `IS_EMPTY()` which accepts any type of field and returns true if the field does not contain data. In the case of scalar fields, if the field is `null` this returns true. In the case of complex fields, which can never be `null`, in the case of lists, the function returns true if the list is empty. In the case of maps, it returns true if all of the map's fields are unpopulated. ## Documentation See above. ## Testing Added unit tests. > Add IS_EMPTY Function. > -- > > Key: DRILL-8354 > URL: https://issues.apache.org/jira/browse/DRILL-8354 > Project: Apache Drill > Issue Type: Improvement > Components: Functions - Drill >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 2.0.0 > > > When analyzing data, there is currently no single function to evaluate > whether a given field is empty. With scalar fields, this can be accomplished > with the `IS NOT NULL` operator, but with complex fields, this is more > challenging as complex fields are never null. > This PR adds a UDF called IS_EMPTY() which accepts any type of field and > returns true if the field does not contain data. > > In the case of scalar fields, if the field is `null` this returns true. In > the case of complex fields, which can never be `null`, in the case of lists, > the function returns true if the list is empty. In the case of maps, it > returns true if all of the map's fields are unpopulated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
