Change in asterixdb[master]: [NO ISSUE] Fix type inference for ANY in UDFs
Xikui Wang has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3405 ) Change subject: [NO ISSUE] Fix type inference for ANY in UDFs .. Patch Set 3: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3405 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6a03749fe041ec6f43de230d3e784122f7664545 Gerrit-Change-Number: 3405 Gerrit-PatchSet: 3 Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-Comment-Date: Sun, 09 Jun 2019 00:11:30 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE] Fix type inference for ANY in UDFs
Xikui Wang has posted comments on this change. (
https://asterix-gerrit.ics.uci.edu/3405 )
Change subject: [NO ISSUE] Fix type inference for ANY in UDFs
..
Patch Set 2:
(1 comment)
https://asterix-gerrit.ics.uci.edu/#/c/3405/2/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
File
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp:
https://asterix-gerrit.ics.uci.edu/#/c/3405/2/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp@21
PS2, Line 21: let i={"id":1, "text_list": [{"text":"lower text1"},
{"text":"lower text2"}], "another_list" : ["string_1", "string_2"]}
> the expected result did not change. why is that?
The open field is not accessed in the function. It's here to make sure the
parameters can be translated into Java objects properly.
--
To view, visit https://asterix-gerrit.ics.uci.edu/3405
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6a03749fe041ec6f43de230d3e784122f7664545
Gerrit-Change-Number: 3405
Gerrit-PatchSet: 2
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Xikui Wang
Gerrit-Comment-Date: Fri, 07 Jun 2019 22:44:49 +
Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIX-2560] edit_distance() is case insensitive
Xikui Wang has abandoned this change. ( https://asterix-gerrit.ics.uci.edu/3380 ) Change subject: [ASTERIX-2560] edit_distance() is case insensitive .. Abandoned The change of edit-distance changes some behaviors of the similarity join, which is not very clear to me. -- To view, visit https://asterix-gerrit.ics.uci.edu/3380 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: Id41bcf58a08435b5611c660609085d320691b990 Gerrit-Change-Number: 3380 Gerrit-PatchSet: 1 Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE] Fix type inference for ANY in UDFs
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3405 to look at the new patch set (#2). Change subject: [NO ISSUE] Fix type inference for ANY in UDFs .. [NO ISSUE] Fix type inference for ANY in UDFs - user model changes: no - storage format changes: no - interface changes: no Details: When data type is available, we should use defined data type. If not, we should try to parse the data type from the data. Change-Id: I6a03749fe041ec6f43de230d3e784122f7664545 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java 2 files changed, 13 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/05/3405/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3405 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6a03749fe041ec6f43de230d3e784122f7664545 Gerrit-Change-Number: 3405 Gerrit-PatchSet: 2 Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE] Fix type inference for ANY in UDFs
Xikui Wang has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/3405
Change subject: [NO ISSUE] Fix type inference for ANY in UDFs
..
[NO ISSUE] Fix type inference for ANY in UDFs
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
When data type is available, we should use defined data type. If not, we
should try to parse the data type from the data.
Change-Id: I6a03749fe041ec6f43de230d3e784122f7664545
---
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
3 files changed, 14 insertions(+), 4 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/05/3405/1
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
index 743d39a..62c2b87 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
@@ -18,7 +18,7 @@
*/
use externallibtest;
-let i={"id":1, "text_list": [{"text":"lower text1"}, {"text":"lower text2"}]}
+let i={"id":1, "text_list": [{"text":"lower text1"}, {"text":"lower text2"}],
"another_list" : ["string_1", "string_2"]}
select value `testlib#toUpper`(i);
let i=`testlib#toUpper`({"id":1, "text_list":[{"text":"lower text"}]})
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
index 65ecd8d..678b3d7 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
@@ -90,7 +90,7 @@
@Override
public String toString() {
-return String.valueOf(value, 0, size == 0 ? 0 : size - 1);
+return String.valueOf(value, 0, size == 0 ? 0 : size);
}
public void endRecord() throws IOException {
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
index ee8308f..3af08fa 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
@@ -583,10 +583,20 @@
public IJObject access(AListVisitablePointable pointable,
IObjectPool objectPool,
IAType listType, JObjectPointableVisitor pointableVisitor)
throws HyracksDataException {
List items = pointable.getItems();
+List itemTags = pointable.getItemTags();
JList list = pointable.ordered() ? new JOrderedList(listType) :
new JUnorderedList(listType);
IJObject listItem;
-for (IVisitablePointable itemPointable : items) {
-final IAType fieldType = ((AbstractCollectionType)
listType).getItemType();
+for (int iter1 = 0; iter1 < items.size(); iter1++) {
+IVisitablePointable itemPointable = items.get(iter1);
+// First, try to get defined type.
+IAType fieldType = ((AbstractCollectionType)
listType).getItemType();
+if (fieldType.getTypeTag() == ATypeTag.ANY) {
+// Second, if defined type is not available, try to infer
it from data
+IVisitablePointable itemTagPointable = itemTags.get(iter1);
+ATypeTag itemTypeTag =
EnumDeserializer.ATYPETAGDESERIALIZER
+
.deserialize(itemTagPointable.getByteArray()[itemTagPointable.getStartOffset()]);
+fieldType = TypeTagUtil.getBuiltinTypeByTag(itemTypeTag);
+}
typeInfo.reset(fieldType, fieldType.getTypeTag());
switch (typeInfo.getTypeTag()) {
case OBJECT:
--
To view, visit https://asterix-gerrit.ics.uci.edu/3405
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6a03749fe041ec6f43de230d3e784122f7664545
Ger
Change in asterixdb[master]: [ASTERIX-2559] Prevent missing parameter in external UDF
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3379 to look at the new patch set (#3). Change subject: [ASTERIX-2559] Prevent missing parameter in external UDF .. [ASTERIX-2559] Prevent missing parameter in external UDF - user model changes: yes UNION typed parameters are not allowed in external UDF parameters - storage format changes: no - interface changes: no Details: When there is a UNION typed parameter in an external UDF, the IntroduceDynamicTypeCastForExternalFunctionRule would repeatedly introduce cast function. When such parameter appears, the external UDF should throw an exception when the value is missing. Change-Id: I1fe2098d1e1c5e179a0635d9d6cc3a1534278259 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.2.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.3.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.6.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.7.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.8.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.1.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.2.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.3.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.4.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.5.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/parameter_casting/parameter_casting.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml 17 files changed, 228 insertions(+), 16 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/79/3379/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3379 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1fe2098d1e1c5e179a0635d9d6cc3a1534278259 Gerrit-Change-Number: 3379 Gerrit-PatchSet: 3 Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIX-2559] Prevent missing parameter in external UDF
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3379 to look at the new patch set (#2). Change subject: [ASTERIX-2559] Prevent missing parameter in external UDF .. [ASTERIX-2559] Prevent missing parameter in external UDF - user model changes: yes UNION typed parameters are not allowed in external UDF parameters - storage format changes: no - interface changes: no Details: When there is a UNION typed parameter in an external UDF, the IntroduceDynamicTypeCastForExternalFunctionRule would repeatedly introduce cast function. When such parameter appears, the external UDF should throw an exception when the value is missing. Change-Id: I1fe2098d1e1c5e179a0635d9d6cc3a1534278259 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.2.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.3.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.1.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.2.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.3.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.4.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.5.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/parameter_casting/parameter_casting.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml 15 files changed, 187 insertions(+), 16 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/79/3379/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3379 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1fe2098d1e1c5e179a0635d9d6cc3a1534278259 Gerrit-Change-Number: 3379 Gerrit-PatchSet: 2 Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIX-2560] edit_distance() is case insensitive
Xikui Wang has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/3380
Change subject: [ASTERIX-2560] edit_distance() is case insensitive
..
[ASTERIX-2560] edit_distance() is case insensitive
- user model changes: yes
edit_distance() now is case sensitive
- storage format changes: no
- interface changes: no
Detail:
The edit_distance() function is not case sensitive. This is not
following the common definition of edit distance.
Change-Id: Id41bcf58a08435b5611c660609085d320691b990
---
M
hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/UTF8StringCharByCharIterator.java
1 file changed, 2 insertions(+), 2 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/80/3380/1
diff --git
a/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/UTF8StringCharByCharIterator.java
b/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/UTF8StringCharByCharIterator.java
index 237d291..5cab892 100644
---
a/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/UTF8StringCharByCharIterator.java
+++
b/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/UTF8StringCharByCharIterator.java
@@ -36,8 +36,8 @@
@Override
public int compare(ISequenceIterator cmpIter) throws HyracksDataException {
-char thisChar = Character.toLowerCase(UTF8StringUtil.charAt(data,
pos));
-char thatChar =
Character.toLowerCase(UTF8StringUtil.charAt(cmpIter.getData(),
cmpIter.getPos()));
+char thisChar = UTF8StringUtil.charAt(data, pos);
+char thatChar = UTF8StringUtil.charAt(cmpIter.getData(),
cmpIter.getPos());
if (thisChar == thatChar) {
return 0;
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/3380
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id41bcf58a08435b5611c660609085d320691b990
Gerrit-Change-Number: 3380
Gerrit-PatchSet: 1
Gerrit-Owner: Xikui Wang
Change in asterixdb[master]: [ASTERIX-2559] Prevent missing parameter in external UDF
Xikui Wang has removed Dmitry Lychagin from this change. ( https://asterix-gerrit.ics.uci.edu/3379 ) Change subject: [ASTERIX-2559] Prevent missing parameter in external UDF .. Removed reviewer Dmitry Lychagin. -- To view, visit https://asterix-gerrit.ics.uci.edu/3379 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: deleteReviewer Gerrit-Change-Id: I1fe2098d1e1c5e179a0635d9d6cc3a1534278259 Gerrit-Change-Number: 3379 Gerrit-PatchSet: 1 Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIX-2559] Prevent missing parameter in external UDF
Xikui Wang has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/3379
Change subject: [ASTERIX-2559] Prevent missing parameter in external UDF
..
[ASTERIX-2559] Prevent missing parameter in external UDF
- user model changes: yes
UNION typed parameters are not allowed in external UDF parameters
- storage format changes: no
- interface changes: no
Details:
When there is a UNION typed parameter in an external UDF, the
IntroduceDynamicTypeCastForExternalFunctionRule would repeatedly
introduce cast function. When such parameter appears, the external UDF
should throw an exception when the value is missing.
Change-Id: I1fe2098d1e1c5e179a0635d9d6cc3a1534278259
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.2.lib.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.3.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.4.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.5.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/parameter_casting/parameter_casting.7.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/parameter_casting/parameter_casting.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml
9 files changed, 166 insertions(+), 14 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/79/3379/1
diff --git
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java
index a00933a..fcf931d 100644
---
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java
+++
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java
@@ -75,40 +75,48 @@
IAType inputType;
IAType reqArgType;
boolean castFlag;
+boolean checkUnknown = false;
for (int i = 0; i < funcCallExpr.getArguments().size(); i++) {
Mutable argExpr =
funcCallExpr.getArguments().get(i);
inputType = (IAType)
op.computeOutputTypeEnvironment(context).getType(argExpr.getValue());
reqArgType = ((ExternalScalarFunctionInfo)
funcCallExpr.getFunctionInfo()).getArgumentTypes().get(i);
-
+while (NonTaggedFormatUtil.isOptional(inputType)) {
+inputType = ((AUnionType) inputType).getActualType();
+checkUnknown = true;
+}
if (reqArgType.getTypeTag() == ATypeTag.OBJECT) {
castFlag =
!IntroduceDynamicTypeCastRule.compatible((ARecordType) reqArgType, inputType,
argExpr.getValue().getSourceLocation());
} else {
castFlag = !reqArgType.equals(inputType);
}
-/**
- * the input record type can be an union type
- * for the case when it comes from a subplan or left-outer join
- */
-boolean checkUnknown = false;
-while (NonTaggedFormatUtil.isOptional(inputType)) {
-/** while-loop for the case there is a nested multi-level
union */
-inputType = ((AUnionType) inputType).getActualType();
-checkUnknown = true;
+
+if (checkUnknown) {
+AbstractFunctionCallExpression chkUnknownFunc =
+new
ScalarFunctionCallExpression(FunctionUtil.getFunctionInfo(BuiltinFunctions.CHECK_UNKNOWN));
+TypeCastUtils.setRequiredAndInputTypes(chkUnknownFunc,
reqArgType, inputType);
+wrapFunctionParameter(funcCallExpr, argExpr, chkUnknownFunc,
i);
+changed = true;
}
-if (castFlag || checkUnknown) {
+
+if (castFlag) {
AbstractFunctionCallExpression castFunc =
new
ScalarFunctionCallExpression(FunctionUtil.getFunctionInfo(BuiltinFunctions.CAST_TYPE));
-
castFunc.setSourceLocation(argExpr.getValue().getSourceLocation());
-castFunc.getArguments().add(argExpr);
TypeCastUtils.setRequiredAndInputType
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Xikui Wang has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3270 ) Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs .. [NO ISSUE][FUN] Fix type inference and casting in UDFs - user model changes: no - storage format changes: no - interface changes: no Details: The current UDF framework handles the argument types in a sloppy way. It takes in the arguments and reads them as the expected data types in the configuration. This could cause exception at runtime when passing in arguments with unexpected datatypes. When setting the arguments, it did a type casting for numeric values only to make sure the int64 from query interface can be evaluated properly. However, this is not robust enough. This patch fixes the type inference for UDFs during the complation time. The ExternalTypeComputer is refactored to return defined data type, and meanwhile checks the argument data types. Also, the IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the type castings for data types besides record type. Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f Reviewed-on: https://asterix-gerrit.ics.uci.edu/3270 Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Xikui Wang --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.1.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.2.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.3.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.6.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.7.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.4.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.6.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/mysum/mysum.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunctionProvider.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFactory.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFactory.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalTypeComputer.java M asterixdb/asterix-metadata/src/test/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtilTest.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/ExternalFunctionInfo.java A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/mysum/mysum.2.adm M asterixdb/asterix-server/src/test/resources/integrationts/library/testsuite.xml 28 files changed, 124 insertions(+), 188 deletions(-) Approvals: Jenkins: Verified; Verified Xikui Wang: Looks good to me, approved Objections: Anon. E. Moose (1000171): Violations found Jenkins: Violations found diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java index 97e2174..a00933a 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Xikui Wang has posted comments on this change. (
https://asterix-gerrit.ics.uci.edu/3270 )
Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs
..
Patch Set 9: Code-Review+2
(6 comments)
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java
File
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java:
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java@45
PS7, Line 45: protected final IExternalFunction externalFunctionInstance;
> MAJOR SonarQube violation:
Done
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java
File
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java:
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java@41
PS7, Line 41: public class ExternalFunctionCompilerUtil {
> MAJOR SonarQube violation:
Done
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java@142
PS7, Line 142: }
> MAJOR SonarQube violation:
save for future uses.
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java@146
PS7, Line 146: private static IFunctionInfo
getUnnestFunctionInfo(MetadataTransactionContext txnCtx, Function function) {
> MAJOR SonarQube violation:
same as above.
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java@150
PS7, Line 150: private static IFunctionInfo
getStatefulFunctionInfo(MetadataTransactionContext txnCtx, Function function) {
> MAJOR SonarQube violation:
same as above.
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/ExternalFunctionInfo.java
File
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/ExternalFunctionInfo.java:
https://asterix-gerrit.ics.uci.edu/#/c/3270/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/ExternalFunctionInfo.java@27
PS7, Line 27: public class ExternalFunctionInfo extends FunctionInfo implements
IExternalFunctionInfo {
> MAJOR SonarQube violation:
dk whether this should be overridden or not. The comparison logic is not changed
--
To view, visit https://asterix-gerrit.ics.uci.edu/3270
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f
Gerrit-Change-Number: 3270
Gerrit-PatchSet: 9
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Michael Blow
Gerrit-Reviewer: Till Westmann
Gerrit-Reviewer: Xikui Wang
Gerrit-Comment-Date: Tue, 16 Apr 2019 05:50:32 +
Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3270 to look at the new patch set (#8). Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs .. [NO ISSUE][FUN] Fix type inference and casting in UDFs - user model changes: no - storage format changes: no - interface changes: no Details: The current UDF framework handles the argument types in a sloppy way. It takes in the arguments and reads them as the expected data types in the configuration. This could cause exception at runtime when passing in arguments with unexpected datatypes. When setting the arguments, it did a type casting for numeric values only to make sure the int64 from query interface can be evaluated properly. However, this is not robust enough. This patch fixes the type inference for UDFs during the complation time. The ExternalTypeComputer is refactored to return defined data type, and meanwhile checks the argument data types. Also, the IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the type castings for data types besides record type. Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.1.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.2.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.3.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.6.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.7.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.4.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.6.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/mysum/mysum.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunctionProvider.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFactory.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFactory.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalTypeComputer.java M asterixdb/asterix-metadata/src/test/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtilTest.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/ExternalFunctionInfo.java A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/mysum/mysum.2.adm M asterixdb/asterix-server/src/test/resources/integrationts/library/testsuite.xml 28 files changed, 124 insertions(+), 188 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3270/8 -- To view, visit https://asterix-gerrit.ics.uci.edu/3270 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f Gerrit-PatchSet: 8 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xiku
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3270 to look at the new patch set (#7). Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs .. [NO ISSUE][FUN] Fix type inference and casting in UDFs - user model changes: no - storage format changes: no - interface changes: no Details: The current UDF framework handles the argument types in a sloppy way. It takes in the arguments and reads them as the expected data types in the configuration. This could cause exception at runtime when passing in arguments with unexpected datatypes. When setting the arguments, it did a type casting for numeric values only to make sure the int64 from query interface can be evaluated properly. However, this is not robust enough. This patch fixes the type inference for UDFs during the complation time. The ExternalTypeComputer is refactored to return defined data type, and meanwhile checks the argument data types. Also, the IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the type castings for data types besides record type. Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.1.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.2.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.3.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.6.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.7.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.4.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.6.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/mysum/mysum.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFactory.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFactory.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalTypeComputer.java M asterixdb/asterix-metadata/src/test/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtilTest.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/ExternalFunctionInfo.java A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/mysum/mysum.2.adm M asterixdb/asterix-server/src/test/resources/integrationts/library/testsuite.xml 27 files changed, 115 insertions(+), 183 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3270/7 -- To view, visit https://asterix-gerrit.ics.uci.edu/3270 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f Gerrit-PatchSet: 7 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3270 to look at the new patch set (#6). Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs .. [NO ISSUE][FUN] Fix type inference and casting in UDFs - user model changes: no - storage format changes: no - interface changes: no Details: The current UDF framework handles the argument types in a sloppy way. It takes in the arguments and reads them as the expected data types in the configuration. This could cause exception at runtime when passing in arguments with unexpected datatypes. When setting the arguments, it did a type casting for numeric values only to make sure the int64 from query interface can be evaluated properly. However, this is not robust enough. This patch fixes the type inference for UDFs during the complation time. The ExternalTypeComputer is refactored to return defined data type, and meanwhile checks the argument data types. Also, the IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the type castings for data types besides record type. Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.1.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.2.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.3.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.6.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.7.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.4.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.6.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/mysum/mysum.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFactory.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFactory.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalTypeComputer.java M asterixdb/asterix-metadata/src/test/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtilTest.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/ExternalFunctionInfo.java A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/mysum/mysum.2.adm M asterixdb/asterix-server/src/test/resources/integrationts/library/testsuite.xml 27 files changed, 116 insertions(+), 172 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3270/6 -- To view, visit https://asterix-gerrit.ics.uci.edu/3270 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f Gerrit-PatchSet: 6 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Xikui Wang has posted comments on this change. Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs .. Patch Set 3: (3 comments) Thanks for your comments! https://asterix-gerrit.ics.uci.edu/#/c/3270/3/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java File asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java: PS3, Line 80: Boolean > /boolean/ Done PS3, Line 81: iter1 > not intro'd in this change, but can we rename iter1 to i or index or someth Is there a particular benefit for changing this, or it's a general stylistic rule? The reason I ask is bc I have been naming iterator variables this way for some time... It would useful to know if that's not the right way. PS3, Line 84: getArgumenTypes > /getArgumentTypes/ Done -- To view, visit https://asterix-gerrit.ics.uci.edu/3270 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3270 to look at the new patch set (#4). Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs .. [NO ISSUE][FUN] Fix type inference and casting in UDFs - user model changes: no - storage format changes: no - interface changes: no Details: The current UDF framework handles the argument types in a sloppy way. It takes in the arguments and reads them as the expected data types in the configuration. This could cause exception at runtime when passing in arguments with unexpected datatypes. When setting the arguments, it did a type casting for numeric values only to make sure the int64 from query interface can be evaluated properly. However, this is not robust enough. This patch fixes the type inference for UDFs during the complation time. The ExternalTypeComputer is refactored to return defined data type, and meanwhile checks the argument data types. Also, the IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the type castings for data types besides record type. Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.1.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.2.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.3.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.6.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.7.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.4.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.6.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/mysum/mysum.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFactory.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFactory.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalTypeComputer.java M asterixdb/asterix-metadata/src/test/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtilTest.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/ExternalFunctionInfo.java A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/mysum/mysum.2.adm M asterixdb/asterix-server/src/test/resources/integrationts/library/testsuite.xml 27 files changed, 115 insertions(+), 172 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3270/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3270 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference in JListAccessor
Xikui Wang has submitted this change and it was merged.
Change subject: [NO ISSUE][FUN] Fix type inference in JListAccessor
..
[NO ISSUE][FUN] Fix type inference in JListAccessor
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
As the list element type is available, there is no need to deserialize
element types from the data. Doing so, for example, will cause the
object type to be fully open type and closed fields to be recognized as
open fields. This will further cause NPE in UDFs. Modified the upperCase
test case to check this issue.
Change-Id: I4f437857779e5c0af175c695938fbd7208da244d
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3264
Sonar-Qube: Jenkins
Tested-by: Jenkins
Contrib: Jenkins
Integration-Tests: Jenkins
Reviewed-by: Dmitry Lychagin
---
D
asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.1.ddl.aql
D
asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.2.lib.aql
D
asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.3.query.aql
D
asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.4.lib.aql
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.1.ddl.sqlpp
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
M
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/upperCase/upperCase.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
M
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/UpperCaseFunction.java
10 files changed, 32 insertions(+), 120 deletions(-)
Approvals:
Jenkins: Verified; No violations found; ; Verified
Dmitry Lychagin: Looks good to me, approved
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.1.ddl.aql
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.1.ddl.aql
deleted file mode 100644
index 2dcd24d..000
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.1.ddl.aql
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-drop dataverse externallibtest if exists;
-create dataverse externallibtest;
-use dataverse externallibtest;
-
-create type TextType if not exists as open {
-id: int32,
-text: string
-};
-
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.2.lib.aql
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.2.lib.aql
deleted file mode 100644
index d1e0e87..000
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.2.lib.aql
+++ /dev/null
@@ -1,19 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-install externallibtest testlib
target/data/externallib/asterix-external-data-testlib.zip
\ No newline at end of file
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/que
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3270 to look at the new patch set (#3). Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs .. [NO ISSUE][FUN] Fix type inference and casting in UDFs - user model changes: no - storage format changes: no - interface changes: no Details: The current UDF framework handles the argument types in a sloppy way. It takes in the arguments and reads them as the expected data types in the configuration. This could cause exception at runtime when passing in arguments with unexpected datatypes. When setting the arguments, it did a type casting for numeric values only to make sure the int64 from query interface can be evaluated properly. However, this is not robust enough. This patch fixes the type inference for UDFs during the complation time. The ExternalTypeComputer is refactored to return defined data type, and meanwhile checks the argument data types. Also, the IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the type castings for data types besides record type. Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.1.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.2.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.3.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.6.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.7.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.4.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.6.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/mysum/mysum.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFactory.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFactory.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalTypeComputer.java M asterixdb/asterix-metadata/src/test/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtilTest.java A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/mysum/mysum.2.adm M asterixdb/asterix-server/src/test/resources/integrationts/library/testsuite.xml 26 files changed, 114 insertions(+), 168 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3270/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3270 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference and casting in UDFs
Xikui Wang has uploaded a new change for review. https://asterix-gerrit.ics.uci.edu/3270 Change subject: [NO ISSUE][FUN] Fix type inference and casting in UDFs .. [NO ISSUE][FUN] Fix type inference and casting in UDFs - user model changes: no - storage format changes: no - interface changes: no Details: The current UDF framework handles the argument types in a sloppy way. It takes in the arguments and reads them as the expected data types in the configuration. This could cause exception at runtime when passing in arguments with unexpected datatypes. When setting the arguments, it did a type casting for numeric values only to make sure the int64 from query interface can be evaluated properly. However, this is not robust enough. This patch fixes the type inference for UDFs during the complation time. The ExternalTypeComputer is refactored to return defined data type, and meanwhile checks the argument data types. Also, the IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the type castings for data types besides record type. Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.1.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.2.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.3.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.6.lib.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/my_array_sum/my_array_sum.7.ddl.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.4.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.6.lib.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/mysum/mysum.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalFunction.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFactory.java C asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MyArraySumFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFactory.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/MySumFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalTypeComputer.java A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/my_array_sum/my_array_sum.1.adm A asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-functions/mysum/mysum.2.adm M asterixdb/asterix-server/src/test/resources/integrationts/library/testsuite.xml 25 files changed, 115 insertions(+), 166 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3270/1 diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java index 97e2174..4041f44 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java @@ -21,11 +21,15 @@ import org.apache.asterix.lang.common.util.FunctionUtil; import org.apache.asterix.metadata.functions.ExternalScalarFunctionInfo; +import org.apache.asterix.om.base.ARecord; import org.apache.asterix.om.functions.BuiltinFunctions; import org.apache.asterix.om.typecomputer.base.TypeCastUtils; import org.a
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference in JListAccessor
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3264 to look at the new patch set (#2). Change subject: [NO ISSUE][FUN] Fix type inference in JListAccessor .. [NO ISSUE][FUN] Fix type inference in JListAccessor - user model changes: no - storage format changes: no - interface changes: no Details: As the list element type is available, there is no need to deserialize element types from the data. Doing so, for example, will cause the object type to be fully open type and closed fields to be recognized as open fields. This will further cause NPE in UDFs. Modified the upperCase test case to check this issue. Change-Id: I4f437857779e5c0af175c695938fbd7208da244d --- D asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.1.ddl.aql D asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.2.lib.aql D asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.3.query.aql D asterixdb/asterix-app/src/test/resources/runtimets/queries/external-library/upperCase/upperCase.4.lib.aql M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.1.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/upperCase/upperCase.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/UpperCaseFunction.java 10 files changed, 32 insertions(+), 120 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/64/3264/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3264 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4f437857779e5c0af175c695938fbd7208da244d Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE][FUN] Fix type inference in JListAccessor
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3264
Change subject: [NO ISSUE][FUN] Fix type inference in JListAccessor
..
[NO ISSUE][FUN] Fix type inference in JListAccessor
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
As the list element type is available, there is no need to deserialize
element types from the data. Doing so, for example, will cause the
object type to be fully open type and closed fields to be recognized as
open fields. This will further cause NPE in UDFs. Modified the upperCase
test case to check this issue.
Change-Id: I4f437857779e5c0af175c695938fbd7208da244d
---
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.1.ddl.sqlpp
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
M
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/upperCase/upperCase.1.adm
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
M
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/UpperCaseFunction.java
5 files changed, 32 insertions(+), 22 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/64/3264/1
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.1.ddl.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.1.ddl.sqlpp
index 8130de2..292e4e1 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.1.ddl.sqlpp
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.1.ddl.sqlpp
@@ -20,7 +20,11 @@
CREATE DATAVERSE externallibtest;
USE externallibtest;
+create type list_element as open{
+text: string
+};
+
create type TextType if not exists as open {
id: int32,
-text: string
+text_list: [list_element]
};
\ No newline at end of file
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
index f361232..743d39a 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/upperCase/upperCase.3.query.sqlpp
@@ -18,11 +18,11 @@
*/
use externallibtest;
-let i={"id":1, "text":"lower text"}
+let i={"id":1, "text_list": [{"text":"lower text1"}, {"text":"lower text2"}]}
select value `testlib#toUpper`(i);
-let i=`testlib#toUpper`({"id":1, "text":"lower text"})
+let i=`testlib#toUpper`({"id":1, "text_list":[{"text":"lower text"}]})
select value i;
-let i= {"field1" : `testlib#toUpper`({"id":1, "text":"lower text"}), "field2":
123}
+let i= {"field1" : `testlib#toUpper`({"id":1, "text_list":[{"text":"lower
text"}]}), "field2": 123}
select value i;
\ No newline at end of file
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/upperCase/upperCase.1.adm
b/asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/upperCase/upperCase.1.adm
index e4af2be..2efaa4f 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/upperCase/upperCase.1.adm
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/upperCase/upperCase.1.adm
@@ -1,3 +1,3 @@
-{ "id": -1, "text": "LOWER TEXT" }
-{ "id": -1, "text": "LOWER TEXT" }
-{ "field1": { "id": -1, "text": "LOWER TEXT" }, "field2": 123 }
+{ "id": -1, "text_list": [ { "text": "lower text1" }, { "text": "lower text2"
} ], "element_n": 2, "capitalized_list": [ "LOWER TEXT1", "LOWER TEXT2" ] }
+{ "id": -1, "text_list": [ { "text": "lower text" } ], "element_n": 1,
"capitalized_list": [ "LOWER TEXT" ] }
+{ "field1": { "id": -1, "text_list": [ { "text": "lower text" } ],
"element_n": 1, "capitalized_list": [ "LOWER TEXT" ] }, "field2": 123 }
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
index 2678a8d..1a6553c 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/JObjectAccessors.java
@@ -82,8 +82,10 @@
import org.apache.asterix.om.pointables.AListVisitablePointable;
import org.apache.asterix.om.p
Change in asterixdb[master]: [NO ISSUE][RT] Fix flush in OneFramePushRuntime
Xikui Wang has abandoned this change. Change subject: [NO ISSUE][RT] Fix flush in OneFramePushRuntime .. Abandoned Not right. -- To view, visit https://asterix-gerrit.ics.uci.edu/3256 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I2c2768a41fb9d23f8ce990c8c5c52ec14c2319e5 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE][RT] Fix flush in OneFramePushRuntime
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3256
Change subject: [NO ISSUE][RT] Fix flush in OneFramePushRuntime
..
[NO ISSUE][RT] Fix flush in OneFramePushRuntime
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
The AbstractOneInputOneOutputOneFramePushRuntime doesn't flush data
records in its flushAndReset() method. This causes the data records are
not flushed as expected if there is a partitioned connected after it. In
data feeds particularly, when the feeding rate is low, the ingested data
will not be able to be queried, even if they are ingested for a very
long time.
Change-Id: I2c2768a41fb9d23f8ce990c8c5c52ec14c2319e5
---
M
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/base/AbstractOneInputOneOutputOneFramePushRuntime.java
1 file changed, 1 insertion(+), 0 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/56/3256/1
diff --git
a/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/base/AbstractOneInputOneOutputOneFramePushRuntime.java
b/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/base/AbstractOneInputOneOutputOneFramePushRuntime.java
index 71b44d3..81edfe0 100644
---
a/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/base/AbstractOneInputOneOutputOneFramePushRuntime.java
+++
b/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/base/AbstractOneInputOneOutputOneFramePushRuntime.java
@@ -72,6 +72,7 @@
protected void flushAndReset() throws HyracksDataException {
if (appender.getTupleCount() > 0) {
appender.write(writer, true);
+writer.flush();
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/3256
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2c2768a41fb9d23f8ce990c8c5c52ec14c2319e5
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Change in asterixdb[master]: [ASTERIXDB-2514] Fix BigInt display in formatted JSON
Xikui Wang has submitted this change and it was merged.
Change subject: [ASTERIXDB-2514] Fix BigInt display in formatted JSON
..
[ASTERIXDB-2514] Fix BigInt display in formatted JSON
- user model changes: no
- storage format changes: no
- interface changes: no
Javascript treats all numbers as doubles. This causes large BigInt
values cannot be displayed properly in the formatted JSON option, and
the lower digits are replaced with "0". In this patch, we cast the
numbers into strings with special characters, and cast those strings
back after they are parsed by the JSON parser. Also, to make sure future
new data types can be displayed properly, the default JSON display is
set to "literal".
Change-Id: I1bbef349efba0da24d01afb1cb17b4f7d5b7f261
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3142
Sonar-Qube: Jenkins
Integration-Tests: Jenkins
Tested-by: Jenkins
Contrib: Jenkins
Reviewed-by: Michael Blow
---
M asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
M asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
2 files changed, 11 insertions(+), 5 deletions(-)
Approvals:
Jenkins: Verified; No violations found; ; Verified
Michael Blow: Looks good to me, approved
diff --git a/asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
b/asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
index 1157c27..31448ee 100644
--- a/asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
+++ b/asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
@@ -181,7 +181,14 @@
if (results[iter1].length < 1) {
continue;
}
- var resultJSON = $.parseJSON(results[iter1]);
+ var str_obj =
results[iter1].replace(/:\s*(\d+)([\s,])/g, ': "#bigint#$1#bigint#"$2');
+ resultJSON = JSON.parse(str_obj, (key, value) => {
+ if (/^#bigint#\d+#bigint#/g.test(value)) {
+value = value.substring(8, value.length - 8);
+return BigInt(value);
+ }
+ return value;
+});
$(this).append($('').attr("id",
"json-record"+idx+"-"+iter1));
$('#json-record'+idx+"-"+iter1).jsonViewer(resultJSON,
{collapsed: true, level: 10});
}
diff --git
a/asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
b/asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
index 5e8a79e..f31b555 100644
---
a/asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
+++
b/asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
@@ -56,10 +56,7 @@
else
html += '"' + json + '"';
}
-else if (typeof json === 'number') {
- html += '' + json + '';
-}
-else if (typeof json === 'boolean') {
+else if (typeof json === 'number' || typeof json === 'boolean' || typeof
json === 'bigint') {
html += '' + json + '';
}
else if (json === null) {
@@ -111,6 +108,8 @@
else {
html += '{}';
}
+} else {
+html += '' + json + '';
}
return html;
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/3142
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I1bbef349efba0da24d01afb1cb17b4f7d5b7f261
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Ian Maxon
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Michael Blow
Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [ASTERIXDB-2514] Fix BigInt display in formatted JSON
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3142
Change subject: [ASTERIXDB-2514] Fix BigInt display in formatted JSON
..
[ASTERIXDB-2514] Fix BigInt display in formatted JSON
- user model changes: no
- storage format changes: no
- interface changes: no
Javascript treats all numbers as doubles. This causes large BigInt
values cannot be displayed properly in the formatted JSON option, and
the lower digits are replaced with "0". In this patch, we cast the
numbers into strings with special characters, and cast those strings
back after they are parsed by the JSON parser. Also, to make sure future
new data types can be displayed properly, the default JSON display is
set to "literal".
Change-Id: I1bbef349efba0da24d01afb1cb17b4f7d5b7f261
---
M asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
M asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
2 files changed, 11 insertions(+), 5 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/42/3142/1
diff --git a/asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
b/asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
index 1157c27..31448ee 100644
--- a/asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
+++ b/asterixdb/asterix-app/src/main/resources/webui/querytemplate.html
@@ -181,7 +181,14 @@
if (results[iter1].length < 1) {
continue;
}
- var resultJSON = $.parseJSON(results[iter1]);
+ var str_obj =
results[iter1].replace(/:\s*(\d+)([\s,])/g, ': "#bigint#$1#bigint#"$2');
+ resultJSON = JSON.parse(str_obj, (key, value) => {
+ if (/^#bigint#\d+#bigint#/g.test(value)) {
+value = value.substring(8, value.length - 8);
+return BigInt(value);
+ }
+ return value;
+});
$(this).append($('').attr("id",
"json-record"+idx+"-"+iter1));
$('#json-record'+idx+"-"+iter1).jsonViewer(resultJSON,
{collapsed: true, level: 10});
}
diff --git
a/asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
b/asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
index 5e8a79e..f31b555 100644
---
a/asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
+++
b/asterixdb/asterix-app/src/main/resources/webui/static/js/jquery.json-viewer.js
@@ -56,10 +56,7 @@
else
html += '"' + json + '"';
}
-else if (typeof json === 'number') {
- html += '' + json + '';
-}
-else if (typeof json === 'boolean') {
+else if (typeof json === 'number' || typeof json === 'boolean' || typeof
json === 'bigint') {
html += '' + json + '';
}
else if (json === null) {
@@ -111,6 +108,8 @@
else {
html += '{}';
}
+} else {
+html += '' + json + '';
}
return html;
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/3142
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1bbef349efba0da24d01afb1cb17b4f7d5b7f261
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Change in asterixdb[master]: [NO ISSUE][EXT] Fix JRecord setField for open fields
Xikui Wang has submitted this change and it was merged.
Change subject: [NO ISSUE][EXT] Fix JRecord setField for open fields
..
[NO ISSUE][EXT] Fix JRecord setField for open fields
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
setField method should be able to handle open fields.
Change-Id: I4ec185d62e78e8100215334bc656586268314797
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3068
Sonar-Qube: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Contrib: Jenkins
Reviewed-by: Ali Alsuliman
---
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
2 files changed, 2 insertions(+), 7 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Ali Alsuliman: Looks good to me, approved
Jenkins: Verified; No violations found; ; Verified
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
index 4402130..464ecb2 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
@@ -35,8 +35,7 @@
};
create type DetectResultType as open{
-id:int64,
-sensitive: boolean
+id:int64
};
create dataset EmpDataset(InputRecordType) primary key id;
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
index 7fa9d0b..10c228d 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
@@ -99,11 +99,7 @@
if (pos >= 0) {
fields[pos] = fieldValue;
} else {
-if (openFields.get(fieldName) != null) {
-openFields.put(fieldName, fieldValue);
-} else {
-throw new
RuntimeDataException(ErrorCode.LIBRARY_JAVA_JOBJECTS_UNKNOWN_FIELD, fieldName);
-}
+openFields.put(fieldName, fieldValue);
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/3068
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I4ec185d62e78e8100215334bc656586268314797
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [NO ISSUE][EXT] Fix JRecord setField for open fields
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3068 to look at the new patch set (#4). Change subject: [NO ISSUE][EXT] Fix JRecord setField for open fields .. [NO ISSUE][EXT] Fix JRecord setField for open fields - user model changes: no - storage format changes: no - interface changes: no Details: setField method should be able to handle open fields. Change-Id: I4ec185d62e78e8100215334bc656586268314797 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java 2 files changed, 2 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/68/3068/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3068 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4ec185d62e78e8100215334bc656586268314797 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds
Xikui Wang has submitted this change and it was merged.
Change subject: [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds
..
[ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds
- user model changes: no
- storage format changes: no
- interface changes: no
LET is not necessary for applying functions to data feeds. We could
inline the function calls when constructing the pipeline query.
Change-Id: I65842f9ac84891b363d7e0a02425258d0df794e7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3072
Sonar-Qube: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Contrib: Jenkins
Reviewed-by: Dmitry Lychagin
---
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.3.lib.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.4.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.5.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/udf_filter_on_feed/udf_filter_on_feed.1.adm
M
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm
A
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/WordInListFactory.java
A
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/WordInListFunction.java
M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml
11 files changed, 240 insertions(+), 16 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Jenkins: Verified; No violations found; ; Verified
Dmitry Lychagin: Looks good to me, approved
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
index 593d7ce..d22e929 100644
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
@@ -184,8 +184,8 @@
argExprs.add(new LiteralExpr(new IntegerLiteral((Integer)
arg)));
} else if (arg instanceof String) {
argExprs.add(new LiteralExpr(new StringLiteral((String) arg)));
-} else if (arg instanceof VariableExpr) {
-argExprs.add((VariableExpr) arg);
+} else if (arg instanceof Expression) {
+argExprs.add((Expression) arg);
}
}
return argExprs;
@@ -216,25 +216,16 @@
whereClause = new WhereClause(whereClauseQuery.getBody());
}
-// TODO: This can be the place to add select predicate for ingestion
// Attaching functions
-int varIdx = 1;
-VariableExpr previousVarExpr = fromTermLeftExpr;
-ArrayList letClauses = new ArrayList<>();
-for (FunctionSignature funcSig : feedConnection.getAppliedFunctions())
{
-VarIdentifier intermediateVar = SqlppVariableUtil
-
.toInternalVariableIdentifier(FEED_DATAFLOW_INTERMEIDATE_VAL_PREFIX +
String.valueOf(varIdx));
-VariableExpr intermediateVarExpr = new
VariableExpr(intermediateVar);
-CallExpr functionCallExpr = new CallExpr(funcSig,
addArgs(previousVarExpr));
-previousVarExpr = intermediateVarExpr;
-LetClause letClause = new LetClause(intermediateVarExpr,
functionCallExpr);
-letClauses.add(letClause);
-varIdx++;
+Expression previousVarExpr = fromTermLeftExpr;
+for (FunctionSignature functionSignature :
feedConnection.getAppliedFunctions()) {
+CallExpr functionCallExpr = new CallExpr(functionSignature,
addArgs(previousVarExpr));
+previousVarExpr = functionCallExpr;
}
// Constructing select clause
SelectElement selectElement = new SelectElement(previousVarExpr);
SelectClause selectClause = new SelectClause(selectElement, null,
false);
-SelectBlock selectBlock = new SelectBlock(selectClause, fromClause,
letClauses, whereClause, null, null, null);
+SelectBlock selectBlock = new SelectBlock(selectClause, fromClause,
null, whereClause, null, null, null);
SelectSetOperation selectSetOperation = new SelectSetOperation(new
SetOperationInput(selectBlock, null), null);
SelectExpression body
Change in asterixdb[master]: [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3072 to look at the new patch set (#3). Change subject: [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds .. [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds - user model changes: no - storage format changes: no - interface changes: no LET is not necessary for applying functions to data feeds. We could inline the function calls when constructing the pipeline query. Change-Id: I65842f9ac84891b363d7e0a02425258d0df794e7 --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.3.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.4.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/udf_filter_on_feed/udf_filter_on_feed.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/WordInListFactory.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/WordInListFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml 11 files changed, 240 insertions(+), 16 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/72/3072/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3072 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I65842f9ac84891b363d7e0a02425258d0df794e7 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3072 to look at the new patch set (#2). Change subject: [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds .. [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds - user model changes: no - storage format changes: no - interface changes: no LET is not necessary for applying functions to data feeds. We could incline the function calls when constructing the pipeline query. Change-Id: I65842f9ac84891b363d7e0a02425258d0df794e7 --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.3.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.4.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/udf_filter_on_feed/udf_filter_on_feed.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/WordInListFactory.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/WordInListFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml 11 files changed, 240 insertions(+), 16 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/72/3072/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3072 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I65842f9ac84891b363d7e0a02425258d0df794e7 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3072
Change subject: [ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds
..
[ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds
- user model changes: no
- storage format changes: no
- interface changes: no
LET is not necessary for applying functions to data feeds. We could
incline the function calls when constructing the pipeline query.
Change-Id: I65842f9ac84891b363d7e0a02425258d0df794e7
---
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.3.lib.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.4.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/udf_filter_on_feed/udf_filter_on_feed.5.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/udf_filter_on_feed/udf_filter_on_feed.1.adm
M
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm
A
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/WordInListFactory.java
A
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/WordInListFunction.java
M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml
11 files changed, 251 insertions(+), 15 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/72/3072/1
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
index 593d7ce..1049e66 100644
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
@@ -184,8 +184,8 @@
argExprs.add(new LiteralExpr(new IntegerLiteral((Integer)
arg)));
} else if (arg instanceof String) {
argExprs.add(new LiteralExpr(new StringLiteral((String) arg)));
-} else if (arg instanceof VariableExpr) {
-argExprs.add((VariableExpr) arg);
+} else if (arg instanceof Expression) {
+argExprs.add((Expression) arg);
}
}
return argExprs;
@@ -216,25 +216,28 @@
whereClause = new WhereClause(whereClauseQuery.getBody());
}
-// TODO: This can be the place to add select predicate for ingestion
// Attaching functions
int varIdx = 1;
-VariableExpr previousVarExpr = fromTermLeftExpr;
-ArrayList letClauses = new ArrayList<>();
-for (FunctionSignature funcSig : feedConnection.getAppliedFunctions())
{
-VarIdentifier intermediateVar = SqlppVariableUtil
-
.toInternalVariableIdentifier(FEED_DATAFLOW_INTERMEIDATE_VAL_PREFIX +
String.valueOf(varIdx));
-VariableExpr intermediateVarExpr = new
VariableExpr(intermediateVar);
-CallExpr functionCallExpr = new CallExpr(funcSig,
addArgs(previousVarExpr));
-previousVarExpr = intermediateVarExpr;
-LetClause letClause = new LetClause(intermediateVarExpr,
functionCallExpr);
-letClauses.add(letClause);
-varIdx++;
+Expression previousVarExpr = fromTermLeftExpr;
+for (FunctionSignature functionSignature :
feedConnection.getAppliedFunctions()) {
+CallExpr functionCallExpr = new CallExpr(functionSignature,
addArgs(previousVarExpr));
+previousVarExpr = functionCallExpr;
}
+//ArrayList letClauses = new ArrayList<>();
+//for (FunctionSignature funcSig :
feedConnection.getAppliedFunctions()) {
+//VarIdentifier intermediateVar = SqlppVariableUtil
+//
.toInternalVariableIdentifier(FEED_DATAFLOW_INTERMEIDATE_VAL_PREFIX +
String.valueOf(varIdx));
+//VariableExpr intermediateVarExpr = new
VariableExpr(intermediateVar);
+//CallExpr functionCallExpr = new CallExpr(funcSig,
addArgs(previousVarExpr));
+//previousVarExpr = intermediateVarExpr;
+//LetClause letClause = new LetClause(intermediateVarExpr,
functionCallExpr);
+//letClauses.add(letClause);
+//
Change in asterixdb[master]: [NO ISSUE][EXT] Fix JRecord setField for open fields
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3068
Change subject: [NO ISSUE][EXT] Fix JRecord setField for open fields
..
[NO ISSUE][EXT] Fix JRecord setField for open fields
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
setField method should be friendly to open fields.
Change-Id: I4ec185d62e78e8100215334bc656586268314797
---
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
2 files changed, 2 insertions(+), 7 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/68/3068/1
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
index 4402130..464ecb2 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/keyword_detector/keyword_detector.1.ddl.sqlpp
@@ -35,8 +35,7 @@
};
create type DetectResultType as open{
-id:int64,
-sensitive: boolean
+id:int64
};
create dataset EmpDataset(InputRecordType) primary key id;
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
index 7fa9d0b..10c228d 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/java/base/JRecord.java
@@ -99,11 +99,7 @@
if (pos >= 0) {
fields[pos] = fieldValue;
} else {
-if (openFields.get(fieldName) != null) {
-openFields.put(fieldName, fieldValue);
-} else {
-throw new
RuntimeDataException(ErrorCode.LIBRARY_JAVA_JOBJECTS_UNKNOWN_FIELD, fieldName);
-}
+openFields.put(fieldName, fieldValue);
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/3068
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4ec185d62e78e8100215334bc656586268314797
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Change in asterixdb-bad[master]: [NO ISSUE][BAD] Coordinated change for window functions (AST...
Xikui Wang has posted comments on this change. Change subject: [NO ISSUE][BAD] Coordinated change for window functions (ASTERIXDB-2488) .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3051 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I2799ef800c67f76d2d6dda17c9e619c63b0bcf4a Gerrit-PatchSet: 1 Gerrit-Project: asterixdb-bad Gerrit-Branch: master Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][ING] TweetParserFactory fix
Xikui Wang has submitted this change and it was merged.
Change subject: [NO ISSUE][ING] TweetParserFactory fix
..
[NO ISSUE][ING] TweetParserFactory fix
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Fix the TweetParserFactory so it returns the correct record class.
Change-Id: If330bf060423027e01787224d1b2e93fd4d33d71
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3050
Sonar-Qube: Jenkins
Tested-by: Jenkins
Contrib: Jenkins
Integration-Tests: Jenkins
Reviewed-by: Wail Alkowaileet
---
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
1 file changed, 2 insertions(+), 2 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Jenkins: Verified; No violations found; ; Verified
Wail Alkowaileet: Looks good to me, approved
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
index 7844924..e5b6d01 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
@@ -54,8 +54,8 @@
}
@Override
-public Class getRecordClass() {
-return String.class;
+public Class getRecordClass() {
+return char[].class;
}
@Override
--
To view, visit https://asterix-gerrit.ics.uci.edu/3050
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: If330bf060423027e01787224d1b2e93fd4d33d71
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Wail Alkowaileet
Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [NO ISSUE][ING] TweetParserFactory fix
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3050
Change subject: [NO ISSUE][ING] TweetParserFactory fix
..
[NO ISSUE][ING] TweetParserFactory fix
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Fix the TweetParserFactory so it returns the correct record class.
Change-Id: If330bf060423027e01787224d1b2e93fd4d33d71
---
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
1 file changed, 2 insertions(+), 2 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/50/3050/1
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
index 7844924..e5b6d01 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
@@ -54,8 +54,8 @@
}
@Override
-public Class getRecordClass() {
-return String.class;
+public Class getRecordClass() {
+return char[].class;
}
@Override
--
To view, visit https://asterix-gerrit.ics.uci.edu/3050
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: If330bf060423027e01787224d1b2e93fd4d33d71
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Change in asterixdb[master]: [NO ISSUE][ING] Tweets ingestion related refactoring
Xikui Wang has submitted this change and it was merged.
Change subject: [NO ISSUE][ING] Tweets ingestion related refactoring
..
[NO ISSUE][ING] Tweets ingestion related refactoring
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
As the incoming Tweets are merely JSON records, the Twitter adapter
should be able to use JSON parser. For that purpose, we changed the
record record to be consistent with JSONDataParser which uses char[]
instead of String. Also, "JSON" should also be a valid way to specify
using JSON parser.
Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3032
Sonar-Qube: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Contrib: Jenkins
Reviewed-by: Wail Alkowaileet
---
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/TweetParser.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/JSONDataParserFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/RecordConverterFactoryProvider.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/StreamRecordReaderProviderTest.java
M
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java
13 files changed, 50 insertions(+), 32 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Jenkins: Verified; No violations found; ; Verified
Wail Alkowaileet: Looks good to me, approved
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
index aff00fc..65ecd8d 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java
@@ -128,4 +128,10 @@
builder.getChars(0, builder.length(), value, 0);
this.size = builder.length();
}
+
+public void set(String strValue) throws IOException {
+ensureCapacity(strValue.length());
+strValue.getChars(0, strValue.length(), value, 0);
+this.size = strValue.length();
+}
}
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
index b466614..81ad5ba 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
@@ -57,7 +57,7 @@
public HttpServerRecordReader(int port, String entryPoint, int queueSize,
HttpServerConfig httpServerConfig)
throws Exception {
this.inputQ = new LinkedBlockingQueue<>(queueSize > 0 ? queueSize :
DEFAULT_QUEUE_SIZE);
-this.record = new CharArrayRecord(0);
+this.record = new CharArrayRecord();
webManager = new WebManager();
webServer = new HttpServer(webManager.getBosses(),
webManager.getWorkers(), port, httpServerConfig);
webServer.addServlet(new HttpFeedServlet(webServer.ctx(),
@@ -77,7 +77,7 @@
if (srecord == null) {
return null;
}
-record.set(srecord.toCharArray());
+record.set(srecord);
return record;
}
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
b/asterixdb/asterix-external-data/
Change in asterixdb[master]: [NO ISSUE][ING] Tweets ingestion related refactoring
Xikui Wang has posted comments on this change. Change subject: [NO ISSUE][ING] Tweets ingestion related refactoring .. Patch Set 5: (5 comments) Addressed all comments. Not sure why the IT failed at the http_json test cases. Local tests show everything is fine. Let's see how this one goes... https://asterix-gerrit.ics.uci.edu/#/c/3032/5/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java: PS5, Line 135: } > add: Done https://asterix-gerrit.ics.uci.edu/#/c/3032/5/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java: PS5, Line 54: 0 > This is not needed anymore. You can make it to take the default buffer size agree https://asterix-gerrit.ics.uci.edu/#/c/3032/5/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java: PS5, Line 63: 0 > same agree! https://asterix-gerrit.ics.uci.edu/#/c/3032/5/asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java File asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java: PS5, Line 73: CharArrayRecord > You can move this out of the loop. Done PS5, Line 99: CharArrayRecord > This one as well Done -- To view, visit https://asterix-gerrit.ics.uci.edu/3032 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][ING] Tweets ingestion related refactoring
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3032 to look at the new patch set (#6). Change subject: [NO ISSUE][ING] Tweets ingestion related refactoring .. [NO ISSUE][ING] Tweets ingestion related refactoring - user model changes: no - storage format changes: no - interface changes: no Details: As the incoming Tweets are merely JSON records, the Twitter adapter should be able to use JSON parser. For that purpose, we changed the record record to be consistent with JSONDataParser which uses char[] instead of String. Also, "JSON" should also be a valid way to specify using JSON parser. Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/TweetParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/JSONDataParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/RecordConverterFactoryProvider.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/StreamRecordReaderProviderTest.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java 13 files changed, 50 insertions(+), 32 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/32/3032/6 -- To view, visit https://asterix-gerrit.ics.uci.edu/3032 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 Gerrit-PatchSet: 6 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet
Change in asterixdb[master]: [NO ISSUE][ING] Tweets ingestion related refactoring
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3032 to look at the new patch set (#5). Change subject: [NO ISSUE][ING] Tweets ingestion related refactoring .. [NO ISSUE][ING] Tweets ingestion related refactoring - user model changes: no - storage format changes: no - interface changes: no Details: As the incoming Tweets are merely JSON records, the Twitter adapter should be able to use JSON parser. For that purpose, we changed the record record to be consistent with JSONDataParser which uses char[] instead of String. Also, "JSON" should also be a valid way to specify using JSON parser. Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/TweetParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/JSONDataParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/RecordConverterFactoryProvider.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/StreamRecordReaderProviderTest.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java 13 files changed, 48 insertions(+), 31 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/32/3032/5 -- To view, visit https://asterix-gerrit.ics.uci.edu/3032 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet
Change in asterixdb[master]: [NO ISSUE][ING] Tweets ingestion related refactoring
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3032 to look at the new patch set (#4). Change subject: [NO ISSUE][ING] Tweets ingestion related refactoring .. [NO ISSUE][ING] Tweets ingestion related refactoring - user model changes: no - storage format changes: no - interface changes: no Details: As the incoming Tweets are merely JSON records, the Twitter adapter should be able to use JSON parser. For that purpose, we changed the record record to be consistent with JSONDataParser which uses char[] instead of String. Also, "JSON" should also be a valid way to specify using JSON parser. Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/TweetParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/JSONDataParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/RecordConverterFactoryProvider.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/StreamRecordReaderProviderTest.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java 13 files changed, 48 insertions(+), 31 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/32/3032/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3032 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet
Change in asterixdb[master]: [NO ISSUE][ING] Tweets ingestion related refactoring
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3032 to look at the new patch set (#3). Change subject: [NO ISSUE][ING] Tweets ingestion related refactoring .. [NO ISSUE][ING] Tweets ingestion related refactoring - user model changes: no - storage format changes: no - interface changes: no Details: As the incoming Tweets are merely JSON records, the Twitter adapter should be able to use JSON parser. For that purpose, we changed the record record to be consistent with JSONDataParser which uses char[] instead of String. Also, "JSON" should also be a valid way to specify using JSON parser. Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/CharArrayRecord.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/TweetParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/JSONDataParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java 11 files changed, 50 insertions(+), 33 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/32/3032/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3032 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet
Change in asterixdb[master]: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON ...
Xikui Wang has submitted this change and it was merged.
Change subject: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON
parser
..
[ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
The generic record used in current http-feed returns length of "-1"
which will cause the JSON parser to fail. For compatibility
consideration, we should use CharArrayRecord instead.
Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3026
Sonar-Qube: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Contrib: Jenkins
Reviewed-by: Wail Alkowaileet
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.3.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.4.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.5.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.6.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.1.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.2.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
10 files changed, 156 insertions(+), 3 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Jenkins: Verified; No violations found; ; Verified
Wail Alkowaileet: Looks good to me, approved
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp
new file mode 100644
index 000..94a773c
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+drop dataverse test if exists;
+create dataverse test;
+use test;
+
+create type WorrisomeNotificationType as {
+dataverseName: string,
+channelName: string
+};
+create type WorrisomeNotificationStoreType as {
+id: uuid
+};
+create feed WorrisomeTweetsFeed with {
+ "adapter-name" : "http_adapter",
+ "addresses" : "asterix_nc1:10001",
+ "address-type" : "NC",
+ "type-name" : "WorrisomeNotificationType",
+ "format" : "json"
+};
+
+create dataset WorrisomeNotifications(WorrisomeNotificationStoreType) primary
key id autogenerated;
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp
new file mode 100644
index 000..cc180a3
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp
@@ -0,0 +1,24 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIE
Change in asterixdb[master]: [NO ISSUE][ING] Tweets ingestion related refactoring
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3032 to look at the new patch set (#2). Change subject: [NO ISSUE][ING] Tweets ingestion related refactoring .. [NO ISSUE][ING] Tweets ingestion related refactoring - user model changes: no - storage format changes: no - interface changes: no Details: As the incoming Tweets are merely JSON records, the Twitter adapter should be able to use JSON parser. For that purpose, we changed the record record to be consistent with JSONDataParser which uses char[] instead of String. Also, "JSON" should also be a valid way to specify using JSON parser. Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/TweetParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/JSONDataParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java 9 files changed, 43 insertions(+), 29 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/32/3032/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3032 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHa...
Xikui Wang has posted comments on this change. Change subject: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin .. Patch Set 5: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3023 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHa...
Xikui Wang has submitted this change and it was merged.
Change subject: [NO ISSUE][RT] Replace HybridHashJoin with
OptimizedHybridHashJoin
..
[NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
The HybridHashJoinOperator is an old implenetation which haven't been
used in the runtime, and it lacks necessary documentation and memory
management. The OptimizedHybridHashJoinOperatorDescriptor serves the
same purpose. We should use this instead and avoid maintaining the old
one.
Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3023
Sonar-Qube: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Contrib: Jenkins
Reviewed-by: Xikui Wang
---
M
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java
D
hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java
D
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java
M
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOrderHashJoinTest.java
4 files changed, 247 insertions(+), 1,164 deletions(-)
Approvals:
Jenkins: Verified; No violations found; ; Verified
Xikui Wang: Looks good to me, approved
diff --git
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java
index 45ec44b..091cc44 100644
---
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java
+++
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java
@@ -45,7 +45,6 @@
import org.apache.hyracks.api.dataflow.IOperatorDescriptor;
import org.apache.hyracks.api.dataflow.value.IBinaryComparator;
import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory;
-import org.apache.hyracks.api.dataflow.value.IBinaryHashFunctionFactory;
import org.apache.hyracks.api.dataflow.value.IBinaryHashFunctionFamily;
import org.apache.hyracks.api.dataflow.value.IMissingWriterFactory;
import org.apache.hyracks.api.dataflow.value.IPredicateEvaluatorFactory;
@@ -55,7 +54,6 @@
import org.apache.hyracks.api.dataflow.value.RecordDescriptor;
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.api.job.IOperatorDescriptorRegistry;
-import org.apache.hyracks.dataflow.std.join.HybridHashJoinOperatorDescriptor;
import
org.apache.hyracks.dataflow.std.join.OptimizedHybridHashJoinOperatorDescriptor;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@@ -65,7 +63,6 @@
// The maximum number of in-memory frames that this hash join can use.
private final int memSizeInFrames;
private final int maxInputBuildSizeInFrames;
-private final int aveRecordsPerFrame;
private final double fudgeFactor;
private static final Logger LOGGER = LogManager.getLogger();
@@ -76,7 +73,6 @@
super(kind, partitioningType, sideLeftOfEqualities,
sideRightOfEqualities);
this.memSizeInFrames = memSizeInFrames;
this.maxInputBuildSizeInFrames = maxInputSizeInFrames;
-this.aveRecordsPerFrame = aveRecordsPerFrame;
this.fudgeFactor = fudgeFactor;
if (LOGGER.isTraceEnabled()) {
LOGGER.trace("HybridHashJoinPOperator constructed with: JoinKind="
+ kind + ", JoinPartitioningType="
@@ -117,8 +113,6 @@
int[] keysLeft = JobGenHelper.variablesToFieldIndexes(keysLeftBranch,
inputSchemas[0]);
int[] keysRight =
JobGenHelper.variablesToFieldIndexes(keysRightBranch, inputSchemas[1]);
IVariableTypeEnvironment env = context.getTypeEnvironment(op);
-IBinaryHashFunctionFactory[] hashFunFactories =
-
JobGenHelper.variablesToBinaryHashFunctionFactories(keysLeftBranch, env,
context);
IBinaryHashFunctionFamily[] hashFunFamilies =
JobGenHelper.variablesToBinaryHashFunctionFamilies(keysLeftBranch, env,
context);
IBinaryComparatorFactory[] comparatorFactories = new
IBinaryComparatorFactory[keysLeft.length];
@@ -138,21 +132,9 @@
JobGenHelper.mkRecordDescriptor(context.getTypeEnvironment(op),
propagatedSchema, context);
IOperatorDescriptorRegistry spec = builder.getJobSpec
Change in asterixdb[master]: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHa...
Xikui Wang has posted comments on this change. Change subject: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin .. Patch Set 5: forward +2 from Dymitry -- To view, visit https://asterix-gerrit.ics.uci.edu/3023 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][ING] Tweets ingestion related refactoring
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3032
Change subject: [NO ISSUE][ING] Tweets ingestion related refactoring
..
[NO ISSUE][ING] Tweets ingestion related refactoring
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
As the incoming Tweets are merely JSON records, the Twitter adapter
should be able to use JSON parser. For that purpose, we changed the
record record to be consistent with JSONDataParser which uses char[]
instead of String. Also, "JSON" should also be a valid way to specify
using JSON parser.
Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078
---
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPushRecordReader.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/TweetParser.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/JSONDataParserFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/factory/TweetParserFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/parser/test/TweetParserTest.java
9 files changed, 42 insertions(+), 27 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/32/3032/1
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
index cc4b7f9..674dd52 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/SemiStructuredRecordReader.java
@@ -41,7 +41,8 @@
private int recordNumber = 0;
private static final List recordReaderFormats =
Collections.unmodifiableList(Arrays.asList(ExternalDataConstants.FORMAT_ADM,
-ExternalDataConstants.FORMAT_JSON,
ExternalDataConstants.FORMAT_SEMISTRUCTURED));
+ExternalDataConstants.FORMAT_JSON,
ExternalDataConstants.FORMAT_JSON_CAP,
+ExternalDataConstants.FORMAT_SEMISTRUCTURED));
private static final String REQUIRED_CONFIGS = "";
@Override
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java
index 2cf5396..f5404f6 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterPullRecordReader.java
@@ -24,6 +24,7 @@
import org.apache.asterix.external.api.IRawRecord;
import org.apache.asterix.external.api.IRecordReader;
import org.apache.asterix.external.dataflow.AbstractFeedDataFlowController;
+import org.apache.asterix.external.input.record.CharArrayRecord;
import org.apache.asterix.external.input.record.GenericRecord;
import org.apache.asterix.external.util.FeedLogManager;
import org.apache.hyracks.api.exceptions.HyracksDataException;
@@ -35,7 +36,7 @@
import twitter4j.TwitterException;
import twitter4j.TwitterObjectFactory;
-public class TwitterPullRecordReader implements IRecordReader {
+public class TwitterPullRecordReader implements IRecordReader {
private Query query;
private Twitter twitter;
@@ -43,7 +44,7 @@
private QueryResult result;
private int nextTweetIndex = 0;
private long lastTweetIdReceived = 0;
-private GenericRecord record;
+private CharArrayRecord record;
private boolean stopped = false;
public TwitterPullRecordReader(Twitter twitter, String keywords, int
requestInterval) {
@@ -51,7 +52,7 @@
this.requestInterval = requestInterval;
this.query = new Query(keywords);
this.query.setCount(100);
-this.record = new GenericRecord<>();
+this.record = new CharArrayRecord(0);
}
Change in asterixdb[master]: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON ...
Xikui Wang has posted comments on this change. Change subject: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser .. Patch Set 4: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/3026/4/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.3.post.http File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.3.post.http: PS4, Line 20: body > It would good if we can have more than one record in a request. Just to mak Done https://asterix-gerrit.ics.uci.edu/#/c/3026/4/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java: PS4, Line 60: CharArrayRecord > Maybe initializing it with inititalCapacity = 0? Done -- To view, visit https://asterix-gerrit.ics.uci.edu/3026 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON ...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3026 to look at the new patch set (#5). Change subject: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser .. [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser - user model changes: no - storage format changes: no - interface changes: no Details: The generic record used in current http-feed returns length of "-1" which will cause the JSON parser to fail. For compatibility consideration, we should use CharArrayRecord instead. Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.3.post.http A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.4.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.6.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java 10 files changed, 156 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/26/3026/5 -- To view, visit https://asterix-gerrit.ics.uci.edu/3026 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet
Change in asterixdb[master]: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHa...
Xikui Wang has abandoned this change. Change subject: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin .. Abandoned Conducted experiments on a 3 nodes cluster (2NC 1CC) with TPCH benchmark twice the size of the aggregated memory. There is no significant benefit of using linear probe hash table. :( -- To view, visit https://asterix-gerrit.ics.uci.edu/1993 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON ...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3026 to look at the new patch set (#4). Change subject: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser .. [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser - user model changes: no - storage format changes: no - interface changes: no Details: The generic record used in current http-feed returns length of "-1" which will cause the JSON parser to fail. For compatibility consideration, we should use CharArrayRecord instead. Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.3.post.http A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.4.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.6.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java 10 files changed, 153 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/26/3026/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3026 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON ...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3026 to look at the new patch set (#3). Change subject: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser .. [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser - user model changes: no - storage format changes: no - interface changes: no Details: The generic record used in current http-feed returns length of "-1" which will cause the JSON parser to fail. For compatibility consideration, we should use CharArrayRecord instead. Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.3.post.http A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.4.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.5.pollquery.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.6.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java 10 files changed, 154 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/26/3026/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3026 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON ...
Xikui Wang has uploaded a new patch set (#2). Change subject: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser .. [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser - user model changes: no - storage format changes: no - interface changes: no Details: The generic record used in current http-feed returns length of "-1" which will cause the JSON parser to fail. For compatibility consideration, we should use CharArrayRecord instead. Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.3.post.http A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.4.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.5.pollquery.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.6.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.2.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java 10 files changed, 154 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/26/3026/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3026 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON ...
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3026
Change subject: [ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON
parser
..
[ASTERIXDB-2480][ING] Fix http-feed compatibility with JSON parser
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
The generic record used in current http-feed returns length of "-1"
which will cause the JSON parser to fail. For compatibility
consideration, we should use CharArrayRecord instead.
Change-Id: I719c98376583a3316dbc291696df65a97dd4b26e
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.3.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.4.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.5.pollquery.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.6.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.1.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed_json/http_feed.2.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
10 files changed, 154 insertions(+), 2 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/26/3026/1
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp
new file mode 100644
index 000..246f7e1
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.1.ddl.sqlpp
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+drop dataverse test if exists;
+create dataverse test;
+use test;
+
+create type WorrisomeNotificationType as {
+ dataverseName: string,
+ channelName: string
+};
+create feed WorrisomeTweetsFeed with {
+ "adapter-name" : "http_adapter",
+ "addresses" : "asterix_nc1:10001",
+ "address-type" : "NC",
+ "type-name" : "WorrisomeNotificationType",
+ "format" : "json"
+};
+
+create dataset WorrisomeNotifications(WorrisomeNotificationType) primary key
dataverseName;
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp
new file mode 100644
index 000..cc180a3
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed_json/http_feed_json.2.update.sqlpp
@@ -0,0 +1,24 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+use test;
+set `wait-for-completion-feed` "false";
+
+connect feed WorrisomeTweet
Change in asterixdb-bad[master]: [NO ISSUE][BAD] BAD clean-ups
Xikui Wang has abandoned this change. Change subject: [NO ISSUE][BAD] BAD clean-ups .. Abandoned -- To view, visit https://asterix-gerrit.ics.uci.edu/3022 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I76c5d201a95f71a8f9bfd2461620145c0c27a08d Gerrit-PatchSet: 2 Gerrit-Project: asterixdb-bad Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: James Fang Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHa...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1993 to look at the new patch set (#11). Change subject: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin .. [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin - user model changes: no - storage format changes: no - interface changes: no Details: The current SerializableHashTable used in InMemoryHashJoin is too complicated and inefficient for in-memory-join. As the tuple number in in memory join is fixed, we don't need to consider spill or overflow issues. Also, SerializableHashTable will cause extra memory lookup (2 per each entry) and have larger probability to have cache misses. LinearHashTable only needs 1 lookup per each entry and have better chance to be cached. Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java M hyracks-fullstack/hyracks/hyracks-api/src/main/resources/errormsg/en.properties D hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoin.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoin.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/IntSerDeBuffer.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/LinearProbeHashTable.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SimpleSerializableHashTable.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/test/java/org/apache/hyracks/dataflow/std/structures/LinearProbeHashTableTest.java D hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOrderHashJoinTest.java 13 files changed, 742 insertions(+), 1,288 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/93/1993/11 -- To view, visit https://asterix-gerrit.ics.uci.edu/1993 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHa...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1993 to look at the new patch set (#10). Change subject: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin .. [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin - user model changes: no - storage format changes: no - interface changes: no Details: The current SerializableHashTable used in InMemoryHashJoin is too complicated and inefficient for in-memory-join. As the tuple number in in memory join is fixed, we don't need to consider spill or overflow issues. Also, SerializableHashTable will cause extra memory lookup (2 per each entry) and have larger probability to have cache misses. LinearHashTable only needs 1 lookup per each entry and have better chance to be cached. Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/ErrorCode.java M hyracks-fullstack/hyracks/hyracks-api/src/main/resources/errormsg/en.properties D hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoin.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoin.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/IntSerDeBuffer.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/LinearProbeHashTable.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SimpleSerializableHashTable.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/test/java/org/apache/hyracks/dataflow/std/structures/LinearProbeHashTableTest.java D hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOrderHashJoinTest.java 13 files changed, 742 insertions(+), 1,288 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/93/1993/10 -- To view, visit https://asterix-gerrit.ics.uci.edu/1993 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 Gerrit-PatchSet: 10 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHa...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1993 to look at the new patch set (#9). Change subject: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin .. [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin - user model changes: no - storage format changes: no - interface changes: no Details: The current SerializableHashTable used in InMemoryHashJoin is too complicated and inefficient for in-memory-join. As the tuple number in in memory join is fixed, we don't need to consider spill or overflow issues. Also, SerializableHashTable will cause extra memory lookup (2 per each entry) and have larger probability to have cache misses. LinearHashTable only needs 1 lookup per each entry and have better chance to be cached. Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java D hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoin.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoinOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoin.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/IntSerDeBuffer.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/LinearProbeHashTable.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SimpleSerializableHashTable.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/test/java/org/apache/hyracks/dataflow/std/structures/LinearProbeHashTableTest.java D hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOrderHashJoinTest.java 12 files changed, 749 insertions(+), 1,341 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/93/1993/9 -- To view, visit https://asterix-gerrit.ics.uci.edu/1993 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 Gerrit-PatchSet: 9 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHa...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1993 to look at the new patch set (#8). Change subject: [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin .. [ASTERIXDB-2086][RT] Use LinearProbeHashTable for InMemoryHashJoin - user model changes: no - storage format changes: no - interface changes: no Details: The current SerializableHashTable used in InMemoryHashJoin is too complicated and inefficient for in-memory-join. As the tuple number in in memory join is fixed, we don't need to consider spill or overflow issues. Also, SerializableHashTable will cause extra memory lookup (2 per each entry) and have larger probability to have cache misses. LinearHashTable only needs 1 lookup per each entry and have better chance to be cached. Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java D hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoin.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/InMemoryHashJoinOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoin.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/IntSerDeBuffer.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/LinearProbeHashTable.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/SimpleSerializableHashTable.java A hyracks-fullstack/hyracks/hyracks-dataflow-std/src/test/java/org/apache/hyracks/dataflow/std/structures/LinearProbeHashTableTest.java D hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOrderHashJoinTest.java 12 files changed, 749 insertions(+), 1,341 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/93/1993/8 -- To view, visit https://asterix-gerrit.ics.uci.edu/1993 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I61e11d8223902cc34e81738660c1b3ed0ab86318 Gerrit-PatchSet: 8 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHa...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3023 to look at the new patch set (#4). Change subject: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin .. [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin - user model changes: no - storage format changes: no - interface changes: no Details: The HybridHashJoinOperator is an old implenetation which haven't been used in the runtime, and it lacks necessary documentation and memory management. The OptimizedHybridHashJoinOperatorDescriptor serves the same purpose. We should use this instead and avoid maintaining the old one. Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82 --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java D hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java D hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOrderHashJoinTest.java 4 files changed, 247 insertions(+), 1,164 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/3023/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3023 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHa...
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3023 to look at the new patch set (#3). Change subject: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin .. [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin - user model changes: no - storage format changes: no - interface changes: no Details: The HybridHashJoinOperator is an old implenetation which haven't been used in the runtime, and it lacks necessary documentation and memory management. The OptimizedHybridHashJoinOperatorDescriptor serves the same purpose. We should use this instead and avoid maintaining the old one. Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82 --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java D hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java D hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOrderHashJoinTest.java 4 files changed, 243 insertions(+), 1,132 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/3023/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3023 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHa...
Xikui Wang has uploaded a new patch set (#2). Change subject: [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin .. [NO ISSUE][RT] Replace HybridHashJoin with OptimizedHybridHashJoin - user model changes: no - storage format changes: no - interface changes: no Details: The HybridHashJoinOperator is an old implenetation which haven't been used in the runtime, and it lacks necessary documentation and memory management. The OptimizedHybridHashJoinOperatorDescriptor serves the same purpose. We should use this instead and avoid maintaining the old one. Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82 --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/HybridHashJoinPOperator.java D hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/HybridHashJoinOperatorDescriptor.java D hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOptimizedHybridHashJoinTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/TPCHCustomerOrderHashJoinTest.java 4 files changed, 245 insertions(+), 1,134 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/23/3023/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3023 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ed612cc233af1b78d453c7b711077b82e721e82 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Jenkins
Change in asterixdb-bad[master]: [NO ISSUE][BAD] BAD clean-ups
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3022 to look at the new patch set (#2). Change subject: [NO ISSUE][BAD] BAD clean-ups .. [NO ISSUE][BAD] BAD clean-ups Details: 1. Cleaned up dependencies which are not used. 2. Changed logging to use log4j. 3. Refactored BADExecutionTest to be consistent with master. 4. Cleaned up test cases to make sure every test case is independent. 5. Unused file removed. Change-Id: I76c5d201a95f71a8f9bfd2461620145c0c27a08d --- M asterix-bad/pom.xml M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java M asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java M asterix-bad/src/main/java/org/apache/asterix/bad/runtime/NotifyBrokerRuntime.java D asterix-bad/src/main/resources/asterix-build-configuration.xml M asterix-bad/src/main/resources/cc.conf M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADExecutionTest.java M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADOptimizerTest.java M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java D asterix-bad/src/test/resources/conf/cluster.xml D asterix-bad/src/test/resources/conf/hyracks-deployment.properties A asterix-bad/src/test/resources/conf/log4j2-bad.xml D asterix-bad/src/test/resources/conf/test.properties A asterix-bad/src/test/resources/runtimets/only_bad.xml M asterix-bad/src/test/resources/runtimets/queries/channel/drop_function/drop_function.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/channel/drop_function_dataverse/drop_function_dataverse.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/channel/drop_index/drop_index.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/channel/drop_results/drop_results.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/channel/drop_subscriptions/drop_subscriptions.1.ddl.sqlpp A asterix-bad/src/test/resources/runtimets/queries/procedure/create_procedure_check_metadata/create_procedure_check_metadata.3.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/procedure/delete_procedure_drop_dataset/delete_procedure_drop_dataset.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/procedure/delete_procedure_drop_function/delete_procedure_drop_function.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/procedure/delete_procedure_drop_index/delete_procedure_drop_index.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/procedure/insert_procedure_drop_dataset/insert_procedure_drop_dataset.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/procedure/insert_procedure_drop_dataverse/insert_procedure_drop_dataverse.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_drop_dataset/query_procedure_drop_dataset.1.ddl.sqlpp M asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_drop_function/query_procedure_drop_function.1.ddl.sqlpp R asterix-bad/src/test/resources/runtimets/testsuite_bad.xml 29 files changed, 251 insertions(+), 405 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad refs/changes/22/3022/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3022 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I76c5d201a95f71a8f9bfd2461620145c0c27a08d Gerrit-PatchSet: 2 Gerrit-Project: asterixdb-bad Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Jenkins
Change in asterixdb-bad[master]: [NO ISSUE][BAD] BAD clean-ups
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3022
Change subject: [NO ISSUE][BAD] BAD clean-ups
..
[NO ISSUE][BAD] BAD clean-ups
Details:
1. Cleaned up dependencies which are not used.
2. Changed logging to use log4j.
3. Refactored BADExecutionTest to be consistent with master.
4. Cleaned up test cases to make sure every test case is independent.
5. Unused file removed.
Change-Id: I76c5d201a95f71a8f9bfd2461620145c0c27a08d
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
M
asterix-bad/src/main/java/org/apache/asterix/bad/runtime/NotifyBrokerRuntime.java
D asterix-bad/src/main/resources/asterix-build-configuration.xml
M asterix-bad/src/main/resources/cc.conf
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADExecutionTest.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADOptimizerTest.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
D asterix-bad/src/test/resources/conf/cluster.xml
D asterix-bad/src/test/resources/conf/hyracks-deployment.properties
A asterix-bad/src/test/resources/conf/log4j2-bad.xml
D asterix-bad/src/test/resources/conf/test.properties
A asterix-bad/src/test/resources/runtimets/only_bad.xml
M
asterix-bad/src/test/resources/runtimets/queries/channel/drop_function/drop_function.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/channel/drop_function_dataverse/drop_function_dataverse.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/channel/drop_index/drop_index.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/channel/drop_results/drop_results.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/channel/drop_subscriptions/drop_subscriptions.1.ddl.sqlpp
A
asterix-bad/src/test/resources/runtimets/queries/procedure/create_procedure_check_metadata/create_procedure_check_metadata.3.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/procedure/delete_procedure_drop_dataset/delete_procedure_drop_dataset.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/procedure/delete_procedure_drop_function/delete_procedure_drop_function.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/procedure/delete_procedure_drop_index/delete_procedure_drop_index.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/procedure/insert_procedure_drop_dataset/insert_procedure_drop_dataset.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/procedure/insert_procedure_drop_dataverse/insert_procedure_drop_dataverse.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_drop_dataset/query_procedure_drop_dataset.1.ddl.sqlpp
M
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_drop_function/query_procedure_drop_function.1.ddl.sqlpp
R asterix-bad/src/test/resources/runtimets/testsuite_bad.xml
29 files changed, 251 insertions(+), 405 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad
refs/changes/22/3022/1
diff --git a/asterix-bad/pom.xml b/asterix-bad/pom.xml
index d6c0602..6396b36 100644
--- a/asterix-bad/pom.xml
+++ b/asterix-bad/pom.xml
@@ -28,7 +28,7 @@
0.9.5-SNAPSHOT
0.3.5-SNAPSHOT
true
-
${root.dir}/../../asterix-app/src/test/resources/log4j2-test.xml
+
${root.dir}/src/test/resources/conf/log4j2-bad.xml
@@ -153,25 +153,16 @@
2.10
-
org.apache.hadoop:hadoop-minicluster
org.apache.asterix:asterix-fuzzyjoin
+
org.apache.hyracks:hyracks-storage-am-lsm-btree-test
+
org.apache.asterix:asterix-common:test-jar
-
-
org.apache.asterix:asterix-common
-
-
- org.apache.hadoop
- hadoop-minicluster
- ${hadoop.version}
- jar
- test
-
org.apache.asterix
asterix-om
@@ -225,13 +216,6 @@
test
- org.apache.asterix
- asterix-common
- ${asterix.version}
- test-jar
- test
-
-
org.apache.hyracks
algebricks-common
@@ -248,10 +232,6 @@
org.apache.asterix
asterix-lang-common
${asterix.version}
-
-
- log4j
- log4j
org.apache.commons
@@ -307,7 +287,22 @@
commons-io
commons-io
- 2.6
+
+
+ org.apache.hyracks
+ hyracks-api
+
+
+ org.apache.logging.log4j
+ log4j-api
+
+
+
+ org.apache.hyracks
+ hyracks-storage-am-lsm-btree-test
+ ${hyracks.version}
+ test-jar
+ test
Change in asterixdb[master]: [ASTERIXDB-2467][STO] Fix locking protocol with bad tuples
Xikui Wang has posted comments on this change. Change subject: [ASTERIXDB-2467][STO] Fix locking protocol with bad tuples .. Patch Set 2: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3003 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I86baa1f21ef7390777c8e2b0ce226095528e7f18 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Luo Chen Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
Xikui Wang has submitted this change and it was merged.
Change subject: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
..
[ASTERIXDB-2264][ING] Introduce Http Feed Adapter
- user-model changes: add http_adapter for feed.
- storage format changes: no
- interface changes: no
Details:
1. Added http feed. User may use following syntax to create a http feed:
create feed TweetFeed with {
"adapter-name" : "http_adapter",
"addresses" : "asterix_nc2:10002,asterix_nc1:10001",
"address-type" : "NC",
"type-name" : "TweetMessageType",
"format" : "adm"
};
2. Refactored insert-feed test case to avoid waiting for 10s.
3. Refactored some constants in feeds.
Change-Id: I3c197a3df557ecc01f07f0907688c4ea81379e40
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2994
Reviewed-by: Murtadha Hubail
Sonar-Qube: Jenkins
Tested-by: Jenkins
Contrib: Jenkins
Integration-Tests: Jenkins
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.3.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.4.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.5.sleep.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.6.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.7.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.8.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.1.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.2.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.3.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-external-data/pom.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IExternalDataSourceFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/StreamRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/SocketServerInputStreamFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedUtils.java
M
asterixdb/asterix-external-data/src/main/resources/META-INF/services/org.apache.asterix.external.api.IRecordReaderFactory
23 files changed, 593 insertions(+), 106 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Jenkins: Verified; No violations found; ; Verified
Murtadha Hubail: Looks good to me, approved
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
new file mode 100644
index 000..b6e5aa3
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+drop dataverse experiments if exists;
+create dataverse experiments;
+use exp
Change in asterixdb[master]: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
Hello Anon. E. Moose #1000171, Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/2994
to look at the new patch set (#3).
Change subject: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
..
[ASTERIXDB-2264][ING] Introduce Http Feed Adapter
- user-model changes: add http_adapter for feed.
- storage format changes: no
- interface changes: no
Details:
1. Added http feed. User may use following syntax to create a http feed:
create feed TweetFeed with {
"adapter-name" : "http_adapter",
"addresses" : "asterix_nc2:10002,asterix_nc1:10001",
"address-type" : "NC",
"type-name" : "TweetMessageType",
"format" : "adm"
};
2. Refactored insert-feed test case to avoid waiting for 10s.
3. Refactored some constants in feeds.
Change-Id: I3c197a3df557ecc01f07f0907688c4ea81379e40
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.3.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.4.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.5.sleep.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.6.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.7.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.8.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.1.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.2.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.3.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-external-data/pom.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IExternalDataSourceFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/StreamRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/SocketServerInputStreamFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedUtils.java
M
asterixdb/asterix-external-data/src/main/resources/META-INF/services/org.apache.asterix.external.api.IRecordReaderFactory
23 files changed, 593 insertions(+), 106 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/94/2994/3
--
To view, visit https://asterix-gerrit.ics.uci.edu/2994
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3c197a3df557ecc01f07f0907688c4ea81379e40
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Murtadha Hubail
Change in asterixdb[master]: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
Hello Anon. E. Moose #1000171, Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/2994
to look at the new patch set (#2).
Change subject: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
..
[ASTERIXDB-2264][ING] Introduce Http Feed Adapter
- user-model changes: add http_adapter for feed.
- storage format changes: no
- interface changes: no
Details:
1. Added http feed. User may use following syntax to create a http feed:
create feed TweetFeed with {
"adapter-name" : "http_adapter",
"addresses" : "asterix_nc2:10002,asterix_nc1:10001",
"address-type" : "NC",
"type-name" : "TweetMessageType",
"format" : "adm"
};
2. Refactored insert-feed test case to avoid waiting for 10s.
3. Refactored some constants in feeds.
Change-Id: I3c197a3df557ecc01f07f0907688c4ea81379e40
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.3.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.4.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.5.sleep.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.6.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.7.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.8.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.1.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.2.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.3.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-external-data/pom.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IExternalDataSourceFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/StreamRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/SocketServerInputStreamFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedUtils.java
M
asterixdb/asterix-external-data/src/main/resources/META-INF/services/org.apache.asterix.external.api.IRecordReaderFactory
23 files changed, 591 insertions(+), 106 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/94/2994/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/2994
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3c197a3df557ecc01f07f0907688c4ea81379e40
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/2994
Change subject: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
..
[ASTERIXDB-2264][ING] Introduce Http Feed Adapter
- user-model changes: add http_adapter for feed.
- storage format changes: no
- interface changes: no
Details:
1. Added http feed. User may use following syntax to create a http feed:
create feed TweetFeed with {
"adapter-name" : "http_adapter",
"addresses" : "asterix_nc2:10002,asterix_nc1:10001",
"address-type" : "NC",
"type-name" : "TweetMessageType",
"format" : "adm"
};
2. Refactored insert-feed test case to avoid waiting for 10s.
3. Refactored some constants in feeds.
Change-Id: I3c197a3df557ecc01f07f0907688c4ea81379e40
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.3.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.4.post.http
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.5.sleep.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.6.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.7.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.8.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.1.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.2.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/http_feed/http_feed.3.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-external-data/pom.xml
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IExternalDataSourceFactory.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReader.java
A
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/http/HttpServerRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/StreamRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/twitter/TwitterRecordReaderFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/SocketServerInputStreamFactory.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/provider/DatasourceFactoryProvider.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedUtils.java
M
asterixdb/asterix-external-data/src/main/resources/META-INF/services/org.apache.asterix.external.api.IRecordReaderFactory
23 files changed, 587 insertions(+), 106 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/94/2994/1
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
new file mode 100644
index 000..b6e5aa3
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+drop dataverse experiments if exists;
+create dataverse experiments;
+use experiments;
+
+create type TweetMessageType as open {
+id : string
+};
+
+create dataset Tweets(TweetMessageType) primary key id;
+
+create feed TweetFeed with {
+ "adapter-name" : "http_
Change in asterixdb[master]: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
Xikui Wang has abandoned this change. Change subject: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter .. Abandoned Replace this with another change. -- To view, visit https://asterix-gerrit.ics.uci.edu/2321 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I9c03bddd0df59b69a1f2d6e989b8ee93cf50a6c0 Gerrit-PatchSet: 33 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail
Change in asterixdb[master]: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
Hello Anon. E. Moose #1000171, Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/2321
to look at the new patch set (#32).
Change subject: [ASTERIXDB-2264][ING] Introduce Http Feed Adapter
..
[ASTERIXDB-2264][ING] Introduce Http Feed Adapter
- user-model changes: add http_adapter for feed.
- storage format changes: no
- interface changes: no
Details:
1. Added http feed. User may use following syntax to create a http feed:
create feed TweetFeed with {
"adapter-name" : "http_adapter",
"addresses" : "asterix_nc2:10002,asterix_nc1:10001",
"address-type" : "NC",
"type-name" : "TweetMessageType",
"format" : "adm"
};
2. Refactored part of the test framework to be able to send http
requests to any port.
3. Refactored insert-feed test case to avoid waiting for 10s.
4. Refactored some constants in feeds.
Change-Id: I9c03bddd0df59b69a1f2d6e989b8ee93cf50a6c0
---
M
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
M
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/LangExecutionUtil.java
M
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/ReplicationExecutionTest.java
M
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionIT.java
R
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.1.ddl.sqlpp
C
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.2.update.sqlpp
R
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.3.post.http
R
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.4.post.http
C
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.5.sleep.sqlpp
C
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.6.update.sqlpp
C
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.7.query.sqlpp
C
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/http_feed/http_feed.8.ddl.sqlpp
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/insert-feed/insert-feed.1.ddl.sqlpp
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/insert-feed/insert-feed.2.update.sqlpp
R
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/insert-feed/insert-feed.3.query.sqlpp
R
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/insert-feed/insert-feed.4.ddl.sqlpp
D
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/insert-feed/insert-feed.7.server.sqlpp
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/add_replica/add_replica.1.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/add_replica/add_replica.2.pollget.http
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/bulkload/bulkload.1.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/bulkload/bulkload.13.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/bulkload/bulkload.14.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/bulkload/bulkload.2.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/bulkload/bulkload.3.pollget.http
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/bulkload/bulkload.4.pollget.http
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/bulkload/bulkload.7.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/bulkload/bulkload.8.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/flushed_component/flushed_component.1.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/flushed_component/flushed_component.10.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/flushed_component/flushed_component.2.pollget.http
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/flushed_component/flushed_component.5.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/mem_component_recovery/mem_component_recovery.1.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/mem_component_recovery/mem_component_recovery.13.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/mem_component_recovery/mem_component_recovery.14.sto.cmd
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/replication/mem_component_recovery/mem_component_recovery.2.sto.cmd
M
ast
Change in asterixdb[master]: [NO ISSUE][FUN] Add more data types for external library
Xikui Wang has submitted this change and it was merged.
Change subject: [NO ISSUE][FUN] Add more data types for external library
..
[NO ISSUE][FUN] Add more data types for external library
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
1. Added more types for external library.
2. Added test case for data types.
3. Fixed minor bug where the parameter type is not trimmed in UDF.
Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2871
Sonar-Qube: Jenkins
Tested-by: Jenkins
Contrib: Jenkins
Integration-Tests: Jenkins
Reviewed-by: Dmitry Lychagin
---
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.1.lib.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.2.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.3.lib.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.4.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/type_validation/type_validation.1.adm
M
asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml
A
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/TypeValidationFunction.java
A
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/TypeValidationFunctionFactory.java
M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml
M
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java
M
asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm
13 files changed, 229 insertions(+), 7 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Jenkins: Verified; No violations found; ; Verified
Dmitry Lychagin: Looks good to me, approved
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
index 0af1b9d..0eac212 100755
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
@@ -222,7 +222,7 @@
String functionType = function.getFunctionType().trim();
List args = new ArrayList<>();
for (String arg : fargs) {
-args.add(arg);
+args.add(arg.trim());
}
FunctionSignature signature = new
FunctionSignature(dataverse, functionFullName, args.size());
Function f = new Function(signature, args,
functionReturnType, functionDefinition, functionLanguage,
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.1.lib.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.1.lib.sqlpp
new file mode 100644
index 000..25aff38
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.1.lib.sqlpp
@@ -0,0 +1,20 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+install externallibtest testlib
target/data/externallib/asterix-external-data-testlib.zip
\ No newline at end of file
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.2.query.sqlpp
b/asterixdb/asterix-app/src/
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Xikui Wang has submitted this change and it was merged. Change subject: [NO ISSUE] User-defined Function Documentation update .. [NO ISSUE] User-defined Function Documentation update - user model changes: no - storage format changes: no - interface changes: no Details: 1. Updated the UDF documentation to be consisten with current master. 2. Cleaned default UDF package to remove useless UDFs. 3. Added the example in documentation as a test case for IT. 4. Reorganized the documentation to keep up with the new structure. 5. Minor changes to other documentation pages to keep style consistent. Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Reviewed-on: https://asterix-gerrit.ics.uci.edu/2804 Sonar-Qube: Jenkins Tested-by: Jenkins Contrib: Jenkins Reviewed-by: Taewoo Kim Integration-Tests: Jenkins --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.2.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.5.pollquery.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.6.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.7.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-function/feed-with-external-function.1.adm M asterixdb/asterix-doc/pom.xml R asterixdb/asterix-doc/src/main/data_ingestion/feeds.md A asterixdb/asterix-doc/src/main/data_ingestion/feeds_title.md M asterixdb/asterix-doc/src/main/installation/ansible_title.md M asterixdb/asterix-doc/src/main/installation/aws_title.md A asterixdb/asterix-doc/src/main/user-defined_function/udf.md A asterixdb/asterix-doc/src/main/user-defined_function/udf_title.md M asterixdb/asterix-doc/src/site/markdown/ncservice.md D asterixdb/asterix-doc/src/site/markdown/udf.md M asterixdb/asterix-doc/src/site/site.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/Datatypes.java D asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsFunction.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsInPlaceFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddMentionedUsersFactory.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/addMentionedUsersFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm 26 files changed, 329 insertions(+), 359 deletions(-) Approvals: Anon. E. Moose #1000171: Taewoo Kim: Looks good to me, approved Jenkins: Verified; No violations found; ; Verified diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp index 9402e1f..a6a1cdc 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp @@ -18,5 +18,4 @@ */ use externallibtest; -let x=testlib#mysum(3,4) -select VALUE x; +testlib#mysum(3,4); diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp index 4fdc669..3bc33de 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp @@ -22,34 +22,20 @@ * Date : 4th Oct 2017 */ -drop dataverse externallibtest if exists; -create dataverse externallibtest; -use e
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Hello Anon. E. Moose #1000171, Taewoo Kim, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2804 to look at the new patch set (#16). Change subject: [NO ISSUE] User-defined Function Documentation update .. [NO ISSUE] User-defined Function Documentation update - user model changes: no - storage format changes: no - interface changes: no Details: 1. Updated the UDF documentation to be consisten with current master. 2. Cleaned default UDF package to remove useless UDFs. 3. Added the example in documentation as a test case for IT. 4. Reorganized the documentation to keep up with the new structure. 5. Minor changes to other documentation pages to keep style consistent. Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.2.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.5.pollquery.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.6.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.7.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-function/feed-with-external-function.1.adm M asterixdb/asterix-doc/pom.xml R asterixdb/asterix-doc/src/main/data_ingestion/feeds.md A asterixdb/asterix-doc/src/main/data_ingestion/feeds_title.md M asterixdb/asterix-doc/src/main/installation/ansible_title.md M asterixdb/asterix-doc/src/main/installation/aws_title.md A asterixdb/asterix-doc/src/main/user-defined_function/udf.md A asterixdb/asterix-doc/src/main/user-defined_function/udf_title.md M asterixdb/asterix-doc/src/site/markdown/ncservice.md D asterixdb/asterix-doc/src/site/markdown/udf.md M asterixdb/asterix-doc/src/site/site.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/Datatypes.java D asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsFunction.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsInPlaceFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddMentionedUsersFactory.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/addMentionedUsersFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm 26 files changed, 329 insertions(+), 359 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/2804/16 -- To view, visit https://asterix-gerrit.ics.uci.edu/2804 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Gerrit-PatchSet: 16 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Ian2 Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Xikui Wang has posted comments on this change. Change subject: [NO ISSUE] User-defined Function Documentation update .. Patch Set 14: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/2804/14/asterixdb/asterix-doc/src/main/user-defined_function/udf.md File asterixdb/asterix-doc/src/main/user-defined_function/udf.md: Line 54: the `asterixdb-external` sub-project. Assuming the UDFs are installed into the `udfs` dataverse and `testlib` library, > Maybe if we have a stand-alone package for the same example under a new rep Yes. There is one repo that I created under my account, but I'm not sure whether we want to make it official. -- To view, visit https://asterix-gerrit.ics.uci.edu/2804 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Ian2 Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Xikui Wang has posted comments on this change. Change subject: [NO ISSUE] User-defined Function Documentation update .. Patch Set 15: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/2804/15/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp: Line 26: connect feed TweetFeed to dataset ProcessedTweets apply function testlib#addMentionedUsers; > Is there a reason to remove the parallelism hint? no particular reason... The hint here is not useful for the test. -- To view, visit https://asterix-gerrit.ics.uci.edu/2804 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Gerrit-PatchSet: 15 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Ian2 Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Xikui Wang has posted comments on this change. Change subject: [NO ISSUE] User-defined Function Documentation update .. Patch Set 14: (7 comments) Thanks for the review! https://asterix-gerrit.ics.uci.edu/#/c/2804/14/asterixdb/asterix-doc/src/main/user-defined_function/udf.md File asterixdb/asterix-doc/src/main/user-defined_function/udf.md: Line 22: Apache AsterixDB supports two types of user-defined functions (UDFs): SQLPP function and Programming language based function. > maybe "Apache AsterixDB supports two languages for writing user-defined fun Done Line 26: mainly focus on how to install/invoke/uninstall a programming language function library using the Ansible script that we provided. > s/programming language function/Java/g Done Line 27: Currently, AsterixDB only supports Java based programming UDF and we are working on supporting more languages. :) > dunno if this really belongs here Done Line 33: to deploy an AsterixDB instance on your local machine or cluster. The UDFs have to be installed offline. Here are the > "This section assumes you have followed the preceding ansible installation Done Line 44: - Step 3: Start the AsterixDB > no 'the' Done Line 48: After AsterixDB starts, you can use the following query to check whether your UDFs are sucessfully installed. > are/have been Done Line 54: the `asterixdb-external` sub-project. Assuming the UDFs are installed into the `udfs` dataverse and `testlib` library, > this is sort of vague, assumes knowledge of maven couldn't find a better way to point people to the right place let's if the others have better ideas about this... -- To view, visit https://asterix-gerrit.ics.uci.edu/2804 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Ian2 Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2804 to look at the new patch set (#15). Change subject: [NO ISSUE] User-defined Function Documentation update .. [NO ISSUE] User-defined Function Documentation update - user model changes: no - storage format changes: no - interface changes: no Details: 1. Updated the UDF documentation to be consisten with current master. 2. Cleaned default UDF package to remove useless UDFs. 3. Added the example in documentation as a test case for IT. 4. Reorganized the documentation to keep up with the new structure. 5. Minor changes to other documentation pages to keep style consistent. Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.2.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.5.pollquery.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.6.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.7.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-function/feed-with-external-function.1.adm M asterixdb/asterix-doc/pom.xml R asterixdb/asterix-doc/src/main/data_ingestion/feeds.md A asterixdb/asterix-doc/src/main/data_ingestion/feeds_title.md M asterixdb/asterix-doc/src/main/installation/ansible_title.md M asterixdb/asterix-doc/src/main/installation/aws_title.md A asterixdb/asterix-doc/src/main/user-defined_function/udf.md A asterixdb/asterix-doc/src/main/user-defined_function/udf_title.md M asterixdb/asterix-doc/src/site/markdown/ncservice.md D asterixdb/asterix-doc/src/site/markdown/udf.md M asterixdb/asterix-doc/src/site/site.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/Datatypes.java D asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsFunction.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsInPlaceFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddMentionedUsersFactory.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/addMentionedUsersFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm 26 files changed, 328 insertions(+), 359 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/2804/15 -- To view, visit https://asterix-gerrit.ics.uci.edu/2804 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Gerrit-PatchSet: 15 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Ian2 Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2804 to look at the new patch set (#14). Change subject: [NO ISSUE] User-defined Function Documentation update .. [NO ISSUE] User-defined Function Documentation update - user model changes: no - storage format changes: no - interface changes: no Details: 1. Updated the UDF documentation to be consisten with current master. 2. Cleaned default UDF package to remove useless UDFs. 3. Added the example in documentation as a test case for IT. 4. Reorganized the documentation to keep up with the new structure. 5. Minor changes to other documentation pages to keep style consistent. Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.2.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.5.pollquery.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.6.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.7.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-function/feed-with-external-function.1.adm M asterixdb/asterix-doc/pom.xml R asterixdb/asterix-doc/src/main/data_ingestion/feeds.md A asterixdb/asterix-doc/src/main/data_ingestion/feeds_title.md M asterixdb/asterix-doc/src/main/installation/ansible_title.md M asterixdb/asterix-doc/src/main/installation/aws_title.md A asterixdb/asterix-doc/src/main/user-defined_function/udf.md A asterixdb/asterix-doc/src/main/user-defined_function/udf_title.md M asterixdb/asterix-doc/src/site/markdown/ncservice.md D asterixdb/asterix-doc/src/site/markdown/udf.md M asterixdb/asterix-doc/src/site/site.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/Datatypes.java D asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsFunction.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsInPlaceFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddMentionedUsersFactory.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/addMentionedUsersFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm 26 files changed, 329 insertions(+), 359 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/2804/14 -- To view, visit https://asterix-gerrit.ics.uci.edu/2804 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Gerrit-PatchSet: 14 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian2 Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2804 to look at the new patch set (#13). Change subject: [NO ISSUE] User-defined Function Documentation update .. [NO ISSUE] User-defined Function Documentation update - user model changes: no - storage format changes: no - interface changes: no Details: 1. Updated the UDF documentation to be consisten with current master. 2. Cleaned default UDF package to remove useless UDFs. 3. Added the example in documentation as a test case for IT. 4. Reorganized the documentation to keep up with the new structure. 5. Minor changes to other documentation pages to keep style consistent. Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.2.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.5.pollquery.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.6.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.7.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-function/feed-with-external-function.1.adm M asterixdb/asterix-doc/pom.xml R asterixdb/asterix-doc/src/main/data_ingestion/feeds.md A asterixdb/asterix-doc/src/main/data_ingestion/feeds_title.md M asterixdb/asterix-doc/src/main/installation/ansible_title.md M asterixdb/asterix-doc/src/main/installation/aws_title.md A asterixdb/asterix-doc/src/main/user-defined_function/udf.md A asterixdb/asterix-doc/src/main/user-defined_function/udf_title.md M asterixdb/asterix-doc/src/site/markdown/ncservice.md D asterixdb/asterix-doc/src/site/markdown/udf.md M asterixdb/asterix-doc/src/site/site.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/Datatypes.java D asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsFunction.java M asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsInPlaceFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddMentionedUsersFactory.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/addMentionedUsersFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm 26 files changed, 337 insertions(+), 367 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/2804/13 -- To view, visit https://asterix-gerrit.ics.uci.edu/2804 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Gerrit-PatchSet: 13 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian2 Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Carey Gerrit-Reviewer: Taewoo Kim Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: Wail Alkowaileet
Change in asterixdb[master]: [NO ISSUE] User-defined Function Documentation update
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2804 to look at the new patch set (#11). Change subject: [NO ISSUE] User-defined Function Documentation update .. [NO ISSUE] User-defined Function Documentation update - user model changes: no - storage format changes: no - interface changes: no Details: 1. Updated the UDF documentation to be consisten with current master. 2. Cleaned default UDF package to remove useless UDFs. 3. Added the example in documentation as a test case for IT. 4. Reorganized the documentation to keep up with the new structure. 5. Minor changes to other documentation pages to keep style consistent. Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/mysum/mysum.3.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.1.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.2.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.3.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.5.pollquery.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.6.lib.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/feeds/feed-with-external-function/feed-with-external-function.7.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/feed-with-external-function/feed-with-external-function.1.adm M asterixdb/asterix-doc/pom.xml R asterixdb/asterix-doc/src/main/data_ingestion/feeds.md A asterixdb/asterix-doc/src/main/data_ingestion/feeds_title.md M asterixdb/asterix-doc/src/main/installation/ansible_title.md M asterixdb/asterix-doc/src/main/installation/aws_title.md A asterixdb/asterix-doc/src/main/user-defined_function/udf.md A asterixdb/asterix-doc/src/main/user-defined_function/udf_title.md M asterixdb/asterix-doc/src/site/markdown/ncservice.md D asterixdb/asterix-doc/src/site/markdown/udf.md M asterixdb/asterix-doc/src/site/site.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/Datatypes.java D asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsFactory.java D asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddHashTagsFunction.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/AddMentionedUsersFactory.java R asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/addMentionedUsersFunction.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm 26 files changed, 271 insertions(+), 403 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/2804/11 -- To view, visit https://asterix-gerrit.ics.uci.edu/2804 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1 Gerrit-PatchSet: 11 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [NO ISSUE][FUN] Add more data types for external library
Xikui Wang has posted comments on this change.
Change subject: [NO ISSUE][FUN] Add more data types for external library
..
Patch Set 7:
(1 comment)
https://asterix-gerrit.ics.uci.edu/#/c/2871/7/asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/TypeValidationFunction.java
File
asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/TypeValidationFunction.java:
PS7, Line 46: System.out.println("De-Initialized");
> Can you please remove this? or at least use a logger?
Done
--
To view, visit https://asterix-gerrit.ics.uci.edu/2871
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Till Westmann
Gerrit-Reviewer: Xikui Wang
Gerrit-Reviewer: abdullah alamoudi
Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][FUN] Add more data types for external library
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2871 to look at the new patch set (#8). Change subject: [NO ISSUE][FUN] Add more data types for external library .. [NO ISSUE][FUN] Add more data types for external library - user model changes: no - storage format changes: no - interface changes: no Details: 1. Added more types for external library. 2. Added test case for data types. 3. Fixed minor bug where the parameter type is not trimmed in UDF. Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31 --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.1.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.3.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/type_validation/type_validation.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/TypeValidationFunction.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/TypeValidationFunctionFactory.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm 13 files changed, 229 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/71/2871/8 -- To view, visit https://asterix-gerrit.ics.uci.edu/2871 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31 Gerrit-PatchSet: 8 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi
Change in asterixdb[master]: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntro...
Xikui Wang has submitted this change and it was merged.
Change subject: [NO-ISSUE][COMP] Avoid adding redundant var in
AbstractIntroduceGroupByCombinerRule
..
[NO-ISSUE][COMP] Avoid adding redundant var in
AbstractIntroduceGroupByCombinerRule
- user model changes: no
- storage format changes: no
- interface changes: no
For live variables added in new Group-by op, they should not be added
again.
Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2933
Sonar-Qube: Jenkins
Tested-by: Jenkins
Contrib: Jenkins
Integration-Tests: Jenkins
Reviewed-by: Dmitry Lychagin
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.3.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.4.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java
7 files changed, 120 insertions(+), 7 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Jenkins: Verified; No violations found; ; Verified
Dmitry Lychagin: Looks good to me, approved
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp
new file mode 100644
index 000..3100af6
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+drop dataverse test if exists;
+create dataverse test;
+use test;
+
+create type TweetType as open {
+ id : int64
+};
+
+create dataset Tweets(TweetType) primary key id;
+
+drop type NearbyBuildingType if exists;
+create type NearbyBuildingType as open {
+buildingId: int64,
+buildingType : string,
+buildingLocation : point
+};
+create dataset NearbyBuildings(NearbyBuildingType) primary key buildingId;
+
+create function annotateTweet(x) {
+LET nearby_buildings = (select r.buildingType as buildingType, count(r) as
cnt
+ from NearbyBuildings r
+ where spatial_intersect(create_point(x.latitude, x.longitude),
create_circle(r.buildingLocation, 3.0))
+ group by r.buildingType)
+select x.*, nearby_buildings
+};
+
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp
new file mode 100644
index 000..ab13484
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed unde
Change in asterixdb[master]: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntro...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2933 to look at the new patch set (#7). Change subject: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule .. [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule - user model changes: no - storage format changes: no - interface changes: no For live variables added in new Group-by op, they should not be added again. Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java 7 files changed, 120 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/33/2933/7 -- To view, visit https://asterix-gerrit.ics.uci.edu/2933 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 Gerrit-PatchSet: 7 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang
Change in asterixdb[master]: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntro...
Xikui Wang has posted comments on this change.
Change subject: [NO-ISSUE][COMP] Avoid adding redundant var in
AbstractIntroduceGroupByCombinerRule
..
Patch Set 4:
(4 comments)
https://asterix-gerrit.ics.uci.edu/#/c/2933/4/asterixdb/asterix-app/src/test/resources/log4j2-asterixdb-test.xml
File asterixdb/asterix-app/src/test/resources/log4j2-asterixdb-test.xml:
Line 44:
> Let's revert these lines. We don't need query plans in the logs by default.
done. this was added mistakenly...
https://asterix-gerrit.ics.uci.edu/#/c/2933/4/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp
File
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp:
Line 38: LET nearby_buildings = (select r.buildingType as Religion,
count(r) as cnt
> why "as Religion"? can we remove it? This is building type, so let this fie
Done
https://asterix-gerrit.ics.uci.edu/#/c/2933/4/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java
File
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java:
Line 87: if (!newGbyLiveVars.contains(usedDecorVars.get(0))) {
> can we extract usedDecorVars.get(0) into a separate variable and use this v
Done
Line 108: if (!propagatedVars.contains(var) &&
!newGbyLiveVars.contains(var)) {
> The only difference between VariableUtilities.getLiveVariables(newGbyOp) an
Done
--
To view, visit https://asterix-gerrit.ics.uci.edu/2933
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Xikui Wang
Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntro...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2933 to look at the new patch set (#6). Change subject: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule .. [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule - user model changes: no - storage format changes: no - interface changes: no For live variables added in new Group-by op, they should not be added again. Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 --- M asterixdb/asterix-app/src/test/resources/optimizerts/only.txt M asterixdb/asterix-app/src/test/resources/runtimets/only_sqlpp.xml A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java 9 files changed, 126 insertions(+), 8 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/33/2933/6 -- To view, visit https://asterix-gerrit.ics.uci.edu/2933 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 Gerrit-PatchSet: 6 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntro...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2933 to look at the new patch set (#5). Change subject: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule .. [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule - user model changes: no - storage format changes: no - interface changes: no For live variables added in new Group-by op, they should not be added again. Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 --- M asterixdb/asterix-app/src/test/resources/optimizerts/only.txt A asterixdb/asterix-app/src/test/resources/optimizerts/results/redundant-var-in-groupby.plan M asterixdb/asterix-app/src/test/resources/runtimets/only_sqlpp.xml A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java 10 files changed, 196 insertions(+), 8 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/33/2933/5 -- To view, visit https://asterix-gerrit.ics.uci.edu/2933 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntro...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2933 to look at the new patch set (#3). Change subject: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule .. [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule - user model changes: no - storage format changes: no - interface changes: no For live variables added in new Group-by op, they should not be added again. Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 --- M asterixdb/asterix-app/src/test/resources/log4j2-asterixdb-test.xml A asterixdb/asterix-app/src/test/resources/optimizerts/results/redundant-var-in-groupby.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java 8 files changed, 170 insertions(+), 1 deletion(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/33/2933/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/2933 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntro...
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2933 to look at the new patch set (#2). Change subject: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule .. [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntroduceGroupByCombinerRule - user model changes: no - storage format changes: no - interface changes: no For live variables added in new Group-by op, they should not be added again. Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 --- M asterixdb/asterix-app/src/test/resources/log4j2-asterixdb-test.xml A asterixdb/asterix-app/src/test/resources/optimizerts/results/redundant-var-in-groupby.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/redundant-var-in-groupby/redundant-var-in-groupby.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/redundant-var-in-groupby/redundant-var-in-groupby.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java 8 files changed, 170 insertions(+), 1 deletion(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/33/2933/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/2933 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO-ISSUE][COMP] Avoid adding redundant var in AbstractIntro...
Xikui Wang has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/2933
Change subject: [NO-ISSUE][COMP] Avoid adding redundant var in
AbstractIntroduceGroupByCombinerRule
..
[NO-ISSUE][COMP] Avoid adding redundant var in
AbstractIntroduceGroupByCombinerRule
- user model changes: no
- storage format changes: no
- interface changes: no
For live variables added in new Group-by op, they should not be added
again.
Change-Id: Ic1ab9aee31db95d5782385bc3d53777da54f6d83
---
A
asterixdb/asterix-app/src/test/resources/optimizerts/queries/redundant-var-in-groupby.sqlpp
A
asterixdb/asterix-app/src/test/resources/optimizerts/results/redundant-var-in-groupby.plan
M
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceGroupByCombinerRule.java
3 files changed, 140 insertions(+), 1 deletion(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/33/2933/1
diff --git
a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/redundant-var-in-groupby.sqlpp
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/redundant-var-in-groupby.sqlpp
new file mode 100644
index 000..202a03d
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/redundant-var-in-groupby.sqlpp
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+drop dataverse test if exists;
+create dataverse test;
+use test;
+
+create type TweetType as open {
+ id : int64,
+ created_at : datetime
+};
+
+create type StoredTweetType as open {
+ tid : uuid
+};
+create dataset Tweets1(StoredTweetType) primary key tid autogenerated;
+create dataset Tweets2(StoredTweetType) primary key tid autogenerated;
+
+drop dataset ReligiousBuildingDataset if exists;
+drop type ReligiousBuildingType if exists;
+create type ReligiousBuildingType as open {
+religiousBuildingId : string,
+religionName : string,
+buildingLocation : point,
+registeredBeliever: int
+};
+create dataset ReligiousBuildingDataset(ReligiousBuildingType) primary key
religiousBuildingId;
+
+drop dataset AttackEventsDataset if exists;
+drop type AttackEventsType if exists;
+create type AttackEventsType as open {
+attackRecordId: string,
+attackDatetime: datetime,
+attackLocation: point,
+relatedReligion: string
+};
+create dataset AttackEventsDataset(AttackEventsType) primary key
attackRecordId;
+
+create function annotateTweet(x) {
+LET nearby_religious_attack = (select r.religionName as Religion,
count(a.attackRecordId) as AttackNum
+ from ReligiousBuildingDataset r, AttackEventsDataset a
+ where spatial_intersect(create_point(x.latitude, x.longitude),
create_circle(r.buildingLocation, 3.0))
+ and x.created_at < a.attackDatetime + duration("P2M")
+ and x.created_at > a.attackDatetime
+ and r.religionName = a.relatedReligion
+ group by r.religionName)
+select x.*, nearby_religious_attack
+};
+
+insert into Tweets2 (
+select annotateTweet(t) from Tweets1
+);
\ No newline at end of file
diff --git
a/asterixdb/asterix-app/src/test/resources/optimizerts/results/redundant-var-in-groupby.plan
b/asterixdb/asterix-app/src/test/resources/optimizerts/results/redundant-var-in-groupby.plan
new file mode 100644
index 000..f3ae099
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/optimizerts/results/redundant-var-in-groupby.plan
@@ -0,0 +1,70 @@
+-- COMMIT |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+-- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ -- INSERT_DELETE |PARTITIONED|
+-- HASH_PARTITION_EXCHANGE [$$105] |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+-- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+-- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+-- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+-- STREAM_PROJECT |PARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+-- PRE_
Change in asterixdb[master]: [ASTERIX-2430][COMP] Avoid inlining function calls in WITH i...
Xikui Wang has posted comments on this change. Change subject: [ASTERIX-2430][COMP] Avoid inlining function calls in WITH in AST rewritting .. Patch Set 5: I've looked at the query in q2-11. For the modified query, the pure function should be inlined. That's something that I didn't notice before. I'll abandon this patch as it's not fully correct. -- To view, visit https://asterix-gerrit.ics.uci.edu/2874 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I18afbe4165ca84c790d7693a7a7bd5f6fd53187a Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIX-2430][COMP] Avoid inlining function calls in WITH i...
Xikui Wang has abandoned this change. Change subject: [ASTERIX-2430][COMP] Avoid inlining function calls in WITH in AST rewritting .. Abandoned -- To view, visit https://asterix-gerrit.ics.uci.edu/2874 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I18afbe4165ca84c790d7693a7a7bd5f6fd53187a Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang
Change in asterixdb-bad[master]: Fix formatting of broker notification
Xikui Wang has posted comments on this change. Change subject: Fix formatting of broker notification .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/2900 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6f73b64d56d06a8ea3f785d465d4e0cda027dfb2 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb-bad Gerrit-Branch: master Gerrit-Owner: Steven Jacobs Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: No
Change in asterixdb-bad[master]: Fixed bug with pull-based channel
Xikui Wang has posted comments on this change. Change subject: Fixed bug with pull-based channel .. Patch Set 3: Verified+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/2876 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: If758f9e8d5c66c5badd819ad877d269bee4408c8 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb-bad Gerrit-Branch: master Gerrit-Owner: Steven Jacobs Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: No
Change in asterixdb-bad[master]: Fixed bug with pull-based channel
Xikui Wang has posted comments on this change. Change subject: Fixed bug with pull-based channel .. Patch Set 3: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/2876 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: If758f9e8d5c66c5badd819ad877d269bee4408c8 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb-bad Gerrit-Branch: master Gerrit-Owner: Steven Jacobs Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Steven Jacobs Gerrit-Reviewer: Xikui Wang Gerrit-HasComments: No
Change in asterixdb-bad[master]: Fixed bug with pull-based channel
Xikui Wang has posted comments on this change.
Change subject: Fixed bug with pull-based channel
..
Patch Set 1:
(3 comments)
https://asterix-gerrit.ics.uci.edu/#/c/2876/1/asterix-bad/src/main/java/org/apache/asterix/bad/runtime/NotifyBrokerRuntime.java
File
asterix-bad/src/main/java/org/apache/asterix/bad/runtime/NotifyBrokerRuntime.java:
Line 98: if (push) {
Could you add two line comments to the following so we would know why it's null.
Line 99: recordPrinterFactory = new
ARecordPrinterFactory((ARecordType) recordType).createPrinter();
record type is the actual data records.
Line 100: } else {
record type is a list of subscription ids
--
To view, visit https://asterix-gerrit.ics.uci.edu/2876
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: If758f9e8d5c66c5badd819ad877d269bee4408c8
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Xikui Wang
Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIX-2430][COMP] Avoid inlining function calls in WITH i...
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2874 to look at the new patch set (#4). Change subject: [ASTERIX-2430][COMP] Avoid inlining function calls in WITH in AST rewritting .. [ASTERIX-2430][COMP] Avoid inlining function calls in WITH in AST rewritting - user model changes: no - storage format changes: no - interface changes: no Details: The inlineWithExperssion() call in QueryRewritter inlines with/let experssion blindly. This can cause problem when the function call is on the DoNotInlineFuncs list. We can safely skip the inlining of function calls in the AST rewritting part as the calls that are not on the DoNotInlineFuncs list will be inlined by InlineVariablesRule later. Change-Id: I18afbe4165ca84c790d7693a7a7bd5f6fd53187a --- A asterixdb/asterix-app/src/test/resources/optimizerts/queries/ASTERIXDB-2430.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2430.plan M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/InlineWithExpressionVisitor.java 3 files changed, 117 insertions(+), 0 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/74/2874/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/2874 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I18afbe4165ca84c790d7693a7a7bd5f6fd53187a Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [NO ISSUE][FUN] Add more data types for external library
Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2871 to look at the new patch set (#3). Change subject: [NO ISSUE][FUN] Add more data types for external library .. [NO ISSUE][FUN] Add more data types for external library - user model changes: no - storage format changes: no - interface changes: no Details: 1. Added more types for external library. 2. Added test case for data types. 3. Fixed minor bug where the parameter type is not trimmed in UDF. Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31 --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.1.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.3.lib.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_validation/type_validation.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/type_validation/type_validation.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/external-library/validate-default-library/validate-default-library.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_it_sqlpp.xml A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/TypeValidationFunction.java A asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library/TypeValidationFunctionFactory.java M asterixdb/asterix-external-data/src/test/resources/library_descriptor.xml M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-server/src/test/resources/integrationts/library/results/library-metadata/functionDataset/functionDataset.1.adm 13 files changed, 234 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/71/2871/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/2871 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Xikui Wang Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins
