[beam] 02/02: fix type error

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit a7501289a23e2d8a3a7e5a3b9f5e920702d4d634
Author: mingmxu <ming...@ebay.com>
AuthorDate: Mon Feb 26 23:39:08 2018 -0800

fix type error
---
 .../java/org/apache/beam/sdk/extensions/sql/BeamSqlDslUdfUdafTest.java  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlDslUdfUdafTest.java
 
b/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlDslUdfUdafTest.java
index 01e45c1..4589d39 100644
--- 
a/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlDslUdfUdafTest.java
+++ 
b/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlDslUdfUdafTest.java
@@ -98,7 +98,7 @@ public class BeamSqlDslUdfUdafTest extends BeamSqlDslBase {
 
 RowType subStrRowType = RowSqlType.builder()
 .withIntegerField("f_int")
-.withIntegerField("sub_string")
+.withVarcharField("sub_string")
 .build();
 Row subStrRow = Row.withRowType(subStrRowType).addValues(2, "s").build();
 PAssert.that(result3).containsInAnyOrder(subStrRow);

-- 
To stop receiving notification emails like this one, please contact
xumingm...@apache.org.

[beam] branch master updated (8a39c80 -> a750128)

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 8a39c80  Merge pull request #4695: Add To/From Proto Round Trip for 
ExecutableStage
 new 880caef  @Parameter annotation does not work for UDFs in Beam SQL
 new a750128  fix type error

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../apache/beam/sdk/extensions/sql/BeamSqlUdf.java |  2 +-
 .../sql/impl/interpreter/BeamSqlFnExecutor.java|  4 
 ...pression.java => BeamSqlDefaultExpression.java} | 15 +++--
 .../sdk/extensions/sql/BeamSqlDslUdfUdafTest.java  | 26 +-
 4 files changed, 33 insertions(+), 14 deletions(-)
 copy 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/{BeamSqlInputRefExpression.java
 => BeamSqlDefaultExpression.java} (74%)

-- 
To stop receiving notification emails like this one, please contact
xumingm...@apache.org.

[beam] 01/02: @Parameter annotation does not work for UDFs in Beam SQL

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 880caef3fc17295f817648ccdffe369c2dc63fa1
Author: mingmxu <ming...@ebay.com>
AuthorDate: Mon Feb 26 15:42:55 2018 -0800

@Parameter annotation does not work for UDFs in Beam SQL
---
 .../apache/beam/sdk/extensions/sql/BeamSqlUdf.java |  2 +-
 .../sql/impl/interpreter/BeamSqlFnExecutor.java|  4 +++
 .../operator/BeamSqlDefaultExpression.java}| 38 ++
 .../sdk/extensions/sql/BeamSqlDslUdfUdafTest.java  | 26 ++-
 4 files changed, 47 insertions(+), 23 deletions(-)

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlUdf.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlUdf.java
index 91bad20..5df046a 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlUdf.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlUdf.java
@@ -35,7 +35,7 @@ import org.apache.beam.sdk.annotations.Experimental;
  * }
  *
  * The first parameter is named "s" and is mandatory,
- * and the second parameter is named "n" and is optional.
+ * and the second parameter is named "n" and is optional(always NULL if not 
specified).
  */
 @Experimental
 public interface BeamSqlUdf extends Serializable {
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
index ae65c2b..dbdd03d 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
@@ -23,6 +23,7 @@ import java.util.Calendar;
 import java.util.List;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlCaseExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlCastExpression;
+import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlDefaultExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlInputRefExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlPrimitive;
@@ -383,6 +384,9 @@ public class BeamSqlFnExecutor implements 
BeamSqlExpressionExecutor {
 case "DATETIME_PLUS":
   return new BeamSqlDatetimePlusExpression(subExps);
 
+//DEFAULT keyword for UDF with optional parameter
+case "DEFAULT":
+  return new BeamSqlDefaultExpression();
 
 case "CASE":
   ret = new BeamSqlCaseExpression(subExps);
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlUdf.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/BeamSqlDefaultExpression.java
similarity index 52%
copy from 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlUdf.java
copy to 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/BeamSqlDefaultExpression.java
index 91bad20..0557600 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlUdf.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/BeamSqlDefaultExpression.java
@@ -15,29 +15,25 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-package org.apache.beam.sdk.extensions.sql;
+package org.apache.beam.sdk.extensions.sql.impl.interpreter.operator;
 
-import java.io.Serializable;
-import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.transforms.windowing.BoundedWindow;
+import org.apache.beam.sdk.values.Row;
+import org.apache.calcite.sql.type.SqlTypeName;
 
 /**
- * Interface to create a UDF in Beam SQL.
- *
- * A static method {@code eval} is required. Here is an example:
- *
- * 
- * public static class MyLeftFunction {
- *   public String eval(
- *   Parameter(name = "s") String s,
- *   Parameter(name = "n", optional = true) Integer n) {
- * return s.substring(0, n == null ? 1 : n);
- *   }
- * }
- *
- * The first parameter is named "s" and is mandatory,
- * and the second parameter is named "n" and is optional.
+ * DEFAULT keyword for UDF with optional parameter.
  */
-@Experimental
-public interface BeamSqlUdf extends Serializable {
-  String UDF_METHOD = &

[beam] 01/02: add setup/teardown for BeamSqlSeekableTable.

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit b0991a47a0e1a7bba416be1c0b3781c298966731
Author: mingmxu <ming...@ebay.com>
AuthorDate: Sun Feb 25 22:14:25 2018 -0800

add setup/teardown for BeamSqlSeekableTable.
---
 .../apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java | 10 ++
 .../extensions/sql/impl/transform/BeamJoinTransforms.java| 12 +++-
 .../sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java  |  8 
 3 files changed, 29 insertions(+), 1 deletion(-)

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java
index d274dd9..95165a5 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java
@@ -29,7 +29,17 @@ import org.apache.beam.sdk.values.Row;
 @Experimental
 public interface BeamSqlSeekableTable extends Serializable{
   /**
+   * prepare the instance.
+   */
+  void setup();
+
+  /**
* return a list of {@code Row} with given key set.
*/
   List seekRow(Row lookupSubRow);
+
+  /**
+   * cleanup resources of the instance.
+   */
+  void teardown();
 }
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
index 648b973..2b44814 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
@@ -219,7 +219,12 @@ public class BeamJoinTransforms {
 
 @Override
 public PCollection expand(PCollection input) {
-  return input.apply("join_as_lookup", ParDo.of(new DoFn<Row, Row>() {
+  return input.apply("join_as_lookup", ParDo.of(new DoFn<Row, Row>(){
+@Setup
+public void setup(){
+  seekableTable.setup();
+}
+
 @ProcessElement
 public void processElement(ProcessContext context) {
   Row factRow = context.element();
@@ -230,6 +235,11 @@ public class BeamJoinTransforms {
   }
 }
 
+@Teardown
+public void teardown(){
+  seekableTable.teardown();
+}
+
 private Row extractJoinSubRow(Row factRow) {
   List joinSubsetValues =
   factJoinIdx
diff --git 
a/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java
 
b/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java
index ca639bd..6a06123 100644
--- 
a/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java
+++ 
b/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java
@@ -131,6 +131,14 @@ public class BeamJoinRelUnboundedVsBoundedTest extends 
BaseRelTest {
 public List seekRow(Row lookupSubRow) {
   return Arrays.asList(Row.withRowType(getRowType()).addValues(1, 
"SITE1").build());
 }
+
+@Override
+public void setup() {
+}
+
+@Override
+public void teardown() {
+}
   }
 
   @Test

-- 
To stop receiving notification emails like this one, please contact
xumingm...@apache.org.

[beam] 02/02: rename method as suggested and declare as default methods.

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit dbaf09838c5d064fdd6d953b91ad9a1ce61af72b
Author: mingmxu <ming...@ebay.com>
AuthorDate: Mon Feb 12 08:02:31 2018 -0800

rename method as suggested and declare as default methods.
---
 .../org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java  | 4 ++--
 .../sdk/extensions/sql/impl/transform/BeamJoinTransforms.java | 4 ++--
 .../sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java   | 8 
 3 files changed, 4 insertions(+), 12 deletions(-)

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java
index 95165a5..7b256c8 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java
@@ -31,7 +31,7 @@ public interface BeamSqlSeekableTable extends Serializable{
   /**
* prepare the instance.
*/
-  void setup();
+  default void setUp(){};
 
   /**
* return a list of {@code Row} with given key set.
@@ -41,5 +41,5 @@ public interface BeamSqlSeekableTable extends Serializable{
   /**
* cleanup resources of the instance.
*/
-  void teardown();
+  default void tearDown(){};
 }
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
index 2b44814..1be47b4 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
@@ -222,7 +222,7 @@ public class BeamJoinTransforms {
   return input.apply("join_as_lookup", ParDo.of(new DoFn<Row, Row>(){
 @Setup
 public void setup(){
-  seekableTable.setup();
+  seekableTable.setUp();
 }
 
 @ProcessElement
@@ -237,7 +237,7 @@ public class BeamJoinTransforms {
 
 @Teardown
 public void teardown(){
-  seekableTable.teardown();
+  seekableTable.tearDown();
 }
 
 private Row extractJoinSubRow(Row factRow) {
diff --git 
a/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java
 
b/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java
index 6a06123..ca639bd 100644
--- 
a/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java
+++ 
b/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRelUnboundedVsBoundedTest.java
@@ -131,14 +131,6 @@ public class BeamJoinRelUnboundedVsBoundedTest extends 
BaseRelTest {
 public List seekRow(Row lookupSubRow) {
   return Arrays.asList(Row.withRowType(getRowType()).addValues(1, 
"SITE1").build());
 }
-
-@Override
-public void setup() {
-}
-
-@Override
-public void teardown() {
-}
   }
 
   @Test

-- 
To stop receiving notification emails like this one, please contact
xumingm...@apache.org.

[beam] branch master updated (2b70d60 -> dbaf098)

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 2b70d60  Merge pull request #4731: [BEAM-3732] Fix broken maven 
profiles
 new b0991a4  add setup/teardown for BeamSqlSeekableTable.
 new dbaf098  rename method as suggested and declare as default methods.

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java | 10 ++
 .../extensions/sql/impl/transform/BeamJoinTransforms.java| 12 +++-
 2 files changed, 21 insertions(+), 1 deletion(-)

-- 
To stop receiving notification emails like this one, please contact
xumingm...@apache.org.

[beam] branch master updated (c8585fa -> 35570f1)

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from c8585fa  This closes #4198
 add 19e8159  convert JOIN as LOOKUP
 add c84e3f2  add BeamSqlSeekableTable to SurfaceTest
 new 35570f1  Merge pull request #4196 from XuMingmin/BEAM-3171

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../sdk/extensions/sql/BeamSqlSeekableTable.java}  | 18 ++---
 .../sdk/extensions/sql/impl/rel/BeamJoinRel.java   | 45 +++-
 .../sql/impl/transform/BeamJoinTransforms.java | 81 ++
 .../sdk/extensions/sql/BeamSqlApiSurfaceTest.java  |  1 +
 .../rel/BeamJoinRelUnboundedVsBoundedTest.java | 63 +
 5 files changed, 197 insertions(+), 11 deletions(-)
 copy sdks/java/{core/src/main/java/org/apache/beam/sdk/metrics/Metric.java => 
extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamSqlSeekableTable.java}
 (66%)

-- 
To stop receiving notification emails like this one, please contact
['"commits@beam.apache.org" <commits@beam.apache.org>'].

[beam] 01/01: Merge pull request #4196 from XuMingmin/BEAM-3171

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 35570f1accfd2e9ef04421c9f02b6cdfcb7c95f4
Merge: c8585fa c84e3f2
Author: James Xu <xumingmi...@gmail.com>
AuthorDate: Sat Dec 2 16:53:17 2017 +0800

Merge pull request #4196 from XuMingmin/BEAM-3171

[BEAM-3171] convert a join into lookup

 .../sdk/extensions/sql/BeamSqlSeekableTable.java   | 35 ++
 .../sdk/extensions/sql/impl/rel/BeamJoinRel.java   | 45 +++-
 .../sql/impl/transform/BeamJoinTransforms.java | 81 ++
 .../sdk/extensions/sql/BeamSqlApiSurfaceTest.java  |  1 +
 .../rel/BeamJoinRelUnboundedVsBoundedTest.java | 63 +
 5 files changed, 223 insertions(+), 2 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
"commits@beam.apache.org" <commits@beam.apache.org>.

[beam] 01/02: [BEAM-3238][SQL] Move Coders to map in BeamRecordSqlType

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit e2c3de092b0a7a5bbfcb5b7797984d5f4bd10138
Author: Anton Kedin <ke...@google.com>
AuthorDate: Wed Nov 22 16:04:05 2017 -0800

[BEAM-3238][SQL] Move Coders to map in BeamRecordSqlType

Improve readability. All coders are supposed to be thread safe and
currently are backed by the static instances.
---
 .../beam/sdk/extensions/sql/BeamRecordSqlType.java | 111 +
 1 file changed, 47 insertions(+), 64 deletions(-)

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlType.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlType.java
index 982494a..1784ec1 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlType.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlType.java
@@ -17,13 +17,13 @@
  */
 package org.apache.beam.sdk.extensions.sql;
 
+import com.google.common.collect.ImmutableMap;
 import java.math.BigDecimal;
 import java.sql.Types;
 import java.util.ArrayList;
 import java.util.Collections;
 import java.util.Date;
 import java.util.GregorianCalendar;
-import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 import org.apache.beam.sdk.coders.BigDecimalCoder;
@@ -50,26 +50,39 @@ import org.apache.beam.sdk.values.BeamRecordType;
  *
  */
 public class BeamRecordSqlType extends BeamRecordType {
-  private static final Map<Integer, Class> SQL_TYPE_TO_JAVA_CLASS = new 
HashMap<>();
-  static {
-SQL_TYPE_TO_JAVA_CLASS.put(Types.TINYINT, Byte.class);
-SQL_TYPE_TO_JAVA_CLASS.put(Types.SMALLINT, Short.class);
-SQL_TYPE_TO_JAVA_CLASS.put(Types.INTEGER, Integer.class);
-SQL_TYPE_TO_JAVA_CLASS.put(Types.BIGINT, Long.class);
-SQL_TYPE_TO_JAVA_CLASS.put(Types.FLOAT, Float.class);
-SQL_TYPE_TO_JAVA_CLASS.put(Types.DOUBLE, Double.class);
-SQL_TYPE_TO_JAVA_CLASS.put(Types.DECIMAL, BigDecimal.class);
-
-SQL_TYPE_TO_JAVA_CLASS.put(Types.BOOLEAN, Boolean.class);
-
-SQL_TYPE_TO_JAVA_CLASS.put(Types.CHAR, String.class);
-SQL_TYPE_TO_JAVA_CLASS.put(Types.VARCHAR, String.class);
-
-SQL_TYPE_TO_JAVA_CLASS.put(Types.TIME, GregorianCalendar.class);
-
-SQL_TYPE_TO_JAVA_CLASS.put(Types.DATE, Date.class);
-SQL_TYPE_TO_JAVA_CLASS.put(Types.TIMESTAMP, Date.class);
-  }
+  private static final Map<Integer, Class> JAVA_CLASSES = ImmutableMap
+  .<Integer, Class>builder()
+  .put(Types.TINYINT, Byte.class)
+  .put(Types.SMALLINT, Short.class)
+  .put(Types.INTEGER, Integer.class)
+  .put(Types.BIGINT, Long.class)
+  .put(Types.FLOAT, Float.class)
+  .put(Types.DOUBLE, Double.class)
+  .put(Types.DECIMAL, BigDecimal.class)
+  .put(Types.BOOLEAN, Boolean.class)
+  .put(Types.CHAR, String.class)
+  .put(Types.VARCHAR, String.class)
+  .put(Types.TIME, GregorianCalendar.class)
+  .put(Types.DATE, Date.class)
+  .put(Types.TIMESTAMP, Date.class)
+  .build();
+
+  private static final Map<Integer, Coder> CODERS = ImmutableMap
+  .<Integer, Coder>builder()
+  .put(Types.TINYINT, ByteCoder.of())
+  .put(Types.SMALLINT, ShortCoder.of())
+  .put(Types.INTEGER, BigEndianIntegerCoder.of())
+  .put(Types.BIGINT, BigEndianLongCoder.of())
+  .put(Types.FLOAT, FloatCoder.of())
+  .put(Types.DOUBLE, DoubleCoder.of())
+  .put(Types.DECIMAL, BigDecimalCoder.of())
+  .put(Types.BOOLEAN, BooleanCoder.of())
+  .put(Types.CHAR, StringUtf8Coder.of())
+  .put(Types.VARCHAR, StringUtf8Coder.of())
+  .put(Types.TIME, TimeCoder.of())
+  .put(Types.DATE, DateCoder.of())
+  .put(Types.TIMESTAMP, DateCoder.of())
+  .build();
 
   public List fieldTypes;
 
@@ -84,54 +97,24 @@ public class BeamRecordSqlType extends BeamRecordType {
   }
 
   public static BeamRecordSqlType create(List fieldNames,
-  List fieldTypes) {
+ List fieldTypes) {
 if (fieldNames.size() != fieldTypes.size()) {
   throw new IllegalStateException("the sizes of 'dataType' and 
'fieldTypes' must match.");
 }
+
 List fieldCoders = new ArrayList<>(fieldTypes.size());
+
 for (int idx = 0; idx < fieldTypes.size(); ++idx) {
-  switch (fieldTypes.get(idx)) {
-  case Types.INTEGER:
-fieldCoders.add(BigEndianIntegerCoder.of());
-break;
-  case Types.SMALLINT:
-fieldCoders.add(ShortCoder.of());
-break;
-  case Types.TINYINT:
-fieldCoders.add(ByteCoder.of());
-break;
-  case Types.DOUBLE:
-fieldCoders.add(DoubleCoder.of());
-break;
-  case Types.FLOAT:
-fieldCoders.add(FloatCoder.of());
- 

[beam] branch master updated (27f505e -> 2eb7de0)

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 27f505e  Merge pull request #4095 from rniemo-g/master
 new e2c3de0  [BEAM-3238][SQL] Move Coders to map in BeamRecordSqlType
 new 2eb7de0  [BEAM-3238][SQL] Add BeamRecordSqlTypeBuilder

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../beam/sdk/extensions/sql/BeamRecordSqlType.java | 190 ++---
 .../sdk/extensions/sql/BeamRecordSqlTypeTest.java  | 115 +
 ...BeamSqlBuiltinFunctionsIntegrationTestBase.java |  56 +++---
 3 files changed, 273 insertions(+), 88 deletions(-)
 create mode 100644 
sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlTypeTest.java

-- 
To stop receiving notification emails like this one, please contact
['"commits@beam.apache.org" <commits@beam.apache.org>'].

[beam] 02/02: [BEAM-3238][SQL] Add BeamRecordSqlTypeBuilder

[xumingming]

This is an automated email from the ASF dual-hosted git repository.

xumingming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 2eb7de0fe6e96da9805fc827294da1e1329ff716
Author: Anton Kedin <ke...@google.com>
AuthorDate: Wed Nov 22 19:17:21 2017 -0800

[BEAM-3238][SQL] Add BeamRecordSqlTypeBuilder

To improve readability of creating BeamRecordSqlTypes.
---
 .../beam/sdk/extensions/sql/BeamRecordSqlType.java |  85 ++-
 .../sdk/extensions/sql/BeamRecordSqlTypeTest.java  | 115 +
 ...BeamSqlBuiltinFunctionsIntegrationTestBase.java |  56 +-
 3 files changed, 229 insertions(+), 27 deletions(-)

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlType.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlType.java
index 1784ec1..a6b23b6 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlType.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlType.java
@@ -17,6 +17,7 @@
  */
 package org.apache.beam.sdk.extensions.sql;
 
+import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
 import java.math.BigDecimal;
 import java.sql.Types;
@@ -108,8 +109,8 @@ public class BeamRecordSqlType extends BeamRecordType {
   Integer fieldType = fieldTypes.get(idx);
 
   if (!CODERS.containsKey(fieldType)) {
-  throw new UnsupportedOperationException(
-  "Data type: " + fieldType + " not supported yet!");
+throw new UnsupportedOperationException(
+"Data type: " + fieldType + " not supported yet!");
   }
 
   fieldCoders.add(CODERS.get(fieldType));
@@ -166,4 +167,84 @@ public class BeamRecordSqlType extends BeamRecordType {
 return "BeamRecordSqlType [fieldNames=" + getFieldNames()
 + ", fieldTypes=" + fieldTypes + "]";
   }
+
+  public static Builder builder() {
+return new Builder();
+  }
+
+  /**
+   * Builder class to construct {@link BeamRecordSqlType}.
+   */
+  public static class Builder {
+
+private ImmutableList.Builder fieldNames;
+private ImmutableList.Builder fieldTypes;
+
+public Builder withField(String fieldName, Integer fieldType) {
+  fieldNames.add(fieldName);
+  fieldTypes.add(fieldType);
+  return this;
+}
+
+public Builder withTinyIntField(String fieldName) {
+  return withField(fieldName, Types.TINYINT);
+}
+
+public Builder withSmallIntField(String fieldName) {
+  return withField(fieldName, Types.SMALLINT);
+}
+
+public Builder withIntegerField(String fieldName) {
+  return withField(fieldName, Types.INTEGER);
+}
+
+public Builder withBigIntField(String fieldName) {
+  return withField(fieldName, Types.BIGINT);
+}
+
+public Builder withFloatField(String fieldName) {
+  return withField(fieldName, Types.FLOAT);
+}
+
+public Builder withDoubleField(String fieldName) {
+  return withField(fieldName, Types.DOUBLE);
+}
+
+public Builder withDecimalField(String fieldName) {
+  return withField(fieldName, Types.DECIMAL);
+}
+
+public Builder withBooleanField(String fieldName) {
+  return withField(fieldName, Types.BOOLEAN);
+}
+
+public Builder withCharField(String fieldName) {
+  return withField(fieldName, Types.CHAR);
+}
+
+public Builder withVarcharField(String fieldName) {
+  return withField(fieldName, Types.VARCHAR);
+}
+
+public Builder withTimeField(String fieldName) {
+  return withField(fieldName, Types.TIME);
+}
+
+public Builder withDateField(String fieldName) {
+  return withField(fieldName, Types.DATE);
+}
+
+public Builder withTimestampField(String fieldName) {
+  return withField(fieldName, Types.TIMESTAMP);
+}
+
+private Builder() {
+  this.fieldNames = ImmutableList.builder();
+  this.fieldTypes = ImmutableList.builder();
+}
+
+public BeamRecordSqlType build() {
+  return create(fieldNames.build(), fieldTypes.build());
+}
+  }
 }
diff --git 
a/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlTypeTest.java
 
b/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlTypeTest.java
new file mode 100644
index 000..78ff221
--- /dev/null
+++ 
b/sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamRecordSqlTypeTest.java
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Ver

[2/2] beam git commit: This closes #4115

[xumingming]

This closes #4115


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2b4a6b5d
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/2b4a6b5d
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/2b4a6b5d

Branch: refs/heads/master
Commit: 2b4a6b5d5cb455fe73971fde4caa7d1446cd55aa
Parents: acfab89 92b3a9a
Author: James Xu 
Authored: Mon Nov 13 14:42:08 2017 +0800
Committer: James Xu 
Committed: Mon Nov 13 14:42:08 2017 +0800

--
 .../sql/meta/provider/kafka/BeamKafkaTable.java | 39 +++-
 1 file changed, 30 insertions(+), 9 deletions(-)
--

[1/2] beam git commit: support topicPartition in BeamKafkaTable

[xumingming]

Repository: beam
Updated Branches:
  refs/heads/master acfab8965 -> 2b4a6b5d5


support topicPartition in BeamKafkaTable


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/92b3a9ac
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/92b3a9ac
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/92b3a9ac

Branch: refs/heads/master
Commit: 92b3a9acf04450313c3c2340f6921bf433c2dc04
Parents: acfab89
Author: mingmxu 
Authored: Fri Nov 10 19:51:37 2017 -0800
Committer: James Xu 
Committed: Mon Nov 13 14:41:29 2017 +0800

--
 .../sql/meta/provider/kafka/BeamKafkaTable.java | 39 +++-
 1 file changed, 30 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/beam/blob/92b3a9ac/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/kafka/BeamKafkaTable.java
--
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/kafka/BeamKafkaTable.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/kafka/BeamKafkaTable.java
index 50f7496..8f663a3 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/kafka/BeamKafkaTable.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/kafka/BeamKafkaTable.java
@@ -34,6 +34,7 @@ import org.apache.beam.sdk.values.KV;
 import org.apache.beam.sdk.values.PBegin;
 import org.apache.beam.sdk.values.PCollection;
 import org.apache.beam.sdk.values.PDone;
+import org.apache.kafka.common.TopicPartition;
 import org.apache.kafka.common.serialization.ByteArrayDeserializer;
 import org.apache.kafka.common.serialization.ByteArraySerializer;
 
@@ -45,6 +46,7 @@ import 
org.apache.kafka.common.serialization.ByteArraySerializer;
 public abstract class BeamKafkaTable extends BaseBeamTable implements 
Serializable {
   private String bootstrapServers;
   private List topics;
+  private List topicPartitions;
   private Map configUpdates;
 
   protected BeamKafkaTable(BeamRecordSqlType beamSqlRowType) {
@@ -58,6 +60,13 @@ public abstract class BeamKafkaTable extends BaseBeamTable 
implements Serializab
 this.topics = topics;
   }
 
+  public BeamKafkaTable(BeamRecordSqlType beamSqlRowType,
+  List topicPartitions, String bootstrapServers) {
+super(beamSqlRowType);
+this.bootstrapServers = bootstrapServers;
+this.topicPartitions = topicPartitions;
+  }
+
   public BeamKafkaTable updateConsumerProperties(Map 
configUpdates) {
 this.configUpdates = configUpdates;
 return this;
@@ -76,15 +85,27 @@ public abstract class BeamKafkaTable extends BaseBeamTable 
implements Serializab
 
   @Override
   public PCollection buildIOReader(Pipeline pipeline) {
-return PBegin.in(pipeline).apply("read",
-KafkaIO.read()
-.withBootstrapServers(bootstrapServers)
-.withTopics(topics)
-.updateConsumerProperties(configUpdates)
-.withKeyDeserializerAndCoder(ByteArrayDeserializer.class, 
ByteArrayCoder.of())
-.withValueDeserializerAndCoder(ByteArrayDeserializer.class, 
ByteArrayCoder.of())
-.withoutMetadata())
-.apply("in_format", getPTransformForInput());
+KafkaIO.Read kafkaRead = null;
+if (topics != null) {
+  kafkaRead = KafkaIO.read()
+  .withBootstrapServers(bootstrapServers)
+  .withTopics(topics)
+  .updateConsumerProperties(configUpdates)
+  .withKeyDeserializerAndCoder(ByteArrayDeserializer.class, 
ByteArrayCoder.of())
+  .withValueDeserializerAndCoder(ByteArrayDeserializer.class, 
ByteArrayCoder.of());
+} else if (topicPartitions != null) {
+  kafkaRead = KafkaIO.read()
+  .withBootstrapServers(bootstrapServers)
+  .withTopicPartitions(topicPartitions)
+  .updateConsumerProperties(configUpdates)
+  .withKeyDeserializerAndCoder(ByteArrayDeserializer.class, 
ByteArrayCoder.of())
+  .withValueDeserializerAndCoder(ByteArrayDeserializer.class, 
ByteArrayCoder.of());
+} else {
+  throw new IllegalArgumentException("One of topics and topicPartitions 
must be configurated.");
+}
+
+return PBegin.in(pipeline).apply("read", kafkaRead.withoutMetadata())
+.apply("in_format", getPTransformForInput());
   }
 
   @Override

[2/2] beam git commit: This closes #4101

[xumingming]

This closes #4101


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/4451d556
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/4451d556
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/4451d556

Branch: refs/heads/master
Commit: 4451d5566d6ffd73d2fd6f946a11ffd1eebd90c6
Parents: 867d816 4a5741f
Author: James Xu 
Authored: Thu Nov 9 13:43:09 2017 +0800
Committer: James Xu 
Committed: Thu Nov 9 13:43:09 2017 +0800

--
 .../beam/sdk/extensions/sql/impl/rel/BeamAggregationRel.java  | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--

[1/2] beam git commit: change `withAllowedTimestampSkew`

[xumingming]

Repository: beam
Updated Branches:
  refs/heads/master 867d81684 -> 4451d5566


change `withAllowedTimestampSkew`


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/4a5741f2
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/4a5741f2
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/4a5741f2

Branch: refs/heads/master
Commit: 4a5741f2f1aabac098e41cfe453b52876a1bcdb0
Parents: 867d816
Author: mingmxu 
Authored: Wed Nov 8 13:44:18 2017 -0800
Committer: James Xu 
Committed: Thu Nov 9 13:42:04 2017 +0800

--
 .../beam/sdk/extensions/sql/impl/rel/BeamAggregationRel.java  | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/beam/blob/4a5741f2/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamAggregationRel.java
--
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamAggregationRel.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamAggregationRel.java
index e49e79c..6ed30b6 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamAggregationRel.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamAggregationRel.java
@@ -80,7 +80,8 @@ public class BeamAggregationRel extends Aggregate implements 
BeamRelNode {
 
BeamSqlRelUtils.getBeamRelInput(input).buildBeamPipeline(inputPCollections, 
sqlEnv);
 if (windowFieldIdx != -1) {
   upstream = upstream.apply(stageName + "assignEventTimestamp", 
WithTimestamps
-  .of(new BeamAggregationTransforms.WindowTimestampFn(windowFieldIdx)))
+  .of(new BeamAggregationTransforms.WindowTimestampFn(windowFieldIdx))
+  .withAllowedTimestampSkew(new Duration(Long.MAX_VALUE)))
   .setCoder(upstream.getCoder());
 }
 

[1/2] beam git commit: [BEAM-2203] Implement 'timestamp - interval'

[xumingming]

Repository: beam
Updated Branches:
  refs/heads/master 299429394 -> e0166ceb6


[BEAM-2203] Implement 'timestamp - interval'

Move TIMESTAMPDIFF implementation into 
BeamSqlTimestampMinusTimestampExpression, add 
BeamSqlTimestampMinusIntervalExpression


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/0a63eb21
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/0a63eb21
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/0a63eb21

Branch: refs/heads/master
Commit: 0a63eb2195575f6cc5ee228d4edb1ed9571f9f5f
Parents: 29942939
Author: Anton Kedin 
Authored: Sun Nov 5 23:26:10 2017 -0800
Committer: James Xu 
Committed: Wed Nov 8 11:58:34 2017 +0800

--
 .../date/BeamSqlDatetimeMinusExpression.java| 104 -
 ...BeamSqlTimestampMinusIntervalExpression.java |  83 +++
 ...eamSqlTimestampMinusTimestampExpression.java |  97 
 .../BeamSqlDatetimeMinusExpressionTest.java | 189 ---
 ...SqlTimestampMinusIntervalExpressionTest.java | 163 +
 ...qlTimestampMinusTimestampExpressionTest.java | 233 +++
 .../BeamSqlDateFunctionsIntegrationTest.java|  18 ++
 7 files changed, 691 insertions(+), 196 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/beam/blob/0a63eb21/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/date/BeamSqlDatetimeMinusExpression.java
--
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/date/BeamSqlDatetimeMinusExpression.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/date/BeamSqlDatetimeMinusExpression.java
index 6a5cbb1..6948ba1 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/date/BeamSqlDatetimeMinusExpression.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/operator/date/BeamSqlDatetimeMinusExpression.java
@@ -28,80 +28,80 @@ import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlPrimi
 import org.apache.beam.sdk.transforms.windowing.BoundedWindow;
 import org.apache.beam.sdk.values.BeamRecord;
 import org.apache.calcite.sql.type.SqlTypeName;
-import org.joda.time.DateTime;
 import org.joda.time.DurationFieldType;
-import org.joda.time.Period;
-import org.joda.time.PeriodType;
 
 /**
  * Infix '-' operation for timestamps.
  *
- * Currently this implementation is specific to how Calcite implements 
'TIMESTAMPDIFF(..)'.
- * It converts the TIMESTAMPDIFF() call into infix minus and normalizes it
- * with corresponding TimeUnit's multiplier.
+ * Implements 2 SQL subtraction operations at the moment:
+ * 'timestampdiff(timeunit, timestamp, timestamp)', and 'timestamp - interval'
  *
- * In addition to this TIMESTAMPDIFF(..) implementation, Calcite also 
supports infix
- * operations 'interval - interval' and 'timestamp - interval'.
- * These are not implemented yet.
+ * Calcite converts both of the above into infix '-' expression, with 
different operands and
+ * return types.
+ *
+ * This class delegates evaluation to specific implementation of one of the 
above operations,
+ * see {@link BeamSqlTimestampMinusTimestampExpression}
+ * and {@link BeamSqlTimestampMinusIntervalExpression}
+ *
+ * Calcite supports one more subtraction kind: 'interval - interval',
+ * but it is not implemented yet.
  */
 public class BeamSqlDatetimeMinusExpression extends BeamSqlExpression {
-  private SqlTypeName intervalType;
 
-  private static final Map 
INTERVALS_DURATIONS_TYPES =
+  static final Map INTERVALS_DURATIONS_TYPES =
   ImmutableMap.builder()
-  .put(SqlTypeName.INTERVAL_SECOND, DurationFieldType.seconds())
-  .put(SqlTypeName.INTERVAL_MINUTE, DurationFieldType.minutes())
-  .put(SqlTypeName.INTERVAL_HOUR, DurationFieldType.hours())
-  .put(SqlTypeName.INTERVAL_DAY, DurationFieldType.days())
-  .put(SqlTypeName.INTERVAL_MONTH, DurationFieldType.months())
-  .put(SqlTypeName.INTERVAL_YEAR, DurationFieldType.years())
-  .build();
-
-  public BeamSqlDatetimeMinusExpression(
-  List operands, SqlTypeName intervalType) {
-super(operands, SqlTypeName.BIGINT);
-this.intervalType = intervalType;
+  .put(SqlTypeName.INTERVAL_SECOND, DurationFieldType.seconds())
+  .put(SqlTypeName.INTERVAL_MINUTE, DurationFieldType.minutes())
+  .put(SqlTypeName.INTERVAL_HOUR, DurationFieldType.hours())
+  .put(SqlTypeName.INTERVAL_DAY, DurationFieldType.days())
+ 

[2/2] beam git commit: This closes #4082

[xumingming]

This closes #4082


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e0166ceb
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/e0166ceb
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/e0166ceb

Branch: refs/heads/master
Commit: e0166ceb657b393e4037ac7c178797a864379745
Parents: 29942939 0a63eb2
Author: James Xu 
Authored: Wed Nov 8 11:59:07 2017 +0800
Committer: James Xu 
Committed: Wed Nov 8 11:59:07 2017 +0800

--
 .../date/BeamSqlDatetimeMinusExpression.java| 104 -
 ...BeamSqlTimestampMinusIntervalExpression.java |  83 +++
 ...eamSqlTimestampMinusTimestampExpression.java |  97 
 .../BeamSqlDatetimeMinusExpressionTest.java | 189 ---
 ...SqlTimestampMinusIntervalExpressionTest.java | 163 +
 ...qlTimestampMinusTimestampExpressionTest.java | 233 +++
 .../BeamSqlDateFunctionsIntegrationTest.java|  18 ++
 7 files changed, 691 insertions(+), 196 deletions(-)
--

[1/2] beam git commit: [BEAM-2203] Implement TIMESTAMPDIFF()

[xumingming]

Repository: beam
Updated Branches:
  refs/heads/master 269bf8946 -> f80bf2fdf


[BEAM-2203] Implement TIMESTAMPDIFF()


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/4fe7480a
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/4fe7480a
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/4fe7480a

Branch: refs/heads/master
Commit: 4fe7480a749699e0d1418dbe0b29e6ecefd08497
Parents: 269bf89
Author: Anton Kedin 
Authored: Mon Oct 30 17:36:21 2017 -0700
Committer: James Xu 
Committed: Tue Nov 7 23:09:20 2017 +0800

--
 sdks/java/extensions/sql/pom.xml|   8 +
 .../sql/impl/interpreter/BeamSqlFnExecutor.java |   9 +-
 .../operator/BeamSqlReinterpretExpression.java  |  55 -
 .../date/BeamSqlDatetimeMinusExpression.java| 107 +
 .../BeamSqlReinterpretExpression.java   |  58 +
 .../DatetimeReinterpretConversions.java |  59 +
 .../IntegerReinterpretConversions.java  |  44 
 .../reinterpret/ReinterpretConversion.java  | 114 +
 .../operator/reinterpret/Reinterpreter.java | 101 
 .../operator/reinterpret/package-info.java  |  22 ++
 .../impl/interpreter/BeamSqlFnExecutorTest.java |  14 ++
 .../BeamSqlReinterpretExpressionTest.java   | 116 +++---
 .../BeamSqlDatetimeMinusExpressionTest.java | 232 +++
 .../DatetimeReinterpretConversionsTest.java |  73 ++
 .../IntegerReinterpretConversionsTest.java  |  81 +++
 .../reinterpret/ReinterpretConversionTest.java  | 116 ++
 .../operator/reinterpret/ReinterpreterTest.java | 155 +
 .../BeamSqlDateFunctionsIntegrationTest.java|  52 +
 18 files changed, 1330 insertions(+), 86 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/beam/blob/4fe7480a/sdks/java/extensions/sql/pom.xml
--
diff --git a/sdks/java/extensions/sql/pom.xml b/sdks/java/extensions/sql/pom.xml
index 742d3b6..34ff670 100644
--- a/sdks/java/extensions/sql/pom.xml
+++ b/sdks/java/extensions/sql/pom.xml
@@ -37,6 +37,7 @@
 -MM-dd 
HH:mm
 1.13.0
 1.10.0
+1.9.5
   
 
   
@@ -271,5 +272,12 @@
   hamcrest-all
   test
 
+
+
+  org.mockito
+  mockito-all
+  ${mockito.version}
+  test
+
   
 

http://git-wip-us.apache.org/repos/asf/beam/blob/4fe7480a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
--
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
index 8770055..31d5022 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/interpreter/BeamSqlFnExecutor.java
@@ -27,7 +27,6 @@ import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlCastE
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlInputRefExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlPrimitive;
-import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlReinterpretExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlUdfExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlWindowEndExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.BeamSqlWindowExpression;
@@ -50,6 +49,7 @@ import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.date.BeamSql
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.date.BeamSqlCurrentTimestampExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.date.BeamSqlDateCeilExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.date.BeamSqlDateFloorExpression;
+import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.date.BeamSqlDatetimeMinusExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.date.BeamSqlDatetimePlusExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.date.BeamSqlExtractExpression;
 import 
org.apache.beam.sdk.extensions.sql.impl.interpreter.operator.date.BeamSqlIntervalMultiplyExpression;
@@ -79,6 +79,7 @@ import 

[2/2] beam git commit: This closes #4065

[xumingming]

This closes #4065


Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/f80bf2fd
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/f80bf2fd
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/f80bf2fd

Branch: refs/heads/master
Commit: f80bf2fdff655b62deb458b1e57d867bfbf03f41
Parents: 269bf89 4fe7480
Author: James Xu 
Authored: Tue Nov 7 23:11:03 2017 +0800
Committer: James Xu 
Committed: Tue Nov 7 23:11:03 2017 +0800

--
 sdks/java/extensions/sql/pom.xml|   8 +
 .../sql/impl/interpreter/BeamSqlFnExecutor.java |   9 +-
 .../operator/BeamSqlReinterpretExpression.java  |  55 -
 .../date/BeamSqlDatetimeMinusExpression.java| 107 +
 .../BeamSqlReinterpretExpression.java   |  58 +
 .../DatetimeReinterpretConversions.java |  59 +
 .../IntegerReinterpretConversions.java  |  44 
 .../reinterpret/ReinterpretConversion.java  | 114 +
 .../operator/reinterpret/Reinterpreter.java | 101 
 .../operator/reinterpret/package-info.java  |  22 ++
 .../impl/interpreter/BeamSqlFnExecutorTest.java |  14 ++
 .../BeamSqlReinterpretExpressionTest.java   | 116 +++---
 .../BeamSqlDatetimeMinusExpressionTest.java | 232 +++
 .../DatetimeReinterpretConversionsTest.java |  73 ++
 .../IntegerReinterpretConversionsTest.java  |  81 +++
 .../reinterpret/ReinterpretConversionTest.java  | 116 ++
 .../operator/reinterpret/ReinterpreterTest.java | 155 +
 .../BeamSqlDateFunctionsIntegrationTest.java|  52 +
 18 files changed, 1330 insertions(+), 86 deletions(-)
--

[GitHub] beam pull request #4041: [BEAM-2528] BeamSQL DDL :: CreateTable

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/4041

[BEAM-2528] BeamSQL DDL :: CreateTable

I started this PR as an initial attempt to implement the BeamSQL `create 
table` statement.  The implementation might not be so mature, but I hope this 
could be a place we can discuss deeper about the create table. I will introduce 
this PR in the following 3 aspects:

* MetaStore
* TableProvider
* Grammar

## MetaStore

Metastore is responsible for handling the CRUD of table during a session. 
e.g. create a table, query all tables, query a table by the specified name etc. 
When a table is created, the table meta info can be persisted by the metastore, 
but the default `InMemoryMetaStore` will only store the meta info in memory, so 
it will NOT be persisted, but user can implement the `MetaStore` interface to 
make a persistent implementation.

The table names in MetaStore need to be unique.

## TableProvider

The tables in MetaStore can come from many different sources, the 
construction of a usable table is the responsibility of a `TableProvider`, 
TableProvider have the similar interface like `MetaStore`, but it only handles 
a specific type of table, e.g. `TextTableProvider` only handle text tables, 
while `KafakaTableProvider` only handle kafka tables.

In this PR, only `TextTableProvider`  and  `KafakaTableProvider` are 
implemented as example.

## Grammar

The grammar for create a TEXT table is:

```sql
CREATE TABLE ORDERS(
   ID INT PRIMARY KEY COMMENT 'this is the primary key',
   NAME VARCHAR(127) COMMENT 'this is the name'
)
COMMENT 'this is the table orders'
LOCATION 'text://home/admin/orders'
TBLPROPERTIES '{"format": "Excel"}'
```

`LOCATION` dictates where the data of the table is stored. The scheme of 
the LOCATION dictate the table type, e.g. in the above example, the table type 
is `text`, using the table type we can find the corresponding 
`TextTableProvider` using the ServiceLoader merchanism.

`TBLPROPERTIES` is used to specify some other properties of the table, in 
the above example, we specified the format of each line of text file: 
`Excel`(one variant of CSV format).

The grammar for create a KAFKA table is:

```sql
 CREATE TABLE ORDERS(
   ID INT PRIMARY KEY COMMENT 'this is the primary key',
   NAME VARCHAR(127) COMMENT 'this is the name'
 )
 COMMENT 'this is the table orders'
 LOCATION 'kafka://localhost:2181/brokers?topic=test'
 TBLPROPERTIES '{"bootstrap.servers":"localhost:9092", "topics": ["topic1", 
"topic2"]}'
```

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2528-create-table-from-master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/4041.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4041


commit ab5ee759116c7e8139d7d17aca631f018a48fb40
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-09-13T12:36:37Z

[BEAM-2528] create table

commit 813b9a7a5e7232d3028bc6b859d96c0d856c1517
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-10-26T12:17:09Z

minor




---

[GitHub] beam pull request #3481: [BEAM-2528] BeamSql: DDL: create table

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3481


---

[GitHub] beam pull request #3708: [BEAM-2740] introduce the new InnerBeamSqlEnv/BeamS...

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3708


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3702: [BEAM-2744] rename BeamRecordType#size()

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3702


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3700: [BEAM-2745] add BeamRecordSqlType.getFieldTypeByInd...

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3700


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3708: [BEAM-2740] introduce the new InnerBeamSqlEnv/BeamS...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3708

[BEAM-2740] introduce the new InnerBeamSqlEnv/BeamSqlEnv

Summary:

* Simplified `BeamSqlEnv`: now you can only register UDF/UDAF in 
`BeamSqlEnv`, and `BeamSqlEnv` just record the udfName -> udfClass mapping, 
actually work will be done by `InnerBeamSqlEnv`.
* `BeamSqlEnv` is only used by the surface API classes.
* `InnerBeamSqlEnv` is actually the old `BeamSqlEnv`, it holds schema and 
planner.
  * `InnerBeamSqlEnv` is only used by the implementation classes.
* `BeamSqlEnv` is converted to `InnerBeamSqlEnv` automatically in `BeamSql` 
and `BeamSqlCli`.
* `BeamSqlEnv` and `InnerBeamSqlEnv` are quite different, so can not share 
the same interface.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2740

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3708.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3708


commit 80491a26456e1a4a1ffc0e4e00290de63b9617e4
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-08-09T10:54:54Z

[BEAM-2740] introduce InnerBeamSqlEnv to extract implementation related 
logic from BeamSqlEnv

commit 5074f830347e07ef0ba8396bc9a464a37eb089ec
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-08-09T11:04:46Z

add licence declaration for BeamRelTest




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3699: [BEAM-2742] change Field type from primitives to bo...

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3699


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3704: [BEAM-2733] update javadoc for BeamSql

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3704


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3704: [BEAM-2733] update javadoc for BeamSql

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3704

[BEAM-2733] update javadoc for BeamSql

Summary:

* `Exception` declaration are removed from `BeamSql.query ()`
* Added javadoc to explain when the sql query use a non-exist table、a 
subset of tables from upstream, what happens.
* Added javadoc for BeamSql.query and QueryTransform to describe the 
lifetime of tables in PCollectionTuple.
* SimpleQueryTransform and QueryTransform are actually user-facing (they 
are return types of query() and simpleQuery()), so they are kept `public`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2733-BeamSql

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3704.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3704


commit df4d24f2104b61b65bbfc027d005415d4282c992
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-08-08T11:52:53Z

[BEAM-2733] update javadoc for BeamSql




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3702: [BEAM-2744] rename BeamRecordType#size()

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3702

[BEAM-2744] rename BeamRecordType#size()

Summary:

* BeamRecord.size() -> BeamRecord.getFieldCount()
* BeamRecordType.size() -> BeamRecordType.getFieldCount()
* remove BeamRecord.valueInString()

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2744-rename-BeamRecordType-size

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3702.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3702


commit 6466a9fcfa0dcc4c73d05a98ffd8727584d190b4
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-08-08T07:15:59Z

[BEAM-2744] rename BeamRecordType#size()

commit ff10e1a5a64e9b7c607b965a9b5bc318465f1995
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-08-08T07:27:48Z

remove BeamRecord#valueInString()




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3700: [BEAM-2745] add BeamRecordSqlType.getFieldTypeByInd...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3700

[BEAM-2745] add BeamRecordSqlType.getFieldTypeByIndex()

Summary:

* BeamSqlRecordType -> BeamRecordSqlType
* fieldsName -> fieldNames
* fieldsType -> fieldTypes
* add getFieldTypeByIndex()

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2745-add-getFieldTypeByIndex

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3700.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3700


commit cec323b1ddde3185082794dd420904eb89374bdb
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-08-08T06:53:18Z

[BEAM-2745] add BeamRecordSqlType.getFieldTypeByIndex()




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3699: [BEAM-2742] change Field type from primitives to bo...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3699

[BEAM-2742] change Field type from primitives to boxed types

change Field type of `BeamRecord` from primitive types to boxed types, and 
removed `isNull` method.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2742-switch-to-boxed-types

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3699.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3699


commit 189959a65f5fc1f7835dcd2a99a76c7717860e07
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-08-08T06:26:09Z

[BEAM-2742] change Field type from primitives to boxed types




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3666: [BEAM-2441] move dsls/sql to sdks/java/extensions/s...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3666

[BEAM-2441] move dsls/sql to sdks/java/extensions/sql

R: @XuMingmin @takidau 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2441

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3666.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3666


commit ba493f85a5a18665cd6ae4eb61e0a86fab1a6c07
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-30T15:39:37Z

move dsls/sql to sdks/java/extensions/sql

commit c1b5482d3fb13a14926f0ffc23d0810b3105ed24
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-30T15:58:02Z

rename package org.apache.beam.dsls.sql to 
org.apache.beam.sdk.extensions.sql

commit febd044ae306a28fa3797a1663e54c1d7fbe43ce
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-31T09:11:53Z

move all implementation classes/packages into impl package




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3583: [BEAM-2613] add integration test for comparison ope...

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3583


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3571: [BEAM-2621] BeamSqlRecordType -> BeamSqlRowType

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3571


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3570: [BEAM-2565] add integration test for conditional fu...

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3570


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3547: [BEAM-2561] add integration test for date functions

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3547


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3583: [BEAM-2613] add integration test for comparison ope...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3583

[BEAM-2613] add integration test for comparison operators

Summary:

1. added integration test for comparison operators.
2. refactored comparison operator expressions into a separate package: 
`comparison`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2613-integration-test-for-comparison-operators

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3583.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3583


commit f5044071c0ff0e909f2bd24ab3a8ea5921161c4b
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-13T10:24:55Z

[BEAM-2613] add integration test for comparison operators




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3571: [BEAM-2621] BeamSqlRecordType -> BeamSqlRowType

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3571

[BEAM-2621] BeamSqlRecordType -> BeamSqlRowType

@XuMingmin @jbonofre @takidau 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2621-rename-BeamSqlRecordType

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3571.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3571


commit 54b939445839a4b2b81d01cb50f75d76ee0fc175
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-17T09:55:56Z

BeamSqlRecordType -> BeamSqlRowType




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3570: [BEAM-2565] add integration test for conditional fu...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3570

[BEAM-2565] add integration test for conditional functions

@XuMingmin @jbonofre @takidau 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2565-integration-test-for-case-operator

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3570.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3570


commit 04f50114960266ec194a75b5d2c4ded53259367f
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-17T09:34:05Z

[BEAM-2565] add integration test for conditional functions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3557: [BEAM-2560] Add integration test for arithmetic ope...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3557

[BEAM-2560] Add integration test for arithmetic operators.

Re-implemented the arithmetic operators & refactored string functions 
integration test to utilize `ExpressionChecker`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2560-integration-test-for-arithmetic

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3557.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3557


commit 715615438ac72b2d4bf154e2816a65d2ccdd46ea
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-07T03:04:46Z

[BEAM-2560] Add integration test for arithmetic operators.

And also refactor BeamSqlStringFunctionsIntegrationTest to use 
ExpressionChecker




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3532: [BEAM-2564] add integration test for string functio...

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3532


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3506: [BEAM-2550] add UnitTest for JOIN in DSL

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3506


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3547: [BEAM-2561] add integration test for date functions

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3547

[BEAM-2561] add integration test for date functions

R: @XuMingmin @takidau 

there are some duplications with PR #3532 , will be some refactor after 
#3532 is merged.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2561-integration-test-for-date-functions

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3547.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3547


commit 3a0b1f57ddca7a08c9ee7ee3a6ea0a760b9a0dc0
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-11T09:49:04Z

[BEAM-2561] add integration test for date functions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3532: [BEAM-2564] add integration test for string functio...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3532

[BEAM-2564] add integration test for string functions

Add integration test for string functions and fixed a bug in 
`BeamSqlTrimExpression`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2564-integration-test-for-string-operators

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3532.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3532


commit 709768f98fa1bd4daa3080bfe1aaec31b78f
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-10T11:58:21Z

[BEAM-2564] add integration test for string functions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3506: [BEAM-2550] add UnitTest for JOIN in DSL

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3506

[BEAM-2550] add UnitTest for JOIN in DSL

@XuMingmin @takidau 

I have only tested `bounded vs bounded` join scenario, since we already 
have enumerated every cases like `bounded vs bounded`, `unbounded vs unbounded` 
in tests for `BeamJoinRel`, test here is mainly to test the integration of JOIN 
with the DSL methods.

Also I kinda fixed a related bug in `BeamIOSourceRel` -- which have not 
setCoder for the rows. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2550-test-join-in-dsl

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3506.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3506


commit 5885d1cad12d28c453603e12c8a11f369d1d6d65
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-07-06T03:29:41Z

[BEAM-2550] add UnitTest for JOIN in DSL




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3478: [BEAM-2515] BeamSql: refactor the MockedBeamSqlTabl...

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3478


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3481: [BEAM-2528] BeamSql: DDL: create table

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3481

[BEAM-2528] BeamSql: DDL: create table

I started this PR as an initial attempt to implement the `create table` 
statement.  The implementation might not be so mature, but I hope this could be 
a place we can discuss deeper about the create table. I will introduce this PR 
in the following 3 aspects:

* MetaStore
* TableProvider
* Grammar

## MetaStore

Metastore is responsible for handling the CRUD of table during a session. 
e.g. create a table, query all tables, query a table by the specified name etc. 
When a table is created, the table meta info can be persisted by the metastore, 
but the default `InMemoryMetaStore` will only store the meta info in memory, so 
it will NOT be persisted, but user can implement the `MetaStore` interface to 
make a persistent implementation.

The table names in MetaStore need to be unique.

## TableProvider

The tables in MetaStore can come from many different sources, the 
construction of a usable table is the responsibility of a `TableProvider`, 
TableProvider have the similar interface like `MetaStore`, but it only handles 
a specific type of table, e.g. `TextTableProvider` only handle text tables, 
while `KafakaTableProvider` only handle kafka tables.

## Grammar

The grammar for create table is:

```sql
CREATE TABLE ORDERS(
   ID INT PRIMARY KEY COMMENT 'this is the primary key',
   NAME VARCHAR(127) COMMENT 'this is the name'
)
COMMENT 'this is the table orders'
LOCATION 'text://home/admin/orders'
TBLPROPERTIES '{"format": "Excel"}'
```

`LOCATION` dictates where the data of the table is stored. The scheme of 
the LOCATION dictate the table type, e.g. in the above example, the table type 
is `text`, using the table type we can find the corresponding 
`TextTableProvider` using the ServiceLoader merchanism.

`TBLPROPERTIES` is used to specify some other properties of the table, in 
the above example, we specified the format of each line of text file: 
`Excel`(one variant of CSV format).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2528-create-table

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3481.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3481


commit 9bbb08f5bdabbb903491c9ec0bfc32145632d63b
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-06-17T11:30:03Z

[BEAM-2528] BeamSql: DDL: create table




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3478: [BEAM-2515] BeamSql: refactor the MockedBeamSqlTabl...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3478

[BEAM-2515] BeamSql: refactor the MockedBeamSqlTable and related tests

Summary:

* rename `MockedBeamSqlTable` to `MockedBoundedTable`
* Use the `RowsBuilder` to build rows in Unit Test.
* use `PAssert` rather than `assertEquals` when test PCollections.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2515-refactor-mock

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3478.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3478


commit 1c03838b88b45e2bb32ddf0a8bfe022de68cafc0
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-06-27T02:42:40Z

use PAssert rather than assertEquals

commit 5b48fc288127429355cc727cadbc657cba719320
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-06-27T09:08:39Z

minor

commit 40b8962e2f5a66d279801f8d4604c442634d902a
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-06-30T06:54:26Z

MockedBeamSqlTable -> MockedBoundedTable




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3277: [BEAM-2193] implement join

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3277


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3377: [BEAM-2443] apply AutoValue to BeamSqlRecordType

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3377


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3377: [BEAM-2443] apply AutoValue to BeamSqlRecordType

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3377

[BEAM-2443] apply AutoValue to BeamSqlRecordType



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2443-autovalues

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3377.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3377


commit f6343235e2e769a9cee80b0b844db8b8761ed884
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-06-16T06:31:55Z

[BEAM-2443] apply AutoValue to BeamSqlRecordType




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3357: [BEAM-2442] BeamSql api surface test

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3357


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3365: [BEAM-2444] BeamSql: use java standard exception

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3365

[BEAM-2444] BeamSql: use java standard exception

R: @XuMingmin 
R: @takidau 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2444-use-java-standard-exception

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3365.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3365


commit 83e6c2046571c14d104e622b2e2ad656143e09aa
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-06-15T02:57:10Z

[BEAM-2444] BeamSql: use java standard exception




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3364: [BEAM-2440] BeamSql: reduce visibility

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3364

[BEAM-2440] BeamSql: reduce visibility

Since the packages are cross-import each other, many of them can't be set 
to `package private` access level.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2440-reduce-visibility

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3364.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3364


commit 3625dbd9e00db28d2175c6ac000792fd5c5d8f65
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2016-08-09T09:17:32Z

Add DSLs module

commit 7867ce62e43bef7bfd8011c605379df05494dfcf
Author: mingmxu <ming...@ebay.com>
Date:   2017-04-10T02:49:08Z

[BEAM-301] Initial skeleton for Beam SQL

commit f1c2b6540652156ad7c51ec89a933aacb8c6dab6
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2017-04-12T20:00:53Z

[BEAM-301] This closes #2479

commit 529bc9d9375ae162d70de274975c547a878053a3
Author: mingmxu <ming...@ebay.com>
Date:   2017-04-13T21:46:28Z

checkstyle and rename package

correct package from org.beam.dsls.sql to org.apache.beam.dsls.sql

update with checkstyle

commit ca8760373b37b540219301f1e719633956a787dc
Author: Davor Bonaci <da...@google.com>
Date:   2017-04-18T21:40:40Z

This closes #2530

commit aa07a1d41b71c810c4968bd35e5ca69dc56b8c30
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-04-19T19:14:05Z

This closes #2584

commit 464cc275952305bf51ecfdf056e784441c9c2272
Author: mingmxu <ming...@ebay.com>
Date:   2017-04-23T05:20:25Z

redesign BeamSqlExpression to execute Calcite SQL expression.

Changes:
1. revert BEAM dependency to 0.6.0 to avoid impact of changes in master 
branch;
2. updates as discussion during review;

refine BeamSqlRowCoder

commit a6845a35fc2a4091dcef24f231c6d3a418561e6f
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-04T07:12:55Z

This closes #2656

commit cf95571d90329287b92946065c93724780f31685
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-04T07:16:34Z

This closes #2878

commit dd70852f6fa811a410938e5baa51cb8e602c931b
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-04T03:44:14Z

Fix inconsistent mapping for SQL FLOAT

commit f7ee8d33e13506a17314720c8c23fff486bff76c
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-07T02:24:04Z

This closes #2886

commit a376f37b908828fa0d44b5d71822d51b9c47f971
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-04T14:15:02Z

[BEAM-2158] Implement the arithmetic operators

commit 4ea38d82359ed443c906de9caed8488d1dbf199a
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2017-05-09T08:24:20Z

[BEAM-2158] This closes #2893

commit b12c7b49b8eab7b208f2701c412123635813ee28
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-05T17:20:54Z

[BEAM-2161] Add support for String operators

commit 3e678a75a746c9e40fdbe6599d20969718c34882
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2017-05-09T12:17:53Z

[BEAM-2161] This closes #2919

commit 59abc7640866f58427817c026af15df30947f3a6
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-04T03:44:14Z

[BEAM-2079] Support TextIO as SQL source/sink

commit 7e918a76e409e5c849f2878702746935f4d7bf90
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2017-05-10T05:24:40Z

[BEAM-2079] This closes #2942

commit 3c6d60fb2f473b1860cfeca0f740ec817499d085
Author: mingmxu <ming...@ebay.com>
Date:   2017-05-10T06:04:17Z

[BEAM-2006] window support
Add support for aggregation: global, HOP, TUMBLE, SESSION, only aggregation 
function COUNT

fix typo

commit 6729a027daea04d0ad38f8a83ad1baafa465169f
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-10T17:03:17Z

This closes #3039

commit f728fbe5c7153341ace046fa8b2465ef8844be1b
Author: mingmxu <ming...@ebay.com>
Date:   2017-05-10T23:38:13Z

Support common-used aggregation functions in SQL, including:
  COUNT,SUM,AVG,MAX,MIN

rename BeamAggregationTransform to BeamAggregationTransforms

update comments

commit 523482be0501a7bce79087f47c7752b900178a00
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-13T00:47:06Z

This closes #3067

commit 12dd804678bfaad4705c8f1d50eaf03a086f6daf
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-10T09:19:48Z

[BEAM-2195] Implement conditional operator (CASE)

commit 8bb59840be32ef945cfb7151ab7f8368fabb19f0
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2017-05-13T06:09:08Z

[BEAM-2195] This closes #3042

commit d74a5c34f0732825c052d4c

[GitHub] beam pull request #3218: [BEAM-2356] add NOT operator & some refactoring

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3218


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3357: [BEAM-2442] BeamSql api surface test

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3357

[BEAM-2442] BeamSql api surface test

Summary:

* The surface api of BeamSql includes the following:
  * BeamSql
  * BeamSqlCli
  * BeamSqlEnv
  * All the classes in package `org.apache.beam.dsls.sql.schema`
* Now `BeamSql` and `BeamSqlCli` both extends from `BeamSqlEnv` to avoid 
expose `SchemaPlus` and `BeamQueryPlanner`.
* Calcite realted methods are encapsulated into `CalciteUtils`(which is not 
part of surface api) to avoid exposure.
* Created a new `BeamSqlTable` which abstracts the beam `table` concept.
* `RelDataType`, `RelProtoDataType` are all removed from surface api, 
`BeamSqlRecordType` is the only class which represents the schema of a table.
  * java.sql.Types is used to represent sql type instead of Calcite 
`SqlTypeName`.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2442-api-surface-test

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3357.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3357


commit 507e0451584d20bd7d2fe59a5ee410cc8959f5d8
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-06-14T15:47:10Z

[BEAM-2442] BeamSql surface api test.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3183: [BEAM-2325] support Set operator: intersect, except...

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3183


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3207: [BEAM-2247] implement date functions

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3207


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3254: [BEAM-2375] upgrade beam-version to 2.0.0

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3254


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3277: [BEAM-2193] implement join

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3277

[BEAM-2193] implement join

implement join

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2193-join

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3277.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3277


commit 4313eede78a541b5739f2bdd1588776213dfcf9f
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-29T03:11:34Z

[BEAM-2375] upgrade beam-version to 2.0.0

commit 23750b46f32b3d854f75d7b8a3bdfcb8a44887bd
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-29T13:59:58Z

[BEAM-2193] implement join




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3267: [BEAM-2378] support FULL OUTER JOIN

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3267

[BEAM-2378] support FULL OUTER JOIN

@davorbonaci can you take a look at this please?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2378-full-outer-join

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3267.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3267


commit 86555d68b86f7b62cea8523e1a97aa99b677d7eb
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-31T02:28:55Z

[BEAM-2378] support FULL OUTER JOIN




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3257: fix javadoc of View

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3257

fix javadoc of View

While reading javadoc of `org.apache.beam.sdk.transforms.View`, find the 
sample code of `View` seems not correct, hope the PR will fix it.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam fix-javadoc-of-View

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3257.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3257


commit 4cef2cbc595af1bbd68255d5404fabf8f57c699b
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-30T03:27:32Z

fix javadoc of View




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3254: [BEAM-2375] upgrade beam-version to 2.0.0

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3254

[BEAM-2375] upgrade beam-version to 2.0.0

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam DSL_SQL

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3254.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3254


commit 4313eede78a541b5739f2bdd1588776213dfcf9f
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-29T03:11:34Z

[BEAM-2375] upgrade beam-version to 2.0.0




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3253: rebase master for feature branch DSL_SQL

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3253


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3248: rebasing DSL_SQL branch from master

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3248


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3253: rebase master for feature branch DSL_SQL

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3253

rebase master for feature branch DSL_SQL

Since it is just a rebase, so didn't create a dedicated issue for it. And 
since it is a rebase, it cann't be merged automatically, need to merge manually 
with `--force`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam DSL_SQL_3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3253.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3253


commit 42066431b45ced1931787650d2210be724fb7f5c
Author: Eugene Kirpichov <kirpic...@google.com>
Date:   2017-05-12T17:57:03Z

This closes #3113

commit 6a90b9d164d98d916d0e0af2be39a740ce769076
Author: Aviem Zur <aviem...@gmail.com>
Date:   2017-05-12T13:02:54Z

[BEAM-2277] Fix URI_SCHEME_PATTERN in FileSystems

commit 0acf3620d90a970f4024ae95b909c3bb4ef827ad
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:10:48Z

This closes #3115

commit abe62feb16f6eb59e294c37abf408f84c9280616
Author: Aviem Zur <aviem...@gmail.com>
Date:   2017-05-12T19:18:02Z

[BEAM-2279] Add HDFS support to Spark runner profiles in archetypes and 
examples

commit c2a36284b510461942205b627a019dfbe83f8b37
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-12T21:54:00Z

This closes #3126

commit a6a5ff7be387ef295fc7f921de36a3ea77327bc1
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T16:20:34Z

[BEAM-2277] Add ResourceIdTester and test existing ResourceId 
implementations

A first cut at some of the parts of the ResourceId spec.

commit 15df211c758e7c8f05c3136f25bbe18e3f394321
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:11:06Z

[BEAM-2277] HadoopFileSystem: normalize implementation

* Drop empty authority always
* Resolve directories

commit fbb0de129d881b6b6a1ae37fc8d75075c9d8df86
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:13:00Z

Remove '/' entirely from determining FileSystem scheme

commit ec956c85efa16d00c5e218ee1257b8ee62a2013d
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:38:54Z

Mark FileSystem and related as Experimental

commit f3540d47f10c18859340a738a7e93643ee57f604
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:46:16Z

Rename FileSystems.setDefaultConfigInWorkers

And document that it's not for users.

commit 3921163829d2a99b0524bf3fa1b85a4cde826f3a
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T20:57:59Z

Fix shading of guava testlib

commit 9f81fd299bd32e0d6056a7da9fa994cf74db0ed9
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T21:59:14Z

This closes #3121

commit f53ceecfb18a52aeeab8f1869c189e4aa2f56d35
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T21:54:33Z

internal comments

commit 89289e498e71d6e9fbd24702032471fa13fbd46c
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T22:08:16Z

This closes #3128

commit 7bade36eb18a48e01a8401f97f10a70a492172c4
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T22:52:51Z

[BEAM-2279] Fix archetype breakages

* Add version for hadoop-file-system dependency.
* Add DirectRunner in test scope in java8, otherwise mvn package fails
  when another runner profile is enabled.

commit 9a43da74a9005507acdc80524edc634c8318674d
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T22:57:21Z

This closes #3132

commit 504bd6a8036946a796a7290853e45bf20d32d07d
Author: Rune Fevang <fev...@exabel.com>
Date:   2017-05-12T20:24:32Z

[BEAM-2290] Fix issue where timestamps weren't set when using 
CompressedSource

commit d4df9079039d57e37cdb1666c8c3120f2fa2909a
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-15T16:52:38Z

This closes #3131

commit 8060d66083973cdab91f68085652ab68e3dee596
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-15T17:00:45Z

Reduce Log Level of PubsubUnboundedSource

commit 6e8d8d65708e3d5b0525a7b7f97597d4af71d51e
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-15T17:07:14Z

This closes #3145

commit de1adefda958ae152eae4b7b94b3b78563e130aa
Author: Neville Li <neville@gmail.com>
Date:   2017-05-15T18:55:09Z

readAvros should't have proto Message upper bound

commit 88ec00860032078dcd33e07d94642912f6aa7bf7
Author: Eugene Kirpichov <kirpic...@google.com>
Date:   2017-05-15T19:22:27Z

This closes #3149

commit 15ec2cd83d4291957a75d9f2ac838ff867dcbe0c
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-15T17:57:24Z

Fix GcsResourceIdTest in postcommits

This test is not 

[GitHub] beam pull request #3244: DSL_SQL: merge master into DSL_SQL feature branch

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3244


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3248: rebasing DSL_SQL branch from master

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3248

rebasing DSL_SQL branch from master

first time do this(rebase feature branch from master), not sure whether it 
is the right way. @jbonofre can you take a look at it?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam DSL_SQL

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3248.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3248


commit e8d829817e3f7e07884a91737b61bfa5133d5724
Author: Borisa Zivkovic <borisa.zivko...@huawei.com>
Date:   2017-05-12T07:44:20Z

BigtableIO should use AutoValue for read and write

commit 42066431b45ced1931787650d2210be724fb7f5c
Author: Eugene Kirpichov <kirpic...@google.com>
Date:   2017-05-12T17:57:03Z

This closes #3113

commit 6a90b9d164d98d916d0e0af2be39a740ce769076
Author: Aviem Zur <aviem...@gmail.com>
Date:   2017-05-12T13:02:54Z

[BEAM-2277] Fix URI_SCHEME_PATTERN in FileSystems

commit 0acf3620d90a970f4024ae95b909c3bb4ef827ad
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:10:48Z

This closes #3115

commit abe62feb16f6eb59e294c37abf408f84c9280616
Author: Aviem Zur <aviem...@gmail.com>
Date:   2017-05-12T19:18:02Z

[BEAM-2279] Add HDFS support to Spark runner profiles in archetypes and 
examples

commit c2a36284b510461942205b627a019dfbe83f8b37
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-12T21:54:00Z

This closes #3126

commit a6a5ff7be387ef295fc7f921de36a3ea77327bc1
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T16:20:34Z

[BEAM-2277] Add ResourceIdTester and test existing ResourceId 
implementations

A first cut at some of the parts of the ResourceId spec.

commit 15df211c758e7c8f05c3136f25bbe18e3f394321
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:11:06Z

[BEAM-2277] HadoopFileSystem: normalize implementation

* Drop empty authority always
* Resolve directories

commit fbb0de129d881b6b6a1ae37fc8d75075c9d8df86
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:13:00Z

Remove '/' entirely from determining FileSystem scheme

commit ec956c85efa16d00c5e218ee1257b8ee62a2013d
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:38:54Z

Mark FileSystem and related as Experimental

commit f3540d47f10c18859340a738a7e93643ee57f604
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:46:16Z

Rename FileSystems.setDefaultConfigInWorkers

And document that it's not for users.

commit 3921163829d2a99b0524bf3fa1b85a4cde826f3a
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T20:57:59Z

Fix shading of guava testlib

commit 9f81fd299bd32e0d6056a7da9fa994cf74db0ed9
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T21:59:14Z

This closes #3121

commit f53ceecfb18a52aeeab8f1869c189e4aa2f56d35
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T21:54:33Z

internal comments

commit 89289e498e71d6e9fbd24702032471fa13fbd46c
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T22:08:16Z

This closes #3128

commit 7bade36eb18a48e01a8401f97f10a70a492172c4
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T22:52:51Z

[BEAM-2279] Fix archetype breakages

* Add version for hadoop-file-system dependency.
* Add DirectRunner in test scope in java8, otherwise mvn package fails
  when another runner profile is enabled.

commit 9a43da74a9005507acdc80524edc634c8318674d
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T22:57:21Z

This closes #3132

commit 504bd6a8036946a796a7290853e45bf20d32d07d
Author: Rune Fevang <fev...@exabel.com>
Date:   2017-05-12T20:24:32Z

[BEAM-2290] Fix issue where timestamps weren't set when using 
CompressedSource

commit d4df9079039d57e37cdb1666c8c3120f2fa2909a
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-15T16:52:38Z

This closes #3131

commit 8060d66083973cdab91f68085652ab68e3dee596
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-15T17:00:45Z

Reduce Log Level of PubsubUnboundedSource

commit 6e8d8d65708e3d5b0525a7b7f97597d4af71d51e
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-15T17:07:14Z

This closes #3145

commit de1adefda958ae152eae4b7b94b3b78563e130aa
Author: Neville Li <neville@gmail.com>
Date:   2017-05-15T18:55:09Z

readAvros should't have proto Message upper bound

commit 88ec00860032078dcd33e07d94642912f6aa7bf7
Author: Eugene Kirpichov <kirpic...@google.com>
Date:   2017-05-15T19:22:27Z

This closes #3149

commit 15ec2cd83d4291957a75d9f2ac838ff867d

[GitHub] beam pull request #3247: rebasing test

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3247


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3247: rebasing test

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3247

rebasing test

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam DSL_SQL

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3247.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3247


commit 42066431b45ced1931787650d2210be724fb7f5c
Author: Eugene Kirpichov <kirpic...@google.com>
Date:   2017-05-12T17:57:03Z

This closes #3113

commit 6a90b9d164d98d916d0e0af2be39a740ce769076
Author: Aviem Zur <aviem...@gmail.com>
Date:   2017-05-12T13:02:54Z

[BEAM-2277] Fix URI_SCHEME_PATTERN in FileSystems

commit 0acf3620d90a970f4024ae95b909c3bb4ef827ad
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:10:48Z

This closes #3115

commit abe62feb16f6eb59e294c37abf408f84c9280616
Author: Aviem Zur <aviem...@gmail.com>
Date:   2017-05-12T19:18:02Z

[BEAM-2279] Add HDFS support to Spark runner profiles in archetypes and 
examples

commit c2a36284b510461942205b627a019dfbe83f8b37
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-12T21:54:00Z

This closes #3126

commit a6a5ff7be387ef295fc7f921de36a3ea77327bc1
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T16:20:34Z

[BEAM-2277] Add ResourceIdTester and test existing ResourceId 
implementations

A first cut at some of the parts of the ResourceId spec.

commit 15df211c758e7c8f05c3136f25bbe18e3f394321
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:11:06Z

[BEAM-2277] HadoopFileSystem: normalize implementation

* Drop empty authority always
* Resolve directories

commit fbb0de129d881b6b6a1ae37fc8d75075c9d8df86
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:13:00Z

Remove '/' entirely from determining FileSystem scheme

commit ec956c85efa16d00c5e218ee1257b8ee62a2013d
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:38:54Z

Mark FileSystem and related as Experimental

commit f3540d47f10c18859340a738a7e93643ee57f604
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:46:16Z

Rename FileSystems.setDefaultConfigInWorkers

And document that it's not for users.

commit 3921163829d2a99b0524bf3fa1b85a4cde826f3a
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T20:57:59Z

Fix shading of guava testlib

commit 9f81fd299bd32e0d6056a7da9fa994cf74db0ed9
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T21:59:14Z

This closes #3121

commit f53ceecfb18a52aeeab8f1869c189e4aa2f56d35
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T21:54:33Z

internal comments

commit 89289e498e71d6e9fbd24702032471fa13fbd46c
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T22:08:16Z

This closes #3128

commit 7bade36eb18a48e01a8401f97f10a70a492172c4
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T22:52:51Z

[BEAM-2279] Fix archetype breakages

* Add version for hadoop-file-system dependency.
* Add DirectRunner in test scope in java8, otherwise mvn package fails
  when another runner profile is enabled.

commit 9a43da74a9005507acdc80524edc634c8318674d
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T22:57:21Z

This closes #3132

commit 504bd6a8036946a796a7290853e45bf20d32d07d
Author: Rune Fevang <fev...@exabel.com>
Date:   2017-05-12T20:24:32Z

[BEAM-2290] Fix issue where timestamps weren't set when using 
CompressedSource

commit d4df9079039d57e37cdb1666c8c3120f2fa2909a
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-15T16:52:38Z

This closes #3131

commit 8060d66083973cdab91f68085652ab68e3dee596
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-15T17:00:45Z

Reduce Log Level of PubsubUnboundedSource

commit 6e8d8d65708e3d5b0525a7b7f97597d4af71d51e
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-15T17:07:14Z

This closes #3145

commit de1adefda958ae152eae4b7b94b3b78563e130aa
Author: Neville Li <neville@gmail.com>
Date:   2017-05-15T18:55:09Z

readAvros should't have proto Message upper bound

commit 88ec00860032078dcd33e0

[GitHub] beam pull request #3246: rebasing DSL_SQL branch from master

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3246


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3246: rebasing DSL_SQL branch from master

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3246

rebasing DSL_SQL branch from master

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam DSL_SQL

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3246.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3246


commit 6a90b9d164d98d916d0e0af2be39a740ce769076
Author: Aviem Zur <aviem...@gmail.com>
Date:   2017-05-12T13:02:54Z

[BEAM-2277] Fix URI_SCHEME_PATTERN in FileSystems

commit 0acf3620d90a970f4024ae95b909c3bb4ef827ad
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:10:48Z

This closes #3115

commit abe62feb16f6eb59e294c37abf408f84c9280616
Author: Aviem Zur <aviem...@gmail.com>
Date:   2017-05-12T19:18:02Z

[BEAM-2279] Add HDFS support to Spark runner profiles in archetypes and 
examples

commit c2a36284b510461942205b627a019dfbe83f8b37
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-12T21:54:00Z

This closes #3126

commit a6a5ff7be387ef295fc7f921de36a3ea77327bc1
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T16:20:34Z

[BEAM-2277] Add ResourceIdTester and test existing ResourceId 
implementations

A first cut at some of the parts of the ResourceId spec.

commit 15df211c758e7c8f05c3136f25bbe18e3f394321
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:11:06Z

[BEAM-2277] HadoopFileSystem: normalize implementation

* Drop empty authority always
* Resolve directories

commit fbb0de129d881b6b6a1ae37fc8d75075c9d8df86
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:13:00Z

Remove '/' entirely from determining FileSystem scheme

commit ec956c85efa16d00c5e218ee1257b8ee62a2013d
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:38:54Z

Mark FileSystem and related as Experimental

commit f3540d47f10c18859340a738a7e93643ee57f604
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T18:46:16Z

Rename FileSystems.setDefaultConfigInWorkers

And document that it's not for users.

commit 3921163829d2a99b0524bf3fa1b85a4cde826f3a
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T20:57:59Z

Fix shading of guava testlib

commit 9f81fd299bd32e0d6056a7da9fa994cf74db0ed9
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T21:59:14Z

This closes #3121

commit f53ceecfb18a52aeeab8f1869c189e4aa2f56d35
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T21:54:33Z

internal comments

commit 89289e498e71d6e9fbd24702032471fa13fbd46c
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T22:08:16Z

This closes #3128

commit 7bade36eb18a48e01a8401f97f10a70a492172c4
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T22:52:51Z

[BEAM-2279] Fix archetype breakages

* Add version for hadoop-file-system dependency.
* Add DirectRunner in test scope in java8, otherwise mvn package fails
  when another runner profile is enabled.

commit 9a43da74a9005507acdc80524edc634c8318674d
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-12T22:57:21Z

This closes #3132

commit 504bd6a8036946a796a7290853e45bf20d32d07d
Author: Rune Fevang <fev...@exabel.com>
Date:   2017-05-12T20:24:32Z

[BEAM-2290] Fix issue where timestamps weren't set when using 
CompressedSource

commit d4df9079039d57e37cdb1666c8c3120f2fa2909a
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-15T16:52:38Z

This closes #3131

commit 8060d66083973cdab91f68085652ab68e3dee596
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-15T17:00:45Z

Reduce Log Level of PubsubUnboundedSource

commit 6e8d8d65708e3d5b0525a7b7f97597d4af71d51e
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-15T17:07:14Z

This closes #3145

commit de1adefda958ae152eae4b7b94b3b78563e130aa
Author: Neville Li <neville@gmail.com>
Date:   2017-05-15T18:55:09Z

readAvros should't have proto Message upper bound

commit 88ec00860032078dcd33e07d94642912f6aa7bf7
Author: Eugene Kirpichov <kirpic...@google.com>
Date:   2017-05-15T19:22:27Z

This closes #3149

co

[GitHub] beam pull request #3244: DSL_SQL: merge master into DSL_SQL feature branch

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3244

DSL_SQL: merge master into DSL_SQL feature branch

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x ] Make sure tests pass via `mvn clean verify`.
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/beam master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3244.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3244


commit 16a3fff18fdd34ebd85fe7d436f8ce3a429666d0
Author: Robert Bradshaw <rober...@gmail.com>
Date:   2017-05-11T19:09:28Z

[BEAM-1345] Mark apache_beam/internal as internal.

commit 7881fd5e6b53f6a26e23f877e47a410f6a515ae7
Author: Robert Bradshaw <rober...@gmail.com>
Date:   2017-05-11T19:54:13Z

[BEAM-1345] Clearly delineate public api in apache_beam/typehints.

commit 4a62fabdb30de3f9d1a539ee7e685b5151ed9ca1
Author: Robert Bradshaw <rober...@gmail.com>
Date:   2017-05-11T22:15:13Z

Closes #3094

commit 0b7356e329f060c3824991cbe3b43bce89042763
Author: Charles Chen <c...@google.com>
Date:   2017-05-11T22:07:30Z

Move assert_that, equal_to, is_empty to apache_beam.testing.util

commit 1b31167fb8bac634d40e5ab5b86b0e928e00f309
Author: Robert Bradshaw <rober...@gmail.com>
Date:   2017-05-11T22:27:59Z

Closes #3086

commit 98e685d85a4cf75faee8156516d7d3aff60077f3
Author: Robert Bradshaw <rober...@google.com>
Date:   2017-05-10T22:55:46Z

Remove some internal details from the public API.

commit fe625678f55d9c9cc288b644749d7916015cf080
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-11T22:43:21Z

This closes #3065

commit d76349beeab5cf6e0513d6272d635ac0b4662239
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-11T21:03:08Z

Don't deploy jdk1.8-tests module

commit 37e0b437e14991b6627efa8c3fc5b215e8925f81
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-05-11T22:49:39Z

This closes #3097

commit fc3e28de37acc9de597518cf9fe793d1ddebcf2a
Author: Robert Bradshaw <rober...@gmail.com>
Date:   2017-05-11T22:47:54Z

Fix due to GBKO name change.

commit 07e5536bc128a324e213eec1a6bd97beb950dcc6
Author: Robert Bradshaw <rober...@gmail.com>
Date:   2017-05-11T22:50:35Z

Closes #3101

commit a45d63f6362e88d034704a4749cef7eedc76a691
Author: Sourabh Bajaj <sourabhba...@google.com>
Date:   2017-05-11T23:41:06Z

Remove unused test data

commit fc77ca7cb18f5dc93dca9427112a3eaaf1494788
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T00:01:38Z

This closes #3106

commit c39c9aecf859095bfde87267e7eb9d9a4fd1682d
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-12T00:16:53Z

Enable SerializableCoder to Serialize with Generic Types

A TypeToken that contains generics is not serializable. However, the
TypeDescriptor does not need to be transmitted via the serialized form,
so mark it as transient.

commit 652fcb7e9e427b3ae44dbfcdd537298d9dd07cb5
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-12T00:55:50Z

This closes #3109

commit 2f177820c4e72fe92dc47d595847ddecdae7afcd
Author: Sourabh Bajaj <sourabhba...@google.com>
Date:   2017-05-12T00:23:40Z

Rename filesink to filebasedsink

commit 36a6cd69aa01b7e122e6e84ff1218adc7ba059c9
Author: Ahmet Altay <al...@google.com>
Date:   2017-05-12T01:04:24Z

This closes #3111

commit db7d71aec5d6c4b7a15474c7ef9db28460fb0043
Author: Thomas Groh <tg...@google.com>
Date:   2017-05-11T23:17:54Z

Mark More values methods Internal

PCollectionTuple#ofPrimitiveOutputsInternal is internal.

TaggedPValue is internal.

commit d6ac39a23e50e82c013e5052b48cb6ffbf929195
Author: Luke Cwik <lc...@google.com>
Date:   2017-05-12T01:34:15Z

[BEAM-1345] Mark More values methods Internal

This closes #3104

commit 5228f7d1f48bf39c0f97c28ab64064089e5f8df9
Author: Borisa Zivkovic <borisa.zivko...@huawei.com>
Date:   2017-05-12T13:03:02Z

[BEAM-2153] Move connection management in JmsIO.write() to setup/teardown 
methods

commit 13c06bf79a45aa35e099730569331aa4d04d25de
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2017-05-12T14:49:20Z

[BEAM-2153] This closes #3116

commit e8d829817e3f7e07884a91737b61bfa5133d5724
Author: Borisa Zivk

[GitHub] beam pull request #3168: [BEAM-2310] support new data type: DECIMAL

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3168


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3166: [BEAM-2309] implement VALUES

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3166


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3218: [BEAM-2356] add NOT operator & some refactoring

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3218

[BEAM-2356] add NOT operator & some refactoring

Seems only `NOT` operator is missing, other operators will be implicitly 
converted to `AND`, `OR`, `NOT`, during parsing phase, e.g. `IS FALSE` is 
converted to `NOT`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2356-logical-operator

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3218.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3218


commit c0502cb74edbdda6fe65168a821ecc0949f82cf5
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-24T04:41:47Z

add NOT operator & some refactoring




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3207: [BEAM-2247] implement date functions

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3207

[BEAM-2247] implement date functions

@XuMingmin @jbonofre can you guys take a look at this one?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2247-date-functions

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3207.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3207


commit 5783fc09545058a8c35a110fcc7284626e9927f9
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-17T03:48:11Z

init implementation(require Values)

commit 48bac0325507c9265d8f6e94ea3747954b31bbc5
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-23T13:38:06Z

implement date functions




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3162: [BEAM-2297] support UNION

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3162


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3183: [BEAM-2325] support Set operator: intersect & excep...

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3183

[BEAM-2325] support Set operator: intersect & except

R: @XuMingmin 
R: @jbonofre 

guys, can you take a look at this one?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2325-intersect-except

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3183.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3183


commit c6d909baa2fa6624d47893e50264d75a535dbfe6
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-19T13:47:10Z

support Set operator: intersect & except




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3168: [BEAM-2310] support new data type: DECIMAL

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3168

[BEAM-2310] support new data type: DECIMAL

1. support new data type: DECIMAL
2. support encoding/decoding for TIME
3. add tests all the supported data types.

R: @XuMingmin 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2310-decimal

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3168.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3168


commit 9a0739dcdfc3b865ea3e74f47bd1188a18abe60a
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-17T14:48:00Z

support encoding/decoding of TIME

commit 12331d5af6482f18ddfaec2b1e22e192a1605c52
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-17T14:52:08Z

support new data type: DECIMAL




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3166: [BEAM-2309] implement VALUES

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3166

[BEAM-2309] implement VALUES

1. implement VALUES
2. add support for data type CHAR(to be able to test VALUES)

R: @XuMingmin 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2309-values

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3166.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3166


commit 72217a0ff8d75842e33bf98b5cf91b258725f1c8
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-17T06:41:18Z

implement VALUES

1. implement VALUES
2. add support for data type CHAR(to be able to test VALUES)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3082: [BEAM-2255] implement ORDER BY

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3082


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3162: [BEAM-2297] support UNION

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3162

[BEAM-2297] support UNION

support UNION

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2297-union

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3162.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3162


commit d540d78359f28a8e7a38af22d69f1136c778bd74
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-16T05:10:56Z

support UNION




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3122: [BEAM-2149] Improved kafka table implemention

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3122


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3042: [BEAM-2195] support conditional operators

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/3042


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3122: [BEAM-2149] Improved kafka table implemention

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3122

[BEAM-2149] Improved kafka table implemention

1. use robust CSV library to parse & print.
2. support different data types rather than just `String`.
3. a little cleanup for TextTable (to extract common methods).

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2149-kafka

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3122.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3122


commit 2074343ff4afc402a8ff50ecc4c99dbbd05281cb
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-12T17:07:01Z

Improved kafka table implemention.

1. use robust CSV library to parse & print.
2. support different data types rather than just `String`.
3. a little cleanup for TextTable (to extract common methods).




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3082: [BEAM-2255] implement ORDER BY

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3082

[BEAM-2255] implement ORDER BY



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2255-order-by

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3082.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3082


commit 95ee7578f77577c60859776539e54a41103f94a4
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-11T17:07:18Z

implement ORDER BY




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #3077: Fix typo in org.apache.beam.sdk.transforms.Top

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/3077

Fix typo in org.apache.beam.sdk.transforms.Top

Hope I understand the method comments correctly:)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam fix-typo-in-Top

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3077.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3077


commit a0377c9e8264ff58bafa6438d19a533f8a17
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-11T08:07:10Z

minor typo fix in comment




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2942: [BEAM-2079] Support TextIO as SQL source/sink

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/2942


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2893: [BEAM-2158] implement the arithmetic operators

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/2893


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2942: [BEAM-2079] Support TextIO as SQL source/sink

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/2942

[BEAM-2079] Support TextIO as SQL source/sink

Support TextIO as SQL source/sink

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2079

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2942.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2942


commit 3625dbd9e00db28d2175c6ac000792fd5c5d8f65
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2016-08-09T09:17:32Z

Add DSLs module

commit 7867ce62e43bef7bfd8011c605379df05494dfcf
Author: mingmxu <ming...@ebay.com>
Date:   2017-04-10T02:49:08Z

[BEAM-301] Initial skeleton for Beam SQL

commit f1c2b6540652156ad7c51ec89a933aacb8c6dab6
Author: Jean-Baptiste Onofré <jbono...@apache.org>
Date:   2017-04-12T20:00:53Z

[BEAM-301] This closes #2479

commit 529bc9d9375ae162d70de274975c547a878053a3
Author: mingmxu <ming...@ebay.com>
Date:   2017-04-13T21:46:28Z

checkstyle and rename package

correct package from org.beam.dsls.sql to org.apache.beam.dsls.sql

update with checkstyle

commit ca8760373b37b540219301f1e719633956a787dc
Author: Davor Bonaci <da...@google.com>
Date:   2017-04-18T21:40:40Z

This closes #2530

commit aa07a1d41b71c810c4968bd35e5ca69dc56b8c30
Author: Dan Halperin <dhalp...@google.com>
Date:   2017-04-19T19:14:05Z

This closes #2584

commit 464cc275952305bf51ecfdf056e784441c9c2272
Author: mingmxu <ming...@ebay.com>
Date:   2017-04-23T05:20:25Z

redesign BeamSqlExpression to execute Calcite SQL expression.

Changes:
1. revert BEAM dependency to 0.6.0 to avoid impact of changes in master 
branch;
2. updates as discussion during review;

refine BeamSqlRowCoder

commit a6845a35fc2a4091dcef24f231c6d3a418561e6f
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-04T07:12:55Z

This closes #2656

commit cf95571d90329287b92946065c93724780f31685
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-04T07:16:34Z

This closes #2878

commit dd70852f6fa811a410938e5baa51cb8e602c931b
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-04T03:44:14Z

Fix inconsistent mapping for SQL FLOAT

commit f7ee8d33e13506a17314720c8c23fff486bff76c
Author: Davor Bonaci <da...@google.com>
Date:   2017-05-07T02:24:04Z

This closes #2886

commit bc9f9f3daffe47426f22f2b944f860b8cd2889da
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-04T03:44:14Z

Support TextIO as SQL source/sink




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2886: [BEAM-2151]Fix inconsistent mapping for SQL FLOAT

[xumingming]

Github user xumingming closed the pull request at:

https://github.com/apache/beam/pull/2886


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2919: [BEAM-2161] support string operators

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/2919

[BEAM-2161] support string operators

https://issues.apache.org/jira/browse/BEAM-2161 @XuMingmin 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2161-support-string-operators

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2919.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2919


commit bc0506b28aaa41da0795facb78ab6dc936d87d3d
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-05T17:20:54Z

implement string operators




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2893: [BEAM-2158] implement the arithmetic operators

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/2893

[BEAM-2158] implement the arithmetic operators

implement the arithmetic operators: 
https://issues.apache.org/jira/browse/BEAM-2158

@XuMingmin can you take a look at this?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam 
BEAM-2158-support-arithmetic-operators

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2893.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2893






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] beam pull request #2886: [BEAM-2151]Fix inconsistent mapping for SQL FLOAT

[xumingming]

GitHub user xumingming opened a pull request:

https://github.com/apache/beam/pull/2886

[BEAM-2151]Fix inconsistent mapping for SQL FLOAT

This pull request is mainly to fix the SQL FLOAT mapping, previously it is 
mapped to Float in `BeamSQLRow`, but mapped to `Double` in `BeamSQLRowCoder`, 
now we consistently map SQL FLOAT to java Float.

When try to unit test this, found another bug in BeamSQLRow: the VARCHAR 
field is not encode/decoded correctly(mainly because OUTTER Context vs NESTED 
Context), so fixed this bug at the same time.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xumingming/beam BEAM-2151

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2886.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2886


commit 02ffb848cf536cf631696f68051fecded2907931
Author: James Xu <xumingmi...@gmail.com>
Date:   2017-05-04T03:44:14Z

Fix inconsistent mapping for SQL FLOAT




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---