subject:"incubator\-hawq git commit\: HAWQ\-703. Serialize HCatalog Complex Types to plain text \(as Hive profile\). \[Forced Update\!\]"

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-05-05 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 bdcb854ad -> a042001ff (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/a042001f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/a042001f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/a042001f

Branch: refs/heads/HAWQ-703
Commit: a042001ff4212c03e0135d57a5cbf9864e062758
Parents: 6717e7d b357dbc
Author: Oleksandr Diachenko 
Authored: Mon May 2 13:00:44 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu May 5 15:33:36 2016 -0700

--
 depends/libyarn/src/CMakeLists.txt  |   2 +-
 .../libyarn/src/libyarnclient/LibYarnClient.cpp |   6 +-
 pxf/Makefile|   5 +-
 pxf/build.gradle|   5 +
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  28 +-
 .../org/apache/hawq/pxf/api/package-info.java   |  23 ++
 .../hawq/pxf/api/utilities/Utilities.java   |  11 +-
 .../org/apache/hawq/pxf/api/MetadataTest.java   |  11 +
 .../hawq/pxf/plugins/hbase/package-info.java|  23 ++
 .../hawq/pxf/plugins/hdfs/AvroResolver.java |   4 +-
 .../hawq/pxf/plugins/hdfs/package-info.java |  23 ++
 .../hawq/pxf/plugins/hive/package-info.java |  23 ++
 .../hive/utilities/EnumHiveToHawqType.java  |   7 +-
 .../plugins/hive/utilities/HiveUtilities.java   |  59 +--
 .../hive/utilities/HiveUtilitiesTest.java   |   8 +-
 .../pxf/service/MetadataResponseFormatter.java  |   1 +
 .../apache/hawq/pxf/service/package-info.java   |  23 ++
 .../pxf/service/utilities/ProtocolData.java |   2 +-
 .../service/MetadataResponseFormatterTest.java  |   2 +-
 src/backend/catalog/external/externalmd.c   |  53 ++-
 src/backend/executor/execMain.c |   2 +-
 .../gpopt/translate/CTranslatorUtils.cpp|  10 +-
 .../communication/rmcomm_RM2RMSEG.c |  13 +-
 src/backend/utils/misc/etc/gpcheck.cnf  |  48 ++-
 src/bin/psql/describe.c |   5 +-
 .../data/hcatalog/invalid_numeric_range.json|   2 +-
 .../hcatalog/invalid_typemod_timestamp.json |   2 +-
 src/test/regress/data/hcatalog/multi_table.json |   2 +-
 .../data/hcatalog/multi_table_duplicates.json   |   2 +-
 .../regress/data/hcatalog/null_field_name.json  |   1 +
 .../data/hcatalog/null_field_source_type.json   |   1 +
 .../regress/data/hcatalog/null_field_type.json  |   1 +
 src/test/regress/data/hcatalog/null_fields.json |   1 +
 src/test/regress/data/hcatalog/null_item.json   |   1 +
 .../regress/data/hcatalog/null_item_name.json   |   1 +
 .../regress/data/hcatalog/null_item_path.json   |   1 +
 .../regress/data/hcatalog/single_table.json |   2 +-
 src/test/regress/input/json_load.source |  35 ++
 src/test/regress/output/hcatalog_lookup.source  |   8 +-
 src/test/regress/output/json_load.source|  35 ++
 tools/bin/gpcheck   | 372 ++-
 tools/bin/gppylib/gpcheckutil.py|  17 +-
 tools/bin/hawqpylib/hawqlib.py  |  18 +
 tools/doc/gpcheck_help  |  44 ++-
 tools/sbin/gpcheck_hostdump |  50 ++-
 45 files changed, 880 insertions(+), 113 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/a042001f/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --cc pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4cb7104,4fc510d..9e1c137
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@@ -75,22 -71,22 +75,26 @@@ public class Metadata 
   */
  public static class Field {
  private String name;
 -private String type; // TODO: change to enum
 +private EnumHawqType type; // field type which PXF exposes
- private String sourceType; // filed type PXF reads from
++private String sourceType; // field type PXF reads from
  private String[] modifiers; // type modifiers, optional field
  
- public Field(String name, EnumHawqType type, String sourceType) {
- if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())
- || StringUtils.isBlank(sourceType)) {
- throw new IllegalArgumentException("Field name, type and source 
type cannot be empty");
 -public Field(String name, String type) {
 -
 -if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
 -throw new

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-05-05 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 d6d556e36 -> 0f556808b (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/0f556808
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/0f556808
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/0f556808

Branch: refs/heads/HAWQ-703
Commit: 0f556808bccd3640f00962a5629978ab9474f4f4
Parents: 6717e7d b357dbc
Author: Oleksandr Diachenko 
Authored: Mon May 2 13:00:44 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu May 5 15:30:54 2016 -0700

--
 depends/libyarn/src/CMakeLists.txt  |   2 +-
 .../libyarn/src/libyarnclient/LibYarnClient.cpp |   6 +-
 pxf/Makefile|   5 +-
 pxf/build.gradle|   5 +
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  28 +-
 .../org/apache/hawq/pxf/api/package-info.java   |  23 ++
 .../hawq/pxf/api/utilities/Utilities.java   |  11 +-
 .../org/apache/hawq/pxf/api/MetadataTest.java   |  11 +
 .../hawq/pxf/plugins/hbase/package-info.java|  23 ++
 .../hawq/pxf/plugins/hdfs/AvroResolver.java |   4 +-
 .../hawq/pxf/plugins/hdfs/package-info.java |  23 ++
 .../hawq/pxf/plugins/hive/package-info.java |  23 ++
 .../hive/utilities/EnumHiveToHawqType.java  |   7 +-
 .../plugins/hive/utilities/HiveUtilities.java   |  59 +--
 .../hive/utilities/HiveUtilitiesTest.java   |   8 +-
 .../pxf/service/MetadataResponseFormatter.java  |   1 +
 .../apache/hawq/pxf/service/package-info.java   |  23 ++
 .../pxf/service/utilities/ProtocolData.java |   2 +-
 .../service/MetadataResponseFormatterTest.java  |   2 +-
 src/backend/catalog/external/externalmd.c   |  53 ++-
 src/backend/executor/execMain.c |   2 +-
 .../gpopt/translate/CTranslatorUtils.cpp|  10 +-
 .../communication/rmcomm_RM2RMSEG.c |  13 +-
 src/backend/utils/misc/etc/gpcheck.cnf  |  48 ++-
 src/bin/psql/describe.c |   5 +-
 .../data/hcatalog/invalid_numeric_range.json|   2 +-
 .../hcatalog/invalid_typemod_timestamp.json |   2 +-
 src/test/regress/data/hcatalog/multi_table.json |   2 +-
 .../data/hcatalog/multi_table_duplicates.json   |   2 +-
 .../regress/data/hcatalog/null_field_name.json  |   1 +
 .../data/hcatalog/null_field_source_type.json   |   1 +
 .../regress/data/hcatalog/null_field_type.json  |   1 +
 src/test/regress/data/hcatalog/null_fields.json |   1 +
 src/test/regress/data/hcatalog/null_item.json   |   1 +
 .../regress/data/hcatalog/null_item_name.json   |   1 +
 .../regress/data/hcatalog/null_item_path.json   |   1 +
 .../regress/data/hcatalog/single_table.json |   2 +-
 src/test/regress/input/json_load.source |  35 ++
 src/test/regress/output/hcatalog_lookup.source  |   8 +-
 src/test/regress/output/json_load.source|  35 ++
 tools/bin/gpcheck   | 372 ++-
 tools/bin/gppylib/gpcheckutil.py|  17 +-
 tools/bin/hawqpylib/hawqlib.py  |  18 +
 tools/doc/gpcheck_help  |  44 ++-
 tools/sbin/gpcheck_hostdump |  50 ++-
 45 files changed, 880 insertions(+), 113 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/0f556808/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --cc pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4cb7104,4fc510d..9e1c137
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@@ -75,22 -71,22 +75,26 @@@ public class Metadata 
   */
  public static class Field {
  private String name;
 -private String type; // TODO: change to enum
 +private EnumHawqType type; // field type which PXF exposes
- private String sourceType; // filed type PXF reads from
++private String sourceType; // field type PXF reads from
  private String[] modifiers; // type modifiers, optional field
  
- public Field(String name, EnumHawqType type, String sourceType) {
- if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())
- || StringUtils.isBlank(sourceType)) {
- throw new IllegalArgumentException("Field name, type and source 
type cannot be empty");
 -public Field(String name, String type) {
 -
 -if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
 -throw new

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-05-05 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 0ef8c9e94 -> d6d556e36 (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/d6d556e3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/d6d556e3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/d6d556e3

Branch: refs/heads/HAWQ-703
Commit: d6d556e3608c19962d4c0ad5f1a0d06288c795b1
Parents: 6717e7d b357dbc
Author: Oleksandr Diachenko 
Authored: Mon May 2 13:00:44 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu May 5 15:26:17 2016 -0700

--
 depends/libyarn/src/CMakeLists.txt  |   2 +-
 .../libyarn/src/libyarnclient/LibYarnClient.cpp |   6 +-
 pxf/Makefile|   5 +-
 pxf/build.gradle|   5 +
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  28 +-
 .../org/apache/hawq/pxf/api/package-info.java   |  23 ++
 .../hawq/pxf/api/utilities/Utilities.java   |  11 +-
 .../org/apache/hawq/pxf/api/MetadataTest.java   |  11 +
 .../hawq/pxf/plugins/hbase/package-info.java|  23 ++
 .../hawq/pxf/plugins/hdfs/AvroResolver.java |   4 +-
 .../hawq/pxf/plugins/hdfs/package-info.java |  23 ++
 .../hawq/pxf/plugins/hive/package-info.java |  23 ++
 .../hive/utilities/EnumHiveToHawqType.java  |   7 +-
 .../plugins/hive/utilities/HiveUtilities.java   |  59 +--
 .../hive/utilities/HiveUtilitiesTest.java   |   8 +-
 .../pxf/service/MetadataResponseFormatter.java  |   1 +
 .../apache/hawq/pxf/service/package-info.java   |  23 ++
 .../pxf/service/utilities/ProtocolData.java |   2 +-
 .../service/MetadataResponseFormatterTest.java  |   2 +-
 src/backend/catalog/external/externalmd.c   |  49 ++-
 src/backend/executor/execMain.c |   2 +-
 .../gpopt/translate/CTranslatorUtils.cpp|  10 +-
 .../communication/rmcomm_RM2RMSEG.c |  13 +-
 src/backend/utils/misc/etc/gpcheck.cnf  |  48 ++-
 src/bin/psql/describe.c |   5 +-
 .../data/hcatalog/invalid_numeric_range.json|   2 +-
 .../hcatalog/invalid_typemod_timestamp.json |   2 +-
 src/test/regress/data/hcatalog/multi_table.json |   2 +-
 .../data/hcatalog/multi_table_duplicates.json   |   2 +-
 .../regress/data/hcatalog/null_field_name.json  |   1 +
 .../data/hcatalog/null_field_source_type.json   |   1 +
 .../regress/data/hcatalog/null_field_type.json  |   1 +
 src/test/regress/data/hcatalog/null_fields.json |   1 +
 src/test/regress/data/hcatalog/null_item.json   |   1 +
 .../regress/data/hcatalog/null_item_name.json   |   1 +
 .../regress/data/hcatalog/null_item_path.json   |   1 +
 .../regress/data/hcatalog/single_table.json |   2 +-
 src/test/regress/input/json_load.source |  35 ++
 src/test/regress/output/hcatalog_lookup.source  |   8 +-
 src/test/regress/output/json_load.source|  35 ++
 tools/bin/gpcheck   | 372 ++-
 tools/bin/gppylib/gpcheckutil.py|  17 +-
 tools/bin/hawqpylib/hawqlib.py  |  18 +
 tools/doc/gpcheck_help  |  44 ++-
 tools/sbin/gpcheck_hostdump |  50 ++-
 45 files changed, 876 insertions(+), 113 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/d6d556e3/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --cc pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4cb7104,4fc510d..9e1c137
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@@ -75,22 -71,22 +75,26 @@@ public class Metadata 
   */
  public static class Field {
  private String name;
 -private String type; // TODO: change to enum
 +private EnumHawqType type; // field type which PXF exposes
- private String sourceType; // filed type PXF reads from
++private String sourceType; // field type PXF reads from
  private String[] modifiers; // type modifiers, optional field
  
- public Field(String name, EnumHawqType type, String sourceType) {
- if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())
- || StringUtils.isBlank(sourceType)) {
- throw new IllegalArgumentException("Field name, type and source 
type cannot be empty");
 -public Field(String name, String type) {
 -
 -if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
 -throw new

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-05-05 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 b4c52b8c7 -> 0ef8c9e94 (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/0ef8c9e9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/0ef8c9e9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/0ef8c9e9

Branch: refs/heads/HAWQ-703
Commit: 0ef8c9e947f33c8114a0d7f0c5bb7834bab96dbb
Parents: 6717e7d b357dbc
Author: Oleksandr Diachenko 
Authored: Mon May 2 13:00:44 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu May 5 15:23:08 2016 -0700

--
 depends/libyarn/src/CMakeLists.txt  |   2 +-
 .../libyarn/src/libyarnclient/LibYarnClient.cpp |   6 +-
 pxf/Makefile|   5 +-
 pxf/build.gradle|   5 +
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  28 +-
 .../org/apache/hawq/pxf/api/package-info.java   |  23 ++
 .../hawq/pxf/api/utilities/Utilities.java   |  11 +-
 .../org/apache/hawq/pxf/api/MetadataTest.java   |  11 +
 .../hawq/pxf/plugins/hbase/package-info.java|  23 ++
 .../hawq/pxf/plugins/hdfs/AvroResolver.java |   4 +-
 .../hawq/pxf/plugins/hdfs/package-info.java |  23 ++
 .../hawq/pxf/plugins/hive/package-info.java |  23 ++
 .../hive/utilities/EnumHiveToHawqType.java  |   7 +-
 .../plugins/hive/utilities/HiveUtilities.java   |  59 +--
 .../hive/utilities/HiveUtilitiesTest.java   |   8 +-
 .../pxf/service/MetadataResponseFormatter.java  |   1 +
 .../apache/hawq/pxf/service/package-info.java   |  23 ++
 .../pxf/service/utilities/ProtocolData.java |   2 +-
 .../service/MetadataResponseFormatterTest.java  |   2 +-
 src/backend/catalog/external/externalmd.c   |  48 ++-
 src/backend/executor/execMain.c |   2 +-
 .../gpopt/translate/CTranslatorUtils.cpp|  10 +-
 .../communication/rmcomm_RM2RMSEG.c |  13 +-
 src/backend/utils/misc/etc/gpcheck.cnf  |  48 ++-
 src/bin/psql/describe.c |   5 +-
 .../data/hcatalog/invalid_numeric_range.json|   2 +-
 .../hcatalog/invalid_typemod_timestamp.json |   2 +-
 src/test/regress/data/hcatalog/multi_table.json |   2 +-
 .../data/hcatalog/multi_table_duplicates.json   |   2 +-
 .../regress/data/hcatalog/null_field_name.json  |   1 +
 .../data/hcatalog/null_field_source_type.json   |   1 +
 .../regress/data/hcatalog/null_field_type.json  |   1 +
 src/test/regress/data/hcatalog/null_fields.json |   1 +
 src/test/regress/data/hcatalog/null_item.json   |   1 +
 .../regress/data/hcatalog/null_item_name.json   |   1 +
 .../regress/data/hcatalog/null_item_path.json   |   1 +
 .../regress/data/hcatalog/single_table.json |   2 +-
 src/test/regress/input/json_load.source |  35 ++
 src/test/regress/output/hcatalog_lookup.source  |   8 +-
 src/test/regress/output/json_load.source|  35 ++
 tools/bin/gpcheck   | 372 ++-
 tools/bin/gppylib/gpcheckutil.py|  17 +-
 tools/bin/hawqpylib/hawqlib.py  |  18 +
 tools/doc/gpcheck_help  |  44 ++-
 tools/sbin/gpcheck_hostdump |  50 ++-
 45 files changed, 875 insertions(+), 113 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/0ef8c9e9/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --cc pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4cb7104,4fc510d..9e1c137
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@@ -75,22 -71,22 +75,26 @@@ public class Metadata 
   */
  public static class Field {
  private String name;
 -private String type; // TODO: change to enum
 +private EnumHawqType type; // field type which PXF exposes
- private String sourceType; // filed type PXF reads from
++private String sourceType; // field type PXF reads from
  private String[] modifiers; // type modifiers, optional field
  
- public Field(String name, EnumHawqType type, String sourceType) {
- if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())
- || StringUtils.isBlank(sourceType)) {
- throw new IllegalArgumentException("Field name, type and source 
type cannot be empty");
 -public Field(String name, String type) {
 -
 -if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
 -throw new

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-05-03 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 4ccb72dd3 -> b4c52b8c7 (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/b4c52b8c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/b4c52b8c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/b4c52b8c

Branch: refs/heads/HAWQ-703
Commit: b4c52b8c74b9d701c479537188771678a1a9
Parents: 6717e7d b357dbc
Author: Oleksandr Diachenko 
Authored: Mon May 2 13:00:44 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Tue May 3 17:39:07 2016 -0700

--
 depends/libyarn/src/CMakeLists.txt  |   2 +-
 .../libyarn/src/libyarnclient/LibYarnClient.cpp |   6 +-
 pxf/Makefile|   5 +-
 pxf/build.gradle|   5 +
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  28 +-
 .../org/apache/hawq/pxf/api/package-info.java   |  23 ++
 .../hawq/pxf/api/utilities/Utilities.java   |  11 +-
 .../org/apache/hawq/pxf/api/MetadataTest.java   |  11 +
 .../hawq/pxf/plugins/hbase/package-info.java|  23 ++
 .../hawq/pxf/plugins/hdfs/AvroResolver.java |   4 +-
 .../hawq/pxf/plugins/hdfs/package-info.java |  23 ++
 .../hawq/pxf/plugins/hive/package-info.java |  23 ++
 .../hive/utilities/EnumHiveToHawqType.java  |   7 +-
 .../plugins/hive/utilities/HiveUtilities.java   |  59 +--
 .../hive/utilities/HiveUtilitiesTest.java   |   8 +-
 .../pxf/service/MetadataResponseFormatter.java  |   1 +
 .../apache/hawq/pxf/service/package-info.java   |  23 ++
 .../pxf/service/utilities/ProtocolData.java |   2 +-
 .../service/MetadataResponseFormatterTest.java  |   2 +-
 src/backend/executor/execMain.c |   2 +-
 .../gpopt/translate/CTranslatorUtils.cpp|  10 +-
 .../communication/rmcomm_RM2RMSEG.c |  13 +-
 src/backend/utils/misc/etc/gpcheck.cnf  |  48 ++-
 src/bin/psql/describe.c |   5 +-
 tools/bin/gpcheck   | 372 ++-
 tools/bin/gppylib/gpcheckutil.py|  17 +-
 tools/bin/hawqpylib/hawqlib.py  |  18 +
 tools/doc/gpcheck_help  |  44 ++-
 tools/sbin/gpcheck_hostdump |  50 ++-
 29 files changed, 746 insertions(+), 99 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/b4c52b8c/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --cc pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4cb7104,4fc510d..9e1c137
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@@ -75,22 -71,22 +75,26 @@@ public class Metadata 
   */
  public static class Field {
  private String name;
 -private String type; // TODO: change to enum
 +private EnumHawqType type; // field type which PXF exposes
- private String sourceType; // filed type PXF reads from
++private String sourceType; // field type PXF reads from
  private String[] modifiers; // type modifiers, optional field
  
- public Field(String name, EnumHawqType type, String sourceType) {
- if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())
- || StringUtils.isBlank(sourceType)) {
- throw new IllegalArgumentException("Field name, type and source 
type cannot be empty");
 -public Field(String name, String type) {
 -
 -if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
 -throw new IllegalArgumentException("Field name and type 
cannot be empty");
++public Field(String name, EnumHawqType type, String sourceType) {
++if (StringUtils.isBlank(name)
++|| type == null
++|| StringUtils.isBlank(sourceType)) {
++throw new IllegalArgumentException(
++"Field name, type and source type cannot be empty");
+ }
 -
+ this.name = name;
+ this.type = type;
++this.sourceType = sourceType;
  }
- this.name = name;
- this.type = type;
- this.sourceType = sourceType;
- }
- public Field(String name, EnumHawqType type, String sourceType, String[] 
modifiers) {
- this(name, type, sourceType);
- this.modifiers = modifiers;
+ 
 -public Field(String name, String type, String[] modifiers) {
 -this(name, type);
++

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-05-03 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 ba21c90c5 -> 4ccb72dd3 (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/4ccb72dd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/4ccb72dd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/4ccb72dd

Branch: refs/heads/HAWQ-703
Commit: 4ccb72dd336c8112eda145f249606a7d29aa0aee
Parents: 6717e7d b357dbc
Author: Oleksandr Diachenko 
Authored: Mon May 2 13:00:44 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Tue May 3 17:37:20 2016 -0700

--
 depends/libyarn/src/CMakeLists.txt  |   2 +-
 .../libyarn/src/libyarnclient/LibYarnClient.cpp |   6 +-
 pxf/Makefile|   5 +-
 pxf/build.gradle|   5 +
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  28 +-
 .../org/apache/hawq/pxf/api/package-info.java   |  23 ++
 .../hawq/pxf/api/utilities/Utilities.java   |  11 +-
 .../org/apache/hawq/pxf/api/MetadataTest.java   |  11 +
 .../hawq/pxf/plugins/hbase/package-info.java|  23 ++
 .../hawq/pxf/plugins/hdfs/AvroResolver.java |   4 +-
 .../hawq/pxf/plugins/hdfs/package-info.java |  23 ++
 .../hawq/pxf/plugins/hive/package-info.java |  23 ++
 .../hive/utilities/EnumHiveToHawqType.java  |   7 +-
 .../plugins/hive/utilities/HiveUtilities.java   |  59 +--
 .../hive/utilities/HiveUtilitiesTest.java   |   8 +-
 .../pxf/service/MetadataResponseFormatter.java  |   1 +
 .../apache/hawq/pxf/service/package-info.java   |  23 ++
 .../pxf/service/utilities/ProtocolData.java |   2 +-
 .../service/MetadataResponseFormatterTest.java  |   2 +-
 src/backend/executor/execMain.c |   2 +-
 .../gpopt/translate/CTranslatorUtils.cpp|  10 +-
 .../communication/rmcomm_RM2RMSEG.c |  13 +-
 src/backend/utils/misc/etc/gpcheck.cnf  |  48 ++-
 src/bin/psql/describe.c |   2 +
 tools/bin/gpcheck   | 372 ++-
 tools/bin/gppylib/gpcheckutil.py|  17 +-
 tools/bin/hawqpylib/hawqlib.py  |  18 +
 tools/doc/gpcheck_help  |  44 ++-
 tools/sbin/gpcheck_hostdump |  50 ++-
 29 files changed, 745 insertions(+), 97 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/4ccb72dd/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --cc pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4cb7104,4fc510d..9e1c137
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@@ -75,22 -71,22 +75,26 @@@ public class Metadata 
   */
  public static class Field {
  private String name;
 -private String type; // TODO: change to enum
 +private EnumHawqType type; // field type which PXF exposes
- private String sourceType; // filed type PXF reads from
++private String sourceType; // field type PXF reads from
  private String[] modifiers; // type modifiers, optional field
  
- public Field(String name, EnumHawqType type, String sourceType) {
- if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())
- || StringUtils.isBlank(sourceType)) {
- throw new IllegalArgumentException("Field name, type and source 
type cannot be empty");
 -public Field(String name, String type) {
 -
 -if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
 -throw new IllegalArgumentException("Field name and type 
cannot be empty");
++public Field(String name, EnumHawqType type, String sourceType) {
++if (StringUtils.isBlank(name)
++|| type == null
++|| StringUtils.isBlank(sourceType)) {
++throw new IllegalArgumentException(
++"Field name, type and source type cannot be empty");
+ }
 -
+ this.name = name;
+ this.type = type;
++this.sourceType = sourceType;
  }
- this.name = name;
- this.type = type;
- this.sourceType = sourceType;
- }
- public Field(String name, EnumHawqType type, String sourceType, String[] 
modifiers) {
- this(name, type, sourceType);
- this.modifiers = modifiers;
+ 
 -public Field(String name, String type, String[] modifiers) {
 -this(name, type);
++

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-05-03 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 b1db3991c -> 08007ba97 (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/08007ba9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/08007ba9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/08007ba9

Branch: refs/heads/HAWQ-703
Commit: 08007ba97e8a09308633fc8e43d937b7d0dd280e
Parents: 6717e7d b357dbc
Author: Oleksandr Diachenko 
Authored: Mon May 2 13:00:44 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Tue May 3 16:53:51 2016 -0700

--
 depends/libyarn/src/CMakeLists.txt  |   2 +-
 .../libyarn/src/libyarnclient/LibYarnClient.cpp |   6 +-
 pxf/Makefile|   5 +-
 pxf/build.gradle|   5 +
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  28 +-
 .../org/apache/hawq/pxf/api/package-info.java   |  23 ++
 .../hawq/pxf/api/utilities/Utilities.java   |  11 +-
 .../org/apache/hawq/pxf/api/MetadataTest.java   |  11 +
 .../hawq/pxf/plugins/hbase/package-info.java|  23 ++
 .../hawq/pxf/plugins/hdfs/AvroResolver.java |   4 +-
 .../hawq/pxf/plugins/hdfs/package-info.java |  23 ++
 .../hawq/pxf/plugins/hive/package-info.java |  23 ++
 .../hive/utilities/EnumHiveToHawqType.java  |   2 +-
 .../hive/utilities/HiveUtilitiesTest.java   |   8 +-
 .../pxf/service/MetadataResponseFormatter.java  |   1 +
 .../apache/hawq/pxf/service/package-info.java   |  23 ++
 .../pxf/service/utilities/ProtocolData.java |   2 +-
 src/backend/executor/execMain.c |   2 +-
 .../gpopt/translate/CTranslatorUtils.cpp|  10 +-
 .../communication/rmcomm_RM2RMSEG.c |  13 +-
 src/backend/utils/misc/etc/gpcheck.cnf  |  48 ++-
 src/bin/psql/describe.c |   2 +
 tools/bin/gpcheck   | 372 ++-
 tools/bin/gppylib/gpcheckutil.py|  17 +-
 tools/bin/hawqpylib/hawqlib.py  |  18 +
 tools/doc/gpcheck_help  |  44 ++-
 tools/sbin/gpcheck_hostdump |  50 ++-
 27 files changed, 711 insertions(+), 65 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/08007ba9/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --cc pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4cb7104,4fc510d..9e1c137
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@@ -75,22 -71,22 +75,26 @@@ public class Metadata 
   */
  public static class Field {
  private String name;
 -private String type; // TODO: change to enum
 +private EnumHawqType type; // field type which PXF exposes
- private String sourceType; // filed type PXF reads from
++private String sourceType; // field type PXF reads from
  private String[] modifiers; // type modifiers, optional field
  
- public Field(String name, EnumHawqType type, String sourceType) {
- if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())
- || StringUtils.isBlank(sourceType)) {
- throw new IllegalArgumentException("Field name, type and source 
type cannot be empty");
 -public Field(String name, String type) {
 -
 -if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
 -throw new IllegalArgumentException("Field name and type 
cannot be empty");
++public Field(String name, EnumHawqType type, String sourceType) {
++if (StringUtils.isBlank(name)
++|| type == null
++|| StringUtils.isBlank(sourceType)) {
++throw new IllegalArgumentException(
++"Field name, type and source type cannot be empty");
+ }
 -
+ this.name = name;
+ this.type = type;
++this.sourceType = sourceType;
  }
- this.name = name;
- this.type = type;
- this.sourceType = sourceType;
- }
- public Field(String name, EnumHawqType type, String sourceType, String[] 
modifiers) {
- this(name, type, sourceType);
- this.modifiers = modifiers;
+ 
 -public Field(String name, String type, String[] modifiers) {
 -this(name, type);
++public Field(String name, EnumHawqType type, String sourceType,
++String[] modifiers) {
++

[1/3] incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-04-28 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 45c63b793 -> 8c5e6f8b2 (forced update)


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/8c5e6f8b/pxf/pxf-service/tmp/generatedSources/org/apache/hawq/pxf/service/rest/FragmenterResource.java
--
diff --git 
a/pxf/pxf-service/tmp/generatedSources/org/apache/hawq/pxf/service/rest/FragmenterResource.java
 
b/pxf/pxf-service/tmp/generatedSources/org/apache/hawq/pxf/service/rest/FragmenterResource.java
new file mode 100644
index 000..d6e8d49
--- /dev/null
+++ 
b/pxf/pxf-service/tmp/generatedSources/org/apache/hawq/pxf/service/rest/FragmenterResource.java
@@ -0,0 +1,154 @@
+package org.apache.hawq.pxf.service.rest;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.apache.hawq.pxf.api.Fragment;
+import org.apache.hawq.pxf.api.Fragmenter;
+import org.apache.hawq.pxf.api.FragmentsStats;
+import org.apache.hawq.pxf.service.FragmenterFactory;
+import org.apache.hawq.pxf.service.FragmentsResponse;
+import org.apache.hawq.pxf.service.FragmentsResponseFormatter;
+import org.apache.hawq.pxf.service.utilities.AnalyzeUtils;
+import org.apache.hawq.pxf.service.utilities.ProtocolData;
+import org.apache.hawq.pxf.service.utilities.SecuredHDFS;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+
+import javax.servlet.ServletContext;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.HttpHeaders;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Class enhances the API of the WEBHDFS REST server. Returns the data 
fragments
+ * that a data resource is made of, enabling parallel processing of the data
+ * resource. Example for querying API FRAGMENTER from a web client
+ * {@code curl -i 
"http://localhost:51200/pxf/{version}/Fragmenter/getFragments?path=/dir1/dir2/*txt"}
+ * /pxf/ is made part of the path when there is a webapp by that
+ * name in tomcat.
+ */
+@Path("/" + Version.PXF_PROTOCOL_VERSION + "/Fragmenter/")
+public class FragmenterResource extends RestResource {
+private static final Log LOG = LogFactory.getLog(FragmenterResource.class);
+
+/**
+ * The function is called when
+ * {@code http://nn:port/pxf/{version}/Fragmenter/getFragments?path=...} 
is used.
+ *
+ * @param servletContext Servlet context contains attributes required by
+ *SecuredHDFS
+ * @param headers Holds HTTP headers from request
+ * @param path Holds URI path option used in this request
+ * @return response object with JSON serialized fragments metadata
+ * @throws Exception if getting fragments info failed
+ */
+@GET
+@Path("getFragments")
+@Produces("application/json")
+public Response getFragments(@Context final ServletContext servletContext,
+ @Context final HttpHeaders headers,
+ @QueryParam("path") final String path)
+throws Exception {
+
+ProtocolData protData = getProtocolData(servletContext, headers, path);
+
+/* Create a fragmenter instance with API level parameters */
+final Fragmenter fragmenter = FragmenterFactory.create(protData);
+
+List fragments = fragmenter.getFragments();
+
+fragments = AnalyzeUtils.getSampleFragments(fragments, protData);
+
+FragmentsResponse fragmentsResponse = 
FragmentsResponseFormatter.formatResponse(
+fragments, path);
+
+return Response.ok(fragmentsResponse, 
MediaType.APPLICATION_JSON_TYPE).build();
+}
+
+/**
+ * The function is called when
+ * {@code 
http://nn:port/pxf/{version}/Fragmenter/getFragmentsStats?path=...} is
+ * used.
+ *
+ * @param servletContext Servlet context contains attributes required by
+ *SecuredHDFS
+ * @param headers Holds HTTP headers from request
+ * @param path Holds URI path option used in this request
+ * @return response object with JSON

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-04-28 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 3dc965fd8 -> fb820580c (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/fb820580
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/fb820580
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/fb820580

Branch: refs/heads/HAWQ-703
Commit: fb820580c0e74681e0a5d0f37bb7361e89f4449d
Parents: 649828f
Author: Oleksandr Diachenko 
Authored: Fri Apr 22 16:34:42 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu Apr 28 14:40:09 2016 -0700

--
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  31 +-
 .../hawq/pxf/api/utilities/EnumHawqType.java|  90 +++
 .../org/apache/hawq/pxf/api/MetadataTest.java   |  54 +
 .../hive/utilities/EnumHiveToHawqTypes.java |  70 
 .../plugins/hive/utilities/HiveUtilities.java   | 109 ++-
 .../plugins/hive/HiveMetadataFetcherTest.java   |  12 +-
 .../hive/utilities/HiveUtilitiesTest.java   |  31 --
 .../hawq/pxf/service/MetadataResponse.java  |   6 +-
 .../pxf/service/MetadataResponseFormatter.java  |   3 +-
 .../service/MetadataResponseFormatterTest.java  |  57 ++
 src/backend/catalog/external/externalmd.c   |   4 +
 src/backend/utils/adt/pxf_functions.c   |   9 +-
 src/bin/psql/describe.c |  32 +-
 src/include/catalog/external/itemmd.h   |   3 +
 src/include/catalog/pg_proc.h   |   4 +-
 src/include/catalog/pg_proc.sql |   2 +-
 16 files changed, 386 insertions(+), 131 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/fb820580/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --git a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4fc510d..7db253e 100644
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@ -23,6 +23,7 @@ package org.apache.hawq.pxf.api;
 import java.util.ArrayList;
 import java.util.List;
 
+import org.apache.hawq.pxf.api.utilities.EnumHawqType;
 import org.apache.commons.lang.StringUtils;
 
 /**
@@ -67,16 +68,20 @@ public class Metadata {
 }
 
 /**
- * Class representing item field - name and type.
+ * Class representing item field - name, type, source type, modifiers.
+ * Type - exposed type of field
+ * Source type - type of field in underlying source
+ * Modifiers - additional attributes which describe type or field
  */
 public static class Field {
 private String name;
-private String type; // TODO: change to enum
+private EnumHawqType type; // field type which PXF exposes
+private String sourceType; // filed type PXF reads from
 private String[] modifiers; // type modifiers, optional field
 
-public Field(String name, String type) {
+public Field(String name, EnumHawqType type) {
 
-if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
+if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())) {
 throw new IllegalArgumentException("Field name and type cannot 
be empty");
 }
 
@@ -84,8 +89,18 @@ public class Metadata {
 this.type = type;
 }
 
-public Field(String name, String type, String[] modifiers) {
+public Field(String name, EnumHawqType type, String sourceType) {
 this(name, type);
+this.sourceType = sourceType;
+}
+
+public Field(String name, EnumHawqType type, String[] modifiers) {
+this(name, type);
+this.modifiers = modifiers;
+}
+
+public Field(String name, EnumHawqType type, String sourceType, 
String[] modifiers) {
+this(name, type, sourceType);
 this.modifiers = modifiers;
 }
 
@@ -93,10 +108,14 @@ public class Metadata {
 return name;
 }
 
-public String getType() {
+public EnumHawqType getType() {
 return type;
 }
 
+public String getSourceType() {
+return sourceType;
+}
+
 public String[] getModifiers() {
 return modifiers;
 }

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/fb820580/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-04-28 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 7b8372399 -> 3dc965fd8 (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/3dc965fd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/3dc965fd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/3dc965fd

Branch: refs/heads/HAWQ-703
Commit: 3dc965fd8ceb9cd98b2cdbdc08afe3ec4f013e8a
Parents: 649828f
Author: Oleksandr Diachenko 
Authored: Fri Apr 22 16:34:42 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu Apr 28 14:21:38 2016 -0700

--
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  31 +-
 .../hawq/pxf/api/utilities/EnumHawqType.java|  77 +
 .../hive/utilities/EnumHiveToHawqTypes.java |  70 
 .../plugins/hive/utilities/HiveUtilities.java   | 109 ++-
 .../plugins/hive/HiveMetadataFetcherTest.java   |  12 +-
 .../hive/utilities/HiveUtilitiesTest.java   |  31 --
 .../hawq/pxf/service/MetadataResponse.java  |   1 +
 .../pxf/service/MetadataResponseFormatter.java  |   3 +-
 .../service/MetadataResponseFormatterTest.java  |  57 ++
 src/backend/catalog/external/externalmd.c   |   4 +
 src/backend/utils/adt/pxf_functions.c   |   9 +-
 src/bin/psql/describe.c |  32 +-
 src/include/catalog/external/itemmd.h   |   3 +
 src/include/catalog/pg_proc.h   |   4 +-
 src/include/catalog/pg_proc.sql |   2 +-
 15 files changed, 317 insertions(+), 128 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/3dc965fd/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --git a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4fc510d..7db253e 100644
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@ -23,6 +23,7 @@ package org.apache.hawq.pxf.api;
 import java.util.ArrayList;
 import java.util.List;
 
+import org.apache.hawq.pxf.api.utilities.EnumHawqType;
 import org.apache.commons.lang.StringUtils;
 
 /**
@@ -67,16 +68,20 @@ public class Metadata {
 }
 
 /**
- * Class representing item field - name and type.
+ * Class representing item field - name, type, source type, modifiers.
+ * Type - exposed type of field
+ * Source type - type of field in underlying source
+ * Modifiers - additional attributes which describe type or field
  */
 public static class Field {
 private String name;
-private String type; // TODO: change to enum
+private EnumHawqType type; // field type which PXF exposes
+private String sourceType; // filed type PXF reads from
 private String[] modifiers; // type modifiers, optional field
 
-public Field(String name, String type) {
+public Field(String name, EnumHawqType type) {
 
-if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
+if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())) {
 throw new IllegalArgumentException("Field name and type cannot 
be empty");
 }
 
@@ -84,8 +89,18 @@ public class Metadata {
 this.type = type;
 }
 
-public Field(String name, String type, String[] modifiers) {
+public Field(String name, EnumHawqType type, String sourceType) {
 this(name, type);
+this.sourceType = sourceType;
+}
+
+public Field(String name, EnumHawqType type, String[] modifiers) {
+this(name, type);
+this.modifiers = modifiers;
+}
+
+public Field(String name, EnumHawqType type, String sourceType, 
String[] modifiers) {
+this(name, type, sourceType);
 this.modifiers = modifiers;
 }
 
@@ -93,10 +108,14 @@ public class Metadata {
 return name;
 }
 
-public String getType() {
+public EnumHawqType getType() {
 return type;
 }
 
+public String getSourceType() {
+return sourceType;
+}
+
 public String[] getModifiers() {
 return modifiers;
 }

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/3dc965fd/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java
--
diff --git

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-04-28 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 9d791730a -> 7b8372399 (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/7b837239
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/7b837239
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/7b837239

Branch: refs/heads/HAWQ-703
Commit: 7b83723990939dc4dd5cf6ce325e21eed291d523
Parents: 649828f
Author: Oleksandr Diachenko 
Authored: Fri Apr 22 16:34:42 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu Apr 28 13:57:30 2016 -0700

--
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  33 --
 .../hawq/pxf/api/utilities/EnumHawqType.java|  77 +
 .../hive/utilities/EnumHiveToHawqTypes.java |  70 
 .../plugins/hive/utilities/HiveUtilities.java   | 109 ++-
 .../plugins/hive/HiveMetadataFetcherTest.java   |  12 +-
 .../hive/utilities/HiveUtilitiesTest.java   |  18 +--
 .../hawq/pxf/service/MetadataResponse.java  |   1 +
 .../pxf/service/MetadataResponseFormatter.java  |   3 +-
 .../service/MetadataResponseFormatterTest.java  |  57 ++
 src/backend/catalog/external/externalmd.c   |   4 +
 src/backend/utils/adt/pxf_functions.c   |   9 +-
 src/bin/psql/describe.c |  32 +-
 src/include/catalog/external/itemmd.h   |   3 +
 src/include/catalog/pg_proc.h   |   4 +-
 src/include/catalog/pg_proc.sql |   2 +-
 15 files changed, 305 insertions(+), 129 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/7b837239/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --git a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4fc510d..a375ad8 100644
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@ -23,6 +23,7 @@ package org.apache.hawq.pxf.api;
 import java.util.ArrayList;
 import java.util.List;
 
+import org.apache.hawq.pxf.api.utilities.EnumHawqType;
 import org.apache.commons.lang.StringUtils;
 
 /**
@@ -67,25 +68,39 @@ public class Metadata {
 }
 
 /**
- * Class representing item field - name and type.
+ * Class representing item field - name, type, source type, modifiers.
+ * Type - exposed type of field
+ * Source type - type of field in underlying source
+ * Modifiers - additional attributes which describe type or field
  */
 public static class Field {
 private String name;
-private String type; // TODO: change to enum
+private EnumHawqType type; // field type which PXF exposes
+private String sourceType; // filed type PXF reads from
 private String[] modifiers; // type modifiers, optional field
 
-public Field(String name, String type) {
+public Field(String name, EnumHawqType type) {
 
-if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
-throw new IllegalArgumentException("Field name and type cannot 
be empty");
+if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())) {
+throw new IllegalArgumentException("Field name and type cannot 
be empty.");
 }
 
 this.name = name;
 this.type = type;
 }
 
-public Field(String name, String type, String[] modifiers) {
+public Field(String name, EnumHawqType type, String sourceType) {
 this(name, type);
+this.sourceType = sourceType;
+}
+
+public Field(String name, EnumHawqType type, String[] modifiers) {
+this(name, type);
+this.modifiers = modifiers;
+}
+
+public Field(String name, EnumHawqType type, String sourceType, 
String[] modifiers) {
+this(name, type, sourceType);
 this.modifiers = modifiers;
 }
 
@@ -93,10 +108,14 @@ public class Metadata {
 return name;
 }
 
-public String getType() {
+public EnumHawqType getType() {
 return type;
 }
 
+public String getSourceType() {
+return sourceType;
+}
+
 public String[] getModifiers() {
 return modifiers;
 }

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/7b837239/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-04-28 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 a5aa89377 -> 9d791730a (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/9d791730
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/9d791730
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/9d791730

Branch: refs/heads/HAWQ-703
Commit: 9d791730ae761c01cf9b0eaec453e6410f90
Parents: 649828f
Author: Oleksandr Diachenko 
Authored: Fri Apr 22 16:34:42 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu Apr 28 13:12:16 2016 -0700

--
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  26 -
 .../hawq/pxf/api/utilities/EnumHawqType.java|  77 +
 .../hive/utilities/EnumHiveToHawqTypes.java |  70 
 .../plugins/hive/utilities/HiveUtilities.java   | 109 ++-
 .../plugins/hive/HiveMetadataFetcherTest.java   |  12 +-
 .../hive/utilities/HiveUtilitiesTest.java   |  18 +--
 .../hawq/pxf/service/MetadataResponse.java  |   1 +
 .../pxf/service/MetadataResponseFormatter.java  |   3 +-
 .../service/MetadataResponseFormatterTest.java  |  58 ++
 src/backend/catalog/external/externalmd.c   |   4 +
 src/backend/utils/adt/pxf_functions.c   |   9 +-
 src/bin/psql/describe.c |  32 +-
 src/include/catalog/external/itemmd.h   |   3 +
 src/include/catalog/pg_proc.h   |   4 +-
 src/include/catalog/pg_proc.sql |   2 +-
 15 files changed, 301 insertions(+), 127 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9d791730/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --git a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4fc510d..b5448b9 100644
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@ -23,6 +23,7 @@ package org.apache.hawq.pxf.api;
 import java.util.ArrayList;
 import java.util.List;
 
+import org.apache.hawq.pxf.api.utilities.EnumHawqType;
 import org.apache.commons.lang.StringUtils;
 
 /**
@@ -71,12 +72,13 @@ public class Metadata {
  */
 public static class Field {
 private String name;
-private String type; // TODO: change to enum
+private EnumHawqType type;
+private String sourceType;
 private String[] modifiers; // type modifiers, optional field
 
-public Field(String name, String type) {
+public Field(String name, EnumHawqType type) {
 
-if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
+if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())) {
 throw new IllegalArgumentException("Field name and type cannot 
be empty");
 }
 
@@ -84,8 +86,18 @@ public class Metadata {
 this.type = type;
 }
 
-public Field(String name, String type, String[] modifiers) {
+public Field(String name, EnumHawqType type, String sourceType) {
 this(name, type);
+this.sourceType = sourceType;
+}
+
+public Field(String name, EnumHawqType type, String[] modifiers) {
+this(name, type);
+this.modifiers = modifiers;
+}
+
+public Field(String name, EnumHawqType type, String sourceType, 
String[] modifiers) {
+this(name, type, sourceType);
 this.modifiers = modifiers;
 }
 
@@ -93,10 +105,14 @@ public class Metadata {
 return name;
 }
 
-public String getType() {
+public EnumHawqType getType() {
 return type;
 }
 
+public String getSourceType() {
+return sourceType;
+}
+
 public String[] getModifiers() {
 return modifiers;
 }

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/9d791730/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java
--
diff --git 
a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java
new file mode 100644
index 000..61189d9
--- /dev/null
+++ 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

2016-04-28 Thread odiachenko

Repository: incubator-hawq
Updated Branches:
  refs/heads/HAWQ-703 85eb46337 -> a5aa89377 (forced update)


HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile).


Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq/commit/a5aa8937
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq/tree/a5aa8937
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq/diff/a5aa8937

Branch: refs/heads/HAWQ-703
Commit: a5aa893777cbf26485e3e18d4a69e0a7aae80a2b
Parents: 649828f
Author: Oleksandr Diachenko 
Authored: Fri Apr 22 16:34:42 2016 -0700
Committer: Oleksandr Diachenko 
Committed: Thu Apr 28 13:05:53 2016 -0700

--
 .../java/org/apache/hawq/pxf/api/Metadata.java  |  26 -
 .../hawq/pxf/api/utilities/EnumHawqType.java|  80 ++
 .../hive/utilities/EnumHiveToHawqTypes.java |  70 
 .../plugins/hive/utilities/HiveUtilities.java   | 109 ++-
 .../plugins/hive/HiveMetadataFetcherTest.java   |  12 +-
 .../hive/utilities/HiveUtilitiesTest.java   |  18 +--
 .../hawq/pxf/service/MetadataResponse.java  |   1 +
 .../pxf/service/MetadataResponseFormatter.java  |   3 +-
 .../service/MetadataResponseFormatterTest.java  |  58 ++
 src/backend/catalog/external/externalmd.c   |   4 +
 src/backend/utils/adt/pxf_functions.c   |   9 +-
 src/bin/psql/describe.c |  32 +-
 src/include/catalog/external/itemmd.h   |   3 +
 src/include/catalog/pg_proc.h   |   4 +-
 src/include/catalog/pg_proc.sql |   2 +-
 15 files changed, 304 insertions(+), 127 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/a5aa8937/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
--
diff --git a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
index 4fc510d..b5448b9 100644
--- a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
+++ b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/Metadata.java
@@ -23,6 +23,7 @@ package org.apache.hawq.pxf.api;
 import java.util.ArrayList;
 import java.util.List;
 
+import org.apache.hawq.pxf.api.utilities.EnumHawqType;
 import org.apache.commons.lang.StringUtils;
 
 /**
@@ -71,12 +72,13 @@ public class Metadata {
  */
 public static class Field {
 private String name;
-private String type; // TODO: change to enum
+private EnumHawqType type;
+private String sourceType;
 private String[] modifiers; // type modifiers, optional field
 
-public Field(String name, String type) {
+public Field(String name, EnumHawqType type) {
 
-if (StringUtils.isBlank(name) || StringUtils.isBlank(type)) {
+if (StringUtils.isBlank(name) || 
StringUtils.isBlank(type.getTypeName())) {
 throw new IllegalArgumentException("Field name and type cannot 
be empty");
 }
 
@@ -84,8 +86,18 @@ public class Metadata {
 this.type = type;
 }
 
-public Field(String name, String type, String[] modifiers) {
+public Field(String name, EnumHawqType type, String sourceType) {
 this(name, type);
+this.sourceType = sourceType;
+}
+
+public Field(String name, EnumHawqType type, String[] modifiers) {
+this(name, type);
+this.modifiers = modifiers;
+}
+
+public Field(String name, EnumHawqType type, String sourceType, 
String[] modifiers) {
+this(name, type, sourceType);
 this.modifiers = modifiers;
 }
 
@@ -93,10 +105,14 @@ public class Metadata {
 return name;
 }
 
-public String getType() {
+public EnumHawqType getType() {
 return type;
 }
 
+public String getSourceType() {
+return sourceType;
+}
+
 public String[] getModifiers() {
 return modifiers;
 }

http://git-wip-us.apache.org/repos/asf/incubator-hawq/blob/a5aa8937/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java
--
diff --git 
a/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java
new file mode 100644
index 000..14a3cc5
--- /dev/null
+++ 
b/pxf/pxf-api/src/main/java/org/apache/hawq/pxf/api/utilities/EnumHawqType.java
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

[1/3] incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

incubator-hawq git commit: HAWQ-703. Serialize HCatalog Complex Types to plain text (as Hive profile). [Forced Update!]

13 matches

Site Navigation

Mail list logo

Footer information