[jira] [Updated] (GOBBLIN-1174) Fail job on FileBasedSource ls invalid source directory

2020-06-01 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1174: --- Summary: Fail job on FileBasedSource ls invalid source directory (was: Fail job on FileBas

[jira] [Created] (GOBBLIN-1174) Fail job on FileBasedSource with invalid source directory

2020-06-01 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1174: -- Summary: Fail job on FileBasedSource with invalid source directory Key: GOBBLIN-1174 URL: https://issues.apache.org/jira/browse/GOBBLIN-1174 Project: Apache Gobbli

[jira] [Created] (GOBBLIN-1146) Allow configuring autocommit in JDBCWriters

2020-05-12 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1146: -- Summary: Allow configuring autocommit in JDBCWriters Key: GOBBLIN-1146 URL: https://issues.apache.org/jira/browse/GOBBLIN-1146 Project: Apache Gobblin Is

[jira] [Updated] (GOBBLIN-1142) Hive Distcp support filter on partitioned or snapshot table

2020-05-05 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1142: --- Description: The change adds support filtering a specific type of tables, e.g snapshot, par

[jira] [Updated] (GOBBLIN-1142) Hive Distcp support filter on partitioned or snapshot table

2020-05-05 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1142: --- Description: The change adds support filtering a specific type of tables, e.g snapshot, par

[jira] [Created] (GOBBLIN-1142) Hive Distcp support filter on partitioned or snapshot table

2020-05-05 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1142: -- Summary: Hive Distcp support filter on partitioned or snapshot table Key: GOBBLIN-1142 URL: https://issues.apache.org/jira/browse/GOBBLIN-1142 Project: Apache Gob

[jira] [Created] (GOBBLIN-1096) Work with DST change in compaction watermark

2020-03-24 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1096: -- Summary: Work with DST change in compaction watermark Key: GOBBLIN-1096 URL: https://issues.apache.org/jira/browse/GOBBLIN-1096 Project: Apache Gobblin I

[jira] [Updated] (GOBBLIN-1066) field projection with namespace

2020-02-28 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1066: --- Description: `AvroProjectionConverter` currently ignores extract namespace to identify fiel

[jira] [Created] (GOBBLIN-1066) field projection with namespace

2020-02-28 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1066: -- Summary: field projection with namespace Key: GOBBLIN-1066 URL: https://issues.apache.org/jira/browse/GOBBLIN-1066 Project: Apache Gobblin Issue Type: Ta

[jira] [Updated] (GOBBLIN-1056) Allow customizing client pool population in KafkaSource

2020-02-20 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1056: --- Description: Put existing logic of consumer client pool population into method `populateCli

[jira] [Updated] (GOBBLIN-1056) Allow customizing client pool population in KafkaSource

2020-02-20 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1056: --- Summary: Allow customizing client pool population in KafkaSource (was: Allow customizing c

[jira] [Updated] (GOBBLIN-1056) Allow customizing client pool creation in KafkaSource

2020-02-20 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1056: --- Description: Put existing logic of consumer client pool population > Allow customizing clie

[jira] [Updated] (GOBBLIN-1056) Allow customizing client pool creation in KafkaSource

2020-02-20 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1056: --- Summary: Allow customizing client pool creation in KafkaSource (was: Allow customize clien

[jira] [Created] (GOBBLIN-1056) Allow customize client pool creation in KafkaSource

2020-02-20 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1056: -- Summary: Allow customize client pool creation in KafkaSource Key: GOBBLIN-1056 URL: https://issues.apache.org/jira/browse/GOBBLIN-1056 Project: Apache Gobblin

[jira] [Updated] (GOBBLIN-1056) Allow customize client pool creation in KafkaSource

2020-02-20 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1056: --- Description: (was: StandardManifestRecord: A standard representation of a record from a

[jira] [Commented] (GOBBLIN-1045) Emit more events in compaction job

2020-02-12 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035769#comment-17035769 ] Zhixiong Chen commented on GOBBLIN-1045: Generalize it in `MRCompactionTask`? -

[jira] [Updated] (GOBBLIN-1045) Emit more events in compaction job

2020-02-10 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1045: --- Description: Emit count event for the following item in compaction job - number of files, c

[jira] [Updated] (GOBBLIN-1045) Emit more events in compaction job

2020-02-10 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1045: --- Summary: Emit more events in compaction job (was: Emit events for hive metadata in compact

[jira] [Updated] (GOBBLIN-1045) Emit events for hive metadata in compaction job

2020-02-10 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1045: --- Description: Emit count event for the following hive metadata in a compaction job - numFile

[jira] [Created] (GOBBLIN-1045) Emit events for hive metadata in compaction job

2020-02-10 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1045: -- Summary: Emit events for hive metadata in compaction job Key: GOBBLIN-1045 URL: https://issues.apache.org/jira/browse/GOBBLIN-1045 Project: Apache Gobblin

[jira] [Updated] (GOBBLIN-1012) Implement CompactionWithWatermarkSuite

2020-01-02 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1012: --- Description: - A `compactionWatermark` is a timestamp indicating the data we've seen up to

[jira] [Updated] (GOBBLIN-1012) Implement CompactionWithWatermarkSuite

2020-01-02 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1012: --- Description: `CompactionWithWatermarkSuite` will report compaction watermark as part of the

[jira] [Created] (GOBBLIN-1012) Implement CompactionWithWatermarkSuite

2020-01-02 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1012: -- Summary: Implement CompactionWithWatermarkSuite Key: GOBBLIN-1012 URL: https://issues.apache.org/jira/browse/GOBBLIN-1012 Project: Apache Gobblin Issue T

[jira] [Updated] (GOBBLIN-1011) Adjust compaction flow to work with virtual partition

2019-12-20 Thread Zhixiong Chen (Jira)
[ https://issues.apache.org/jira/browse/GOBBLIN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-1011: --- Description: - Update existing `CompactionVerifier`s and `CompactionCompleteAction`s to wor

[jira] [Created] (GOBBLIN-1011) Adjust compaction flow to work with virtual partition

2019-12-20 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1011: -- Summary: Adjust compaction flow to work with virtual partition Key: GOBBLIN-1011 URL: https://issues.apache.org/jira/browse/GOBBLIN-1011 Project: Apache Gobblin

[jira] [Created] (GOBBLIN-1001) Implement TimePartitionGlobFinder

2019-12-10 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-1001: -- Summary: Implement TimePartitionGlobFinder Key: GOBBLIN-1001 URL: https://issues.apache.org/jira/browse/GOBBLIN-1001 Project: Apache Gobblin Issue Type:

[jira] [Created] (GOBBLIN-993) Support job level hive configuration override

2019-12-04 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-993: - Summary: Support job level hive configuration override Key: GOBBLIN-993 URL: https://issues.apache.org/jira/browse/GOBBLIN-993 Project: Apache Gobblin Issu

[jira] [Created] (GOBBLIN-896) Clone schema or field props in AvroFieldRemover

2019-10-03 Thread Zhixiong Chen (Jira)
Zhixiong Chen created GOBBLIN-896: - Summary: Clone schema or field props in AvroFieldRemover Key: GOBBLIN-896 URL: https://issues.apache.org/jira/browse/GOBBLIN-896 Project: Apache Gobblin Is

[jira] [Created] (GOBBLIN-831) Fix NPE in KafkaWorkUnitPacker when there is no WorkUnit created

2019-07-18 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-831: - Summary: Fix NPE in KafkaWorkUnitPacker when there is no WorkUnit created Key: GOBBLIN-831 URL: https://issues.apache.org/jira/browse/GOBBLIN-831 Project: Apache Go

[jira] [Created] (GOBBLIN-827) Add more events

2019-07-15 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-827: - Summary: Add more events Key: GOBBLIN-827 URL: https://issues.apache.org/jira/browse/GOBBLIN-827 Project: Apache Gobblin Issue Type: Task Repor

[jira] [Updated] (GOBBLIN-769) Support string record timestamp in TimeBasedAvroWriterPartitioner

2019-05-13 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-769: -- Summary: Support string record timestamp in TimeBasedAvroWriterPartitioner (was: Support reco

[jira] [Updated] (GOBBLIN-769) Support string record timestamp in TimeBasedAvroWriterPartitioner

2019-05-13 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-769: -- Description: Currently, if a record timestamp is a string, `TimeBasedAvroWriterPartitioner` wi

[jira] [Created] (GOBBLIN-769) Support record timestamp as string in TimeBasedWriterPartitioner

2019-05-13 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-769: - Summary: Support record timestamp as string in TimeBasedWriterPartitioner Key: GOBBLIN-769 URL: https://issues.apache.org/jira/browse/GOBBLIN-769 Project: Apache Go

[jira] [Updated] (GOBBLIN-767) Support different time units in TimeBasedWriterPartitioner

2019-05-10 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-767: -- Description: Currently, `TimeBasedWriterPartitioner` assumes the timestamp value from a record

[jira] [Created] (GOBBLIN-767) Support different time units in TimeBasedWriterPartitioner

2019-05-10 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-767: - Summary: Support different time units in TimeBasedWriterPartitioner Key: GOBBLIN-767 URL: https://issues.apache.org/jira/browse/GOBBLIN-767 Project: Apache Gobblin

[jira] [Updated] (GOBBLIN-763) Fix incorrect AvroUtils.removeUncomparableFields implementation

2019-05-02 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-763: -- Description: - Remove fields, specified by configuration `compaction.job.key.fieldBlacklist`,

[jira] [Updated] (GOBBLIN-763) Support fields removal for compaction dedup key schema

2019-05-02 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-763: -- Summary: Support fields removal for compaction dedup key schema (was: Fix incorrect AvroUtils

[jira] [Updated] (GOBBLIN-763) Fix incorrect AvroUtils.removeUncomparableFields implementation

2019-05-02 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-763: -- Description: Currently, `AvroUtils.removeUncomparableFields` will only keep the first field of

[jira] [Created] (GOBBLIN-763) Fix incorrect removeUncomparableFields implementation in AvroUtils

2019-05-02 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-763: - Summary: Fix incorrect removeUncomparableFields implementation in AvroUtils Key: GOBBLIN-763 URL: https://issues.apache.org/jira/browse/GOBBLIN-763 Project: Apache

[jira] [Updated] (GOBBLIN-763) Fix incorrect AvroUtils.removeUncomparableFields implementation

2019-05-02 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-763: -- Summary: Fix incorrect AvroUtils.removeUncomparableFields implementation (was: Fix incorrect

[jira] [Updated] (GOBBLIN-763) Fix incorrect removeUncomparableFields implementation in AvroUtils

2019-05-02 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-763: -- Description: Currently, (was: StandardManifestRecord: A standard representation of a record

[jira] [Created] (GOBBLIN-738) Open a way to customize decoding KafkaConsumerRecord

2019-04-15 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-738: - Summary: Open a way to customize decoding KafkaConsumerRecord Key: GOBBLIN-738 URL: https://issues.apache.org/jira/browse/GOBBLIN-738 Project: Apache Gobblin

[jira] [Updated] (GOBBLIN-716) Add lineage in FileBasedSource

2019-03-27 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-716: -- Description: Add lineage in `FileBasedSource` - By default, `FileBasedSource` marks dataset le

[jira] [Updated] (GOBBLIN-716) Add lineage in FileBasedSource

2019-03-27 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-716: -- Summary: Add lineage in FileBasedSource (was: Add FileBasedSource lineage event) > Add linea

[jira] [Updated] (GOBBLIN-716) Add FileBasedSource lineage event

2019-03-27 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-716: -- Description: (was: It'd be useful to support configuration properties to override the defa

[jira] [Created] (GOBBLIN-716) Add FileBasedSource lineage event

2019-03-27 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-716: - Summary: Add FileBasedSource lineage event Key: GOBBLIN-716 URL: https://issues.apache.org/jira/browse/GOBBLIN-716 Project: Apache Gobblin Issue Type: Bug

[jira] [Updated] (GOBBLIN-716) Add FileBasedSource lineage event

2019-03-27 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-716: -- Issue Type: Task (was: Bug) > Add FileBasedSource lineage event > ---

[jira] [Updated] (GOBBLIN-716) Add FileBasedSource lineage event

2019-03-27 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-716: -- External issue URL: (was: https://github.com/linkedin/gobblin/issues/1904) > Add FileBasedSo

[jira] [Updated] (GOBBLIN-543) Opensource StandardManifestRecord and DistributedClasspathManager

2018-12-21 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-543: -- Description: StandardManifestRecord: A standardized record that represents an input record fo

[jira] [Updated] (GOBBLIN-543) Opensource StandardManifestRecord and DistributedClasspathManager

2018-12-21 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-543: -- Description: StandardManifestRecord: A standard representation of a record from a service tha

[jira] [Updated] (GOBBLIN-543) Opensource StandardManifestRecord and DistributedClasspathManager

2018-12-21 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-543: -- Summary: Opensource StandardManifestRecord and DistributedClasspathManager (was: Opensource D

[jira] [Updated] (GOBBLIN-543) Opensource distributedClasspathManager

2018-12-21 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-543: -- Summary: Opensource distributedClasspathManager (was: DistributedClasspathManager) > Opensou

[jira] [Updated] (GOBBLIN-543) Opensource DistributedClasspathManager

2018-12-21 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-543: -- Summary: Opensource DistributedClasspathManager (was: Opensource distributedClasspathManager)

[jira] [Updated] (GOBBLIN-621) Add utilities

2018-10-26 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-621: -- Description: - `TestIOUtils.readAllRecords`: read json data as `GenericRecord` with avro sche

[jira] [Created] (GOBBLIN-621) Add utilities

2018-10-26 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-621: - Summary: Add utilities Key: GOBBLIN-621 URL: https://issues.apache.org/jira/browse/GOBBLIN-621 Project: Apache Gobblin Issue Type: Task Reporte

[jira] [Closed] (GOBBLIN-565) Implement partition level lineage event for job using TimePartitionedDataPublisher

2018-10-26 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen closed GOBBLIN-565. - Resolution: Fixed > Implement partition level lineage event for job using > TimePartitionedData

[jira] [Updated] (GOBBLIN-587) Implement partition level lineage for fs based destination

2018-09-12 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-587: -- Summary: Implement partition level lineage for fs based destination (was: Implement gobblin f

[jira] [Updated] (GOBBLIN-587) Implement gobblin fs sink partition level lineage

2018-09-12 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-587: -- Description: Currently, gobblin lineage is sent at dataset level. The task is to send partiti

[jira] [Updated] (GOBBLIN-587) Implement gobblin fs sink partition level lineage

2018-09-12 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-587: -- Description: Currently, gobblin lineage is sent at dataset level. The task is to send partiti

[jira] [Created] (GOBBLIN-587) Implement gobblin fs sink partition level lineage

2018-09-12 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-587: - Summary: Implement gobblin fs sink partition level lineage Key: GOBBLIN-587 URL: https://issues.apache.org/jira/browse/GOBBLIN-587 Project: Apache Gobblin

[jira] [Updated] (GOBBLIN-576) Send partition level lineage in hive distcp

2018-09-05 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-576: -- Description: Currently hive distcp only supports dataset/table level lineage. The task is to s

[jira] [Created] (GOBBLIN-576) Send partition level lineage in hive distcp

2018-09-05 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-576: - Summary: Send partition level lineage in hive distcp Key: GOBBLIN-576 URL: https://issues.apache.org/jira/browse/GOBBLIN-576 Project: Apache Gobblin Issue

[jira] [Created] (GOBBLIN-565) Implement partition level lineage event for job using TimePartitionedDataPublisher

2018-08-15 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-565: - Summary: Implement partition level lineage event for job using TimePartitionedDataPublisher Key: GOBBLIN-565 URL: https://issues.apache.org/jira/browse/GOBBLIN-565

[jira] [Created] (GOBBLIN-564) Implement partition descriptor

2018-08-15 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-564: - Summary: Implement partition descriptor Key: GOBBLIN-564 URL: https://issues.apache.org/jira/browse/GOBBLIN-564 Project: Apache Gobblin Issue Type: Task

[jira] [Created] (GOBBLIN-543) DistributedClasspathManager

2018-07-26 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-543: - Summary: DistributedClasspathManager Key: GOBBLIN-543 URL: https://issues.apache.org/jira/browse/GOBBLIN-543 Project: Apache Gobblin Issue Type: Task

[jira] [Created] (GOBBLIN-517) Add missing apache license info

2018-06-26 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-517: - Summary: Add missing apache license info Key: GOBBLIN-517 URL: https://issues.apache.org/jira/browse/GOBBLIN-517 Project: Apache Gobblin Issue Type: Bug

[jira] [Updated] (GOBBLIN-489) Implement PusherFactory

2018-06-11 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-489: -- Summary: Implement PusherFactory (was: Create PusherFactory) > Implement PusherFactory >

[jira] [Updated] (GOBBLIN-489) Implement PusherFactory

2018-06-11 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-489: -- Description: A `PusherFactory` creates a `Pusher`. Changes are: * `PusherFactory` and gobblin

[jira] [Updated] (GOBBLIN-489) Create PusherFactory

2018-06-11 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-489: -- Description: An `PusherFactory` creates a `Pusher`. Changes are: * `PusherFactory` and gobbli

[jira] [Updated] (GOBBLIN-489) Create PusherFactory

2018-06-11 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-489: -- Summary: Create PusherFactory (was: Create general EventProducer with a Pusher) > Create Pus

[jira] [Created] (GOBBLIN-501) Fix NPE thrown from read after EOF of LazyMaterializeDecryptorInputStream

2018-05-24 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-501: - Summary: Fix NPE thrown from read after EOF of LazyMaterializeDecryptorInputStream Key: GOBBLIN-501 URL: https://issues.apache.org/jira/browse/GOBBLIN-501 Project:

[jira] [Created] (GOBBLIN-493) Fix build issue in GithubDataEventTypesPartitioner

2018-05-15 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-493: - Summary: Fix build issue in GithubDataEventTypesPartitioner Key: GOBBLIN-493 URL: https://issues.apache.org/jira/browse/GOBBLIN-493 Project: Apache Gobblin

[jira] [Assigned] (GOBBLIN-489) Create general EventProducer with a Pusher

2018-05-08 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen reassigned GOBBLIN-489: - Assignee: Zhixiong Chen > Create general EventProducer with a Pusher > -

[jira] [Created] (GOBBLIN-489) Create general EventProducer with a Pusher

2018-05-08 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-489: - Summary: Create general EventProducer with a Pusher Key: GOBBLIN-489 URL: https://issues.apache.org/jira/browse/GOBBLIN-489 Project: Apache Gobblin Issue T

[jira] [Created] (GOBBLIN-488) Make `AsyncRequest` aware of records

2018-05-04 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-488: - Summary: Make `AsyncRequest` aware of records Key: GOBBLIN-488 URL: https://issues.apache.org/jira/browse/GOBBLIN-488 Project: Apache Gobblin Issue Type: T

[jira] [Created] (GOBBLIN-487) Integrate PasswordManager in R2RestWriterBuilder

2018-05-04 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-487: - Summary: Integrate PasswordManager in R2RestWriterBuilder Key: GOBBLIN-487 URL: https://issues.apache.org/jira/browse/GOBBLIN-487 Project: Apache Gobblin

[jira] [Updated] (GOBBLIN-482) Add http write documentation

2018-04-30 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-482: -- Description: {color:#33}The old http write framework under `AbstractHttpWriter` and `Abstra

[jira] [Created] (GOBBLIN-482) Add http write documentation

2018-04-30 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-482: - Summary: Add http write documentation Key: GOBBLIN-482 URL: https://issues.apache.org/jira/browse/GOBBLIN-482 Project: Apache Gobblin Issue Type: Task

[jira] [Updated] (GOBBLIN-482) Add http write documentation

2018-04-30 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-482: -- Description: The old http write framework under [AbstractHttpWriter|{color:#ffc66d}[https://gi

[jira] [Created] (GOBBLIN-442) Add lineage for mysql sqlserver postgresql

2018-03-23 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-442: - Summary: Add lineage for mysql sqlserver postgresql Key: GOBBLIN-442 URL: https://issues.apache.org/jira/browse/GOBBLIN-442 Project: Apache Gobblin Issue T

[jira] [Created] (GOBBLIN-435) Fix data publisher created from job broker not closed

2018-03-22 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-435: - Summary: Fix data publisher created from job broker not closed Key: GOBBLIN-435 URL: https://issues.apache.org/jira/browse/GOBBLIN-435 Project: Apache Gobblin

[jira] [Updated] (GOBBLIN-430) Add lineage in SalesforceSource

2018-03-20 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-430: -- Description: - Set source lineage info into work units generated by `SalesforceSource`   - Full

[jira] [Updated] (GOBBLIN-430) Add lineage in SalesforceSource

2018-03-20 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-430: -- Summary: Add lineage in SalesforceSource (was: Add lineage for salesforce source) > Add linea

[jira] [Created] (GOBBLIN-430) Add lineage for salesforce source

2018-03-20 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-430: - Summary: Add lineage for salesforce source Key: GOBBLIN-430 URL: https://issues.apache.org/jira/browse/GOBBLIN-430 Project: Apache Gobblin Issue Type: Task

[jira] [Created] (GOBBLIN-395) Add lineage for copying config based dataset

2018-01-30 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-395: - Summary: Add lineage for copying config based dataset Key: GOBBLIN-395 URL: https://issues.apache.org/jira/browse/GOBBLIN-395 Project: Apache Gobblin Issue

[jira] [Closed] (GOBBLIN-374) GobblinMetrics failed to close event reporters

2018-01-30 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen closed GOBBLIN-374. - Resolution: Fixed > GobblinMetrics failed to close event reporters >

[jira] [Closed] (GOBBLIN-380) Add log about time elapsed for waiting services to be healthy

2018-01-30 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen closed GOBBLIN-380. - Resolution: Fixed > Add log about time elapsed for waiting services to be healthy > -

[jira] [Created] (GOBBLIN-380) Add log about time elapsed for waiting services to be healthy

2018-01-18 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-380: - Summary: Add log about time elapsed for waiting services to be healthy Key: GOBBLIN-380 URL: https://issues.apache.org/jira/browse/GOBBLIN-380 Project: Apache Gobbl

[jira] [Created] (GOBBLIN-374) GobblinMetrics failed to close event reporters

2018-01-16 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-374: - Summary: GobblinMetrics failed to close event reporters Key: GOBBLIN-374 URL: https://issues.apache.org/jira/browse/GOBBLIN-374 Project: Apache Gobblin Iss

[jira] [Assigned] (GOBBLIN-374) GobblinMetrics failed to close event reporters

2018-01-16 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen reassigned GOBBLIN-374: - Assignee: Zhixiong Chen > GobblinMetrics failed to close event reporters > -

[jira] [Created] (GOBBLIN-364) Exclude JobState from WorkUnit created by PartitionedFileSourceBase

2018-01-10 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-364: - Summary: Exclude JobState from WorkUnit created by PartitionedFileSourceBase Key: GOBBLIN-364 URL: https://issues.apache.org/jira/browse/GOBBLIN-364 Project: Apache

[jira] [Updated] (GOBBLIN-354) Support DynamicConfig in AzkabanCompactionJobLauncher

2018-01-02 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-354: -- Description: Allow old compaction job to accept dynamic configurations > Support DynamicConfig

[jira] [Created] (GOBBLIN-354) Support DynamicConfig in AzkabanCompactionJobLauncher

2018-01-02 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-354: - Summary: Support DynamicConfig in AzkabanCompactionJobLauncher Key: GOBBLIN-354 URL: https://issues.apache.org/jira/browse/GOBBLIN-354 Project: Apache Gobblin

[jira] [Created] (GOBBLIN-353) Fix low watermark overridden by high watermark in SalesforceSource

2018-01-02 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-353: - Summary: Fix low watermark overridden by high watermark in SalesforceSource Key: GOBBLIN-353 URL: https://issues.apache.org/jira/browse/GOBBLIN-353 Project: Apache

[jira] [Updated] (GOBBLIN-344) Fix help method getResolver in LineageInfo is private

2017-12-11 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-344: -- Summary: Fix help method getResolver in LineageInfo is private (was: Fix getResolver help meth

[jira] [Created] (GOBBLIN-344) Fix getResolver help method in LineageInfo is private

2017-12-11 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-344: - Summary: Fix getResolver help method in LineageInfo is private Key: GOBBLIN-344 URL: https://issues.apache.org/jira/browse/GOBBLIN-344 Project: Apache Gobblin

[jira] [Created] (GOBBLIN-334) Implement SharedResourceFactory for LineageInfo

2017-12-05 Thread Zhixiong Chen (JIRA)
Zhixiong Chen created GOBBLIN-334: - Summary: Implement SharedResourceFactory for LineageInfo Key: GOBBLIN-334 URL: https://issues.apache.org/jira/browse/GOBBLIN-334 Project: Apache Gobblin Is

[jira] [Updated] (GOBBLIN-319) Add DatasetResolver to transform raw Gobblin dataset to application specific dataset

2017-11-28 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-319: -- Description: - Add DatasetResolver to transform raw Gobblin dataset to application specific da

[jira] [Updated] (GOBBLIN-319) Add DatasetResolver to transform raw Gobblin dataset to application specific dataset

2017-11-28 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-319: -- Summary: Add DatasetResolver to transform raw Gobblin dataset to application specific dataset

[jira] [Updated] (GOBBLIN-319) Add exampleDataDir when sending file system based dataset lineage

2017-11-27 Thread Zhixiong Chen (JIRA)
[ https://issues.apache.org/jira/browse/GOBBLIN-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated GOBBLIN-319: -- Summary: Add exampleDataDir when sending file system based dataset lineage (was: Lineage even

  1   2   >