[jira] [Commented] (BEAM-12) Apply GroupByKey transforms on PCollection of normal type other than KV

2016-02-14 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-12?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146657#comment-15146657 ] Daniel Halperin commented on BEAM-12: - Hi, I'm assuming that `ExtractFn` is some sort of

[jira] [Created] (BEAM-45) Unbounded source for Google Cloud Bigtable

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-45: --- Summary: Unbounded source for Google Cloud Bigtable Key: BEAM-45 URL: https://issues.apache.org/jira/browse/BEAM-45 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-46) Unbounded sink for Google Cloud Bigtable

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-46: --- Summary: Unbounded sink for Google Cloud Bigtable Key: BEAM-46 URL: https://issues.apache.org/jira/browse/BEAM-46 Project: Beam Issue Type: New Feature

[jira] [Updated] (BEAM-46) Unbounded sink for Google Cloud Bigtable

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-46: Assignee: (was: Daniel Halperin) > Unbounded sink for Google Cloud Bigtable > -

[jira] [Updated] (BEAM-45) Unbounded source for Google Cloud Bigtable

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-45?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-45: Assignee: (was: Daniel Halperin) > Unbounded source for Google Cloud Bigtable > ---

[jira] [Created] (BEAM-47) UnboundedSource should support dynamic work rebalancing

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-47: --- Summary: UnboundedSource should support dynamic work rebalancing Key: BEAM-47 URL: https://issues.apache.org/jira/browse/BEAM-47 Project: Beam Issue Type: New

[jira] [Created] (BEAM-48) BigQueryIO.Read reimplemented as BoundedSource

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-48: --- Summary: BigQueryIO.Read reimplemented as BoundedSource Key: BEAM-48 URL: https://issues.apache.org/jira/browse/BEAM-48 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-49) BigQueryIO: support BYTES

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-49: --- Summary: BigQueryIO: support BYTES Key: BEAM-49 URL: https://issues.apache.org/jira/browse/BEAM-49 Project: Beam Issue Type: New Feature Components:

[jira] [Created] (BEAM-50) BigQueryIO.Write: reimplement in Java

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-50: --- Summary: BigQueryIO.Write: reimplement in Java Key: BEAM-50 URL: https://issues.apache.org/jira/browse/BEAM-50 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-51) Implement a CSV file reader

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-51: --- Summary: Implement a CSV file reader Key: BEAM-51 URL: https://issues.apache.org/jira/browse/BEAM-51 Project: Beam Issue Type: New Feature Components

[jira] [Created] (BEAM-52) KafkaIO - bounded/unbounded, source/sink

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-52: --- Summary: KafkaIO - bounded/unbounded, source/sink Key: BEAM-52 URL: https://issues.apache.org/jira/browse/BEAM-52 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-54) PubSubIO: add validation of topic/subscription

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-54: --- Summary: PubSubIO: add validation of topic/subscription Key: BEAM-54 URL: https://issues.apache.org/jira/browse/BEAM-54 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-53) PubSubIO: reimplement in Java

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-53: --- Summary: PubSubIO: reimplement in Java Key: BEAM-53 URL: https://issues.apache.org/jira/browse/BEAM-53 Project: Beam Issue Type: New Feature Componen

[jira] [Created] (BEAM-55) Allow users to compress FileBasedSink output files

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-55: --- Summary: Allow users to compress FileBasedSink output files Key: BEAM-55 URL: https://issues.apache.org/jira/browse/BEAM-55 Project: Beam Issue Type: New Featu

[jira] [Created] (BEAM-56) Support ZIP compression in CompressedSource

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-56: --- Summary: Support ZIP compression in CompressedSource Key: BEAM-56 URL: https://issues.apache.org/jira/browse/BEAM-56 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-57) FileBasedSink should write intermediate output to a different folder

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-57: --- Summary: FileBasedSink should write intermediate output to a different folder Key: BEAM-57 URL: https://issues.apache.org/jira/browse/BEAM-57 Project: Beam Is

[jira] [Commented] (BEAM-52) KafkaIO - bounded/unbounded, source/sink

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167336#comment-15167336 ] Daniel Halperin commented on BEAM-52: - +1 [~amitsela]! In-progress work against Kafka 0.

[jira] [Created] (BEAM-58) Support Google Cloud Storage encryption keys

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-58: --- Summary: Support Google Cloud Storage encryption keys Key: BEAM-58 URL: https://issues.apache.org/jira/browse/BEAM-58 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-59) IOChannelFactory rethinking/redesign

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-59: --- Summary: IOChannelFactory rethinking/redesign Key: BEAM-59 URL: https://issues.apache.org/jira/browse/BEAM-59 Project: Beam Issue Type: New Feature C

[jira] [Created] (BEAM-60) FileBasedSource/IOChannelFactory: Custom glob expansion

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-60: --- Summary: FileBasedSource/IOChannelFactory: Custom glob expansion Key: BEAM-60 URL: https://issues.apache.org/jira/browse/BEAM-60 Project: Beam Issue Type: New

[jira] [Created] (BEAM-62) DatastoreIO: upgrade error handling

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-62: --- Summary: DatastoreIO: upgrade error handling Key: BEAM-62 URL: https://issues.apache.org/jira/browse/BEAM-62 Project: Beam Issue Type: New Feature Co

[jira] [Created] (BEAM-63) DatastoreIO.Read: dynamic work rebalancing

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-63: --- Summary: DatastoreIO.Read: dynamic work rebalancing Key: BEAM-63 URL: https://issues.apache.org/jira/browse/BEAM-63 Project: Beam Issue Type: New Feature

[jira] [Assigned] (BEAM-60) FileBasedSource/IOChannelFactory: Custom glob expansion

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin reassigned BEAM-60: --- Assignee: Daniel Halperin (was: Davor Bonaci) > FileBasedSource/IOChannelFactory: Custom glob e

[jira] [Updated] (BEAM-60) FileBasedSource/IOChannelFactory: Custom glob expansion

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-60: Assignee: (was: Daniel Halperin) > FileBasedSource/IOChannelFactory: Custom glob expansion > --

[jira] [Updated] (BEAM-51) Implement a CSV file reader

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-51?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-51: Assignee: (was: James Malone) > Implement a CSV file reader > --- > >

[jira] [Updated] (BEAM-57) FileBasedSink should write intermediate output to a different folder

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-57?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-57: Assignee: (was: Davor Bonaci) > FileBasedSink should write intermediate output to a different folde

[jira] [Updated] (BEAM-55) Allow users to compress FileBasedSink output files

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-55?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-55: Assignee: (was: Davor Bonaci) > Allow users to compress FileBasedSink output files > --

[jira] [Updated] (BEAM-56) Support ZIP compression in CompressedSource

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-56?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-56: Assignee: (was: Davor Bonaci) > Support ZIP compression in CompressedSource > -

[jira] [Created] (BEAM-61) DatastoreIO: unbounded sink

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-61: --- Summary: DatastoreIO: unbounded sink Key: BEAM-61 URL: https://issues.apache.org/jira/browse/BEAM-61 Project: Beam Issue Type: New Feature Components

[jira] [Created] (BEAM-64) General decompression registry

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-64: --- Summary: General decompression registry Key: BEAM-64 URL: https://issues.apache.org/jira/browse/BEAM-64 Project: Beam Issue Type: New Feature Compone

[jira] [Created] (BEAM-65) SplittableDoFn

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-65: --- Summary: SplittableDoFn Key: BEAM-65 URL: https://issues.apache.org/jira/browse/BEAM-65 Project: Beam Issue Type: New Feature Components: beam-model

[jira] [Created] (BEAM-66) Unbounded source for directory/file scanning

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-66: --- Summary: Unbounded source for directory/file scanning Key: BEAM-66 URL: https://issues.apache.org/jira/browse/BEAM-66 Project: Beam Issue Type: New Feature

[jira] [Updated] (BEAM-66) Unbounded source for directory/file scanning

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-66?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-66: Assignee: (was: Daniel Halperin) > Unbounded source for directory/file scanning > -

[jira] [Created] (BEAM-67) Apache Sqoop connector

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-67: --- Summary: Apache Sqoop connector Key: BEAM-67 URL: https://issues.apache.org/jira/browse/BEAM-67 Project: Beam Issue Type: New Feature Components: sdk

[jira] [Assigned] (BEAM-66) Unbounded source for directory/file scanning

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-66?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin reassigned BEAM-66: --- Assignee: Daniel Halperin (was: Davor Bonaci) > Unbounded source for directory/file scanning >

[jira] [Created] (BEAM-68) Support for limiting parallelism of a step

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-68: --- Summary: Support for limiting parallelism of a step Key: BEAM-68 URL: https://issues.apache.org/jira/browse/BEAM-68 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-69) TextIO/AvroIO numShards may not work in different runners

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-69: --- Summary: TextIO/AvroIO numShards may not work in different runners Key: BEAM-69 URL: https://issues.apache.org/jira/browse/BEAM-69 Project: Beam Issue Type: Ne

[jira] [Created] (BEAM-70) More expressive UnboundedSource hierarchy and utilities

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-70: --- Summary: More expressive UnboundedSource hierarchy and utilities Key: BEAM-70 URL: https://issues.apache.org/jira/browse/BEAM-70 Project: Beam Issue Type: New

[jira] [Commented] (BEAM-70) More expressive UnboundedSource hierarchy and utilities

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-70?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167458#comment-15167458 ] Daniel Halperin commented on BEAM-70: - see also BEAM-71 > More expressive UnboundedSourc

[jira] [Created] (BEAM-71) Watermark library

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-71: --- Summary: Watermark library Key: BEAM-71 URL: https://issues.apache.org/jira/browse/BEAM-71 Project: Beam Issue Type: New Feature Components: sdk-java

[jira] [Created] (BEAM-72) Source design pattern: option to retain metadata

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-72: --- Summary: Source design pattern: option to retain metadata Key: BEAM-72 URL: https://issues.apache.org/jira/browse/BEAM-72 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-73) IO design pattern: Decouple Parsers and Coders

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-73: --- Summary: IO design pattern: Decouple Parsers and Coders Key: BEAM-73 URL: https://issues.apache.org/jira/browse/BEAM-73 Project: Beam Issue Type: New Feature

[jira] [Updated] (BEAM-73) IO design pattern: Decouple Parsers and Coders

2016-02-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-73: Description: Many Sources can be thought of as providing a byte[] payload -- e.g. TextIO bytes between

[jira] [Created] (BEAM-74) Testing harness for Unbounded Sources

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-74: --- Summary: Testing harness for Unbounded Sources Key: BEAM-74 URL: https://issues.apache.org/jira/browse/BEAM-74 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-75) Running tests of dynamic work rebalancing

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-75: --- Summary: Running tests of dynamic work rebalancing Key: BEAM-75 URL: https://issues.apache.org/jira/browse/BEAM-75 Project: Beam Issue Type: New Feature

[jira] [Created] (BEAM-76) FileBasedSource should take a list of files/globs

2016-02-25 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-76: --- Summary: FileBasedSource should take a list of files/globs Key: BEAM-76 URL: https://issues.apache.org/jira/browse/BEAM-76 Project: Beam Issue Type: New Featur

[jira] [Comment Edited] (BEAM-53) PubSubIO: reimplement in Java

2016-03-01 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174266#comment-15174266 ] Daniel Halperin edited comment on BEAM-53 at 3/1/16 7:31 PM: - Thi

[jira] [Commented] (BEAM-50) BigQueryIO.Write: reimplement in Java

2016-03-01 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174264#comment-15174264 ] Daniel Halperin commented on BEAM-50: - This BEAM issue subsumes https://github.com/Googl

[jira] [Commented] (BEAM-53) PubSubIO: reimplement in Java

2016-03-01 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174266#comment-15174266 ] Daniel Halperin commented on BEAM-53: - This BEAM issue subsumes https://github.com/Googl

[jira] [Created] (BEAM-85) DataflowAssert (BeamAssert ;) needs sanity check that it's used correctly

2016-03-01 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-85: --- Summary: DataflowAssert (BeamAssert ;) needs sanity check that it's used correctly Key: BEAM-85 URL: https://issues.apache.org/jira/browse/BEAM-85 Project: Beam

[jira] [Updated] (BEAM-85) DataflowAssert (BeamAssert ;) needs sanity check that it's used correctly

2016-03-02 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-85: Description: It is important that assert is applied to pipeline before the pipeline is run, otherwise

[jira] [Created] (BEAM-86) CountingSource should expose PTransform> rather than sources directly

2016-03-02 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-86: --- Summary: CountingSource should expose PTransform> rather than sources directly Key: BEAM-86 URL: https://issues.apache.org/jira/browse/BEAM-86 Project: Beam Is

[jira] [Commented] (BEAM-92) Data-dependent sinks

2016-03-03 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178173#comment-15178173 ] Daniel Halperin commented on BEAM-92: - Abstractly, this is very easy to accomplish when t

[jira] [Updated] (BEAM-92) Data-dependent sinks

2016-03-03 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-92: Component/s: beam-model > Data-dependent sinks > > > Key: BEAM-92

[jira] [Commented] (BEAM-91) Retractions

2016-03-10 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1513#comment-1513 ] Daniel Halperin commented on BEAM-91: - This is the BEAM version of https://github.com/Go

[jira] [Commented] (BEAM-92) Data-dependent sinks

2016-03-10 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1518#comment-1518 ] Daniel Halperin commented on BEAM-92: - Also duping GitHub https://github.com/GoogleCloud

[jira] [Commented] (BEAM-123) Skip header row from csv

2016-03-19 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15199112#comment-15199112 ] Daniel Halperin commented on BEAM-123: -- Hi Davin, Which implementation of a CSV reader

[jira] [Updated] (BEAM-125) PubSubIO.Read should be implemented by an UnboundedSource

2016-03-19 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-125: - Component/s: (was: runner-core) sdk-java-gcp > PubSubIO.Read should be implemen

[jira] [Updated] (BEAM-136) PipelineOptionsFactory should detect all runners

2016-03-19 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-136: - Assignee: Thomas Groh > PipelineOptionsFactory should detect all runners > -

[jira] [Updated] (BEAM-53) PubSubIO: reimplement in Java

2016-03-19 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-53: Assignee: Mark Shields > PubSubIO: reimplement in Java > - > >

[jira] [Updated] (BEAM-123) Skip header row from csv

2016-03-19 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-123: - Component/s: sdk-java-extensions > Skip header row from csv > - > >

[jira] [Created] (BEAM-136) PipelineOptionsFactory should detect all runners

2016-03-19 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-136: Summary: PipelineOptionsFactory should detect all runners Key: BEAM-136 URL: https://issues.apache.org/jira/browse/BEAM-136 Project: Beam Issue Type: Improve

[jira] [Commented] (BEAM-128) The transform BigQueryIO.Read is currently not supported.

2016-03-19 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201946#comment-15201946 ] Daniel Halperin commented on BEAM-128: -- I would expect that this bug is blocked by [BEA

[jira] [Resolved] (BEAM-125) PubSubIO.Read should be implemented by an UnboundedSource

2016-03-19 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin resolved BEAM-125. -- Resolution: Duplicate I went ahead and assigned that one to you. Feel free to change the title/de

[jira] [Commented] (BEAM-138) Extend TextIO to new protocols (and maybe rename to FileIO)

2016-03-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204600#comment-15204600 ] Daniel Halperin commented on BEAM-138: -- https://issues.apache.org/jira/browse/BEAM-59 c

[jira] [Commented] (BEAM-138) Extend TextIO to new protocols (and maybe rename to FileIO)

2016-03-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204605#comment-15204605 ] Daniel Halperin commented on BEAM-138: -- JB, want to mark this as a dupe of BEAM-59? >

[jira] [Commented] (BEAM-131) Write to jdbc/database

2016-03-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204636#comment-15204636 ] Daniel Halperin commented on BEAM-131: -- This may be a silly question, but why is this i

[jira] [Comment Edited] (BEAM-131) Write to jdbc/database

2016-03-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205025#comment-15205025 ] Daniel Halperin edited comment on BEAM-131 at 3/21/16 8:19 PM: ---

[jira] [Commented] (BEAM-131) Write to jdbc/database

2016-03-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205025#comment-15205025 ] Daniel Halperin commented on BEAM-131: -- [and of course if the JDBC connector is only si

[jira] [Commented] (BEAM-131) Write to jdbc/database

2016-03-21 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205022#comment-15205022 ] Daniel Halperin commented on BEAM-131: -- +1. Re your #2: I hope not. Specifically, I am

[jira] [Created] (BEAM-148) Jenkins build failure on start project

2016-03-23 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-148: Summary: Jenkins build failure on start project Key: BEAM-148 URL: https://issues.apache.org/jira/browse/BEAM-148 Project: Beam Issue Type: Bug Com

[jira] [Commented] (BEAM-148) Jenkins build failure on start project

2016-03-23 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209779#comment-15209779 ] Daniel Halperin commented on BEAM-148: -- Is there a way to version the Jenkins config wi

[jira] [Commented] (BEAM-148) Jenkins build failure on start project

2016-03-23 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209807#comment-15209807 ] Daniel Halperin commented on BEAM-148: -- On my local disk `mvn verify` indeed fails whil

[jira] [Commented] (BEAM-148) Jenkins build failure on start project

2016-03-23 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209808#comment-15209808 ] Daniel Halperin commented on BEAM-148: -- And pointing out that the test that failed on J

[jira] [Commented] (BEAM-136) PipelineOptionsFactory should detect all runners

2016-03-25 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15212154#comment-15212154 ] Daniel Halperin commented on BEAM-136: -- [~lcwik] can probably answer that question > P

[jira] [Commented] (BEAM-50) BigQueryIO.Write: reimplement in Java

2016-03-28 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214474#comment-15214474 ] Daniel Halperin commented on BEAM-50: - Other things to include: When users exceed BigQue

[jira] [Created] (BEAM-157) CombineTest.testGlobalCombineWithDefaultsAndTriggers is broken

2016-03-29 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-157: Summary: CombineTest.testGlobalCombineWithDefaultsAndTriggers is broken Key: BEAM-157 URL: https://issues.apache.org/jira/browse/BEAM-157 Project: Beam Issu

[jira] [Commented] (BEAM-85) DataflowAssert (BeamAssert ;) needs sanity check that it's used correctly

2016-03-29 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15217468#comment-15217468 ] Daniel Halperin commented on BEAM-85: - See [BEAM-157] > DataflowAssert (BeamAssert ;) ne

[jira] [Commented] (BEAM-157) CombineTest.testGlobalCombineWithDefaultsAndTriggers is broken

2016-03-29 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15217467#comment-15217467 ] Daniel Halperin commented on BEAM-157: -- See also, [BEAM-85] > CombineTest.testGlobalCo

[jira] [Commented] (BEAM-159) Support fixed number of shards in custom sinks

2016-03-30 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218769#comment-15218769 ] Daniel Halperin commented on BEAM-159: -- https://issues.apache.org/jira/browse/BEAM-68

[jira] [Resolved] (BEAM-159) Support fixed number of shards in custom sinks

2016-03-30 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin resolved BEAM-159. -- Resolution: Duplicate > Support fixed number of shards in custom sinks > -

[jira] [Commented] (BEAM-68) Support for limiting parallelism of a step

2016-03-30 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219353#comment-15219353 ] Daniel Halperin commented on BEAM-68: - Eugene: I disagree with the premise that even the

[jira] [Comment Edited] (BEAM-68) Support for limiting parallelism of a step

2016-03-30 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219353#comment-15219353 ] Daniel Halperin edited comment on BEAM-68 at 3/31/16 5:08 AM: -- E

[jira] [Commented] (BEAM-68) Support for limiting parallelism of a step

2016-03-30 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219376#comment-15219376 ] Daniel Halperin commented on BEAM-68: - Okay, I think I'm partially wrong. KV> -> ParDo(p

[jira] [Commented] (BEAM-159) Support fixed number of shards in custom sinks

2016-03-31 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220133#comment-15220133 ] Daniel Halperin commented on BEAM-159: -- Thanks Eugene -- this makes more sense. Appreci

[jira] [Created] (BEAM-164) Dynamic splitting for unbounded sources

2016-04-01 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-164: Summary: Dynamic splitting for unbounded sources Key: BEAM-164 URL: https://issues.apache.org/jira/browse/BEAM-164 Project: Beam Issue Type: Bug Co

[jira] [Resolved] (BEAM-164) Dynamic splitting for unbounded sources

2016-04-01 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin resolved BEAM-164. -- Resolution: Duplicate > Dynamic splitting for unbounded sources >

[jira] [Closed] (BEAM-164) Dynamic splitting for unbounded sources

2016-04-01 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin closed BEAM-164. > Dynamic splitting for unbounded sources > --- > > Ke

[jira] [Commented] (BEAM-47) UnboundedSource should support dynamic work rebalancing

2016-04-01 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221974#comment-15221974 ] Daniel Halperin commented on BEAM-47: - I don't think so. SplittableDoFn is a replacement

[jira] [Commented] (BEAM-47) UnboundedSource should support dynamic work rebalancing

2016-04-01 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221977#comment-15221977 ] Daniel Halperin commented on BEAM-47: - (SplittableDoFn could make this obsolete) > Un

[jira] [Created] (BEAM-168) IntervalBoundedBackOff change broke Beam-on-Dataflow

2016-04-04 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-168: Summary: IntervalBoundedBackOff change broke Beam-on-Dataflow Key: BEAM-168 URL: https://issues.apache.org/jira/browse/BEAM-168 Project: Beam Issue Type: Bug

[jira] [Updated] (BEAM-168) IntervalBoundedExponentialBackOff change broke Beam-on-Dataflow

2016-04-04 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin updated BEAM-168: - Summary: IntervalBoundedExponentialBackOff change broke Beam-on-Dataflow (was: IntervalBoundedBackO

[jira] [Commented] (BEAM-167) TextIO can't read concatenated gzip files

2016-04-04 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225122#comment-15225122 ] Daniel Halperin commented on BEAM-167: -- Thanks Eugene! > TextIO can't read concatenate

[jira] [Closed] (BEAM-168) IntervalBoundedExponentialBackOff change broke Beam-on-Dataflow

2016-04-04 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin closed BEAM-168. Resolution: Fixed > IntervalBoundedExponentialBackOff change broke Beam-on-Dataflow >

[jira] [Commented] (BEAM-123) Skip header row from csv

2016-04-04 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225740#comment-15225740 ] Daniel Halperin commented on BEAM-123: -- An alternative would be to update FileBasedSour

[jira] [Commented] (BEAM-123) Skip header row from csv

2016-04-04 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225738#comment-15225738 ] Daniel Halperin commented on BEAM-123: -- Hi Davin, I see. We do not currently have a CS

[jira] [Created] (BEAM-174) BigQueryIO.Read should give good error message for views and federated tables

2016-04-05 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-174: Summary: BigQueryIO.Read should give good error message for views and federated tables Key: BEAM-174 URL: https://issues.apache.org/jira/browse/BEAM-174 Project: Beam

[jira] [Created] (BEAM-178) stdout vs logging in DataflowPipelineRunner?

2016-04-06 Thread Daniel Halperin (JIRA)
Daniel Halperin created BEAM-178: Summary: stdout vs logging in DataflowPipelineRunner? Key: BEAM-178 URL: https://issues.apache.org/jira/browse/BEAM-178 Project: Beam Issue Type: Bug

[jira] [Assigned] (BEAM-188) Merging WindowFn + GBK + Write => InvalidWindows throws UnsupportedOperationException

2016-04-11 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin reassigned BEAM-188: Assignee: Daniel Halperin (was: Kenneth Knowles) > Merging WindowFn + GBK + Write => Invalid

[jira] [Closed] (BEAM-188) Merging WindowFn + GBK + Write => InvalidWindows throws UnsupportedOperationException

2016-04-14 Thread Daniel Halperin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Halperin closed BEAM-188. Resolution: Fixed > Merging WindowFn + GBK + Write => InvalidWindows throws > UnsupportedOperationExc

  1   2   3   4   5   6   7   8   9   10   >