Re: Apache Beam a Complete Guide - Review?

2020-06-28 Thread Wesley Peng
Hi Rion Rion Williams wrote: I considered that one as well but was in the same boat in terms of not pulling the trigger (lack of reviews, price point, etc.). I eventually landed on Streaming Systems, which I highly, highly recommend if you want to learn more about the Beam model: -

Re: Apache Beam a Complete Guide - Review?

2020-06-28 Thread Rion Williams
Hi Wesley, I considered that one as well but was in the same boat in terms of not pulling the trigger (lack of reviews, price point, etc.). I eventually landed on Streaming Systems, which I highly, highly recommend if you want to learn more about the Beam model: - http://streamingsystems.net/

DoFn with SideInput

2020-06-28 Thread Praveen K Viswanathan
Hi All - I am facing an issue while using *side-input*. *What am I doing:* >From my main program, I am calling a custom PTransform with a PCollectionView as parameter. Inside custom PTransform, I am passing the PCollectionView as a side-input to a DoFn. *Issue:* When I run the pipeline, I am

Apache Beam a Complete Guide - Review?

2020-06-28 Thread Wesley Peng
Hello Has anyone bought this book? Can you give a simple review, good or not? https://www.thriftbooks.com/w/apache-beam-a-complete-guide---2020-edition/26243355/item/36997139/ I want to buy a book for beam the specific topic, but this book seems too new to have enough reviews. Thanks.

Re: Can SpannerIO read data from different GCP project?

2020-06-28 Thread Austin Bennett
I havent tried yet, but looks like the connection string asks for the project to be specified. Based on that (and cross project working for other circumstances), I would imagine it will work, but...? Give it a try! One tricky place might be ensuring proper permissions, in both projects (and

Concurrency issue with KafkaIO

2020-06-28 Thread wang Wu
Hi, We run Beam pipeline on Spark in the streaming mode. We subscribe to multiple Kafka topics. Our job run fine until it is under heavy load: millions of Kafka messages coming per seconds. The exception look like concurrency issue. Is it a known bug in Beam or some Spark configuration we could