Max is right, it isn't generally possible to upgrade Beam version for scio.
Scio doesn't use much of Beam internals, but Beam API isn't guaranteed to
be binary compatible in a minor release. There is a small chance that it
might work.
I would recommend using the latest version 0.7.0-beta3 built
@Gleb Thanks for pointing out some of the features unique to Scio! Have yet to
try it out, but I see many Beam folks liking it a lot.
@Jan I think the Beam version is hardcoded in Scio because Scio is essentially a
layer on top of Beam and some of its internals. While the public API does not
Hi all,
I think I'm going to stick with scala and scio :-).
I'm curious though: why is there a hard coupling between scio and beam
versions? I was hoping to use latest scio 0.7.0-beta2 with beam 2.9.0 but
that appears to get blocked, which was unexpected to me.
Regarding the suggestion to add
Agree with Max that scio is lagging behind. However, it also has features
that significantly reduce boilerplate, and even improve performance. For
instance, the latest version (0.7.0) automatically derives binary coders
for case classes using macro at compile-time, that is a way better
than
Interesting project, Jan! I think we could add your project to this page:
https://beam.apache.org/community/integrations/
The benefit of using the Java DSL would be to be able to directly track Beam.
The Scio Scala DSL usually lags a bit behind. But since you probably don't
require the latest
Scio is a Scala wrapper on top of Beam's Java SDK. So it still benefits
from the maturity of Beam Java in terms of performance and reliability.
Using Scala will definitely be less verbose than Java. You can use Scio or
the Java SDK directly with Scala's support for calling Java libraries.
Kenn
Hi Chak,
I'm not sure if it's the correct decision...
To be completely honest, the first iterations (which I haven't made public
so far) were actually in java.
However I find java to be a bit verbose for my taste.
The past 5 years I've worked in OCaml, and despite the lacking
tooling/ecosystem I
Hi Jan,
This is quite interesting. As far as I know, Beam and Flink have more
mature and stable API in Java. What is the motivation here to use
scala/scio in your project?
Kind regards,
Chak
On Sun, Dec 30, 2018 at 1:02 PM wrote:
> Hi all,
>
> I figured out how to build deterministic
