Github user jkff closed the pull request at:
https://github.com/apache/incubator-beam/pull/1473
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1636
[BEAM-498] Moves OldDoFn to runners-core
This will of course need the usual Dataflow worker surgery.
R: @kennknowles
You can merge this pull request into a Git repository by running
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1620
[BEAM-1154] Get side input from proper window in ReduceFn
R: @kennknowles
CC: @peihe
You can merge this pull request into a Git repository by running:
$ git pull https://github.com
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1601
[BEAM-1149] Explode windows when fn uses side inputs
This basically reverts
https://github.com/apache/incubator-beam/commit/38f0b11cc9028cf347e3c96b6e6116e5a5a9972d
and adds some comments
Github user jkff closed the pull request at:
https://github.com/apache/incubator-beam/pull/1584
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1584
DO NOT REVIEW. Debugging MqttIO failures.
(can't reproduce the error locally, and don't know how to run something on
Travis - so creating a PR just for the sake of getting a Travis run)
You
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1565
Practically removes OldDoFn from SDK
After this change, the only usages of OldDoFn in the SDK are in combine
context stuff, which itself should be moved to runners-core (outside the scope
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1486
Adds a category for tests using splittable DoFn
Excludes the category in all runners except direct.
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1473
Adds TransformEvaluator.startBundle()
For symmetry with finishBundle() and to allow decoupling creation of
a ParDoEvaluator from the time it calls fn.startBundle().
R: @tgroh
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1471
Preserves compressed windows in PushbackSideInputDoFnRunner
R: @tgroh
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jkff/incubator-beam
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1461
[BEAM-1060] Makes DoFnTester use new DoFn internally.
There were 2 remaining users of DoFnTester.of(OldDoFn):
- SplittableParDo.ProcessElements: this is fixed in
https://github.com
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1373
Properly uses result of transform override
Previously direct runner would use the transform override to .apply(),
but would keep the original transform in the pipeline, e.g. it would use
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1367
Declare archetype starter dependency on slf4j
Example failed log before this PR:
https://api.travis-ci.org/jobs/176202650/log.txt?deansi=true
R: @lukecwik
You can merge this pull
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1364
Fixes after #1332
#1332 introduced a few precommit failures. This PR fixes them and makes a
couple of other minor changes noticed while doing the fixes.
R: @amitsela
You can merge
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1353
Liberates ReduceFnRunner from WindowingInternals, and lets
WindowingInternals do windowed side outputs
- Introduces WindowingInternals.sideOutputWindowedValue (will be necessary
Github user jkff closed the pull request at:
https://github.com/apache/incubator-beam/pull/1036
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1260
Reduces exposure of WindowedValue in the SDK
- Introduces ValueInSingleWindow for purposes of PAssert
- Moves several classes that should be in runners-core
and that used
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1243
Fixes crash of shade plugin on SparkProcessContext
Error discovered by debugging the Maven plugin itself
(using mvnDebug) and setting a breakpoint for the exception
and then inspecting
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1203
Deduplicates ParDo{Single,Multi}EvaluatorFactory
This is in preparation for adding a third one for a future ParDo-like
primitive
transform to be introduced inside SplittableParDo
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1201
Makes PTransformOverrideFactory type-safe
This removes a bunch of casting boilerplate from subclasses.
Also, removes unused class `UncommittedBundleOutputManager` (seems not
worth
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1050
Makes FileBasedSink use a temporary directory
When writing to `/path/to/foo`, temporary files would be written to
`/path/too/foo-temp-$uid` (or something like that), i.e. as siblings
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1048
Converts KafkaIO builders to @AutoValue
This is in the same spirit as
https://github.com/apache/incubator-beam/pull/1031,
https://github.com/jbonofre/incubator-beam/pull/1 and
https
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1036
Introduces the Rebundle transform
It's similar to Reshuffle in that it prevents fusion of the surrounding
transforms, however while Reshuffle
requires the input collection to be KVs
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1031
Converts Datastore to use AutoValue
This style saves a good amount of code and is easier to mimic in new
connectors.
I have a few more changes like this lined up.
R
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1028
Enables -Xlint:rawtypes
Except for Spark and Flink runners and the microbenchmarks module.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1023
Deduplicates the 3 in-memory TimerInternals
There were 3 implementations of TimerInternals doing
basically the same thing:
- TestTimerInternals in ReduceFnTester
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/1011
Support for @Setup and @Teardown in DoFnTester
- To allow testing DoFn reuse, adds a third cloning mode - CLONE_ONCE. The
current CLONE is renamed to CLONE_PER_BUNDLE because it corresponds
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/973
Deprecate TeardownPolicy for Dataflow service
We are moving towards supporting only TEARDOWN_ALWAYS.
(This is a clone of
https://github.com/pjesa/DataflowJavaSDK/commit
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/954
Enables BytecodeVerificationLocal on travis
This is to achieve stricter verification of bytecode,
mainly bytecode generated by DoFnInvokers dynamically.
See https
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/953
[BEAM-630] Fixes wrong type name in generated code
See https://issues.apache.org/jira/browse/BEAM-630 for details. We
generated the wrong local variable type (with a semicolon - "descr
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/926
More changes to DoFn{Signatures,Invokers}.
In preparation for Splittable DoFn.
* More generic code generation in DoFnInvokers:
supports methods with return values (thanks @bjchambers
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/896
[BEAM-65] SplittableDoFn prototype.
Work in progress. Currently only runs in direct runner,
and not ready for any use by real users.
No tests for most of the code yet (will add
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/812
Rewrites DoFnReflector to go via DoFnSignature
DoFnSignature encapsulates type information about a DoFn,
in particular which arguments/features its methods
actually use.
Before
GitHub user jkff opened a pull request:
https://github.com/apache/incubator-beam/pull/144
[BEAM-182] Bound trials in assertSplitAtFraction
At most 100 trials per item and 1000 trials total.
This appears to be enough for tests such as AvroSourceTest
in the majority of cases
34 matches
Mail list logo
