Hello,
I just started playing with Beam and I wanted to debug what happens
between transforms in pipelines. I wrote a simple 'Debug' transform for
this.
The idea is to apply a function based on a predicate to any element in a
collection without changing the collection, or in other words, a transform
that
does not transform but produces side effects.
The idea is better illustrated with this simple example:
.apply(FlatMapElements.via((String text) -> Arrays.asList(text.split("
")))
.withOutputType(new TypeDescriptor<String>() {
}))
.apply(Debug
.when((String s) -> s.startsWith("A"))
.with((String s) -> {
System.out.println(s);
return null;
}));
.apply(Filter.byPredicate((String text) -> text.length() > 5))
.apply(Debug.print()); // sugared method, same as above
I think this can be useful (at least for debugging purposes), is there
something
like this already in the SDK ? If this is not the case, can you please give
me some
feedback/ideas to improve my transform.
Thanks,
-Ismael
ps. You can find the code of the first version of the transform here:
https://github.com/iemejia/beam-playground/blob/master/src/main/java/org/apache/beam/transforms/Debug.java
