Ok,
found it. The
edgeBuilder.setAdditionalConfiguration(TezRuntimeConfiguration.TEZ_RUNTIME_COMPRESS,
"true”);
does work for me!
So let me describe my use case a little bit...
Basically i have one Configuration like object on the client side. This is
assembled by multiple sources and the only way a user can set custom Tez
properties (do not use tez-site.xml in any perspective).
Then i’m building my DAG with its vertices and edges programatically.
Now, do you have any recommendation for me how to route the right Tez
properties effectively to the corresponding Tez components ? (with tez
components i mean like vertex properties, dag properties, AM properties, edge
properties, etc..)
Should i simply set all tez.* properties to any component or is there a smarter
way ?
And what components/properties might i’m missing ?
Any help appreciated!
Johannes
> On 14 Sep 2015, at 16:57, Johannes Zillmann <[email protected]> wrote:
>
> Hey guys,
>
> question. How do i enabled tez.runtime.compress programatically ?
> When i set this property in the tez-site.xml it is picket up correctly.
> But all other options i tried:
> - dag.setConf(TezRuntimeConfiguration.TEZ_RUNTIME_COMPRESS, "true");
> - mapVertex.setConf(TezRuntimeConfiguration.TEZ_RUNTIME_COMPRESS, "true");
> - reduceVertex.setConf(TezRuntimeConfiguration.TEZ_RUNTIME_COMPRESS, "true”);
>
> do not have any effect! (Checking the log output of the Shuffle class)
>
> Johannes
