Hi Sid I created Tez-963 which is related to the problem that I met.
Regards! Chen On Wed, Mar 19, 2014 at 8:20 PM, Chen He <[email protected]> wrote: > Hi Sid > > Thank you for the reply. Actually, I think it related to Tez-710 and > Tez-717. I created TezLocalRunner which extends TezClient. The > TezLocalRunner need to start DAGAppMaster which is in charge of maintaining > Tez DAG and let LocalContainerLauncher run tasks in a single > node.(Tez-710). > > Here is the error when LocalContainerLauncher launches task: > > java.io.FileNotFoundException: tez-conf.pb (No such file or directory) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.<init>(FileInputStream.java:146) > at java.io.FileInputStream.<init>(FileInputStream.java:101) > at > org.apache.tez.common.TezUtils.addUserSpecifiedTezConfiguration(TezUtils.java:58) > at > org.apache.hadoop.mapred.YarnTezDagChild.runTask(YarnTezDagChild.java:296) > at > org.apache.tez.dag.app.launcher.LocalContainerLauncher$1.run(LocalContainerLauncher.java:263) > at java.lang.Thread.run(Thread.java:744) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > > I did some research and found out that the tez-conf.pb was successfully > written to the Tez staging directory. But the YarnTezDagChild can not get > the value of "TezConfiguration.TEZ_PB_BINARY_CONF_NAME" (get "null") > > Any suggestion? Thank you in advance! > > Regards! > > Chen > > > On Wed, Mar 19, 2014 at 7:50 PM, Siddharth Seth <[email protected]> wrote: > >> Right, the configuration provided by the user gets written out and is made >> available to the AM and tasks. >> >> I¹m not sure what you mean by OEI always get ³null²¹ - assuming this is >> while trying to read a specific key. The Tez client filters out all >> non-tez configuration parameters before writing out this Configuration - >> which would cause an empty lookup. >> >> Could you describe why you need to pass in a Configuration - typically, >> I/P/O should have their own payload - which is what they use to configure >> themselves. >> >> >> On 3/19/14, 3:57 PM, "Chen He" <[email protected]> wrote: >> >> >Dear Tez developers: >> > >> >I met a problem when I was working on TezLocalRunner. I need to know how >> >user's configuration is passed to the YarnTezDagChild so that it can load >> >it using: >> >TezUtils.addUserSpecifiedTezConfiguration(conf); >> > >> >Here is my understanding: >> > >> >1) Users create their TezConfiguration and the TezClientUtils will write >> >it >> >to staging directory; >> >2) DAGAppMaster's main method load TezConfiguration using >> >TezUtils.addUserSpecifiedTezConfiguration(conf); >> >3) when the YarnTezDagChild starts to run, its main method also call >> >TezUtils.addUserSpecifiedTezConfiguration(conf); >> > >> >I met a problem that I always get "null" whenever I call the >> >TezUtils.addUserSpecifiedTezConfiguration(conf) in the TezClient. >> > >> >Any reply will be appreciated! >> > >> >Regards! >> > >> >Chen >> >> >> >
