[ https://issues.apache.org/jira/browse/PIG-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208139#comment-15208139 ]
shenxianqiang commented on PIG-4849: ------------------------------------ In src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java In newVertex function calls (vertexInfo, payloadConf). This is the cause to lead tez-ui to crash. But,In tez-0.6,2,there is a property:tez.runtime.convert.user-payload.to.history-text It is mainly used to control the output of user.payload information. We modify this bug in tez or pig? Any suggestions? > pig on tez will cause tez-ui to crash,because the content from timeline > server is too long. > -------------------------------------------------------------------------------------------- > > Key: PIG-4849 > URL: https://issues.apache.org/jira/browse/PIG-4849 > Project: Pig > Issue Type: Bug > Affects Versions: 0.15.0 > Environment: pig:0.15,tez:0.6.2,hadoop:2.5.0 > Reporter: shenxianqiang > > After running several Pig Job,the tez-ui server crashed,because the content > from timeline server exceed 80MB. > When I input : > http://timeline:48188/ws/v1/timeline/TEZ_DAG_ID?limit=51 > The content is too long... > Like this : > {quote} > {"vertexName":"scope-18","processorClass":"org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor","userPayloadAsText":"{\"desc\":\"wordcount[4,12] > > (GROUP_BY)\",\"config\":{\"dfs.datanode.data.dir\":\"file:\\/\\/\\/search\\/hadoop\\/dfs_data,\",\"dfs.namenode.checkpoint.txns\":\"1000000\",\"s3.replication\":\"3\",\"mapreduce.output.fileoutputformat.compress.type\":\"RECORD\",\"mapreduce.jobtracker.jobhistory.lru.cache.size\":\"5\",\"hadoop.http.filter.initializers\":\"org.apache.hadoop.http.lib.StaticUserWebFilter\",\"yarn.nodemanager.keytab\":\"\\/etc\\/krb5.keytab\",\"nfs.mountd.port\":\"4242\",\"yarn.resourcemanager.zk-acl\":\"world:anyone:rwcda\",\"dfs.https.server.keystore.resource\":\"ssl-server.xml\",\"mapr...... > {quote} > I am surprised that each vertex has a "userPayloadAsText",and "config" > information is particularly large. > When I run a more complex pig on tez job, the tez-ui server is easy to crash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)