[ https://issues.apache.org/jira/browse/TEZ-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hitesh Shah updated TEZ-2105: ----------------------------- Labels: gsoc gsoc2015 hadoop java pig tez (was: ) > Totally Sorted Edge with auto-parallelism > ----------------------------------------- > > Key: TEZ-2105 > URL: https://issues.apache.org/jira/browse/TEZ-2105 > Project: Apache Tez > Issue Type: New Feature > Reporter: Gopal V > Labels: gsoc, gsoc2015, hadoop, java, pig, tez > > Pig-on-Tez supports an edge configuration using a sampled Output along with a > vertex manager for automatic parallelism estimation. > This is referred to in the Pig-on-Tez Hadoop Summit presentation. > http://www.slideshare.net/Hadoop_Summit/pig-on-tez-low-latency-etl-with-big-data/19 > Migrating that plan-model into Tez as a native edge type would allow for much > more efficient scheduling of the downstream edges and effectively turn the > auto-parallelism implementation into a runtime skew-correcting mechanism > within this edge. > The Tez Edge has enough information to sample, determine partitioning order > and correct parallelism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)