[jira] [Commented] (PIG-4621) Enable Illustrate in spark
[ https://issues.apache.org/jira/browse/PIG-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028218#comment-15028218 ] Syed Zulfiqar Ali commented on PIG-4621: CC: [~xuefuz] [~mohitsabharwal] > Enable Illustrate in spark > -- > > Key: PIG-4621 > URL: https://issues.apache.org/jira/browse/PIG-4621 > Project: Pig > Issue Type: Sub-task > Components: spark >Reporter: liyunzhang_intel >Assignee: Syed Zulfiqar Ali > Fix For: spark-branch > > > Current we don't support illustrate in spark mode. > How illustrate works > see:http://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#ILLUSTRATE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4621) Enable Illustrate in spark
[ https://issues.apache.org/jira/browse/PIG-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026205#comment-15026205 ] Syed Zulfiqar Ali commented on PIG-4621: Patch uploaded for review: https://reviews.apache.org/r/40692/ > Enable Illustrate in spark > -- > > Key: PIG-4621 > URL: https://issues.apache.org/jira/browse/PIG-4621 > Project: Pig > Issue Type: Sub-task > Components: spark >Reporter: liyunzhang_intel >Assignee: Syed Zulfiqar Ali > Fix For: spark-branch > > > Current we don't support illustrate in spark mode. > How illustrate works > see:http://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#ILLUSTRATE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (PIG-4621) Enable Illustrate in spark
[ https://issues.apache.org/jira/browse/PIG-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on PIG-4621 started by Syed Zulfiqar Ali. -- > Enable Illustrate in spark > -- > > Key: PIG-4621 > URL: https://issues.apache.org/jira/browse/PIG-4621 > Project: Pig > Issue Type: Sub-task > Components: spark >Reporter: liyunzhang_intel >Assignee: Syed Zulfiqar Ali > Fix For: spark-branch > > > Current we don't support illustrate in spark mode. > How illustrate works > see:http://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#ILLUSTRATE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PIG-4621) Enable Illustrate in spark
[ https://issues.apache.org/jira/browse/PIG-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Syed Zulfiqar Ali updated PIG-4621: --- Assignee: Syed Zulfiqar Ali > Enable Illustrate in spark > -- > > Key: PIG-4621 > URL: https://issues.apache.org/jira/browse/PIG-4621 > Project: Pig > Issue Type: Sub-task > Components: spark >Reporter: liyunzhang_intel >Assignee: Syed Zulfiqar Ali > Fix For: spark-branch > > > Current we don't support illustrate in spark mode. > How illustrate works > see:http://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#ILLUSTRATE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-4616) Fix UT errors of TestPigRunner in Spark mode
[ https://issues.apache.org/jira/browse/PIG-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973798#comment-14973798 ] Syed Zulfiqar Ali commented on PIG-4616: TestPigRunner depends heavily on SparkPigStats API for validating the executed PIG script (..and underlying Spark jobs). Some of these tests are failing due to unimplemented methods in SparkPigStats API (Eg: getRecordWritten(), getBytesWritten() etc). I dug into few tests and found that though underlying Spark jobs are doing correct computation but SparkPigStats API is not returning the correct stats and metrics, hence tests are failing. > Fix UT errors of TestPigRunner in Spark mode > > > Key: PIG-4616 > URL: https://issues.apache.org/jira/browse/PIG-4616 > Project: Pig > Issue Type: Sub-task > Components: spark >Reporter: Xianda Ke >Assignee: Xianda Ke > Fix For: spark-branch > > > Following unit test failed: > org.apache.pig.test.TestPigRunner.returnCodeTest > org.apache.pig.test.TestPigRunner.testEmptyFileCounter > org.apache.pig.test.TestPigRunner.testDisablePigCounters2 > org.apache.pig.test.TestPigRunner.simpleTest > org.apache.pig.test.TestPigRunner.simpleTest2 > org.apache.pig.test.TestPigRunner.MQDepJobFailedTest > org.apache.pig.test.TestPigRunner.scriptsInDfsTest > org.apache.pig.test.TestPigRunner.testGetHadoopCounters > org.apache.pig.test.TestPigRunner.simpleMultiQueryTest > org.apache.pig.test.TestPigRunner.testDuplicateCounterName > org.apache.pig.test.TestPigRunner.testRegisterExternalJar > org.apache.pig.test.TestPigRunner.simpleMultiQueryTest2 > org.apache.pig.test.TestPigRunner.testDuplicateCounterName2 > org.apache.pig.test.TestPigRunner.returnCodeTest2 > org.apache.pig.test.TestPigRunner.orderByTest > org.apache.pig.test.TestPigRunner.testDisablePigCounters > org.apache.pig.test.TestPigRunner.testLongCounterName > org.apache.pig.test.TestPigRunner.testEmptyFileCounter2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
