[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-09 Thread Adam Szita (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119583#comment-16119583 ] Adam Szita commented on PIG-5283: - [^PIG-5283.1.patch] committed to trunk. Thanks for the review Rohini,

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-09 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119488#comment-16119488 ] liyunzhang_intel commented on PIG-5283: --- [~szita]: thanks for your explaination. +1 > Configuration

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-09 Thread Adam Szita (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119467#comment-16119467 ] Adam Szita commented on PIG-5283: - [~kellyzly] because insisde {{readFields}} [compression configs are

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119287#comment-16119287 ] liyunzhang_intel commented on PIG-5283: --- [~szita]: I can understand the reason why need set

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-08 Thread Adam Szita (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118306#comment-16118306 ] Adam Szita commented on PIG-5283: - Attached [^PIG-5283.1.patch] with the feature of only writing out the

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118075#comment-16118075 ] liyunzhang_intel commented on PIG-5283: --- [~szita]: {quote} My only question is that if we should

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-07 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116449#comment-16116449 ] Rohini Palaniswamy commented on PIG-5283: - bq. My only question is that if we should only write

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-07 Thread Adam Szita (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116368#comment-16116368 ] Adam Szita commented on PIG-5283: - [~kellyzly], it doesn't matter what we set in the configuration,

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-05 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115582#comment-16115582 ] Rohini Palaniswamy commented on PIG-5283: - bq. HADOOP-8588 was committed in Hadoop 3, supposed to

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-03 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113817#comment-16113817 ] liyunzhang_intel commented on PIG-5283: --- [~nkollar] and [~szita]: Can we set the correct value of

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-03 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112502#comment-16112502 ] Nandor Kollar commented on PIG-5283: I think Adam's patch is a good workaround for this problem, looks

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-03 Thread Nandor Kollar (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112500#comment-16112500 ] Nandor Kollar commented on PIG-5283: This is a problem with Hadoop 3 (HADOOP-14459) and Spark with Pig.

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-03 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112449#comment-16112449 ] liyunzhang_intel commented on PIG-5283: --- [~szita]: In

[jira] [Commented] (PIG-5283) Configuration is not passed to SparkPigSplits on the backend

2017-08-02 Thread Adam Szita (JIRA)
[ https://issues.apache.org/jira/browse/PIG-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111076#comment-16111076 ] Adam Szita commented on PIG-5283: - I propose we make use of SparkPigSplit and write the conf along with the