[jira] Updated: (PIG-1094) Fix unit tests corresponding to source changes so far
[
https://issues.apache.org/jira/browse/PIG-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Ding updated PIG-1094:
--
Attachment: PIG-1094_7.patch
This patch fixed several failed test cases caused by removing split by 'file'
from Pig Latin.
Resutls of 'test-patch':
{code}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 9 new or
modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the
total number of release audit warnings.
{code}
Fix unit tests corresponding to source changes so far
-
Key: PIG-1094
URL: https://issues.apache.org/jira/browse/PIG-1094
Project: Pig
Issue Type: Sub-task
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Attachments: PIG-1094.patch, PIG-1094_2.patch, PIG-1094_3.patch,
PIG-1094_4.patch, PIG-1094_5.patch, PIG-1094_6.patch, PIG-1094_7.patch
The check-in's so far on load-store-redesign branch have nor addressed unit
test failures due to interface changes. This jira is to track the task of
making the common case unit tests work with the new interfaces. Some aspects
of the new proposal like using LoadCaster interface for casting, making local
mode work have not been completed yet. Tests which are failing due to those
reasons will not be fixed in this jira and addressed in the jiras
corresponding to those tasks
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1094) Fix unit tests corresponding to source changes so far
[ https://issues.apache.org/jira/browse/PIG-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-1094: --- Attachment: PIG-1094_5.patch This patch (PIG-1094_5.patch) fixes the order-by, skew-join and merge-join test failures. TestPoissonSampleLoader.java - testNumSamples() - Unlike earlier version of sampler, if there are very few rows (3 in this case) only one sample is selected. WeightedRangePartitioner.java - If the sample file is empty, there was a check to ensure that the input is also empty , using FileLocalizer.getSize(). Removed that check. Input location need not be a file. PoissonSampleLoader.java - additional comments, fixed indentation . GetMemNumRows.java - handling the case where 2nd last column is null (while looking for the specially marked last tuple). output of test-patch - [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 6 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. Fix unit tests corresponding to source changes so far - Key: PIG-1094 URL: https://issues.apache.org/jira/browse/PIG-1094 Project: Pig Issue Type: Sub-task Reporter: Pradeep Kamath Assignee: Pradeep Kamath Attachments: PIG-1094.patch, PIG-1094_2.patch, PIG-1094_3.patch, PIG-1094_4.patch, PIG-1094_5.patch The check-in's so far on load-store-redesign branch have nor addressed unit test failures due to interface changes. This jira is to track the task of making the common case unit tests work with the new interfaces. Some aspects of the new proposal like using LoadCaster interface for casting, making local mode work have not been completed yet. Tests which are failing due to those reasons will not be fixed in this jira and addressed in the jiras corresponding to those tasks -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1094) Fix unit tests corresponding to source changes so far
[ https://issues.apache.org/jira/browse/PIG-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-1094: --- Attachment: PIG-1094_6.patch This patch replaces PIG-1094_5.patch As per Pradeep's suggestion, keeping the check in WeightedRangePartitioner.java to ensure that input is empty if sample file is empty. Also merged with latest changes in LSR branch. Fix unit tests corresponding to source changes so far - Key: PIG-1094 URL: https://issues.apache.org/jira/browse/PIG-1094 Project: Pig Issue Type: Sub-task Reporter: Pradeep Kamath Assignee: Pradeep Kamath Attachments: PIG-1094.patch, PIG-1094_2.patch, PIG-1094_3.patch, PIG-1094_4.patch, PIG-1094_5.patch, PIG-1094_6.patch The check-in's so far on load-store-redesign branch have nor addressed unit test failures due to interface changes. This jira is to track the task of making the common case unit tests work with the new interfaces. Some aspects of the new proposal like using LoadCaster interface for casting, making local mode work have not been completed yet. Tests which are failing due to those reasons will not be fixed in this jira and addressed in the jiras corresponding to those tasks -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1094) Fix unit tests corresponding to source changes so far
[
https://issues.apache.org/jira/browse/PIG-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Kamath updated PIG-1094:
Attachment: PIG-1094_3.patch
Attached patch addresses more test failures.
Here is the session output from running ant test-patch target locally:
{noformat}
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 70 new or
modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] -1 release audit. The applied patch generated 382 release
audit warnings (more than the trunk's current 381 warnings).
[exec]
{noformat}
The release audit warnings are due to html issues with jdiff and can be ignored.
The list of tests which still fail are:
||Testcase class||Testcase method||Probable Cause||
|TestBinaryStorage|testBinaryStorageWithAsciiData|Streaming needs to be fixed|
|TestBinaryStorage|testBinaryStorageWithBinaryData|Streaming needs to be fixed|
|TestBZip|testBzipInPig|bzip needs to be fixed|
|TestBZip|testEmptyBzipInPig|bzip needs to be fixed|
|TestCounters|testMapOnly|Counters need to be fixed - dependent on hadoop|
|TestCounters|testMapOnlyBinStorage|Counters need to be fixed - dependent on
hadoop|
|TestCounters|testMapReduceOnly|Counters need to be fixed - dependent on hadoop|
|TestCounters|testMapReduceOnlyBinStorage|Counters need to be fixed - dependent
on hadoop|
|TestCounters|testMapCombineReduce|Counters need to be fixed - dependent on
hadoop|
|TestCounters|testMapCombineReduceBinStorage|Counters need to be fixed -
dependent on hadoop|
|TestCounters|testMultipleMRJobs|Counters need to be fixed - dependent on
hadoop|
|TestCounters|testLocal|Counters need to be fixed - dependent on hadoop|
|TestEvalPipeline2|testEmptySort|order by related failure|
|TestGrunt|testCD|Reparsing logic needs to be fixed|
|TestHBaseStorage|testLoadFromHBase|HBaseStorage needs to be fixed for new
load-store redesign|
|TestHBaseStorage|testBackwardsCompatibility|HBaseStorage needs to be fixed for
new load-store redesign|
|TestMRCompiler|testMergeJoin|test needs to be updated per new Merge join
implementation|
|TestMultiQuery|testMultiQueryPhase3StreamingInReducer2|Streaming needs to be
fixed|
|TestPoissonSampleLoader|testComputeSamples|test needs to be updated per new
Skew join code?|
|TestStreamingLocal|testSimpleMapSideStreaming|Streaming needs to be fixed|
|TestStreamingLocal|testSimpleMapSideStreamingWithOutputSchema|Streaming needs
to be fixed|
|TestStreamingLocal|testSimpleReduceSideStreamingAfterFlatten|Streaming needs
to be fixed|
|TestStreamingLocal|testSimpleOrderedReduceSideStreamingAfterFlatten|Streaming
needs to be fixed|
|TestStreamingLocal|testSimpleMapSideStreamingWithUnixPipes|Streaming needs to
be fixed|
|TestStreamingLocal|testLocalNegativeLoadStoreOptimization|Streaming needs to
be fixed|
Fix unit tests corresponding to source changes so far
-
Key: PIG-1094
URL: https://issues.apache.org/jira/browse/PIG-1094
Project: Pig
Issue Type: Sub-task
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Attachments: PIG-1094.patch, PIG-1094_2.patch, PIG-1094_3.patch
The check-in's so far on load-store-redesign branch have nor addressed unit
test failures due to interface changes. This jira is to track the task of
making the common case unit tests work with the new interfaces. Some aspects
of the new proposal like using LoadCaster interface for casting, making local
mode work have not been completed yet. Tests which are failing due to those
reasons will not be fixed in this jira and addressed in the jiras
corresponding to those tasks
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1094) Fix unit tests corresponding to source changes so far
[ https://issues.apache.org/jira/browse/PIG-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1094: -- Attachment: PIG-1094_2.patch This patch replaces Util.generateURI(String) with Util.generateURI(String, PigContext) to handle hadoopifying of test files. Fix unit tests corresponding to source changes so far - Key: PIG-1094 URL: https://issues.apache.org/jira/browse/PIG-1094 Project: Pig Issue Type: Sub-task Reporter: Pradeep Kamath Assignee: Pradeep Kamath Attachments: PIG-1094.patch, PIG-1094_2.patch The check-in's so far on load-store-redesign branch have nor addressed unit test failures due to interface changes. This jira is to track the task of making the common case unit tests work with the new interfaces. Some aspects of the new proposal like using LoadCaster interface for casting, making local mode work have not been completed yet. Tests which are failing due to those reasons will not be fixed in this jira and addressed in the jiras corresponding to those tasks -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1094) Fix unit tests corresponding to source changes so far
[
https://issues.apache.org/jira/browse/PIG-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Kamath updated PIG-1094:
Attachment: PIG-1094.patch
Attached patch which fixes most test cases which are due to the core changes in
load store interfaces. As noted in the description, the patch does not address
specific test failures like those due to local mode, streaming etc not being
fixed yet to work with the new load store redesign.
* The patch ensures all tests now compile
* The main changes were to ensure new interface methods are now used. Also in
many tests were earlier running in local mode, now they have been changed to
run in mapreduce mode - these are tests not specific to local mode - so the
switch should not result in loss of any test coverage.
* The patch also fixes LoadCaster related changes to POCast and related tests
* TestPigLineRecordReader.java and TestStreaming have been excluded from
compile-test and test-core targets since local mode and streaming needs to be
fixed for them to work
Here are the results from running test-patch ant target locally:
{noformat}
[exec]
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 77 new or
modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the
total number of release audit warnings.
{noformat}
I will add a comment separately on what tests still fail and which area of the
code needs to be fixed to address them.
Fix unit tests corresponding to source changes so far
-
Key: PIG-1094
URL: https://issues.apache.org/jira/browse/PIG-1094
Project: Pig
Issue Type: Sub-task
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Attachments: PIG-1094.patch
The check-in's so far on load-store-redesign branch have nor addressed unit
test failures due to interface changes. This jira is to track the task of
making the common case unit tests work with the new interfaces. Some aspects
of the new proposal like using LoadCaster interface for casting, making local
mode work have not been completed yet. Tests which are failing due to those
reasons will not be fixed in this jira and addressed in the jiras
corresponding to those tasks
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
