[
https://issues.apache.org/jira/browse/PIG-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060758#comment-13060758
]
Ken Goodhope commented on PIG-2153:
-----------------------------------
Thanks Pradeep, that is actually very helpful. If I understand you correctly,
the outer tuple isn't part of the schema returned by LoadFunc.getSchema(). Is
it possible that the result of LoadFunc.getNext used to be wrapped in an
implicit tuple, and that is no longer happening?
The results of the unit tests with the fix I suggested in my last comment
showed 11 tests now working that were broke before, and 11 tests now breaking
that used to work. This makes me wonder if some of the tests have been written
with the expectation there is an implicit wrapping tuple, and some have been
written with expectation that there is no implicit wrapper. Am I missing
something?
Here are the test results.
Test that were broke and now work.
> [junit] Test org.apache.pig.test.TestBestFitCast
> [junit] Test org.apache.pig.test.TestCounters
> [junit] Test org.apache.pig.test.TestDataBagAccess
> [junit] Test org.apache.pig.test.TestEmptyInputDir
> [junit] Test org.apache.pig.test.TestImplicitSplit
> [junit] Test org.apache.pig.test.TestInvoker
> [junit] Test org.apache.pig.test.TestPigRunner
> [junit] Test org.apache.pig.test.TestPigSplit
> [junit] Test org.apache.pig.test.TestScriptLanguage
> [junit] Test org.apache.pig.test.TestScriptUDF
> [junit] Test org.apache.pig.test.TestSkewedJoin
Tests that used to work, but break with the fix I tried.
< [junit] Test org.apache.pig.test.TestCombiner FAILED
< [junit] Test org.apache.pig.test.TestCommit FAILED
< [junit] Test org.apache.pig.test.TestEvalPipeline2 FAILED
< [junit] Test org.apache.pig.test.TestEvalPipelineLocal FAILED
< [junit] Test org.apache.pig.test.TestForEachNestedPlanLocal FAILED
< [junit] Test org.apache.pig.test.TestLimitAdjuster FAILED
< [junit] Test org.apache.pig.test.TestMergeJoinOuter FAILED
< [junit] Test org.apache.pig.test.TestProject FAILED
< [junit] Test org.apache.pig.test.TestProjectRange FAILED
< [junit] Test org.apache.pig.test.TestPruneColumn FAILED
< [junit] Test org.apache.pig.test.TestUnionOnSchema FAILED
> POProject throws an error with tuples containing a single non-tuple field
> -------------------------------------------------------------------------
>
> Key: PIG-2153
> URL: https://issues.apache.org/jira/browse/PIG-2153
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Reporter: Ken Goodhope
>
> When POProject.getNext(tuple) processes a tuple with one field, the field is
> pulled out. If that field is not a tuple, a cast exception is thrown. This
> is happening in the folliwing block of code at line 401.
> if(columns.size() == 1) {
> try{
> ret = inpValue.get(columns.get(0));
> ...
> res.result = (Tuple)ret;
> I am seeing this error in a unit test that is loading an array of floats.
> The LoadFunc is converting the array to bag, and wrapping the bag in a tuple.
>
> ({(3.3),(1.2),(5.6)})
> This results on POProject attempting to cast the bag to a tuple. Looking at
> the code, it appears that if I wrapped the previous tuple in another tuple,
> then it would work.
> (({(3.3),(1.2),(5.6)}))
> In this case it would work because POProject would extract the first inner
> tuple and return it. But this would require the LoadFunc to check for tuples
> with a single non-tuple field and only wrap those.
> This could be fixed by first checking that the tuple does actually wrap
> another tuple.
> if(columns.size() == 1 && inpValue.getType(0) == DataType.TUPLE)
> {...
> I don't know the original intent of this code well enough to say this is the
> appropriate fix or not. Hoping someone with more Pig experience can help
> here. Right now this is preventing the unit tests in AvroStorage from
> working. I can change the unit test, but I think in this case the unit test
> is catching a real bug.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
