[ https://issues.apache.org/jira/browse/PIG-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated PIG-5059: ------------------------------------ Fix Version/s: (was: 0.17.0) 0.18.0 > Pig 0.16 e2e Types_Order tests failed with Sort check failed > ------------------------------------------------------------ > > Key: PIG-5059 > URL: https://issues.apache.org/jira/browse/PIG-5059 > Project: Pig > Issue Type: Bug > Components: e2e harness > Affects Versions: 0.16.0 > Environment: Ubuntu 14.04 > Mac OS > CentOS 7.1 > Reporter: Konstantin Harasov > Fix For: 0.18.0 > > > Env: core 5.2, centOS 7.1 > pig: pig-0.16 > Pig 0.16 e2e tests Types_Order_1,2,3,4,11,12,13,14,15,16 failed because of > Sort check failed. > test-base: > [exec] ========================================================= > [exec] LOGGING RESULTS TO > /opt/pig/pig-0.16/test/e2e/pig/testdist/out/log/test_harnesss_1478952742 > [exec] ========================================================= > [exec] Results so far, PASSED: 0 FAILED: 1 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 0 FAILED: 2 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 0 FAILED: 3 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 0 FAILED: 4 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 1 FAILED: 4 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 2 FAILED: 4 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 3 FAILED: 4 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 4 FAILED: 4 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 5 FAILED: 4 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 6 FAILED: 4 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 6 FAILED: 5 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 6 FAILED: 6 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 6 FAILED: 7 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 6 FAILED: 8 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 6 FAILED: 9 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Results so far, PASSED: 6 FAILED: 10 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > [exec] Final results , PASSED: 6 FAILED: 10 SKIPPED: 0 > ABORTED: 0 FAILED DEPENDENCY: 0 > BUILD FAILED > TEST: Types_Order_1 > sort > ./out/pigtest/-1478952742-nightly.conf/Types_Order_1_benchmark.out/out_original > test cksum: 1595601925 208685 > benchmark cksum: 1595601925 208685 > Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n > ./out/pigtest/-1478952742-nightly.conf/Types_Order_1.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_1.out/out_original:27: > disorder: 18 > Sort check failed > TEST: Types_Order_2 > Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr > ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original:23: > disorder: zach young 3.34 > Sort check failed > TEST: Types_Order_3 > Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n > ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original:27: > disorder: 18 > Sort check failed > TEST: Types_Order_4 > Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr > ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original:23: > disorder: zach young 3.34 > Sort check failed > TEST: Types_Order_11 > Going to run sort check command: sort -cs -t -k 3n > ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original:731: > disorder: oscar underhill 58 0.1 > Sort check failed > TEST: Types_Order_12 > Going to run sort check command: sort -cs -t -k 3nr > ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original:4: > disorder: irene johnson 61 3.99 > Sort check failed > TEST: Types_Order_13 > Going to run sort check command: sort -cs -t -k 3n > ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original:731: > disorder: oscar underhill 58 0.1 > Sort check failed > TEST: Types_Order_14 > Going to run sort check command: sort -cs -t -k 3nr > ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original:4: > disorder: irene johnson 61 3.99 > Sort check failed > TEST: Types_Order_15 > Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n > ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original:27: > disorder: 18 > Sort check failed > TEST: Types_Order_16 > Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr > ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original > /bin/sort: > ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original:23: > disorder: zach young 3.34 > Sort check failed > I did some research and create one assumption about the cause of the error: > ROOT-CAUSE: > In the process of investigation it became clear the following: when we > store data in pig, we assign them the data type. This leads to the fact that > the file size changed (probably because of the removal of extra delimiters). > As a result of subsequent actions sorting is carried out already with the > changed data (different file size). In bash (on Cent OS, Ubuntu and Mac OS), > in turn, we do not have the ability to work with data types, and when we > perform any sorting, we make them with the same "raw data". As a result, the > check at the end of the test is incorrect because the files that are compared > have different size. > We also have different results during sorting on Cent OS/Ubuntu and Mac > OS. All these inaccuracies lead to failure in sort check. -- This message was sent by Atlassian JIRA (v6.3.15#6346)