[
https://issues.apache.org/jira/browse/PIG-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-5059:
------------------------------------
Fix Version/s: (was: 0.17.0)
0.18.0
> Pig 0.16 e2e Types_Order tests failed with Sort check failed
> ------------------------------------------------------------
>
> Key: PIG-5059
> URL: https://issues.apache.org/jira/browse/PIG-5059
> Project: Pig
> Issue Type: Bug
> Components: e2e harness
> Affects Versions: 0.16.0
> Environment: Ubuntu 14.04
> Mac OS
> CentOS 7.1
> Reporter: Konstantin Harasov
> Fix For: 0.18.0
>
>
> Env: core 5.2, centOS 7.1
> pig: pig-0.16
> Pig 0.16 e2e tests Types_Order_1,2,3,4,11,12,13,14,15,16 failed because of
> Sort check failed.
> test-base:
> [exec] =========================================================
> [exec] LOGGING RESULTS TO
> /opt/pig/pig-0.16/test/e2e/pig/testdist/out/log/test_harnesss_1478952742
> [exec] =========================================================
> [exec] Results so far, PASSED: 0 FAILED: 1 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 0 FAILED: 2 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 0 FAILED: 3 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 0 FAILED: 4 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 1 FAILED: 4 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 2 FAILED: 4 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 3 FAILED: 4 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 4 FAILED: 4 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 5 FAILED: 4 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 4 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 5 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 6 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 7 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 8 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 9 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 10 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Final results , PASSED: 6 FAILED: 10 SKIPPED: 0
> ABORTED: 0 FAILED DEPENDENCY: 0
> BUILD FAILED
> TEST: Types_Order_1
> sort
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_1_benchmark.out/out_original
> test cksum: 1595601925 208685
> benchmark cksum: 1595601925 208685
> Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_1.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_1.out/out_original:27:
> disorder: 18
> Sort check failed
> TEST: Types_Order_2
> Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original:23:
> disorder: zach young 3.34
> Sort check failed
> TEST: Types_Order_3
> Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original:27:
> disorder: 18
> Sort check failed
> TEST: Types_Order_4
> Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original:23:
> disorder: zach young 3.34
> Sort check failed
> TEST: Types_Order_11
> Going to run sort check command: sort -cs -t -k 3n
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original:731:
> disorder: oscar underhill 58 0.1
> Sort check failed
> TEST: Types_Order_12
> Going to run sort check command: sort -cs -t -k 3nr
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original:4:
> disorder: irene johnson 61 3.99
> Sort check failed
> TEST: Types_Order_13
> Going to run sort check command: sort -cs -t -k 3n
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original:731:
> disorder: oscar underhill 58 0.1
> Sort check failed
> TEST: Types_Order_14
> Going to run sort check command: sort -cs -t -k 3nr
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original:4:
> disorder: irene johnson 61 3.99
> Sort check failed
> TEST: Types_Order_15
> Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original:27:
> disorder: 18
> Sort check failed
> TEST: Types_Order_16
> Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original
> /bin/sort:
> ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original:23:
> disorder: zach young 3.34
> Sort check failed
> I did some research and create one assumption about the cause of the error:
> ROOT-CAUSE:
> In the process of investigation it became clear the following: when we
> store data in pig, we assign them the data type. This leads to the fact that
> the file size changed (probably because of the removal of extra delimiters).
> As a result of subsequent actions sorting is carried out already with the
> changed data (different file size). In bash (on Cent OS, Ubuntu and Mac OS),
> in turn, we do not have the ability to work with data types, and when we
> perform any sorting, we make them with the same "raw data". As a result, the
> check at the end of the test is incorrect because the files that are compared
> have different size.
> We also have different results during sorting on Cent OS/Ubuntu and Mac
> OS. All these inaccuracies lead to failure in sort check.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
