zhangliang created TRAFODION-2373: ------------------------------------- Summary: odb load 500k rows while set parameter max 505k Key: TRAFODION-2373 URL: https://issues.apache.org/jira/browse/TRAFODION-2373 Project: Apache Trafodion Issue Type: Bug Components: db-utility-odb Reporter: zhangliang Priority: Minor
I want to load 505k rows to trafodion with odb, by setting parameter like max=505000/rows=5000/parallel=6. However, odb just load 500k rows indeed. Logs here: [root@gy08 odb_test]# ./odb64luo -u zale -p zale -d gy_13_1 -l src=data_500k:tgt=trafodion.odb_copy.load_2:max=505000:rows=5000:parallel=6:loadcmd=UL:fs=,:bad=bad2 -vv odb [main(1512)] - Execution Table content (7 items): etab[0]: Type (.type): l Thread ID (.id): 0 Parent EID (.parent): 0 Child EID (.child): 0 TBE flag (.tbe): 1 Field Separator (.fs): 44 (decimal value) Record Separator (.rs): 10 (decimal value) String Qualifier (.sq): 0 (decimal value) Escape Character (.ec): 92 (decimal value) Embed file Character (.em): 0 (decimal value) Pad Character (.pc): 0 (decimal value) Run (.run): (null) Max Rows (.mr): 505000 Max Errors (.mer): 0 EID flags (.flg/.flg2): 16001000400/200000 (octal values) Rowset (.r): 5000 rows Rowset Buffer Size (.rbs): 0 bytes Row Buffer Size (.s): 0 bytes IO Buffer Size (.iobuff): 0 bytes RW Buffer Size (.buffsz): 262144 bytes Pre SQL (.pre): (null) MPre SQL (.mpre): (null) Post SQL (.post): (null) Database Type (.dbt): 0 Source (.src): data_500k Target (.Ocso[0-2]): (null).(null).(null) Map File (.map): (null) Lines to skip (.k): 0 Parallel Streams (.ps): 6 Null String (.ns): (null) Commit (.cmt): 0 (-1=end, 0=auto, >0 num rows) Field Truncation (.fldtr): 0 Load Command (.loadcmd): UL etab[1]: Type (.type): L Thread ID (.id): 1 Parent EID (.parent): 0 Child EID (.child): 0 TBE flag (.tbe): 1 Field Separator (.fs): 44 (decimal value) Record Separator (.rs): 10 (decimal value) String Qualifier (.sq): 0 (decimal value) Escape Character (.ec): 92 (decimal value) Embed file Character (.em): 0 (decimal value) Pad Character (.pc): 0 (decimal value) Run (.run): (null) Max Rows (.mr): 505000 Max Errors (.mer): 0 EID flags (.flg/.flg2): 16001000400/200000 (octal values) Rowset (.r): 5000 rows Rowset Buffer Size (.rbs): 0 bytes Row Buffer Size (.s): 0 bytes IO Buffer Size (.iobuff): 0 bytes RW Buffer Size (.buffsz): 262144 bytes Pre SQL (.pre): (null) MPre SQL (.mpre): (null) Post SQL (.post): (null) Database Type (.dbt): 0 Source (.src): data_500k Target (.Ocso[0-2]): (null).(null).(null) Map File (.map): (null) Lines to skip (.k): 0 Parallel Streams (.ps): 6 Null String (.ns): (null) Commit (.cmt): 0 (-1=end, 0=auto, >0 num rows) Field Truncation (.fldtr): 0 Load Command (.loadcmd): UL etab[2]: Type (.type): L Thread ID (.id): 2 Parent EID (.parent): 0 Child EID (.child): 0 TBE flag (.tbe): 1 Field Separator (.fs): 44 (decimal value) Record Separator (.rs): 10 (decimal value) String Qualifier (.sq): 0 (decimal value) Escape Character (.ec): 92 (decimal value) Embed file Character (.em): 0 (decimal value) Pad Character (.pc): 0 (decimal value) Run (.run): (null) Max Rows (.mr): 505000 Max Errors (.mer): 0 EID flags (.flg/.flg2): 16001000400/200000 (octal values) Rowset (.r): 5000 rows Rowset Buffer Size (.rbs): 0 bytes Row Buffer Size (.s): 0 bytes IO Buffer Size (.iobuff): 0 bytes RW Buffer Size (.buffsz): 262144 bytes Pre SQL (.pre): (null) MPre SQL (.mpre): (null) Post SQL (.post): (null) Database Type (.dbt): 0 Source (.src): data_500k Target (.Ocso[0-2]): (null).(null).(null) Map File (.map): (null) Lines to skip (.k): 0 Parallel Streams (.ps): 6 Null String (.ns): (null) Commit (.cmt): 0 (-1=end, 0=auto, >0 num rows) Field Truncation (.fldtr): 0 Load Command (.loadcmd): UL etab[3]: Type (.type): L Thread ID (.id): 3 Parent EID (.parent): 0 Child EID (.child): 0 TBE flag (.tbe): 1 Field Separator (.fs): 44 (decimal value) Record Separator (.rs): 10 (decimal value) String Qualifier (.sq): 0 (decimal value) Escape Character (.ec): 92 (decimal value) Embed file Character (.em): 0 (decimal value) Pad Character (.pc): 0 (decimal value) Run (.run): (null) Max Rows (.mr): 505000 Max Errors (.mer): 0 EID flags (.flg/.flg2): 16001000400/200000 (octal values) Rowset (.r): 5000 rows Rowset Buffer Size (.rbs): 0 bytes Row Buffer Size (.s): 0 bytes IO Buffer Size (.iobuff): 0 bytes RW Buffer Size (.buffsz): 262144 bytes Pre SQL (.pre): (null) MPre SQL (.mpre): (null) Post SQL (.post): (null) Database Type (.dbt): 0 Source (.src): data_500k Target (.Ocso[0-2]): (null).(null).(null) Map File (.map): (null) Lines to skip (.k): 0 Parallel Streams (.ps): 6 Null String (.ns): (null) Commit (.cmt): 0 (-1=end, 0=auto, >0 num rows) Field Truncation (.fldtr): 0 Load Command (.loadcmd): UL etab[4]: Type (.type): L Thread ID (.id): 4 Parent EID (.parent): 0 Child EID (.child): 0 TBE flag (.tbe): 1 Field Separator (.fs): 44 (decimal value) Record Separator (.rs): 10 (decimal value) String Qualifier (.sq): 0 (decimal value) Escape Character (.ec): 92 (decimal value) Embed file Character (.em): 0 (decimal value) Pad Character (.pc): 0 (decimal value) Run (.run): (null) Max Rows (.mr): 505000 Max Errors (.mer): 0 EID flags (.flg/.flg2): 16001000400/200000 (octal values) Rowset (.r): 5000 rows Rowset Buffer Size (.rbs): 0 bytes Row Buffer Size (.s): 0 bytes IO Buffer Size (.iobuff): 0 bytes RW Buffer Size (.buffsz): 262144 bytes Pre SQL (.pre): (null) MPre SQL (.mpre): (null) Post SQL (.post): (null) Database Type (.dbt): 0 Source (.src): data_500k Target (.Ocso[0-2]): (null).(null).(null) Map File (.map): (null) Lines to skip (.k): 0 Parallel Streams (.ps): 6 Null String (.ns): (null) Commit (.cmt): 0 (-1=end, 0=auto, >0 num rows) Field Truncation (.fldtr): 0 Load Command (.loadcmd): UL etab[5]: Type (.type): L Thread ID (.id): 5 Parent EID (.parent): 0 Child EID (.child): 0 TBE flag (.tbe): 1 Field Separator (.fs): 44 (decimal value) Record Separator (.rs): 10 (decimal value) String Qualifier (.sq): 0 (decimal value) Escape Character (.ec): 92 (decimal value) Embed file Character (.em): 0 (decimal value) Pad Character (.pc): 0 (decimal value) Run (.run): (null) Max Rows (.mr): 505000 Max Errors (.mer): 0 EID flags (.flg/.flg2): 16001000400/200000 (octal values) Rowset (.r): 5000 rows Rowset Buffer Size (.rbs): 0 bytes Row Buffer Size (.s): 0 bytes IO Buffer Size (.iobuff): 0 bytes RW Buffer Size (.buffsz): 262144 bytes Pre SQL (.pre): (null) MPre SQL (.mpre): (null) Post SQL (.post): (null) Database Type (.dbt): 0 Source (.src): data_500k Target (.Ocso[0-2]): (null).(null).(null) Map File (.map): (null) Lines to skip (.k): 0 Parallel Streams (.ps): 6 Null String (.ns): (null) Commit (.cmt): 0 (-1=end, 0=auto, >0 num rows) Field Truncation (.fldtr): 0 Load Command (.loadcmd): UL etab[6]: Type (.type): L Thread ID (.id): 6 Parent EID (.parent): 0 Child EID (.child): 0 TBE flag (.tbe): 1 Field Separator (.fs): 44 (decimal value) Record Separator (.rs): 10 (decimal value) String Qualifier (.sq): 0 (decimal value) Escape Character (.ec): 92 (decimal value) Embed file Character (.em): 0 (decimal value) Pad Character (.pc): 0 (decimal value) Run (.run): (null) Max Rows (.mr): 505000 Max Errors (.mer): 0 EID flags (.flg/.flg2): 16001000400/200000 (octal values) Rowset (.r): 5000 rows Rowset Buffer Size (.rbs): 0 bytes Row Buffer Size (.s): 0 bytes IO Buffer Size (.iobuff): 0 bytes RW Buffer Size (.buffsz): 262144 bytes Pre SQL (.pre): (null) MPre SQL (.mpre): (null) Post SQL (.post): (null) Database Type (.dbt): 0 Source (.src): data_500k Target (.Ocso[0-2]): (null).(null).(null) Map File (.map): (null) Lines to skip (.k): 0 Parallel Streams (.ps): 6 Null String (.ns): (null) Commit (.cmt): 0 (-1=end, 0=auto, >0 num rows) Field Truncation (.fldtr): 0 Load Command (.loadcmd): UL odb [2016-11-29 10:45:57]: starting ODBC connection(s)... (1) 1 2 3 4 5 6 Connected to Trafodion [1] 5,000 records inserted [commit] [2] 5,000 records inserted [commit] [3] 5,000 records inserted [commit] [4] 5,000 records inserted [commit] [5] 5,000 records inserted [commit] [6] 5,000 records inserted [commit] [1] 10,000 records inserted [commit] [2] 10,000 records inserted [commit] [3] 10,000 records inserted [commit] [4] 10,000 records inserted [commit] [5] 10,000 records inserted [commit] [6] 10,000 records inserted [commit] [1] 15,000 records inserted [commit] [2] 15,000 records inserted [commit] [3] 15,000 records inserted [commit] [4] 15,000 records inserted [commit] [5] 15,000 records inserted [commit] [6] 15,000 records inserted [commit] [1] 20,000 records inserted [commit] [2] 20,000 records inserted [commit] [3] 20,000 records inserted [commit] [4] 20,000 records inserted [commit] [5] 20,000 records inserted [commit] [6] 20,000 records inserted [commit] [1] 25,000 records inserted [commit] [2] 25,000 records inserted [commit] [3] 25,000 records inserted [commit] [4] 25,000 records inserted [commit] [5] 25,000 records inserted [commit] [6] 25,000 records inserted [commit] [1] 30,000 records inserted [commit] [2] 30,000 records inserted [commit] [3] 30,000 records inserted [commit] [4] 30,000 records inserted [commit] [5] 30,000 records inserted [commit] [6] 30,000 records inserted [commit] [1] 35,000 records inserted [commit] [2] 35,000 records inserted [commit] [3] 35,000 records inserted [commit] [4] 35,000 records inserted [commit] [5] 35,000 records inserted [commit] [6] 35,000 records inserted [commit] [1] 40,000 records inserted [commit] [2] 40,000 records inserted [commit] [3] 40,000 records inserted [commit] [4] 40,000 records inserted [commit] [5] 40,000 records inserted [commit] [6] 40,000 records inserted [commit] [1] 45,000 records inserted [commit] [2] 45,000 records inserted [commit] [3] 45,000 records inserted [commit] [4] 45,000 records inserted [commit] [5] 45,000 records inserted [commit] [6] 45,000 records inserted [commit] [1] 50,000 records inserted [commit] [2] 50,000 records inserted [commit] [3] 50,000 records inserted [commit] [4] 50,000 records inserted [commit] [5] 50,000 records inserted [commit] [6] 50,000 records inserted [commit] [1] 55,000 records inserted [commit] [2] 55,000 records inserted [commit] [3] 55,000 records inserted [commit] [4] 55,000 records inserted [commit] [5] 55,000 records inserted [commit] [6] 55,000 records inserted [commit] [1] 60,000 records inserted [commit] [2] 60,000 records inserted [commit] [3] 60,000 records inserted [commit] [4] 60,000 records inserted [commit] [5] 60,000 records inserted [commit] [6] 60,000 records inserted [commit] [1] 65,000 records inserted [commit] [2] 65,000 records inserted [commit] [3] 65,000 records inserted [commit] [4] 65,000 records inserted [commit] [5] 65,000 records inserted [commit] [6] 65,000 records inserted [commit] [1] 70,000 records inserted [commit] [2] 70,000 records inserted [commit] [3] 70,000 records inserted [commit] [4] 70,000 records inserted [commit] [5] 70,000 records inserted [commit] [6] 70,000 records inserted [commit] [1] 75,000 records inserted [commit] [2] 75,000 records inserted [commit] [3] 75,000 records inserted [commit] [4] 75,000 records inserted [commit] [5] 75,000 records inserted [commit] [6] 75,000 records inserted [commit] [1] 80,000 records inserted [commit] [2] 80,000 records inserted [commit] [3] 80,000 records inserted [commit] [4] 80,000 records inserted [commit] [5] 80,000 records inserted [commit] [1] 85,000 records inserted [commit] [2] 85,000 records inserted [commit] [3] 85,000 records inserted [commit] [4] 85,000 records inserted [commit] [6] 80,000 records inserted [commit] [0] odb version 1.1.0 Load statistics: [0] Target table: TRAFODION.ODB_COPY.LOAD_2 [0] Source: data_500k [0] Pre-loading time: 28.536 s (00:00:28.536) [0] Loading time: 25.695 s(00:00:25.695) [0] Total records read: 500,000 [0] Total records inserted: 500,000 [0] Total number of columns: 2 [0] Total bytes read: 5,388,895 [0] Average input row size: 10.8 B [0] ODBC row size: 14 B (data) + 16 B (len ind) [0] Rowset size: 5,000 [0] Rowset buffer size: 146.48 KiB [0] Load throughput (real data): 204.810 KiB/s [0] Load throughput (ODBC): 266.042 KiB/s [0] Reader Total/Wait Cycles: 100/94 This issue occurs when I set max=501k,502k and so on, odb just load 500k instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)