[jira] [Commented] (TRAFODION-1609) Change swstatus to give a simple "up" or "down" answer, instead of the current, confusing display
[
https://issues.apache.org/jira/browse/TRAFODION-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358455#comment-15358455
]
ASF GitHub Bot commented on TRAFODION-1609:
---
Github user mkby commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/562#discussion_r69251632
--- Diff: core/sqf/sql/scripts/install_local_hadoop ---
@@ -847,20 +847,31 @@ EOF
cat <$MY_SW_SCRIPTS_DIR/swstatus
#!/bin/sh
-cd ${MY_SW_ROOT}
-. $MY_SW_SCRIPTS_DIR/sw_env.sh
-NUM_JAVA_PROCS=\`ps -aef | grep \$USER | grep java | grep -v grep | wc -l\`
-NUM_MYSQLD_PROCS=\`ps -aef | grep \$USER | grep mysqld | grep -v grep | wc
-l\`
+cd \${MY_SW_ROOT}
+. \$MY_SW_SCRIPTS_DIR/sw_env.sh
+JPS_OUTPUT=\`jps\`
+SERVICES='HMaster NodeManager ResourceManager NameNode DataNode
SecondaryNameNode'
+for s in \$SERVICES; do
+if [[ ! \$JPS_OUTPUT =~ \$s ]]; then
+MISS_SERVICE="\$s \$MISS_SERVICE"
+fi
+done
-if [ "\$1" == "-v" ]; then
- ps -aef | grep \$USER | grep java | grep -v grep
- ps -aef | grep \$USER | grep mysqld | grep -v grep
+if [[ \$MISS_SERVICE != '' ]]; then
+echo "ERROR: Service \"\$MISS_SERVICE\" are not up!"
+exit 1
+else
+echo "The local hadoop services are up!"
fi
-echo "\$NUM_JAVA_PROCS java servers and \$NUM_MYSQLD_PROCS mysqld
processes are running"
-
-jps | grep -v Jps
-
+NUM_MYSQLD_PROCS=\`ps -aef | grep \$USER | grep mysqld | grep -v grep | wc
-l\`
+if [[ \$NUM_MYSQLD_PROCS -ne 0 ]]; then
+echo "\$NUM_MYSQLD_PROCS mysqld processes are running"
+exit 0
+else
+echo "ERROR: mysqld process is not running!"
+exit 1
+fi
EOF
--- End diff --
Thanks, I see. But they're all start/stop scripts, I don't want to modify
them.
> Change swstatus to give a simple "up" or "down" answer, instead of the
> current, confusing display
> -
>
> Key: TRAFODION-1609
> URL: https://issues.apache.org/jira/browse/TRAFODION-1609
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-general
>Reporter: Hans Zeller
>Assignee: Eason Zhang
>Priority: Minor
>
> The "swstatus" script used in cases where we did install_local_hadoop counts
> the MySQL processes and then simply invokes jps.
> It would be better if the script would check specifically for HMaster,
> NameNode, DataNode and maybe others, like NodeManager and ResourceManager. If
> any are missing, it should print out which ones are missing, otherwise it
> should just say something like "processes are up".
> It should also return an exit code of 0 if everything is up and 1 if some
> processes are missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TRAFODION-2034) Cannot output bad records in the loaded file even if specified file for storing bad records
[ https://issues.apache.org/jira/browse/TRAFODION-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358285#comment-15358285 ] ASF GitHub Bot commented on TRAFODION-2034: --- GitHub user lanbb opened a pull request: https://github.com/apache/incubator-trafodion/pull/570 [TRAFODION-2034] write failed row to bad record file when load with p… …arameter bad You can merge this pull request into a Git repository by running: $ git pull https://github.com/lanbb/incubator-trafodion master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafodion/pull/570.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #570 commit 0f28ece37e5bb85de1777df0ff29e499dd4c0b2d Author: Zale <498358...@qq.com> Date: 2016-06-30T17:56:44Z [TRAFODION-2034] write failed row to bad record file when load with parameter bad > Cannot output bad records in the loaded file even if specified file for > storing bad records > --- > > Key: TRAFODION-2034 > URL: https://issues.apache.org/jira/browse/TRAFODION-2034 > Project: Apache Trafodion > Issue Type: Bug > Components: db-utility-odb > Environment: centos6.7 HBase 1.0.0-cdh5.4.8 >Reporter: zhangliang >Priority: Minor > > This issue is from mantis 330. > Load file where is bad record in it, with option bad=output_data/bad_records > to hold the bad records, but the result is the file output_data/bad_records > generated but it is empty lines. > [root@suse-1 odb_test]# ./odb64luo -u trafodion -p traf123 -d traf -l > src=output_data/ext_person3.csv:pre=@scripts/ddl_person3.sql:tgt=trafodion.odb_test.person3:max=1000:rows=5000:parallel=5:loadcmd=UL:fs=\|:sq=\":bad=output_data/bad_records > odb [2016-04-25 19:17:05]: starting ODBC connection(s)... 0 1 2 3 4 5 > [0.0.0]Executing: 'drop table TRAFODION.odb_test.person3;' > [0.0.0]--- command executed in 7.279s (prep 0.001s, exec 7.278s, fetch > 0.000s/0.000s) > [0.0.1]Executing: 'CREATE TABLE TRAFODION.odb_test."PERSON3" ( > PID BIGINT SIGNED NOT NULL > ,FNAME CHAR(20) NOT NULL > ,LNAME CHAR(20) NOT NULL > ,COUNTRY VARCHAR(40) NOT NULL > ,CITY VARCHAR(40) NOT NULL > ,BDATE DATE NOT NULL > ,SEX CHAR(1) NOT NULL > ,EMAIL VARCHAR(40) NOT NULL > ,SALARY NUMERIC(9,2) NOT NULL > ,EMPL VARCHAR(40) NOT NULL > ,NOTES VARCHAR(80) > ,LOADTS TIMESTAMP(0) > ,PRIMARY KEY (PID) > );' > [0.0.1]--- command executed in 1.388s (prep 0.002s, exec 1.386s, fetch > 0.000s/0.000s) > Connected to Trafodion > [1] odb [Oloadbuff(9438)] - Error loading row 5 (State: 23000, Native 0) > [Trafodion ODBC Driver] GENERAL ERROR. Null Value in a non nullable column. > Row: 5 Column: 1 > [1] 999 records inserted [commit] > [0] odb version 1.1.0 Load statistics: > [0] Target table: TRAFODION.ODB_TEST.PERSON3 > [0] Source: output_data/ext_person3.csv > [0] Pre-loading time: 10.920 s (00:00:10.920) > [0] Loading time: 0.127 s(00:00:00.127) > [0] Total records read: 1,000 > [0] Total records inserted: 999 > [0] Total number of columns: 12 > [0] Total bytes read: 180,517 > [0] Average input row size: 180.5 B > [0] ODBC row size: 341 B (data) + 96 B (len ind) > [0] Rowset size: 1,000 > [0] Rowset buffer size: 426.76 KiB > [0] Load throughput (real data): 1,388.080 KiB/s > [0] Load throughput (ODBC): 2,619.487 KiB/s > [0] Reader Total/Wait Cycles: 1/0 > odb [2016-04-25 19:17:16]: exiting. Session Elapsed time 11.060 seconds > (00:00:11.060) > SQL>showddl person3; > > CREATE TABLE TRAFODION.ODB_TEST.PERSON3 > ( > PID LARGEINT NO DEFAULT NOT NULL NOT DROPPABLE > SERIALIZED > , FNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , LNAME CHAR(20) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , COUNTRY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , CITY VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , BDATE DATE NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , SEX CHAR(1) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , EMAIL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT DROPPABLE SERIALIZED > , SALARY NUMERIC(9, 2) NO DEFAULT NOT NULL NOT > DROPPABLE SERIALIZED > , EMPL VARCHAR(40) CHARACTER SET ISO88591 COLLATE > DEFAULT NO DEFAULT NOT NULL NOT D
[jira] [Updated] (TRAFODION-2085) Installer will not able to upgrade instance successfully if hbase_trx_jar file version changed
[ https://issues.apache.org/jira/browse/TRAFODION-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming updated TRAFODION-2085: Assignee: Amanda Moran (was: liu ming) > Installer will not able to upgrade instance successfully if hbase_trx_jar > file version changed > -- > > Key: TRAFODION-2085 > URL: https://issues.apache.org/jira/browse/TRAFODION-2085 > Project: Apache Trafodion > Issue Type: Bug > Components: installer >Affects Versions: 2.1-incubating >Reporter: Jianguo Zhou (Jason) >Assignee: Amanda Moran > > The installer will try to hbase_trx_jar file from /var/lib/trafodion to the > Hadoop distros location, and there was one hbase-trx-hdp2_3-2.0.0.jar out > there, then we run the upgrade, it will have anther > hbase-trx-hdp2_3-2.1.0.jar in the /var/lib/trafodion too, means 2 trx jar > file will be copied to Hadoop distros location, then the installer will fail > for checking logic as following: > for node in $HBASE_NODES > do >copiedOver=$(ssh -q -n $node sudo ls $HADOOP_PATH/hbase-trx* | wc -l) >if [[ $copiedOver -ne "1" ]]; then <-- 2 not equal 1, so failed then > echo "***ERROR: $hbase_trx_jar was not copied on $node" > echo "***ERROR: Please investigate why this happened" > echo "***ERROR: Trafodion can not start without this. EXITING..." > exit -1 >fi > done -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-2085) Installer will not able to upgrade instance successfully if hbase_trx_jar file version changed
[ https://issues.apache.org/jira/browse/TRAFODION-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-2085: --- Assignee: liu ming > Installer will not able to upgrade instance successfully if hbase_trx_jar > file version changed > -- > > Key: TRAFODION-2085 > URL: https://issues.apache.org/jira/browse/TRAFODION-2085 > Project: Apache Trafodion > Issue Type: Bug > Components: installer >Affects Versions: 2.1-incubating >Reporter: Jianguo Zhou (Jason) >Assignee: liu ming > > The installer will try to hbase_trx_jar file from /var/lib/trafodion to the > Hadoop distros location, and there was one hbase-trx-hdp2_3-2.0.0.jar out > there, then we run the upgrade, it will have anther > hbase-trx-hdp2_3-2.1.0.jar in the /var/lib/trafodion too, means 2 trx jar > file will be copied to Hadoop distros location, then the installer will fail > for checking logic as following: > for node in $HBASE_NODES > do >copiedOver=$(ssh -q -n $node sudo ls $HADOOP_PATH/hbase-trx* | wc -l) >if [[ $copiedOver -ne "1" ]]; then <-- 2 not equal 1, so failed then > echo "***ERROR: $hbase_trx_jar was not copied on $node" > echo "***ERROR: Please investigate why this happened" > echo "***ERROR: Trafodion can not start without this. EXITING..." > exit -1 >fi > done -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-2089) JDBC_T2 Makefile not add INCLUDE path for log4cxx installed by traf_tools_setup.sh
[ https://issues.apache.org/jira/browse/TRAFODION-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming resolved TRAFODION-2089. - Resolution: Fixed Fix Version/s: 2.1-incubating > JDBC_T2 Makefile not add INCLUDE path for log4cxx installed by > traf_tools_setup.sh > -- > > Key: TRAFODION-2089 > URL: https://issues.apache.org/jira/browse/TRAFODION-2089 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming > Fix For: 2.1-incubating > > > In Development enviorment, one can simply use traf_tools_setup.sh to install > thirdparty libraries, including log4cxx. > Most components Makefile will contain include path of that log4cxx, but > JDBC_T2 not, so if one doesn't install log4cxx in system path, build fail. > It is better to add log4cxx path into Makefile -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1676) support better runtime error message when a SQL function meet fital error
[
https://issues.apache.org/jira/browse/TRAFODION-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358086#comment-15358086
]
ASF GitHub Bot commented on TRAFODION-1676:
---
Github user traflm commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/564#discussion_r69231004
--- Diff: core/sql/cli/Statement.cpp ---
@@ -2298,6 +2298,10 @@ RETCODE Statement::resolveNames(LateNameInfoList *
lnil,
if (retcode != ex_expr::EXPR_OK)
{
diagsArea << DgSqlCode(-EXE_CONVERT_STRING_ERROR);
+ char hexstr[MAX_OFFENDING_SOURCE_DATA_DISPLAY_LEN];
--- End diff --
thanks Dave, I will change it. This may core dump the program.
> support better runtime error message when a SQL function meet fital error
> -
>
> Key: TRAFODION-1676
> URL: https://issues.apache.org/jira/browse/TRAFODION-1676
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-exe
>Reporter: liu ming
>Assignee: liu ming
>Priority: Minor
>
> A sql contains some SQL function, for example UPPER(), when the input data of
> UPPER() contains invalid data, UPPER() will fail. In this case, the whole
> query abort, and the error message cannot tell which row has invalid data, so
> it is very hard to fix the problem.
> It will be good to have row id or the real value of the error row in the
> error message to help trouble shooting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TRAFODION-2089) JDBC_T2 Makefile not add INCLUDE path for log4cxx installed by traf_tools_setup.sh
[ https://issues.apache.org/jira/browse/TRAFODION-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358009#comment-15358009 ] ASF GitHub Bot commented on TRAFODION-2089: --- Github user asfgit closed the pull request at: https://github.com/apache/incubator-trafodion/pull/567 > JDBC_T2 Makefile not add INCLUDE path for log4cxx installed by > traf_tools_setup.sh > -- > > Key: TRAFODION-2089 > URL: https://issues.apache.org/jira/browse/TRAFODION-2089 > Project: Apache Trafodion > Issue Type: Bug >Reporter: liu ming >Assignee: liu ming > > In Development enviorment, one can simply use traf_tools_setup.sh to install > thirdparty libraries, including log4cxx. > Most components Makefile will contain include path of that log4cxx, but > JDBC_T2 not, so if one doesn't install log4cxx in system path, build fail. > It is better to add log4cxx path into Makefile -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-2039) Add support for ALTER LIBRARY FILE ''
[ https://issues.apache.org/jira/browse/TRAFODION-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357634#comment-15357634 ] ASF GitHub Bot commented on TRAFODION-2039: --- Github user asfgit closed the pull request at: https://github.com/apache/incubator-trafodion/pull/563 > Add support for ALTER LIBRARY FILE '' > > > Key: TRAFODION-2039 > URL: https://issues.apache.org/jira/browse/TRAFODION-2039 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-general >Affects Versions: 2.0-incubating >Reporter: Suresh Subbiah >Assignee: Suresh Subbiah > > Add suppport for an ALTER statement that will allow the file associated with > a library object to be changed. > ALTER LIBRARY FILE ' ;' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1676) support better runtime error message when a SQL function meet fital error
[
https://issues.apache.org/jira/browse/TRAFODION-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357549#comment-15357549
]
ASF GitHub Bot commented on TRAFODION-1676:
---
Github user DaveBirdsall commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/564#discussion_r69178912
--- Diff: core/sql/cli/Statement.cpp ---
@@ -2298,6 +2298,10 @@ RETCODE Statement::resolveNames(LateNameInfoList *
lnil,
if (retcode != ex_expr::EXPR_OK)
{
diagsArea << DgSqlCode(-EXE_CONVERT_STRING_ERROR);
+ char hexstr[MAX_OFFENDING_SOURCE_DATA_DISPLAY_LEN];
--- End diff --
Need to allow room for a trailing null. You'll have to declare one more
byte here than you pass as the second parameter to stringToHex. Alternatively,
change the stringToHex function to allow for room (for example, change the
statement "outLen = (outLen / 2)" to "outLen = (outLen / 2) - 1"
> support better runtime error message when a SQL function meet fital error
> -
>
> Key: TRAFODION-1676
> URL: https://issues.apache.org/jira/browse/TRAFODION-1676
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-exe
>Reporter: liu ming
>Assignee: liu ming
>Priority: Minor
>
> A sql contains some SQL function, for example UPPER(), when the input data of
> UPPER() contains invalid data, UPPER() will fail. In this case, the whole
> query abort, and the error message cannot tell which row has invalid data, so
> it is very hard to fix the problem.
> It will be good to have row id or the real value of the error row in the
> error message to help trouble shooting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (TRAFODION-2090) improve the very first time of query compiling
liu ming created TRAFODION-2090: --- Summary: improve the very first time of query compiling Key: TRAFODION-2090 URL: https://issues.apache.org/jira/browse/TRAFODION-2090 Project: Apache Trafodion Issue Type: Improvement Reporter: liu ming Currently, the very first query run in master executor: sqlci or mxosrvr will take seconds to compile. Initial analysis shows most time spend on metadata retrieving. Need to improve the compile time for the very first time, at least from end user's perspective. Speed up the metadata retrieval speed. Redesign metadata scheme Prefetch metadata are all among possible solutions, the aim is to shorten the time spend on the SQL compile for the first time. Initial goal is within 1 second, faster the better. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-2016) doesn't support vanilla hbase 1.1
[ https://issues.apache.org/jira/browse/TRAFODION-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356862#comment-15356862 ] ASF GitHub Bot commented on TRAFODION-2016: --- GitHub user mashengchen opened a pull request: https://github.com/apache/incubator-trafodion/pull/568 TRAFODION-2016 support hbase 1.1 & cdh 5.5 for developer pls do set HBASE_DISTRO before source sqenv.sh export HBASE_DISTRO=APACHE1.0 (APACHE HBASE1.0) export HBASE_DISTRO=APACHE1.1 (APACHE HBASE1.1) export HBASE_DISTRO=CDH5.4(cloudera 1.0.0-cdh5.4.4) export HBASE_DISTRO=CDH5.5(cloudera 1.0.0-cdh5.5.1) export HBASE_DISTRO=HDP (hortonworks 2.3) You can merge this pull request into a Git repository by running: $ git pull https://github.com/mashengchen/incubator-trafodion hbase1.1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafodion/pull/568.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #568 commit b1ff89f145063ef633c8852495db875668c2fbd2 Author: traf Date: 2016-06-27T05:21:01Z jira1743 commit 324a981c69e9c150bd7730d63ad5049483749918 Author: mashengchen Date: 2016-06-29T07:42:59Z Merge pull request #1 from apache/master update head commit 6d08029a1cbd162b5573e3055ecfee6b45f8371a Author: traf Date: 2016-06-30T09:48:59Z TRAFODION-2016 support apache hbase1.1 & cdh5.5 commit 443b0c4abfcb8a4493deecb805b76568d8d1ed74 Author: traf Date: 2016-06-30T10:08:32Z remove other jira changes commit bacd8bff71ca271d1f8be92c03940bb2fbd4e575 Author: traf Date: 2016-06-30T10:10:34Z remove other jira changes > doesn't support vanilla hbase 1.1 > - > > Key: TRAFODION-2016 > URL: https://issues.apache.org/jira/browse/TRAFODION-2016 > Project: Apache Trafodion > Issue Type: Bug >Reporter: mashengchen >Assignee: mashengchen > > trafodion 2.1 support cdh5.4 & hdp2.3 & vanilla hbase 1.0. > but when use hbase 1.1 ,it can pass compile, the reason is some APIs and > method return type changed. > Need generate another suit of codes like what hdp2.3 do (hdp2.3 based on > hbase 1.1). > also need to support cdh5.5 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TRAFODION-2016) doesn't support vanilla hbase 1.1
[ https://issues.apache.org/jira/browse/TRAFODION-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mashengchen updated TRAFODION-2016: --- Description: trafodion 2.1 support cdh5.4 & hdp2.3 & vanilla hbase 1.0. but when use hbase 1.1 ,it can pass compile, the reason is some APIs and method return type changed. Need generate another suit of codes like what hdp2.3 do (hdp2.3 based on hbase 1.1). also need to support cdh5.5 was: trafodion 2.1 support cdh5.4 & hdp2.3 & vanilla hbase 1.0. but when use hbase 1.1 ,it can pass compile, the reason is some APIs and method return type changed. Need generate another suit of codes like what hdp2.3 do (hdp2.3 based on hbase 1.1). > doesn't support vanilla hbase 1.1 > - > > Key: TRAFODION-2016 > URL: https://issues.apache.org/jira/browse/TRAFODION-2016 > Project: Apache Trafodion > Issue Type: Bug >Reporter: mashengchen >Assignee: mashengchen > > trafodion 2.1 support cdh5.4 & hdp2.3 & vanilla hbase 1.0. > but when use hbase 1.1 ,it can pass compile, the reason is some APIs and > method return type changed. > Need generate another suit of codes like what hdp2.3 do (hdp2.3 based on > hbase 1.1). > also need to support cdh5.5 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
