[GitHub] trafodion pull request #1688: [TRAFODION-3034] Support Oracle Hierarchy Quer...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1688#discussion_r223166271 --- Diff: core/sql/optimizer/ValueDesc.cpp --- @@ -4555,7 +4555,7 @@ void ValueIdSet::unparse(NAString &result, NAString connectorText; - if ((form == MVINFO_FORMAT) || (form == QUERY_FORMAT)) + if ((form == MVINFO_FORMAT) || (form == QUERY_FORMAT) || (form == CONNECT_BY_FORMAT)) --- End diff -- CONNECT_BY_FORMAT has only one difference from QUERY_FORMAT, that the column name only has the last part, not qualified. The reason is: the column name will be used in the work() method of CONNECT_BY, where alias of the table name is lost. But we can make sure, there is only one table in that query in the work() method, so only the column name is required. ---
[GitHub] trafodion pull request #1714: [TRAFODION-3038] fix HDP download link in inst...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1714 [TRAFODION-3038] fix HDP download link in install_local_hadoop script You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3038 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1714.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1714 commit 94786dc9d47b225ae9233d86e8de51c6f669e33b Author: Liu Ming Date: 2018-09-13T13:21:17Z [TRAFODION-3038] fix HDP download link in install_local_hadoop script ---
[GitHub] trafodion pull request #1711: [TRAFODION-3201] revert wrong changes in TRAFO...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1711 [TRAFODION-3201] revert wrong changes in TRAFODION-3175 I had a wrong understanding of the outputInfo class, so a wrong 'fix' in TRAFODION-3175 it can save as many rows , the numEntries is for columns, not rows You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3201 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1711.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1711 commit 159a26dd038fc5078b0e29725efdfbaeeb9b55d8 Author: Liu Ming Date: 2018-09-09T11:14:22Z [TRAFODION-3201] revert wrong changes in TRAFODION-3175 ---
[GitHub] trafodion pull request #1710: [TRAFODION-3193]update cqd reference manual ab...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1710 [TRAFODION-3193]update cqd reference manual about the new CQD Add a new CQD which end user will use, so update the manual You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3193 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1710.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1710 commit c499dc009d8e208b4336a6541ca0b02a2263bbac Author: Liu Ming Date: 2018-09-09T02:47:24Z [TRAFODION-3193]update cqd reference manual about the new CQD ---
[GitHub] trafodion pull request #1704: [TRAFODION-2952] large amount of data will cau...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1704#discussion_r213515995 --- Diff: core/sql/cli/Cli.cpp --- @@ -10599,7 +10599,10 @@ static Lng32 SeqGenCliInterfaceUpdAndValidateMulti( numTries++; - DELAY(100 + numTries*25); + if( 100 + numTries*25 < 1000) //MAX is 1 second + DELAY(100 + numTries*25); + else + DELAY(1000); --- End diff -- Yes, this is a good suggestion ---
[GitHub] trafodion pull request #1704: [TRAFODION-2952] large amount of data will cau...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1704#discussion_r213515950 --- Diff: core/sql/sqlcomp/DefaultConstants.h --- @@ -2946,6 +2946,12 @@ enum DefaultConstants // this is used to change cache size of sequence numbers for a session. // It overwrites the cache size that was specified during sequence creation. TRAF_SEQUENCE_CACHE_SIZE, + + // this is used to set the retry time if two concurrent update of sequence + // conflict, and how many times will retry + // by default it is 250, when you saw error 1583, you can try to increase --- End diff -- thanks Dave, yes, I will change this ---
[GitHub] trafodion pull request #1704: [TRAFODION-2952] large amount of data will cau...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1704 [TRAFODION-2952] large amount of data will cause error in sequence ge⦠â¦nerating Currently, the sequence next range value is NOT protected by DTM, but a retry logic. If multiple processes try to use the same sequence, when the cache is used, all of them try to get the next range and go into race. So one may get a range that another get. The logic is any one get the next range will check the updated timestamp, if it is itself, then it is good for him. Otherwise, the range is got by someone else. So the code wait and retry. For the use case described in JIRA TRAFODION-2952, there are 48 partitioned table and want to do a load of 4 billion rows. It is high concurrency, and many times conflict. The that large load, it is very frequent to fail, since the retry time is hardcoded to 10. This patch introduce a CQD to control the retry number. It is set to a higher number by default, so in most cases, users don't need to modify this. But if still run into issue, one can change the CQD. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2952 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1704.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1704 commit 8958ce6aa28a96a15ed067dc49f556e0945165a6 Author: Liu Ming Date: 2018-08-25T15:20:32Z [TRAFODION-2952] large amount of data will cause error in sequence generating ---
[GitHub] trafodion pull request #1698: [TRAFODION-2968] update SQL manual for MySQL f...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1698#discussion_r212510010 --- Diff: docs/sql_reference/src/asciidoc/_chapters/sql_functions_and_expressions.adoc --- @@ -11290,6 +11295,99 @@ ID --- 1 row(s) selected. ``` +<<< +[[uuid_function]] +== UUID Function +Returns a Universal Unique Identifier (UUID) generated according to RFC 4122. +A UUID is designed as a number that is globally unique in space and time. +Two calls to UUID() are expected to generate two different values, even if +these calls are performed on two separate devices not connected to each other. + +NOTE: But UUID() can only be used in SELECT list, or very simple expressions to get a UUID. --- End diff -- We don't prevent it from being used in the WHERE clause. It can be used in some special case. It just we don't know where optimizer will choose to evaluate this function. If in ROOT, it will be one value for per statement, if it pushdown to SCAN, it will be one value for each row. So I will update the words as you suggested. ---
[GitHub] trafodion pull request #1698: [TRAFODION-2968] update SQL manual for MySQL f...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1698#discussion_r212510081 --- Diff: docs/sql_reference/src/asciidoc/_chapters/sql_functions_and_expressions.adoc --- @@ -442,7 +444,10 @@ the expressions have NULL values, the function returns a NULL value. | <> | Returns the value of the first operand unless it is NULL, in which case it returns the value of the second operand. | <> | Returns either the database user name of the current user who invoked the function or the database user name associated with the specified user ID number. -| <> | Returns a globally unique identifier. +| <> | Returns a globally unique identifier. (Oracle extention) --- End diff -- thank you, I will update the words ---
[GitHub] trafodion pull request #1698: [TRAFODION-2968] update SQL manual for MySQL f...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1698#discussion_r212509865 --- Diff: docs/sql_reference/src/asciidoc/_chapters/sql_functions_and_expressions.adoc --- @@ -11290,6 +11295,99 @@ ID --- 1 row(s) selected. ``` +<<< --- End diff -- Hi, Dave, we already have a SYS_GUID example ---
[GitHub] trafodion pull request #1698: [TRAFODION-2968] update SQL manual for MySQL f...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1698#discussion_r212509677 --- Diff: docs/sql_reference/src/asciidoc/_chapters/sql_functions_and_expressions.adoc --- @@ -442,7 +444,10 @@ the expressions have NULL values, the function returns a NULL value. | <> | Returns the value of the first operand unless it is NULL, in which case it returns the value of the second operand. | <> | Returns either the database user name of the current user who invoked the function or the database user name associated with the specified user ID number. -| <> | Returns a globally unique identifier. +| <> | Returns a globally unique identifier. (Oracle extention) +| <> | Returns a globally unique identifier. --- End diff -- UUID is MySQL function. I don't think ANSI define standard of SQL functions. In SQL Server, it is called newId(). So let me add comments like SYS_GUID, by adding "similar to MySQL UUID function"? ---
[GitHub] trafodion pull request #1699: [TRAFODION-2404] enhance the regexpr performan...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1699#discussion_r211479574 --- Diff: core/sql/exp/exp_like.cpp --- @@ -513,11 +512,16 @@ ex_expr::exp_return_type ExRegexpClauseChar::eval(char *op_data[], str_cpy_all(pattern, op_data[2], len2); str_cpy_all(srcStr, op_data[1], len1); - - z = regcomp(®, pattern, cflags); + if(rpattern_ != pattern) --- End diff -- Hi, Dave, rpattern_ is NAString, so I think this comparison is correct? My intent is to compare the content. Let me check the NAString further. ---
[GitHub] trafodion pull request #1699: [TRAFODION-2404] enhance the regexpr performan...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1699 [TRAFODION-2404] enhance the regexpr performance Previously, each time evaluate a row, it needs to invoke the regcomp(), by this change, only when the pattern is changed, invoke the regcomp() You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2404 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1699.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1699 commit cadd9e90526a5b857912805894d17d2bb1f5e8de Author: Liu Ming Date: 2018-08-19T08:49:07Z first check in, must switch to other task commit c41d24b8c73cc583c13e978e76ffb3ac2391d9a1 Author: Liu Ming Date: 2018-08-19T10:13:04Z enhance by adding comments and adding error check ---
[GitHub] trafodion pull request #1698: [TRAFODION-2968] update SQL manual for MySQL f...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1698 [TRAFODION-2968] update SQL manual for MySQL functions unix_timestamp, uuid, sleep etc add documentation for some newly added MySQL functions. For comparability enhancement of Trafodion. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2968 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1698.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1698 commit 694d6d407a6def098dfbac4223c0404b852d7081 Author: LiuMing Date: 2018-08-19T09:10:51Z add documentation for uuid() commit 95348f435b9c7585c09283a95093c9dc47b0cf03 Author: LiuMing Date: 2018-08-19T09:26:09Z add uuid_short() documentation commit aece4997c66b13fe845f980082713f17e07cc4d1 Author: LiuMing Date: 2018-08-19T09:36:19Z add documentation for sleep() function commit d8ad0c7a647863601c69bfa0ff52c168d7783138 Author: LiuMing Date: 2018-08-19T09:52:30Z add unix_timestamp documentation ---
[GitHub] trafodion pull request #1691: [TRAFODION-3182] cleanup monitor log content
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1691 [TRAFODION-3182] cleanup monitor log content At field, when we want to do trafodion trouble shooting. Open mon.xxx.log file, in many times, it is flood with message related STDIN redirection entries. These log entries are never real issues, and it is very hard to understand, so this PR is to move these log entries into DEBUG level. So only when an expert want to debug very complicated issue and need to turn-on the DEBUG log/trace, these info will be printed out. I am trying to make log file more useful, this is the first attempt. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3182 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1691.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1691 commit 5c8c990be5af0aba055b84b28477abae671ffa59 Author: Liu Ming Date: 2018-08-12T03:12:14Z [TRAFODION-3182] cleanup monitor log content ---
[GitHub] trafodion pull request #1690: [TRAFODION-1881] A better way to solve TRAFODI...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1690 [TRAFODION-1881] A better way to solve TRAFODION-1858 In my previous fix to TRAFODION-1858, I add a new method to check if a column is a system column by comparing the column name with "_SALT_", "_DIVISION_" etc, that is not a good way. This is a new implementation, it use more robust way to tell if a column is a system column. To check the RI, the predicate only need to check the user specified Primary Column. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-1881 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1690.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1690 commit cad4418c141fe5ea73888dc4b0a91272197cc14d Author: Liu Ming Date: 2018-08-12T02:05:31Z [TRAFODION-1881] A better way to solve TRAFODION-1858 ---
[GitHub] trafodion pull request #1689: [TRAFODION-3175] enhance ExeCliInterface get i...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1689 [TRAFODION-3175] enhance ExeCliInterface get interface to return null indicator enhance the ExeCliInterface get method to return NULL ptr for NULL value. So the caller is able to determine the null value. This is only for the fetchAllRow interface, where later the caller need to call OutputInfo method get, instead of adding a new array to save null indicator, I choose to reuse the data_ pointer You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3175 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1689.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1689 commit e2a4b805dbac396d9892fe696b1b60307f04ec1e Author: Liu Ming Date: 2018-08-11T13:03:47Z [TRAFODION-3175] enhance ExeCliInterface get interface to return null indicator commit 5ddd7bfc89c9230d14aeb4b61272036a9797830a Author: Liu Ming Date: 2018-08-11T20:17:39Z for not nullalble column bypass the indicator get to avoid core dump ---
[GitHub] trafodion pull request #1688: [TRAFODION-3034] Support Oracle Hierarchy Quer...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1688#discussion_r209260745 --- Diff: core/sql/optimizer/BindRelExpr.cpp --- @@ -2505,7 +2505,62 @@ RelExpr *RelExpr::bindSelf(BindWA *bindWA) if (bindWA->inViewWithCheckOption()) bindWA->predsOfViewWithCheckOption() += selectionPred(); } +#if 0 + ItemExpr *startWithTree = removeStartWithTree(); + if (startWithTree) { +bindWA->getCurrentScope()->context()->inWhereClause() = TRUE; +startWithTree->convertToValueIdSet(getStartWith(), bindWA, ITM_AND); +bindWA->getCurrentScope()->context()->inWhereClause() = FALSE; + +if (bindWA->errStatus()) return this; +// If this is an embedded insert, then subquery predicates are not +// allowed. +// For example: To handle this query and issue an error stating +// subqueries are not allowed in embedded inserts +// +// select a from (insert into t901t01 values(22,22,222))t(a,b,c) +// where t.a IN (select m from t901t03 where t901t03.m = 77); + +if (getGroupAttr()->isEmbeddedInsert()) +{ + if (!getStartWith().isEmpty() && getStartWith().containsSubquery()) + { + *CmpCommon::diags() << DgSqlCode(-4337); + bindWA->setErrStatus(); + return this; + } +} + } + + ItemExpr *connectByTree = removeConnectByTree(); + if (connectByTree) { + +bindWA->getCurrentScope()->context()->inWhereClause() = TRUE; +connectByTree->convertToValueIdSet(getConnectBy(), bindWA, ITM_AND); +bindWA->getCurrentScope()->context()->inWhereClause() = FALSE; + +if (bindWA->errStatus()) return this; + +// If this is an embedded insert, then subquery predicates are not +// allowed. +// For example: To handle this query and issue an error stating +// subqueries are not allowed in embedded inserts +// +// select a from (insert into t901t01 values(22,22,222))t(a,b,c) +// where t.a IN (select m from t901t03 where t901t03.m = 77); + +if (getGroupAttr()->isEmbeddedInsert()) +{ + if (!getConnectBy().isEmpty() && getConnectBy().containsSubquery()) + { + *CmpCommon::diags() << DgSqlCode(-4337); + bindWA->setErrStatus(); + return this; + } +} + } +#endif --- End diff -- oops, I will remove these dead code. ---
[GitHub] trafodion pull request #1688: [TRAFODION-3034] Support Oracle Hierarchy Quer...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1688 [TRAFODION-3034] Support Oracle Hierarchy Query (Connect By) This PR is the first patch to support Oracle Hierarchy query feature (CONNECT BY). In this PR, the feature is implemented as a new SQL utility, It is standalone, rather clear isolated with all other SQL functions, so the impact is minimal. In the long run, we should finish the ANSI recursive feature (recursive WITH) , and at that time, this feature can be considered to refactor to use that infrastructure. This is just the first phase of this feature. The basic logic is simple: the utility will run a query to get all start values (specified by the START WITH clause), then it will construct queries to search for children of the root, and loop until no children can be found. Oracle has 3 pseudo columns, to support the ISLEAF and CONNECT_BY_PATH, the utility will have to run a query for each parent, it will be rather slow. If a query doesn't have those two pseudo columns required, the utility will run in batch mode, for each iteration, get all children in one query. That will be much faster. One can check the executor/TEST021 for how this feature works first. This will be a long review process, there must be many places to be modified and enhanced, thank you all for help in advance. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3034 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1688.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1688 commit 5fb8f21f20c397e077b2f7a8f96a7be9819f4883 Author: Liu Ming Date: 2018-06-01T22:44:58Z [TRAFODION-3034] add parser changes commit b29f4ddb417b087a3c888b17e9fae2ee6b060828 Author: Liu Ming Date: 2018-07-17T09:21:17Z basic work finish commit 88e8b20d91ed7093ea06448a64bcdc3090d91ba9 Author: Liu Ming Date: 2018-07-21T09:56:44Z fix parser, add loop detection commit 9fb3a5943f6440540b4894bc554d94b317fd9ece Author: Liu Ming Date: 2018-07-23T05:28:08Z refactor the code, next need to add CQD and where clause support commit b875ec3cdaef2c33cc3d127c24e62be478a78781 Author: Liu Ming Date: 2018-07-25T04:22:21Z support where clause, next add CQD to control memory usage commit 1b853c5da8210d77bd8d84dac374e57a67e38bef Author: Liu Ming Date: 2018-07-26T05:00:23Z support where clause, next add regression test commit f20b63c875f3a56ba3da1a83c2f1e0f255b29b71 Author: Liu Ming Date: 2018-07-27T12:03:44Z support both where clause and order by, next add test cases commit 5757f25e9c2fa91b25751e24f28b04fbf903df4e Author: Liu Ming Date: 2018-07-28T02:54:56Z add test case, next to rebase and try to support is_cycle commit 73a175a19c4f77e2af761ecc2a0b01779f9b7175 Author: Liu Ming Date: 2018-07-28T05:15:14Z Merge branch 'master' of git://git.apache.org/trafodion into TRAFODION-3034 Conflicts: core/sql/generator/GenRelExeUtil.cpp core/sql/optimizer/RelExeUtil.h core/sql/sqlcomp/DefaultConstants.h commit 1b32b6b27b7ba4686b174c41a349f320807221fe Author: Liu Ming Date: 2018-08-04T10:44:33Z support PATH, next try to support is leaf commit 81ef30d0e97797e033522b711f1aeb5ba359cd94 Author: Liu Ming Date: 2018-08-05T09:43:03Z add ISLEAF support, but need to wait for another enhancement commit d781d881ad400e75fcd765d1856bdb509e2c6adb Author: Liu Ming Date: 2018-08-07T20:48:56Z finish is leaf commit cc9057d27aec9d0e5413330f72cc8df932103e7d Author: Liu Ming Date: 2018-08-07T20:49:08Z Merge branch 'master' of git://git.apache.org/trafodion into TRAFODION-3034 commit 8ae2ed9c5fcbb1ccb47f6b952438dedc06b0d90e Author: Liu Ming Date: 2018-08-10T05:27:13Z first phase finished ---
[GitHub] trafodion pull request #1686: [TRAFODION-3172] update documentation for sys_...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1686 [TRAFODION-3172] update documentation for sys_guid function update the doc for new added function You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3172 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1686.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1686 commit e1e35258af0c165a4afbbd08c8b98a999c6eb2a3 Author: LiuMing Date: 2018-08-08T00:01:54Z [TRAFODION-3172] update documentation for sys_guid function ---
[GitHub] trafodion pull request #1679: [TRAFODION-3163] Add script to do STFS health ...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1679#discussion_r208065409 --- Diff: core/sqf/src/tm/tools/dtmci.cpp --- @@ -1801,6 +1801,20 @@ int main(int argc, char *argv[]) << "' for status command." << endl; } } +else if (!strcmp(lp_nextcmd, "showenv")) +{ + get_cmd(lp_inputstr, lp_nextcmd); + if (lp_nextcmd[0] == '\0') + cout <<"TODO: show all env var\n" << endl; + else + { + const char* v = ms_getenv_str((const char*)lp_nextcmd); + if(v == NULL) +cout <<"*** Enviorement Var not exist" << endl; --- End diff -- thanks Dave, I will change this typo ---
[GitHub] trafodion pull request #1680: [TRAFODION-3173] enhance ExeCliInterface fetch...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1680 [TRAFODION-3173] enhance ExeCliInterface fetchAllRows to save the dat⦠â¦a type You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3173 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1680.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1680 commit f75ea3f7f101f67089a4874b750c58e8a3f24b39 Author: Liu Ming Date: 2018-08-06T04:30:29Z [TRAFODION-3173] enhance ExeCliInterface fetchAllRows to save the data type ---
[GitHub] trafodion pull request #1679: [TRAFODION-3163] Add script to do STFS health ...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1679 [TRAFODION-3163] Add script to do STFS health check It is hard to test the STFS status if the cluster has many nodes and the STFS configured with many directories. This script will help to do basic checking. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3163 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1679.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1679 commit a864ee1c947182443ba370158529162628b46d76 Author: LiuMing Date: 2018-08-01T15:40:40Z add command to dtmci commit 399b2bdf5b65d2dff0e578461cdb03c8d2430577 Author: LiuMing Date: 2018-08-05T04:05:55Z fix bugs commit 0b3b5b9f4c2ff77b6fd05f495cf908ed32421ff5 Author: LiuMing Date: 2018-08-05T06:34:07Z add a wrapper script ---
[GitHub] trafodion pull request #1676: TRAFODION-3165 HDFS storage option not work we...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1676 TRAFODION-3165 HDFS storage option not work well if table use namespace If the table has namespace, the path of that table in HDFS should not in /hbase/data/default, but in /hbase/data// You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3165 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1676.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1676 commit 4ac689a67a80e3627e20a3146565513bd1e5a4ba Author: Liu Ming Date: 2018-08-02T18:51:28Z TRAFODION-3165 HDFS storage option not work well if table use namespace ---
[GitHub] trafodion pull request #1666: TRAFODION-3167 add Maven Central Repository UR...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1666 TRAFODION-3167 add Maven Central Repository URL for sql pom.xml You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3167 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1666.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1666 commit 8dedd6f002c0ffa4c93a9a656d0c80ac59c33f92 Author: LiuMing Date: 2018-07-28T07:25:04Z TRAFODION-3167 add Maven Central Repository URL for sql pom.xml ---
[GitHub] trafodion pull request #1661: fix regression failure
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1661 fix regression failure Sorry, need yet another fix to the GET command display change, which causing failures. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3162-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1661.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1661 commit 74650ac9a44726edf4e74ec132533552c8bafd71 Author: Liu Ming Date: 2018-07-24T12:49:54Z fix regression failure ---
[GitHub] trafodion pull request #1659: [TRAFODION-3162] fix regression failure get co...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1659 [TRAFODION-3162] fix regression failure get command fix regression failure You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3162 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1659.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1659 commit 8956a5b11d2b49846d95326924a52d0038efd361 Author: Liu Ming Date: 2018-07-23T09:53:07Z [TRAFODION-3162] fix regression failure get command ---
[GitHub] trafodion pull request #1657: [TRAFODION-3147] support sys_guid function
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1657 [TRAFODION-3147] support sys_guid function Support Oracle sys_guid function as described: https://docs.oracle.com/cd/B12037_01/server.101/b10759/functions153.htm This is required by some users You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3147 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1657.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1657 commit 6ad70a36245a6dacedda0d8740c7d84cff5bf707 Author: Liu Ming Date: 2018-07-21T15:03:49Z [TRAFODION-3147] support sys_guid function commit a748fbb91c131d9c257886aba59d8042d9f7215c Author: Liu Ming Date: 2018-07-21T15:33:19Z add regress test for sys_guid ---
[GitHub] trafodion pull request #1646: [TRAFODION-3145] Make "hbase_options" as defau...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1646#discussion_r202908302 --- Diff: core/sql/sqlcomp/CmpSeabaseDDLcommon.cpp --- @@ -10991,7 +10997,20 @@ CmpSeabaseDDL::setupHbaseOptions(ElemDDLHbaseOptions * hbaseOptionsClause, hbaseOptionsStr += optionStr; } } + if (!flushSize.isNull() && !memstoreFlushSizeOptionSpecified) +{ + hbaseOption = new(STMTHEAP) HbaseCreateOption("MEMSTORE_FLUSH_SIZE", +flushSize.data()); + hbaseCreateOptions.insert(hbaseOption); + if (ActiveSchemaDB()->getDefaults().userDefault + (HBASE_MEMSTORE_FLUSH_SIZE_OPTION) == TRUE) +{ + numHbaseOptions += 1; + sprintf(optionStr, "MEMSTORE_FLUSH_SIZE='%s'|", flushSize.data()); --- End diff -- I agree, I can change this into a more robust coding style, using snprintf. ---
[GitHub] trafodion pull request #1646: [TRAFODION-3145] Make "hbase_options" as defau...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1646 [TRAFODION-3145] Make "hbase_options" as default setting and use cqd ⦠â¦to change the default setting As we know, for performance thinking, we always need to add below syntax in âcreate tableâ syntax. And I know in latest version, we have made âATTRIBUTES ALIGNED FORMATâ as default, do we have a plan that make HBASE_OPTIONS(including ENCODINGãCOMPRESSIONãMEMESTORE) as the default setting when creating table? I think this will be easier for end users? ATTRIBUTES ALIGNED FORMAT HBASE_OPTIONS ( DATA_BLOCK_ENCODING = 'FAST_DIFF', COMPRESSION = 'SNAPPY', MEMSTORE_FLUSH_SIZE = '1073741824' ) ; Trafodion has cqd HBASE_DATA_BLOCK_ENCODING_OPTION âFAST_DIFFâ; cqd HBASE_COMPRESSION_OPTION âSNAPPYâ; Add cqd HBASE_MEMSTORE_FLUSH_SIZE_OPTION '1073741824' ; So one can use these values without explicitly specify them when create table. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3145 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1646.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1646 commit 86fdb25c09785da8d17b05e38e4f802af97932e9 Author: Liu Ming Date: 2018-07-15T22:37:40Z [TRAFODION-3145] Make "hbase_options" as default setting and use cqd to change the default setting ---
[GitHub] trafodion pull request #1619: [TRAFODION-3101] enhance get command to show r...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1619#discussion_r197692791 --- Diff: core/sql/regress/compGeneral/EXPECTED023 --- @@ -23,6 +23,10 @@ SB_HISTOGRAMS SB_HISTOGRAM_INTERVALS SB_PERSISTENT_SAMPLES +=== --- End diff -- Let me change it step by step. ---
[GitHub] trafodion pull request #1619: [TRAFODION-3101] enhance get command to show r...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1619#discussion_r197692662 --- Diff: core/sql/regress/compGeneral/EXPECTED023 --- @@ -23,6 +23,10 @@ SB_HISTOGRAMS SB_HISTOGRAM_INTERVALS SB_PERSISTENT_SAMPLES +=== --- End diff -- I agree, this looks much better! ---
[GitHub] trafodion pull request #1619: [TRAFODION-3101] enhance get command to show r...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1619#discussion_r197666711 --- Diff: core/sql/executor/ExExeUtilGet.cpp --- @@ -949,6 +949,7 @@ short ExExeUtilGetMetadataInfoTcb::displayHeading() { if (getMItdb().noHeader()) { +printf("LMDBG noheader is true\n"); --- End diff -- sorry, I will remove this... ---
[GitHub] trafodion pull request #1619: [TRAFODION-3101] enhance get command to show r...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1619 [TRAFODION-3101] enhance get command to show rows get At field, engineers issue get tables to list all tables, and want to count how many tables got, it will be good to show that info by 'get' command itself. The current implementation is to show that count as part of get command output, just as header info. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3101 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1619.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1619 commit 591e0fa60a13f9f57eee1bc7a9b35fe5a10145a2 Author: Liu Ming Date: 2018-06-25T00:48:37Z [TRAFODION-3101] enhance get command to show rows get ---
[GitHub] trafodion pull request #1613: TRAFODION-3111 CentOS 7 build error
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1613 TRAFODION-3111 CentOS 7 build error Some new code checkin broke the build on CentOS 7 I will continously monitor the CentOS 7 build until the jenkins setup. CentOS 7 jenkins test now failed on some UDR regression, it is hard to find out root cause yet, still need some time. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3111 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1613.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1613 commit ac974faaa9523bd385efe5d73fff9766453e47be Author: Liu Ming Date: 2018-06-19T00:58:24Z TRAFODION-3111 CentOS 7 build error ---
[GitHub] trafodion pull request #1532: [TRAFODION-3026] add create option storage pol...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1532#discussion_r193922170 --- Diff: core/sql/src/main/java/org/trafodion/sql/HBaseClient.java --- @@ -538,7 +564,21 @@ public boolean createk(String tblName, Object[] tableOptions, admin.createTable(desc); } } -admin.close(); + +if(setDescRet!= null) +{ + if(setDescRet.storagePolicyChanged()) + { + Object tableOptionsStoragePolicy[] = new Object[HBASE_HDFS_STORAGE_POLICY+1]; + for(int i=0; i
[GitHub] trafodion pull request #1532: [TRAFODION-3026] add create option storage pol...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1532#discussion_r193921823 --- Diff: core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/coprocessor/transactional/TrxRegionEndpoint.java.tmpl --- @@ -2404,6 +2410,72 @@ CoprocessorService, Coprocessor { done.run(TlogDel_response); } + public void setStoragePolicy(RpcController controller, + TrafSetStoragePolicyRequest request, + RpcCallback done) { +String path = request.getPath(); +String policy = request.getPolicy(); +if (LOG.isTraceEnabled()) LOG.trace("setStoragePolicy ENTRY. path " + path + " policy " + policy ); + +IOException t=null; +try { + invokeSetStoragePolicy(fs, path, policy); +} +catch (IOException e) { + t = e; +} + + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrafSetStoragePolicyResponse.Builder setStoragePolicyResponseBuilder = + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrafSetStoragePolicyResponse.newBuilder(); + +if(t != null) +{ + LOG.error("setStoragePolicy error : " + t.toString() ); + setStoragePolicyResponseBuilder.setStatus(false); + setStoragePolicyResponseBuilder.setException(t.toString()); +} +else +{ + setStoragePolicyResponseBuilder.setStatus(true); + setStoragePolicyResponseBuilder.setException(""); +} + +TrafSetStoragePolicyResponse resp = setStoragePolicyResponseBuilder.build(); + +done.run(resp); + + } + + private static void invokeSetStoragePolicy(final FileSystem fs, final String pathstr, + final String storagePolicy) + throws IOException { +Path path = new Path(pathstr); +Method m = null; +try { +m = fs.getClass().getDeclaredMethod("setStoragePolicy", +new Class[] { Path.class, String.class }); +m.setAccessible(true); --- End diff -- Yes, Selva, this is a good suggestion. ---
[GitHub] trafodion pull request #1532: [TRAFODION-3026] add create option storage pol...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1532#discussion_r193921772 --- Diff: core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/client/transactional/TransactionManager.java --- @@ -3226,5 +3260,65 @@ public RecoveryRequestResponse call(TrxRegionService instance) throws IOExceptio return resultArray[0].getResultList(); } + +public void setStoragePolicy(String tblName, String policy) + throws IOException { + + int retryCount = 0; + int retrySleep = TM_SLEEP; + boolean retry = false; + try { +Table tbl = connection.getTable(TableName.valueOf(tblName)); +String rowkey = "0"; +CoprocessorRpcChannel channel = tbl.coprocessorService(rowkey.getBytes()); + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrxRegionService.BlockingInterface service = + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrxRegionService.newBlockingStub(channel); + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrafSetStoragePolicyRequest.Builder request = + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrafSetStoragePolicyRequest.newBuilder(); +String hbaseRoot = config.get("hbase.rootdir"); +FileSystem fs = FileSystem.get(config); + //Construct the HDFS dir + //find out if namespace is there + String[] parts = tblName.split(":"); + String namespacestr=""; + String fullPath = hbaseRoot + "/data/" ; + String fullPath2 = hbaseRoot + "/data/default/"; + if(fs.exists(new Path(fullPath2))) +fullPath = fullPath2; + + if(parts.length >1) //have namespace +fullPath = fullPath + parts[0] + "/" + parts[1]; + else +fullPath = fullPath + tblName; + + request.setPath(fullPath); + request.setPolicy(policy); + + do { + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrafSetStoragePolicyResponse ret = +service.setStoragePolicy(null,request.build()); + + //handle result and error + if( ret == null) + { +LOG.error("setStoragePolicy Response ret null "); + } + else if (ret.getStatus() == false) + { +LOG.error("setStoragePolicy Response ret false: " + ret.getException()); +throw new IOException(ret.getException()); + } + if(retryCount == RETRY_ATTEMPTS) + { +throw new IOException("coprocessor not response"); + } + if (retry) + retrySleep = retry(retrySleep); +} while (retry && retryCount++ < RETRY_ATTEMPTS); + } + catch (Exception e) { + throw new IOException(e); + } + } --- End diff -- thanks Selva! Sorry, I should be more careful, I never understand the LOG and exception clear, will change this. ---
[GitHub] trafodion pull request #1532: [TRAFODION-3026] add create option storage pol...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1532#discussion_r193921336 --- Diff: core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/client/transactional/TransactionManager.java --- @@ -2996,6 +3022,11 @@ else if (tableOption.equalsIgnoreCase("USE_DEFAULT")) (Long.parseLong(tableOption)); returnStatus.setTableDescriptorChanged(); break ; + case HBASE_HDFS_STORAGE_POLICY: --- End diff -- Hi, Prashanth, the hdfs path of the table is available only after the table is created. And it is easier to set this in alter, since it directly support the SQL ALTER TABLE to change this option. But I was planning to test and support other DDL including ALTER in phase two. But add code here has one consideration of the reason above. Let me consider how to test this out, as I understand, it will never be late, since the policy can be changed at anytime. ---
[GitHub] trafodion pull request #1532: [TRAFODION-3026] add create option storage pol...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1532#discussion_r193921620 --- Diff: core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/client/transactional/TransactionManager.java --- @@ -3226,5 +3260,65 @@ public RecoveryRequestResponse call(TrxRegionService instance) throws IOExceptio return resultArray[0].getResultList(); } + +public void setStoragePolicy(String tblName, String policy) + throws IOException { + + int retryCount = 0; + int retrySleep = TM_SLEEP; + boolean retry = false; + try { +Table tbl = connection.getTable(TableName.valueOf(tblName)); +String rowkey = "0"; +CoprocessorRpcChannel channel = tbl.coprocessorService(rowkey.getBytes()); + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrxRegionService.BlockingInterface service = + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrxRegionService.newBlockingStub(channel); + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrafSetStoragePolicyRequest.Builder request = + org.apache.hadoop.hbase.coprocessor.transactional.generated.TrxRegionProtos.TrafSetStoragePolicyRequest.newBuilder(); +String hbaseRoot = config.get("hbase.rootdir"); +FileSystem fs = FileSystem.get(config); + //Construct the HDFS dir + //find out if namespace is there + String[] parts = tblName.split(":"); + String namespacestr=""; --- End diff -- The API is only available at HBase 2.0. But I don't know when Trafodion will support HBase 2.0. And this feature is required by a customer, so we cannnot wait for months. This is a temp solution, once we move to HBase 2.0, most of these code can be eliminated and using HBase public API. ---
[GitHub] trafodion pull request #1592: Trafodion 3025 support order by with limit N i...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1592 Trafodion 3025 support order by with limit N in subquery ORDER BY in subquery is ignored by Trafodion, when in the subquery, there is a limit N, it is good to keep the order by: select * from t ( select * from t order by c limit 20); for example You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3025 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1592.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1592 commit e45745fece09450f59a1adebc7b3413b35e54c50 Author: Liu Ming Date: 2018-06-02T06:37:54Z [TRAFODION-3025] support order by in subquery if there is limit N commit fda906dcd94ff28957b32c825e04206e57160841 Author: Liu Ming Date: 2018-06-02T07:33:32Z add test ---
[GitHub] trafodion pull request #1591: TRAFODION-3076 update pom.xml.hdp
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1591 TRAFODION-3076 update pom.xml.hdp for protobuf compile You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3076 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1591.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1591 commit 2f791126896b05fde20b181366e7b6c9bca77a08 Author: Liu Ming Date: 2018-05-19T11:36:55Z TRAFODION-3076 update pom.xml.hdp ---
[GitHub] trafodion pull request #1571: [TRAFODION-3075] DCS start fail at CentOS 7
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1571 [TRAFODION-3075] DCS start fail at CentOS 7 This is a temp solution. Root cause need much more time to figure out. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3075 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1571.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1571 commit 92e34167fd03b230f329624c76dc55323a9f27cd Author: LiuMing Date: 2018-05-20T10:33:28Z [TRAFODION-3075] DCS start fail at CentOS 7 ---
[GitHub] trafodion pull request #1570: [TRAFODION-3015] query no result when comparin...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1570 [TRAFODION-3015] query no result when comparing string with BIGNUM When the column is BIGNUM, and the predicate is comparing to a string. Trafodion cast the string into FLOAT and the convert FLOAT into BIGNUM, There is an issue when convert FLOAT into BIGNUM that didn't considering the rounding: so 2.738046768E+007 will become 27380467 instead of 27380468 You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3015-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1570.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1570 commit fac342d0b06c1e248b6389c070ccb62ce37a60a9 Author: Liu Ming Date: 2018-05-19T10:47:14Z [TRAFODION-3015] compare string to BIGNUM return no result commit a08afe4940b828309e80f91ee0f2c29aaf08f5c6 Author: Liu Ming Date: 2018-05-19T11:56:59Z add regression test ---
[GitHub] trafodion pull request #1569: [TRAFODION-3075] Installer needs to support RH...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1569 [TRAFODION-3075] Installer needs to support RH7 / Centos7 You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3075 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1569.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1569 commit 580be156af361f7d9ab0302b4468fb4324480776 Author: LiuMing Date: 2018-05-19T04:32:17Z [TRAFODION-3075] Installer needs to support RH7 / Centos7 ---
[GitHub] trafodion pull request #1564: [TRAFODION-3015] retrieve a value from numeric...
Github user traflm closed the pull request at: https://github.com/apache/trafodion/pull/1564 ---
[GitHub] trafodion pull request #1564: [TRAFODION-3015] retrieve a value from numeric...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1564 [TRAFODION-3015] retrieve a value from numeric type get no result When the column is defined as NUMERIC(19,0), underlying data type will use FLOAT to save the number. So an implicit cast will generate wrong result when user input is string literal. For example: create table t3015 (a1 NUMERIC(19,0) ); insert into t3015 values(27380468); select * from t3015 where a1='27380468'; will have wrong output. The fix is to report error, if the precision of NUMERIC is greater than 18, and try to cast STRING into it. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3015-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1564.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1564 commit 3e8703cbb433deebf616156617db95874aaafcdd Author: Liu Ming Date: 2018-05-13T10:20:14Z [TRAFODION-3015] retrieve a value from numeric type get no result ---
[GitHub] trafodion pull request #1563: [TRAFODION-3015] retrieve a value from numeric...
Github user traflm closed the pull request at: https://github.com/apache/trafodion/pull/1563 ---
[GitHub] trafodion pull request #1563: [TRAFODION-3015] retrieve a value from numeric...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1563 [TRAFODION-3015] retrieve a value from numeric type get no result if⦠⦠using xx='value' When the column is defined as NUMERIC(19,0), underlying data type will use FLOAT to save the number. So an implicit cast will generate wrong result when user input is string literal. For example: create table t3015 (a1 NUMERIC(19,0) ); insert into t3015 values(27380468); select * from t3015 where a1='27380468'; will have wrong output. The fix is to report error, if the precision of NUMERIC is greater than 18, and try to cast STRING into it. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3015 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1563.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1563 commit 56b175ed6c41c202ce6ceef4fbadb16df37045cf Author: Liu Ming Date: 2018-05-12T14:58:25Z [TRAFODION-3015] retrieve a value from numeric type get no result if using xx='value' ---
[GitHub] trafodion pull request #1553: [TRAFODION-1861] Run Trafodion on CentOS 7
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1553#discussion_r186920564 --- Diff: core/sqf/src/seabed/src/Makefile --- @@ -55,6 +55,16 @@ LIBSBUTIL= $(LIBEXPDIR)/libsbutil.so LIBSQSTATESB = $(LIBEXPDIR)/libsqstatesb.so # neet -lrt for clock_gettime LIBSX += -lrt +# +# the order of these objects have to be swapped based on the GCC version +MS_MPITMSG_O = $(OUTDIR)/mpitmsg.o \ + $(OUTDIR)/ms.o +ifeq ($(GCC_MAJOR_VERSION_GT_3),1) +ifeq ($(GCC_MINOR_VERSION_GT_4),1) --- End diff -- You are right Dave, so it should be GCC_MAJOR_VERSION_EQ_4 and GCC_MINOR_VERSION_GT_4 We don't know how GCC 5.1 behave now, so let only check GCC 4.4 vs. GCC 4.8 at present. Let me change the logic more accurate That is, we only change the link order for GCC 4.x , where x>4 ---
[GitHub] trafodion pull request #1553: [TRAFODION-1861] Run Trafodion on CentOS 7
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1553 [TRAFODION-1861] Run Trafodion on CentOS 7 Solve the link issue when using GCC 4.8 on CentOS 7 So Trafodion can start on CentOS 7 with this change It also depends on https://github.com/apache/trafodion/pull/1552 to make the code build success You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-1861 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1553.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1553 commit 4addf763ee18eb9cb01e2bd0fecde297a60f485c Author: LiuMing Date: 2018-05-08T04:01:46Z [TRAFODION-1861] Run Trafodion on CentOS 7 ---
[GitHub] trafodion pull request #1552: [TRAFODION-3061] eliminate building errors on ...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1552 [TRAFODION-3061] eliminate building errors on CentOS7 There are some coding cannot be compiled using GCC in CentOS 7. We need to continuously check and fix these issues until CentOS7 is supported and jenkins test established. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3061 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1552.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1552 commit a8f9197b0e3043f0c2e1cfca841d010bca42ef63 Author: LiuMing Date: 2018-05-08T01:17:57Z [TRAFODION-3061] eliminate building errors on CentOS7 ---
[GitHub] trafodion pull request #1550: [TRAFODION-3059] remove "mt_" column family
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1550 [TRAFODION-3059] remove "mt_" column family All Trafodion tables have a second column family named "mt_", which was originally designed for SSCC. SSCC is not enabled now SSCC plan to not use a new column family So it is good to remove this extra, no use column family You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3059 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1550.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1550 commit 1ad5a1265f5def7c3c2b914c628eec9ad33ccd1a Author: Liu Ming Date: 2018-05-06T08:48:16Z [TRAFODION-3059] remove "mt_" column family ---
[GitHub] trafodion pull request #1549: [TRAFODION-3054] update pom.xml.cdh for hbase-...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1549 [TRAFODION-3054] update pom.xml.cdh for hbase-trx to build protobuf The various pom.xml.cdh file can be used as pom.xml for maven input, to build protobuf java output for hbase-trx. As we want to modify the .proto to add a new coprocessor, we found that it failed with ERROR: Caused by: java.lang.UnsupportedClassVersionError: org/apache/hadoop/maven/plugin/protoc/ProtocMojo : Unsupported major.minor version 52.0 And the plug-in information is [INFO] --- hadoop-maven-plugins:3.1.0:protoc (compile-protoc) @ hbase-trx-cdh5_4 --- So the lasted Hadoop maven plugins for protoc is 3.1.0 which need JDK 1.8. It also means the pom.xml.cdh files should be updated to match the Hadoop-Maven-plugin version same as its hadoop version, instead of using latest version (without specifying the version tag) You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3054 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1549.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1549 commit 1cc728fa7eaf444e7dc6f61a4bd7ed65b12ad7e1 Author: Liu Ming Date: 2018-05-05T16:15:40Z [TRAFODION-3054] update pom.xml.cdh for hbase-trx to build protobuf ---
[GitHub] trafodion pull request #1540: [TRAFODION-2953] initial implementation of sma...
GitHub user traflm reopened a pull request: https://github.com/apache/trafodion/pull/1540 [TRAFODION-2953] initial implementation of small table in one Hbase Region This is the very first code drop for TRAFODION-2953, the major purpose is to get feedback to see if the design is OK and shall we continue with this effort or not. Please check https://issues.apache.org/jira/browse/TRAFODION-2953, I attached a design document there. This PR contains initial prototype of the design. It still has a major code-refactor to do, but I would like to hear from community first, before I spend more time on it, in case there are something fundamentally wrong. The major code-refactor I mentioned above is for this prototype code is to add a new PartitionFunction, instead of modifying the SinglePartitionPartitioningFunction directly. I will inherit a new SinglePartitionWithSmallTablePartitioningFunction from SinglePartitionPartitioningFunction next. One can check the TEST035 to get a demo of how it works. There are many other tasks following this initial code drop. For example, index/alter/create-super-table etc. The major idea to support this is to add a new SYSTEM column _TBLNM_, so in a single Region, rows belong to different tables can be distinguished. When doing IUDS, always add predicate _TBLNM_ = 'theNameOfTheTable' to filter out the correct rows. Please help to review and let me know your comments, many thanks. The requirement is strong. In some use cases, there must be many small tables. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2953-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1540.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1540 commit 04123ad4168fbd7305caf37d5c8f02deb6eafa07 Author: Liu Ming Date: 2018-04-24T21:28:26Z AFODION-2953] initial implementation commit c195c6fe23a214cad3b8db3359c19b282ac7a5c1 Author: Liu Ming Date: 2018-04-24T22:14:22Z [TRAFODION-2953] first code drop for small table in one HBase Region ---
[GitHub] trafodion pull request #1540: [TRAFODION-2953] initial implementation of sma...
Github user traflm closed the pull request at: https://github.com/apache/trafodion/pull/1540 ---
[GitHub] trafodion pull request #1540: [TRAFODION-2953] initial implementation of sma...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1540#discussion_r183936345 --- Diff: core/sql/regress/tools/sbdefs --- @@ -22,7 +22,7 @@ -- defs included during seabase regr run #ifdef SEABASE_REGRESS -cqd mode_seabase 'ON'; +--cqd mode_seabase 'ON'; --- End diff -- typo, will fix this in next commit. Sorry ---
[GitHub] trafodion pull request #1540: [TRAFODION-2953] initial implementation of sma...
GitHub user traflm reopened a pull request: https://github.com/apache/trafodion/pull/1540 [TRAFODION-2953] initial implementation of small table in one Hbase Region This is the very first code drop for TRAFODION-2953, the major purpose is to get feedback to see if the design is OK and shall we continue with this effort or not. Please check https://issues.apache.org/jira/browse/TRAFODION-2953, I attached a design document there. This PR contains initial prototype of the design. It still has a major code-refactor to do, but I would like to hear from community first, before I spend more time on it, in case there are something fundamentally wrong. The major change for this prototype code is to add a new PartitionFunction, instead of modifying the SinglePartitionPartitioningFunction directly. I will inherit a new SinglePartitionWithSmallTablePartitioningFunction from SinglePartitionPartitioningFunction next. One can check the TEST035 to get a demo of how it works. There are many other tasks following this initial code drop. For example, index/alter/create-super-table etc. The major idea to support this is to add a new SYSTEM column _TBLNM_, so in a single Region, rows belong to different tables can be distinguished. When doing IUDS, always add predicate _TBLNM_ = 'theNameOfTheTable' to filter out the correct rows. Please help to review and let me know your comments! You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2953-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1540.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1540 commit 04123ad4168fbd7305caf37d5c8f02deb6eafa07 Author: Liu Ming Date: 2018-04-24T21:28:26Z AFODION-2953] initial implementation commit c195c6fe23a214cad3b8db3359c19b282ac7a5c1 Author: Liu Ming Date: 2018-04-24T22:14:22Z [TRAFODION-2953] first code drop for small table in one HBase Region ---
[GitHub] trafodion pull request #1540: [TRAFODION-2953] initial implementation of sma...
Github user traflm closed the pull request at: https://github.com/apache/trafodion/pull/1540 ---
[GitHub] trafodion pull request #1540: [TRAFODION-2953] initial implementation
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1540 [TRAFODION-2953] initial implementation This is the very first code drop for TRAFODION-2953, the major purpose is to get feedback to see if the design is OK and shall we continue with this effort or not. Please check https://issues.apache.org/jira/browse/TRAFODION-2953, I attached a design document there. This PR contains initial prototype of the design. It still has a major code-refactor to do, but I would like to hear from community first, before I spend more time on it, in case there are something fundamentally wrong. The major change for this prototype code is to add a new PartitionFunction, instead of modifying the SinglePartitionPartitioningFunction directly. I will inherit a new SinglePartitionWithSmallTablePartitioningFunction from SinglePartitionPartitioningFunction next. One can check the TEST035 to get a demo of how it works. There are many other tasks following this initial code drop. For example, index/alter/create-super-table etc. The major idea to support this is to add a new SYSTEM column _TBLNM_, so in a single Region, rows belong to different tables can be distinguished. When doing IUDS, always add predicate _TBLNM_ = 'theNameOfTheTable' to filter out the correct rows. Please help to review and let me know your comments! You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2953-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1540.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1540 commit 04123ad4168fbd7305caf37d5c8f02deb6eafa07 Author: Liu Ming Date: 2018-04-24T21:28:26Z AFODION-2953] initial implementation ---
[GitHub] trafodion pull request #1521: Do not merge
Github user traflm closed the pull request at: https://github.com/apache/trafodion/pull/1521 ---
[GitHub] trafodion pull request #1533: [TRAFODION-3037] install_local_hadoop script o...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1533 [TRAFODION-3037] install_local_hadoop script out of date (APACHE) On a brandnew develop machine, if one set the HBASE_DISTRO=APACHE, install_local_hadoop cannot correctly download required tar files. The download link changed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3035 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1533.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1533 commit a3971a2365d23f66303c0a33d36b70cb47fb6c21 Author: Liu Ming Date: 2018-04-22T06:32:24Z [TRAFODION-3037] install_local_hadoop script out of date (APACHE) ---
[GitHub] trafodion pull request #1532: [TRAFODION-3026] add create option storage pol...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1532 [TRAFODION-3026] add create option storage policy Some user want Trafodion to support HSM provided by Hadoop 2.6.0. Tables can be stored on different type of storage hardware (SSD/HDD). Hadoop provides HSM in HDFS, but Hbase doesn't provide the support until HBase 2.0. There is HBASE-19858 and plan to support HSM in HBase 1.5.0. But still not released. So before that HBase feature availible, we now use HDFS API to directly set the storage policy on Hbase table path. Trafodion adds new HBASE_OPTIONS to support this feature. So when creating a Trafodion table, it is possible to specify the storage policy (by default, HDFS provide hot/warm/cold/one_ssd/all_ssd etc) This provide a basic support of user's requirement. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3026 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1532.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1532 commit 5724227c48f1fa33e993a5642803792fb0478867 Author: Liu Ming Date: 2018-04-21T15:26:52Z [TRAFODION-3025] add create option storage policy commit ad05d5c0c6be22760266a0a2b365d4a7c5d2747a Author: Liu Ming Date: 2018-04-21T15:27:44Z remove typos ---
[GitHub] trafodion pull request #1521: Do not merge
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1521 Do not merge Just for test You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2953 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1521.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1521 commit a85518ac10886dcf3284e3af7a026241879143b3 Author: Liu Ming Date: 2018-03-26T22:41:29Z [TRAFODION-2953] initial change in ddl commit b83d238c0bc8c4bcdea2652d974a059111bfeb98 Author: Liu Ming Date: 2018-03-31T04:57:52Z second commit, basic works commit 5e64fa8b1133eb5dac9ac92993e863fa25427864 Author: Liu Ming Date: 2018-04-03T19:23:06Z third commit, pass the isSmallTable to NATable commit 92d6d0a5764d877006674c4d3103d4769c0c0697 Author: Liu Ming Date: 2018-04-14T00:04:45Z Merge branch 'master' of git://git.apache.org/trafodion into TRAFODION-2953 ---
[GitHub] trafodion pull request #1518: [TRAFODION-3021] union, intersect, except shou...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1518 [TRAFODION-3021] union, intersect, except should be cacheable Make UNION, INTERSECT, EXCEPT cacheable. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3021 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1518.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1518 commit e2b9a66d92f63e01f897f1ca1d7f3a6801a52756 Author: Liu Ming Date: 2018-04-14T02:13:53Z [TRAFODION-3021] union, intersect, except should be cacheable ---
[GitHub] trafodion pull request #1510: [TRAFODION-3020] regression issue
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1510 [TRAFODION-3020] regression issue one line of unused code in parser You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3020 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1510.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1510 commit 927df7240050324817640a9ec2c2f0880f405b69 Author: Liu Ming Date: 2018-04-09T08:41:57Z [TRAFODION-3020] regression issue ---
[GitHub] trafodion pull request #1485: [TRAFODION-2999] make UUID non reserved keywor...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1485#discussion_r178998606 --- Diff: core/sql/common/ComResWords.cpp --- @@ -395,8 +395,7 @@ const ComResWord ComResWords::resWords_[] = { ComResWord("USAGE",ANS_|RESWORD_), ComResWord("USER", ANS_|RESWORD_), ComResWord("USING",ANS_|RESWORD_), - ComResWord("UUID",ANS_|RESWORD_), - ComResWord("VALUE",ANS_|RESWORD_), + ComResWord("VALUE", ANS_|RESWORD_|MPWORD_), --- End diff -- Thanks Dave, let me remove this line. ---
[GitHub] trafodion pull request #1500: [TRAFODION-3014] merge R2.2 changes into mainb...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1500 [TRAFODION-3014] merge R2.2 changes into mainbranch This is an effort to merge R2.2 branch into main branch. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3014 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1500.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1500 commit ef73b7719c746bc721587f2b4b4a032d71d03c65 Author: liu.yu Date: 2017-08-15T04:12:48Z cherry-pick altercolumn commit a465ec83025a8a3664a1b0f9a28c86021287952a Author: liu.yu Date: 2017-08-15T03:22:21Z Capitalize XML commit 08b41edb08d313ed524e64392271d17c38ee9b19 Author: liu.yu Date: 2017-08-14T03:56:00Z cherry pick add alter column 2 commit fb007c00469a36b37e58d1821581fdbe38527626 Author: liu.yu Date: 2017-08-13T08:06:25Z cherry-pick Load XML Files commit 89576f3dc7f5e37977ec3cdd9f0fd5ecadee0d36 Author: Sandhya Sundaresan Date: 2017-08-22T17:38:42Z Changes to take care of closing internal cursor statement when extract operation is done and the exact length of data is passed in to be extracted. commit 8a260f425e0527100c8039b8db8a8b4db64b1101 Author: Sandhya Sundaresan Date: 2017-08-22T17:41:39Z Expected file change to show result of unload. commit 2048d253ca0ec8e01bf9f8b2c959a67f82da530a Author: Liu Ming Date: 2017-08-24T03:09:27Z Merge Fix for Cherry-pick changes of [TRAFODION-2703] & [TRAFODION-2710] to 2.2 commit f18644841f8d3c8aaa5c594c1018798d515821b0 Author: Sandhya Sundaresan Date: 2017-08-31T20:03:05Z Main change: Added a method for JDBC driver to be able to close an open cursor abruptly by passing in a 0 length to the extract syntax. This will ensure internal opens are not left around if the application does not retrieve all the data with the stream API. This abrupt close can be called when the result set is closed or the connection is closed by keeping track of all the open Blob objects. (in the driver) Added a freeResources method and destructor to LobExtract and LobUpdate Tcbs. Minor changes and fixes : Refactored the ExLob::closeCursor Removed the LOB_CLI_SELECT_CLOSE call from ::fetchCursor so it gets closed only when ExLob::closeCursor is called. Makes the code consistent and readable. Took care of a few memory allocations and deletions. Refactored the LOB length calculation into on ecommon method so it works correctly for both regular LOB dat aas well as external LOB files. Earlier the external lobs length was being returned incorrectly. Now it reports the length of the external HDFS file . Fixed one place in generator where the CQD value was not being multiplied by the MB units correctly. commit 615b8dd26d38e19ba17cf9451453bc38e8879dcc Author: Sandhya Sundaresan Date: 2017-09-04T06:58:05Z Merge remote branch 'origin/pr/1216/head' into merge_1216 commit e8c09ebb678ab6560130bf687dc1e52c9b04d64a Author: Sandhya Sundaresan Date: 2017-09-11T17:57:02Z Changes to handle errors during drop of lob tables that leave the table inconsistent. (cherry picked from commit 749295ad774620d49db79d961b1bbacac8d589c7) commit 42b2b0b4a56e9b125d15eaa227976d11dcec1b3c Author: Sandhya Sundaresan Date: 2017-09-12T17:51:49Z Merge remote branch 'origin/pr/1230/head' into merge_1230 commit 808c87450f343e6df3dcc82e0799919581958513 Author: Weiqing Xu Date: 2017-09-20T05:49:28Z [TRAFODION-2704]close unused statment in JDBC regression commit bd792ff7ef0a81c6b0106ce1d2aaeb3ff34d9701 Author: Weiqing Xu Date: 2017-09-20T05:57:26Z [TRAFODION-2740]JDBC extract LOB context in chunks commit ea80ab383da807fe4fd61d9a9193c08ad2c79283 Author: Weiqing Xu Date: 2017-09-22T16:44:40Z [TRAFODION-2740]remove unused code commit 835a48945afe3cc19ec58ac94e22b323db69d5cf Author: Weiqing Xu Date: 2017-09-23T17:47:33Z [TRAFODION-2740]add property to set the LOB chunk size commit 1ade4208f1e3551fa4ce2c7cbfda29a4c8c68b15 Author: Weiqing Xu Date: 2017-09-23T17:48:39Z [TRAFODION-2704]reslove the random schema not exist issue commit e26ec04a7815cb46f3d10d9816398c10ad13df67 Author: Sandhya Sundaresan Date: 2017-09-25T04:32:50Z Merge remote branch 'origin/pr/1240/head' into merge_1240 commit de664c37a95f89573ccc4466357d5ebf94df22a5 Author: Weiqing Xu Date: 2017-09-25T05:47:56Z [TRAFODION-2753]LOB: filetolob() no longer works in trafci commit 3254cc16b259a6a513cabc391699f44f29850cf1 Author: Sandhya Sundaresan Date: 2017-09-26T05:31:16Z Merge remote branch 'origin/pr/1242/head' into merge_1242 commit f077620ae434bfdc115a599a591bec1f5eb7bff6 Author
[GitHub] trafodion pull request #1485: [TRAFODION-2999] make UUID non reserved keywor...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1485#discussion_r177260101 --- Diff: core/sql/common/ComResWords.cpp --- @@ -395,8 +395,6 @@ const ComResWord ComResWords::resWords_[] = { ComResWord("USAGE",ANS_|RESWORD_), ComResWord("USER", ANS_|RESWORD_), ComResWord("USING",ANS_|RESWORD_), - ComResWord("UUID",ANS_|RESWORD_), - ComResWord("VALUE",ANS_|RESWORD_), --- End diff -- Yes, @DaveBirdsall , Let me revert the changes to 'VALUE'. The reason is we found 'VALUE' is also a popular string for column/table names and there is no strict reason to keep it as reserved word. And we changed it in a commercial branch and test and no side effect. So I want to change it together with 'UUID' ( UUID for sure is very popular in the naming of table/columns ) But for clarity, let me change it, unless you change your mind and allow this :-) ---
[GitHub] trafodion pull request #1497: [TRAFODION-2986] minor issues in adoc found du...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1497 [TRAFODION-2986] minor issues in adoc found during mvn site During the release 2.2, trying to update the documentation, found several minor typos and syntax errors Doc fixing. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2986 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1497.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1497 commit b6157d5fa11688c0c8a1a26d574d479eb376988d Author: Liu Ming Date: 2018-03-25T21:48:20Z [TRAFODION-2986] minor issues in adoc found during mvn site ---
[GitHub] trafodion pull request #1481: [Trafodion-2335] support functions in the colu...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1481#discussion_r176618285 --- Diff: core/sql/regress/seabase/EXPECTED020 --- @@ -3945,4 +3945,35 @@ ?2 --- 1 row(s) selected. >> +>>obey TEST020(trafodion_2335); +>>--create seqence test020_seq; +>>create table test020t45(a largeint not null default unix_timestamp(), ++>b char(36) not null default uuid(), ++>c varchar(10) default to_char(sysdate,'MMDD'), ++>--support sequence as default in next check-in ++>--d int not null default testi020_seq.nextval, ++>e int ); + +--- SQL operation complete. +>>insert into test020t45(e) values(1),(2),(3); + +--- 3 row(s) inserted. +>>select count(*) from test020t45 where c = to_char(sysdate,'MMDD'); + +(EXPR) + + + 3 + +--- 1 row(s) selected. +>>--negative tests +>>--the function is not variable-free, so should fail +>>create table test020t45(a largeint not null default unix_timestamp(), ++>b varchar(10) default to_char(test020t45.c,'MMDD'), ++>e int ); + +*** ERROR[1084] An invalid default value was specified for column B. + +*** ERROR[8822] The statement was not prepared. + --- End diff -- @selvaganesang By adding new test case for upsert mode 'merge', the missing column of unix_timestamp() default value is updated. So still something wrong, I need to fix it. Thanks for reminding this. ---
[GitHub] trafodion pull request #1491: [TRAFODION-3006] unix_timestamp runtime core d...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1491 [TRAFODION-3006] unix_timestamp runtime core dump issue Wrong input param for strptime, null pointer You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-3006 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1491.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1491 commit bde4454db07386817a754666a17ce819b9b456f2 Author: Liu Ming Date: 2018-03-21T19:52:55Z [TRAFODION-3006] unix_timestamp runtime core dump issue ---
[GitHub] trafodion pull request #1481: [Trafodion-2335] support functions in the colu...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1481#discussion_r176278679 --- Diff: core/sql/common/ComSmallDefs.h --- @@ -609,47 +609,59 @@ enum ComColumnClass { COM_UNKNOWN_CLASS #define COM_ALTERED_USER_COLUMN_LIT "C " enum ComColumnDefaultClass { COM_CURRENT_DEFAULT + , COM_CURRENT_UT_DEFAULT --- End diff -- Thanks all, I now understand what is the issue! If there are old metadata, and there is a upgrade, then things are messed up. But I think this enum can be changed by adding new value at the end, instead of "cannot be changed". So let me add them at the end, and add big comments. ---
[GitHub] trafodion pull request #1481: [Trafodion-2335] support functions in the colu...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1481#discussion_r175966341 --- Diff: core/sql/regress/seabase/EXPECTED020 --- @@ -3945,4 +3945,35 @@ ?2 --- 1 row(s) selected. >> +>>obey TEST020(trafodion_2335); +>>--create seqence test020_seq; +>>create table test020t45(a largeint not null default unix_timestamp(), ++>b char(36) not null default uuid(), ++>c varchar(10) default to_char(sysdate,'MMDD'), ++>--support sequence as default in next check-in ++>--d int not null default testi020_seq.nextval, ++>e int ); + +--- SQL operation complete. +>>insert into test020t45(e) values(1),(2),(3); + +--- 3 row(s) inserted. +>>select count(*) from test020t45 where c = to_char(sysdate,'MMDD'); + +(EXPR) + + + 3 + +--- 1 row(s) selected. +>>--negative tests +>>--the function is not variable-free, so should fail +>>create table test020t45(a largeint not null default unix_timestamp(), ++>b varchar(10) default to_char(test020t45.c,'MMDD'), ++>e int ); + +*** ERROR[1084] An invalid default value was specified for column B. + +*** ERROR[8822] The statement was not prepared. + --- End diff -- Thanks Selva, Let me add such a test case. ---
[GitHub] trafodion pull request #1481: [Trafodion-2335] support functions in the colu...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1481#discussion_r175964689 --- Diff: core/sql/common/ComSmallDefs.h --- @@ -609,47 +609,59 @@ enum ComColumnClass { COM_UNKNOWN_CLASS #define COM_ALTERED_USER_COLUMN_LIT "C " enum ComColumnDefaultClass { COM_CURRENT_DEFAULT + , COM_CURRENT_UT_DEFAULT --- End diff -- @robertamarton Could you help to provide some comments here? @selvaganesang I think new values will NOT cause problems, the MD table structure is not changed. I don't find any checking code relying on values in this column. But Roberta should have more insight here. ---
[GitHub] trafodion pull request #1481: [Trafodion-2335] support functions in the colu...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1481#discussion_r175651051 --- Diff: core/sql/parser/ElemDDLColDef.h --- @@ -309,6 +312,8 @@ class ElemDDLColDef : public ElemDDLNode NABoolean seabaseSerialized_; NABoolean isColDefaultSpec_; + + Int32 errCode_; --- End diff -- This is leftover code, I will remove it. When there are more comments, and change together ---
[GitHub] trafodion pull request #1485: [TRAFODION-2999] make UUID non reserved keywor...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1485 [TRAFODION-2999] make UUID non reserved keyword You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2999 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1485.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1485 ---
[GitHub] trafodion pull request #1481: [Trafodion-2335] support functions in the colu...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1481#discussion_r175641075 --- Diff: core/sql/parser/sqlparser.y --- @@ -8734,6 +8738,30 @@ datetime_value_function : TOK_CURDATE '(' ')' $$ = new (PARSERHEAP()) Cast(uniqueId,type); } +/* type item */ +datetime_misc_function_used_as_default: TOK_TO_CHAR '(' value_expression ',' character_string_literal ')' + { + NAString * ves= unicodeToChar + (ToTokvalPlusYYText(&$3)->yytext, + ToTokvalPlusYYText(&$3)->yyleng, + (CharInfo::CharSet) ( + ComGetNameInterfaceCharSet() // CharInfo::UTF8 + ), + PARSERHEAP()); + //save the original text + NAString fullstr; + fullstr += "TO_CHAR("; --- End diff -- to_char(sysdate, 'MMDD') unparse to TO_CHAR(CAST(currrenttimestmp), 'MMDD') Which cannot be compiled later. Given more time, I may be able to find the right place to modify the unparse for sysdate, but the CAST is something that very hard to make it generic. Other unparse usage place may need the CAST. So for now, I have to keep the original text in this way. ---
[GitHub] trafodion pull request #1483: [TRAFODION-2998] sleep execution code cannot b...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1483 [TRAFODION-2998] sleep execution code cannot build on CentOS 7 Wrong implementation in the sleep runtime code. And made it not able to be built on CentOS 7 because of bad data type conversion. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2998-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1483.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1483 commit 80b76cd4ce9102d3f52e82392af213819a54023c Author: Liu Ming Date: 2018-03-19T03:34:09Z [TRAFODION-2998] sleep execution code cannot build on CentOS 7 ---
[GitHub] trafodion pull request #1482: [TRAFODION-2998] sleep execution code cannot b...
Github user traflm closed the pull request at: https://github.com/apache/trafodion/pull/1482 ---
[GitHub] trafodion pull request #1482: [TRAFODION-2998] sleep execution code cannot b...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1482 [TRAFODION-2998] sleep execution code cannot build on CentOS 7 Wrong data type conversion , and wrong usage of labs You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2998 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1482.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1482 commit f253a3818887baa10d4f21e12e67a8d9414f02e1 Author: Liu Ming Date: 2018-03-19T03:10:27Z [TRAFODION-2998] sleep execution code cannot build on CentOS 7 ---
[GitHub] trafodion pull request #1481: [Trafodion-2335] support functions in the colu...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1481 [Trafodion-2335] support functions in the column default definition This PR add the support in column default definition the expansion to use following functions: uuid unix_timestamp to_char To support to_char, it saves the original text of full to_char function, and save it as DEFAULT_VALUE in "_MD_".COLUMNS. And the INSERT binder will get this original string, and parse into the ItemExpr tree. The semantic check is done in setDefaultAttribute. UUID/UNIX_TIMESTAMP follow the current_timestamp implementation. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2335 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1481.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1481 commit 6b800c34a4f9cb4386223067ff7c3801bdbec002 Author: Liu Ming Date: 2018-03-14T09:07:42Z [TRAFODION-2335] support some functions as column default, part1 commit commit 5f58cd95aa72f4cbe944aa8d0aeb11eb0202f524 Author: Liu Ming Date: 2018-03-18T09:19:35Z update the regression test and various fixes ---
[GitHub] trafodion pull request #1471: [TRAFODION-2985] fix wrong download link
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1471 [TRAFODION-2985] fix wrong download link Some download links are wrong, fix them. This time, try each link by hand one by one. Should be all correct... You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2985 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1471.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1471 commit cb886d3bfaf09cc9d7ce171fb553c7b08b8fed85 Author: Liu Ming Date: 2018-03-13T01:05:32Z [TRAFODION-2985] fix wrong download link ---
[GitHub] trafodion pull request #1467: [TRAFODION-2985] Update Project Web Site for A...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1467#discussion_r173617726 --- Diff: docs/src/site/markdown/download.md --- @@ -37,34 +72,34 @@ To build Trafodion from source code, see the [Trafodion Contributor Guide](https * [Ambari Plugin][ap210] - [PGP][appgp210] [MD5][apmd5210] [SHA1][apsha210] * [Documentation](documentation.html#210_Release) -[src210]: http://www.apache.org/dyn/closer.lua/trafodion/apache-trafodion-2.1.0-incubating/src/apache-trafodion-2.1.0-incubating-src.tar.gz -[pgp210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/src/apache-trafodion-2.1.0-incubating-src.tar.gz.asc -[md5210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/src/apache-trafodion-2.1.0-incubating-src.tar.gz.md5 -[sha210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/src/apache-trafodion-2.1.0-incubating-src.tar.gz.sha -[ins210]: http://www.apache.org/dyn/closer.lua/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_installer-2.1.0-incubating.tar.gz -[inpgp210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_installer-2.1.0-incubating.tar.gz.asc -[inmd5210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_installer-2.1.0-incubating.tar.gz.md5 -[insha210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_installer-2.1.0-incubating.tar.gz.sha -[pins210]: http://www.apache.org/dyn/closer.lua/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_pyinstaller-2.1.0-incubating.tar.gz -[pinpgp210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_pyinstaller-2.1.0-incubating.tar.gz.asc -[pinmd5210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_pyinstaller-2.1.0-incubating.tar.gz.md5 -[pinsha210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_pyinstaller-2.1.0-incubating.tar.gz.sha -[ser210]: http://www.apache.org/dyn/closer.lua/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_server-2.1.0-RH6-x86_64-incubating.tar.gz -[sepgp210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_server-2.1.0-RH6-x86_64-incubating.tar.gz.asc -[semd5210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_server-2.1.0-RH6-x86_64-incubating.tar.gz.md5 -[sesha210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_server-2.1.0-RH6-x86_64-incubating.tar.gz.sha -[cl210]: http://www.apache.org/dyn/closer.lua/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_clients-2.1.0-RH6-x86_64-incubating.tar.gz -[clpgp210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_clients-2.1.0-RH6-x86_64-incubating.tar.gz.asc -[clmd5210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_clients-2.1.0-RH6-x86_64-incubating.tar.gz.md5 -[clsha210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/apache-trafodion_clients-2.1.0-RH6-x86_64-incubating.tar.gz.sha -[ar210]: http://www.apache.org/dyn/closer.lua/trafodion/apache-trafodion-2.1.0-incubating/bin/traf_ambari_rpms/apache-trafodion_server-2.1.0-1.x86_64.rpm -[arpgp210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/traf_ambari_rpms/apache-trafodion_server-2.1.0-1.x86_64.rpm.asc -[armd5210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/traf_ambari_rpms/apache-trafodion_server-2.1.0-1.x86_64.rpm.md5 -[arsha210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/traf_ambari_rpms/apache-trafodion_server-2.1.0-1.x86_64.rpm.sha -[ap210]: http://www.apache.org/dyn/closer.lua/trafodion/apache-trafodion-2.1.0-incubating/bin/traf_ambari_rpms/traf_ambari-2.1.0-1.noarch.rpm -[appgp210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/traf_ambari_rpms/traf_ambari-2.1.0-1.noarch.rpm.asc -[apmd5210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/traf_ambari_rpms/traf_ambari-2.1.0-1.noarch.rpm.md5 -[apsha210]: http://www.apache.org/dist/trafodion/apache-trafodion-2.1.0-incubating/bin/traf_ambari_rpms/traf_ambari-2.1.0-1.noarch.rpm.sha +[src210]: http://archive.apache.org/dyn/closer.lua/trafodion/apache-trafodion-2.1.0-incubating/src/apache-trafodion-2.1.0-incubating-src.tar.gz --- End diff -- Thanks Steve, I will change it ---
[GitHub] trafodion pull request #1467: [TRAFODION-2985] Update Project Web Site for A...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1467 [TRAFODION-2985] Update Project Web Site for Apache Trafodion Release⦠⦠2.2.0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2985 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1467.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1467 commit a8230a9e31932108dd278c32292e37955f308124 Author: Liu Ming Date: 2018-03-10T01:33:38Z [TRAFODION-2985] Update Project Web Site for Apache Trafodion Release 2.2.0 ---
[GitHub] trafodion pull request #1444: [TRAFODION-2954] add MySQL function unix_times...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1444#discussion_r169868459 --- Diff: core/sql/regress/executor/TEST002 --- @@ -1188,6 +1188,38 @@ select * from regexp_test where c1 regexp '^(25[0-5]|2[0-4][0-9]|[0-1]{1}[0-9]{2 select * from regexp_test where c1 regexp '(ä¸ææµè¯)'; select * from regexp_test where c1 regexp '[^\'; drop table regexp_test; + +--create table have 1K rows +create table T002T1K (uniq int not null, + c1K int, c100 int, + c10 int, c1 int, c0 int ) + STORE BY (uniq) + ATTRIBUTES ALIGNED FORMAT + SALT USING 8 PARTITIONS + ; + +upsert using load into T002T1K select +0 + (1000 * x10) + (100 * x1) + (10 * x1) + (1 * x01), +0 + (100 * x10) + (10 * x1) + (1 * x01), +0 + (10 * x1) + (1 * x01), +0 + (1 * x01), +0, +X01 +from (values(0)) t +transpose 0,1,2,3,4,5,6,7,8,9 as x10 +transpose 0,1,2,3,4,5,6,7,8,9 as x1 +transpose 0,1,2,3,4,5,6,7,8,9 as X01; + +create table t002timert (c0 int, c1 int, c2 largeint); +create table t002tmp1 (c1 int); +insert into t002tmp1 values(1),(2),(3); + +insert into t002timert select 1, sleep(5) , unix_timestamp() from t002tmp1; +insert into t002timert select 2, sleep(5) , unix_timestamp() from t002tmp1; +select 'sleeptimetest002', di from (select ( max(c2) - min(c2)) as di from t002timert ) where di between 5 and 9; --- End diff -- above two INSERTS will insert two different unix_timestamp() , the difference is the second between their run, so if the sleep(5) eval once, the diff should be around 5 seconds, and less than 10. So the query is using 'between 5 and 9'. If something wrong, this test query will return 0 rows and fail the test. ---
[GitHub] trafodion pull request #1444: [TRAFODION-2954] add MySQL function unix_times...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1444#discussion_r169511829 --- Diff: core/sql/optimizer/GroupAttr.cpp --- @@ -1793,6 +1793,8 @@ void GroupAttributes::resolveCharacteristicInputs(const ValueIdSet& externalInpu ItemExpr * vidExpr = vid.getItemExpr(); if ((vidExpr->getOperatorType() == ITM_CURRENT_USER) || (vidExpr->getOperatorType() == ITM_CURRENT_TIMESTAMP) || +(vidExpr->getOperatorType() == ITM_UNIQUE_SHORT_ID) || +(vidExpr->getOperatorType() == ITM_UNIQUE_ID) || --- End diff -- You are right, Hans, I also think in the opposite, I just simply follow the current_timestamp behavior, and this really is not what I originally want. So I will change uuid() to be evaluated once per row. ---
[GitHub] trafodion pull request #1444: [TRAFODION-2157] add MySQL function unix_times...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1444#discussion_r168649121 --- Diff: core/sql/exp/exp_function.cpp --- @@ -2501,6 +2518,100 @@ ex_expr::exp_return_type ExFunctionReverseStr::eval(char *op_data[], return ex_expr::EXPR_OK; }; +ex_expr::exp_return_type ex_function_sleep::eval(char *op_data[], + CollHeap* heap, + ComDiagsArea** diagsArea) +{ + Int32 sec = 0; + switch (getOperand(1)->getDatatype()) + { +case REC_BIN8_SIGNED: + sec = *(Int8 *)op_data[1] ; + if(sec < 0 ) + { +ExRaiseSqlError(heap, diagsArea, EXE_BAD_ARG_TO_MATH_FUNC); +*(*diagsArea) << DgString0("SLEEP"); +return ex_expr::EXPR_ERROR; + } + sleep(sec); + *(Int64 *)op_data[0] = 1; + break; + +case REC_BIN16_SIGNED: + sec = *(short *)op_data[1] ; + if(sec < 0 ) + { +ExRaiseSqlError(heap, diagsArea, EXE_BAD_ARG_TO_MATH_FUNC); +*(*diagsArea) << DgString0("SLEEP"); +return ex_expr::EXPR_ERROR; + } + sleep(sec); + *(Int64 *)op_data[0] = 1; + break; + +case REC_BIN32_SIGNED: + *(Lng32 *)sec = labs(*(Lng32 *)op_data[1]); + if(sec < 0 ) + { +ExRaiseSqlError(heap, diagsArea, EXE_BAD_ARG_TO_MATH_FUNC); +*(*diagsArea) << DgString0("SLEEP"); +return ex_expr::EXPR_ERROR; + } + sleep(sec); + *(Int64 *)op_data[0] = 1; + break; + +case REC_BIN64_SIGNED: + sec = *(Int64 *)op_data[1]; + if(sec < 0 ) + { +ExRaiseSqlError(heap, diagsArea, EXE_BAD_ARG_TO_MATH_FUNC); +*(*diagsArea) << DgString0("SLEEP"); +return ex_expr::EXPR_ERROR; + } + sleep(sec); + *(Int64 *)op_data[0] = 1; + break; + +default: +ExRaiseSqlError(heap, diagsArea, EXE_BAD_ARG_TO_MATH_FUNC); +*(*diagsArea) << DgString0("SLEEP"); + return ex_expr::EXPR_ERROR; + break; + } + //get the seconds to sleep + return ex_expr::EXPR_OK; +} + +ex_expr::exp_return_type ex_function_unixtime::eval(char *op_data[], + CollHeap* heap, + ComDiagsArea** diagsArea) +{ + char *opData = op_data[1]; + //if there is input value + if(opData[0] != 0 && getNumOperands() == 2) + { +struct tm* ptr; +char* r = strptime(opData, "%Y-%m-%d %H:%M:%S", ptr); +if( r == NULL) --- End diff -- Thanks Dave, you are correct. We want to be very strict here. I will add this checking. ---
[GitHub] trafodion pull request #1444: [TRAFODION-2157] add MySQL function unix_times...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1444 [TRAFODION-2157] add MySQL function unix_timestamp,uuid,sleep This is to add three MySQL compatible functions into Trafodion. It is required for another JIRA to add unix_timestamp, uuid as default value of a column definition. sleep is simple to add, so add with this change as well. The document will be updated for a separate JIRA asap. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2157 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1444.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1444 commit dbe4cac9c8e58c8dc9c6b86b5c473320df5c44c3 Author: Liu Ming Date: 2018-02-14T08:57:58Z [TRAFODION-2157] add MySQL function unix_timestamp,uuid,sleep commit 3555983cdeb03059277913babd646400f5f319ff Author: Liu Ming Date: 2018-02-14T08:58:30Z Merge branch 'master' of git://git.apache.org/trafodion into TRAFODION-2157 commit 3c6bce6e10e8f319bcdeffa272ffc4ac31a74f48 Author: Liu Ming Date: 2018-02-14T21:51:10Z [TRAFODION-2157] fix various issues ---
[GitHub] trafodion pull request #1441: [TRAFODION-2908] create table with wrong char ...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1441#discussion_r166279557 --- Diff: core/sql/bin/SqlciErrors.txt --- @@ -140,7 +140,7 @@ 1138 Z 9 ADVANCED CRTCL DIALOUT --- unused --- 1139 Z 9 BEGINNER MAJOR DBADMIN System-generated column $0~ColumnName of base table $1~TableName cannot appear in the search condition of a check constraint definition. 1140 Z 9 BEGINNER MAJOR DBADMIN Row-length $0~int0 exceeds the maximum allowed row-length of $1~int1 for table $2~TableName. -1141 Z 9 BEGINNER MAJOR DBADMIN --- unused --- +1141 Z 9 BEGINNER MAJOR DBADMIN Key-length $0~int0 exceeds the maximum allowed rowkey length $1~int1. --- End diff -- Thanks Dave for the review, I will update teh Message Guide and modify the workding ---
[GitHub] trafodion pull request #1441: [TRAFODION-2908] create table with wrong char ...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1441 [TRAFODION-2908] create table with wrong char length hang and crash add a check if the key length is too long report error. Otherwise, SQL will go into a very long process to build the encoded key and finally failed. And in some cases crash, it makes little sense to check why it crash, so just add check and return error when key is too long. HBase max rowkey len is 32K, so it is the hard limit for now. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2908 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1441.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1441 commit 5f5808ab9d66bc2fba02292b3a0e5c371d3f5678 Author: Liu Ming Date: 2018-02-05T08:11:50Z [TRAFODION-2908] create table with wrong char length hang and crash ---
[GitHub] trafodion pull request #1438: [TRAFODION-2951] group_concat on utf8 column w...
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1438 [TRAFODION-2951] group_concat on utf8 column with chinese character get messy code the pivot synthesize type function not consider the charset of output. Add code to copy the charset from child. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2951 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1438.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1438 commit c7039b920846ba084f8404ba48ea5c8ab2dd3dc5 Author: Liu Ming Date: 2018-02-03T10:09:27Z [TRAFODION-2951] group_concat on utf8 column with chinese character get messy code commit 3c963796066d13a2a55bb6a8fe903d82b5ea49da Author: Liu Ming Date: 2018-02-03T11:12:02Z add regression test ---
[GitHub] trafodion pull request #1435: Trafodion 2942
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1435 Trafodion 2942 update the NOTICE file year This is for Release R2.2 change. You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2942 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1435.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1435 commit ef73b7719c746bc721587f2b4b4a032d71d03c65 Author: liu.yu Date: 2017-08-15T04:12:48Z cherry-pick altercolumn commit a465ec83025a8a3664a1b0f9a28c86021287952a Author: liu.yu Date: 2017-08-15T03:22:21Z Capitalize XML commit 08b41edb08d313ed524e64392271d17c38ee9b19 Author: liu.yu Date: 2017-08-14T03:56:00Z cherry pick add alter column 2 commit fb007c00469a36b37e58d1821581fdbe38527626 Author: liu.yu Date: 2017-08-13T08:06:25Z cherry-pick Load XML Files commit 89576f3dc7f5e37977ec3cdd9f0fd5ecadee0d36 Author: Sandhya Sundaresan Date: 2017-08-22T17:38:42Z Changes to take care of closing internal cursor statement when extract operation is done and the exact length of data is passed in to be extracted. commit 8a260f425e0527100c8039b8db8a8b4db64b1101 Author: Sandhya Sundaresan Date: 2017-08-22T17:41:39Z Expected file change to show result of unload. commit 2048d253ca0ec8e01bf9f8b2c959a67f82da530a Author: Liu Ming Date: 2017-08-24T03:09:27Z Merge Fix for Cherry-pick changes of [TRAFODION-2703] & [TRAFODION-2710] to 2.2 commit f18644841f8d3c8aaa5c594c1018798d515821b0 Author: Sandhya Sundaresan Date: 2017-08-31T20:03:05Z Main change: Added a method for JDBC driver to be able to close an open cursor abruptly by passing in a 0 length to the extract syntax. This will ensure internal opens are not left around if the application does not retrieve all the data with the stream API. This abrupt close can be called when the result set is closed or the connection is closed by keeping track of all the open Blob objects. (in the driver) Added a freeResources method and destructor to LobExtract and LobUpdate Tcbs. Minor changes and fixes : Refactored the ExLob::closeCursor Removed the LOB_CLI_SELECT_CLOSE call from ::fetchCursor so it gets closed only when ExLob::closeCursor is called. Makes the code consistent and readable. Took care of a few memory allocations and deletions. Refactored the LOB length calculation into on ecommon method so it works correctly for both regular LOB dat aas well as external LOB files. Earlier the external lobs length was being returned incorrectly. Now it reports the length of the external HDFS file . Fixed one place in generator where the CQD value was not being multiplied by the MB units correctly. commit 615b8dd26d38e19ba17cf9451453bc38e8879dcc Author: Sandhya Sundaresan Date: 2017-09-04T06:58:05Z Merge remote branch 'origin/pr/1216/head' into merge_1216 commit e8c09ebb678ab6560130bf687dc1e52c9b04d64a Author: Sandhya Sundaresan Date: 2017-09-11T17:57:02Z Changes to handle errors during drop of lob tables that leave the table inconsistent. (cherry picked from commit 749295ad774620d49db79d961b1bbacac8d589c7) commit 42b2b0b4a56e9b125d15eaa227976d11dcec1b3c Author: Sandhya Sundaresan Date: 2017-09-12T17:51:49Z Merge remote branch 'origin/pr/1230/head' into merge_1230 commit 808c87450f343e6df3dcc82e0799919581958513 Author: Weiqing Xu Date: 2017-09-20T05:49:28Z [TRAFODION-2704]close unused statment in JDBC regression commit bd792ff7ef0a81c6b0106ce1d2aaeb3ff34d9701 Author: Weiqing Xu Date: 2017-09-20T05:57:26Z [TRAFODION-2740]JDBC extract LOB context in chunks commit ea80ab383da807fe4fd61d9a9193c08ad2c79283 Author: Weiqing Xu Date: 2017-09-22T16:44:40Z [TRAFODION-2740]remove unused code commit 835a48945afe3cc19ec58ac94e22b323db69d5cf Author: Weiqing Xu Date: 2017-09-23T17:47:33Z [TRAFODION-2740]add property to set the LOB chunk size commit 1ade4208f1e3551fa4ce2c7cbfda29a4c8c68b15 Author: Weiqing Xu Date: 2017-09-23T17:48:39Z [TRAFODION-2704]reslove the random schema not exist issue commit e26ec04a7815cb46f3d10d9816398c10ad13df67 Author: Sandhya Sundaresan Date: 2017-09-25T04:32:50Z Merge remote branch 'origin/pr/1240/head' into merge_1240 commit de664c37a95f89573ccc4466357d5ebf94df22a5 Author: Weiqing Xu Date: 2017-09-25T05:47:56Z [TRAFODION-2753]LOB: filetolob() no longer works in trafci commit 3254cc16b259a6a513cabc391699f44f29850cf1 Author: Sandhya Sundaresan Date: 2017-09-26T05:31:16Z Merge remote branch 'origin/pr/1242/head' into merge_1242 commit f077620ae434bfdc115a599a591bec1f5eb7bff6 Author: Steve Varnau Date: 2017-12-22T19:23:5
[GitHub] trafodion pull request #1406: [TRAFODION-2918] fix regression tests
GitHub user traflm opened a pull request: https://github.com/apache/trafodion/pull/1406 [TRAFODION-2918] fix regression tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/trafodion TRAFODION-2918 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/trafodion/pull/1406.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1406 commit ed07e6c3273abe48532cc81a3ce535eabe8b9164 Author: Liu Ming Date: 2018-01-18T19:17:50Z [TRAFODION-2918] fix regression tests ---
[GitHub] trafodion pull request #1399: [TRAFODION-2909] Add ROLLUP Function for *Traf...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1399#discussion_r162052172 --- Diff: docs/sql_reference/src/asciidoc/_chapters/sql_functions_and_expressions.adoc --- @@ -6337,6 +6337,300 @@ UPDATE persnl.job SET jobdesc = RIGHT (jobdesc, 12); ``` +<<< +[[rollup_function]] +== ROLLUP Function + +The ROLLUP function calculates multiple levels of subtotals aggregating from right to left through the comma-separated list of columns, and provides a grand total. It is a an extension to the `GROUP BY` clause and can be used with `ORDER BY` to sort the results. + +``` +SELECTâ¦GROUP BY ROLLUP (column 1, [column 2,]â¦[column n]) +``` + +ROLLUP generates n+1 levels of subtotals and grand total, where n is the number of the selected column(s). + +For example, a query that contains three rollup columns returns the following rows: + +* First-level: stand aggregate values calculated by GROUP BY clause without using ROLLUP. +* Second-level: subtotals aggregating across column 3 for each combination of column 1 and column 2. +* Third-level: subtotals aggregating across column 2 and column 3 for each column 1. +* Fourth-level: the grand total row. + +NOTE: Trafodion does not support CUBE function which works slightly differently from ROLLUP. + +[[considerations_for_rollup]] +=== Considerations for ROLLUP + +[[null_in_result_sets]] + NULL in Result Sets + +* The NULLs in each super-aggregate row represent subtotals and grand total. +* The NULLs in selected columns are considered equal and sorted into one NULL group in result sets. + +[[using_rollup_with_the_column_order_reversed]] + Using ROLLUP with the Column Order Reversed + +ROLLUP removes the right-most column at each step, therefore the result sets vary with the column order specified in the comma-separated list. + +[cols="50%,50%"] +|=== +| If the column order is _country_, _state_, _city_ and _name_, ROLLUP returns following groupings. +| If the column order is _name_, _city_, _state_ and _country_, ROLLUP returns following groupings. +| _country_, _state_, _city_ and _name_ | _name_, _city_, _state_ and _country_ +| _country_, _state_ and _city_ | _name_, _city_ and _state_ +| _country_ and _state_ | _name_ and _city_ +| _country_ | _name_ +| grand total| grand total +|=== + +[[examples_of_rollup]] +=== Examples of ROLLUP + +[[examples_of_grouping_by_one_or_multiple_rollup_columns]] + Examples of Grouping By One or Multiple Rollup Columns + +Suppose that we have a _sales1_ table like this: + +``` +SELECT * FROM sales1; + +DELIVERY_YEAR REGION PRODUCT REVENUE +- -- --- + 2016 A Dress100 + 2016 A Dress200 + 2016 A Pullover 300 + 2016 B Dress400 + 2017 A Pullover 500 + 2017 B Dress600 + 2017 B Pullover 700 + 2017 B Pullover 800 + +--- 8 row(s) selected. +``` + +* This is an example of grouping by one rollup column. ++ +``` +SELECT delivery_year, SUM (revenue) AS total_revenue +FROM sales1 +GROUP BY ROLLUP (delivery_year); +``` + ++ +``` +DELIVERY_YEAR TOTAL_REVENUE +- + 2016 1000 + 2017 2600 + NULL 3600 + +--- 3 row(s) selected. +``` + +* This is an example of grouping by two rollup columns. ++ +ROLLUP firstly aggregates at the lowest level (_region_) and then rollup those aggregations to the next +level (_delivery_year_), finally it produces a grand total across these two levels. + ++ +``` +SELECT delivery_year, region, SUM (revenue) AS total_revenue +FROM sales1 +GROUP BY ROLLUP (delivery_year, region); +``` + ++ +``` +DELIVERY_YEAR REGION TOTAL_REVENUE +- -- + 2016 A 600 + 2016 B 400 + 2016 NULL 1000 + 2017 A 500 + 2017 B 2100
[GitHub] trafodion pull request #1399: [TRAFODION-2909] Add ROLLUP Function for *Traf...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1399#discussion_r162050572 --- Diff: docs/sql_reference/src/asciidoc/_chapters/sql_functions_and_expressions.adoc --- @@ -6337,6 +6337,300 @@ UPDATE persnl.job SET jobdesc = RIGHT (jobdesc, 12); ``` +<<< +[[rollup_function]] --- End diff -- I think this is not a function but a clause, so I suggest move this chapter into chapter 6? I am not sure about this, @DaveBirdsall what do you think here? ---
[GitHub] trafodion pull request #1394: [TRAFODION-2891] fix the bufoverrun Critical e...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1394#discussion_r161806802 --- Diff: core/sqf/src/seatrans/tm/hbasetmlib2/hbasetm.h --- @@ -139,9 +139,9 @@ class CHbaseTM : public JavaObjectInterfaceTM JM_REGTRUNCABORT, JM_DROPTABLE, JM_RQREGINFO, - JM_LAST + JM_TMLAST }; - JavaMethodInit JavaMethods_[JM_LAST]; + JavaMethodInit JavaMethods_[JM_TMLAST]; --- End diff -- This is critical! ---
[GitHub] trafodion pull request #1394: [TRAFODION-2891] fix the bufoverrun Critical e...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1394#discussion_r161806495 --- Diff: core/conn/odbc/src/odbc/nsksrvrcore/srvrothers.cpp --- @@ -5153,7 +5153,7 @@ odbc_SQLSvc_GetSQLCatalogs_sme_( { ERROR_DESC_def *p_buffer = QryCatalogSrvrStmt->sqlError.errorList._buffer; strncpy(RequestError, p_buffer->errorText,sizeof(RequestError) -1); -RequestError[sizeof(RequestError)] = '\0'; +RequestError[sizeof(RequestError) - 1] = '\0'; --- End diff -- very good catch! ---
[GitHub] trafodion pull request #1394: [TRAFODION-2891] fix the bufoverrun Critical e...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1394#discussion_r161806194 --- Diff: core/conn/odbc/src/odbc/Common/linux/sqmem.cpp --- @@ -400,7 +400,7 @@ void TestPool(void * membase, int length) long j, index; short error; long pass; - void * pool_ptrs[256]; + void * pool_ptrs[256 + 1]; --- End diff -- very good catch! ---
[GitHub] trafodion pull request #1394: [TRAFODION-2891] fix the bufoverrun Critical e...
Github user traflm commented on a diff in the pull request: https://github.com/apache/trafodion/pull/1394#discussion_r161805913 --- Diff: core/conn/odb/src/odb.c --- @@ -5313,7 +5313,7 @@ static void etabadd(char type, char *run, int id) } } if ( etab[no].type == 'e' ) { /* name & create output file */ -for ( i = j = 0; etab[no].tgt[i] && i < sizeof(buff); i++ ) { +for ( i = j = 0; i < sizeof(buff) && etab[no].tgt[i]; i++ ) { --- End diff -- I don't really get this fully understand. If i is out of bound for arry tgt, why this change will prevent the tag[i] to be executed? ---