[jira] [Commented] (TRAFODION-1834) ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in sqconfig are fully qualified
[ https://issues.apache.org/jira/browse/TRAFODION-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153850#comment-15153850 ] liu ming commented on TRAFODION-1834: - The issue is slightly different. The FQDN is not a problem. The binder will build the initial node_map for a given table, at that time, the API invoked is called createNodeMapForHbase(), in that function, it will truncate the FQDN into hostname. And the NAClusterInfo class which collecting the hostname to nodeId mapping will also do the truncation. So it is a match. But there is a new problem that this jira will fix: In a system, there is a cluster with 10 nodes, HBase RS installed on 8 of them. Table bltest have 100M rows, 250G in one-replica size, split into 100 regions, evenly spread over 8 nodes. do a very simple test: select [last 1]* from bltest; Without this CQD set, it launched 10 ESPs over 10 nodes, and the ESP node_map printed as: esp_2_node_map . (\NSK:-1:-1:-1:-1:-1:-1:-1:-1:-1:-1) which means randomly locate each ESP With this CQD, each time, the node_map is different, such as: esp_2_node_map . (\NSK:0:0:0:0:0:0:0:0:0:0) esp_2_node_map . (\NSK:7:7:7:7:7:7:7:7:7:7) you can notice, the 10 ESP will be put into same node when CQD is ‘on’, and the node number is random. This bug is in optimizer’s NodeMap::getPopularNodeNumber() function, it tries to find out a most popular node. In above case, 10 ESPs try to read 100 regions, so for the first ESP, it needs to read 10 regions, 0~9, but region 0 and region 1 may be in different RS node, so this function is trying to find a RS which serves the most regions from Region 0 to Region 9, and locate first ESP there. But this function use an uninitialized array to do the job. It malloc a new buffer and use as counter array, but GCC is not always clear the newly alloc buffer, so you never know what init value in that array, so if node[0] init with a very big number, it will always win, and we saw: esp_2_node_map . (\NSK:0:0:0:0:0:0:0:0:0:0) a fix is to init the array. And in this change, we remove the bias in getPopularNodeNumber() to lowest node ID, but make it random. > ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in > sqconfig are fully qualified > -- > > Key: TRAFODION-1834 > URL: https://issues.apache.org/jira/browse/TRAFODION-1834 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 1.3-incubating >Reporter: Atanu Mishra >Assignee: liu ming > > Paraphrasing Hans, who looked into this briefly: > Looking at the code, I can see that we remove anything from the first dot in > the node name of the region. Then we search for the unqualified node name in > the list of node names of the Trafodion cluster. > Both of these clusters use FQDNs in their configuration, though, therefore we > won't find the node names. > To Reproduce: > Do an EXPLAIN of a parallel query with and without TRAF_ALLOW_ESP_COLOCATION > set, and the node map will not show a co-located plan when the CQD is on - if > the system uses FQDNs in its sqconfig file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1834) ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in sqconfig are fully qualified
[ https://issues.apache.org/jira/browse/TRAFODION-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153840#comment-15153840 ] ASF GitHub Bot commented on TRAFODION-1834: --- GitHub user traflm opened a pull request: https://github.com/apache/incubator-trafodion/pull/330 [TRAFODION-1834] fix issue in ESP colocation You can merge this pull request into a Git repository by running: $ git pull https://github.com/traflm/incubator-trafodion TRAFODION-1834 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafodion/pull/330.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #330 commit 2f0956fee3cac2e9410a716c2dd7c1ceb15af264 Author: Liu Ming Date: 2016-02-19T07:16:51Z [TRAFODION-1834] fix issue in ESP colocation > ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in > sqconfig are fully qualified > -- > > Key: TRAFODION-1834 > URL: https://issues.apache.org/jira/browse/TRAFODION-1834 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 1.3-incubating >Reporter: Atanu Mishra >Assignee: liu ming > > Paraphrasing Hans, who looked into this briefly: > Looking at the code, I can see that we remove anything from the first dot in > the node name of the region. Then we search for the unqualified node name in > the list of node names of the Trafodion cluster. > Both of these clusters use FQDNs in their configuration, though, therefore we > won't find the node names. > To Reproduce: > Do an EXPLAIN of a parallel query with and without TRAF_ALLOW_ESP_COLOCATION > set, and the node map will not show a co-located plan when the CQD is on - if > the system uses FQDNs in its sqconfig file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1839) Trafodion Installer Evolution
Gunnar Tapper created TRAFODION-1839: Summary: Trafodion Installer Evolution Key: TRAFODION-1839 URL: https://issues.apache.org/jira/browse/TRAFODION-1839 Project: Apache Trafodion Issue Type: Improvement Components: documentation, installer Affects Versions: 2.0-incubating Reporter: Gunnar Tapper Umbrella for evolution of the Trafodion Installer. Overall motivation is: * Position Trafodion as a pure Hadoop add-on: the Hadoop environment must be installed and fully functional before installing Trafodion. * Use as much of the existing function as possible. * Define distinct phases that fit into common IT practices. * Enhanced user experience. Consideration: IT departments often separate responsibilities such as security administration, configuration changes, etc. These type of activities need to be planned and executed during specific maintenance windows. All acivities are audited in order to be compliant with local regulations; for examle, the Sarbanes-Oxley Act. In contrast, the current behavior of the Trafodion Installer, which assumes that it can obtain configuration information, change Hadoop configuration, install software packages, set up user IDs, and install/deploy Trafodion in one session. The following distinct install phases are defined. 1. Environment validation: Ensure that the environment is suited for Trafodion. Identify required changes. The Trafodion Scanner should be able to fulfill this need. The current Scanner Configuration file is out-of-date so it generates false positives such as flagging hyper threading, which is now OK. NOTE: Need to spend more time on the Scanner. The design and implementation looks quite elegant and robust. 2. Configuration builder: Automated Mode (aka expert mode) uses a documented configuration file with environmental information to drive the installation. Guided Mode prompts the user for information creating a comment-free configuration file. Change the Guided Mode to be a configuration builder that builds a fully-commented configuration file. Enhance Guided Mode to do more discovery using the REST APIs/Hadoop configuration files to obtain information that the user have to provide today. The goal here is to reduce user-supplied information to the absolute minimum: Trafodion user password, target install directory, tar file, and distribution-manager user and password. >From past experiences, it's possible to automatically detect what type of Hadoop installation is used, URL to distribution manager or location of configuration files, and then rely on REST APIs/configuration files to discover nodes, current settings, and so forth. All prompts are done in a loop to allow for review/approval of the input. When completed, the user is shown the overall settings in a numbered menu thereby allowing for new edits, if needed. Once approved, the commented configuration file is created thereby allowing the user to directly edit the file in later upgrades. As today, new runs of the Installer reads the configuration file for prompt default values. It's probably best to implement discovery functions in Python because it has good handling of REST APIs, reading XML files, and parsing JSON responses. 3. Configuration changer: Trafodion requires specific system, HDFS, and HBase settings. These should be implemented BEFORE installing Trafodion so that the user can a) time the required changes b) choose to make the changes using an alternative means c) validate that the changes did not break their Hadoop environment. Keep today's capability of restarting Hadoop services but don't invoke configuration change unless required. (Don't do it blindly like today; restarting Hadoop services is a big deal and should be avoided as much as possible.) 4. User Setup: Create/validate mode to allow for cases where security administrators must perform user creation. 5. Package Download: Validate/download mode for required packages. 6. Trafodion Install: As today. 7. Activation: As today trafodion_install adds a set of new arguments to invoke the different steps per the list above: No argument displays help text -a, --automated Run installer in automated mode. Cannot be combined with -b. -c, --config_file Location of configuration file -v, --validate Validatation mode. Applies to -s and -p. -s, --scan Checks the current Hadoop and system environment for Trafodion usage. -b, --build Guided Mode. Builds --config_file. -u, --user Creates/validates Trafodion user. -v defines validate mode. -p, --package Installs/validates required software packages. -v defines validate mode. -i, --install Installs the Trafodion software. -a, --activate Starts Trafodion. Other changes: * Move installer settings to a separate file that's sourced in. Today, most installer files defines the se
[jira] [Updated] (TRAFODION-1628) Implement T2 Driver's Rowsets ability to enhance the batch insert performance
[ https://issues.apache.org/jira/browse/TRAFODION-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian Jin updated TRAFODION-1628: Assignee: RuoYu Zuo > Implement T2 Driver's Rowsets ability to enhance the batch insert performance > - > > Key: TRAFODION-1628 > URL: https://issues.apache.org/jira/browse/TRAFODION-1628 > Project: Apache Trafodion > Issue Type: Improvement > Components: client-jdbc-t2 >Reporter: RuoYu Zuo >Assignee: RuoYu Zuo >Priority: Critical > Labels: features, performance > > JDBC T2 Driver now has very poor performance of batch insert, because it does > not have rowsets ability. Implement rowsets functionality will allow T2 > Driver performs batch insert operation much faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1838) Upadate statistics with sampling does not work occasionally when table has nullable key column
Suresh Subbiah created TRAFODION-1838: - Summary: Upadate statistics with sampling does not work occasionally when table has nullable key column Key: TRAFODION-1838 URL: https://issues.apache.org/jira/browse/TRAFODION-1838 Project: Apache Trafodion Issue Type: Bug Components: sql-cmp Affects Versions: 1.2-incubating Reporter: Suresh Subbiah Assignee: David Wayne Birdsall If a table has a nullable key column then update statistics is not able to create a sample table. This is true even if the cqd allow_nullable_unique_key_constraint 'on' is set before issuing update stats statement. To reproduce drop table test1 ; cqd allow_nullable_unique_key_constraint 'on' ; create table test1(a int) store by (a) ; insert into test1 values (1),(2),(3),(4),(5),(6),(7),(8) ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; upsert using load into test1 select * from test1 ; -- should have about a million rows in table now exit; cqd ustat_use_is_when_no_stats 'off' ; update statistics for table test1 on every column sample ; If IS is used then for such narrow tables we don't bother to create a sample table even if we do sample. Its all done in one pass. This may have something to do with the secondary compiler gets the cqd cqd allow_nullable_unique_key_constraint 'on' ; Even if this cqd is set before update stats command is issued error is seen. Though sometimes if there is some DDL before update stats and the cqd is already issued before the DDL, update stats with sampling on such tables does seem to work. I am not certain about this though. Error seen >>update statistics for table test1 on every column sample ; *** ERROR[9214] Object TRAFODION.SEABASE.SQLMX_7303240222670804273_1455851118_723941 could not be created. *** ERROR[1135] Clustering key column A must be assigned a NOT NULL NOT DROPPABLE constraint. *** ERROR[9201] Unable to DROP object TRAFODION.SEABASE.SQLMX_7303240222670804273_1455851118_723941. *** ERROR[1389] Object TRAFODION.SEABASE.SQLMX_7303240222670804273_1455851118_723941 does not exist in Trafodion. --- SQL operation failed with errors. Ustat code is setting the necessary cqd though it does not always seem to get to where it is needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1835) LICENSE framework for source and binary distributions
[ https://issues.apache.org/jira/browse/TRAFODION-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153477#comment-15153477 ] ASF GitHub Bot commented on TRAFODION-1835: --- Github user svarnau commented on a diff in the pull request: https://github.com/apache/incubator-trafodion/pull/329#discussion_r53409882 --- Diff: licenses/Makefile --- @@ -0,0 +1,46 @@ +# @@@ START COPYRIGHT @@@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +# @@@ END COPYRIGHT @@@ + +all: LICENSE-src LICENSE-dcs LICENSE-server LICENSE-install LICENSE-rest LICENSE-clients + +# All source code included in Trafodion source +LICENSE-src: + cat Apache lic-components-src lic-dcs-src lic-server-src lic-test-src > $@ + +# Binary licenses for each separtely packaged component --- End diff -- Fixing now, along with makefile bug. > LICENSE framework for source and binary distributions > - > > Key: TRAFODION-1835 > URL: https://issues.apache.org/jira/browse/TRAFODION-1835 > Project: Apache Trafodion > Issue Type: Sub-task > Components: Build Infrastructure >Affects Versions: 2.0-incubating >Reporter: Steve Varnau >Assignee: Steve Varnau > > Create single location for license information that can be used to generate > appropriate LICENSE files for source distribution as well as binary > distributions for each of the package files we generate: > - server > - clients > - install > - dcs > - rest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1835) LICENSE framework for source and binary distributions
[ https://issues.apache.org/jira/browse/TRAFODION-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153430#comment-15153430 ] ASF GitHub Bot commented on TRAFODION-1835: --- Github user DaveBirdsall commented on a diff in the pull request: https://github.com/apache/incubator-trafodion/pull/329#discussion_r53406054 --- Diff: licenses/Makefile --- @@ -0,0 +1,46 @@ +# @@@ START COPYRIGHT @@@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +# @@@ END COPYRIGHT @@@ + +all: LICENSE-src LICENSE-dcs LICENSE-server LICENSE-install LICENSE-rest LICENSE-clients + +# All source code included in Trafodion source +LICENSE-src: + cat Apache lic-components-src lic-dcs-src lic-server-src lic-test-src > $@ + +# Binary licenses for each separtely packaged component --- End diff -- Spelling error in comment (fix it later): "separtely" > LICENSE framework for source and binary distributions > - > > Key: TRAFODION-1835 > URL: https://issues.apache.org/jira/browse/TRAFODION-1835 > Project: Apache Trafodion > Issue Type: Sub-task > Components: Build Infrastructure >Affects Versions: 2.0-incubating >Reporter: Steve Varnau >Assignee: Steve Varnau > > Create single location for license information that can be used to generate > appropriate LICENSE files for source distribution as well as binary > distributions for each of the package files we generate: > - server > - clients > - install > - dcs > - rest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1835) LICENSE framework for source and binary distributions
[ https://issues.apache.org/jira/browse/TRAFODION-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153390#comment-15153390 ] ASF GitHub Bot commented on TRAFODION-1835: --- GitHub user svarnau opened a pull request: https://github.com/apache/incubator-trafodion/pull/329 [TRAFODION-1835] Re-factor license files Enable building license files for source and binary distributions. Since we build several different distribution packages, the LICENSE files need to be specific to the components packaged. The licenses/Makefile indicates how we combine each component source and binary license files for source package and the various binary distribution packages. Binary distribution license info is still incomplete. This change sets the framework, and license content will be completed later. Several of the licenses/*-bin files have "To Do" items indicating remaining work to be done related to TRAFODION-1816. You can merge this pull request into a Git repository by running: $ git pull https://github.com/svarnau/incubator-trafodion lic1835 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-trafodion/pull/329.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #329 commit d6830446562ffafbea4e9fc4c8542f4e2fdee39f Author: Steve Varnau Date: 2016-02-18T02:23:09Z [TRAFODION-1835] Re-factor license files Enable building license files for source and binary distributions. Since we build several different distribution packages, the LICENSE files need to be specific to the components packaged. The licenses/Makefile indicates how we combine each component source and binary license files for source package and the various binary distribution packages. Binary distribution license info is still incomplete. This change sets the framework, and license content will be completed later. Several of the licenses/*-bin files have "To Do" items indicating remaining work to be done related to TRAFODION-1816. > LICENSE framework for source and binary distributions > - > > Key: TRAFODION-1835 > URL: https://issues.apache.org/jira/browse/TRAFODION-1835 > Project: Apache Trafodion > Issue Type: Sub-task > Components: Build Infrastructure >Affects Versions: 2.0-incubating >Reporter: Steve Varnau >Assignee: Steve Varnau > > Create single location for license information that can be used to generate > appropriate LICENSE files for source distribution as well as binary > distributions for each of the package files we generate: > - server > - clients > - install > - dcs > - rest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1837) License information for binary distribution of linux clients
Steve Varnau created TRAFODION-1837: --- Summary: License information for binary distribution of linux clients Key: TRAFODION-1837 URL: https://issues.apache.org/jira/browse/TRAFODION-1837 Project: Apache Trafodion Issue Type: Sub-task Reporter: Steve Varnau Identify licenses for third-party sw bundled in linux client builds -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1836) License information for binary distribution of win-odbc
Steve Varnau created TRAFODION-1836: --- Summary: License information for binary distribution of win-odbc Key: TRAFODION-1836 URL: https://issues.apache.org/jira/browse/TRAFODION-1836 Project: Apache Trafodion Issue Type: Sub-task Components: client-odbc-windows Reporter: Steve Varnau Gather license information for any software bundled into the win-odbc build and packaged with other trafodion clients -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-1835) LICENSE framework for source and binary distributions
Steve Varnau created TRAFODION-1835: --- Summary: LICENSE framework for source and binary distributions Key: TRAFODION-1835 URL: https://issues.apache.org/jira/browse/TRAFODION-1835 Project: Apache Trafodion Issue Type: Sub-task Components: Build Infrastructure Affects Versions: 2.0-incubating Reporter: Steve Varnau Create single location for license information that can be used to generate appropriate LICENSE files for source distribution as well as binary distributions for each of the package files we generate: - server - clients - install - dcs - rest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (TRAFODION-1835) LICENSE framework for source and binary distributions
[ https://issues.apache.org/jira/browse/TRAFODION-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on TRAFODION-1835 started by Steve Varnau. --- > LICENSE framework for source and binary distributions > - > > Key: TRAFODION-1835 > URL: https://issues.apache.org/jira/browse/TRAFODION-1835 > Project: Apache Trafodion > Issue Type: Sub-task > Components: Build Infrastructure >Affects Versions: 2.0-incubating >Reporter: Steve Varnau >Assignee: Steve Varnau > > Create single location for license information that can be used to generate > appropriate LICENSE files for source distribution as well as binary > distributions for each of the package files we generate: > - server > - clients > - install > - dcs > - rest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1835) LICENSE framework for source and binary distributions
[ https://issues.apache.org/jira/browse/TRAFODION-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Varnau reassigned TRAFODION-1835: --- Assignee: Steve Varnau > LICENSE framework for source and binary distributions > - > > Key: TRAFODION-1835 > URL: https://issues.apache.org/jira/browse/TRAFODION-1835 > Project: Apache Trafodion > Issue Type: Sub-task > Components: Build Infrastructure >Affects Versions: 2.0-incubating >Reporter: Steve Varnau >Assignee: Steve Varnau > > Create single location for license information that can be used to generate > appropriate LICENSE files for source distribution as well as binary > distributions for each of the package files we generate: > - server > - clients > - install > - dcs > - rest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1825) Phoenix tests failure while running DescColumnSortOrderTest
[ https://issues.apache.org/jira/browse/TRAFODION-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153086#comment-15153086 ] Arvind Narain commented on TRAFODION-1825: -- Reply from Anoop: yes, primary key date column causes this crash. Looks like some issue with external date format to internal date format conversion during input values. let me check it out. SQL>drop table td; --- SQL operation complete. SQL>create table td (a int not null, b date not null primary key); --- SQL operation complete. SQL>prepare s from upsert into td values (?, ?); --- SQL command prepared. SQL>execute s using 1, '2012-01-01'; *** ERROR[29157] There was a problem reading from the server *** ERROR[29160] The message header was not long enough SQL> > Phoenix tests failure while running DescColumnSortOrderTest > --- > > Key: TRAFODION-1825 > URL: https://issues.apache.org/jira/browse/TRAFODION-1825 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-exe >Affects Versions: 2.0-incubating > Environment: Daily builds starting from 2/9, possibly linked to > changes in JIRA 1807. >Reporter: Arvind Narain > > Daily builds are failing while running Phoenix test DescColumnSortOrderTest > for both T2 and T4 env. > To simulate (for T4) > ./phoenix_test.py --target=localhost: --user=dontcare --pw=dontcare \ > --targettype=TR --javahome=$JAVA_HOME > --jdbccp=$MY_SQROOT/export/lib/jdbcT4.jar --tests=DescColumnSortOrderTest > Following is the core stack generated for T2: > #12 > #13 0x0230840fc085 in ?? () > #14 0x7fb3bb1b4cea in ExScheduler::work (this=0x7fb36c90a9c8, > prevWaitTime=0) at ../executor/ExScheduler.cpp:331 > #15 0x7fb3bb088cff in ex_root_tcb::oltExecute (this=0x7fb36c90bab8, > glob=0x7fb36c928150, input_desc=0x7fb36c92d238, output_desc=0x0, > diagsArea=@0x7fb3d4b19a98) at ../executor/ex_root.cpp:2166 > #16 0x7fb3ba8ff435 in CliStatement::doOltExecute (this=0x7fb36c9268d8, > cliGlobals=0x7fb3ccbf3c30, input_desc=0x7fb36c92d238, output_desc=0x0, > diagsArea=..., doNormalExecute=@0x7fb3d4b19bdc, reExecute=@0x7fb3d4b19bd8) at > ../cli/Statement.cpp:5504 > #17 0x7fb3ba87ead6 in SQLCLI_PerformTasks(CliGlobals *, ULng32, > SQLSTMT_ID *, SQLDESC_ID *, SQLDESC_ID *, Lng32, Lng32, typedef __va_list_tag > __va_list_tag *, SQLCLI_PTR_PAIRS *, SQLCLI_PTR_PAIRS *) > (cliGlobals=0x7fb3ccbf3c30, tasks=8063, statement_id=0x7fb3cd04e800, > input_descriptor=0x7fb3cd04e830, output_descriptor=0x0, > num_input_ptr_pairs=0, num_output_ptr_pairs=0, ap=0x7fb3d4b19dd0, > input_ptr_pairs=0x0, output_ptr_pairs=0x0) at ../cli/Cli.cpp:3257 > #18 0x7fb3ba87fef2 in SQLCLI_ClearExecFetchClose(CliGlobals *, SQLSTMT_ID > *, SQLDESC_ID *, SQLDESC_ID *, Lng32, Lng32, Lng32, typedef __va_list_tag > __va_list_tag *, SQLCLI_PTR_PAIRS *, SQLCLI_PTR_PAIRS *) > (cliGlobals=0x7fb3ccbf3c30, statement_id=0x7fb3cd04e800, > input_descriptor=0x7fb3cd04e830, output_descriptor=0x0, > num_input_ptr_pairs=0, num_output_ptr_pairs=0, num_total_ptr_pairs=0, > ap=0x7fb3d4b19dd0, input_ptr_pairs=0x0, output_ptr_pairs=0x0) at > ../cli/Cli.cpp:3788 > #19 0x7fb3ba9133c9 in SQL_EXEC_ClearExecFetchClose > (statement_id=0x7fb3cd04e800, input_descriptor=0x7fb3cd04e830, > output_descriptor=0x0, num_input_ptr_pairs=0, num_output_ptr_pairs=0, > num_total_ptr_pairs=0) at ../cli/CliExtern.cpp:2630 > #20 0x7fb3c0f3b15a in EXECUTE (pSrvrStmt=0x7fb3cd04e1f0) at > native/SqlInterface.cpp:1142 > #21 0x7fb3c0f361b6 in SRVR_STMT_HDL::Execute (this=0x7fb3cd04e1f0, > inCursorName=0x0, totalRowCount=1, inSqlStmtType=0, > inValueList=0x7fb3d4b1a2b0, inSqlAsyncEnable=0, inQueryTimeout=0, > outValueList=0x7fb3d4b1a2c0) at native/CSrvrStmt.cpp:260 > #22 0x7fb3c0f57a7e in odbc_SQLSvc_ExecuteN_sme_ (objtag_=0x0, > call_id_=0x0, exception_=0x7fb3d4b1a280, dialogueId=140410204143200, > stmtId=140410215522800, cursorName=0x0, sqlStmtType=0, totalRowCount=1, > inputValueList=0x7fb3d4b1a2b0, sqlAsyncEnable=0, queryTimeout=0, > outputValueList=0x7fb3d4b1a2c0, sqlWarning=0x7fb3d4b1a2e0) at > native/SrvrOthers.cpp:310 > #23 0x7fb3c0f50451 in > Java_org_trafodion_jdbc_t2_SQLMXPreparedStatement_execute > (jenv=0x7fb3cc0089d8, jobj=0x7fb3d4b1a4d8, server=0x0, > dialogueId=140410204143200, txid=0, autoCommit=1 '\001', txnMode=2, > stmtId=140410215522800, cursorName=0x0, isSelect=0 '\000', paramRowCount=1, > paramCount=2, paramValues=0x7fb3d4b1a470, queryTimeout=0, isAnyLob=0 '\000', > iso88591Encoding=0x0, resultSet=0x0, contBatchOnError=0 '\000') at > native/SQLMXPreparedStatement.cpp:365 > #24 0x7fb3c9012d98 in ?? () > #25 0x0002 in ?? () > #26 0x7fb3cd04e1f0 in ?? () > #27 0x in ?? () > Similar stack seen for T4, though currentl
[jira] [Commented] (TRAFODION-1825) Phoenix tests failure while running DescColumnSortOrderTest
[ https://issues.apache.org/jira/browse/TRAFODION-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153032#comment-15153032 ] Arvind Narain commented on TRAFODION-1825: -- Hi Selva, I didn't have my environment setup, hence couldn't try earlier. Following works under sqlci but fails with trafci, works before change for TRAFODION-1807. SQL>prepare s1 from upsert into DescColumnSortOrderTest (id, date1) values (?, ?); --- SQL command prepared. SQL>execute s1 using 1, '2012-01-01'; *** ERROR[29157] There was a problem reading from the server *** ERROR[29160] The message header was not long enough SQL> > Phoenix tests failure while running DescColumnSortOrderTest > --- > > Key: TRAFODION-1825 > URL: https://issues.apache.org/jira/browse/TRAFODION-1825 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-exe >Affects Versions: 2.0-incubating > Environment: Daily builds starting from 2/9, possibly linked to > changes in JIRA 1807. >Reporter: Arvind Narain > > Daily builds are failing while running Phoenix test DescColumnSortOrderTest > for both T2 and T4 env. > To simulate (for T4) > ./phoenix_test.py --target=localhost: --user=dontcare --pw=dontcare \ > --targettype=TR --javahome=$JAVA_HOME > --jdbccp=$MY_SQROOT/export/lib/jdbcT4.jar --tests=DescColumnSortOrderTest > Following is the core stack generated for T2: > #12 > #13 0x0230840fc085 in ?? () > #14 0x7fb3bb1b4cea in ExScheduler::work (this=0x7fb36c90a9c8, > prevWaitTime=0) at ../executor/ExScheduler.cpp:331 > #15 0x7fb3bb088cff in ex_root_tcb::oltExecute (this=0x7fb36c90bab8, > glob=0x7fb36c928150, input_desc=0x7fb36c92d238, output_desc=0x0, > diagsArea=@0x7fb3d4b19a98) at ../executor/ex_root.cpp:2166 > #16 0x7fb3ba8ff435 in CliStatement::doOltExecute (this=0x7fb36c9268d8, > cliGlobals=0x7fb3ccbf3c30, input_desc=0x7fb36c92d238, output_desc=0x0, > diagsArea=..., doNormalExecute=@0x7fb3d4b19bdc, reExecute=@0x7fb3d4b19bd8) at > ../cli/Statement.cpp:5504 > #17 0x7fb3ba87ead6 in SQLCLI_PerformTasks(CliGlobals *, ULng32, > SQLSTMT_ID *, SQLDESC_ID *, SQLDESC_ID *, Lng32, Lng32, typedef __va_list_tag > __va_list_tag *, SQLCLI_PTR_PAIRS *, SQLCLI_PTR_PAIRS *) > (cliGlobals=0x7fb3ccbf3c30, tasks=8063, statement_id=0x7fb3cd04e800, > input_descriptor=0x7fb3cd04e830, output_descriptor=0x0, > num_input_ptr_pairs=0, num_output_ptr_pairs=0, ap=0x7fb3d4b19dd0, > input_ptr_pairs=0x0, output_ptr_pairs=0x0) at ../cli/Cli.cpp:3257 > #18 0x7fb3ba87fef2 in SQLCLI_ClearExecFetchClose(CliGlobals *, SQLSTMT_ID > *, SQLDESC_ID *, SQLDESC_ID *, Lng32, Lng32, Lng32, typedef __va_list_tag > __va_list_tag *, SQLCLI_PTR_PAIRS *, SQLCLI_PTR_PAIRS *) > (cliGlobals=0x7fb3ccbf3c30, statement_id=0x7fb3cd04e800, > input_descriptor=0x7fb3cd04e830, output_descriptor=0x0, > num_input_ptr_pairs=0, num_output_ptr_pairs=0, num_total_ptr_pairs=0, > ap=0x7fb3d4b19dd0, input_ptr_pairs=0x0, output_ptr_pairs=0x0) at > ../cli/Cli.cpp:3788 > #19 0x7fb3ba9133c9 in SQL_EXEC_ClearExecFetchClose > (statement_id=0x7fb3cd04e800, input_descriptor=0x7fb3cd04e830, > output_descriptor=0x0, num_input_ptr_pairs=0, num_output_ptr_pairs=0, > num_total_ptr_pairs=0) at ../cli/CliExtern.cpp:2630 > #20 0x7fb3c0f3b15a in EXECUTE (pSrvrStmt=0x7fb3cd04e1f0) at > native/SqlInterface.cpp:1142 > #21 0x7fb3c0f361b6 in SRVR_STMT_HDL::Execute (this=0x7fb3cd04e1f0, > inCursorName=0x0, totalRowCount=1, inSqlStmtType=0, > inValueList=0x7fb3d4b1a2b0, inSqlAsyncEnable=0, inQueryTimeout=0, > outValueList=0x7fb3d4b1a2c0) at native/CSrvrStmt.cpp:260 > #22 0x7fb3c0f57a7e in odbc_SQLSvc_ExecuteN_sme_ (objtag_=0x0, > call_id_=0x0, exception_=0x7fb3d4b1a280, dialogueId=140410204143200, > stmtId=140410215522800, cursorName=0x0, sqlStmtType=0, totalRowCount=1, > inputValueList=0x7fb3d4b1a2b0, sqlAsyncEnable=0, queryTimeout=0, > outputValueList=0x7fb3d4b1a2c0, sqlWarning=0x7fb3d4b1a2e0) at > native/SrvrOthers.cpp:310 > #23 0x7fb3c0f50451 in > Java_org_trafodion_jdbc_t2_SQLMXPreparedStatement_execute > (jenv=0x7fb3cc0089d8, jobj=0x7fb3d4b1a4d8, server=0x0, > dialogueId=140410204143200, txid=0, autoCommit=1 '\001', txnMode=2, > stmtId=140410215522800, cursorName=0x0, isSelect=0 '\000', paramRowCount=1, > paramCount=2, paramValues=0x7fb3d4b1a470, queryTimeout=0, isAnyLob=0 '\000', > iso88591Encoding=0x0, resultSet=0x0, contBatchOnError=0 '\000') at > native/SQLMXPreparedStatement.cpp:365 > #24 0x7fb3c9012d98 in ?? () > #25 0x0002 in ?? () > #26 0x7fb3cd04e1f0 in ?? () > #27 0x in ?? () > Similar stack seen for T4, though currently in daily builds the logs do not > have the stack trace: > Running test.java.org.trafodion.phoenix.end2end.DescColumnSortOrderTest > test.j
[jira] [Commented] (TRAFODION-1825) Phoenix tests failure while running DescColumnSortOrderTest
[ https://issues.apache.org/jira/browse/TRAFODION-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152968#comment-15152968 ] Selvaganesan Govindarajan commented on TRAFODION-1825: -- I looked at the core file. The core file indicates that the query is UPSERT INTO DescColumnSortOrderTest (id,date1) VALUES(?,?) Can we attempt to reproduce this issue using sqlci. > Phoenix tests failure while running DescColumnSortOrderTest > --- > > Key: TRAFODION-1825 > URL: https://issues.apache.org/jira/browse/TRAFODION-1825 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-exe >Affects Versions: 2.0-incubating > Environment: Daily builds starting from 2/9, possibly linked to > changes in JIRA 1807. >Reporter: Arvind Narain > > Daily builds are failing while running Phoenix test DescColumnSortOrderTest > for both T2 and T4 env. > To simulate (for T4) > ./phoenix_test.py --target=localhost: --user=dontcare --pw=dontcare \ > --targettype=TR --javahome=$JAVA_HOME > --jdbccp=$MY_SQROOT/export/lib/jdbcT4.jar --tests=DescColumnSortOrderTest > Following is the core stack generated for T2: > #12 > #13 0x0230840fc085 in ?? () > #14 0x7fb3bb1b4cea in ExScheduler::work (this=0x7fb36c90a9c8, > prevWaitTime=0) at ../executor/ExScheduler.cpp:331 > #15 0x7fb3bb088cff in ex_root_tcb::oltExecute (this=0x7fb36c90bab8, > glob=0x7fb36c928150, input_desc=0x7fb36c92d238, output_desc=0x0, > diagsArea=@0x7fb3d4b19a98) at ../executor/ex_root.cpp:2166 > #16 0x7fb3ba8ff435 in CliStatement::doOltExecute (this=0x7fb36c9268d8, > cliGlobals=0x7fb3ccbf3c30, input_desc=0x7fb36c92d238, output_desc=0x0, > diagsArea=..., doNormalExecute=@0x7fb3d4b19bdc, reExecute=@0x7fb3d4b19bd8) at > ../cli/Statement.cpp:5504 > #17 0x7fb3ba87ead6 in SQLCLI_PerformTasks(CliGlobals *, ULng32, > SQLSTMT_ID *, SQLDESC_ID *, SQLDESC_ID *, Lng32, Lng32, typedef __va_list_tag > __va_list_tag *, SQLCLI_PTR_PAIRS *, SQLCLI_PTR_PAIRS *) > (cliGlobals=0x7fb3ccbf3c30, tasks=8063, statement_id=0x7fb3cd04e800, > input_descriptor=0x7fb3cd04e830, output_descriptor=0x0, > num_input_ptr_pairs=0, num_output_ptr_pairs=0, ap=0x7fb3d4b19dd0, > input_ptr_pairs=0x0, output_ptr_pairs=0x0) at ../cli/Cli.cpp:3257 > #18 0x7fb3ba87fef2 in SQLCLI_ClearExecFetchClose(CliGlobals *, SQLSTMT_ID > *, SQLDESC_ID *, SQLDESC_ID *, Lng32, Lng32, Lng32, typedef __va_list_tag > __va_list_tag *, SQLCLI_PTR_PAIRS *, SQLCLI_PTR_PAIRS *) > (cliGlobals=0x7fb3ccbf3c30, statement_id=0x7fb3cd04e800, > input_descriptor=0x7fb3cd04e830, output_descriptor=0x0, > num_input_ptr_pairs=0, num_output_ptr_pairs=0, num_total_ptr_pairs=0, > ap=0x7fb3d4b19dd0, input_ptr_pairs=0x0, output_ptr_pairs=0x0) at > ../cli/Cli.cpp:3788 > #19 0x7fb3ba9133c9 in SQL_EXEC_ClearExecFetchClose > (statement_id=0x7fb3cd04e800, input_descriptor=0x7fb3cd04e830, > output_descriptor=0x0, num_input_ptr_pairs=0, num_output_ptr_pairs=0, > num_total_ptr_pairs=0) at ../cli/CliExtern.cpp:2630 > #20 0x7fb3c0f3b15a in EXECUTE (pSrvrStmt=0x7fb3cd04e1f0) at > native/SqlInterface.cpp:1142 > #21 0x7fb3c0f361b6 in SRVR_STMT_HDL::Execute (this=0x7fb3cd04e1f0, > inCursorName=0x0, totalRowCount=1, inSqlStmtType=0, > inValueList=0x7fb3d4b1a2b0, inSqlAsyncEnable=0, inQueryTimeout=0, > outValueList=0x7fb3d4b1a2c0) at native/CSrvrStmt.cpp:260 > #22 0x7fb3c0f57a7e in odbc_SQLSvc_ExecuteN_sme_ (objtag_=0x0, > call_id_=0x0, exception_=0x7fb3d4b1a280, dialogueId=140410204143200, > stmtId=140410215522800, cursorName=0x0, sqlStmtType=0, totalRowCount=1, > inputValueList=0x7fb3d4b1a2b0, sqlAsyncEnable=0, queryTimeout=0, > outputValueList=0x7fb3d4b1a2c0, sqlWarning=0x7fb3d4b1a2e0) at > native/SrvrOthers.cpp:310 > #23 0x7fb3c0f50451 in > Java_org_trafodion_jdbc_t2_SQLMXPreparedStatement_execute > (jenv=0x7fb3cc0089d8, jobj=0x7fb3d4b1a4d8, server=0x0, > dialogueId=140410204143200, txid=0, autoCommit=1 '\001', txnMode=2, > stmtId=140410215522800, cursorName=0x0, isSelect=0 '\000', paramRowCount=1, > paramCount=2, paramValues=0x7fb3d4b1a470, queryTimeout=0, isAnyLob=0 '\000', > iso88591Encoding=0x0, resultSet=0x0, contBatchOnError=0 '\000') at > native/SQLMXPreparedStatement.cpp:365 > #24 0x7fb3c9012d98 in ?? () > #25 0x0002 in ?? () > #26 0x7fb3cd04e1f0 in ?? () > #27 0x in ?? () > Similar stack seen for T4, though currently in daily builds the logs do not > have the stack trace: > Running test.java.org.trafodion.phoenix.end2end.DescColumnSortOrderTest > test.java.org.trafodion.phoenix.end2end.DescColumnSortOrderTest > Tests run: 32, Failures: 1, Errors: 8, Skipped: 0, Time elapsed: 308.115 sec > <<< FAILURE! - in > test.java.org.trafodion.phoenix.end2end.DescColumnSortOrderTest > lessThanLeadi
[jira] [Updated] (TRAFODION-1618) Row count estimation method used in Update stats can be improved
[ https://issues.apache.org/jira/browse/TRAFODION-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall updated TRAFODION-1618: Fix Version/s: (was: 2.0-incubating) 1.3-incubating > Row count estimation method used in Update stats can be improved > > > Key: TRAFODION-1618 > URL: https://issues.apache.org/jira/browse/TRAFODION-1618 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Reporter: Qifan Chen >Assignee: David Wayne Birdsall > Fix For: 1.3-incubating > > > For a customer table, the row count is estimated as 2343750004, while the > actual row account is 25055150. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1618) Row count estimation method used in Update stats can be improved
[ https://issues.apache.org/jira/browse/TRAFODION-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall resolved TRAFODION-1618. - Resolution: Fixed Fix Version/s: 2.0-incubating Fixed. > Row count estimation method used in Update stats can be improved > > > Key: TRAFODION-1618 > URL: https://issues.apache.org/jira/browse/TRAFODION-1618 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Reporter: Qifan Chen >Assignee: David Wayne Birdsall > Fix For: 2.0-incubating > > > For a customer table, the row count is estimated as 2343750004, while the > actual row account is 25055150. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1740) Use CQDs to prevent time-outs and speed sampling in UPDATE STATS
[ https://issues.apache.org/jira/browse/TRAFODION-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall resolved TRAFODION-1740. - Resolution: Fixed Fix Version/s: 2.0-incubating This was fixed by adding logic to UPDATE STATISTICS to set the PARALLEL_NUM_ESPS explicity to get a higher degree of parallelism for the query to populate the sample table, and to set the HBASE_NUM_CACHE_ROWS_MIN and HBASE_NUM_CACHE_ROWS_MAX CQDs conservatively to avoid HBase timeouts. > Use CQDs to prevent time-outs and speed sampling in UPDATE STATS > > > Key: TRAFODION-1740 > URL: https://issues.apache.org/jira/browse/TRAFODION-1740 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 2.0-incubating, 1.3-incubating > Environment: All; however this is particularly noticeable in clusters >Reporter: David Wayne Birdsall >Assignee: David Wayne Birdsall > Fix For: 2.0-incubating > > > When doing UPDATE STATISTICS on large tables (one example had 17.7 billion > rows), we sometimes time-out with an error like the following: > SQL>update statistics for table table1 on every column sample; > *** ERROR[9200] UPDATE STATISTICS for table TRAFODION.SEABASE.TABLE1 encounter > ed an error (8448) from statement Process_Query. [2015-12-15 12:18:45] > *** ERROR[8448] Unable to access Hbase interface. Call to > ExpHbaseInterface::nex > tRow returned error HBASE_ACCESS_ERROR(-706). Cause: > java.util.concurrent.ExecutionException: > org.apache.hadoop.hbase.client.RetriesE > xhaustedException: Failed after attempts=36, exceptions: > Tue Dec 15 17:18:19 GMT 2015, null, java.net.SocketTimeoutException: > callTimeout > =6, callDuration=60303: row '?. [2015-12-15 12:18:45] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1777) UPDATE STATS on a non-existent object gives repetitive error messages
[ https://issues.apache.org/jira/browse/TRAFODION-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall resolved TRAFODION-1777. - Resolution: Fixed Fix Version/s: 2.0-incubating This was fixed in the same pull request as JIRA TRAFODION-1775. > UPDATE STATS on a non-existent object gives repetitive error messages > - > > Key: TRAFODION-1777 > URL: https://issues.apache.org/jira/browse/TRAFODION-1777 > Project: Apache Trafodion > Issue Type: Bug > Environment: All >Reporter: David Wayne Birdsall >Assignee: David Wayne Birdsall >Priority: Minor > Fix For: 2.0-incubating > > > Example: > >>update statistics for table non_existent on every column; > *** ERROR[4082] Object TRAFODION.SEABASE.NON_EXISTENT does not exist or is > inaccessible. > *** ERROR[4082] Object TRAFODION.SEABASE.NON_EXISTENT does not exist or is > inaccessible. > *** ERROR[4082] Object TRAFODION.SEABASE.NON_EXISTENT does not exist or is > inaccessible. > --- SQL operation failed with errors. > >> > One 4082 error would be enough. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1775) Raise a warning or error if UPDATE STATS is done w/o SAMPLE on a large table
[ https://issues.apache.org/jira/browse/TRAFODION-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall resolved TRAFODION-1775. - Resolution: Fixed Fix Version/s: 2.0-incubating Fixed. Added code to raise a warning or error if an UPDATE STATS is done on a large table without the SAMPLE clause. > Raise a warning or error if UPDATE STATS is done w/o SAMPLE on a large table > > > Key: TRAFODION-1775 > URL: https://issues.apache.org/jira/browse/TRAFODION-1775 > Project: Apache Trafodion > Issue Type: Improvement > Components: sql-cmp >Affects Versions: 2.0-incubating, 1.3-incubating > Environment: All >Reporter: David Wayne Birdsall >Assignee: David Wayne Birdsall > Fix For: 2.0-incubating > > > If UPDATE STATISTICS is done on a very large table (say, 17 billion rows) > without a SAMPLE clause, it can take hours to finish. The reason is that > there is not enough memory to do things in memory, so it will in the worst > case rescan the entire table for each histogram to create. The larger the > table the worse it gets. > The suggestion, then, is to raise an error if this is done unless syntax > overrides it. And in the latter case raise a warning. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1819) UPDATE STATS should not be allowed in a user transaction
[ https://issues.apache.org/jira/browse/TRAFODION-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall resolved TRAFODION-1819. - Resolution: Fixed Fix Version/s: 2.0-incubating Fixed. Added a check to disallow UPDATE STATS in a user-defined transaction. > UPDATE STATS should not be allowed in a user transaction > > > Key: TRAFODION-1819 > URL: https://issues.apache.org/jira/browse/TRAFODION-1819 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 2.0-incubating, 1.3-incubating > Environment: Any >Reporter: David Wayne Birdsall >Assignee: David Wayne Birdsall > Fix For: 2.0-incubating > > > UPDATE STATISTICS sometimes will do a mix of DDL and DML statements. These > cannot be mixed in one transaction. Therefore, UPDATE STATISTICS should not > be allowed in a user transaction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1831) Wrong error message when traf_coprocessor.properties is absent
[ https://issues.apache.org/jira/browse/TRAFODION-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall resolved TRAFODION-1831. - Resolution: Fixed Fixed by changing error message raised from 1394 to 1398. The latter gives the HBase error string, which points to a coprocessor issue. > Wrong error message when traf_coprocessor.properties is absent > -- > > Key: TRAFODION-1831 > URL: https://issues.apache.org/jira/browse/TRAFODION-1831 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmu >Affects Versions: 2.0-incubating >Reporter: David Wayne Birdsall >Assignee: David Wayne Birdsall >Priority: Minor > Fix For: 2.0-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1467) Update statistics does not generate multi-column key stats on a salted table
[ https://issues.apache.org/jira/browse/TRAFODION-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall resolved TRAFODION-1467. - Resolution: Fixed Fixed by changing default for CQD USTAT_ADD_SALTED_KEY_PREFIXES_FOR_MC to 'ON'. > Update statistics does not generate multi-column key stats on a salted table > > > Key: TRAFODION-1467 > URL: https://issues.apache.org/jira/browse/TRAFODION-1467 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmu >Affects Versions: 1.1 (pre-incubation) >Reporter: Rohit Jain >Assignee: David Wayne Birdsall >Priority: Minor > Labels: performance > Fix For: 2.0-incubating > > > The EVERY KEY or EVERY COLUMN option in update statistics generates > multi-column statistics for all the key columns. It does not do this if the > table is salted, potentially leading to poor plans. > Example of when it works: > create table t1 (a int NOT NULL NOT DROPPABLE, b int NOT NULL NOT DROPPABLE, > c int NOT NULL NOT DROPPABLE, primary key (a, b, c)); > update statistics for table t1 on every key; > showstats for table t1 on existing columns; > Histogram data for Table TRAFODION.BCI.T1 > Table ID: 7427073929238755441 >Hist ID # IntsRowcount UEC Colname(s) > == == === === === > 868236218 1 0 0 C > 868236223 1 0 0 B > 868236228 1 0 0 A > 868236233 1 0 0 A, B > 868236238 1 0 0 A, B, C > Example of when it doesn't (salted table): > create table t1 (a int NOT NULL NOT DROPPABLE, b int NOT NULL NOT > DROPPABLE, c int NOT NULL NOT DROPPABLE, primary key (a, b, c)) salt using 4 > partitions on (a, b); > update statistics for table t1 on every key; > showstats for table t1 on existing columns; > Histogram data for Table TRAFODION.BCI.T1 > Table ID: 7427073929238776270 >Hist ID # IntsRowcount UEC Colname(s) > == == === === === > 998895314 1 0 0 C > 998895319 1 0 0 B > 998895324 1 0 0 A > 998895329 1 0 0 "_SALT_" > 998895334 1 0 0 "_SALT_", A, B, C -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1472) SHOWSTATS EVERY COLUMN does not show multi-key-column stats for salted tables
[
https://issues.apache.org/jira/browse/TRAFODION-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Wayne Birdsall resolved TRAFODION-1472.
-
Resolution: Fixed
Fix Version/s: 2.0-incubating
Fixed by changing default for CQD USTAT_ADD_SALTED_KEY_PREFIXES_FOR_MC to 'ON'.
> SHOWSTATS EVERY COLUMN does not show multi-key-column stats for salted tables
> -
>
> Key: TRAFODION-1472
> URL: https://issues.apache.org/jira/browse/TRAFODION-1472
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmu
>Affects Versions: 1.1 (pre-incubation)
>Reporter: Rohit Jain
>Assignee: David Wayne Birdsall
>Priority: Minor
> Fix For: 2.0-incubating
>
>
> If Update Statistics is done on an unsalted table using the EVERY KEY option
> not only are all leading multi-key-column statistics gathered but SHOWSTATS
> display these statistics either with the EVERY COLUMN OPTION or the EXISTING
> COLUMNS option. It does not do so for tables that are salted.
> Works for table that is not salted
> -
> create table t2 (cust int not null not droppable, account int not null not
> droppable, txn_timestamp timestamp not null not droppable, amt numeric
> (18,4), primary key (cust, account, txn_timestamp));
> update statistics for table t2 on every key;
> showstats for table t2 on every column;
>Hist ID # IntsRowcount UEC Colname(s)
> == == === === ===
> 1409472594 1 0 0 CUST
> 1409472589 1 0 0 ACCOUNT
> 1409472584 1 0 0 TXN_TIMESTAMP
> 1409472604 1 0 0 CUST, ACCOUNT, TXN_TIMESTAMP
> 1409472599 1 0 0 CUST, ACCOUNT
> showstats for table t2 on existing columns;
>Hist ID # IntsRowcount UEC Colname(s)
> == == === === ===
> 1409472584 1 0 0 TXN_TIMESTAMP
> 1409472589 1 0 0 ACCOUNT
> 1409472594 1 0 0 CUST
> 1409472599 1 0 0 CUST, ACCOUNT
> 1409472604 1 0 0 CUST, ACCOUNT, TXN_TIMESTAMP
> Does not work for table that is salted
> ---
> create table t1 (cust int not null not droppable, account int not null not
> droppable, txn_timestamp timestamp not null not droppable, amt numeric
> (18,4), primary key (cust, account, txn_timestamp)) salt using 8 partitions
> on (cust, account) division by (date_part('YEARMONTH', txn_timestamp));
> update statistics for table t1 on every key;
> This does not generate leading multi-key-column statistics as reported in
> another JIRA. So follow this by:
> update statistics for table t1 on ("_SALT_", "_DIVISION_1_"), ("_SALT_",
> "_DIVISION_1_", CUST), ("_SALT_", "_DIVISION_1_", CUST, ACCOUNT);
> showstats for table t1 on every column;
>Hist ID # IntsRowcount UEC Colname(s)
> == == === === ===
> 270748103 1 0 0 CUST
> 270748096 1 0 0 ACCOUNT
> 270748092 1 0 0 TXN_TIMESTAMP
> 270748113 1 0 0 "_SALT_"
> 270748106 1 0 0 "_DIVISION_1_"
> 270748117 1 0 0 "_SALT_", "_DIVISION_1_", CUST,
> ACCOUNT, TXN_TIMESTAMP
> showstats for table t1 on existing columns;
>Hist ID # IntsRowcount UEC Colname(s)
> == == === === ===
> 270748092 1 0 0 TXN_TIMESTAMP
> 270748096 1 0 0 ACCOUNT
> 270748103 1 0 0 CUST
> 270748106 1 0 0 "_DIVISION_1_"
> 270748113 1 0 0 "_SALT_"
> 270748117 1 0 0 "_SALT_", "_DIVISION_1_", CUST,
> ACCOUNT, TXN_TIMESTAMP
> 270748122 1 0 0 "_SALT_", "_DIVISION_1_"
> 270748127 1 0 0 "_SALT_", "_DIVISION_1_", CUST
> 270748132 1 0 0 "_SALT_", "_DIVISION_1_", CUST,
> ACCOUNT
> The EXISTING COLUMNS option displays all leading multi-key-column statistics,
> ON EVERY COLUMN doesn't.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TRAFODION-1467) Update statistics does not generate multi-column key stats on a salted table
[ https://issues.apache.org/jira/browse/TRAFODION-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Wayne Birdsall updated TRAFODION-1467: Fix Version/s: 2.0-incubating > Update statistics does not generate multi-column key stats on a salted table > > > Key: TRAFODION-1467 > URL: https://issues.apache.org/jira/browse/TRAFODION-1467 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmu >Affects Versions: 1.1 (pre-incubation) >Reporter: Rohit Jain >Assignee: David Wayne Birdsall >Priority: Minor > Labels: performance > Fix For: 2.0-incubating > > > The EVERY KEY or EVERY COLUMN option in update statistics generates > multi-column statistics for all the key columns. It does not do this if the > table is salted, potentially leading to poor plans. > Example of when it works: > create table t1 (a int NOT NULL NOT DROPPABLE, b int NOT NULL NOT DROPPABLE, > c int NOT NULL NOT DROPPABLE, primary key (a, b, c)); > update statistics for table t1 on every key; > showstats for table t1 on existing columns; > Histogram data for Table TRAFODION.BCI.T1 > Table ID: 7427073929238755441 >Hist ID # IntsRowcount UEC Colname(s) > == == === === === > 868236218 1 0 0 C > 868236223 1 0 0 B > 868236228 1 0 0 A > 868236233 1 0 0 A, B > 868236238 1 0 0 A, B, C > Example of when it doesn't (salted table): > create table t1 (a int NOT NULL NOT DROPPABLE, b int NOT NULL NOT > DROPPABLE, c int NOT NULL NOT DROPPABLE, primary key (a, b, c)) salt using 4 > partitions on (a, b); > update statistics for table t1 on every key; > showstats for table t1 on existing columns; > Histogram data for Table TRAFODION.BCI.T1 > Table ID: 7427073929238776270 >Hist ID # IntsRowcount UEC Colname(s) > == == === === === > 998895314 1 0 0 C > 998895319 1 0 0 B > 998895324 1 0 0 A > 998895329 1 0 0 "_SALT_" > 998895334 1 0 0 "_SALT_", A, B, C -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-1467) Update statistics does not generate multi-column key stats on a salted table
[ https://issues.apache.org/jira/browse/TRAFODION-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152516#comment-15152516 ] ASF GitHub Bot commented on TRAFODION-1467: --- Github user asfgit closed the pull request at: https://github.com/apache/incubator-trafodion/pull/327 > Update statistics does not generate multi-column key stats on a salted table > > > Key: TRAFODION-1467 > URL: https://issues.apache.org/jira/browse/TRAFODION-1467 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmu >Affects Versions: 1.1 (pre-incubation) >Reporter: Rohit Jain >Assignee: David Wayne Birdsall >Priority: Minor > Labels: performance > > The EVERY KEY or EVERY COLUMN option in update statistics generates > multi-column statistics for all the key columns. It does not do this if the > table is salted, potentially leading to poor plans. > Example of when it works: > create table t1 (a int NOT NULL NOT DROPPABLE, b int NOT NULL NOT DROPPABLE, > c int NOT NULL NOT DROPPABLE, primary key (a, b, c)); > update statistics for table t1 on every key; > showstats for table t1 on existing columns; > Histogram data for Table TRAFODION.BCI.T1 > Table ID: 7427073929238755441 >Hist ID # IntsRowcount UEC Colname(s) > == == === === === > 868236218 1 0 0 C > 868236223 1 0 0 B > 868236228 1 0 0 A > 868236233 1 0 0 A, B > 868236238 1 0 0 A, B, C > Example of when it doesn't (salted table): > create table t1 (a int NOT NULL NOT DROPPABLE, b int NOT NULL NOT > DROPPABLE, c int NOT NULL NOT DROPPABLE, primary key (a, b, c)) salt using 4 > partitions on (a, b); > update statistics for table t1 on every key; > showstats for table t1 on existing columns; > Histogram data for Table TRAFODION.BCI.T1 > Table ID: 7427073929238776270 >Hist ID # IntsRowcount UEC Colname(s) > == == === === === > 998895314 1 0 0 C > 998895319 1 0 0 B > 998895324 1 0 0 A > 998895329 1 0 0 "_SALT_" > 998895334 1 0 0 "_SALT_", A, B, C -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TRAFODION-1834) ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in sqconfig are fully qualified
[ https://issues.apache.org/jira/browse/TRAFODION-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liu ming reassigned TRAFODION-1834: --- Assignee: liu ming > ESP colocation (CQD TRAF_ALLOW_ESP_COLOCATION) not working when node names in > sqconfig are fully qualified > -- > > Key: TRAFODION-1834 > URL: https://issues.apache.org/jira/browse/TRAFODION-1834 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-cmp >Affects Versions: 1.3-incubating >Reporter: Atanu Mishra >Assignee: liu ming > > Paraphrasing Hans, who looked into this briefly: > Looking at the code, I can see that we remove anything from the first dot in > the node name of the region. Then we search for the unqualified node name in > the list of node names of the Trafodion cluster. > Both of these clusters use FQDNs in their configuration, though, therefore we > won't find the node names. > To Reproduce: > Do an EXPLAIN of a parallel query with and without TRAF_ALLOW_ESP_COLOCATION > set, and the node map will not show a co-located plan when the CQD is on - if > the system uses FQDNs in its sqconfig file. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
