[jira] [Created] (TRAFODION-2011) better logging or exception messaging for getScanner issue due to lease timeout
liu ming created TRAFODION-2011: --- Summary: better logging or exception messaging for getScanner issue due to lease timeout Key: TRAFODION-2011 URL: https://issues.apache.org/jira/browse/TRAFODION-2011 Project: Apache Trafodion Issue Type: Improvement Reporter: liu ming When an active transaction is too long, its lease will timeout. And the transaction will be retired internally, however, the client doesn't know and still issue new get/put operations, and will get strange exception, which is hard to understand. It will be better to enhance the error message or logging to help identify the issue. Reproduce: simulate a long transaction, >begin; wait for 2 hours, or change the hbase.transaction.lease.timeout to shorter timeout. >do an update; some error like this, it is confusing: *** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-706). Cause: java.util.concurrent.ExecutionException: java.io.IOException: PerformScan error on coprocessor call, scannerID: 1 java.io.IOException: performScan encountered Exception txID: 25769804282 Exception: org.apache.hadoop.hbase.UnknownScannerException: TrxRegionEndpoint getScanner - scanner id 1, already closed? java.util.concurrent.FutureTask.report(FutureTask.java:122) java.util.concurrent.FutureTask.get(FutureTask.java:188) org.trafodion.sql.HTableClient.fetchRows(HTableClient.java:1251) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-2010) UPDATE/DELETE cannot generate a parallel plan properly
liu ming created TRAFODION-2010: --- Summary: UPDATE/DELETE cannot generate a parallel plan properly Key: TRAFODION-2010 URL: https://issues.apache.org/jira/browse/TRAFODION-2010 Project: Apache Trafodion Issue Type: Bug Reporter: liu ming For big UPDATE/DELETE, that is, affecting a lot of rows, for example, 10M rows to update, Trafodion cannot generate a proper parallel plan by default. So very slow. Reproduce steps: DDL CREATE TABLE T113B ( UNIQ INT NO DEFAULT NOT NULL , C100KINT DEFAULT NULL , C10K INT DEFAULT NULL , C1K INT DEFAULT NULL , C100 INT DEFAULT NULL , C10 INT DEFAULT NULL , C1 INT DEFAULT NULL , C0 INT DEFAULT NULL , Primary key(uniq) ) SALT USING 80 PARTITIONS ATTRIBUTES ALIGNED FORMAT ; Populate data: upsert using load into t113b select 0 + (10 * x10) + (1 * x1) + (1000 * x1000) + (100 * x100) + (10 * x10) +( 1 * x1), 0 + (1 * x1) + (1000 * x1000) + (100 * x100) + (10 * x10) +( 1 * x1), 0 + (1000 * x1000) + (100 * x100) + (10 * x10) + (1 * x1), 0 + (100 * x100) + (10 * x10) + (1 * x1), 0 + (10 * x10) + (1 * x1), 0 + (1 * x1), 0, X0 from (values(0)) t transpose 0,1,2,3,4,5,6,7,8,9 as x10 transpose 0,1,2,3,4,5,6,7,8,9 as x1 transpose 0,1,2,3,4,5,6,7,8,9 as x1000 transpose 0,1,2,3,4,5,6,7,8,9 as x100 transpose 0,1,2,3,4,5,6,7,8,9 as x10 transpose 0,1,2,3,4,5,6,7,8,9 as x1 transpose 0,1,2,3,4,5,6,7,8,9 as x0; The update DML explain options 'f' update t113b set c0 = 0 where c1 = 0; DELETE is similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TRAFODION-2009) parallel scanner failing on tpcds.store_sales table
[ https://issues.apache.org/jira/browse/TRAFODION-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297190#comment-15297190 ] Eric Owhadi commented on TRAFODION-2009: set severity to minor because parallel_scanner is experimental feature disabled by default. > parallel scanner failing on tpcds.store_sales table > --- > > Key: TRAFODION-2009 > URL: https://issues.apache.org/jira/browse/TRAFODION-2009 > Project: Apache Trafodion > Issue Type: Bug > Components: sql-exe >Affects Versions: 2.1-incubating > Environment: on dev workstation >Reporter: Eric Owhadi >Assignee: Eric Owhadi >Priority: Minor > Original Estimate: 24h > Remaining Estimate: 24h > > CREATE TABLE TRAFODION.SEABASE.STORE_SALES > ( > SS_SOLD_DATE_SK INT DEFAULT NULL NOT SERIALIZED > , SS_ITEM_SK INT NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , SS_TICKET_NUMBER INT NO DEFAULT NOT NULL NOT DROPPABLE NOT > SERIALIZED > , SS_SOLD_TIME_SK INT DEFAULT NULL NOT SERIALIZED > , SS_CUSTOMER_SK INT DEFAULT NULL NOT SERIALIZED > , SS_CDEMO_SK INT DEFAULT NULL NOT SERIALIZED > , SS_HDEMO_SK INT DEFAULT NULL NOT SERIALIZED > , SS_ADDR_SK INT DEFAULT NULL NOT SERIALIZED > , SS_STORE_SK INT DEFAULT NULL NOT SERIALIZED > , SS_PROMO_SK INT DEFAULT NULL NOT SERIALIZED > , SS_QUANTITY INT DEFAULT NULL NOT SERIALIZED > , SS_WHOLESALE_COSTREAL DEFAULT NULL NOT SERIALIZED > , SS_LIST_PRICEREAL DEFAULT NULL NOT SERIALIZED > , SS_SALES_PRICE REAL DEFAULT NULL NOT SERIALIZED > , SS_EXT_DISCOUNT_AMT REAL DEFAULT NULL NOT SERIALIZED > , SS_EXT_SALES_PRICE REAL DEFAULT NULL NOT SERIALIZED > , SS_EXT_WHOLESALE_COSTREAL DEFAULT NULL NOT SERIALIZED > , SS_EXT_LIST_PRICEREAL DEFAULT NULL NOT SERIALIZED > , SS_EXT_TAX REAL DEFAULT NULL NOT SERIALIZED > , SS_COUPON_AMTREAL DEFAULT NULL NOT SERIALIZED > , SS_NET_PAID REAL DEFAULT NULL NOT SERIALIZED > , SS_NET_PAID_INC_TAX REAL DEFAULT NULL NOT SERIALIZED > , SS_NET_PROFITREAL DEFAULT NULL NOT SERIALIZED > , PRIMARY KEY (SS_SOLD_DATE_SK ASC, SS_ITEM_SK ASC, SS_TICKET_NUMBER ASC) > ) > SALT USING 8 PARTITIONS >ON (SS_ITEM_SK, SS_TICKET_NUMBER) > ATTRIBUTES ALIGNED FORMAT > HBASE_OPTIONS > ( > DATA_BLOCK_ENCODING = 'FAST_DIFF', > BLOCKSIZE = '131072' > ) > ; > load into store_sales select >SS_SOLD_DATE_SK > , SS_ITEM_SK > , SS_TICKET_NUMBER > , SS_SOLD_TIME_SK > , SS_CUSTOMER_SK > , SS_CDEMO_SK > , SS_HDEMO_SK > , SS_ADDR_SK > , SS_STORE_SK > , SS_PROMO_SK > , SS_QUANTITY > , SS_WHOLESALE_COST > , SS_LIST_PRICE > , SS_SALES_PRICE > , SS_EXT_DISCOUNT_AMT > , SS_EXT_SALES_PRICE > , SS_EXT_WHOLESALE_COST > , SS_EXT_LIST_PRICE > , SS_EXT_TAX > , SS_COUPON_AMT > , SS_NET_PAID > , SS_NET_PAID_INC_TAX > , SS_NET_PROFIT > from hive.hive.store_sales; > set statistics on; > cqd parallel_num_esps '1'; > cqd hbase_dop_parallel_scanner '1.0'; > prepare xx from select count(*) from store_sales where ss_customer_sk between > 1000 and 2; > execute xx; > the result will return wrong count. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-2009) parallel scanner failing on tpcds.store_sales table
Eric Owhadi created TRAFODION-2009: -- Summary: parallel scanner failing on tpcds.store_sales table Key: TRAFODION-2009 URL: https://issues.apache.org/jira/browse/TRAFODION-2009 Project: Apache Trafodion Issue Type: Bug Components: sql-exe Affects Versions: 2.1-incubating Environment: on dev workstation Reporter: Eric Owhadi Assignee: Eric Owhadi Priority: Minor CREATE TABLE TRAFODION.SEABASE.STORE_SALES ( SS_SOLD_DATE_SK INT DEFAULT NULL NOT SERIALIZED , SS_ITEM_SK INT NO DEFAULT NOT NULL NOT DROPPABLE NOT SERIALIZED , SS_TICKET_NUMBER INT NO DEFAULT NOT NULL NOT DROPPABLE NOT SERIALIZED , SS_SOLD_TIME_SK INT DEFAULT NULL NOT SERIALIZED , SS_CUSTOMER_SK INT DEFAULT NULL NOT SERIALIZED , SS_CDEMO_SK INT DEFAULT NULL NOT SERIALIZED , SS_HDEMO_SK INT DEFAULT NULL NOT SERIALIZED , SS_ADDR_SK INT DEFAULT NULL NOT SERIALIZED , SS_STORE_SK INT DEFAULT NULL NOT SERIALIZED , SS_PROMO_SK INT DEFAULT NULL NOT SERIALIZED , SS_QUANTITY INT DEFAULT NULL NOT SERIALIZED , SS_WHOLESALE_COSTREAL DEFAULT NULL NOT SERIALIZED , SS_LIST_PRICEREAL DEFAULT NULL NOT SERIALIZED , SS_SALES_PRICE REAL DEFAULT NULL NOT SERIALIZED , SS_EXT_DISCOUNT_AMT REAL DEFAULT NULL NOT SERIALIZED , SS_EXT_SALES_PRICE REAL DEFAULT NULL NOT SERIALIZED , SS_EXT_WHOLESALE_COSTREAL DEFAULT NULL NOT SERIALIZED , SS_EXT_LIST_PRICEREAL DEFAULT NULL NOT SERIALIZED , SS_EXT_TAX REAL DEFAULT NULL NOT SERIALIZED , SS_COUPON_AMTREAL DEFAULT NULL NOT SERIALIZED , SS_NET_PAID REAL DEFAULT NULL NOT SERIALIZED , SS_NET_PAID_INC_TAX REAL DEFAULT NULL NOT SERIALIZED , SS_NET_PROFITREAL DEFAULT NULL NOT SERIALIZED , PRIMARY KEY (SS_SOLD_DATE_SK ASC, SS_ITEM_SK ASC, SS_TICKET_NUMBER ASC) ) SALT USING 8 PARTITIONS ON (SS_ITEM_SK, SS_TICKET_NUMBER) ATTRIBUTES ALIGNED FORMAT HBASE_OPTIONS ( DATA_BLOCK_ENCODING = 'FAST_DIFF', BLOCKSIZE = '131072' ) ; load into store_sales select SS_SOLD_DATE_SK , SS_ITEM_SK , SS_TICKET_NUMBER , SS_SOLD_TIME_SK , SS_CUSTOMER_SK , SS_CDEMO_SK , SS_HDEMO_SK , SS_ADDR_SK , SS_STORE_SK , SS_PROMO_SK , SS_QUANTITY , SS_WHOLESALE_COST , SS_LIST_PRICE , SS_SALES_PRICE , SS_EXT_DISCOUNT_AMT , SS_EXT_SALES_PRICE , SS_EXT_WHOLESALE_COST , SS_EXT_LIST_PRICE , SS_EXT_TAX , SS_COUPON_AMT , SS_NET_PAID , SS_NET_PAID_INC_TAX , SS_NET_PROFIT from hive.hive.store_sales; set statistics on; cqd parallel_num_esps '1'; cqd hbase_dop_parallel_scanner '1.0'; prepare xx from select count(*) from store_sales where ss_customer_sk between 1000 and 2; execute xx; the result will return wrong count. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-2008) Support for external file LOBs in Trafodion
Sandhya Sundaresan created TRAFODION-2008: - Summary: Support for external file LOBs in Trafodion Key: TRAFODION-2008 URL: https://issues.apache.org/jira/browse/TRAFODION-2008 Project: Apache Trafodion Issue Type: New Feature Components: sql-exe Reporter: Sandhya Sundaresan Fix For: 2.1-incubating Add support for a new lobtype and lob function (externaltolob) that will allow files to reside in HDFS outside o Trafodion. The file is not imported nor part of Trafodion but the file lcoator will be a part of the Trafodion column. Once a column is declared with the "external" attribute, it cannot contain an internal LOB type . All data that's part of that column will have to reside outside Trafodion. Similarly if a column is declared as a regular blob/clb, then an external file cannot be linked to that column using the externaltolob function. Also external file LOBs can be inserted or updated. They cannot be "appended". So each a particular LOB value in a column can be pointing to just one external file . It cannot point ot multiple files so append is not allowed. If an external LOB is updated or modified by the user outside Trafodion,t he user will need to do an "update" to update the information in the Trafodion LOB column. If not, the information in the Trafodion table will be stale. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (TRAFODION-1979) values clause gives incorrect results or crashes mxosrvr when called on the LOB type
[
https://issues.apache.org/jira/browse/TRAFODION-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandhya Sundaresan resolved TRAFODION-1979.
---
Resolution: Fixed
https://github.com/apache/incubator-trafodion/pull/473
> values clause gives incorrect results or crashes mxosrvr when called on the
> LOB type
>
>
> Key: TRAFODION-1979
> URL: https://issues.apache.org/jira/browse/TRAFODION-1979
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-exe
>Affects Versions: 2.1-incubating
>Reporter: Sandhya Sundaresan
>Assignee: Sandhya Sundaresan
> Fix For: 2.1-incubating
>
>
> The following issues were seen :
> When using values() to display lob columns, it returns illegible output as
> shown in the following execution output. Presumably it’s because the
> character set returned by values() has deviated from the original character
> set.
> This was seen on the EntAdv2.0 v1007 build installed on a workstation.
> >>control query default TRAF_BLOB_AS_VARCHAR 'OFF';
> --- SQL operation complete.
> >>control query default TRAF_CLOB_AS_VARCHAR 'OFF';
> --- SQL operation complete.
> >>
> >>create schema mytest16;
> --- SQL operation complete.
> >>set schema mytest16;
> --- SQL operation complete.
> >>
> >>create table t (c1 blob);
> --- SQL operation complete.
> >>insert into table t values (stringtolob('column 1'));
> --- 1 row(s) inserted.
> >>
> >>values ((select * from t));
> C1
>
> !U▒`H(U▒▒E▒`H▒SF▒=▒`H▒"TRAFODION"."MYTEST16"
> --- 1 row(s) selected.
> >>
> >>drop table t;
> --- SQL operation complete.
> >>
> >>drop schema mytest16 cascade;
> --- SQL operation complete.
> >>
> >>exit;
> End of MXCI Session
> with LOB functions, the problems is a little worse and can crash the process:
> >>values (stringtolob('column 1'));
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x7fb2833bfe22, pid=20142, tid=140404765395296
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_67-b01) (build
> 1.7.0_67-b01)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode
> linux-amd64 compressed oops)
> # Problematic frame:
> # C [libcommon.so+0xeee22] str_len(char const*)+0x2
> #
> # Core dump written. Default location:
> /home/trafodion/v0502a/sql/scripts/core or core.20142
> #
> # An error report file with more information is saved as:
> # /home/trafodion/v0502a/sql/scripts/hs_err_pid20142.log
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.sun.com/bugreport/crash.jsp [^]
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> Aborted (core dumped)
> (gdb) bt
> #0 0x7fb2851a5625 in raise () from /lib64/libc.so.6
> 001 0x7fb2851a6d8d in abort () from /lib64/libc.so.6
> 002 0x7fb2871d2a55 in os::abort(bool) ()
>from /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> 003 0x7fb287352f87 in VMError::report_and_die() ()
>from /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> 004 0x7fb2871d796f in JVM_handle_linux_signal ()
>from /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> 005
> 006 0x7fb2833bfe22 in str_len (s=0x0) at ../common/str.cpp:457
> 007 0x7fb2833c1803 in str_sprintf (
> buffer=0x7fffe8749280 "lobDescChunksName
> \300\252t\350\377\177.\"LOBDescHandle__-", '0' , "1_-001\"",
> format=0x7fb284f39d26 "lobDescChunksName %s") at ../common/str.cpp:722
> 008 0x7fb284ecd021 in SQLCLI_LOBcliInterface (
> cliGlobals=, inLobHandle=0x7fffe874d7b0 "\377\377",
> inLobHandleLen=72, blackBox=0x0, blackBoxLen=0x7fffe874d5c8,
> outLobHandle=0x7fb2735e9380 "", outLobHandleLen=0x7fb2735e9780,
> qType=LOB_CLI_INSERT, qPhase=LOB_CLI_ExecImmed, dataOffset=0x7fffe874bea8,
> dataLen=0x7fffe874d580, inoutDescPartnKey=0x7fffe874bea0,
> inoutDescSyskey=0x7fffe874be98, inCliInterface=0x0, xnId=0, lobTrace=0)
> at ../cli/Cli.cpp:9473
> 009 0x7fb284f33448 in SQL_EXEC_LOBcliInterface (
> inLobHandle=0x7fffe874d7b0 "\377\377", inLobHandleLen=72, blackBox=0x0,
> blackBoxLen=0x7fffe874d5c8, outLobHandle=0x7fb2735e9
[jira] [Updated] (TRAFODION-2001) Trafodion Elasticity enhancements
[ https://issues.apache.org/jira/browse/TRAFODION-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Subbiah updated TRAFODION-2001: -- Assignee: Gonzalo E Correa > Trafodion Elasticity enhancements > - > > Key: TRAFODION-2001 > URL: https://issues.apache.org/jira/browse/TRAFODION-2001 > Project: Apache Trafodion > Issue Type: New Feature > Components: documentation, foundation, installer >Affects Versions: 2.1-incubating >Reporter: Gonzalo E Correa >Assignee: Gonzalo E Correa > Attachments: TRAFODION-2001-Elasticity.docx > > Original Estimate: 672h > Remaining Estimate: 672h > > Abstract > This JIRA proposes changes to the Apache Trafodion Incubation configuration > and operational elements used to define and operate a Trafodion cluster in a > Trafodion instance. The changes proposed build on existing functionality for > enhancing Trafodion to support elasticity by implementing the ability to add > and delete nodes used by Trafodion components. > The concept of a cluster is defined in Trafodion through the configuration of > nodes in the ‘sqconfig’ text file, the Trafodion configuration file. This > configuration file is compiled to generate a set of files used to start and > stop a Trafodion instance. The files generated consist of a configuration > database and scripts which presently contain fixed, i.e., hardcoded, > configuration topology attributes of a configured Trafodion instance. This > proposal addresses this hardcoded configuration topology issue by adding new > configuration commands in the Trafodion Foundation’s monitor components, > changing the methodology in the scripts generated, and extending the use of > the configuration database currently in use. > Objectives > The objective of this proposal is to implement elasticity capabilities in > Trafodion to match or exceed the elasticity capabilities of other modules in > the Hadoop eco system. > This focuses on the Trafodion Foundation functionality. Specifically, the > monitor components which use the configuration of nodes that define a > Trafodion cluster are enhanced to provide at set of primitive commands to add > and delete nodes as well as the generation of events to existing processes in > the instance of the addition or deletion of member nodes to the cluster > configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TRAFODION-2007) dsc can't start if use install_local_hadoop to install hadoop
mashengchen created TRAFODION-2007: -- Summary: dsc can't start if use install_local_hadoop to install hadoop Key: TRAFODION-2007 URL: https://issues.apache.org/jira/browse/TRAFODION-2007 Project: Apache Trafodion Issue Type: Bug Reporter: mashengchen Priority: Minor if user use `install_local_hadoop -p rand` to install hadoop env. The script will set zookeeper client port randomly. The 'dcsstart' use default port (2181) to start dcs. so if the port doesn't be 2181, it should change to the random value. the suggested solution is , copy hbase.zookeeper.property.clientPort value which is in $HBASE/conf/hbase-site.xml to dcs.zookeeper.property.clientPort which is in $DCS_DIR/conf/dcs-site.xml. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
