date:20160523

[jira] [Created] (TRAFODION-2010) UPDATE/DELETE cannot generate a parallel plan properly

2016-05-23 Thread liu ming (JIRA)

liu ming created TRAFODION-2010:
---

 Summary: UPDATE/DELETE cannot generate a parallel plan properly
 Key: TRAFODION-2010
 URL: https://issues.apache.org/jira/browse/TRAFODION-2010
 Project: Apache Trafodion
  Issue Type: Bug
Reporter: liu ming


For big UPDATE/DELETE, that is, affecting a lot of rows, for example, 10M rows 
to update, Trafodion cannot generate a proper parallel plan by default. So very 
slow.

Reproduce steps:

DDL
CREATE TABLE T113B
  (
UNIQ INT NO DEFAULT NOT NULL 
  , C100KINT DEFAULT NULL  
  , C10K INT DEFAULT NULL  
  , C1K  INT DEFAULT NULL 
  , C100 INT DEFAULT NULL 
  , C10  INT DEFAULT NULL  
  , C1   INT DEFAULT NULL  
  , C0   INT DEFAULT NULL ,
  Primary key(uniq)
  )
  SALT USING 80 PARTITIONS
ATTRIBUTES ALIGNED FORMAT
;
 
Populate data:
upsert using load into t113b select
0 + (10 * x10) + (1 * x1) + (1000 * x1000) +
(100 * x100) + (10 * x10) +( 1 * x1),
0 + (1 * x1) + (1000 * x1000) + (100 * x100) +
(10 * x10) +( 1 * x1),
0 + (1000 * x1000) + (100 * x100) + (10 * x10) + (1 * x1),
0 + (100 * x100) + (10 * x10) + (1 * x1),
0 + (10 * x10) + (1 * x1),
0 + (1 * x1),
0,
X0
from (values(0)) t
transpose 0,1,2,3,4,5,6,7,8,9 as x10
transpose 0,1,2,3,4,5,6,7,8,9 as x1
transpose 0,1,2,3,4,5,6,7,8,9 as x1000
transpose 0,1,2,3,4,5,6,7,8,9 as x100
transpose 0,1,2,3,4,5,6,7,8,9 as x10
transpose 0,1,2,3,4,5,6,7,8,9 as x1
transpose 0,1,2,3,4,5,6,7,8,9 as x0;
 
The update DML
explain options 'f' update t113b set c0 = 0 where c1 = 0;


DELETE is similar.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-2009) parallel scanner failing on tpcds.store_sales table

2016-05-23 Thread Eric Owhadi (JIRA)


[ 
https://issues.apache.org/jira/browse/TRAFODION-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297190#comment-15297190
 ] 

Eric Owhadi commented on TRAFODION-2009:


set severity to minor because parallel_scanner is experimental feature disabled 
by default.

> parallel scanner failing on tpcds.store_sales table
> ---
>
> Key: TRAFODION-2009
> URL: https://issues.apache.org/jira/browse/TRAFODION-2009
> Project: Apache Trafodion
>  Issue Type: Bug
>  Components: sql-exe
>Affects Versions: 2.1-incubating
> Environment: on dev workstation
>Reporter: Eric Owhadi
>Assignee: Eric Owhadi
>Priority: Minor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> CREATE TABLE TRAFODION.SEABASE.STORE_SALES
>   (
> SS_SOLD_DATE_SK  INT DEFAULT NULL NOT SERIALIZED
>   , SS_ITEM_SK   INT NO DEFAULT NOT NULL NOT DROPPABLE NOT
>   SERIALIZED
>   , SS_TICKET_NUMBER INT NO DEFAULT NOT NULL NOT DROPPABLE NOT
>   SERIALIZED
>   , SS_SOLD_TIME_SK  INT DEFAULT NULL NOT SERIALIZED
>   , SS_CUSTOMER_SK   INT DEFAULT NULL NOT SERIALIZED
>   , SS_CDEMO_SK  INT DEFAULT NULL NOT SERIALIZED
>   , SS_HDEMO_SK  INT DEFAULT NULL NOT SERIALIZED
>   , SS_ADDR_SK   INT DEFAULT NULL NOT SERIALIZED
>   , SS_STORE_SK  INT DEFAULT NULL NOT SERIALIZED
>   , SS_PROMO_SK  INT DEFAULT NULL NOT SERIALIZED
>   , SS_QUANTITY  INT DEFAULT NULL NOT SERIALIZED
>   , SS_WHOLESALE_COSTREAL DEFAULT NULL NOT SERIALIZED
>   , SS_LIST_PRICEREAL DEFAULT NULL NOT SERIALIZED
>   , SS_SALES_PRICE   REAL DEFAULT NULL NOT SERIALIZED
>   , SS_EXT_DISCOUNT_AMT  REAL DEFAULT NULL NOT SERIALIZED
>   , SS_EXT_SALES_PRICE   REAL DEFAULT NULL NOT SERIALIZED
>   , SS_EXT_WHOLESALE_COSTREAL DEFAULT NULL NOT SERIALIZED
>   , SS_EXT_LIST_PRICEREAL DEFAULT NULL NOT SERIALIZED
>   , SS_EXT_TAX   REAL DEFAULT NULL NOT SERIALIZED
>   , SS_COUPON_AMTREAL DEFAULT NULL NOT SERIALIZED
>   , SS_NET_PAID  REAL DEFAULT NULL NOT SERIALIZED
>   , SS_NET_PAID_INC_TAX  REAL DEFAULT NULL NOT SERIALIZED
>   , SS_NET_PROFITREAL DEFAULT NULL NOT SERIALIZED
>   , PRIMARY KEY (SS_SOLD_DATE_SK ASC, SS_ITEM_SK ASC, SS_TICKET_NUMBER ASC)
>   )
>   SALT USING 8 PARTITIONS
>ON (SS_ITEM_SK, SS_TICKET_NUMBER)
>  ATTRIBUTES ALIGNED FORMAT
>   HBASE_OPTIONS
>   (
> DATA_BLOCK_ENCODING = 'FAST_DIFF',
> BLOCKSIZE = '131072'
>   )
> ;
> load into store_sales select
>SS_SOLD_DATE_SK 
>   , SS_ITEM_SK   
>   , SS_TICKET_NUMBER
>   , SS_SOLD_TIME_SK
>   , SS_CUSTOMER_SK 
>   , SS_CDEMO_SK  
>   , SS_HDEMO_SK
>   , SS_ADDR_SK
>   , SS_STORE_SK   
>   , SS_PROMO_SK  
>   , SS_QUANTITY 
>   , SS_WHOLESALE_COST  
>   , SS_LIST_PRICE   
>   , SS_SALES_PRICE  
>   , SS_EXT_DISCOUNT_AMT  
>   , SS_EXT_SALES_PRICE  
>   , SS_EXT_WHOLESALE_COST   
>   , SS_EXT_LIST_PRICE
>   , SS_EXT_TAX   
>   , SS_COUPON_AMT
>   , SS_NET_PAID  
>   , SS_NET_PAID_INC_TAX  
>   , SS_NET_PROFIT 
> from hive.hive.store_sales;
> set statistics on;
> cqd parallel_num_esps '1';
> cqd hbase_dop_parallel_scanner '1.0';
> prepare xx from select count(*) from store_sales where ss_customer_sk between 
> 1000 and 2;
> execute xx;
> the result will return wrong count.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (TRAFODION-2009) parallel scanner failing on tpcds.store_sales table

2016-05-23 Thread Eric Owhadi (JIRA)

Eric Owhadi created TRAFODION-2009:
--

 Summary: parallel scanner failing on tpcds.store_sales table
 Key: TRAFODION-2009
 URL: https://issues.apache.org/jira/browse/TRAFODION-2009
 Project: Apache Trafodion
  Issue Type: Bug
  Components: sql-exe
Affects Versions: 2.1-incubating
 Environment: on dev workstation
Reporter: Eric Owhadi
Assignee: Eric Owhadi
Priority: Minor


CREATE TABLE TRAFODION.SEABASE.STORE_SALES
  (
SS_SOLD_DATE_SK  INT DEFAULT NULL NOT SERIALIZED
  , SS_ITEM_SK   INT NO DEFAULT NOT NULL NOT DROPPABLE NOT
  SERIALIZED
  , SS_TICKET_NUMBER INT NO DEFAULT NOT NULL NOT DROPPABLE NOT
  SERIALIZED
  , SS_SOLD_TIME_SK  INT DEFAULT NULL NOT SERIALIZED
  , SS_CUSTOMER_SK   INT DEFAULT NULL NOT SERIALIZED
  , SS_CDEMO_SK  INT DEFAULT NULL NOT SERIALIZED
  , SS_HDEMO_SK  INT DEFAULT NULL NOT SERIALIZED
  , SS_ADDR_SK   INT DEFAULT NULL NOT SERIALIZED
  , SS_STORE_SK  INT DEFAULT NULL NOT SERIALIZED
  , SS_PROMO_SK  INT DEFAULT NULL NOT SERIALIZED
  , SS_QUANTITY  INT DEFAULT NULL NOT SERIALIZED
  , SS_WHOLESALE_COSTREAL DEFAULT NULL NOT SERIALIZED
  , SS_LIST_PRICEREAL DEFAULT NULL NOT SERIALIZED
  , SS_SALES_PRICE   REAL DEFAULT NULL NOT SERIALIZED
  , SS_EXT_DISCOUNT_AMT  REAL DEFAULT NULL NOT SERIALIZED
  , SS_EXT_SALES_PRICE   REAL DEFAULT NULL NOT SERIALIZED
  , SS_EXT_WHOLESALE_COSTREAL DEFAULT NULL NOT SERIALIZED
  , SS_EXT_LIST_PRICEREAL DEFAULT NULL NOT SERIALIZED
  , SS_EXT_TAX   REAL DEFAULT NULL NOT SERIALIZED
  , SS_COUPON_AMTREAL DEFAULT NULL NOT SERIALIZED
  , SS_NET_PAID  REAL DEFAULT NULL NOT SERIALIZED
  , SS_NET_PAID_INC_TAX  REAL DEFAULT NULL NOT SERIALIZED
  , SS_NET_PROFITREAL DEFAULT NULL NOT SERIALIZED
  , PRIMARY KEY (SS_SOLD_DATE_SK ASC, SS_ITEM_SK ASC, SS_TICKET_NUMBER ASC)
  )
  SALT USING 8 PARTITIONS
   ON (SS_ITEM_SK, SS_TICKET_NUMBER)
 ATTRIBUTES ALIGNED FORMAT
  HBASE_OPTIONS
  (
DATA_BLOCK_ENCODING = 'FAST_DIFF',
BLOCKSIZE = '131072'
  )
;

load into store_sales select
   SS_SOLD_DATE_SK 
  , SS_ITEM_SK   
  , SS_TICKET_NUMBER
  , SS_SOLD_TIME_SK
  , SS_CUSTOMER_SK 
  , SS_CDEMO_SK  
  , SS_HDEMO_SK
  , SS_ADDR_SK
  , SS_STORE_SK   
  , SS_PROMO_SK  
  , SS_QUANTITY 
  , SS_WHOLESALE_COST  
  , SS_LIST_PRICE   
  , SS_SALES_PRICE  
  , SS_EXT_DISCOUNT_AMT  
  , SS_EXT_SALES_PRICE  
  , SS_EXT_WHOLESALE_COST   
  , SS_EXT_LIST_PRICE
  , SS_EXT_TAX   
  , SS_COUPON_AMT
  , SS_NET_PAID  
  , SS_NET_PAID_INC_TAX  
  , SS_NET_PROFIT 
from hive.hive.store_sales;

set statistics on;
cqd parallel_num_esps '1';
cqd hbase_dop_parallel_scanner '1.0';
prepare xx from select count(*) from store_sales where ss_customer_sk between 
1000 and 2;
execute xx;

the result will return wrong count.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (TRAFODION-2001) Trafodion Elasticity enhancements

2016-05-23 Thread Suresh Subbiah (JIRA)


 [ 
https://issues.apache.org/jira/browse/TRAFODION-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suresh Subbiah updated TRAFODION-2001:
--
Assignee: Gonzalo E Correa

> Trafodion Elasticity enhancements
> -
>
> Key: TRAFODION-2001
> URL: https://issues.apache.org/jira/browse/TRAFODION-2001
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: documentation, foundation, installer
>Affects Versions: 2.1-incubating
>Reporter: Gonzalo E Correa
>Assignee: Gonzalo E Correa
> Attachments: TRAFODION-2001-Elasticity.docx
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> Abstract
> This JIRA proposes changes to the Apache Trafodion Incubation configuration 
> and operational elements used to define and operate a Trafodion cluster in a 
> Trafodion instance. The changes proposed build on existing functionality for 
> enhancing Trafodion to support elasticity by implementing the ability to add 
> and delete nodes used by Trafodion components.
> The concept of a cluster is defined in Trafodion through the configuration of 
> nodes in the ‘sqconfig’ text file, the Trafodion configuration file. This 
> configuration file is compiled to generate a set of files used to start and 
> stop a Trafodion instance. The files generated consist of a configuration 
> database and scripts which presently contain fixed, i.e., hardcoded, 
> configuration topology attributes of a configured Trafodion instance. This 
> proposal addresses this hardcoded configuration topology issue by adding new 
> configuration commands in the Trafodion Foundation’s monitor components, 
> changing the methodology in the scripts generated, and extending the use of 
> the configuration database currently in use. 
> Objectives
> The objective of this proposal is to implement elasticity capabilities in 
> Trafodion to match or exceed the elasticity capabilities of other modules in 
> the Hadoop eco system.
> This focuses on the Trafodion Foundation functionality. Specifically, the 
> monitor components which use the configuration of nodes that define a 
> Trafodion cluster are enhanced to provide at set of primitive commands to add 
> and delete nodes as well as the generation of events to existing processes in 
> the instance of the addition or deletion of member nodes to the cluster 
> configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (TRAFODION-2007) dsc can't start if use install_local_hadoop to install hadoop

2016-05-23 Thread mashengchen (JIRA)

mashengchen created TRAFODION-2007:
--

 Summary: dsc can't start if use install_local_hadoop to install 
hadoop
 Key: TRAFODION-2007
 URL: https://issues.apache.org/jira/browse/TRAFODION-2007
 Project: Apache Trafodion
  Issue Type: Bug
Reporter: mashengchen
Priority: Minor


if user use `install_local_hadoop -p rand` to install hadoop env. The script 
will set zookeeper client port randomly. The 'dcsstart' use default port (2181) 
to start dcs. so if the port doesn't be 2181, it should change to the random 
value.
the suggested solution is , copy hbase.zookeeper.property.clientPort value 
which is in $HBASE/conf/hbase-site.xml to dcs.zookeeper.property.clientPort 
which is in $DCS_DIR/conf/dcs-site.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1931) ODB tool support windows platform

2016-05-23 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/TRAFODION-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295993#comment-15295993
 ] 

ASF GitHub Bot commented on TRAFODION-1931:
---

GitHub user xwq opened a pull request:

https://github.com/apache/incubator-trafodion/pull/498

[TRAFODION-1931]add a script to build odb for windows

Add a script to build odb for windows. The target will be copied to
C:\Build\odb. User can modify the PACKDIR to other place if needed .

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xwq/incubator-trafodion master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-trafodion/pull/498.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #498


commit cf3418a62c3e9ed5d8f1898e8d1ab8ac3b1d5dcf
Author: Weiqing Xu 
Date:   2016-05-20T08:50:22Z

[TRAFODION-1931]add a script to build odb for windows

Add a script to build odb for windows. The target will be copied to
C:\Build\odb. User can modify the PACKDIR to other place if needed .

commit 8bc4f72e85f6bdd774bd90130fa1d2901df40546
Author: Weiqing Xu 
Date:   2016-05-23T06:44:39Z

update the VS project file




> ODB tool support windows platform
> -
>
> Key: TRAFODION-1931
> URL: https://issues.apache.org/jira/browse/TRAFODION-1931
> Project: Apache Trafodion
>  Issue Type: Improvement
>  Components: db-utility-odb
>Reporter: Weiqing Xu
>Assignee: Weiqing Xu
>
> ODB need to be compiled and run in Windows platform. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1931) ODB tool support windows platform

2016-05-23 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/TRAFODION-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295988#comment-15295988
 ] 

ASF GitHub Bot commented on TRAFODION-1931:
---

Github user xwq closed the pull request at:

https://github.com/apache/incubator-trafodion/pull/497


> ODB tool support windows platform
> -
>
> Key: TRAFODION-1931
> URL: https://issues.apache.org/jira/browse/TRAFODION-1931
> Project: Apache Trafodion
>  Issue Type: Improvement
>  Components: db-utility-odb
>Reporter: Weiqing Xu
>Assignee: Weiqing Xu
>
> ODB need to be compiled and run in Windows platform. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TRAFODION-1931) ODB tool support windows platform

2016-05-23 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/TRAFODION-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295987#comment-15295987
 ] 

ASF GitHub Bot commented on TRAFODION-1931:
---

GitHub user xwq opened a pull request:

https://github.com/apache/incubator-trafodion/pull/497

[TRAFODION-1931]add a script to build odb for windows

Add a script to build odb for windows. The target will be copied to
C:\Build\odb. User can modify the PACKDIR to other place if needed .

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xwq/incubator-trafodion master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-trafodion/pull/497.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #497


commit cf3418a62c3e9ed5d8f1898e8d1ab8ac3b1d5dcf
Author: Weiqing Xu 
Date:   2016-05-20T08:50:22Z

[TRAFODION-1931]add a script to build odb for windows

Add a script to build odb for windows. The target will be copied to
C:\Build\odb. User can modify the PACKDIR to other place if needed .

commit f32fcf5cc66730cfdb2d3e25219a954a3b058e94
Author: Weiqing Xu 
Date:   2016-05-23T06:40:17Z

update the VS project file




> ODB tool support windows platform
> -
>
> Key: TRAFODION-1931
> URL: https://issues.apache.org/jira/browse/TRAFODION-1931
> Project: Apache Trafodion
>  Issue Type: Improvement
>  Components: db-utility-odb
>Reporter: Weiqing Xu
>Assignee: Weiqing Xu
>
> ODB need to be compiled and run in Windows platform. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (TRAFODION-2010) UPDATE/DELETE cannot generate a parallel plan properly

[jira] [Commented] (TRAFODION-2009) parallel scanner failing on tpcds.store_sales table

[jira] [Created] (TRAFODION-2009) parallel scanner failing on tpcds.store_sales table

[jira] [Updated] (TRAFODION-2001) Trafodion Elasticity enhancements

[jira] [Created] (TRAFODION-2007) dsc can't start if use install_local_hadoop to install hadoop

[jira] [Commented] (TRAFODION-1931) ODB tool support windows platform

[jira] [Commented] (TRAFODION-1931) ODB tool support windows platform

[jira] [Commented] (TRAFODION-1931) ODB tool support windows platform

8 matches

Site Navigation

Mail list logo

Footer information