[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-18 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383431#comment-15383431
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-trafodion/pull/577


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-18 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382720#comment-15382720
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71197478
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -6953,13 +6968,22 @@ del_stmt_w_acc_type_rtn_list_and_as_clause_col_list 
: '('  delete_statement acce
 
 table_name_as_clause_and_hint : table_name as_clause optimizer_hint 
hbase_access_options
 {
-  $1->setCorrName(*$2);
-  $$ = new (PARSERHEAP()) Scan(*$1);
-  if ($3) 
-$$->setHint($3);
-  if ($4)
-((Scan*)$$)->setHbaseAccessOptions($4);
-   
+   NAString tmp = 
((*$1).getQualifiedNameAsString());
+   
if(SqlParser_CurrentParser->hasWithDefinition(&tmp) )
+   { 
+ RelExpr *re = 
SqlParser_CurrentParser->getWithDefinition(&tmp);
+ RenameTable *rt = new (PARSERHEAP()) 
RenameTable(re, *$2);
+ $$=rt->copyTree(PARSERHEAP());
--- End diff --

Same as above, could give an error if we see a hint or an HBase access 
option.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-18 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382718#comment-15382718
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71197264
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -6647,15 +6651,26 @@ table_as_tmudf_function : TOK_UDF '(' 
table_mapping_function_invocation ')'
 
 table_name_and_hint : table_name optimizer_hint hbase_access_options
   {
-$$ = new (PARSERHEAP()) Scan(*$1);
-if ($2) 
-  $$->setHint($2);
+NAString tmp = ((*$1).getQualifiedNameAsString());
+
if(SqlParser_CurrentParser->hasWithDefinition(&tmp) )
+{
+  RelExpr *re = 
SqlParser_CurrentParser->getWithDefinition(&tmp);
+  $$=re->copyTree(PARSERHEAP());
+  delete $1;
--- End diff --

One thing we could do here is to raise an error if we have an optimizer 
hint or HBase access options at this point. Those are allowed in the grammar, 
but we wouldn't be able to apply them to a more complicated WITH clause.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-17 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381367#comment-15381367
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71083286
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

sorry, I made mistake, by running TESTOK2, it shows the change still 
introduce one more reduce/reduce conflict. Still looking at it.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-15 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380400#comment-15380400
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71058116
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -7000,6 +7024,46 @@ rel_subquery_and_as_clause : rel_subquery as_clause
CheckModeSpecial1();
$$ = $2;
  }
+with_clause : 
+  TOK_WITH with_clause_elements
+  {
+ $$ = NULL;
+  }
+  | TOK_WITH TOK_RECURSIVE with_clause_elements
+  {
+*SqlParser_Diags << DgSqlCode(-3022) << 
DgString0("WITH RECURSIVE");
+YYERROR;
+  }
+
+with_clause_elements : with_clause_element
+  {
+ $$ = NULL;
+  }
+  | with_clause_elements ',' with_clause_element
+  {
+ $$ = NULL;
+  }
+
+with_clause_element : correlation_name TOK_AS '(' query_expression ')'
+  {
+ RelRoot *root = new (PARSERHEAP())
+RelRoot($4, REL_ROOT);
+ $$= new (PARSERHEAP()) RenameTable(root, *$1); 
+
+ //Duplicated definition of WITH
+ if(SqlParser_CurrentParser->hasWithDefinition($1) 
)
+  {
+*SqlParser_Diags << DgSqlCode(-3288)
+ << 
DgString0((*$1).toCharStar());
+ delete $1;
+ delete $4;
+ YYERROR;
+  }
+
+  SqlParser_CurrentParser->insertWithDefinition($1 
, $$);
+  delete $1;
+  delete $4;
--- End diff --

I agree, will change it.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-15 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380398#comment-15380398
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71058095
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -1,14 +13396,41 @@ query_specification : select_token set_quantifier 
query_spec_body
   ColReference(new (PARSERHEAP()) 
ColRefName(TRUE, PARSERHEAP()))
 )
  );
-  assert($3->getOperatorType() == REL_ROOT);
+   assert($3->getOperatorType() == REL_ROOT);
+   RelRoot *root1 = (RelRoot *) $$;
+   RelRoot *root2 = (RelRoot *) $3;
+   root1->assignmentStTree() = root2->assignmentStTree();
+   root2->assignmentStTree() = NULL;
+   }
+   else
+ $$ = $3;
+
+   if (CmpCommon::getDefault(MVQR_LOG_QUERY_DESCRIPTORS) == DF_DUMP ||
+   CmpCommon::getDefault(MVQR_LOG_QUERY_DESCRIPTORS) == DF_DUMP_MV)
+ ((RelRoot*)$$)->setAnalyzeOnly();
+
+}
+
+/* type relx */
+query_specification :with_clause select_token set_quantifier 
query_spec_body 
+ {
+   if ($3) {
+ $$ = new (PARSERHEAP())
+   RelRoot(new (PARSERHEAP())
+   GroupByAgg($4
--- End diff --

if ($3) means there is a set_quantifier token, which is DISTINCT for 
example, so need a GroupByAgg node here.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-15 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380394#comment-15380394
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71057841
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -13418,6 +13507,55 @@ query_specification : select_token '[' 
firstn_sorted NUMERIC_LITERAL_EXACT_NO_SC
   if (CmpCommon::getDefault(MVQR_LOG_QUERY_DESCRIPTORS) == DF_DUMP 
||
   CmpCommon::getDefault(MVQR_LOG_QUERY_DESCRIPTORS) == 
DF_DUMP_MV)
 ((RelRoot*)$$)->setAnalyzeOnly();
+   } 
+/* type relx */
+query_specification : with_clause select_token '[' firstn_sorted 
NUMERIC_LITERAL_EXACT_NO_SCALE ']' set_quantifier query_spec_body
+   {
+ if ($7) {
+   $$ = new (PARSERHEAP())
+ RelRoot(new (PARSERHEAP())
+ GroupByAgg($8
--- End diff --

if($7) means there is a set_quantifier like (DISTINCT), so it will add a 
GroupByAgg node here.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-15 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379745#comment-15379745
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71010941
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -7000,6 +7024,46 @@ rel_subquery_and_as_clause : rel_subquery as_clause
CheckModeSpecial1();
$$ = $2;
  }
+with_clause : 
+  TOK_WITH with_clause_elements
+  {
+ $$ = NULL;
+  }
+  | TOK_WITH TOK_RECURSIVE with_clause_elements
+  {
+*SqlParser_Diags << DgSqlCode(-3022) << 
DgString0("WITH RECURSIVE");
+YYERROR;
+  }
+
+with_clause_elements : with_clause_element
+  {
+ $$ = NULL;
+  }
+  | with_clause_elements ',' with_clause_element
+  {
+ $$ = NULL;
+  }
+
+with_clause_element : correlation_name TOK_AS '(' query_expression ')'
+  {
+ RelRoot *root = new (PARSERHEAP())
+RelRoot($4, REL_ROOT);
+ $$= new (PARSERHEAP()) RenameTable(root, *$1); 
+
+ //Duplicated definition of WITH
+ if(SqlParser_CurrentParser->hasWithDefinition($1) 
)
+  {
+*SqlParser_Diags << DgSqlCode(-3288)
+ << 
DgString0((*$1).toCharStar());
+ delete $1;
+ delete $4;
+ YYERROR;
+  }
+
+  SqlParser_CurrentParser->insertWithDefinition($1 
, $$);
+  delete $1;
+  delete $4;
--- End diff --

I don't think we want to delete $1 and $4 here, since the RelRoot node 
points to $4 and the RenameTable node points to $1.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-15 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379735#comment-15379735
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71010585
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -13418,6 +13507,55 @@ query_specification : select_token '[' 
firstn_sorted NUMERIC_LITERAL_EXACT_NO_SC
   if (CmpCommon::getDefault(MVQR_LOG_QUERY_DESCRIPTORS) == DF_DUMP 
||
   CmpCommon::getDefault(MVQR_LOG_QUERY_DESCRIPTORS) == 
DF_DUMP_MV)
 ((RelRoot*)$$)->setAnalyzeOnly();
+   } 
+/* type relx */
+query_specification : with_clause select_token '[' firstn_sorted 
NUMERIC_LITERAL_EXACT_NO_SCALE ']' set_quantifier query_spec_body
+   {
+ if ($7) {
+   $$ = new (PARSERHEAP())
+ RelRoot(new (PARSERHEAP())
+ GroupByAgg($8
--- End diff --

I don't understand why we add a GroupByAgg node here.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-15 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379733#comment-15379733
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r71010430
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -1,14 +13396,41 @@ query_specification : select_token set_quantifier 
query_spec_body
   ColReference(new (PARSERHEAP()) 
ColRefName(TRUE, PARSERHEAP()))
 )
  );
-  assert($3->getOperatorType() == REL_ROOT);
+   assert($3->getOperatorType() == REL_ROOT);
+   RelRoot *root1 = (RelRoot *) $$;
+   RelRoot *root2 = (RelRoot *) $3;
+   root1->assignmentStTree() = root2->assignmentStTree();
+   root2->assignmentStTree() = NULL;
+   }
+   else
+ $$ = $3;
+
+   if (CmpCommon::getDefault(MVQR_LOG_QUERY_DESCRIPTORS) == DF_DUMP ||
+   CmpCommon::getDefault(MVQR_LOG_QUERY_DESCRIPTORS) == DF_DUMP_MV)
+ ((RelRoot*)$$)->setAnalyzeOnly();
+
+}
+
+/* type relx */
+query_specification :with_clause select_token set_quantifier 
query_spec_body 
+ {
+   if ($3) {
+ $$ = new (PARSERHEAP())
+   RelRoot(new (PARSERHEAP())
+   GroupByAgg($4
--- End diff --

I don't understand why we stick a GroupByAgg node here.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-14 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378533#comment-15378533
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70897000
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

Thanks, that's excellent news!


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376435#comment-15376435
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70756198
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -6647,6 +6652,26 @@ table_as_tmudf_function : TOK_UDF '(' 
table_mapping_function_invocation ')'
 
 table_name_and_hint : table_name optimizer_hint hbase_access_options
   {
+NAString tmp = ((*$1).getQualifiedNameAsString());
+
if(SqlParser_CurrentParser->with_clauses_->contains(&tmp) )
+{
+  RelExpr *re = 
SqlParser_CurrentParser->with_clauses_->getFirstValue(&tmp);
+  $$=re->copyTree(PARSERHEAP());
--- End diff --

yes, I am not quite sure how memory was managed in the parser, I  changed 
to delete $1


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376429#comment-15376429
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70755998
  
--- Diff: core/sql/sqlcomp/parser.h ---
@@ -261,6 +261,8 @@ ItemExpr *get_w_ItemExprTree(const NAWchar * str,
   NABoolean isHQCCacheable()
 { return HQCKey_?HQCKey_->isCacheable():FALSE;  }
 
+  NAHashDictionary *with_clauses_;
--- End diff --

yes, changed


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376434#comment-15376434
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70756042
  
--- Diff: core/sql/bin/SqlciErrors.txt ---
@@ -410,6 +410,7 @@
 1430 3F000 9 BEGINNER MAJOR DBADMIN A schema name that starts and ends 
with an "_"(underscore) is reserved for internal usage. It cannot be used to 
create a user schema.
 1431 Z 9 BEGINNER MINOR DBADMIN Object $0~String0 exists in HBase. 
This could be due to a concurrent transactional ddl operation in progress on 
this table.
 1432 Z 9 BEGINNER MINOR DBADMIN Input LOB type $0~Int0 does not 
match column's storage type: $1~Int1 Column name: $0~String0 .
+1433 Z 9 BEGINNER MINOR DBADMIN WITH clause redefined. WITH name 
$0~String0 .
--- End diff --

yes, changed to 3288


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376157#comment-15376157
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70736959
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

I found a way to avoid the reduce/shift and reduce/reduce error by add 
with_clause down to the underlying rules of non_join_query_expression, instead 
of add it at same level of non_join_query_expression. non_join_query_expression 
can represent many different styles of query expression, including utilities, 
which we don't want a with_clause as prefix, so in the underlying rules, only 
add with_clause before those SELECT syntax, and it solve the problem.

And it seems if use redundant rule explicitly (looks annoying) , we can 
avoid the error. 
For example:
  A : b  | c
is better to change to 
  A : b 
  A : c
I am still trying to understand this.

I checked with PostgreSQL parser, they are doing similar things, comments 
from postgresql parser:

 * This rule parses the equivalent of the standard's .
 * The duplicative productions are annoying, but hard to get rid of without
 * creating shift/reduce conflicts.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15375270#comment-15375270
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70656018
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

Yes, I got a similar result. I tried this:

query_expression : with_clause non_join_query_expression
 | non_join_query_expression

That gives me three additional reduce/reduce conflicts which I don't 
understand. Here are two of them:

State 1941

1870 query_expression: with_clause non_join_query_expression .
1871 | non_join_query_expression .

  TOK_EXCEPT  reduce using rule 1870 (query_expression)
  TOK_EXCEPT  [reduce using rule 1871 (query_expression)]
  TOK_UNION   reduce using rule 1870 (query_expression)
  TOK_UNION   [reduce using rule 1871 (query_expression)]
  $defaultreduce using rule 1870 (query_expression)


I also tried this:

query_expression : optional_with_clause non_join_query_expression

That doesn't cause additional reduce/reduce conflicts, but we get a very 
large number of additional shift/reduce conflicts, something in the 
neighborhood of 50-100.

If we can't get the "correct" parser rules to work, then maybe we can stick 
with the approach you chose (assuming it accepts all the valid WITH clauses) 
and handle any illegal syntax in the semantic actions.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-13 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374956#comment-15374956
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70620967
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

hi, Hans, Dave,
the current change doesn't get higher number of conflicts, it is still 
73/12, but if I tried to move the rule into query_expression, it will increase 
the shift/reduce conflicts, I still cannot find a good way to write a correct 
syntax.  It is rather difficult to find the root cause of shift/reduce conflict


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-12 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373646#comment-15373646
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70517961
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -6579,6 +6581,9 @@ table_reference : table_name_and_hint
   | rel_subquery_and_as_clause
 { $$ = $1; }
 
+  | with_clause_list
--- End diff --

I agree with Hans; it would be better to move the wish_clause non-terminal 
to the query_expression production. Right now, that production reads:

query_expression : non_join_query_expression

You could change it to:

query_expression: with_clause non_join_query_expression
| non_join_query_expression,

or alternatively,

query_expression : with_clause non_join_query_expression

And then code with_clause to allow an empty production, for example:

with_clause : { $$ = NULL; /* empty with_clause case */ }
|  TOK_WITH with_clause_elements


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-12 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373629#comment-15373629
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70514345
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -6579,6 +6581,9 @@ table_reference : table_name_and_hint
   | rel_subquery_and_as_clause
 { $$ = $1; }
 
+  | with_clause_list
--- End diff --

I think this is incorrect. Making "with_clause_list" an alternative for 
table_reference would allow things like: "select * from with c1 as (select * 
from t)".


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-12 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373528#comment-15373528
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70505167
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -7001,6 +7036,46 @@ rel_subquery_and_as_clause : rel_subquery as_clause
$$ = $2;
  }
 
+with_clause_list : with_clause
+ {  $$ = $1 ; }
+
+   | with_clause_list ',' correlation_name TOK_AS 
rel_subquery 
--- End diff --

One way would be:

with_clause : WITH with_clause_elements

with_clause_elements : with_clause element
  |
 with_clause_elements 
with_clause_element

with_clause_element : correlation_name TOK_AS rel_subquery

By the way, I notice the way this is coded, the "with_clause" non-terminal 
is a single correlation_name TOK_AS rel_subquery element. It seems more natural 
(to me anyway) to think of the with_clause as being everything from the WITH 
keyword through the entire list of subqueries.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-12 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373507#comment-15373507
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user DaveBirdsall commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70504097
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -6647,6 +6652,26 @@ table_as_tmudf_function : TOK_UDF '(' 
table_mapping_function_invocation ')'
 
 table_name_and_hint : table_name optimizer_hint hbase_access_options
   {
+NAString tmp = ((*$1).getQualifiedNameAsString());
+
if(SqlParser_CurrentParser->with_clauses_->contains(&tmp) )
+{
+  RelExpr *re = 
SqlParser_CurrentParser->with_clauses_->getFirstValue(&tmp);
+  $$=re->copyTree(PARSERHEAP());
--- End diff --

Should we delete $1 in this code path as well?


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-12 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372910#comment-15372910
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r70441662
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -7001,6 +7036,46 @@ rel_subquery_and_as_clause : rel_subquery as_clause
$$ = $2;
  }
 
+with_clause_list : with_clause
+ {  $$ = $1 ; }
+
+   | with_clause_list ',' correlation_name TOK_AS 
rel_subquery 
--- End diff --

@zellerh , I cannot figure out how to do this. Could you please give me an 
example? I am very new to the parser.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366302#comment-15366302
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69932659
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

That is the number of conflicts encountered by the bison parser when it 
processes file core/sql/parser/sqlparser.y. Here is how I get this information:

```
touch core/sql/parser/sqlparser.y
cd core/sql/nskgmake
gmake -j 4 -ks linuxdebug
```

Here is the current output:

```
../parser/sqlparser.y: warning: 73 shift/reduce conflicts [-Wconflicts-sr]
../parser/sqlparser.y: warning: 12 reduce/reduce conflicts [-Wconflicts-rr]
```

If you get a different (higher) number, then we would have to investigate 
whether there are any potential problems with the new syntax.



> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread Anoop Sharma (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365622#comment-15365622
 ] 

Anoop Sharma commented on TRAFODION-1673:
-

when a make is done, the parser component shows shift/reduce and reduce/reduce 
conflicts.
Something like this will be part of make output:
  ../parser/sqlparser.y: warning: 73 shift/reduce conflicts [-Wconflicts-sr]
  ../parser/sqlparser.y: warning: 12 reduce/reduce conflicts [-Wconflicts-rr]
It is currently at 73/12.

This will also be caught by regress/compGeneral/TESTTOK2.
If that test passes, then conflicts have not changed. Check for that.

> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365573#comment-15365573
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user traflm commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69848822
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

@zellerh , I don't understand what is "number of shift/reduce and 
reduce/reduce conflicts". How to check this number? What is this?


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365454#comment-15365454
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69839237
  
--- Diff: core/sql/bin/SqlciErrors.txt ---
@@ -410,6 +410,7 @@
 1430 3F000 9 BEGINNER MAJOR DBADMIN A schema name that starts and ends 
with an "_"(underscore) is reserved for internal usage. It cannot be used to 
create a user schema.
 1431 Z 9 BEGINNER MINOR DBADMIN Object $0~String0 exists in HBase. 
This could be due to a concurrent transactional ddl operation in progress on 
this table.
 1432 Z 9 BEGINNER MINOR DBADMIN Input LOB type $0~Int0 does not 
match column's storage type: $1~Int1 Column name: $0~String0 .
+1433 Z 9 BEGINNER MINOR DBADMIN WITH clause redefined. WITH name 
$0~String0 .
--- End diff --

Parser errors should be in the range 3000-3999. Also, since this is a 
syntax error, use 42000 (the SQLSTATE value for syntax errors) instead of 
"Z".


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365423#comment-15365423
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69837244
  
--- Diff: core/sql/sqlcomp/parser.h ---
@@ -261,6 +261,8 @@ ItemExpr *get_w_ItemExprTree(const NAWchar * str,
   NABoolean isHQCCacheable()
 { return HQCKey_?HQCKey_->isCacheable():FALSE;  }
 
+  NAHashDictionary *with_clauses_;
--- End diff --

Can you make this a private data member?


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365421#comment-15365421
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69837060
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

Related question: When you made this change, did the number of shift/reduce 
and reduce/reduce conflicts change?


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365413#comment-15365413
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69836802
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -14684,6 +14759,11 @@ optional_limit_spec : TOK_LIMIT 
NUMERIC_LITERAL_EXACT_NO_SCALE
 
 dml_statement : dml_query { $$ = $1; }
 
+   | with_clause_list dml_query
--- End diff --

Same comment as a above about where the parser rule for the WITH clause 
should be.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365409#comment-15365409
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69836669
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -7001,6 +7036,46 @@ rel_subquery_and_as_clause : rel_subquery as_clause
$$ = $2;
  }
 
+with_clause_list : with_clause
+ {  $$ = $1 ; }
+
+   | with_clause_list ',' correlation_name TOK_AS 
rel_subquery 
--- End diff --

Can we do this in a way that doesn't have two different rules for the first 
and the subsequent parts of the WITH clause? See comment above for how the ANSI 
standard handles the WITH clause.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365402#comment-15365402
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69836226
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -6579,6 +6581,9 @@ table_reference : table_name_and_hint
   | rel_subquery_and_as_clause
 { $$ = $1; }
 
+  | with_clause_list
--- End diff --

Here is the syntax rule from the ANSI standard:

 ::=
[  ] 


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365383#comment-15365383
 ] 

ASF GitHub Bot commented on TRAFODION-1673:
---

Github user zellerh commented on a diff in the pull request:

https://github.com/apache/incubator-trafodion/pull/577#discussion_r69835003
  
--- Diff: core/sql/parser/sqlparser.y ---
@@ -6579,6 +6581,9 @@ table_reference : table_name_and_hint
   | rel_subquery_and_as_clause
 { $$ = $1; }
 
+  | with_clause_list
--- End diff --

The ANSI SQL standard defines the "with" clause as part of a 
query_expression, so I would recommend doing the same here 
(non_join_query_expression in our case). Also, it would be good to accept the 
WITH RECURSIVE syntax and to give an error that this is not yet supported.


> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-06 Thread liu ming (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364404#comment-15364404
 ] 

liu ming commented on TRAFODION-1673:
-

To fully support WITH, I agree with both Hans and QiFan. Especially the idea of 
evaluating the CTE only once. However, we can do this step by step.
The very first step is as Hans' suggest, Trafodion support WITH function in the 
parser. The 'Temp View' can be simply be a 'subquery', which is a RenameTable 
in the parse tree. As initial testing, it can support recursive reference as 
well, but with limitation that the WITH definition must occur before it is 
referred , for example:

with w1 as (select * from witht1),
w2 as (select * from w1)
select * from w2; 

is good, since w1 is defined before it is used in w2.

But 
with w1 as (select * from w2),
w2 as (select * from t1)
select * from w1;

will not work, however, I don't see the second usage in all TPCDS queries. So I 
feel it should be good enough to support the first syntax.

This first iteration can implement the functionality, and parser change can be 
reused later of a 'common CTE', and test case can be used also.

Later, we can implement a new CTE type of node, which can feed multiple data 
consumers, but I feel that is not easy in current architecture.

> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2016-07-01 Thread liu ming (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359961#comment-15359961
 ] 

liu ming commented on TRAFODION-1673:
-

It can be first implemented using sub-query, that is, to replace each WITH 
clause by a sub query. (RenamedTable)

CTE mentioned by QiFan is a great idea, and we can set it as a final goal. To 
meet the functional requirement, we can simply use subquery in the first 
iteration.

> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>Assignee: liu ming
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TRAFODION-1673) Implement the WITH clause in Trafodion SQL for simple use cases

2015-12-07 Thread Qifan Chen (JIRA)

[ 
https://issues.apache.org/jira/browse/TRAFODION-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15044992#comment-15044992
 ] 

Qifan Chen commented on TRAFODION-1673:
---

On the scope of WITH support.  In the following CTE = common table expression 
defined by in a WITH clause.

By looking TPCDS, I can see the following.
1. Multiple CTEs can be defined within one query
2. One CTE can be defined based on another CTE 
   Q2: wswscs is dependent on wscs
  Q47: v2 is based on v1
  Q54: my_revenue is dependent on my_customers, segments is dependent on 
my_revenue
  Q57: v2 is based on v1
  Q65: cross_sales is based on cs_ui
3. Repeated references of the same CTE in a single query 
  Q2: wswscs is used in two separate joins
  Q39: inv is used in join with itself as inv1 and inv2
  Q4: year_total is self joined 6 times
  Q59: wss is used in separate sub-queries
  Q65: cross_sales is self joined
  Q74: year_total is self-joined 3 times
  Q95: ws_wh is used in two sub-queries
4. CTEs can be used just like a table expression
  joined with another table
  joined with another CTE
  Union-ed with another CTE
5. CTEs can require extensive computations

So it looks like we need to deal with all above aspects of WITH. 

In addition, to efficiently handle repeated references of the same CET and CTEs 
require extensive computations to compute, I feel we should consider computing 
CTE only once per query. Here is one idea to achieve it. 

For example, to self join a CBE C, we could have the following query tree. S is 
the single computation of C. R is a reference to C. S "smartly" broadcasts data 
to two Rs. 

  join
 / \
RR
|   /
S

> Implement the WITH clause in Trafodion SQL for simple use cases
> ---
>
> Key: TRAFODION-1673
> URL: https://issues.apache.org/jira/browse/TRAFODION-1673
> Project: Apache Trafodion
>  Issue Type: New Feature
>  Components: sql-cmp
>Reporter: Hans Zeller
>
> We keep running into queries that use a WITH clause to define a temporary 
> view that can be used once or multiple times in a FROM clause in the query.
> For non-recursive queries, the WITH clause could probably be handled very 
> similar to a view. When it is defined, we create an in-memory view 
> descriptor, containing the name and the definition. When it is used in a FROM 
> clause, we could go through a code path similar to that of a view - bind the 
> (temporary) view text and substitute it in the query. The fix could probably 
> be handled entirely in the binder.
> This JIRA is *not* about recursive queries, those would require a lot more 
> effort, involving many components in addition to the binder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)