Re: doc on predicate pushdown in joins

[Lefty Leverenz]

Easy as 3.14159  (I can take a hint.)

-- Lefty


On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair  wrote:

> FYI, Harish has a written a very nice doc describing predicate push
> down rules for join. I have attached it to the design doc page. It
> will be very useful for anyone looking at joins.
>
> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
>
> (any help converting it to wiki format from html is welcome!).
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: doc on predicate pushdown in joins

[Thejas Nair]

:)


On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz  wrote:
> Easy as 3.14159  (I can take a hint.)
>
> -- Lefty
>
>
> On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair  wrote:
>
>> FYI, Harish has a written a very nice doc describing predicate push
>> down rules for join. I have attached it to the design doc page. It
>> will be very useful for anyone looking at joins.
>>
>> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
>>
>> (any help converting it to wiki format from html is welcome!).
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: doc on predicate pushdown in joins

[Lefty Leverenz]

The Outer Join Behavior
wikidocis
done, with links from the Design
Docs  page and
the Joins 
doc
.

Harish (or anyone else) would you please review the changes I made to
the definition
for "Null Supplying
table"
?

-- Lefty


On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair  wrote:

> :)
>
>
> On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz 
> wrote:
> > Easy as 3.14159  (I can take a hint.)
> >
> > -- Lefty
> >
> >
> > On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair 
> wrote:
> >
> >> FYI, Harish has a written a very nice doc describing predicate push
> >> down rules for join. I have attached it to the design doc page. It
> >> will be very useful for anyone looking at joins.
> >>
> >>
> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
> >>
> >> (any help converting it to wiki format from html is welcome!).
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or
> entity to
> >> which it is addressed and may contain information that is confidential,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> immediately
> >> and delete it from your system. Thank You.
> >>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: doc on predicate pushdown in joins

[Thejas Nair]

Looks great!
Thanks for also linking to it from the other relevant docs!

On Mon, Dec 9, 2013 at 1:52 AM, Lefty Leverenz  wrote:
> The Outer Join Behavior
> wikidocis
> done, with links from the Design
> Docs  page and
> the Joins 
> doc
> .
>
> Harish (or anyone else) would you please review the changes I made to
> the definition
> for "Null Supplying
> table"
> ?
>
> -- Lefty
>
>
> On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair  wrote:
>
>> :)
>>
>>
>> On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz 
>> wrote:
>> > Easy as 3.14159  (I can take a hint.)
>> >
>> > -- Lefty
>> >
>> >
>> > On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair 
>> wrote:
>> >
>> >> FYI, Harish has a written a very nice doc describing predicate push
>> >> down rules for join. I have attached it to the design doc page. It
>> >> will be very useful for anyone looking at joins.
>> >>
>> >>
>> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
>> >>
>> >> (any help converting it to wiki format from html is welcome!).
>> >>
>> >> --
>> >> CONFIDENTIALITY NOTICE
>> >> NOTICE: This message is intended for the use of the individual or
>> entity to
>> >> which it is addressed and may contain information that is confidential,
>> >> privileged and exempt from disclosure under applicable law. If the
>> reader
>> >> of this message is not the intended recipient, you are hereby notified
>> that
>> >> any printing, copying, dissemination, distribution, disclosure or
>> >> forwarding of this communication is strictly prohibited. If you have
>> >> received this communication in error, please contact the sender
>> immediately
>> >> and delete it from your system. Thank You.
>> >>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: doc on predicate pushdown in joins

[Harish Butani]

Looks good.  Thanks for doing this.

Minor point:

Rule 1: During QBJoinTree construction in Plan Gen, the parse Join Condition 
logic applies this rule.
Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias logic 
applies this rule.

FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the 
SemanticAnalyzer and JoinPPD classes respectively. 
Writing these as separate words maybe confusing. You are better judge of how to 
represent this(quoted/bold etc.)

regards,
Harish.


On Dec 9, 2013, at 1:52 AM, Lefty Leverenz  wrote:

> The Outer Join Behavior
> wikidocis
> done, with links from the Design
> Docs  page and
> the Joins 
> doc
> .
> 
> Harish (or anyone else) would you please review the changes I made to
> the definition
> for "Null Supplying
> table"
> ?
> 
> -- Lefty
> 
> 
> On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair  wrote:
> 
>> :)
>> 
>> 
>> On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz 
>> wrote:
>>> Easy as 3.14159  (I can take a hint.)
>>> 
>>> -- Lefty
>>> 
>>> 
>>> On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair 
>> wrote:
>>> 
 FYI, Harish has a written a very nice doc describing predicate push
 down rules for join. I have attached it to the design doc page. It
 will be very useful for anyone looking at joins.
 
 
>> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
 
 (any help converting it to wiki format from html is welcome!).
 
 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or
>> entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the
>> reader
 of this message is not the intended recipient, you are hereby notified
>> that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender
>> immediately
 and delete it from your system. Thank You.
 
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: doc on predicate pushdown in joins

[Lefty Leverenz]

Okay, then monospace with "()" after the method name is a good way to show
them:  parseJoinCondition() and getQualifiedAlias() ... but I only found
the latter pluralized, instead of singular, so should it be
getQualifiedAliases() or am I missing something?

trunk> *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'*
>
> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221:   *
>> the comments for getQualifiedAliases function.
>
> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230:
>>  Set aliases = getQualifiedAliases((JoinOperator) nd, owi
>
> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:
>>  // be pushed down per getQualifiedAliases
>
> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471:
>>  private Set getQualifiedAliases(JoinOperator op, RowResolver
>> rr) {
>
>
>
-- Lefty


On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani wrote:

> Looks good.  Thanks for doing this.
>
> Minor point:
>
> *Rule 1:* During *QBJoinTree* construction in Plan Gen, the parse Join
> Condition logic applies this rule.
> *Rule 2:* During *JoinPPD* (Join Predicate Pushdown) the get Qualified
> Alias logic applies this rule.
>
> FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the
> SemanticAnalyzer and JoinPPD classes respectively.
> Writing these as separate words maybe confusing. You are better judge of
> how to represent this(quoted/bold etc.)
>
> regards,
> Harish.
>
>
> On Dec 9, 2013, at 1:52 AM, Lefty Leverenz 
> wrote:
>
> The Outer Join Behavior
> wikidoc >is
>
> done, with links from the Design
> Docs  page
> and
> the Joins doc<
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization
> >
> .
>
> Harish (or anyone else) would you please review the changes I made to
> the definition
> for "Null Supplying
> table"<
> https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions
> >
>
> ?
>
> -- Lefty
>
>
> On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair 
> wrote:
>
> :)
>
>
> On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz 
> wrote:
>
> Easy as 3.14159  (I can take a hint.)
>
> -- Lefty
>
>
> On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair 
>
> wrote:
>
>
> FYI, Harish has a written a very nice doc describing predicate push
> down rules for join. I have attached it to the design doc page. It
> will be very useful for anyone looking at joins.
>
>
>
> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
>
>
> (any help converting it to wiki format from html is welcome!).
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
>
> entity to
>
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the
>
> reader
>
> of this message is not the intended recipient, you are hereby notified
>
> that
>
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender
>
> immediately
>
> and delete it from your system. Thank You.
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: doc on predicate pushdown in joins

[Harish Butani]

You are correct, it is plural.

regards,
Harish.
On Dec 10, 2013, at 4:03 AM, Lefty Leverenz  wrote:

> Okay, then monospace with "()" after the method name is a good way to show 
> them:  parseJoinCondition() and getQualifiedAlias() ... but I only found the 
> latter pluralized, instead of singular, so should it be getQualifiedAliases() 
> or am I missing something?
> 
> trunk> grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'
> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221:   * the 
> comments for getQualifiedAliases function.
> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230:  
> Set aliases = getQualifiedAliases((JoinOperator) nd, owi
> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:// 
> be pushed down per getQualifiedAliases
> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471:
> private Set getQualifiedAliases(JoinOperator op, RowResolver rr) {
> 
> 
> -- Lefty
> 
> 
> On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani  wrote:
> Looks good.  Thanks for doing this.
> 
> Minor point:
> 
> Rule 1: During QBJoinTree construction in Plan Gen, the parse Join Condition 
> logic applies this rule.
> Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias 
> logic applies this rule.
> 
> FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the 
> SemanticAnalyzer and JoinPPD classes respectively. 
> Writing these as separate words maybe confusing. You are better judge of how 
> to represent this(quoted/bold etc.)
> 
> regards,
> Harish.
> 
> 
> On Dec 9, 2013, at 1:52 AM, Lefty Leverenz  wrote:
> 
>> The Outer Join Behavior
>> wikidocis
>> 
>> done, with links from the Design
>> Docs  page and
>> the Joins 
>> doc
>> .
>> 
>> Harish (or anyone else) would you please review the changes I made to
>> the definition
>> for "Null Supplying
>> table"
>> 
>> ?
>> 
>> -- Lefty
>> 
>> 
>> On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair  wrote:
>> 
>>> :)
>>> 
>>> 
>>> On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz 
>>> wrote:
 Easy as 3.14159  (I can take a hint.)
 
 -- Lefty
 
 
 On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair 
>>> wrote:
 
> FYI, Harish has a written a very nice doc describing predicate push
> down rules for join. I have attached it to the design doc page. It
> will be very useful for anyone looking at joins.
> 
> 
>>> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
> 
> (any help converting it to wiki format from html is welcome!).
> 
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
>>> entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the
>>> reader
> of this message is not the intended recipient, you are hereby notified
>>> that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender
>>> immediately
> and delete it from your system. Thank You.
> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>> 
> 
> 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader of 
> this message is not the intended recipient, you are hereby notified that any 
> printing, copying, dissemination, distribution, disclosure or forwarding of 
> this communication is strictly prohibited. If you have received this 
> communication in error, please contact the sender immediately and delete it 
> from your system. Thank You.
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information t

Re: doc on predicate pushdown in joins

[Lefty Leverenz]

How's this?  Hive
Implementation

Also, I moved the link on the Design Docs
pagefrom
*Proposed* to *Other*.  (It's called SQL Outer Join Predicate Pushdown
Rules
which
doesn't match the title, but seems okay because it's more descriptive.)

-- Lefty


On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani wrote:

> You are correct, it is plural.
>
> regards,
> Harish.
>
> On Dec 10, 2013, at 4:03 AM, Lefty Leverenz 
> wrote:
>
> Okay, then monospace with "()" after the method name is a good way to show
> them:  parseJoinCondition() and getQualifiedAlias() ... but I only found
> the latter pluralized, instead of singular, so should it be
> getQualifiedAliases() or am I missing something?
>
> trunk> *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'*
>>
>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221:   *
>>> the comments for getQualifiedAliases function.
>>
>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230:
>>>  Set aliases = getQualifiedAliases((JoinOperator) nd, owi
>>
>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:
>>>  // be pushed down per getQualifiedAliases
>>
>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471:
>>>  private Set getQualifiedAliases(JoinOperator op, RowResolver
>>> rr) {
>>
>>
>>
> -- Lefty
>
>
> On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani wrote:
>
>> Looks good.  Thanks for doing this.
>>
>> Minor point:
>>
>> *Rule 1:* During *QBJoinTree* construction in Plan Gen, the parse Join
>> Condition logic applies this rule.
>> *Rule 2:* During *JoinPPD* (Join Predicate Pushdown) the get Qualified
>> Alias logic applies this rule.
>>
>> FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the
>> SemanticAnalyzer and JoinPPD classes respectively.
>> Writing these as separate words maybe confusing. You are better judge of
>> how to represent this(quoted/bold etc.)
>>
>> regards,
>> Harish.
>>
>>
>> On Dec 9, 2013, at 1:52 AM, Lefty Leverenz 
>> wrote:
>>
>> The Outer Join Behavior
>> wikidoc<
>> https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior>is
>>
>> done, with links from the Design
>> Docs  page
>> and
>> the Joins doc<
>> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization
>> >
>> .
>>
>> Harish (or anyone else) would you please review the changes I made to
>> the definition
>> for "Null Supplying
>> table"<
>> https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions
>> >
>>
>> ?
>>
>> -- Lefty
>>
>>
>> On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair 
>> wrote:
>>
>> :)
>>
>>
>> On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz 
>> wrote:
>>
>> Easy as 3.14159  (I can take a hint.)
>>
>> -- Lefty
>>
>>
>> On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair 
>>
>> wrote:
>>
>>
>> FYI, Harish has a written a very nice doc describing predicate push
>> down rules for join. I have attached it to the design doc page. It
>> will be very useful for anyone looking at joins.
>>
>>
>>
>> https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
>>
>>
>> (any help converting it to wiki format from html is welcome!).
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or
>>
>> entity to
>>
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the
>>
>> reader
>>
>> of this message is not the intended recipient, you are hereby notified
>>
>> that
>>
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>>
>> immediately
>>
>> and delete it from your system. Thank You.
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosu

Re: doc on predicate pushdown in joins

[Harish Butani]

I can see why you would rename.

But this sentence is not correct:
'Hive enforces the predicate pushdown rules by these methods in the 
SemanticAnalyzer and JoinPPD classes:'

It should be:
Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD 
classes:

(The implementation involves both predicate pushdown and analyzing join 
conditions)
Sorry about this.

So the link should say 'Hive Outer Join Behavior'

regards,
Harish.


On Dec 10, 2013, at 2:01 PM, Lefty Leverenz  wrote:

> How's this?  Hive Implementation
> 
> Also, I moved the link on the Design Docs page from Proposed to Other.  (It's 
> called SQL Outer Join Predicate Pushdown Rules which doesn't match the title, 
> but seems okay because it's more descriptive.)
> 
> -- Lefty
> 
> 
> On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani  
> wrote:
> You are correct, it is plural.
> 
> regards,
> Harish.
> 
> On Dec 10, 2013, at 4:03 AM, Lefty Leverenz  wrote:
> 
>> Okay, then monospace with "()" after the method name is a good way to show 
>> them:  parseJoinCondition() and getQualifiedAlias() ... but I only found the 
>> latter pluralized, instead of singular, so should it be 
>> getQualifiedAliases() or am I missing something?
>> 
>> trunk> grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'
>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221:   * the 
>> comments for getQualifiedAliases function.
>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230:  
>> Set aliases = getQualifiedAliases((JoinOperator) nd, owi
>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:
>> // be pushed down per getQualifiedAliases
>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471:
>> private Set getQualifiedAliases(JoinOperator op, RowResolver rr) {
>> 
>> 
>> -- Lefty
>> 
>> 
>> On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani  
>> wrote:
>> Looks good.  Thanks for doing this.
>> 
>> Minor point:
>> 
>> Rule 1: During QBJoinTree construction in Plan Gen, the parse Join Condition 
>> logic applies this rule.
>> Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias 
>> logic applies this rule.
>> 
>> FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the 
>> SemanticAnalyzer and JoinPPD classes respectively. 
>> Writing these as separate words maybe confusing. You are better judge of how 
>> to represent this(quoted/bold etc.)
>> 
>> regards,
>> Harish.
>> 
>> 
>> On Dec 9, 2013, at 1:52 AM, Lefty Leverenz  wrote:
>> 
>>> The Outer Join Behavior
>>> wikidocis
>>> 
>>> done, with links from the Design
>>> Docs  page and
>>> the Joins 
>>> doc
>>> .
>>> 
>>> Harish (or anyone else) would you please review the changes I made to
>>> the definition
>>> for "Null Supplying
>>> table"
>>> 
>>> ?
>>> 
>>> -- Lefty
>>> 
>>> 
>>> On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair  wrote:
>>> 
 :)
 
 
 On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz 
 wrote:
> Easy as 3.14159  (I can take a hint.)
> 
> -- Lefty
> 
> 
> On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair 
 wrote:
> 
>> FYI, Harish has a written a very nice doc describing predicate push
>> down rules for join. I have attached it to the design doc page. It
>> will be very useful for anyone looking at joins.
>> 
>> 
 https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html
>> 
>> (any help converting it to wiki format from html is welcome!).
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or
 entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the
 reader
>> of this message is not the intended recipient, you are hereby notified
 that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
 immediately
>> and delete it from your system. Thank You.
>> 
 
 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified that
 any printing, copying, dissemination, distribution, disclosure or
 forward

Re: doc on predicate pushdown in joins

[Lefty Leverenz]

Happy to fix the sentence and the link.  I pointed out the name change just
so you would review it, so please don't apologize!

One more question:  why am I not finding getQualifiedAliases() in the
SemanticAnalyzer class?  It turns up in OpProcFactory.java with javadoc
comments, but I can't find it anywhere in the API docs -- not even in the
index (Hive 0.12.0 API ):

*getQMap()*
-
Method in class
org.apache.hadoop.hive.ql.QTestUtil
 
*getQualifiedName()*
-
Method in class
org.apache.hadoop.hive.serde2.typeinfo.TypeInfoString
representing the qualified type
name.*getQualifiers()*
-
Method in class
org.apache.hive.service.cli.thrift.TTypeQualifiers
 
*getQualifiersSize()*
-
Method in class
org.apache.hive.service.cli.thrift.TTypeQualifiers
Most mysterious.

-- Lefty


On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani wrote:

> I can see why you would rename.
>
> But this sentence is not correct:
> 'Hive enforces the predicate pushdown rules by these methods in the
> SemanticAnalyzer and JoinPPD classes:'
>
> It should be:
> Hive enforces the rules by these methods in the SemanticAnalyzer and
> JoinPPD classes:
>
> (The implementation involves both predicate pushdown and analyzing join
> conditions)
> Sorry about this.
>
> So the link should say 'Hive Outer Join Behavior'
>
> regards,
> Harish.
>
>
> On Dec 10, 2013, at 2:01 PM, Lefty Leverenz 
> wrote:
>
> How's this?  Hive 
> Implementation
>
> Also, I moved the link on the Design Docs 
> pagefrom
> *Proposed* to *Other*.  (It's called SQL Outer Join Predicate Pushdown
> Rules  
> which
> doesn't match the title, but seems okay because it's more descriptive.)
>
> -- Lefty
>
>
> On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani wrote:
>
>> You are correct, it is plural.
>>
>> regards,
>> Harish.
>>
>> On Dec 10, 2013, at 4:03 AM, Lefty Leverenz 
>> wrote:
>>
>> Okay, then monospace with "()" after the method name is a good way to
>> show them:  parseJoinCondition() and getQualifiedAlias() ... but I only
>> found the latter pluralized, instead of singular, so should it be
>> getQualifiedAliases() or am I missing something?
>>
>> trunk> *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'*
>>>
>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221:   *
 the comments for getQualifiedAliases function.
>>>
>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230:
  Set aliases = getQualifiedAliases((JoinOperator) nd, owi
>>>
>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:
// be pushed down per getQualifiedAliases
>>>
>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471:
  private Set getQualifiedAliases(JoinOperator op, RowResolver
 rr) {
>>>
>>>
>>>
>> -- Lefty
>>
>>
>> On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani wrote:
>>
>>> Looks good.  Thanks for doing this.
>>>
>>> Minor point:
>>>
>>> *Rule 1:* During *QBJoinTree* construction in Plan Gen, the parse Join
>>> Condition logic applies this rule.
>>> *Rule 2:* During *JoinPPD* (Join Predicate Pushdown) the get Qualified
>>> Alias logic applies this rule.
>>>
>>> FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the
>>> SemanticAnalyzer and JoinPPD classes respectively.
>>> Writing these as separate words maybe confusing. You are better judge of
>>> how to represent this(quoted/bold etc.)
>>>
>>> regards,
>>> Harish.
>>>
>>>
>>> On Dec 9, 2013, at 1:52 AM, Lefty Leverenz 
>>> wrote:
>>>
>>> The Outer Join Behavior
>>> wikidoc<
>>> https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior>is
>>>
>>> done, with links from the Design
>>> Docs  page
>>> and
>>> the Joins doc<
>>> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization
>>> >
>>> .
>>>
>>> Harish (or anyone else) would you please review the changes I made to

Re: doc on predicate pushdown in joins

[Harish Butani]

getQualifiedAliases is a private method in JoinPPD.

Maybe we should remove the section on Hive Implementation here. It is in the 
Design doc; this information only concerns developers.

regards,
Harish.


On Dec 11, 2013, at 3:05 AM, Lefty Leverenz  wrote:

> Happy to fix the sentence and the link.  I pointed out the name change just 
> so you would review it, so please don't apologize!
> 
> One more question:  why am I not finding getQualifiedAliases() in the 
> SemanticAnalyzer class?  It turns up in OpProcFactory.java with javadoc 
> comments, but I can't find it anywhere in the API docs -- not even in the 
> index (Hive 0.12.0 API): 
> 
> getQMap() - Method in class org.apache.hadoop.hive.ql.QTestUtil
>  
> getQualifiedName() - Method in class 
> org.apache.hadoop.hive.serde2.typeinfo.TypeInfo
> String representing the qualified type name.
> getQualifiers() - Method in class 
> org.apache.hive.service.cli.thrift.TTypeQualifiers
>  
> getQualifiersSize() - Method in class 
> org.apache.hive.service.cli.thrift.TTypeQualifiers
> 
> Most mysterious.
> 
> -- Lefty
> 
> 
> On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani  
> wrote:
> I can see why you would rename.
> 
> But this sentence is not correct:
> 'Hive enforces the predicate pushdown rules by these methods in the 
> SemanticAnalyzer and JoinPPD classes:'
> 
> It should be:
> Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD 
> classes:
> 
> (The implementation involves both predicate pushdown and analyzing join 
> conditions)
> Sorry about this.
> 
> So the link should say 'Hive Outer Join Behavior'
> 
> regards,
> Harish.
> 
> 
> On Dec 10, 2013, at 2:01 PM, Lefty Leverenz  wrote:
> 
>> How's this?  Hive Implementation
>> 
>> Also, I moved the link on the Design Docs page from Proposed to Other.  
>> (It's called SQL Outer Join Predicate Pushdown Rules which doesn't match the 
>> title, but seems okay because it's more descriptive.)
>> 
>> -- Lefty
>> 
>> 
>> On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani  
>> wrote:
>> You are correct, it is plural.
>> 
>> regards,
>> Harish.
>> 
>> On Dec 10, 2013, at 4:03 AM, Lefty Leverenz  wrote:
>> 
>>> Okay, then monospace with "()" after the method name is a good way to show 
>>> them:  parseJoinCondition() and getQualifiedAlias() ... but I only found 
>>> the latter pluralized, instead of singular, so should it be 
>>> getQualifiedAliases() or am I missing something?
>>> 
>>> trunk> grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'
>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221:   * the 
>>> comments for getQualifiedAliases function.
>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230:  
>>> Set aliases = getQualifiedAliases((JoinOperator) nd, owi
>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:
>>> // be pushed down per getQualifiedAliases
>>> ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471:
>>> private Set getQualifiedAliases(JoinOperator op, RowResolver rr) {
>>> 
>>> 
>>> -- Lefty
>>> 
>>> 
>>> On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani  
>>> wrote:
>>> Looks good.  Thanks for doing this.
>>> 
>>> Minor point:
>>> 
>>> Rule 1: During QBJoinTree construction in Plan Gen, the parse Join 
>>> Condition logic applies this rule.
>>> Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias 
>>> logic applies this rule.
>>> 
>>> FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the 
>>> SemanticAnalyzer and JoinPPD classes respectively. 
>>> Writing these as separate words maybe confusing. You are better judge of 
>>> how to represent this(quoted/bold etc.)
>>> 
>>> regards,
>>> Harish.
>>> 
>>> 
>>> On Dec 9, 2013, at 1:52 AM, Lefty Leverenz  wrote:
>>> 
 The Outer Join Behavior
 wikidocis
 
 done, with links from the Design
 Docs  page and
 the Joins 
 doc
 .
 
 Harish (or anyone else) would you please review the changes I made to
 the definition
 for "Null Supplying
 table"
 
 ?
 
 -- Lefty
 
 
 On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair  wrote:
 
> :)
> 
> 
> On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz 
> wrote:
>> Easy as 3.14159  (I can take a hint.)
>> 
>> -- Lefty
>> 
>> 
>> On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair 
> wrote:
>> 
>>> FYI, Harish has a written a very nice doc describing predicate push
>>> down rules for join. I have attached it to the design doc page. It
>>> will be very useful for anyone looking at joins.
>>> 
>>> 
> http

Re: doc on predicate pushdown in joins

[Lefty Leverenz]

> Maybe we should remove the section on Hive Implementation here.
> It is in the Design doc; this information only concerns developers.

But this is the Design doc (unless there's another one somewhere -- maybe
attached to a JIRA ticket?) and it's in the Resources for Contributors part
of the wiki, so it seems appropriate to me.  I'll delete the implementation
section if that's your preference.

Here are the links again, with fixes:

   - Design Docs
(bottom
   of list)
   - Predicate Pushdown
Rules


Speaking of JIRA tickets, is there one for this and should I add any
version information?

-- Lefty


On Wed, Dec 11, 2013 at 7:59 AM, Harish Butani wrote:

> getQualifiedAliases is a private method in JoinPPD.
>
> Maybe we should remove the section on Hive Implementation here. It is in
> the Design doc; this information only concerns developers.
>
> regards,
> Harish.
>
>
> On Dec 11, 2013, at 3:05 AM, Lefty Leverenz 
> wrote:
>
> Happy to fix the sentence and the link.  I pointed out the name change
> just so you would review it, so please don't apologize!
>
> One more question:  why am I not finding getQualifiedAliases() in the
> SemanticAnalyzer class?  It turns up in OpProcFactory.java with javadoc
> comments, but I can't find it anywhere in the API docs -- not even in the
> index (Hive 0.12.0 API ):
>
> *getQMap()*
>  -
> Method in class 
> org.apache.hadoop.hive.ql.QTestUtil
>  
> *getQualifiedName()*
>  -
> Method in class 
> org.apache.hadoop.hive.serde2.typeinfo.TypeInfo
>  String
> representing the qualified type 
> name.*getQualifiers()*
>  -
> Method in class 
> org.apache.hive.service.cli.thrift.TTypeQualifiers
>  
> *getQualifiersSize()*
>  -
> Method in class 
> org.apache.hive.service.cli.thrift.TTypeQualifiers
> Most mysterious.
>
> -- Lefty
>
>
> On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani wrote:
>
>> I can see why you would rename.
>>
>> But this sentence is not correct:
>> 'Hive enforces the predicate pushdown rules by these methods in the
>> SemanticAnalyzer and JoinPPD classes:'
>>
>> It should be:
>> Hive enforces the rules by these methods in the SemanticAnalyzer and
>> JoinPPD classes:
>>
>> (The implementation involves both predicate pushdown and analyzing join
>> conditions)
>> Sorry about this.
>>
>> So the link should say 'Hive Outer Join Behavior'
>>
>> regards,
>> Harish.
>>
>>
>> On Dec 10, 2013, at 2:01 PM, Lefty Leverenz 
>> wrote:
>>
>> How's this?  Hive 
>> Implementation
>>
>> Also, I moved the link on the Design Docs 
>> pagefrom
>> *Proposed* to *Other*.  (It's called SQL Outer Join Predicate Pushdown
>> Rules 
>> which
>> doesn't match the title, but seems okay because it's more descriptive.)
>>
>> -- Lefty
>>
>>
>> On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani 
>> wrote:
>>
>>> You are correct, it is plural.
>>>
>>> regards,
>>> Harish.
>>>
>>> On Dec 10, 2013, at 4:03 AM, Lefty Leverenz 
>>> wrote:
>>>
>>> Okay, then monospace with "()" after the method name is a good way to
>>> show them:  parseJoinCondition() and getQualifiedAlias() ... but I only
>>> found the latter pluralized, instead of singular, so should it be
>>> getQualifiedAliases() or am I missing something?
>>>
>>> trunk> *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'*

 ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221:   *
> the comments for getQualifiedAliases function.

 ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230:
>  Set aliases = getQualifiedAliases((JoinOperator) nd, owi

 ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:
>// be pushed down per getQualifiedAliases

 ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471:
>  private 

Re: doc on predicate pushdown in joins

[Harish Butani]

I see.  Let's leave it in.

This is old code, hard to attribute to  jiras:
- The PPD code comes from: HIVE-279, HIVE-2337
- I cannot tell when the join condition parsing code was added.

regards,
Harish.

On Dec 11, 2013, at 5:17 PM, Lefty Leverenz  wrote:

> > Maybe we should remove the section on Hive Implementation here. 
> > It is in the Design doc; this information only concerns developers.
> 
> But this is the Design doc (unless there's another one somewhere -- maybe 
> attached to a JIRA ticket?) and it's in the Resources for Contributors part 
> of the wiki, so it seems appropriate to me.  I'll delete the implementation 
> section if that's your preference. 
> 
> Here are the links again, with fixes: 
> Design Docs (bottom of list)
> Predicate Pushdown Rules 
> Speaking of JIRA tickets, is there one for this and should I add any version 
> information? 
> 
> -- Lefty
> 
> 
> On Wed, Dec 11, 2013 at 7:59 AM, Harish Butani  
> wrote:
> getQualifiedAliases is a private method in JoinPPD.
> 
> Maybe we should remove the section on Hive Implementation here. It is in the 
> Design doc; this information only concerns developers.
> 
> regards,
> Harish.
> 
> 
> On Dec 11, 2013, at 3:05 AM, Lefty Leverenz  wrote:
> 
>> Happy to fix the sentence and the link.  I pointed out the name change just 
>> so you would review it, so please don't apologize!
>> 
>> One more question:  why am I not finding getQualifiedAliases() in the 
>> SemanticAnalyzer class?  It turns up in OpProcFactory.java with javadoc 
>> comments, but I can't find it anywhere in the API docs -- not even in the 
>> index (Hive 0.12.0 API): 
>> 
>> getQMap() - Method in class org.apache.hadoop.hive.ql.QTestUtil
>>  
>> getQualifiedName() - Method in class 
>> org.apache.hadoop.hive.serde2.typeinfo.TypeInfo
>> String representing the qualified type name.
>> getQualifiers() - Method in class 
>> org.apache.hive.service.cli.thrift.TTypeQualifiers
>>  
>> getQualifiersSize() - Method in class 
>> org.apache.hive.service.cli.thrift.TTypeQualifiers
>> 
>> Most mysterious.
>> 
>> -- Lefty
>> 
>> 
>> On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani  
>> wrote:
>> I can see why you would rename.
>> 
>> But this sentence is not correct:
>> 'Hive enforces the predicate pushdown rules by these methods in the 
>> SemanticAnalyzer and JoinPPD classes:'
>> 
>> It should be:
>> Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD 
>> classes:
>> 
>> (The implementation involves both predicate pushdown and analyzing join 
>> conditions)
>> Sorry about this.
>> 
>> So the link should say 'Hive Outer Join Behavior'
>> 
>> regards,
>> Harish.
>> 
>> 
>> On Dec 10, 2013, at 2:01 PM, Lefty Leverenz  wrote:
>> 
>>> How's this?  Hive Implementation
>>> 
>>> Also, I moved the link on the Design Docs page from Proposed to Other.  
>>> (It's called SQL Outer Join Predicate Pushdown Rules which doesn't match 
>>> the title, but seems okay because it's more descriptive.)
>>> 
>>> -- Lefty
>>> 
>>> 
>>> On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani  
>>> wrote:
>>> You are correct, it is plural.
>>> 
>>> regards,
>>> Harish.
>>> 
>>> On Dec 10, 2013, at 4:03 AM, Lefty Leverenz  wrote:
>>> 
 Okay, then monospace with "()" after the method name is a good way to show 
 them:  parseJoinCondition() and getQualifiedAlias() ... but I only found 
 the latter pluralized, instead of singular, so should it be 
 getQualifiedAliases() or am I missing something?
 
 trunk> grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'
 ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221:   * 
 the comments for getQualifiedAliases function.
 ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230:  
 Set aliases = getQualifiedAliases((JoinOperator) nd, owi
 ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:
 // be pushed down per getQualifiedAliases
 ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471:
 private Set getQualifiedAliases(JoinOperator op, RowResolver rr) {
 
 
 -- Lefty
 
 
 On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani  
 wrote:
 Looks good.  Thanks for doing this.
 
 Minor point:
 
 Rule 1: During QBJoinTree construction in Plan Gen, the parse Join 
 Condition logic applies this rule.
 Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias 
 logic applies this rule.
 
 FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the 
 SemanticAnalyzer and JoinPPD classes respectively. 
 Writing these as separate words maybe confusing. You are better judge of 
 how to represent this(quoted/bold etc.)
 
 regards,
 Harish.
 
 
 On Dec 9, 2013, at 1:52 AM, Lefty Leverenz  wrote:
 
> The Outer Join Behavior
> wikidocis
> 
>

Re: doc on predicate pushdown in joins

[Lefty Leverenz]

>
> This is old code, hard to attribute to  jiras
>

Okay, HIVE-2337 was way back in release 0.8.0 so I think we can safely omit
version information.  If anyone objects, I can link the wikidoc to that
ticket.  (And I've marked HIVE-2337 as being related to HIVE-279 in the
JIRA.)

Thanks for all the help.

-- Lefty


On Wed, Dec 11, 2013 at 7:03 PM, Harish Butani wrote:

> I see.  Let's leave it in.
>
> This is old code, hard to attribute to  jiras:
> - The PPD code comes from: HIVE-279, HIVE-2337
> - I cannot tell when the join condition parsing code was added.
>
> regards,
> Harish.
>
> On Dec 11, 2013, at 5:17 PM, Lefty Leverenz 
> wrote:
>
> > Maybe we should remove the section on Hive Implementation here.
> > It is in the Design doc; this information only concerns developers.
>
> But this is the Design doc (unless there's another one somewhere -- maybe
> attached to a JIRA ticket?) and it's in the Resources for Contributors part
> of the wiki, so it seems appropriate to me.  I'll delete the implementation
> section if that's your preference.
>
> Here are the links again, with fixes:
>
>- Design Docs 
> (bottom
>of list)
>- Predicate Pushdown 
> Rules
>
>
> Speaking of JIRA tickets, is there one for this and should I add any
> version information?
>
> -- Lefty
>
>
> On Wed, Dec 11, 2013 at 7:59 AM, Harish Butani wrote:
>
>> getQualifiedAliases is a private method in JoinPPD.
>>
>> Maybe we should remove the section on Hive Implementation here. It is in
>> the Design doc; this information only concerns developers.
>>
>> regards,
>> Harish.
>>
>>
>> On Dec 11, 2013, at 3:05 AM, Lefty Leverenz 
>> wrote:
>>
>> Happy to fix the sentence and the link.  I pointed out the name change
>> just so you would review it, so please don't apologize!
>>
>> One more question:  why am I not finding getQualifiedAliases() in the
>> SemanticAnalyzer class?  It turns up in OpProcFactory.java with javadoc
>> comments, but I can't find it anywhere in the API docs -- not even in the
>> index (Hive 0.12.0 API ):
>>
>> *getQMap()*
>>  -
>> Method in class 
>> org.apache.hadoop.hive.ql.QTestUtil
>>  
>> *getQualifiedName()*
>>  -
>> Method in class 
>> org.apache.hadoop.hive.serde2.typeinfo.TypeInfo
>>  String
>> representing the qualified type 
>> name.*getQualifiers()*
>>  -
>> Method in class 
>> org.apache.hive.service.cli.thrift.TTypeQualifiers
>>  
>> *getQualifiersSize()*
>>  -
>> Method in class 
>> org.apache.hive.service.cli.thrift.TTypeQualifiers
>> Most mysterious.
>>
>> -- Lefty
>>
>>
>> On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani 
>> wrote:
>>
>>> I can see why you would rename.
>>>
>>> But this sentence is not correct:
>>> 'Hive enforces the predicate pushdown rules by these methods in the
>>> SemanticAnalyzer and JoinPPD classes:'
>>>
>>> It should be:
>>> Hive enforces the rules by these methods in the SemanticAnalyzer and
>>> JoinPPD classes:
>>>
>>> (The implementation involves both predicate pushdown and analyzing join
>>> conditions)
>>> Sorry about this.
>>>
>>> So the link should say 'Hive Outer Join Behavior'
>>>
>>> regards,
>>> Harish.
>>>
>>>
>>> On Dec 10, 2013, at 2:01 PM, Lefty Leverenz 
>>> wrote:
>>>
>>> How's this?  Hive 
>>> Implementation
>>>
>>> Also, I moved the link on the Design Docs 
>>> pagefrom
>>> *Proposed* to *Other*.  (It's called SQL Outer Join Predicate Pushdown
>>> Rules 
>>> which
>>> doesn't match the title, but seems okay because it's more descriptive.)
>>>
>>> -- Lefty
>>>
>>>
>>> On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani 
>>> wrote:
>>>
 You are correct, it is plural.

 regards,
 Harish.

 On Dec 10, 2013, at 4:03 AM, Lefty Leverenz 
 wrote:

 Okay, then monospace with "()" after the method name