Re: doc on predicate pushdown in joins
This is old code, hard to attribute to jiras Okay, HIVE-2337 was way back in release 0.8.0 so I think we can safely omit version information. If anyone objects, I can link the wikidoc to that ticket. (And I've marked HIVE-2337 as being related to HIVE-279 in the JIRA.) Thanks for all the help. -- Lefty On Wed, Dec 11, 2013 at 7:03 PM, Harish Butani hbut...@hortonworks.comwrote: I see. Let's leave it in. This is old code, hard to attribute to jiras: - The PPD code comes from: HIVE-279, HIVE-2337 - I cannot tell when the join condition parsing code was added. regards, Harish. On Dec 11, 2013, at 5:17 PM, Lefty Leverenz leftylever...@gmail.com wrote: Maybe we should remove the section on Hive Implementation here. It is in the Design doc; this information only concerns developers. But this is the Design doc (unless there's another one somewhere -- maybe attached to a JIRA ticket?) and it's in the Resources for Contributors part of the wiki, so it seems appropriate to me. I'll delete the implementation section if that's your preference. Here are the links again, with fixes: - Design Docshttps://cwiki.apache.org/confluence/display/Hive/DesignDocs (bottom of list) - Predicate Pushdown Ruleshttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-PredicatePushdownRules Speaking of JIRA tickets, is there one for this and should I add any version information? -- Lefty On Wed, Dec 11, 2013 at 7:59 AM, Harish Butani hbut...@hortonworks.comwrote: getQualifiedAliases is a private method in JoinPPD. Maybe we should remove the section on Hive Implementation here. It is in the Design doc; this information only concerns developers. regards, Harish. On Dec 11, 2013, at 3:05 AM, Lefty Leverenz leftylever...@gmail.com wrote: Happy to fix the sentence and the link. I pointed out the name change just so you would review it, so please don't apologize! One more question: why am I not finding getQualifiedAliases() in the SemanticAnalyzer class? It turns up in OpProcFactory.java with javadoc comments, but I can't find it anywhere in the API docs -- not even in the index (Hive 0.12.0 API http://hive.apache.org/docs/r0.12.0/api/): *getQMap()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html#getQMap() - Method in class org.apache.hadoop.hive.ql.QTestUtilhttp://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html *getQualifiedName()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html#getQualifiedName() - Method in class org.apache.hadoop.hive.serde2.typeinfo.TypeInfohttp://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html String representing the qualified type name.*getQualifiers()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiers() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiershttp://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html *getQualifiersSize()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiersSize() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiershttp://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html Most mysterious. -- Lefty On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani hbut...@hortonworks.comwrote: I can see why you would rename. But this sentence is not correct: 'Hive enforces the predicate pushdown rules by these methods in the SemanticAnalyzer and JoinPPD classes:' It should be: Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD classes: (The implementation involves both predicate pushdown and analyzing join conditions) Sorry about this. So the link should say 'Hive Outer Join Behavior' regards, Harish. On Dec 10, 2013, at 2:01 PM, Lefty Leverenz leftylever...@gmail.com wrote: How's this? Hive Implementationhttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-HiveImplementation Also, I moved the link on the Design Docs pagehttps://cwiki.apache.org/confluence/display/Hive/DesignDocsfrom *Proposed* to *Other*. (It's called SQL Outer Join Predicate Pushdown Ruleshttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior which doesn't match the title, but seems okay because it's more descriptive.) -- Lefty On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani hbut...@hortonworks.comwrote: You are correct, it is plural. regards, Harish. On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com wrote: Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of
Re: doc on predicate pushdown in joins
Happy to fix the sentence and the link. I pointed out the name change just so you would review it, so please don't apologize! One more question: why am I not finding getQualifiedAliases() in the SemanticAnalyzer class? It turns up in OpProcFactory.java with javadoc comments, but I can't find it anywhere in the API docs -- not even in the index (Hive 0.12.0 API http://hive.apache.org/docs/r0.12.0/api/): *getQMap()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html#getQMap() - Method in class org.apache.hadoop.hive.ql.QTestUtilhttp://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html *getQualifiedName()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html#getQualifiedName() - Method in class org.apache.hadoop.hive.serde2.typeinfo.TypeInfohttp://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.htmlString representing the qualified type name.*getQualifiers()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiers() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiershttp://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html *getQualifiersSize()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiersSize() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiershttp://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html Most mysterious. -- Lefty On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani hbut...@hortonworks.comwrote: I can see why you would rename. But this sentence is not correct: 'Hive enforces the predicate pushdown rules by these methods in the SemanticAnalyzer and JoinPPD classes:' It should be: Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD classes: (The implementation involves both predicate pushdown and analyzing join conditions) Sorry about this. So the link should say 'Hive Outer Join Behavior' regards, Harish. On Dec 10, 2013, at 2:01 PM, Lefty Leverenz leftylever...@gmail.com wrote: How's this? Hive Implementationhttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-HiveImplementation Also, I moved the link on the Design Docs pagehttps://cwiki.apache.org/confluence/display/Hive/DesignDocsfrom *Proposed* to *Other*. (It's called SQL Outer Join Predicate Pushdown Rules https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior which doesn't match the title, but seems okay because it's more descriptive.) -- Lefty On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani hbut...@hortonworks.comwrote: You are correct, it is plural. regards, Harish. On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com wrote: Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of singular, so should it be getQualifiedAliases() or am I missing something? trunk *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'* ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: * the comments for getQualifiedAliases function. ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: SetString aliases = getQualifiedAliases((JoinOperator) nd, owi ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242: // be pushed down per getQualifiedAliases ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: private SetString getQualifiedAliases(JoinOperator op, RowResolver rr) { -- Lefty On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani hbut...@hortonworks.comwrote: Looks good. Thanks for doing this. Minor point: *Rule 1:* During *QBJoinTree* construction in Plan Gen, the parse Join Condition logic applies this rule. *Rule 2:* During *JoinPPD* (Join Predicate Pushdown) the get Qualified Alias logic applies this rule. FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the SemanticAnalyzer and JoinPPD classes respectively. Writing these as separate words maybe confusing. You are better judge of how to represent this(quoted/bold etc.) regards, Harish. On Dec 9, 2013, at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidoc https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins doc https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying table
Re: doc on predicate pushdown in joins
getQualifiedAliases is a private method in JoinPPD. Maybe we should remove the section on Hive Implementation here. It is in the Design doc; this information only concerns developers. regards, Harish. On Dec 11, 2013, at 3:05 AM, Lefty Leverenz leftylever...@gmail.com wrote: Happy to fix the sentence and the link. I pointed out the name change just so you would review it, so please don't apologize! One more question: why am I not finding getQualifiedAliases() in the SemanticAnalyzer class? It turns up in OpProcFactory.java with javadoc comments, but I can't find it anywhere in the API docs -- not even in the index (Hive 0.12.0 API): getQMap() - Method in class org.apache.hadoop.hive.ql.QTestUtil getQualifiedName() - Method in class org.apache.hadoop.hive.serde2.typeinfo.TypeInfo String representing the qualified type name. getQualifiers() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiers getQualifiersSize() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiers Most mysterious. -- Lefty On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani hbut...@hortonworks.com wrote: I can see why you would rename. But this sentence is not correct: 'Hive enforces the predicate pushdown rules by these methods in the SemanticAnalyzer and JoinPPD classes:' It should be: Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD classes: (The implementation involves both predicate pushdown and analyzing join conditions) Sorry about this. So the link should say 'Hive Outer Join Behavior' regards, Harish. On Dec 10, 2013, at 2:01 PM, Lefty Leverenz leftylever...@gmail.com wrote: How's this? Hive Implementation Also, I moved the link on the Design Docs page from Proposed to Other. (It's called SQL Outer Join Predicate Pushdown Rules which doesn't match the title, but seems okay because it's more descriptive.) -- Lefty On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani hbut...@hortonworks.com wrote: You are correct, it is plural. regards, Harish. On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com wrote: Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of singular, so should it be getQualifiedAliases() or am I missing something? trunk grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn' ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: * the comments for getQualifiedAliases function. ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: SetString aliases = getQualifiedAliases((JoinOperator) nd, owi ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242: // be pushed down per getQualifiedAliases ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: private SetString getQualifiedAliases(JoinOperator op, RowResolver rr) { -- Lefty On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani hbut...@hortonworks.com wrote: Looks good. Thanks for doing this. Minor point: Rule 1: During QBJoinTree construction in Plan Gen, the parse Join Condition logic applies this rule. Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias logic applies this rule. FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the SemanticAnalyzer and JoinPPD classes respectively. Writing these as separate words maybe confusing. You are better judge of how to represent this(quoted/bold etc.) regards, Harish. On Dec 9, 2013, at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidochttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins dochttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying tablehttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions ? -- Lefty On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair the...@hortonworks.com wrote: :) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY
Re: doc on predicate pushdown in joins
Maybe we should remove the section on Hive Implementation here. It is in the Design doc; this information only concerns developers. But this is the Design doc (unless there's another one somewhere -- maybe attached to a JIRA ticket?) and it's in the Resources for Contributors part of the wiki, so it seems appropriate to me. I'll delete the implementation section if that's your preference. Here are the links again, with fixes: - Design Docshttps://cwiki.apache.org/confluence/display/Hive/DesignDocs (bottom of list) - Predicate Pushdown Ruleshttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-PredicatePushdownRules Speaking of JIRA tickets, is there one for this and should I add any version information? -- Lefty On Wed, Dec 11, 2013 at 7:59 AM, Harish Butani hbut...@hortonworks.comwrote: getQualifiedAliases is a private method in JoinPPD. Maybe we should remove the section on Hive Implementation here. It is in the Design doc; this information only concerns developers. regards, Harish. On Dec 11, 2013, at 3:05 AM, Lefty Leverenz leftylever...@gmail.com wrote: Happy to fix the sentence and the link. I pointed out the name change just so you would review it, so please don't apologize! One more question: why am I not finding getQualifiedAliases() in the SemanticAnalyzer class? It turns up in OpProcFactory.java with javadoc comments, but I can't find it anywhere in the API docs -- not even in the index (Hive 0.12.0 API http://hive.apache.org/docs/r0.12.0/api/): *getQMap()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html#getQMap() - Method in class org.apache.hadoop.hive.ql.QTestUtilhttp://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/ql/QTestUtil.html *getQualifiedName()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html#getQualifiedName() - Method in class org.apache.hadoop.hive.serde2.typeinfo.TypeInfohttp://hive.apache.org/docs/r0.12.0/api/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.html String representing the qualified type name.*getQualifiers()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiers() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiershttp://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html *getQualifiersSize()*http://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html#getQualifiersSize() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiershttp://hive.apache.org/docs/r0.12.0/api/org/apache/hive/service/cli/thrift/TTypeQualifiers.html Most mysterious. -- Lefty On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani hbut...@hortonworks.comwrote: I can see why you would rename. But this sentence is not correct: 'Hive enforces the predicate pushdown rules by these methods in the SemanticAnalyzer and JoinPPD classes:' It should be: Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD classes: (The implementation involves both predicate pushdown and analyzing join conditions) Sorry about this. So the link should say 'Hive Outer Join Behavior' regards, Harish. On Dec 10, 2013, at 2:01 PM, Lefty Leverenz leftylever...@gmail.com wrote: How's this? Hive Implementationhttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-HiveImplementation Also, I moved the link on the Design Docs pagehttps://cwiki.apache.org/confluence/display/Hive/DesignDocsfrom *Proposed* to *Other*. (It's called SQL Outer Join Predicate Pushdown Ruleshttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior which doesn't match the title, but seems okay because it's more descriptive.) -- Lefty On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani hbut...@hortonworks.comwrote: You are correct, it is plural. regards, Harish. On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com wrote: Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of singular, so should it be getQualifiedAliases() or am I missing something? trunk *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'* ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: * the comments for getQualifiedAliases function. ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: SetString aliases = getQualifiedAliases((JoinOperator) nd, owi ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242: // be pushed down per getQualifiedAliases ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: private SetString getQualifiedAliases(JoinOperator op, RowResolver rr) { -- Lefty On Mon, Dec 9, 2013 at
Re: doc on predicate pushdown in joins
I see. Let's leave it in. This is old code, hard to attribute to jiras: - The PPD code comes from: HIVE-279, HIVE-2337 - I cannot tell when the join condition parsing code was added. regards, Harish. On Dec 11, 2013, at 5:17 PM, Lefty Leverenz leftylever...@gmail.com wrote: Maybe we should remove the section on Hive Implementation here. It is in the Design doc; this information only concerns developers. But this is the Design doc (unless there's another one somewhere -- maybe attached to a JIRA ticket?) and it's in the Resources for Contributors part of the wiki, so it seems appropriate to me. I'll delete the implementation section if that's your preference. Here are the links again, with fixes: Design Docs (bottom of list) Predicate Pushdown Rules Speaking of JIRA tickets, is there one for this and should I add any version information? -- Lefty On Wed, Dec 11, 2013 at 7:59 AM, Harish Butani hbut...@hortonworks.com wrote: getQualifiedAliases is a private method in JoinPPD. Maybe we should remove the section on Hive Implementation here. It is in the Design doc; this information only concerns developers. regards, Harish. On Dec 11, 2013, at 3:05 AM, Lefty Leverenz leftylever...@gmail.com wrote: Happy to fix the sentence and the link. I pointed out the name change just so you would review it, so please don't apologize! One more question: why am I not finding getQualifiedAliases() in the SemanticAnalyzer class? It turns up in OpProcFactory.java with javadoc comments, but I can't find it anywhere in the API docs -- not even in the index (Hive 0.12.0 API): getQMap() - Method in class org.apache.hadoop.hive.ql.QTestUtil getQualifiedName() - Method in class org.apache.hadoop.hive.serde2.typeinfo.TypeInfo String representing the qualified type name. getQualifiers() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiers getQualifiersSize() - Method in class org.apache.hive.service.cli.thrift.TTypeQualifiers Most mysterious. -- Lefty On Tue, Dec 10, 2013 at 2:35 PM, Harish Butani hbut...@hortonworks.com wrote: I can see why you would rename. But this sentence is not correct: 'Hive enforces the predicate pushdown rules by these methods in the SemanticAnalyzer and JoinPPD classes:' It should be: Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD classes: (The implementation involves both predicate pushdown and analyzing join conditions) Sorry about this. So the link should say 'Hive Outer Join Behavior' regards, Harish. On Dec 10, 2013, at 2:01 PM, Lefty Leverenz leftylever...@gmail.com wrote: How's this? Hive Implementation Also, I moved the link on the Design Docs page from Proposed to Other. (It's called SQL Outer Join Predicate Pushdown Rules which doesn't match the title, but seems okay because it's more descriptive.) -- Lefty On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani hbut...@hortonworks.com wrote: You are correct, it is plural. regards, Harish. On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com wrote: Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of singular, so should it be getQualifiedAliases() or am I missing something? trunk grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn' ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: * the comments for getQualifiedAliases function. ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: SetString aliases = getQualifiedAliases((JoinOperator) nd, owi ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242: // be pushed down per getQualifiedAliases ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: private SetString getQualifiedAliases(JoinOperator op, RowResolver rr) { -- Lefty On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani hbut...@hortonworks.com wrote: Looks good. Thanks for doing this. Minor point: Rule 1: During QBJoinTree construction in Plan Gen, the parse Join Condition logic applies this rule. Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias logic applies this rule. FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the SemanticAnalyzer and JoinPPD classes respectively. Writing these as separate words maybe confusing. You are better judge of how to represent this(quoted/bold etc.) regards, Harish. On Dec 9, 2013, at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidochttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins
Re: doc on predicate pushdown in joins
Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of singular, so should it be getQualifiedAliases() or am I missing something? trunk *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'* ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: * the comments for getQualifiedAliases function. ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: SetString aliases = getQualifiedAliases((JoinOperator) nd, owi ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242: // be pushed down per getQualifiedAliases ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: private SetString getQualifiedAliases(JoinOperator op, RowResolver rr) { -- Lefty On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani hbut...@hortonworks.comwrote: Looks good. Thanks for doing this. Minor point: *Rule 1:* During *QBJoinTree* construction in Plan Gen, the parse Join Condition logic applies this rule. *Rule 2:* During *JoinPPD* (Join Predicate Pushdown) the get Qualified Alias logic applies this rule. FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the SemanticAnalyzer and JoinPPD classes respectively. Writing these as separate words maybe confusing. You are better judge of how to represent this(quoted/bold etc.) regards, Harish. On Dec 9, 2013, at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidochttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior is done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins doc https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying table https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions ? -- Lefty On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair the...@hortonworks.com wrote: :) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: doc on predicate pushdown in joins
You are correct, it is plural. regards, Harish. On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com wrote: Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of singular, so should it be getQualifiedAliases() or am I missing something? trunk grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn' ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: * the comments for getQualifiedAliases function. ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: SetString aliases = getQualifiedAliases((JoinOperator) nd, owi ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242:// be pushed down per getQualifiedAliases ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: private SetString getQualifiedAliases(JoinOperator op, RowResolver rr) { -- Lefty On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani hbut...@hortonworks.com wrote: Looks good. Thanks for doing this. Minor point: Rule 1: During QBJoinTree construction in Plan Gen, the parse Join Condition logic applies this rule. Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias logic applies this rule. FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the SemanticAnalyzer and JoinPPD classes respectively. Writing these as separate words maybe confusing. You are better judge of how to represent this(quoted/bold etc.) regards, Harish. On Dec 9, 2013, at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidochttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins dochttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying tablehttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions ? -- Lefty On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair the...@hortonworks.com wrote: :) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you
Re: doc on predicate pushdown in joins
How's this? Hive Implementationhttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-HiveImplementation Also, I moved the link on the Design Docs pagehttps://cwiki.apache.org/confluence/display/Hive/DesignDocsfrom *Proposed* to *Other*. (It's called SQL Outer Join Predicate Pushdown Ruleshttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior which doesn't match the title, but seems okay because it's more descriptive.) -- Lefty On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani hbut...@hortonworks.comwrote: You are correct, it is plural. regards, Harish. On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com wrote: Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of singular, so should it be getQualifiedAliases() or am I missing something? trunk *grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn'* ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: * the comments for getQualifiedAliases function. ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: SetString aliases = getQualifiedAliases((JoinOperator) nd, owi ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242: // be pushed down per getQualifiedAliases ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: private SetString getQualifiedAliases(JoinOperator op, RowResolver rr) { -- Lefty On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani hbut...@hortonworks.comwrote: Looks good. Thanks for doing this. Minor point: *Rule 1:* During *QBJoinTree* construction in Plan Gen, the parse Join Condition logic applies this rule. *Rule 2:* During *JoinPPD* (Join Predicate Pushdown) the get Qualified Alias logic applies this rule. FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the SemanticAnalyzer and JoinPPD classes respectively. Writing these as separate words maybe confusing. You are better judge of how to represent this(quoted/bold etc.) regards, Harish. On Dec 9, 2013, at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidoc https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins doc https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying table https://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions ? -- Lefty On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair the...@hortonworks.com wrote: :) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying,
Re: doc on predicate pushdown in joins
I can see why you would rename. But this sentence is not correct: 'Hive enforces the predicate pushdown rules by these methods in the SemanticAnalyzer and JoinPPD classes:' It should be: Hive enforces the rules by these methods in the SemanticAnalyzer and JoinPPD classes: (The implementation involves both predicate pushdown and analyzing join conditions) Sorry about this. So the link should say 'Hive Outer Join Behavior' regards, Harish. On Dec 10, 2013, at 2:01 PM, Lefty Leverenz leftylever...@gmail.com wrote: How's this? Hive Implementation Also, I moved the link on the Design Docs page from Proposed to Other. (It's called SQL Outer Join Predicate Pushdown Rules which doesn't match the title, but seems okay because it's more descriptive.) -- Lefty On Tue, Dec 10, 2013 at 7:27 AM, Harish Butani hbut...@hortonworks.com wrote: You are correct, it is plural. regards, Harish. On Dec 10, 2013, at 4:03 AM, Lefty Leverenz leftylever...@gmail.com wrote: Okay, then monospace with () after the method name is a good way to show them: parseJoinCondition() and getQualifiedAlias() ... but I only found the latter pluralized, instead of singular, so should it be getQualifiedAliases() or am I missing something? trunk grep -nr 'getQualifiedAlias' ./ql/src/java/* | grep -v 'svn' ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:221: * the comments for getQualifiedAliases function. ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:230: SetString aliases = getQualifiedAliases((JoinOperator) nd, owi ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:242: // be pushed down per getQualifiedAliases ./ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java:471: private SetString getQualifiedAliases(JoinOperator op, RowResolver rr) { -- Lefty On Mon, Dec 9, 2013 at 2:12 PM, Harish Butani hbut...@hortonworks.com wrote: Looks good. Thanks for doing this. Minor point: Rule 1: During QBJoinTree construction in Plan Gen, the parse Join Condition logic applies this rule. Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias logic applies this rule. FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the SemanticAnalyzer and JoinPPD classes respectively. Writing these as separate words maybe confusing. You are better judge of how to represent this(quoted/bold etc.) regards, Harish. On Dec 9, 2013, at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidochttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins dochttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying tablehttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions ? -- Lefty On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair the...@hortonworks.com wrote: :) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY
Re: doc on predicate pushdown in joins
The Outer Join Behavior wikidochttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins dochttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying tablehttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions ? -- Lefty On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair the...@hortonworks.com wrote: :) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: doc on predicate pushdown in joins
Looks great! Thanks for also linking to it from the other relevant docs! On Mon, Dec 9, 2013 at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidochttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins dochttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying tablehttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions ? -- Lefty On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair the...@hortonworks.com wrote: :) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: doc on predicate pushdown in joins
Looks good. Thanks for doing this. Minor point: Rule 1: During QBJoinTree construction in Plan Gen, the parse Join Condition logic applies this rule. Rule 2: During JoinPPD (Join Predicate Pushdown) the get Qualified Alias logic applies this rule. FYI 'parseJoinCondition' and 'getQualifiedAlias' are methods in the SemanticAnalyzer and JoinPPD classes respectively. Writing these as separate words maybe confusing. You are better judge of how to represent this(quoted/bold etc.) regards, Harish. On Dec 9, 2013, at 1:52 AM, Lefty Leverenz leftylever...@gmail.com wrote: The Outer Join Behavior wikidochttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavioris done, with links from the Design Docs https://cwiki.apache.org/confluence/display/Hive/DesignDocs page and the Joins dochttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinOptimization . Harish (or anyone else) would you please review the changes I made to the definition for Null Supplying tablehttps://cwiki.apache.org/confluence/display/Hive/OuterJoinBehavior#OuterJoinBehavior-Definitions ? -- Lefty On Mon, Dec 2, 2013 at 6:46 PM, Thejas Nair the...@hortonworks.com wrote: :) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: doc on predicate pushdown in joins
Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: doc on predicate pushdown in joins
:) On Mon, Dec 2, 2013 at 6:18 PM, Lefty Leverenz leftylever...@gmail.com wrote: Easy as 3.14159 (I can take a hint.) -- Lefty On Mon, Dec 2, 2013 at 5:34 PM, Thejas Nair the...@hortonworks.com wrote: FYI, Harish has a written a very nice doc describing predicate push down rules for join. I have attached it to the design doc page. It will be very useful for anyone looking at joins. https://cwiki.apache.org/confluence/download/attachments/27362075/OuterJoinBehavior.html (any help converting it to wiki format from html is welcome!). -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.