Re: Tungsten's Vectorized Execution
Yes that's exactly the reason. On Sat, May 23, 2015 at 12:37 AM, Yijie Shen henry.yijies...@gmail.com wrote: Davies and Reynold, Glad to hear about the status. I’ve seen [SPARK-7813](https://issues.apache.org/jira/browse/SPARK-7813) and watching it now. If I understand correctly, it’s aimed at moving CodeGenerator’s expressionEvaluator’s code-gen logic into each expressions’ eval() and eliminating to chose between row evaluation methods in Physical Operators? What’s the reason motives this refactoring job? to use code-gen version aggressively in evaluation? On May 22, 2015 at 3:05:24 PM, Xin Reynold (r...@databricks.com) wrote: Yijie, As Davies said, it will take us a while to get to vectorized execution. However, before that, we are going to refactor code generation to push it into each expression: https://issues.apache.org/jira/browse/SPARK-7813 Once this one is in (probably in the next 2 or 3 weeks), there will be lots of expressions to create code-gen versions, and it'd be great to get as much help as possible from the community. On Thu, May 21, 2015 at 1:59 PM, Davies Liu dav...@databricks.com wrote: We have not start to prototype the vectorized one yet, will evaluated in 1.5 and may targeted for 1.6. We're glad to hear some feedback/suggestions/comments from your side! On Thu, May 21, 2015 at 9:37 AM, Yijie Shen henry.yijies...@gmail.com wrote: Hi all, I’ve seen the Blog of Project Tungsten here, it sounds awesome to me! I’ve also noticed there is a plan to change the code generation from record-at-a-time evaluation to a vectorized one, which interests me most. What’s the status of vectorized evaluation? Is this an inner effort of Databricks or welcome to be involved? Since I’ve done similar stuffs on Spark SQL, I would like to get involved if that’s possible. Yours, Yijie - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: Tungsten's Vectorized Execution
Davies and Reynold, Glad to hear about the status. I’ve seen [SPARK-7813](https://issues.apache.org/jira/browse/SPARK-7813) and watching it now. If I understand correctly, it’s aimed at moving CodeGenerator’s expressionEvaluator’s code-gen logic into each expressions’ eval() and eliminating to chose between row evaluation methods in Physical Operators? What’s the reason motives this refactoring job? to use code-gen version aggressively in evaluation? On May 22, 2015 at 3:05:24 PM, Xin Reynold (r...@databricks.com) wrote: Yijie, As Davies said, it will take us a while to get to vectorized execution. However, before that, we are going to refactor code generation to push it into each expression: https://issues.apache.org/jira/browse/SPARK-7813 Once this one is in (probably in the next 2 or 3 weeks), there will be lots of expressions to create code-gen versions, and it'd be great to get as much help as possible from the community. On Thu, May 21, 2015 at 1:59 PM, Davies Liu dav...@databricks.com wrote: We have not start to prototype the vectorized one yet, will evaluated in 1.5 and may targeted for 1.6. We're glad to hear some feedback/suggestions/comments from your side! On Thu, May 21, 2015 at 9:37 AM, Yijie Shen henry.yijies...@gmail.com wrote: Hi all, I’ve seen the Blog of Project Tungsten here, it sounds awesome to me! I’ve also noticed there is a plan to change the code generation from record-at-a-time evaluation to a vectorized one, which interests me most. What’s the status of vectorized evaluation? Is this an inner effort of Databricks or welcome to be involved? Since I’ve done similar stuffs on Spark SQL, I would like to get involved if that’s possible. Yours, Yijie - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: Tungsten's Vectorized Execution
Yijie, As Davies said, it will take us a while to get to vectorized execution. However, before that, we are going to refactor code generation to push it into each expression: https://issues.apache.org/jira/browse/SPARK-7813 Once this one is in (probably in the next 2 or 3 weeks), there will be lots of expressions to create code-gen versions, and it'd be great to get as much help as possible from the community. On Thu, May 21, 2015 at 1:59 PM, Davies Liu dav...@databricks.com wrote: We have not start to prototype the vectorized one yet, will evaluated in 1.5 and may targeted for 1.6. We're glad to hear some feedback/suggestions/comments from your side! On Thu, May 21, 2015 at 9:37 AM, Yijie Shen henry.yijies...@gmail.com wrote: Hi all, I’ve seen the Blog of Project Tungsten here, it sounds awesome to me! I’ve also noticed there is a plan to change the code generation from record-at-a-time evaluation to a vectorized one, which interests me most. What’s the status of vectorized evaluation? Is this an inner effort of Databricks or welcome to be involved? Since I’ve done similar stuffs on Spark SQL, I would like to get involved if that’s possible. Yours, Yijie - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: Tungsten's Vectorized Execution
We have not start to prototype the vectorized one yet, will evaluated in 1.5 and may targeted for 1.6. We're glad to hear some feedback/suggestions/comments from your side! On Thu, May 21, 2015 at 9:37 AM, Yijie Shen henry.yijies...@gmail.com wrote: Hi all, I’ve seen the Blog of Project Tungsten here, it sounds awesome to me! I’ve also noticed there is a plan to change the code generation from record-at-a-time evaluation to a vectorized one, which interests me most. What’s the status of vectorized evaluation? Is this an inner effort of Databricks or welcome to be involved? Since I’ve done similar stuffs on Spark SQL, I would like to get involved if that’s possible. Yours, Yijie - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
