Re: Spark on Oracle available as an Apache licensed open source repo

2022-01-14 Thread Harish Butani
or destruction of data or any other property which may arise from > relying on this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such > loss, damage or destruction. > > > > On Fri, 14

Spark on Oracle available as an Apache licensed open source repo

2022-01-13 Thread Harish Butani
gt; on your Spark cluster. More details can be found in our blog <https://hbutani.github.io/blogs/blog/Spark_on_Oracle_Blog.html> and the project wiki. <https://github.com/oracle/spark-oracle/wiki> regards, Harish Butani

Spark on your Oracle Data Warehouse

2021-03-23 Thread Harish Butani
, comments from the Spark community. regards, Harish Butani.

Spark SQL Macros

2021-02-19 Thread Harish Butani
utani/spark-sql-macros/wiki/Spark_SQL_Macro_examples) provides even more examples. regards, Harish Butani.

Re: Design patterns involving Spark

2017-04-12 Thread Harish Butani
BTW, we now support OLAP functionality natively in spark w/o the need for Druid, through our Spark native BI platform(SNAP): https://www.linkedin.com/pulse/integrated-business-intelligence-big-data-stacks-harish-butani - we provide SQL commands to: create star schema, create olap index, and

Re: Spark + Druid

2015-09-18 Thread Harish Butani
Hi, I have just posted a Blog on this: https://www.linkedin.com/pulse/combining-druid-spark-interactive-flexible-analytics-scale-butani regards, Harish Butani. On Tue, Sep 1, 2015 at 11:46 PM, Paolo Platter wrote: > Fantastic!!! I will look into that and I hope to contribute > &

Spark + Druid

2015-09-01 Thread Harish Butani
ign document, which also describes a benchmark of representative queries on the TPCH dataset. Looking for folks who would be willing to try this out and/or contribute. regards, Harish Butani.

Re: Data frames select and where clause dependency

2015-07-20 Thread Harish Butani
Yes via: org.apache.spark.sql.catalyst.optimizer.ColumnPruning See DefaultOptimizer.batches for list of logical rewrites. You can see the optimized plan by printing: df.queryExecution.optimizedPlan On Mon, Jul 20, 2015 at 5:22 PM, Mohammed Guller wrote: > Michael, > > How would the Catalyst o

Re: Joda Time best practice?

2015-07-20 Thread Harish Butani
Can you post details on how to reproduce the NPE On Mon, Jul 20, 2015 at 1:19 PM, algermissen1971 wrote: > Hi Harish, > > On 20 Jul 2015, at 20:37, Harish Butani wrote: > > > Hey Jan, > > > > Can you provide more details on the serialization and cache issues. &

Re: Joda Time best practice?

2015-07-20 Thread Harish Butani
Hey Jan, Can you provide more details on the serialization and cache issues. If you are looking for datetime functionality with spark-sql please consider: https://github.com/SparklineData/spark-datetime It provides a simple way to combine joda datetime expressions with spark sql. regards, Haris

Re: [SPARK-SQL] Window Functions optimization

2015-07-13 Thread Harish Butani
Just once. You can see this by printing the optimized logical plan. You will see just one repartition operation. So do: val df = sql("your sql...") println(df.queryExecution.analyzed) On Mon, Jul 13, 2015 at 6:37 AM, Hao Ren wrote: > Hi, > > I would like to know: Is there any optimization has b

Re: Spark query

2015-07-08 Thread Harish Butani
try the spark-datetime package: https://github.com/SparklineData/spark-datetime Follow this example https://github.com/SparklineData/spark-datetime#a-basic-example to get the different attributes of a DateTime. On Wed, Jul 8, 2015 at 9:11 PM, prosp4300 wrote: > As mentioned in Spark sQL programm