Adding new build modules is pretty high overhead, so if this is a case where a small amount of duplicated code could get rid of the dependency, that could also be a good short-term option.
- Patrick On Mon, Jul 14, 2014 at 2:15 PM, Matei Zaharia <matei.zaha...@gmail.com> wrote: > Yeah, I'd just add a spark-util that has these things. > > Matei > > On Jul 14, 2014, at 1:04 PM, Michael Armbrust <mich...@databricks.com> > wrote: > > Yeah, sadly this dependency was introduced when someone consolidated the > logging infrastructure. However, the dependency should be very small and > thus easy to remove, and I would like catalyst to be usable outside of > Spark. A pull request to make this possible would be welcome. > > Ideally, we'd create some sort of spark common package that has things like > logging. That way catalyst could depend on that, without pulling in all of > Hadoop, etc. Maybe others have opinions though, so I'm cc-ing the dev list. > > > On Mon, Jul 14, 2014 at 12:21 AM, Yanbo Liang <yanboha...@gmail.com> wrote: >> >> Make Catalyst independent of Spark is the goal of Catalyst, maybe need >> time and evolution. >> I awared that package org.apache.spark.sql.catalyst.util embraced >> org.apache.spark.util.{Utils => SparkUtils}, >> so that Catalyst has a dependency on Spark core. >> I'm not sure whether it will be replaced by other component independent of >> Spark in later release. >> >> >> 2014-07-14 11:51 GMT+08:00 Aniket Bhatnagar <aniket.bhatna...@gmail.com>: >> >>> As per the recent presentation given in Scala days >>> (http://people.apache.org/~marmbrus/talks/SparkSQLScalaDays2014.pdf), it was >>> mentioned that Catalyst is independent of Spark. But on inspecting pom.xml >>> of sql/catalyst module, it seems it has a dependency on Spark Core. Any >>> particular reason for the dependency? I would love to use Catalyst outside >>> Spark >>> >>> (reposted as previous email bounced. Sorry if this is a duplicate). >> >> > >