+1 I tested the source and Hadoop 2.4 release. Checksums and
signatures are OK. Compiles fine with Java 8 on OS X. Tests... don't
fail any more than usual.
FWIW I've also been using the 1.1.0-SNAPSHOT for some time in another
project and have encountered no problems.
I notice that the 1.1.0
Hey Sean,
The reason there are no longer CDH-specific builds is that all newer
versions of CDH and HDP work with builds for the upstream Hadoop
projects. I dropped CDH4 in favor of a newer Hadoop version (2.4) and
the Hadoop-without-Hive (also 2.4) build.
For MapR - we can't officially post
(Copying my reply since I don't know if it goes to the mailing list)
Great, thanks for explaining the reasoning. You're saying these aren't
going into the final release? I think that moots any issue surrounding
distributing them then.
This is all I know of from the ASF:
The reason I'm asking about the columnar compressed format is that
there are some problems for which Parquet is not practical.
Can you elaborate?
Sure.
- Organization or co has no Hadoop, but significant investment in some
other NoSQL store.
- Need to efficiently add a new column to
Personally I'd actually consider putting CDH4 back if there are still users on
it. It's always better to be inclusive, and the convenience of a one-click
download is high. Do we have a sense on what % of CDH users still use CDH4?
Matei
On August 28, 2014 at 11:31:13 PM, Sean Owen
Yeah, we can't/won't post MapR binaries on the ASF web space for the
release. However, I have been linking to them (at their request) with
a clear identifier that it is an incompatible license and a 3rd party
build.
The only vendor specific build property we provide is compatibility
with
I have preliminary patch against spark1.0.2, which is attached to spark-2706.
Now I am working on supporting both hive-0.12 and hive-0.13.1 with
non-intrusive way (not breaking any existing hive-0.12 when introduce
supporting new version). I will attach a proposal to solve multi-version
support
On Fri, Aug 29, 2014 at 7:42 AM, Patrick Wendell pwend...@gmail.com wrote:
In terms of vendor support for this approach - In the early days
Cloudera asked us to add CDH4 repository and more recently Pivotal and
MapR also asked us to allow linking against their hadoop-client
libraries. So we've
FWIW we use CDH4 extensively and would very much appreciate having a
prebuilt version of Spark for it.
We're doing a CDH 4.4 to 4.7 upgrade across all the clusters now and have
plans for a 5.x transition after that.
On Aug 28, 2014 11:57 PM, Sean Owen so...@cloudera.com wrote:
On Fri, Aug 29,
Hi,
My requirement is to run Spark on Yarn without using the script
spark-submit.
I have a servlet and a tomcat server. As and when request comes, it creates
a new SC and keeps it alive for the further requests, I ma setting my
master in sparkConf
as sparkConf.setMaster(yarn-cluster)
but the
including u...@spark.apache.org.
On Fri, Aug 29, 2014 at 2:03 PM, Archit Thakur archit279tha...@gmail.com
wrote:
Hi,
My requirement is to run Spark on Yarn without using the script
spark-submit.
I have a servlet and a tomcat server. As and when request comes, it
creates a new SC and
i suspect there are more cdh4 than cdh5 clusters. most people plan to move
to cdh5 within say 6 months.
On Fri, Aug 29, 2014 at 3:57 AM, Andrew Ash and...@andrewash.com wrote:
FWIW we use CDH4 extensively and would very much appreciate having a
prebuilt version of Spark for it.
We're doing
Archit
We are using yarn-cluster mode , and calling spark via Client class
directly from servlet server. It works fine.
To establish a communication channel to give further requests,
It should be possible with yarn client, but not with yarn server. Yarn
client mode, spark driver
reminder: this is happening right now. jenkins is currently in quiet
mode, and in ~30 minutes, will be briefly going down.
On Thu, Aug 28, 2014 at 1:03 PM, shane knapp skn...@berkeley.edu wrote:
as with all software upgrades, sometimes things don't always work as
expected.
a recent
Cheng Lian-2 wrote
You can just start the work :)
Given 100+ contributors, starting work without a JIRA issue assigned to you
could lead to duplication of effort by well meaning people that have no idea
they are working on the same issue. This does happen and I don't think it's
a good thing.
this is done.
On Fri, Aug 29, 2014 at 7:32 AM, shane knapp skn...@berkeley.edu wrote:
reminder: this is happening right now. jenkins is currently in quiet
mode, and in ~30 minutes, will be briefly going down.
On Thu, Aug 28, 2014 at 1:03 PM, shane knapp skn...@berkeley.edu wrote:
as
We just used CDH 4.7 for our production cluster. And I believe we won't use CDH
5 in the next year.
Sent from my iPhone
On 2014年8月29日, at 14:39, Matei Zaharia matei.zaha...@gmail.com wrote:
Personally I'd actually consider putting CDH4 back if there are still users
on it. It's always
Okay I'll plan to add cdh4 binary as well for the final release!
---
sent from my phone
On Aug 29, 2014 8:26 AM, Ye Xianjin advance...@gmail.com wrote:
We just used CDH 4.7 for our production cluster. And I believe we won't
use CDH 5 in the next year.
Sent from my iPhone
On 2014年8月29日, at
There were several formatting and typographical errors in the SQL docs that
I've fixed in this PR https://github.com/apache/spark/pull/2201. Dunno if
we want to roll that into the release.
On Fri, Aug 29, 2014 at 12:17 PM, Patrick Wendell pwend...@gmail.com
wrote:
Okay I'll plan to add cdh4
Hey Nicholas,
Thanks for this, we can merge in doc changes outside of the actual
release timeline, so we'll make sure to loop those changes in before
we publish the final 1.1 docs.
- Patrick
On Fri, Aug 29, 2014 at 9:24 AM, Nicholas Chammas
nicholas.cham...@gmail.com wrote:
There were several
Hi Josh,
Can you add me as well?
Thanks,
Ron
On Aug 28, 2014, at 3:56 PM, Josh Rosen rosenvi...@gmail.com wrote:
A JIRA admin needs to add you to the ‘’Contributors” role group in order to
allow you to assign issues to yourself. I’ve added this email address to
that group, so you should
Added you; you should be set!
If anyone else wants me to add them, please email me off-list so that we don’t
end up flooding the dev list with replies. Thanks!
On August 29, 2014 at 10:03:41 AM, Ron's Yahoo! (zlgonza...@yahoo.com) wrote:
Hi Josh,
Can you add me as well?
Thanks,
Ron
[Let me know if I should be posting these comments in a different thread.]
Should the default Spark version in spark-ec2
https://github.com/apache/spark/blob/e1535ad3c6f7400f2b7915ea91da9c60510557ba/ec2/spark_ec2.py#L86
be updated for this release?
Nick
On Fri, Aug 29, 2014 at 12:55 PM,
i have always found the 'Rebuild' plugin super useful:
https://wiki.jenkins-ci.org/display/JENKINS/Rebuild+Plugin
this is installed and enables. enjoy!
shane
Oh darn - I missed this update. GRR, unfortunately I think this means
I'll need to cut a new RC. Thanks for catching this Nick.
On Fri, Aug 29, 2014 at 10:18 AM, Nicholas Chammas
nicholas.cham...@gmail.com wrote:
[Let me know if I should be posting these comments in a different thread.]
Should
Hi, Devl!
I got the same problem.
You can try to upgrade your scala plugins to 0.41.2
It works on my mac.
On Aug 12, 2014, at 15:19, Devl Devel devl.developm...@gmail.com wrote:
When compiling the master checkout of spark. The Intellij compile fails
with:
Error:(45, 8) not found:
In our internal projects we use this bit of code in the maven pom to
create a properties file with build information (sorry for the messy
indentation). Then we have code that reads this property file
somewhere and provides that info. This should make it easier to not
have to change version numbers
Just noticed one thing: although --with-hive is deprecated by -Phive,
make-distribution.sh still relies on $SPARK_HIVE (which was controlled by
--with-hive) to determine whether to include datanucleus jar files. This
means we have to do something like SPARK_HIVE=true ./make-distribution.sh
... to
+1. Validated several custom analysis pipelines on a private cluster in
standalone mode. Tested new PySpark support for arbitrary Hadoop input
formats, works great!
-- Jeremy
--
View this message in context:
Hi,
We are developing an app in Spring in which we are using Cassandra and
calling datastax api's from Java to query it. The internal library is
responsible for calling cassandra and other data sources like RDS. We are
calling several client API's from Spark provided by the client-jar to
perform
In some cases IntelliJ's Scala compiler can't compile valid Scala
source files. Hopefully they fix (or have fixed) this in a newer
version.
- Patrick
On Fri, Aug 29, 2014 at 11:38 AM, Yi Tian tianyi.asiai...@gmail.com wrote:
Hi, Devl!
I got the same problem.
You can try to upgrade your
31 matches
Mail list logo
