Thanks gents.
Removal of 'clean package -U' made the difference.
On Tue, Oct 27, 2015 at 6:39 PM, Todd Nist wrote:
> I issued the same basic command and it worked fine.
>
> RADTech-MBP:spark $ ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn
> -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Ph
I issued the same basic command and it worked fine.
RADTech-MBP:spark $ ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn
-Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests
Which created: spark-1.6.0-SNAPSHOT-bin-hadoop-2.6.tgz in the root
directory of the project.
FW
I used the following command:
make-distribution.sh --name custom-spark --tgz -Phadoop-2.4 -Phive
-Phive-thriftserver -Pyarn
spark-1.6.0-SNAPSHOT-bin-custom-spark.tgz was generated (with patch from
SPARK-11348)
Can you try above command ?
Thanks
On Tue, Oct 27, 2015 at 7:03 AM, Kayode Odeyemi w
Seems the build and directory structure in dist is similar to the .gz file
downloaded from the
downloads page. Can the dist directory be used as is?
On Tue, Oct 27, 2015 at 4:03 PM, Kayode Odeyemi wrote:
> Ted, I switched to this:
>
> ./make-distribution.sh --name spark-latest --tgz -Dhadoop.ver
Ted, I switched to this:
./make-distribution.sh --name spark-latest --tgz -Dhadoop.version=2.6.0
-Phadoop-2.6 -Phive -Phive-thriftserver -Pyarn -DskipTests clean package -U
Same error. No .gz file. Here's the bottom output log:
+ rm -rf /home/emperor/javaprojects/spark/dist
+ mkdir -p /home/empe
Can you try the same command shown in the pull request ?
Thanks
> On Oct 27, 2015, at 12:40 AM, Kayode Odeyemi wrote:
>
> Thank you.
>
> But I'm getting same warnings and it's still preventing the archive from
> being generated.
>
> I've ran this on both OSX Lion and Ubuntu 12. Same error. N
Thank you.
But I'm getting same warnings and it's still preventing the archive from
being generated.
I've ran this on both OSX Lion and Ubuntu 12. Same error. No .gz file
On Mon, Oct 26, 2015 at 9:10 PM, Ted Yu wrote:
> Looks like '-Pyarn' was missing in your command.
>
> On Mon, Oct 26, 2015
Looks like '-Pyarn' was missing in your command.
On Mon, Oct 26, 2015 at 12:06 PM, Kayode Odeyemi wrote:
> I used this command which is synonymous to what you have:
>
> ./make-distribution.sh --name spark-latest --tgz --mvn mvn
> -Dhadoop.version=2.6.0 -Phadoop-2.6 -Phive -Phive-thriftserver -Ds
I used this command which is synonymous to what you have:
./make-distribution.sh --name spark-latest --tgz --mvn mvn
-Dhadoop.version=2.6.0 -Phadoop-2.6 -Phive -Phive-thriftserver -DskipTests
clean package -U
But I still see WARNINGS like this in the output and no .gz file created:
cp: /usr/loca
If you use the command shown in:
https://github.com/apache/spark/pull/9281
You should have got the following:
./dist/python/test_support/sql/parquet_partitioned/year=2014/month=9/day=1/part-r-8.gz.parquet
./dist/python/test_support/sql/parquet_partitioned/year=2015/month=9/day=1/part-r-7.
I see a lot of stuffs like this after the a successful maven build:
cp: /usr/local/spark-latest/spark-[WARNING] See
http://docs.codehaus.org/display/MAVENUSER/Shade+Plugin-bin-spark-latest/python/test_support/sql/parquet_partitioned/year=2014/month=9/day=1/
part-r-8.gz.parquet: No such file or
In 1.4 ./make_distribution produces a .tgz file in the root directory (same
directory that make_distribution is in)
On Mon, Oct 26, 2015 at 8:46 AM, Kayode Odeyemi wrote:
> Hi,
>
> The ./make_distribution task completed. However, I can't seem to locate the
> .tar.gz file.
>
> Where does Spark
Hi,
The ./make_distribution task completed. However, I can't seem to locate the
.tar.gz file.
Where does Spark save this? or should I just work with the dist directory?
On Fri, Oct 23, 2015 at 4:23 PM, Kayode Odeyemi wrote:
> I saw this when I tested manually (without ./make-distribution)
>
>
I saw this when I tested manually (without ./make-distribution)
Detected Maven Version: 3.2.2 is not in the allowed range 3.3.3.
So I simply upgraded maven to 3.3.3.
Resolved. Thanks
On Fri, Oct 23, 2015 at 3:17 PM, Sean Owen wrote:
> This doesn't show the actual error output from Maven. I ha
This doesn't show the actual error output from Maven. I have a strong
guess that you haven't set MAVEN_OPTS to increase the memory Maven can
use.
On Fri, Oct 23, 2015 at 6:14 AM, Kayode Odeyemi wrote:
> Hi,
>
> I can't seem to get a successful maven build. Please see command output
> below:
>
> b
Hi,
I can't seem to get a successful maven build. Please see command output
below:
bash-3.2$ ./make-distribution.sh --name spark-latest --tgz --mvn mvn
-Dhadoop.version=2.7.0 -Phadoop-2.7 -Phive -Phive-thriftserver -DskipTests
clean package
+++ dirname ./make-distribution.sh
++ cd .
++ pwd
+ SPAR
16 matches
Mail list logo