Thanks, that was the command. :thumbsup: On Mon, Apr 4, 2016 at 6:28 PM Jakob Odersky <ja...@odersky.com> wrote:
> I just found out how the hash is calculated: > > gpg --print-md sha512 <spark-archive>.tgz > > you can use that to check if the resulting output matches the contents > of <spark-archive>.tgz.sha > > On Mon, Apr 4, 2016 at 3:19 PM, Jakob Odersky <ja...@odersky.com> wrote: > > The published hash is a SHA512. > > > > You can verify the integrity of the packages by running `sha512sum` on > > the archive and comparing the computed hash with the published one. > > Unfortunately however, I don't know what tool is used to generate the > > hash and I can't reproduce the format, so I ended up manually > > comparing the hashes. > > > > On Mon, Apr 4, 2016 at 2:39 PM, Nicholas Chammas > > <nicholas.cham...@gmail.com> wrote: > >> An additional note: The Spark packages being served off of CloudFront > (i.e. > >> the “direct download” option on spark.apache.org) are also corrupt. > >> > >> Btw what’s the correct way to verify the SHA of a Spark package? I’ve > tried > >> a few commands on working packages downloaded from Apache mirrors, but I > >> can’t seem to reproduce the published SHA for > spark-1.6.1-bin-hadoop2.6.tgz. > >> > >> > >> On Mon, Apr 4, 2016 at 11:45 AM Ted Yu <yuzhih...@gmail.com> wrote: > >>> > >>> Maybe temporarily take out the artifacts on S3 before the root cause is > >>> found. > >>> > >>> On Thu, Mar 24, 2016 at 7:25 AM, Nicholas Chammas > >>> <nicholas.cham...@gmail.com> wrote: > >>>> > >>>> Just checking in on this again as the builds on S3 are still broken. > :/ > >>>> > >>>> Could it have something to do with us moving release-build.sh? > >>>> > >>>> > >>>> On Mon, Mar 21, 2016 at 1:43 PM Nicholas Chammas > >>>> <nicholas.cham...@gmail.com> wrote: > >>>>> > >>>>> Is someone going to retry fixing these packages? It's still a > problem. > >>>>> > >>>>> Also, it would be good to understand why this is happening. > >>>>> > >>>>> On Fri, Mar 18, 2016 at 6:49 PM Jakob Odersky <ja...@odersky.com> > wrote: > >>>>>> > >>>>>> I just realized you're using a different download site. Sorry for > the > >>>>>> confusion, the link I get for a direct download of Spark 1.6.1 / > >>>>>> Hadoop 2.6 is > >>>>>> http://d3kbcqa49mib13.cloudfront.net/spark-1.6.1-bin-hadoop2.6.tgz > >>>>>> > >>>>>> On Fri, Mar 18, 2016 at 3:20 PM, Nicholas Chammas > >>>>>> <nicholas.cham...@gmail.com> wrote: > >>>>>> > I just retried the Spark 1.6.1 / Hadoop 2.6 download and got a > >>>>>> > corrupt ZIP > >>>>>> > file. > >>>>>> > > >>>>>> > Jakob, are you sure the ZIP unpacks correctly for you? Is it the > same > >>>>>> > Spark > >>>>>> > 1.6.1/Hadoop 2.6 package you had a success with? > >>>>>> > > >>>>>> > On Fri, Mar 18, 2016 at 6:11 PM Jakob Odersky <ja...@odersky.com> > >>>>>> > wrote: > >>>>>> >> > >>>>>> >> I just experienced the issue, however retrying the download a > second > >>>>>> >> time worked. Could it be that there is some load balancer/cache > in > >>>>>> >> front of the archive and some nodes still serve the corrupt > >>>>>> >> packages? > >>>>>> >> > >>>>>> >> On Fri, Mar 18, 2016 at 8:00 AM, Nicholas Chammas > >>>>>> >> <nicholas.cham...@gmail.com> wrote: > >>>>>> >> > I'm seeing the same. :( > >>>>>> >> > > >>>>>> >> > On Fri, Mar 18, 2016 at 10:57 AM Ted Yu <yuzhih...@gmail.com> > >>>>>> >> > wrote: > >>>>>> >> >> > >>>>>> >> >> I tried again this morning : > >>>>>> >> >> > >>>>>> >> >> $ wget > >>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> > https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz > >>>>>> >> >> --2016-03-18 07:55:30-- > >>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> > https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz > >>>>>> >> >> Resolving s3.amazonaws.com... 54.231.19.163 > >>>>>> >> >> ... > >>>>>> >> >> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz > >>>>>> >> >> > >>>>>> >> >> gzip: stdin: unexpected end of file > >>>>>> >> >> tar: Unexpected EOF in archive > >>>>>> >> >> tar: Unexpected EOF in archive > >>>>>> >> >> tar: Error is not recoverable: exiting now > >>>>>> >> >> > >>>>>> >> >> On Thu, Mar 17, 2016 at 8:57 AM, Michael Armbrust > >>>>>> >> >> <mich...@databricks.com> > >>>>>> >> >> wrote: > >>>>>> >> >>> > >>>>>> >> >>> Patrick reuploaded the artifacts, so it should be fixed now. > >>>>>> >> >>> > >>>>>> >> >>> On Mar 16, 2016 5:48 PM, "Nicholas Chammas" > >>>>>> >> >>> <nicholas.cham...@gmail.com> > >>>>>> >> >>> wrote: > >>>>>> >> >>>> > >>>>>> >> >>>> Looks like the other packages may also be corrupt. I’m > getting > >>>>>> >> >>>> the > >>>>>> >> >>>> same > >>>>>> >> >>>> error for the Spark 1.6.1 / Hadoop 2.4 package. > >>>>>> >> >>>> > >>>>>> >> >>>> > >>>>>> >> >>>> > >>>>>> >> >>>> > >>>>>> >> >>>> > https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.4.tgz > >>>>>> >> >>>> > >>>>>> >> >>>> Nick > >>>>>> >> >>>> > >>>>>> >> >>>> > >>>>>> >> >>>> On Wed, Mar 16, 2016 at 8:28 PM Ted Yu <yuzhih...@gmail.com > > > >>>>>> >> >>>> wrote: > >>>>>> >> >>>>> > >>>>>> >> >>>>> On Linux, I got: > >>>>>> >> >>>>> > >>>>>> >> >>>>> $ tar zxf spark-1.6.1-bin-hadoop2.6.tgz > >>>>>> >> >>>>> > >>>>>> >> >>>>> gzip: stdin: unexpected end of file > >>>>>> >> >>>>> tar: Unexpected EOF in archive > >>>>>> >> >>>>> tar: Unexpected EOF in archive > >>>>>> >> >>>>> tar: Error is not recoverable: exiting now > >>>>>> >> >>>>> > >>>>>> >> >>>>> On Wed, Mar 16, 2016 at 5:15 PM, Nicholas Chammas > >>>>>> >> >>>>> <nicholas.cham...@gmail.com> wrote: > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> > https://s3.amazonaws.com/spark-related-packages/spark-1.6.1-bin-hadoop2.6.tgz > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> Does anyone else have trouble unzipping this? How did this > >>>>>> >> >>>>>> happen? > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> What I get is: > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> $ gzip -t spark-1.6.1-bin-hadoop2.6.tgz > >>>>>> >> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: unexpected end of > file > >>>>>> >> >>>>>> gzip: spark-1.6.1-bin-hadoop2.6.tgz: uncompress failed > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> Seems like a strange type of problem to come across. > >>>>>> >> >>>>>> > >>>>>> >> >>>>>> Nick > >>>>>> >> >>>>> > >>>>>> >> >>>>> > >>>>>> >> >> > >>>>>> >> > > >>> > >>> > >> >
