Re: [Discuss] Exclude Spark from Kylin's binary package

[ShaoFeng Shi]

Thanks for billy's comment; A JIRA is created for this change:
https://issues.apache.org/jira/browse/KYLIN-3571

Billy Liu  于2018年9月18日周二 下午10:35写道：

> I like the idea to remove the built-in spark distribution. I think
> Kylin could work well with all spark 2.1+
>
> With Warm regards
>
> Billy Liu
>
> ShaoFeng Shi  于2018年9月18日周二 下午5:35写道：
> >
> > Hello,
> >
> > Today Kylin's binary package ships an Apache Spark binary, the total
> package is 297 MB in v2.5.0; I noticed that if the package is above 300 MB,
> it would be rejected by Apache SVN. Now in the whole package, Spark takes
> about 200 MB. Later when we upgrade to a higher version, the package can be
> bigger. Without spark, the package is around 100 MB.
> >
> > Since we don't have customization to Spark, we can provide a
> download-spark.sh script for the user to easily download. If the user
> already has that in local, he just needs to copy or create a symbolic link
> in Kylin folder.
> >
> > This will reduce 2/3 package size, and save time/bandwidth for
> distributing Kylin. If you see a problem this change may bring, please
> complain.
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi 史少锋
> >
>


-- 
Best regards,

Shaofeng Shi 史少锋

Re: [Discuss] Exclude Spark from Kylin's binary package

[Billy Liu]

I like the idea to remove the built-in spark distribution. I think
Kylin could work well with all spark 2.1+

With Warm regards

Billy Liu

ShaoFeng Shi  于2018年9月18日周二 下午5:35写道：
>
> Hello,
>
> Today Kylin's binary package ships an Apache Spark binary, the total package 
> is 297 MB in v2.5.0; I noticed that if the package is above 300 MB, it would 
> be rejected by Apache SVN. Now in the whole package, Spark takes about 200 
> MB. Later when we upgrade to a higher version, the package can be bigger. 
> Without spark, the package is around 100 MB.
>
> Since we don't have customization to Spark, we can provide a 
> download-spark.sh script for the user to easily download. If the user already 
> has that in local, he just needs to copy or create a symbolic link in Kylin 
> folder.
>
> This will reduce 2/3 package size, and save time/bandwidth for distributing 
> Kylin. If you see a problem this change may bring, please complain.
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>

[Discuss] Exclude Spark from Kylin's binary package

[ShaoFeng Shi]

Hello,

Today Kylin's binary package ships an Apache Spark binary, the total
package is 297 MB in v2.5.0; I noticed that if the package is above 300 MB,
it would be rejected by Apache SVN. Now in the whole package, Spark takes
about 200 MB. Later when we upgrade to a higher version, the package can be
bigger. Without spark, the package is around 100 MB.

Since we don't have customization to Spark, we can provide a
download-spark.sh script for the user to easily download. If the user
already has that in local, he just needs to copy or create a symbolic link
in Kylin folder.

This will reduce 2/3 package size, and save time/bandwidth for distributing
Kylin. If you see a problem this change may bring, please complain.

-- 
Best regards,

Shaofeng Shi 史少锋