Re: How to reduce zeppelin docker image size ( from custom build)

[Jongyoul Lee]

Hello Rajat,

Thank you for the detailed information.

I recommend changing Dockerfile using "AS Builder". Zeppelin's docker
images install all related dependencies even if you don't use those ones.
Basically, It's fine to include your distribution, Java, and Flink if you
want.

Can you try to do it?

JL

2023년 8월 8일 (화) 오후 5:19, Rajat Ahuja 님이 작성:

> Hi,
>
> I am building my own image. so when i run the image it has following
> content which is around *1.2 GB*
>
>
> du -sh *
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *80K bin180K conf758M interpreter24K k8s79M lib20K LICENSE76K licenses8.0K
> local-repo166M logs14M notebook4.0K NOTICE104M plugins4.0K README.md4.0K
> run400K scripts8.0K webapps28M zeppelin-web-0.11.0-SNAPSHOT.war*
>
> But docker size is
>
>
>
> *ahujarajat261/library_26_again
> latest e26aed940485   3 days ago
>  9.34GBartifactory.us.bank-dns.com:5000/flink/zeppelin-flink-us-bank-image
> 
>  latest e26aed940485   3 days ago  9.34GB*
>
>
> *Here's the dockerfile
> (https://github.com/apache/zeppelin/blob/master/scripts/docker/zeppelin/bin/Dockerfile
> )
> I am using after some modifications to build the image. Any suggestions as
> to how I can reduce the size ?*
>
> https://github.com/apache/zeppelin/blob/master/scripts/docker/zeppelin/bin/Dockerfile
> Thanks
>
>
> On Mon, Aug 7, 2023 at 6:55 AM Vishal Rao  wrote:
>
>> Yes I meant for Rajat (email starter) see
>> https://zeppelin.apache.org/download.html there is the "all" package
>> and the smaller "netinst" version too.
>>
>> On Mon, 7 Aug 2023 at 19:13, Jongyoul Lee  wrote:
>> >
>> > Thank you.
>> >
>> > I'll make it as well and get back to you. We have some redundant
>> libraries so it can be removed as well.
>> >
>> > Best regards,
>> > Jongyoul
>> >
>> > 2023년 8월 7일 (월) 오후 10:38, Vishal Rao 님이 작성:
>> >>
>> >> Yep, instead of starting with the full package and removing stuff, try
>> >> the "netinst" (tiny) package and add only what you need?
>> >>
>> >> On Mon, 7 Aug 2023 at 18:49, Jongyoul Lee  wrote:
>> >> >
>> >> > Hello,
>> >> >
>> >> > I'm not sure but did you check your tgz? What's inside it?
>> >> >
>> >> > 2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성:
>> >> >>
>> >> >> Hi team,
>> >> >>
>> >> >> I am trying to build a zeppelin from source code to run Flink. Here
>> are the things i have done to build the image for only Flink interpreter
>> only but seems like size is 9gb something of docker. How can i reduce the
>> size of the docker image
>> >> >>
>> >> >>
>> >> >> 1) I removed all possible interpreter modules ( like hbase,
>> cassandra, mark down) from pom.xml.
>> >> >> 2) mvn package clean -DskipTests
>> >> >> 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to
>> zeppelin/scripts/docker/bin
>> >> >> 4) change file names and docker file to build the image
>> >> >> and image size is 9gb. Any idea how can i reduce this image size ?
>> >> >>
>> >> >> Thanks
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > 이종열, Jongyoul Lee, 李宗烈
>> >> > http://madeng.net
>> >>
>> >>
>> >>
>> >> --
>> >> "The World is a book, and those who do not travel read only a page." -
>> >> St. Augustine.
>> >
>> >
>> >
>> > --
>> > 이종열, Jongyoul Lee, 李宗烈
>> > http://madeng.net
>>
>>
>>
>> --
>> "The World is a book, and those who do not travel read only a page." -
>> St. Augustine.
>>
>

-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: How to reduce zeppelin docker image size ( from custom build)

[Rajat Ahuja]

Hi,

I am building my own image. so when i run the image it has following
content which is around *1.2 GB*


du -sh *
















*80K bin180K conf758M interpreter24K k8s79M lib20K LICENSE76K licenses8.0K
local-repo166M logs14M notebook4.0K NOTICE104M plugins4.0K README.md4.0K
run400K scripts8.0K webapps28M zeppelin-web-0.11.0-SNAPSHOT.war*

But docker size is



*ahujarajat261/library_26_again
latest e26aed940485   3 days ago
 9.34GBartifactory.us.bank-dns.com:5000/flink/zeppelin-flink-us-bank-image

 latest e26aed940485   3 days ago  9.34GB*


*Here's the dockerfile
(https://github.com/apache/zeppelin/blob/master/scripts/docker/zeppelin/bin/Dockerfile
)
I am using after some modifications to build the image. Any suggestions as
to how I can reduce the size ?*
https://github.com/apache/zeppelin/blob/master/scripts/docker/zeppelin/bin/Dockerfile
Thanks


On Mon, Aug 7, 2023 at 6:55 AM Vishal Rao  wrote:

> Yes I meant for Rajat (email starter) see
> https://zeppelin.apache.org/download.html there is the "all" package
> and the smaller "netinst" version too.
>
> On Mon, 7 Aug 2023 at 19:13, Jongyoul Lee  wrote:
> >
> > Thank you.
> >
> > I'll make it as well and get back to you. We have some redundant
> libraries so it can be removed as well.
> >
> > Best regards,
> > Jongyoul
> >
> > 2023년 8월 7일 (월) 오후 10:38, Vishal Rao 님이 작성:
> >>
> >> Yep, instead of starting with the full package and removing stuff, try
> >> the "netinst" (tiny) package and add only what you need?
> >>
> >> On Mon, 7 Aug 2023 at 18:49, Jongyoul Lee  wrote:
> >> >
> >> > Hello,
> >> >
> >> > I'm not sure but did you check your tgz? What's inside it?
> >> >
> >> > 2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성:
> >> >>
> >> >> Hi team,
> >> >>
> >> >> I am trying to build a zeppelin from source code to run Flink. Here
> are the things i have done to build the image for only Flink interpreter
> only but seems like size is 9gb something of docker. How can i reduce the
> size of the docker image
> >> >>
> >> >>
> >> >> 1) I removed all possible interpreter modules ( like hbase,
> cassandra, mark down) from pom.xml.
> >> >> 2) mvn package clean -DskipTests
> >> >> 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to
> zeppelin/scripts/docker/bin
> >> >> 4) change file names and docker file to build the image
> >> >> and image size is 9gb. Any idea how can i reduce this image size ?
> >> >>
> >> >> Thanks
> >> >
> >> >
> >> >
> >> > --
> >> > 이종열, Jongyoul Lee, 李宗烈
> >> > http://madeng.net
> >>
> >>
> >>
> >> --
> >> "The World is a book, and those who do not travel read only a page." -
> >> St. Augustine.
> >
> >
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
>
>
>
> --
> "The World is a book, and those who do not travel read only a page." -
> St. Augustine.
>

Re: How to reduce zeppelin docker image size ( from custom build)

[Vishal Rao]

Yes I meant for Rajat (email starter) see
https://zeppelin.apache.org/download.html there is the "all" package
and the smaller "netinst" version too.

On Mon, 7 Aug 2023 at 19:13, Jongyoul Lee  wrote:
>
> Thank you.
>
> I'll make it as well and get back to you. We have some redundant libraries so 
> it can be removed as well.
>
> Best regards,
> Jongyoul
>
> 2023년 8월 7일 (월) 오후 10:38, Vishal Rao 님이 작성:
>>
>> Yep, instead of starting with the full package and removing stuff, try
>> the "netinst" (tiny) package and add only what you need?
>>
>> On Mon, 7 Aug 2023 at 18:49, Jongyoul Lee  wrote:
>> >
>> > Hello,
>> >
>> > I'm not sure but did you check your tgz? What's inside it?
>> >
>> > 2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성:
>> >>
>> >> Hi team,
>> >>
>> >> I am trying to build a zeppelin from source code to run Flink. Here are 
>> >> the things i have done to build the image for only Flink interpreter only 
>> >> but seems like size is 9gb something of docker. How can i reduce the size 
>> >> of the docker image
>> >>
>> >>
>> >> 1) I removed all possible interpreter modules ( like hbase, cassandra, 
>> >> mark down) from pom.xml.
>> >> 2) mvn package clean -DskipTests
>> >> 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to 
>> >> zeppelin/scripts/docker/bin
>> >> 4) change file names and docker file to build the image
>> >> and image size is 9gb. Any idea how can i reduce this image size ?
>> >>
>> >> Thanks
>> >
>> >
>> >
>> > --
>> > 이종열, Jongyoul Lee, 李宗烈
>> > http://madeng.net
>>
>>
>>
>> --
>> "The World is a book, and those who do not travel read only a page." -
>> St. Augustine.
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net



-- 
"The World is a book, and those who do not travel read only a page." -
St. Augustine.

Re: How to reduce zeppelin docker image size ( from custom build)

[Jongyoul Lee]

Thank you.

I'll make it as well and get back to you. We have some redundant libraries
so it can be removed as well.

Best regards,
Jongyoul

2023년 8월 7일 (월) 오후 10:38, Vishal Rao 님이 작성:

> Yep, instead of starting with the full package and removing stuff, try
> the "netinst" (tiny) package and add only what you need?
>
> On Mon, 7 Aug 2023 at 18:49, Jongyoul Lee  wrote:
> >
> > Hello,
> >
> > I'm not sure but did you check your tgz? What's inside it?
> >
> > 2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성:
> >>
> >> Hi team,
> >>
> >> I am trying to build a zeppelin from source code to run Flink. Here are
> the things i have done to build the image for only Flink interpreter only
> but seems like size is 9gb something of docker. How can i reduce the size
> of the docker image
> >>
> >>
> >> 1) I removed all possible interpreter modules ( like hbase, cassandra,
> mark down) from pom.xml.
> >> 2) mvn package clean -DskipTests
> >> 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to
> zeppelin/scripts/docker/bin
> >> 4) change file names and docker file to build the image
> >> and image size is 9gb. Any idea how can i reduce this image size ?
> >>
> >> Thanks
> >
> >
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
>
>
>
> --
> "The World is a book, and those who do not travel read only a page." -
> St. Augustine.
>


-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: How to reduce zeppelin docker image size ( from custom build)

[Vishal Rao]

Yep, instead of starting with the full package and removing stuff, try
the "netinst" (tiny) package and add only what you need?

On Mon, 7 Aug 2023 at 18:49, Jongyoul Lee  wrote:
>
> Hello,
>
> I'm not sure but did you check your tgz? What's inside it?
>
> 2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성:
>>
>> Hi team,
>>
>> I am trying to build a zeppelin from source code to run Flink. Here are the 
>> things i have done to build the image for only Flink interpreter only but 
>> seems like size is 9gb something of docker. How can i reduce the size of the 
>> docker image
>>
>>
>> 1) I removed all possible interpreter modules ( like hbase, cassandra, mark 
>> down) from pom.xml.
>> 2) mvn package clean -DskipTests
>> 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to 
>> zeppelin/scripts/docker/bin
>> 4) change file names and docker file to build the image
>> and image size is 9gb. Any idea how can i reduce this image size ?
>>
>> Thanks
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net



-- 
"The World is a book, and those who do not travel read only a page." -
St. Augustine.

Re: How to reduce zeppelin docker image size ( from custom build)

[Jongyoul Lee]

Hello,

I'm not sure but did you check your tgz? What's inside it?

2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성:

> Hi team,
>
> I am trying to build a zeppelin from source code to run Flink. Here are
> the things i have done to build the image for only Flink interpreter only
> but seems like size is 9gb something of docker. How can i reduce the size
> of the docker image
>
>
> 1) I removed all possible interpreter modules ( like hbase, cassandra,
> mark down) from pom.xml.
> 2) mvn package clean -DskipTests
> 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to
> zeppelin/scripts/docker/bin
> 4) change file names and docker file to build the image
> and image size is 9gb. Any idea how can i reduce this image size ?
>
> Thanks
>


-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

How to reduce zeppelin docker image size ( from custom build)

[Rajat Ahuja]

Hi team,

I am trying to build a zeppelin from source code to run Flink. Here are the
things i have done to build the image for only Flink interpreter only but
seems like size is 9gb something of docker. How can i reduce the size of
the docker image


1) I removed all possible interpreter modules ( like hbase, cassandra, mark
down) from pom.xml.
2) mvn package clean -DskipTests
3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to
zeppelin/scripts/docker/bin
4) change file names and docker file to build the image
and image size is 9gb. Any idea how can i reduce this image size ?

Thanks