Hello Jungtaek,
I agree that it's relatively easy to run Zeppelin with JDK 8 because
Zeppelin is not linked with another project directly. But as you told, it
bothers admin or system engineer, sometimes it's impossible for some
reasons. Thus personally, we should keep backward compatibility.
On Wed, Oct 5, 2016 at 4:05 PM, Jungtaek Lim <kabh...@gmail.com> wrote:
> I also think there's also many places to use old version of JDK in
> production. But unlike library projects, Zeppelin doesn't become dependency
> of other projects, so no need to consider other projects and also users'
> own projects, which greatly lowers the barrier.
> Installing JDK/JRE 8 and using it from Zeppelin doesn't harm their business
> logic and projects since users can maintain multiple version of JVMs
> anyway. (yes it could be bothering but possible to do... :) )
>
> We just need to consider compatibility of existing interpreters, and user
> codes which will be run in interpreters.
>
> - Jungtaek Lim (HeartSaVioR)
>
> 2016년 10월 5일 (수) 오후 3:51, Jongyoul Lee <jongy...@gmail.com>님이 작성:
>
> > Hello,
> >
> > Concerning JDK, I mean it, too. In production level, it may be hard to
> > change it.
> >
> > On Wed, Oct 5, 2016 at 3:36 PM, DuyHai Doan <doanduy...@gmail.com>
> wrote:
> >
> > > > "Concerning #2, Zeppelin's backend has a lot of map and the map has
> > > mutable states. Thus we should also think of read-write-lock."
> > >
> > > Agree, the impl of ConcurrentHashMap deals with this by splitting the
> > > internal map into buckets and the access to each bucket is synchronized
> > so
> > > that if you have 2 concurrent operations on the map and they fall into
> > > different buckets it doesn't incur any lock
> > >
> > > For List, the CopyOnWriteArrayList is expensive on write because it
> > copies
> > > the content of the list for each mutation. But this impl makes the
> > > assumption that the read ratio is much higher than the write ratio.
> > >
> > > Etc ... We need to check on case by case basis which impl is the best
> > >
> > > > "Personally, moving to Java8 is very attractive but we divide it with
> > > others because upgrading to the version of JDK influences users'
> > > experiences."
> > >
> > > Upgrading to Java 8 would require users to upgrade JDK server-side
> only,
> > > what is the impact on user experience ?
> > >
> > > On Wed, Oct 5, 2016 at 8:08 AM, Jongyoul Lee <jongy...@gmail.com>
> wrote:
> > >
> > > > Thanks for starting this thread.
> > > >
> > > > Concerning #2, Zeppelin's backend has a lot of map and the map has
> > > mutable
> > > > states. Thus we should also think of read-write-lock.
> > > >
> > > > Personally, moving to Java8 is very attractive but we divide it with
> > > others
> > > > because upgrading to the version of JDK influences users'
> experiences.
> > > >
> > > > On Wed, Oct 5, 2016 at 11:28 AM, Anthony Corbacho <
> > > > anthonycorba...@apache.org> wrote:
> > > >
> > > > > I think about the abuse of @Inject and circular deps, it is just
> > matter
> > > > of
> > > > > education.
> > > > >
> > > > > On Tue, Oct 4, 2016 at 8:05 PM, DuyHai Doan <doanduy...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > About DI I have no strong opinion on the topic.
> > > > > >
> > > > > > I have coded frameworks with just manual DI (through constructor
> > and
> > > > > > context objects) and it works pretty well, even for a big
> project,
> > as
> > > > > long
> > > > > > as the context objects have meaningfull names
> > > > > >
> > > > > > Using DI frameworks like Spring or Guice is also a valid choice,
> > > > > especially
> > > > > > for a backend. The only thing to be really cautious about are
> > > circular
> > > > > > dependencies. Using @Inject is very easy and people tend to abuse
> > it
> > > > > > everywhere and end up with horrible cyclic dependencies
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 4, 2016 at 12:54 PM, Anthony Corbacho <
> > > > > > anthonycorba...@apache.org> wrote:
> > > > > >
> > > > > > > You made my day, this is the kind of email i really like !!
> > > > > > >
> > > > > > > I think its a great idea and i am willing to spend sometime on
> > it.
> > > > > > >
> > > > > > > I also want to move to a DI (guice) architecture , let me know
> > what
> > > > you
> > > > > > > think about it.
> > > > > > >
> > > > > > > On Tuesday, 4 October 2016, DuyHai Doan <doanduy...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > Hello devs
> > > > > > > >
> > > > > > > > The code base of Zeppelin has grown very fast in the last 12
> > > months
> > > > > and
> > > > > > > > it's great. It means that we have more and more contributors.
> > > > > > > >
> > > > > > > > However, to make the project maintainable at long term, we
> need
> > > > > regular
> > > > > > > > code refactoring.
> > > > > > > >
> > > > > > > > I have some ideas to share with you
> > > > > > > >
> > > > > > > > 1) Use Java 8 to benefit from Lambda & streams.
> > > > > > > >
> > > > > > > > Now that Java 8 is well established, it is a good time to
> > > upgrade
> > > > > the
> > > > > > > > project. I believe some interpreters also need Java 8.
> > Cassandra
> > > > > > > > interpreter right now does not have unit tests for the latest
> > > > > features
> > > > > > > > because the Embedded Cassandra server used for testing
> requires
> > > > Java
> > > > > 8.
> > > > > > > >
> > > > > > > > It would also be a good opportunity to go through the code
> > base
> > > > and
> > > > > > > > replace some boilerplate for() loop with manual filtering by
> > the
> > > > > stream
> > > > > > > > shortcut : list.stream().filter(..).map(). It would improve
> > > > greatly
> > > > > > > code
> > > > > > > > readability
> > > > > > > >
> > > > > > > > 2) Multi threading
> > > > > > > >
> > > > > > > > I've seen the usage of synchronize block at a few places in
> > the
> > > > code
> > > > > > > base.
> > > > > > > > Although perfectly valid, it has a cost at runtime and since
> > more
> > > > and
> > > > > > > more
> > > > > > > > people are asking for multi-tenancy or using a single
> Zeppelin
> > > > > instance
> > > > > > > to
> > > > > > > > server multiple users, I guess the synchronized blocks has a
> > huge
> > > > > cost.
> > > > > > > >
> > > > > > > > There are some solid alternatives:
> > > > > > > >
> > > > > > > > - ConcurrentHashMap if we synchronized on a map
> > > > > > > > - CopyOnWriteArrayList if we synchronized on a list.
> > > > > > > >
> > > > > > > > Of cours each sychronize block should be taken carefully not
> to
> > > > > > introduce
> > > > > > > > regression
> > > > > > > >
> > > > > > > > 3) Thread management
> > > > > > > >
> > > > > > > > I've seen some usage of new Thread() {...}.run(); it may be a
> > > good
> > > > > time
> > > > > > > to
> > > > > > > > introduce ThreadPool and pass them along (inside context
> > objects
> > > > for
> > > > > > > > example) to have a more centralized thread management
> > > > > > > >
> > > > > > > > The advantage of having thread pool is that we can manage
> them
> > > in a
> > > > > > > single
> > > > > > > > place, monitor them and expose the info through JMX and also
> > > > control
> > > > > > > system
> > > > > > > > resource by defining max thread number and thread pool queue
> > > > > > > >
> > > > > > > > 4) Server monitoring
> > > > > > > > I hear many users on the field complain about the fact that
> > they
> > > > have
> > > > > > to
> > > > > > > > restart Zeppelin server regularly because it "hangs" after
> > > running
> > > > a
> > > > > > long
> > > > > > > > time.
> > > > > > > >
> > > > > > > > If we can expose some system metrics through JMX, it would
> help
> > > > > people
> > > > > > > > monitor the state of Zeppelin server and take appropriate
> > actions
> > > > > > > >
> > > > > > > > Right now we may only focus on monitoring the server itself,
> > not
> > > > the
> > > > > > > > interpreter JVMs processes. It can be done in a 2nd step
> > > > > > > >
> > > > > > > >
> > > > > > > > What do you think about the ideas ?
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > 이종열, Jongyoul Lee, 李宗烈
> > > > http://madeng.net
> > > >
> > >
> >
> >
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
> >
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
