LGTM… Thanks Ankur
From: moon soo Lee [mailto:m...@apache.org] Sent: 20 February 2016 11:24 AM To: users@zeppelin.incubator.apache.org Subject: Re: Unable to access AVRO method UI way requires interpreter process restart. So the behavior will be different from %spark z.load() you use right now. And...new UI way keeps list of dependencies. It does not reset after interpreter/zeppelin restart. So without api for removing dependency, dependencies will never decrease. Hence, to me, the best option seems - bringing %spark z.load() back with correct implementation - deprecate and remove %dep z.load() while it was alternative for buggy %spark z.load() - keep new UI way, while not only Spark but all other interpreters can leverage it. Thanks, moon On Fri, Feb 19, 2016 at 2:30 PM Aish Fenton <afen...@netflix.com<mailto:afen...@netflix.com>> wrote: I mean add in the dependency in the same way that the UI does. On Fri, Feb 19, 2016 at 2:30 PM Aish Fenton <afen...@netflix.com<mailto:afen...@netflix.com>> wrote: Hmm, Maybe you could change the semantics of z.load to be automatically add in the dependency? The only advantage I was really after with z.load was that self-documenting feature. On Fri, Feb 19, 2016 at 2:11 PM moon soo Lee <m...@apache.org<mailto:m...@apache.org>> wrote: Right, new UI provides more generalized way for dependency loading for all interpreters. not only spark but also jdbc, etc. For spark, We have had %spark z.load() at first, but it was too buggy, so we have created %dep z.load(), which is far less buggy but still some library couldn't loaded in this way. Finally, the new UI way, handles all the case correctly. I agree if we could have z.load for spark, it's self-documenting. And it may be convenient for multiple people working. If you're interested, i can work on bringing %spark z.load() back. But then we'll have 3 different way of dependency loading for Spark interpreter. - %spark z.load() (removed in master branch) - %dep z.load() (marked as deprecated in master branch) - new UI way. I think new UI way is for all interpreters. so we definitely want to keep it. So i want to ask people which one do we want to keep? %spark z.load(), %dep z.load(). Neither, both, one of it? Thanks, moon On Fri, Feb 19, 2016 at 10:52 AM Aish Fenton <afen...@netflix.com<mailto:afen...@netflix.com>> wrote: z.load was buggy, but it was self-documenting. I'm wondering with the new UI way, how will this work for people sharing/importing notebook? We quite commonly share notebooks. On Fri, Feb 19, 2016 at 10:09 AM moon soo Lee <m...@apache.org<mailto:m...@apache.org>> wrote: Hi Ankur, It's bit depends on you Zeppelin version, but basically z.load() inside of Spark interpreter was buggy and we have now removed from master branch. You can check "Dependency Management" section, from http://zeppelin.incubator.apache.org/docs/latest/interpreter/spark.html Basically, two different way one using %dep interpreter, the other using spark properties. If you can use master branch, you can load dependencies from Interpreter menu via GUI. Hope this helps. Best, moon On Fri, Feb 19, 2016 at 5:50 AM Ankur Jain <ankur.j...@yash.com<mailto:ankur.j...@yash.com>> wrote: Hi Team, I loaded avro dependencies, and then tried to read avro files, but it is throwing below exception… [image002.jpg] Can you please guide on this? My Environment details are as mentioned below… https://blogs.aws.amazon.com/bigdata/post/Tx2HJD3Z74J2U8U/Running-an-External-Zeppelin-Instance-using-S3-Backed-Notebooks-with-Spark-on-Am Thanks, Ankur Information transmitted by this e-mail is proprietary to YASH Technologies and/ or its Customers and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please notify us immediately at i...@yash.com<mailto:i...@yash.com> and delete this mail from your records. Information transmitted by this e-mail is proprietary to YASH Technologies and/ or its Customers and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please notify us immediately at i...@yash.com and delete this mail from your records.
