Regarding data in the note.json, In case of user doesn't want include data in exported note.json, user can clean the outputs before export, for now.
We might think displaying two export options with / without data when click export button, if exporting notebook without data is important and need to user aware everytime they export the notebook. But please consider many different possible use cases. Some people might have important information inside of the code (like credentials) but result of query can be made public, some people might want to restrict access to the raw data but want to share query result to other people. Best, moon On Tue, Sep 13, 2016 at 10:20 PM Vikash Kumar <vikash.ku...@resilinc.com> wrote: > Hi, > > But storing the data in a separate file approach will need to maintain the > link between both files. And also this approach is not preferable when the > data is obtained on access basis. like in my case data which comes from > hbase through phoenix is tenant base. So storing that data into note.json > or in different file is breaking the point of multi tenancy. > > So as an approach can we store only configuration and retrieve the data > when we are loading the note by running the all paragraph for first time > load. > > > > But at the same time, i think having data in the note.json helps make > import/export simple and make notebook render able without run it. > > > > So for import/export providing the data is it good? Data is always > confidential and cannot be shared with anyone in form of json. So in this > approach any one can open the note.json and can access the data. > > Thanks & Regards, > > *Vikash Kumar* > > *From:* Felix Cheung [mailto:felixcheun...@hotmail.com] > *Sent:* Wednesday, September 14, 2016 6:24 AM > *To:* users@zeppelin.apache.org; users@zeppelin.apache.org > > > *Subject:* Re: Hbase configuration storage without data > > > > I like that approach - though you should be able to clear result output > before exporting the note, if all you want is the config? The should remove > all output data, keeping it smaller? > > > > > > _____________________________ > From: Mohit Jaggi <mohitja...@gmail.com> > Sent: Monday, September 12, 2016 10:38 AM > Subject: Re: Hbase configuration storage without data > To: <users@zeppelin.apache.org> > > > > one option is to keep the data in separate files. notes.json can contain > the code and the data can be a pointer to /path/to/file. import/export can > choose to include or exclude the data. when it is included the data files > are added to a tgz file containing notes.json otherwise you just export > notes.json > > > > > > > > On Mon, Sep 12, 2016 at 10:33 AM, moon soo Lee <m...@apache.org> wrote: > > Right big note.json file is a problem. > > But at the same time, i think having data in the note.json helps make > import/export simple and make notebook renderable without run it. > > > > So far, i didn't see much discussion about this subject on mailing list or > on the issue tracker. > > > > If there's an good idea that can handle large data while keeping > import/export simple and ability to render without run, that would be a > great starting point of the discussions. > > > > Thanks, > > moon > > > > On Wed, Sep 7, 2016 at 9:40 PM Vikash Kumar <vikash.ku...@resilinc.com> > wrote: > > Hi moon, > > Yes that was the way that I was using. But is there any plan for future > releases to removing the data from note and storing only configuration? > > Because storing the configuration with data when there is no max result > limit will create a big note.json file. > > > > Thanks & Regards, > > *Vikash Kumar* > > *From:* moon soo Lee [mailto:m...@apache.org] > *Sent:* Wednesday, September 7, 2016 8:39 PM > *To:* users@zeppelin.apache.org > *Subject:* Re: Hbase configuration storage without data > > > > Hi, > > > > For now, code and result data are mixed in note.json, which is represented > by 'class Note' [1]. And every Notebook storage layer need to implement > 'NotebookRepo.get()' [2] to read note.json from underlying storage and > convert it into 'class Note'. > > > > As you see the related API and class definition, NotebookRepo actually > doesn't have any restriction how 'class Note' is serialized and saved in > the storage. > > > > So you can event new format, you can exclude result data from saving, and > so on. > > > > Hop this helps. > > > > Thanks, > > moon > > > > [1] > https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java > > [2] > https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepo.java#L47 > > > > On Wed, Sep 7, 2016 at 3:47 AM Vikash Kumar <vikash.ku...@resilinc.com> > wrote: > > Hi all, > > We are storing the note.json configuration into hbase as > it is stored into File system. As default behavior in note.json the query > data is stored along with configuration. But we want to store the > configurations only and when user loading its note then query should get > executed and data generated. This feature we are using for phoenix > interpreter. So how can we remove the data from note.json? Is there any > plan for that? > > > > > > Thanks & Regards, > > *Vikash Kumar* > > > > >
