Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-07 Thread Isabel Drost
On 01.11.2011 Grant Ingersoll wrote: > FWIW, in Lucene, we do the following: > > [...] For documentation purposes I added a summary of the discussion to our wiki (including a disclaimer that we are still working on the draft): https://cwiki.apache.org/confluence/display/MAHOUT/Downloads (Sectio

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-03 Thread Isabel Drost
On 31.10.2011 Lance Norskog wrote: > 6) Quick access to the online algorithms > > The servlet implementation in taste is simple. It should be possible to > package a lot of the online algorithms in one big servlet. Call it " Mahout > Online"? Just a thought: Is that something that a) Mahout users

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-01 Thread Isabel Drost
On 01.11.2011 Grant Ingersoll wrote: > FWIW, in Lucene, we do the following: > > 1. All minor versions within a major release can read prior versions index > within the same major release. That is, 3.4 can read a 3.3 index. > However, 3.3 cannot read a 3.4 index. When a user reads a 3.3 index w

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-01 Thread Isabel Drost
On 01.11.2011 Grant Ingersoll wrote: > On Nov 1, 2011, at 12:15 PM, Ted Dunning wrote: > > I think the trend is away from an explicit version in serialized data and > > toward systems like protobufs or avro which allow much more flexibility. > > +1 +1 > > Sent from my iPhone > > > > On Nov 1, 2

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-01 Thread Grant Ingersoll
On Nov 1, 2011, at 12:15 PM, Ted Dunning wrote: > I think the trend is away from an explicit version in serialized data and > toward systems like protobufs or avro which allow much more flexibility. +1 > > Sent from my iPhone > > On Nov 1, 2011, at 5:09, Grant Ingersoll wrote: > >> For th

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-01 Thread Ted Dunning
I think the trend is away from an explicit version in serialized data and toward systems like protobufs or avro which allow much more flexibility. Sent from my iPhone On Nov 1, 2011, at 5:09, Grant Ingersoll wrote: > For the most part this works and I would recommend we take similar steps.

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-01 Thread Grant Ingersoll
On Nov 1, 2011, at 8:09 AM, Grant Ingersoll wrote: > FWIW, in Lucene, we do the following: > > 1. All minor versions within a major release can read prior versions index > within the same major release. That is, 3.4 can read a 3.3 index. However, > 3.3 cannot read a 3.4 index. When a user r

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-01 Thread Grant Ingersoll
FWIW, in Lucene, we do the following: 1. All minor versions within a major release can read prior versions index within the same major release. That is, 3.4 can read a 3.3 index. However, 3.3 cannot read a 3.4 index. When a user reads a 3.3 index w/ 3.4, it is silently upgraded to 3.4. I th

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-11-01 Thread Isabel Drost
On 31.10.2011 Jeff Eastman wrote: > I think users would benefit a lot by 1) to 3) and would be dismayed if we > could not maintain data consistency between releases > (maybe just point releases?). Good point that I forgot to define in the original mail: Levels of back-compat should depend on wh

RE: Towards 1.0 - Defining backwards compatibility guarantees

2011-10-31 Thread Jeff Eastman
I think users would benefit a lot by 1) to 3) and would be dismayed if we could not maintain data consistency between releases (maybe just point releases?). This could require us to build and ship migrating tools along with any releases which change these formats. 4) and 5) are related and it i

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-10-30 Thread Lance Norskog
6) Quick access to the online algorithms The servlet implementation in taste is simple. It should be possible to package a lot of the online algorithms in one big servlet. Call it " Mahout Online"? One problem here is that uploading and downloading data for each operation is not practical. MO wou

Re: Towards 1.0 - Defining backwards compatibility guarantees

2011-10-30 Thread Lance Norskog
2) Model formats Proposal: a few common structures with higher-level conventions about how to compose them. . For matrix data, the R "dataframe" is a time-tested format for dense vectors, matrices and tensors. Something like this that also handles most sparsity cases would allow ditching a lot of h