Re: [google-appengine] Re: MapReduce Roadmap

[Sridhar Ragunathan]

Hi DX,

For a beginner like it would be really helpful if you have a sample for
using filter.
Badly in need of it.
Thanks,
Sridhar R


On Sun, Dec 8, 2013 at 12:06 AM, D X  wrote:

> So we had written our own input readers that applied filtering, and it was
> pretty simple by extending the DatastoreInputReader code.  I didn't have
> the actual code in front of me, so I took a look back at the library code
> to see what I needed to override.
>
> Then I noticed, filtering functionality is included now!  Just not
> documented online.  Look through
> mapreduce.input_readers.DatastoreInputReader.validate_filters
>
> Looks like you need to include a 'filters' dictionary in your mapper_spec
>
> This supports my poor documentation point...
>
>
> On Friday, December 6, 2013 2:11:56 PM UTC-5, Amit S wrote:
>>
>> I agree with D X that packaged version would really help.
>>
>> One functionality we are badly in need is to be able to supply a query
>> filter in MapReduce (for e.g. where x="vvv"). Currently, it just iterates
>> through all entities of a given object in datastore. So, if I want to
>> update 10,000 rows of a given entity which consists of 5M records, it
>> iterates through all 5M entities and then we have to put the filter logic
>> in our code. This really defeats the purpose and also has huge implications
>> on read costs.
>>
>> On Friday, December 6, 2013 10:51:48 AM UTC-8, D X wrote:
>>>
>>> I've been using the mapreduce library for the last 18mo or so.
>>> In addition to what's already been mentioned, some additional comments:
>>>
>>> - The docs are kinda confusing because there's different sets of docs.
>>>  Just the fact that docs are disorganized gives the impression that it's a
>>> low priority project that's not well maintained.
>>> i.e. prettier: https://developers.google.com/appengine/docs/
>>> python/dataprocessing/helloworld
>>> but more detailed: https://code.google.com/p/appengine-mapreduce/
>>> wiki/MapReduceDemoApp
>>> Keeping one set of well maintained docs would help give the sense that
>>> MapReduce is a higher class citizen
>>>
>>> - Using MapReduce for schema changes is probably a very common yet
>>> simple use case.  I've heard more than one comment that the MapReduce
>>> pipeline seems to complicated to pick up to do a simple task of updating a
>>> bunch of entities.
>>> mapper_spec?  reducer_spec?  input_reader?  output_reader?  Do I have to
>>> learn all these things just to add an extra field to my entities?  While
>>> the wordcount demo shows more of the pipeline, it would probably be easier
>>> for users to pick up if there was a simple demo of how to update your
>>> 'schema' with 5 lines of python.  (a DatastoreInputReader that supports
>>> filtering would be great too)
>>>
>>> If this sounds too negative, you can interepret these comments as saying
>>> that the rest of the GAE docs are great and easy to follow.
>>>
>>> - Packaged versions would be great.  I mean, it was great.  I'm not sure
>>> why you guys got rid of it.  Maybe I'm not hardcore enough to just sync
>>> with the repo (actually, I did).  However, a packaged version suggests that
>>> it's tested and stable.  If I see bugs, I can check online to see if
>>> anyone's seeing the same issue.  When syncing with the repo, I have not
>>> idea how stable the latest checkins are.  Maybe something just broke and
>>> I'm the one person who sync'd after the broken change, and I'm obviously
>>> not going to be constantly syncing the MR library, because I actually have
>>> other things to work on.
>>> What about the version that's included in the SDK?  I toyed with using
>>> that, but the docs indicate I should be downloading from the repo.  So is
>>> the repo more recent, and the SDK version outdated?  Would the SDK version
>>> be more stable?  Again, confusion.
>>>
>>>
>>>
>>>
>>> On Friday, December 6, 2013 1:10:37 AM UTC-5, Chris Ramsdale wrote:

 thanks, PK.  we'll follow-up on the issue you cited.

 -- Chris


 On Thu, Dec 5, 2013 at 9:59 PM, PK  wrote:

> Thanks Chris.
>
> Great to hear that you plan to move map/reduce to GA. The reasons I
> asked are:
>
> 1. it has been experimental for about 3 years if not longer so I had
> started wondering… I think it is fair.
> 2. More important some bugs still in NEW state make me wonder whether
> it follows along with other changes in the platform. In particular seeing
> issues staying in NEW state for months concerns me the most about how
> active the effort is.
>
> (examples of such bugs:
> e.g. Issue 
> 182
>  mapreduce/include.yaml doesn't work with python2.7 and thread 
> safe
>   or Issue 
> 203

Re: [google-appengine] Re: MapReduce Roadmap

[D X]

So we had written our own input readers that applied filtering, and it was 
pretty simple by extending the DatastoreInputReader code.  I didn't have 
the actual code in front of me, so I took a look back at the library code 
to see what I needed to override.

Then I noticed, filtering functionality is included now!  Just not 
documented online.  Look through 
mapreduce.input_readers.DatastoreInputReader.validate_filters

Looks like you need to include a 'filters' dictionary in your mapper_spec

This supports my poor documentation point...

On Friday, December 6, 2013 2:11:56 PM UTC-5, Amit S wrote:
>
> I agree with D X that packaged version would really help.
>
> One functionality we are badly in need is to be able to supply a query 
> filter in MapReduce (for e.g. where x="vvv"). Currently, it just iterates 
> through all entities of a given object in datastore. So, if I want to 
> update 10,000 rows of a given entity which consists of 5M records, it 
> iterates through all 5M entities and then we have to put the filter logic 
> in our code. This really defeats the purpose and also has huge implications 
> on read costs.
>
> On Friday, December 6, 2013 10:51:48 AM UTC-8, D X wrote:
>>
>> I've been using the mapreduce library for the last 18mo or so.
>> In addition to what's already been mentioned, some additional comments:
>>
>> - The docs are kinda confusing because there's different sets of docs. 
>>  Just the fact that docs are disorganized gives the impression that it's a 
>> low priority project that's not well maintained.
>> i.e. prettier: 
>> https://developers.google.com/appengine/docs/python/dataprocessing/helloworld
>> but more detailed: 
>> https://code.google.com/p/appengine-mapreduce/wiki/MapReduceDemoApp
>> Keeping one set of well maintained docs would help give the sense that 
>> MapReduce is a higher class citizen
>>
>> - Using MapReduce for schema changes is probably a very common yet simple 
>> use case.  I've heard more than one comment that the MapReduce pipeline 
>> seems to complicated to pick up to do a simple task of updating a bunch of 
>> entities.
>> mapper_spec?  reducer_spec?  input_reader?  output_reader?  Do I have to 
>> learn all these things just to add an extra field to my entities?  While 
>> the wordcount demo shows more of the pipeline, it would probably be easier 
>> for users to pick up if there was a simple demo of how to update your 
>> 'schema' with 5 lines of python.  (a DatastoreInputReader that supports 
>> filtering would be great too)
>>
>> If this sounds too negative, you can interepret these comments as saying 
>> that the rest of the GAE docs are great and easy to follow.
>>
>> - Packaged versions would be great.  I mean, it was great.  I'm not sure 
>> why you guys got rid of it.  Maybe I'm not hardcore enough to just sync 
>> with the repo (actually, I did).  However, a packaged version suggests that 
>> it's tested and stable.  If I see bugs, I can check online to see if 
>> anyone's seeing the same issue.  When syncing with the repo, I have not 
>> idea how stable the latest checkins are.  Maybe something just broke and 
>> I'm the one person who sync'd after the broken change, and I'm obviously 
>> not going to be constantly syncing the MR library, because I actually have 
>> other things to work on.
>> What about the version that's included in the SDK?  I toyed with using 
>> that, but the docs indicate I should be downloading from the repo.  So is 
>> the repo more recent, and the SDK version outdated?  Would the SDK version 
>> be more stable?  Again, confusion.
>>
>>
>>
>>
>> On Friday, December 6, 2013 1:10:37 AM UTC-5, Chris Ramsdale wrote:
>>>
>>> thanks, PK.  we'll follow-up on the issue you cited.
>>>
>>> -- Chris
>>>
>>>
>>> On Thu, Dec 5, 2013 at 9:59 PM, PK  wrote:
>>>
 Thanks Chris.

 Great to hear that you plan to move map/reduce to GA. The reasons I 
 asked are:

 1. it has been experimental for about 3 years if not longer so I had 
 started wondering… I think it is fair.
 2. More important some bugs still in NEW state make me wonder whether 
 it follows along with other changes in the platform. In particular seeing 
 issues staying in NEW state for months concerns me the most about how 
 active the effort is.

 (examples of such bugs:
 e.g. Issue 
 182
  mapreduce/include.yaml doesn't work with python2.7 and thread 
 safe
   or Issue 
 203  
 map reduce is broken since r534)

 Thanks,
 PK

 On December 5, 2013 at 8:27:33 PM, Chris Ramsdale (cram...@google.com) 
 wrote:

 PK,

 We're definitely planning on moving mapreduce to GA. Plan is

Re: [google-appengine] Re: MapReduce Roadmap

[Jacob Taylor]

It would also be great to offer some caching support. If we could easily
decide which kinds of entities should still be subject to caching, that
would be great.

i.e. we are iterating through Users and reference some common stuff. We
either need to build our own cache for the common stuff or pound on the
datastore.

I would also love to see shard status represented in the graph (green if
done?) and keep the shard graph after the mapreduce is done. This will help
me understand how balanced the shards are.

We have too many namespaces now and sharding is happening on alpha sort of
namespace names. This is a huge problem since a few of our larger
namespaces are adjacent. We end up with about 10x the response time. I
think this might also be affecting the production index builder.

Thanks,
Jacob


On Fri, Dec 6, 2013 at 11:11 AM, amits  wrote:

> I agree with D X that packaged version would really help.
>
> One functionality we are badly in need is to be able to supply a query
> filter in MapReduce (for e.g. where x="vvv"). Currently, it just iterates
> through all entities of a given object in datastore. So, if I want to
> update 10,000 rows of a given entity which consists of 5M records, it
> iterates through all 5M entities and then we have to put the filter logic
> in our code. This really defeats the purpose and also has huge implications
> on read costs.
>
>
> On Friday, December 6, 2013 10:51:48 AM UTC-8, D X wrote:
>>
>> I've been using the mapreduce library for the last 18mo or so.
>> In addition to what's already been mentioned, some additional comments:
>>
>> - The docs are kinda confusing because there's different sets of docs.
>>  Just the fact that docs are disorganized gives the impression that it's a
>> low priority project that's not well maintained.
>> i.e. prettier: https://developers.google.com/appengine/docs/
>> python/dataprocessing/helloworld
>> but more detailed: https://code.google.com/p/appengine-mapreduce/
>> wiki/MapReduceDemoApp
>> Keeping one set of well maintained docs would help give the sense that
>> MapReduce is a higher class citizen
>>
>> - Using MapReduce for schema changes is probably a very common yet simple
>> use case.  I've heard more than one comment that the MapReduce pipeline
>> seems to complicated to pick up to do a simple task of updating a bunch of
>> entities.
>> mapper_spec?  reducer_spec?  input_reader?  output_reader?  Do I have to
>> learn all these things just to add an extra field to my entities?  While
>> the wordcount demo shows more of the pipeline, it would probably be easier
>> for users to pick up if there was a simple demo of how to update your
>> 'schema' with 5 lines of python.  (a DatastoreInputReader that supports
>> filtering would be great too)
>>
>> If this sounds too negative, you can interepret these comments as saying
>> that the rest of the GAE docs are great and easy to follow.
>>
>> - Packaged versions would be great.  I mean, it was great.  I'm not sure
>> why you guys got rid of it.  Maybe I'm not hardcore enough to just sync
>> with the repo (actually, I did).  However, a packaged version suggests that
>> it's tested and stable.  If I see bugs, I can check online to see if
>> anyone's seeing the same issue.  When syncing with the repo, I have not
>> idea how stable the latest checkins are.  Maybe something just broke and
>> I'm the one person who sync'd after the broken change, and I'm obviously
>> not going to be constantly syncing the MR library, because I actually have
>> other things to work on.
>> What about the version that's included in the SDK?  I toyed with using
>> that, but the docs indicate I should be downloading from the repo.  So is
>> the repo more recent, and the SDK version outdated?  Would the SDK version
>> be more stable?  Again, confusion.
>>
>>
>>
>>
>> On Friday, December 6, 2013 1:10:37 AM UTC-5, Chris Ramsdale wrote:
>>>
>>> thanks, PK.  we'll follow-up on the issue you cited.
>>>
>>> -- Chris
>>>
>>>
>>> On Thu, Dec 5, 2013 at 9:59 PM, PK  wrote:
>>>
 Thanks Chris.

 Great to hear that you plan to move map/reduce to GA. The reasons I
 asked are:

 1. it has been experimental for about 3 years if not longer so I had
 started wondering… I think it is fair.
 2. More important some bugs still in NEW state make me wonder whether
 it follows along with other changes in the platform. In particular seeing
 issues staying in NEW state for months concerns me the most about how
 active the effort is.

 (examples of such bugs:
 e.g. Issue 
 182
  mapreduce/include.yaml doesn't work with python2.7 and thread 
 safe
   or Issue 
 203
 map reduce is broken since

Re: [google-appengine] Re: MapReduce Roadmap

[amits]

I agree with D X that packaged version would really help.

One functionality we are badly in need is to be able to supply a query 
filter in MapReduce (for e.g. where x="vvv"). Currently, it just iterates 
through all entities of a given object in datastore. So, if I want to 
update 10,000 rows of a given entity which consists of 5M records, it 
iterates through all 5M entities and then we have to put the filter logic 
in our code. This really defeats the purpose and also has huge implications 
on read costs.

On Friday, December 6, 2013 10:51:48 AM UTC-8, D X wrote:
>
> I've been using the mapreduce library for the last 18mo or so.
> In addition to what's already been mentioned, some additional comments:
>
> - The docs are kinda confusing because there's different sets of docs. 
>  Just the fact that docs are disorganized gives the impression that it's a 
> low priority project that's not well maintained.
> i.e. prettier: 
> https://developers.google.com/appengine/docs/python/dataprocessing/helloworld
> but more detailed: 
> https://code.google.com/p/appengine-mapreduce/wiki/MapReduceDemoApp
> Keeping one set of well maintained docs would help give the sense that 
> MapReduce is a higher class citizen
>
> - Using MapReduce for schema changes is probably a very common yet simple 
> use case.  I've heard more than one comment that the MapReduce pipeline 
> seems to complicated to pick up to do a simple task of updating a bunch of 
> entities.
> mapper_spec?  reducer_spec?  input_reader?  output_reader?  Do I have to 
> learn all these things just to add an extra field to my entities?  While 
> the wordcount demo shows more of the pipeline, it would probably be easier 
> for users to pick up if there was a simple demo of how to update your 
> 'schema' with 5 lines of python.  (a DatastoreInputReader that supports 
> filtering would be great too)
>
> If this sounds too negative, you can interepret these comments as saying 
> that the rest of the GAE docs are great and easy to follow.
>
> - Packaged versions would be great.  I mean, it was great.  I'm not sure 
> why you guys got rid of it.  Maybe I'm not hardcore enough to just sync 
> with the repo (actually, I did).  However, a packaged version suggests that 
> it's tested and stable.  If I see bugs, I can check online to see if 
> anyone's seeing the same issue.  When syncing with the repo, I have not 
> idea how stable the latest checkins are.  Maybe something just broke and 
> I'm the one person who sync'd after the broken change, and I'm obviously 
> not going to be constantly syncing the MR library, because I actually have 
> other things to work on.
> What about the version that's included in the SDK?  I toyed with using 
> that, but the docs indicate I should be downloading from the repo.  So is 
> the repo more recent, and the SDK version outdated?  Would the SDK version 
> be more stable?  Again, confusion.
>
>
>
>
> On Friday, December 6, 2013 1:10:37 AM UTC-5, Chris Ramsdale wrote:
>>
>> thanks, PK.  we'll follow-up on the issue you cited.
>>
>> -- Chris
>>
>>
>> On Thu, Dec 5, 2013 at 9:59 PM, PK  wrote:
>>
>>> Thanks Chris.
>>>
>>> Great to hear that you plan to move map/reduce to GA. The reasons I 
>>> asked are:
>>>
>>> 1. it has been experimental for about 3 years if not longer so I had 
>>> started wondering… I think it is fair.
>>> 2. More important some bugs still in NEW state make me wonder whether it 
>>> follows along with other changes in the platform. In particular seeing 
>>> issues staying in NEW state for months concerns me the most about how 
>>> active the effort is.
>>>
>>> (examples of such bugs:
>>> e.g. Issue 
>>> 182
>>>  mapreduce/include.yaml doesn't work with python2.7 and thread 
>>> safe
>>>   or Issue 
>>> 203  
>>> map reduce is broken since r534)
>>>
>>> Thanks,
>>> PK
>>>
>>> On December 5, 2013 at 8:27:33 PM, Chris Ramsdale (cram...@google.com) 
>>> wrote:
>>>
>>> PK,
>>>
>>> We're definitely planning on moving mapreduce to GA. Plan is to get the 
>>> API finalized and then move it through the std Preview => GA channel.
>>>
>>> Questions for you:
>>>
>>> - how would prefer to access the library itself?
>>>
>>> - is there something about having it outside of the SDK that causes 
>>> substantial friction?
>>>
>>> - or, is it the fact that it's sat in experimental for way too long that 
>>> is the larger concern?
>>>
>>> -- Chris
>>>
>>> Product Manager, Google App Engine
>>>
>>> -- Chris
>>> On Dec 5, 2013 6:12 PM, "PK"  wrote:
>>>
  I am resending this in case the right Product Manager(s) at Google 
 missed my question.
  
  Thanks,
 PK
 http://www.gae123.com

 On November 25, 2013 at 12:53:44 AM, PK (p

Re: [google-appengine] Re: MapReduce Roadmap

[D X]

I've been using the mapreduce library for the last 18mo or so.
In addition to what's already been mentioned, some additional comments:

- The docs are kinda confusing because there's different sets of docs. 
 Just the fact that docs are disorganized gives the impression that it's a 
low priority project that's not well maintained.
i.e. prettier: 
https://developers.google.com/appengine/docs/python/dataprocessing/helloworld
but more detailed: 
https://code.google.com/p/appengine-mapreduce/wiki/MapReduceDemoApp
Keeping one set of well maintained docs would help give the sense that 
MapReduce is a higher class citizen

- Using MapReduce for schema changes is probably a very common yet simple 
use case.  I've heard more than one comment that the MapReduce pipeline 
seems to complicated to pick up to do a simple task of updating a bunch of 
entities.
mapper_spec?  reducer_spec?  input_reader?  output_reader?  Do I have to 
learn all these things just to add an extra field to my entities?  While 
the wordcount demo shows more of the pipeline, it would probably be easier 
for users to pick up if there was a simple demo of how to update your 
'schema' with 5 lines of python.  (a DatastoreInputReader that supports 
filtering would be great too)

If this sounds too negative, you can interepret these comments as saying 
that the rest of the GAE docs are great and easy to follow.

- Packaged versions would be great.  I mean, it was great.  I'm not sure 
why you guys got rid of it.  Maybe I'm not hardcore enough to just sync 
with the repo (actually, I did).  However, a packaged version suggests that 
it's tested and stable.  If I see bugs, I can check online to see if 
anyone's seeing the same issue.  When syncing with the repo, I have not 
idea how stable the latest checkins are.  Maybe something just broke and 
I'm the one person who sync'd after the broken change, and I'm obviously 
not going to be constantly syncing the MR library, because I actually have 
other things to work on.
What about the version that's included in the SDK?  I toyed with using 
that, but the docs indicate I should be downloading from the repo.  So is 
the repo more recent, and the SDK version outdated?  Would the SDK version 
be more stable?  Again, confusion.




On Friday, December 6, 2013 1:10:37 AM UTC-5, Chris Ramsdale wrote:
>
> thanks, PK.  we'll follow-up on the issue you cited.
>
> -- Chris
>
>
> On Thu, Dec 5, 2013 at 9:59 PM, PK > wrote:
>
>> Thanks Chris.
>>
>> Great to hear that you plan to move map/reduce to GA. The reasons I asked 
>> are:
>>
>> 1. it has been experimental for about 3 years if not longer so I had 
>> started wondering… I think it is fair.
>> 2. More important some bugs still in NEW state make me wonder whether it 
>> follows along with other changes in the platform. In particular seeing 
>> issues staying in NEW state for months concerns me the most about how 
>> active the effort is.
>>
>> (examples of such bugs:
>> e.g. Issue 
>> 182
>>  mapreduce/include.yaml doesn't work with python2.7 and thread 
>> safe
>>   or Issue 
>> 203  
>> map reduce is broken since r534)
>>
>> Thanks,
>> PK
>>
>> On December 5, 2013 at 8:27:33 PM, Chris Ramsdale 
>> (cram...@google.com) 
>> wrote:
>>
>> PK,
>>
>> We're definitely planning on moving mapreduce to GA. Plan is to get the 
>> API finalized and then move it through the std Preview => GA channel.
>>
>> Questions for you:
>>
>> - how would prefer to access the library itself?
>>
>> - is there something about having it outside of the SDK that causes 
>> substantial friction?
>>
>> - or, is it the fact that it's sat in experimental for way too long that 
>> is the larger concern?
>>
>> -- Chris
>>
>> Product Manager, Google App Engine
>>
>> -- Chris
>> On Dec 5, 2013 6:12 PM, "PK" > wrote:
>>
>>>  I am resending this in case the right Product Manager(s) at Google 
>>> missed my question.
>>>  
>>>  Thanks,
>>> PK
>>> http://www.gae123.com
>>>
>>> On November 25, 2013 at 12:53:44 AM, PK (p...@gae123.com ) 
>>> wrote:
>>>
>>>   For a very very long time MapReduce has not been integrated with the 
>>> GAE SDK and remains experimental (
>>> https://developers.google.com/appengine/docs/python/dataprocessing/).
>>>  
>>>  Could somebody shed some light on the roadmap plan?
>>>  
>>> Thanks,
>>> PK
>>> http://www.gae123.com
>>>  
>>>  --
>>> You received this message because you are subscribed to the Google 
>>> Groups "Google App Engine" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to google-appengi...@googlegroups.com .
>>> To post to this group, send email to 
>>> google-a...@googlegroups.com
>>> .
>>> Visit this group at http://groups.google.co

Re: [google-appengine] Re: MapReduce Roadmap

[Chris Ramsdale]

thanks, PK.  we'll follow-up on the issue you cited.

-- Chris


On Thu, Dec 5, 2013 at 9:59 PM, PK  wrote:

> Thanks Chris.
>
> Great to hear that you plan to move map/reduce to GA. The reasons I asked
> are:
>
> 1. it has been experimental for about 3 years if not longer so I had
> started wondering… I think it is fair.
> 2. More important some bugs still in NEW state make me wonder whether it
> follows along with other changes in the platform. In particular seeing
> issues staying in NEW state for months concerns me the most about how
> active the effort is.
>
> (examples of such bugs:
> e.g. Issue 
> 182
>  mapreduce/include.yaml doesn't work with python2.7 and thread 
> safe
>   or Issue 
> 203
> map reduce is broken since r534)
>
> Thanks,
> PK
>
> On December 5, 2013 at 8:27:33 PM, Chris Ramsdale 
> (cramsd...@google.com)
> wrote:
>
> PK,
>
> We're definitely planning on moving mapreduce to GA. Plan is to get the
> API finalized and then move it through the std Preview => GA channel.
>
> Questions for you:
>
> - how would prefer to access the library itself?
>
> - is there something about having it outside of the SDK that causes
> substantial friction?
>
> - or, is it the fact that it's sat in experimental for way too long that
> is the larger concern?
>
> -- Chris
>
> Product Manager, Google App Engine
>
> -- Chris
> On Dec 5, 2013 6:12 PM, "PK"  wrote:
>
>>  I am resending this in case the right Product Manager(s) at Google
>> missed my question.
>>
>>  Thanks,
>> PK
>> http://www.gae123.com
>>
>> On November 25, 2013 at 12:53:44 AM, PK (p...@gae123.com )
>> wrote:
>>
>>   For a very very long time MapReduce has not been integrated with the
>> GAE SDK and remains experimental (
>> https://developers.google.com/appengine/docs/python/dataprocessing/).
>>
>>  Could somebody shed some light on the roadmap plan?
>>
>> Thanks,
>> PK
>> http://www.gae123.com
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to google-appengine+unsubscr...@googlegroups.com.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> Visit this group at http://groups.google.com/group/google-appengine.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: MapReduce Roadmap

[PK]

Thanks Chris.

Great to hear that you plan to move map/reduce to GA. The reasons I asked are:

1. it has been experimental for about 3 years if not longer so I had started 
wondering… I think it is fair.
2. More important some bugs still in NEW state make me wonder whether it 
follows along with other changes in the platform. In particular seeing issues 
staying in NEW state for months concerns me the most about how active the 
effort is.

(examples of such bugs:
e.g. Issue 182 mapreduce/include.yaml doesn't work with python2.7 and thread 
safe
          or Issue 203  map reduce is broken since r534)

Thanks,
PK

On December 5, 2013 at 8:27:33 PM, Chris Ramsdale (cramsd...@google.com) wrote:

PK,

We're definitely planning on moving mapreduce to GA. Plan is to get the API 
finalized and then move it through the std Preview => GA channel.

Questions for you:

- how would prefer to access the library itself?

- is there something about having it outside of the SDK that causes substantial 
friction?

- or, is it the fact that it's sat in experimental for way too long that is the 
larger concern?

-- Chris

Product Manager, Google App Engine

-- Chris

On Dec 5, 2013 6:12 PM, "PK"  wrote:
I am resending this in case the right Product Manager(s) at Google missed my 
question.

Thanks,
PK
http://www.gae123.com

On November 25, 2013 at 12:53:44 AM, PK (p...@gae123.com) wrote:

For a very very long time MapReduce has not been integrated with the GAE SDK 
and remains experimental 
(https://developers.google.com/appengine/docs/python/dataprocessing/).

Could somebody shed some light on the roadmap plan?

Thanks,
PK
http://www.gae123.com
--
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: MapReduce Roadmap

[Chris Ramsdale]

PK,

We're definitely planning on moving mapreduce to GA. Plan is to get the API
finalized and then move it through the std Preview => GA channel.

Questions for you:

- how would prefer to access the library itself?

- is there something about having it outside of the SDK that causes
substantial friction?

- or, is it the fact that it's sat in experimental for way too long that is
the larger concern?

-- Chris

Product Manager, Google App Engine

-- Chris
On Dec 5, 2013 6:12 PM, "PK"  wrote:

> I am resending this in case the right Product Manager(s) at Google missed
> my question.
>
> Thanks,
> PK
> http://www.gae123.com
>
> On November 25, 2013 at 12:53:44 AM, PK (p...@gae123.com )
> wrote:
>
>  For a very very long time MapReduce has not been integrated with the GAE
> SDK and remains experimental (
> https://developers.google.com/appengine/docs/python/dataprocessing/).
>
>  Could somebody shed some light on the roadmap plan?
>
> Thanks,
> PK
> http://www.gae123.com
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: MapReduce Roadmap

[PK]

I am resending this in case the right Product Manager(s) at Google missed my 
question.

Thanks,
PK
http://www.gae123.com

On November 25, 2013 at 12:53:44 AM, PK (p...@gae123.com) wrote:

For a very very long time MapReduce has not been integrated with the GAE SDK 
and remains experimental 
(https://developers.google.com/appengine/docs/python/dataprocessing/).

Could somebody shed some light on the roadmap plan?

Thanks,
PK
http://www.gae123.com

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.