[appengine-java] Re: 1.4.2: Can't make https call from dev server
Hey everybody, In order to help me diagnose the problems, could you respond with what JDK (Platform, JDK manufacturer (e.g. Sun, OpenJDK), and version) you're running? As a temporary stopgap, if you're using URLFetch service, you can use the doNotValidateCertificate() method in FetchOptions (http://code.google.com/appengine/docs/java/javadoc/com/ google/appengine/api/urlfetch/FetchOptions.html) to disable validation. -Fred On Feb 13, 5:02 am, harri mauria wrote: > I am having similar problems with the recently released version > com.google.appengine.eclipse.sdkbundle.1.4.2_1.4.2.v201102111811 > Problems happen with the development server, not in production. I am > performing Facebook login credentials authentication in the callback > url. Interrupt stack is below: > > HTTP ERROR 500 > Problem accessing /loginfacebookcallback. Reason: > > Could not verify SSL certificate > for:https://graph.facebook.com/oauth/access_token?omitted > > Caused by: > javax.net.ssl.SSLHandshakeException: Could not verify SSL certificate > for:https://graph.facebook.com/oauth/access_token?omitted > at > com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationExc > eption(URLFetchServiceImpl.java: > 114) > at > com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchService > Impl.java: > 41) > at > com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler > $Connection.fetchResponse(URLFetchServiceStreamHandler.java:418) > at > com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler > $Connection.getInputStream(URLFetchServiceStreamHandler.java:297) > atwww.superpass.de.server.UrlFetcher.get(UrlFetcher.java:60) > > Following message is printed to the console: > I/O exception (javax.net.ssl.SSLPeerUnverifiedException) caught when > processing request: HTTPS hostname invalid: expected > 'graph.facebook.com', received '*.facebook.com' > Retrying request > > Regards, Harri > > On Feb 13, 11:27 am, Lior Harsat wrote: > > > > > > > > > I am having a similar problem with https: > > > I have been testing the SDK and I found one bug which seems critical. > > When I call the URLFetchService.fetch API with a https URL I get a > > javax.net.ssl.SSLHandshakeException. > > for example when I fetch "https://spreadsheets.google.com/feeds/ > > list/" I get > > javax.net.ssl.SSLHandshakeException: Could not verify SSL certificate > > for:https://spreadsheets.google.com/feeds/list/... > > When I switch back to an older SDK this doesn't happen. > > Is there a "Help us test Prerelease SDK 1.4.2" thread I can post this > > to? > > Can anyone confirm this bug reproduces? > > Thanx, Lior > > > On Feb 12, 11:57 am, Peter Liu wrote: > > > > We are using a custom remote delegate to route datastore rpc call to > > > live production test site on appspot. We did this because sometimes we > > > want to develop with live data. > > > > However after upgrading to 1.4.2, https outbound call starts failing > > > (see below). If we change to http, then it works again. > > > > It seems like some class is marked as restricted in this release. Is > > > it intentional? What's the reason behind not allowing outbound https > > > call? > > > > Thanks. > > > > java.lang.NoClassDefFoundError: javax.net.ssl.SSLHandshakeException is > > > a restricted class. Please see the Google App Engine developer's > > > guide for more details. > > > at > > > com.google.appengine.tools.development.agent.runtime.Runtime.reject(Runtime > > > .java: > > > 51) > > > at > > > com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationExc > > > eption(URLFetchServiceImpl.java: > > > 114) > > > at com.google.appengine.api.urlfetch.URLFetchServiceImpl.access > > > $100(URLFetchServiceImpl.java:22) > > > at com.google.appengine.api.urlfetch.URLFetchServiceImpl > > > $1.convertException(URLFetchServiceImpl.java:86) > > > at > > > com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java: > > > 69) > > > at > > > com.pekca.main.delegate.RemoteDelegate$1.get(RemoteDelegate.java: > > > 163) -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: mapreduce - passing filters
This sounds reasonable. The challenge is mostly that finding split points for arbitrary queries is hard at the moment. I would hold off on doing this until after the 1.4.0 rollout is done. We have a new feature we're rolling out (enabled by 1.4.0, although we're going to have to do some work to backfill the sample with preexisting entities) that involves exposing a uniform random sample of keys as an index so that we can find split points easily. I think it should make this a lot easier. One other thought: instead of adding a GQL interpreter, you might just add a hook for loading a class provided by the user. That class would implement a Filter interface with a method that takes a Configuration and returns a Query object so in your example, mailing and timestamp would get passed in as Configuration parameters and a query object corresponding to the GQL statement you put would be built by a Filter class provided by the user. It would act kind of like a templating language for building queries. Make sense/sound like a good idea? On Nov 18, 7:01 am, Nacho Coloma wrote: > > I'm not entirely sure I understand > > the scope of the proposed patch. Are you thinking about adding filters > > > at the DatastoreRecordReader level? It's not entirely clear to me that > > that provides a benefit over just applying the filter at the start of > > the map() function. Totally willing to believe I'm missing something, > > though. > > The map() filter runs against your quota. This is OK for once-only tasks > such as schema upgrades, but Mappers can also be used for repetitive tasks > such as mailing, data cleanup, etc. For these cases, being able to work on a > subset of data is important (process only user accounts with mailing > enabled, for example). > > The biggest problem to resolve is how to specify the filter clause in > mapreduce.xml. I am considering implementing a GQL parser as simple as > possible, and inject servlet request parameters. Something like: > > > mapreduce.mapper.inputformat.datastoreinputformat.query > select * from users where mailing=:value1 and > timestamp<=:value2 > > > This implies porting the GQL implementation from python to Java, or > implementing an ANTLR-based parser. I feel like I am reinventing the wheel, > so any suggestion to use something that exists (or aim to a simpler design) > is welcome. > > On a logistical note, for nontrivial contributions, we require a CLA > > > from either you or your employer (depending on who owns the copyright > > for your work) before we can accept significant contributions. The > > relevant forms are at: > >http://code.google.com/legal/individual-cla-v1.0.html > > andhttp://code.google.com/legal/corporate-cla-v1.0.html. Feel free to > > email me privately if this is an issue. > > No problem with that. > > Regards, > > Nacho. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: mapreduce - passing filters
Hi, I'm the main author of the Java side of appengine-mapreduce. Sorry for the delay in responding - I don't normally lurk this group, but Ikai brought to my attention. We're happy to accept patches. We haven't been as good as we'd have liked about responding to them in a timely manner, since we've been buckling down on getting shuffle/reduce out, but no time like the present to try and correct that. I'm not entirely sure I understand the scope of the proposed patch. Are you thinking about adding filters at the DatastoreRecordReader level? It's not entirely clear to me that that provides a benefit over just applying the filter at the start of the map() function. Totally willing to believe I'm missing something, though. On a logistical note, for nontrivial contributions, we require a CLA from either you or your employer (depending on who owns the copyright for your work) before we can accept significant contributions. The relevant forms are at: http://code.google.com/legal/individual-cla-v1.0.html and http://code.google.com/legal/corporate-cla-v1.0.html. Feel free to email me privately if this is an issue. Thanks! -Fred On Nov 11, 1:06 am, Nacho Coloma wrote: > I know, but I would prefer to discuss it first (offline is OK). I don't want > to start submitting patches without first confirming that > > * I got the concepts correctly (there are a lot of possible > implementations). > * The design is coherent with whatever direction the project is taking. > * The project is open to submissions of this kind. > > Otherwise, it's a waste of time for everyone. > > > > > > > > On Wed, Nov 10, 2010 at 6:58 PM, Maxim Veksler wrote: > > Don't think Google would mind if you submit a patch, you have the source > > code under SVN available from here > >http://code.google.com/p/appengine-mapreduce/source/browse/ > > > On Wed, Nov 10, 2010 at 6:08 PM, Nacho Coloma wrote: > > >> Is there any option to contribute to this project? I am using hacks to > >> do the filtering, and it seems easy to implement. > > >> I would love to discuss things first, but appengine-mapreduce seems > >> like a read-only project to me. It's fine but I would like to help it > >> move forward. > > >> On Nov 8, 10:11 pm, "Ikai Lan (Google)" > >> > > >> wrote: > >> > There was talk about supporting it, but the priority is getting out the > >> > shuffle/reduce steps first. > > >> > appengine-mapreduce is an open source project. You can probably just > >> > edit/subclass DatastoreInputFormat.java and add the filter > >> functionality: > > >> >http://www.google.com/codesearch/p?hl=en#XwsseYUY0Ps/trunk/java/src/c. > >> .. > > >> > -- > >> > Ikai Lan > >> > Developer Programs Engineer, Google App Engine > >> > Blogger:http://googleappengine.blogspot.com > >> > Reddit:http://www.reddit.com/r/appengine > >> > Twitter:http://twitter.com/app_engine > > >> > On Mon, Nov 8, 2010 at 12:28 PM, Brad > >> wrote: > >> > > Ikai, > > >> > > Are there any plans to support filter functionality in the future? I > >> > > have a similar situation where I want to iterate over a collection > >> > > that is big enough to be used by the mapper framework, but much > >> > > smaller than the actual collection size of all entities of that kind. > > >> > > Brad > > >> > > On Nov 8, 11:56 am, "Ikai Lan (Google)" > >> > > > >> > > > >> > > wrote: > >> > > > No, the mapper will go over everything. If the number of entities is > >> > > small, > >> > > > you are better off placing an indexed property on ONLY these fields, > >> then > >> > > > just iterating over them with a cursor and either a local client > >> making > >> > > > multiple HTTP calls to your App Engine app or chained task queues. > > >> > > > -- > >> > > > Ikai Lan > >> > > > Developer Programs Engineer, Google App Engine > >> > > > Blogger:http://googleappengine.blogspot.com > >> > > > Reddit:http://www.reddit.com/r/appengine > >> > > > Twitter:http://twitter.com/app_engine > > >> > > > On Sun, Nov 7, 2010 at 5:58 AM, aswath satrasala < > >> > > aswath.satras...@gmail.com > > >> > > > > wrote: > >> > > > > Hello, > >> > > > > I want to perform few aggregations on entities of a kind that are > >> > > filtered > >> > > > > by the property. > >> > > > > The entities I want to perform the aggregation is small compared > >> to the > >> > > > > complete entity set. > >> > > > > Is there any way to pass the filters to the mapper, so that only > >> those > >> > > > > entities are retrieved and aggregation is performed > > >> > > > > *Regards > >> > > > > -Aswath > >> > > > >http://vs-accounting.appspot.com > >> > > > > Accounting for Indian markets.* > > >> > > > > -- > >> > > > > You received this message because you are subscribed to the Google > >> > > Groups > >> > > > > "Google App Engine for Java" group. > >> > > > > To post to this group, send email to > >> > > > > google-appengine-j...@googlegroups.com. > >> > > > > To unsubscribe from this group, send email to > >> > > > > google-appengine-java+unsubscr...@goo
