[appengine-java] Re: URL Fetch problems

[aldrinm]

Can you try encoding the access token value ,

"https://graph.facebook.com/me?
access_token="+URLEncoder.encode("316426390227|
2.MHJgJ9J6fuNwKern3vyHgg__.3600.129933-1185632060|oZFo-
ku78icKHAx_aFvNw dItx9U", "UTF-8")

?


On Mar 6, 10:04 am, Kim Kha Nguyen  wrote:
> OS: Ubuntu 10.10
> Java SDK: OpenJDK 6
> AppEngine: 1.4.2
> WebToolkit: 2.2.0
>
> I use URLFetch (doc here:http://goo.gl/hPraM), and url 
> is:https://graph.facebook.com/me?access_token=316426390227|2.MHJgJ9J6fuNwKern3vyHgg__.3600.129933-1185632060|oZFo-ku78icKHAx_aFvNw
>  dItx9U&expires_in=5101
>
> If I run in Firefox, error message is:
>
> java.lang.IllegalArgumentException: Invalid uri 
> 'https://graph.facebook.com/me?access_token=316426390227|2.MHJgJ9J6fuNwKern3vyHgg__.3600.129933-1185632060|oZFo-ku78icKHAx_aFvNw
>  dItx9U&expires_in=5101': Invalid query
>
> But if I run in Chrome, error message is:
>
> javax.net.ssl.SSLHandshakeException: Could not verify SSL certificate 
> for:https://graph.facebook.com/me?access_token=316426390227%7C2.MHJgJ9J6f...
>
> Please help me... I'm amateur in GAE.
>
> Thank all of you!
>
> Note: I opened a issue here:http://goo.gl/bxLgu

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: uploading 100k lines into the datastore is enough to max out the CPU time?

[Didier Durand]

Hi,

The first issue that comes to mind in missing indexes: so huge scans
of all existing data when you upload an additional line.

What are your indexes ?

regards

didier

On Mar 6, 12:43 pm, Cláudio Coelho  wrote:
> Hi,
> My application needs a db of the world cities. I have a file that has about
> 100k lines  and I want to use it to populate the datastore.
> My first attempt failed because I didn't use tasks, so I was only able to
> upload a limited set of cities. It spent 2% of CPU total time on the app
> engine.
> I then changed the approach to use tasks.
> I basically read the file once to determine how many lines does it have and
> then invoke tasks to read batches of 15k lines (supplying a start and end
> indexes).
> These invoked tasks (createCities(int start, int end)) read the file again,
> get a list of 15k lines and then process it.
> The whole process takes about 15 seconds on my machine, however, when I do
> this in the app engine, it takes 15 minutes (or more) and maxes out the 6.5
> hours of CPU time! I know there are plenty of optimizations I should do to
> this process, but that doesn't seem to justify the 6.5 hours, so I must be
> doing something seriously wrong.
>
> Does anybody have any clue of what I'm doing wrong? (Code attached)
>
> Thanks!
>
> private void createCities() {
> try
> {
> int count = countCitiesToLoad();
> final int batchSize = 15000;
> for(int start=0;start {
> int end=start+batchSize;
> Queue queue = QueueFactory.getQueue("dbQueue");
> TaskOptions topt = TaskOptions.Builder.withUrl("/dbservlet");
> topt.param("command", "createCities");
> topt.param("start", ""+start);
> topt.param("end", ""+end);
> queue.add(topt);
> logInfo("Dispatched order to create cities "+start+" to "+end);
> Thread.sleep(500);}
> }catch(Exception e)
>
> {
> e.printStackTrace();
> logError(e.getLocalizedMessage());
>
> }
> }
>
> private void createCities(int start, int end)
> {
> try
> {
>  logInfo("Reading cities "+start+" to "+end);
> BufferedReader br= new BufferedReader(new FileReader("data/cities.csv"));
> String line=br.readLine();
> int counter=0;
> PersistenceManager pm = PMF.get().getPersistenceManager();
> ArrayList lines = new ArrayList();
> while(line!=null || counter {
> if(counter>=start && counter  lines.add(line);
> counter++;
> line=br.readLine();}
>
> br.close();
> logInfo("Adding cities "+start+" to "+end);
> createCities(lines);
> logInfo("Done cities "+start+" to "+end);}
>
> catch(Exception e)
> {
> e.printStackTrace();
> logError(e.getLocalizedMessage());
>
> }
> }
>
> private void createCities(ArrayList lines)
> {
> if(lines==null)
> return;
> PersistenceManager pm = PMF.get().getPersistenceManager();
> HashMap countryMap = loadCountryMap();
> try{
> for(String line : lines)
> if(line!=null)
> {
> String fields[]=line.split(",");
> if(fields.length<7)
> logError("length error in line:"+line);
> else
> {
> Location loc = new Location();
> loc.setName(fields[2]);
> loc.setLatitude(Double.parseDouble(fields[3]));
> loc.setLongitude(Double.parseDouble(fields[4]));
> loc.setPopulation(Integer.parseInt(fields[6]));
> {
> Country c = countryMap.get(fields[5]);
> if(c==null)
> logError("Failed to get country for:"+line);
> else
> {
> loc.setCountryId(c.getCountryId());
> loc.setCountry(c.getName());
> pm.makePersistent(loc);}
> }
> }
> }
> } catch (Exception e) {
>
> e.printStackTrace();
>
> } finally {
> pm.close();
> }
> }
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] What is maximum number of entities I can update within the 30 sec time limit of App engine ?

[suersh babu]

Hi,

I like to known what is the maximum number of entities I can update within
the 30 sec
time limit of App engine.

Let say If I have a kind called "Employee" which  has 2000 entities and each
of this entity have
10 Properties,  so if I need to update all of this 2000 entities of
Employee,  what is the maximum
entities that I can update  within 30 sec time limit.

Any suggestion much appreciated.


*Regards**

Suresh Babu G*

http://www.AccountingGuru.in 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Design question for SOA / Facebook like Platform on GAE

[charming30]

Hello..

I am planning for a SOA-Like Architecture for my Facebook-like App on
GAE Java.

- All Apps are PRIVATE only meant for my platform.
- One (Appspot App) will act like a service and provide Social
Networking and User Database services - This is the main App
- While others will act like individual websites that use the services
of the above to provide social networking features in their local
site.

Architecture Design I have in mind so far:
- All Apps will be individual Apps on Appspot, since building and
maintaining a single App to include all this will not be practical.
- Web Services using REST/JSON, SSL for authentication
- All Apps on Appspot will be Billing enabled, so Google is Happy

Please advise if you see any risks and what improvements can be made
to this?

Thanks a ton.

Regards
C30



-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] in my logs

[lp]

hi all

i am getting loads of the following trace in my logs.

#
I 2011-03-06 20:14:32.566 [minglegeo/1.348824067992053184].:
04

for each request i get about 30 lines of this stuff. why?

i dont know what is causing it to be generated?

i have searched for System.out but i am only using log.info, error
etc.

any ideas?

-lp

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: appcfg deployment hangs at jsp compile

[lp]

ok got to the bottom of this.

it seems that the javac wasnt hung, just runing slow. about 60 s/
jsp page.

by removing the datanucleus-enhancer-1.1.4.jar from my WEB-INF/lib the
problem was resolved.

i guess datanuclues jars should be in there but what a waste of 8
hours ;-(

hope this help another.


-lp

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: how would I setup ssl in eclipse dev http server

[Mark]

My app will require secure https connections and I would like to ensure the 
code that detects if the application is connected via a secure connection is 
always in place.  

I am thinking things are not going to be as straight forward if Google is 
using an embedded jetty server.  I haven't found where jetty exists to edit 
configuration files.  I have looked into setting up Jetty although I was 
derailed when I could not find a "jetty" folder.

My other options are to have code that detects if it is the development 
environment assume secure connection.  Although I would have to ensure that 
code is fully tested when deployed at the server.  I am not a fan of this to 
have code not tested until it is deployed into a production environment.

I'll keep researching to see if I can figure this out.  I am hoping another 
developer might have some tips for me.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Data store transfer

[Neill]

The way I've exported/imported data in the past was using CSV file 
transfers. I have some code laying around to handle it, if you want to send 
me a PM.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Could Not Verify SSL Certificate

[Neill]

Have you tried turning on java.io.net debugging to get more information?

Add this JVM arg to the appengine startup runtime, -Djavax.net.debug=ssl

You may need to add keys to the cert file, but it should be installed by 
default by the plugin.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: how would I setup ssl in eclipse dev http server

[Neill]

I'm not sure why you would need or want to, but since the dev server is 
using Jetty, have you looked into how to set it up? I've also seen an option 
to run on a remote server, but there shouldn't be any need to run https from 
an application code standpoint, which is the same whether SSL or not.

A quick search turned up some information on using Jetty and SSL. Good luck!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Logging file size

[JT]

Hi didier, thanks for your reply.
On Mar 4, 2011 11:11 AM, "Didier Durand"  wrote:
> Hi,
>
> Interesting question!
>
> From the quota page: http://code.google.com/appengine/docs/quotas.html,
> it doesn't seem that there is not any official limit on logging.
>
> Maybe some googler will let us know if there is one anyway or not.
>
> On Mar 4, 4:30 pm, JT  wrote:
>> I know there is a limit of 6.5 hours CPU time per day for GAE4J, but if I
>> turn on verbose logging onto the console like using System.out.println,
does
>> the size of the logging limited by GAE in any way?
>>
>> Thanks for any hint!
>
> --
> You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
> To post to this group, send email to
google-appengine-java@googlegroups.com.
> To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Collection object not being updated

[Cosmin Stefan]

Hey,

I cannot use a transaction as I need to change multiple entries (as u
see, for each BigClass participant i have to remove an object from the
collection).

Regarding the pm.makePersistent() call, I have tried with it before
and it didn't do anything different. Plus that on the GAE
documentation page they say it's not needed, as the pm tracks all the
changes.

I have discovered another thing related to this problem. As I've shown
you, in the same persistence call I try to do these (remove smth from
the collection) changes and I also modify other objects. The weird
fact is that the last changes I make are persisted, while the others
aren't. Why is that?

Regarding JPA I'll look into it, but it's kind of difficult as I
already have a huge part of my code written with JDO.

Anyway, looking forward to hearing new suggestions!

Thanks,
Cosmin

On Mar 1, 6:59 pm, Ian Marshall  wrote:
> Hi Cosmin,
>
> I do not see any calls to
>
>   pm.makePersistent(...);
>
> I use this to persist newly-created persistent instances.
>
> I know that you do not use transactions, but I do. Within an active
> transaction, one can update persistent instances and even persist or
> delete entity group child instances without calling
>
>   pm.makePersistent(...);
>
> Have you looked at the GAE persistence blog of Max Ross of Google?
> There are some excellent working examples there...
>
> Ian
>
> On Feb 28, 6:55 pm, Cosmin Stefan 
> wrote:
>
> > So no ideea anyone?
>
> > On Feb 25, 11:42 pm, Cosmin Stefan 
> > wrote:
>
> > > Hey,
>
> > > I have an issue while trying to update one object in a collection,
> > > using JDO.
>
> > > Here are the facts:
> > >    o i have a class (let's call it BigClass), that has an embedded
> > > class(SmallClass) containing an ArrayList.
> > >    o I DONT use/need a transaction
> > >    o I query the database to get a List of BigClass items that should
> > > be modified. I iterate through each of them and I...
> > >    o I remove an element from the list in the SmallClass embedded in
> > > the current BigClass, the changes are not ALWAYS persisted
> > >    o if I print (log) the object after the change, it looks modified,
> > > but if i check the DataViewer, the object was not updated
> > >    o i even tried using JDOHelper.makeDirty on the BigClass, with the
> > > fieldName SmallClass, and it still doesn't work.
>
> > > Some relevant code:
>
> > >                                 Query q = 
> > > pm.newQuery(BigClass.class,"id==:ids");
> > >                                 List 
> > > participatingUsers=(List)
> > > q.execute(participantIDs);
>
> > >                                 //Update the participants
> > >                                 ListIterator 
> > > it=participatingUsers.listIterator();
> > >                                 BigClass participant;
> > >                                 boolean modified;
> > >                                 while(it.hasNext())
> > >                                 {
> > >                                         participant=it.next();
> > >                                         participant.list.remove(smth);
>
> > > JDOHelper.makeDirty(participant,"collection");
> > >                                }
> > >                                
> > >                                modify other objects
> > >                                
> > >                                pm.close()
>
> > > Another thing is that some of the changes I make after this part are
> > > persisted...
>
> > > So, if you have any suggestions, shoot pls!
>
> > > Cosmin

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] uploading 100k lines into the datastore is enough to max out the CPU time?

[Cláudio Coelho]

Hi,
My application needs a db of the world cities. I have a file that has about 
100k lines  and I want to use it to populate the datastore.
My first attempt failed because I didn't use tasks, so I was only able to 
upload a limited set of cities. It spent 2% of CPU total time on the app 
engine.
I then changed the approach to use tasks.
I basically read the file once to determine how many lines does it have and 
then invoke tasks to read batches of 15k lines (supplying a start and end 
indexes).
These invoked tasks (createCities(int start, int end)) read the file again, 
get a list of 15k lines and then process it.
The whole process takes about 15 seconds on my machine, however, when I do 
this in the app engine, it takes 15 minutes (or more) and maxes out the 6.5 
hours of CPU time! I know there are plenty of optimizations I should do to 
this process, but that doesn't seem to justify the 6.5 hours, so I must be 
doing something seriously wrong.

Does anybody have any clue of what I'm doing wrong? (Code attached)

Thanks!


private void createCities() {
try
{
int count = countCitiesToLoad();
final int batchSize = 15000;
for(int start=0;start lines = new ArrayList();
while(line!=null || counter=start && counter  lines) 
{
if(lines==null)
return;
PersistenceManager pm = PMF.get().getPersistenceManager();
HashMap countryMap = loadCountryMap();
try{
for(String line : lines)
if(line!=null)
{
String fields[]=line.split(",");
if(fields.length<7)
logError("length error in line:"+line);
else
{
Location loc = new Location();
loc.setName(fields[2]);
loc.setLatitude(Double.parseDouble(fields[3]));
loc.setLongitude(Double.parseDouble(fields[4]));
loc.setPopulation(Integer.parseInt(fields[6]));
{
Country c = countryMap.get(fields[5]);
if(c==null) 
logError("Failed to get country for:"+line);
else
{
loc.setCountryId(c.getCountryId());
loc.setCountry(c.getName());
pm.makePersistent(loc);
}
}
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
pm.close();
}
}



-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.