Re: [google-appengine] Re: Frameworks on GAE
Look likes there's a niche for a new framework. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/9RiGLtk17W4J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Frameworks on GAE
Jeff... regarding your Objectify optimisations... I'd be interested in details if you have any to hand please :-) I've personally made the introspection stage lazy load (moving it to trigger inside the getMetadata methods)... it's probably limited in scope to my specific needs (I'm not using any polymorphic queries, and I've made assumptions that all entities are @cached)... but it does the trick for me and reduces startup time significantly on my 60+ entities. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/AwtigY9GOCEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Frameworks on GAE
What I've taken away from these long threads is that: 1. GAE is already faster than other AWS and others when it spins up new instances 2. GAE's job is harder when there's a lot of initialization to do and lots of jars to scan. Precompilation helps but not enough for large apps 3. the scheduler doesn't suit all the needs. Sometimes it directs request to cold instances Number 3 cannot be fixed at this stage, unless Google allows us to tweak the scheduler directly. All I can do is trying to limit the app's size and static initialization as much as possible. I'm thinking of splitting my apps into multiple versions and removing jars from the frontend altogether. I use Vosao and am quite satisfied with its performances. I also use Restlet which is a bit slow at startup (I think it used to be slower with earlier versions of GAE). When I hit jaxb, that's terrible so I'm doing all XML processing in task queus. Haven't split the app yet but having non-default versions for task queues and cron jobs looks like acceptable logical partitioning. PS At the last IO they announced changes in the way they define apps, they will introduce the concept of multiple apps running on a virtual servers. Have you guys heard anything else? On Tuesday, 7 August 2012 09:04:07 UTC+12, Steve James wrote: Look likes there's a niche for a new framework. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/X5coDxp2UpAJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Frameworks on GAE
I have been struggling with the same issue of whether GAE is suitable for business apps or not (by business apps, I am thinking of order processing, credit processing system, shipping systems, billing, etc. eCommerce, etc.). For the past 10 years I have built web-based SaaS applications and there is hard to envision building applications that have hundreds of users divided into many dozens of roles that govern access to many hundreds of various parts of the application with lots of workflow management without a framework. I have no problem being able to develop my own user and permissions management system, but since almost everyone developing business applications needs this same functionality, it appears to be a great redundancy of reinventing the wheel by everyone. Almost every other application platform has a very active eco-system where developers make their modules available to others.Having said that, I don't favor a single comprehensive framework that attempts to do everything, but rather an approach where I can pick and chose which functionality/modules I want to add to my application. I would imagine there would be a place where developers would share such functionality and one could just download the code and then include it in the applicable files. This would apply not only to the core of a framework that deals with users, roles and permissions, but also to plug-in functionality such as discussion forum comments, file (Excel) import/export, CRUD scaffolding, etc. I also noticed that almost everyone who had been trying to build a framework on GAE seems to have given up. It is scary committing to build a web development business around GAE when one does not know if this platform is going to survive or whether the powers to be at Google are committed to protecting the investment of their business partners in their technology (in other words, ensuring that evolution provides a path for those who developed on previous versions to migrate to the newer versions relatively easily). Sooo. I hope someone will tell me where the GAE developers share their modules and building blocks and where I can find some tutorials about how to connect these building blocks to build a solid foundation to almost any web application because otherwise I really like much of what I see at GAE. On Sunday, July 22, 2012 5:32:41 AM UTC-4, glimmung wrote: Hi All, I've been reading, initially with amusement but more recently with concern, the dialogue (for want of a better word) between Brandom Wirtz and Jeff Schnitzer re. startup time/optimisation. Brandom has now made the following very strong statement: NO FRAMEWORKS. NONE. Deal with it. This leads me to ask the Google team for their position on this: Is it your position that GAE is an unsuitable platform for framework-driven apps? I'm using a framework, and trust the framework's authors to optimise their part of the piece as much as possible, but I'm paid to solve business problems, and am not about to dive into that timesink of esoterica. It's outside my skill-set, and properly so in my view. I've only really tinkered with GAE so far, and this is putting me off investing more time in what is starting to look like a risky platform for me. So Google peeps, if I want to write B2B apps using a framework, am I your market or not? Or is it your position that the low-level optimisation to balance start-up time and hosting cost will always be required? -- Cheers, PhilK -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/6Pxstk-bnmcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Frameworks on GAE
I doubt that Google will commit to any official statement. So here's my 2 cents, from a low-usage, rather complex B2B application. Frameworks are okay, to a certain degree, but be prepared to rework if you're successful We're using one heavy framework (Apache Wicket) and plenty of the regular libs and frameworks like JodaTime, Objectify, OpenID, LessCss (which depends on Rhino), Bouncy Castle etc. It's not a super-heavy app, but it's not trivial anymore either. We had to make certain modifications to make things work fine on GAE, conduct some profiling, etc, but mostly we're just writing business logic. I guess we would have had to spend some time on any platform we chose, no matter if on AWS, or Heroku, or anything else. Most of the issues we encountered were quite obvious right from the start. (Most notably, Wicket is really not that suited for GAE, but it made feature-development super-fast, so we kept it.) Some other issues only started appearing after switching to the HR datastore, but since that's the default these days, you'll probably encounter those issues right away as well. So I'd recommend you write a prototype, cram in as much of the technology you want to, and keep it running for a couple of days or weeks. Make sure you're mostly happy with performance and startup time. There are times when GAE behaves oddly and is 2-3 times slower than normal. Memcaches is usually super-reliable, until it some day isn't. So, you need to expect the unexpected, and optimise your requests so they are always reasonably fast. We aim for 500ms or less, so on a bad day if a request takes 2 or even 4 seconds, our clients may be unhappy, but not rioting. A request that's typically taking 2 seconds is too slow though (IMO). If your prototype has that kind of performance, rethink it. If you're happy with the prototype and want to move forward, I'd suggest also spending some time ensuring your service and database layers have clean interfaces. I never had to move away from GAE yet, but the recent Memcache issues had me very worried and looking for alternatives. Email, Cron, Memcache, Deploy-scripts, etc are great, but these are the things you can probably rework in a couple of weeks, even later on, especially since you can split the work between your team to a certain extent. But if the entire application (and not just a db-layer) was strongly dependent on GAE specifics, you might find it hard to port it over. There are times when things change, and then I'm happy we don't have sudden load spikes, but that thanks to a sales process that takes weeks, we know what's coming. If we see a certain part of the application is slower than it should be, we spend a couple of days improving it. That's something that simply doesn't work in Consumer apps. If you're getting featured, and it's your once-in-a-lifetime, then you probably have to heed Brandon's wise words and optimise (very) early, and not rely on frameworks, But if it's a normal B2B application, I think you can use a framework or two. Just make sure you test it with a prototype, yadda yadda etc. If you're super-successful, you'll hopefully make enough money in B2B-land, so you can afford paying someone to drop a framework that helped you kickstart your project faster. That's my strategy at least :) Cheers Per On Sunday, July 22, 2012 11:32:41 AM UTC+2, glimmung wrote: Hi All, I've been reading, initially with amusement but more recently with concern, the dialogue (for want of a better word) between Brandom Wirtz and Jeff Schnitzer re. startup time/optimisation. Brandom has now made the following very strong statement: NO FRAMEWORKS. NONE. Deal with it. This leads me to ask the Google team for their position on this: Is it your position that GAE is an unsuitable platform for framework-driven apps? I'm using a framework, and trust the framework's authors to optimise their part of the piece as much as possible, but I'm paid to solve business problems, and am not about to dive into that timesink of esoterica. It's outside my skill-set, and properly so in my view. I've only really tinkered with GAE so far, and this is putting me off investing more time in what is starting to look like a risky platform for me. So Google peeps, if I want to write B2B apps using a framework, am I your market or not? Or is it your position that the low-level optimisation to balance start-up time and hosting cost will always be required? -- Cheers, PhilK -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/OX-pCOHb8qYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at
Re: [google-appengine] Re: Frameworks on GAE
That's an interesting comment. I had the impression that with Java you would have only a small surface that's actually exposed to App Engine and porting would be very, very simple (with the occasional required profiling because it is a very distributed system). With Python, it's very hard to avoid contact with App Engine-specific interfaces. Coming from a mostly Django background, I found it interesting how conspicuously non-opinionated App Engine is. From my last contacts with Java web applications, I came out with the impression opinionated frameworks were frowned upon and cherry-picking libraries was the norm. Overall, it's been a fun experience. I like both approaches, but the more opinionated the framework, the more assumptions it makes about the underlying infrastructure - Django, for instance, really wants a relational database underneath it - it is and more friction should be expected when deploying on App Engine. Having said that, I had good experiences with micro-frameworks on App Engine. I used Tipfy once and never again wanted to be without the Werkzeug debugger (and, even though I never managed to make it work with the Python 2.7 runtime, I'haven't given up). Even my Django stuff running on EC2 use it. Of course, not in production. On Mon, Jul 23, 2012 at 2:32 PM, Per per.fragem...@gmail.com wrote: I doubt that Google will commit to any official statement. So here's my 2 cents, from a low-usage, rather complex B2B application. Frameworks are okay, to a certain degree, but be prepared to rework if you're successful We're using one heavy framework (Apache Wicket) and plenty of the regular libs and frameworks like JodaTime, Objectify, OpenID, LessCss (which depends on Rhino), Bouncy Castle etc. It's not a super-heavy app, but it's not trivial anymore either. We had to make certain modifications to make things work fine on GAE, conduct some profiling, etc, but mostly we're just writing business logic. I guess we would have had to spend some time on any platform we chose, no matter if on AWS, or Heroku, or anything else. Most of the issues we encountered were quite obvious right from the start. (Most notably, Wicket is really not that suited for GAE, but it made feature-development super-fast, so we kept it.) Some other issues only started appearing after switching to the HR datastore, but since that's the default these days, you'll probably encounter those issues right away as well. So I'd recommend you write a prototype, cram in as much of the technology you want to, and keep it running for a couple of days or weeks. Make sure you're mostly happy with performance and startup time. There are times when GAE behaves oddly and is 2-3 times slower than normal. Memcaches is usually super-reliable, until it some day isn't. So, you need to expect the unexpected, and optimise your requests so they are always reasonably fast. We aim for 500ms or less, so on a bad day if a request takes 2 or even 4 seconds, our clients may be unhappy, but not rioting. A request that's typically taking 2 seconds is too slow though (IMO). If your prototype has that kind of performance, rethink it. If you're happy with the prototype and want to move forward, I'd suggest also spending some time ensuring your service and database layers have clean interfaces. I never had to move away from GAE yet, but the recent Memcache issues had me very worried and looking for alternatives. Email, Cron, Memcache, Deploy-scripts, etc are great, but these are the things you can probably rework in a couple of weeks, even later on, especially since you can split the work between your team to a certain extent. But if the entire application (and not just a db-layer) was strongly dependent on GAE specifics, you might find it hard to port it over. There are times when things change, and then I'm happy we don't have sudden load spikes, but that thanks to a sales process that takes weeks, we know what's coming. If we see a certain part of the application is slower than it should be, we spend a couple of days improving it. That's something that simply doesn't work in Consumer apps. If you're getting featured, and it's your once-in-a-lifetime, then you probably have to heed Brandon's wise words and optimise (very) early, and not rely on frameworks, But if it's a normal B2B application, I think you can use a framework or two. Just make sure you test it with a prototype, yadda yadda etc. If you're super-successful, you'll hopefully make enough money in B2B-land, so you can afford paying someone to drop a framework that helped you kickstart your project faster. That's my strategy at least :) Cheers Per On Sunday, July 22, 2012 11:32:41 AM UTC+2, glimmung wrote: Hi All, I've been reading, initially with amusement but more recently with concern, the dialogue (for want of a better word) between Brandom Wirtz and Jeff Schnitzer re. startup time/optimisation. Brandom has now made the
Re: [google-appengine] Re: Frameworks on GAE
On Mon, Jul 23, 2012 at 10:32 AM, Per per.fragem...@gmail.com wrote: I doubt that Google will commit to any official statement. So here's my 2 cents, from a low-usage, rather complex B2B application. Frameworks are okay, to a certain degree, but be prepared to rework if you're successful While I certainly would not disagree with this, I think the important discussion - the one relevant to the question of whether Google should spend effort refining the scheduler - is what kind of frameworks are reasonable? Let's take persistence. Does anyone honestly believe that it's reasonable to write complicated applications using only the Java low-level api? So you pick a lightweight persistence framework like Objectify. Great choice! ;-) I'd like to think I've done a pretty good job of optimizing Ofy, but there is a problem I do not have any way of getting around: In order to map POJOs to the datastore, Objectify has to introspect the POJOs. Because queries can return polymorphic entities, this introspection has to happen up front. Everything is great when your app has a handful of classes. But you add features and add features and now you have 30, 50, 100+ entity classes. My domain model currently has 56 classes and it takes Objectify 4-5s to load and introspect them. This isn't a classpath scanning issue - the bottleneck seems to be the classloader itself. My point: The more complicated your application is, the more startup time is going to be impacted by a framework like Objectify. Also, the more complicated your application is, the more you need a framework like Objectify. It's a catch-22. So this 4-5s hit is just something I have to accept. There's no magic optimization I can do to make it go away - believe me, I'm in the best position of anyone to do it, and I don't see any fixes for Objectify that preserve important features like polymorphism. This hit is going to get worse when my domain model doubles, triples, etc over the next few years. If we say all apps must start up in 5s then we've basically said that Brandon is right and we must stop using all frameworks, even at the foundational level of persistence. I certainly hope nobody takes that seriously. I'm not saying that it's a great idea to throw in frameworks that make your app take 50s to start up... but I am saying that Java startup latency is not a problem we can optimize away. It's something we have to deal with one way or another, and it would be best for everyone if Google dealt with it at the scheduler. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Frameworks on GAE
Hi Brandon, I use both Objectify and low-level access. Objectify for most fixed-field entities, and low-level for small entities (for session, as i override GAE's Java session) and entities where it can have 30+ fields but only 5+ filled in average (but need to store them in 1 entity since people often query them together). On Monday, 23 July 2012 11:29:15 UTC+8, Brandon Wirtz wrote: Thomas, How do you access the datastore? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/oYYMWhDXEIMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Frameworks on GAE
Hi Phil, Coming from Java world, I'm very much spoiled with frameworks. When I first created an app for GAE, I used Spring. But due to unacceptable start-up time, I ditched it and just use Servlet+JSP. I miss a lot of convenience (esp. Spring Security), but luckily GAE provides other goodies, things like sending email, cron, persistence, are now taken for granted. I still use a handful of 3rd party components: joda time, SiteMesh, Scala (I'm talking from JAR point of view, an extra 8MB), and some other small libraries. Oh, and you can always-on your instances. On Sunday, 22 July 2012 17:32:41 UTC+8, glimmung wrote: Hi All, I've been reading, initially with amusement but more recently with concern, the dialogue (for want of a better word) between Brandom Wirtz and Jeff Schnitzer re. startup time/optimisation. Brandom has now made the following very strong statement: NO FRAMEWORKS. NONE. Deal with it. This leads me to ask the Google team for their position on this: Is it your position that GAE is an unsuitable platform for framework-driven apps? I'm using a framework, and trust the framework's authors to optimise their part of the piece as much as possible, but I'm paid to solve business problems, and am not about to dive into that timesink of esoterica. It's outside my skill-set, and properly so in my view. I've only really tinkered with GAE so far, and this is putting me off investing more time in what is starting to look like a risky platform for me. So Google peeps, if I want to write B2B apps using a framework, am I your market or not? Or is it your position that the low-level optimisation to balance start-up time and hosting cost will always be required? -- Cheers, PhilK -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/3SVruDN4gUUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: Frameworks on GAE
Thomas, How do you access the datastore? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
