[web2py] Zero-downtime deploy and route mapping script for Bluemix
Hi all,
A small script I'd like to share -- I use it for zero-downtime Web2Py
deploys to Bluemix (using a so-called Blue-Green deploy method).
The script takes advantage of knowing that domain-mapping to apps is in
'routes.py', using that to map routes in Bluemix to the new deploy.
I posted earlier about how to prepare a manifest.yml file which does the
rest of the work mapping services and setting memory/instances, etc.
In short, I run this deploy.py from my app directory and it does the rest.
Regs,
Duncan.
import subprocess, getpass
# Set your config items here:
username = 'per...@example.com'
org = 'per...@example.com'
space = 'dev'
app = 'thisismyappname-main'
# Get password and log in
pwd = getpass.getpass('Password:')
print('Please wait')
print(subprocess.check_output(['cf','auth',username,pwd]))
print(subprocess.check_output(['cf','target','-o',org,'-s',space]))
# Check if we are green- or blue-mode currently
apps = subprocess.check_output(['cf','apps'])
if app + 'green' in apps:
color = 'blue'
else:
color = 'green'
print('Pushing: '+color+'. Please be patient This could take a while
depending on your upload speed.')
print(subprocess.check_output(['cf','push',app+color,'-f','./manifest.yml']))
execfile('app/routes.py')
for domain in routers['BASE']['domains']:
print('\nMapping route: '+domain)
try:
print(subprocess.check_output(['cf','map-route',app+color,domain]))
except:
print(Skipping +domain)
try:
print(subprocess.check_output(['cf','map-route',app+color,'.'.join(domain.split('.')[1:]),'-n',domain.split('.')[0]]))
except:
print(Skipping with host +domain)
if color == 'blue':
print(subprocess.check_output(['cf','delete','-f',app + 'green']))
else:
print(subprocess.check_output(['cf','delete','-f',app + 'blue']))
print(Blue-green deploy complete.)
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
web2py-users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Here's how to run Web2Py on IBM BlueMix
Hi Massimo, My pleasure. You may remember me from trying to get Web2Py running under Jython in a Websphere servlet container environment a while back. I didn't ever get there fully. So having this way to run in the IBM ecosystem is nice. I have been emailing a bit with one of the contacts from IBM. I'll ask about a blog, and perhaps if warranted, a short video tutorial would also be good. One minor detail I'd add to a tutorial is remembering to add the fake-migration flag to the JazzHub Web2Py codebase after initial deploy, to prevent subsequent deploys without db changes from getting confused with a set up db, but no updates in database migration files. In fact, this would probably a good reason for showing off BlueMix spaces, where one space could be called 'for_single-instance_migration_only' or similar. Also a minor error in my original post: where I mentioned moving sessions to the db. It seems that is already the default, you just need all instances pointing at the same db. I'll let you know how I get on with an IBM update/blog post. Regs, Duncan. On Thursday, 8 May 2014 22:25:49 UTC+10, Massimo Di Pierro wrote: Thank you Duncan, this is very valuable. There should be a setup script for this shipping with web2py. Is there any IBM blog wjere we could advertise it? Massimo On Tuesday, 6 May 2014 19:29:48 UTC-5, duncan macneil wrote: Hi all, I've been trying out the www.bluemix.net IBM PaaS. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Here's how to run Web2Py on IBM BlueMix
On Thursday, 8 May 2014 22:29:31 UTC+10, Willoughby wrote: Is Azure in your shoot-out? I'd be interested in hearing those results... Hey, woah, I'm the cloud shootout guy now? ;-) Cool! I have had a small-to-serious look at Azure, Google Compute, Amazon, Rackspace, Heroku, etc. There are more serious cloud comparison efforts than mine, for sure! In the end, my prediction is that cloud computing, storage, CDN etc will become such a hyper-commodity that your future app-hosting dash will instead be via a cloud aggregator. That future cloud aggregator would report something like: Good morning, Dave. Your application 'MyCoolApp' is healthy. It is currently running across 4 continents and 17 cloud providers. CDN has been weighted to 3 providers based on our 'Gamma' speed/price ratio, last updated on Tuesday. One provider was dropped due to a terms-of-service change. Your current redundancy profile allows you to drop 1 continent or up to 5 providers at any time without business interruption. Your application load balancers route to compute-intensive servers in the UK between 2:30 and 4:30pm weekdays GMT due to a weekday spike detected for which your compute pricing was previously not optimal. Your data is hosted on a shared-secrets algorithm model. At least 6 providers would need to collaborate in order to decrypt the data they host for you. Your monthly fees total is currently 16.2 Cayman Bitcoins, for which you have earned 0.001 Frequent SpaceMiles points. Thank you. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Here's how to run Web2Py on IBM BlueMix
I don't wanna be one of those guys who just compares on price alone but I will obviously be comparing price to other options. (By which I mean 'value for money' not 'lowest price' SmartCloud got criticism for being 'more expensive' than the lowest common denominator, but I doubt those who complained actually took out those servers for a test-run to see how snappy they were for the price-points on offer.) I'd also need to spend time working out how things like backups, billing (and on-billing), alerts, CDN and other features work before diving in. Nothing too serious, but it does take an investment of time to learn these things. But to sum it all up it really has to do with fit for purpose. On smaller projects, there's not too much going on in one release cycle. So a simple 'post-and-host' will do for me. On larger projects where there are big releases, bug fixes, custom field-office releases and the need for continuous testing between all these, I can see how the near-automated devops would save me having a full-time devops costs. If you'll indulge me: There's a real opportunity here for IBM to sell not just to its own direct customers, but to its customers customers. To see what I mean, put yourself in the shoes of a small-to-medium software business. The whole time is spent selling and re-selling the business the proposition. What helps me is not so much the marketing collateral which convinces *me *to use BlueMix, but marketing collateral that I can use to convince my customers that I'm in the right zone using BlueMix. When my customers ask Where will my data live? I often answer With Amazon. You'd be surprised how many respond, What, the book sellers?. I'd rather say: Your data and business-continuity will live with IBM. Take a look at this (imaginary) url, which explains all the details: http://bluemix.net/once-my-developer-is-done-building-my-awesome-software-how-will-IBM-safeguard-my-data-and-keep-my-app-running-smoothly? That's obviously a ridiculous example, but you get the idea. On Thursday, 8 May 2014 03:22:14 UTC+10, Dave S wrote: On Tuesday, May 6, 2014 5:29:48 PM UTC-7, duncan macneil wrote: Hi all, I've been trying out the www.bluemix.net IBM PaaS. It took me a bit of digging to realise there is in fact a Python runtime. But you need a buildpack. The setup is different enough from Heroku to warrant this mini-HowTo: Hey, thanks for posting this. Sounds like you'd give BlueMix a thumbs up; what would sway you to [or not to] go into production with it, compared with other hosting? /dps -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Here's how to run Web2Py on IBM BlueMix
Hi all, I've been trying out the www.bluemix.net IBM PaaS. It took me a bit of digging to realise there is in fact a Python runtime. But you need a buildpack. The setup is different enough from Heroku to warrant this mini-HowTo: Step 1: Use the source version of Web2Py and copy the entire lot into a folder path: 'env/app' so that web2py.py is in the 'app' folder. Step 2: You need to have a requirements.txt file in the 'app' folder -- even if it is empty. (Although you might want to add a single line with just psycopg2, for example.) Step 3: You need to have a manifest.yml in the 'en'v folder (top level of the project) with something like the following content: applications: - host: mycoolapp name: mycoolapp command: python web2py.py -a 'yourpassword' -i 0.0.0.0 -p $PORT path: app domain: ng.bluemix.net mem: 128M buildpack: git://github.com/ephoning/heroku-buildpack-python.git instances: 1 You need to make sure that 'mycoolapp' is changed to be unique and 'yourpassword' is changed, too, obviously. Step 4: Push this entire 'env' directory to BlueMix by navigating to the 'env' directory and running: cf push This assumes you have installed the CloudFoundry CLI command line tool, although it also works if you put all the same resources described above into an empty/new linked JazzHub git project from BlueMix and simply press 'Deploy' from the web page provided. Handy. Some points to consider: 1. The file structure will hold and run the default SQLite db internally, but that's probably not what you want because the files are not designed/guaranteed to be persistent (flushed with a new/fresh deploy, for instance). Instead you can add PostgreSQL and use that for data persistence -- BlueMix gives you a Postgres connection URL, ready to use! You just add it as a service to the app. 2. If you change instances = 1 to a larger number, then you'll probably also want to add a Redis service and use Redis for sessions (or a db for sessions). This will most likely allow people to remain logged in when they are load balanced via round robin to another instance, but I have not tested that. (What I *did* test is that you don't stay logged in with multiple instances running, which suggests for larger-scale apps with multiple instances running you'l want a central Redis for sessions.) 3. You get a SSL-ready address at something like https://mycoolapp.ng.bluemix.net (wildcard certificate exists for *.ng.bluemic.net), but since it seems to have some kind of reverse proxy, Web2Py doesn't detect that the admin is being used under HTTPS: so you might want to also store errors in the DB, too. The default with Web2Py is errors in stored the file path. Or modify the app admin to ignore HTTPS (just remember to use HTTPS when logging in!) I hope this helps someone. BlueMix is free during Beta. Pricing to be announced in June, I think. It will be interesting to see what the pricing is. Overall I think the service is pretty good -- especially if you need formal DevOps. IBM have made the DevOps thing quite neat in BlueMix. I particularly like being able to 'click the Postgres hexagon' and get a DB connection URL back without any install/setup issues. Regs, Duncan. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] This DAL feature was not obvious to me at first, may help others (avoid overly-complex sql joins)
Hi all, So I always knew there was *this *shorthand for getting the field attributes of a 'reference' type field: row.competition_entry.entrant.name What I did not discover until recently is that ALL references can be followed as far as they have been defined. For example: row.competition_entry.entrant.registered_by.name Works even if the query is just: db(db.competition_entry).select() I was jumping through hoops because before even attempting the above, I was doing: db(db.competition_entry.registered_by == db.registrant.id).select() Which is not needed if registered_by is a reference to registrant. I had assumed that you needed the joins to have second and third degree information. In fact, for a complex data structure, this works just fine with that simple single-table select: row.competition_entry.competition_section.category.sponsor.name That is, we a sponsor for each whole competition category (containing multiple sections), into which competitors place their entries. Getting the name of the sponsor really does work like that, even if the query was just db(db.competition_entry).select() -- You would probably appreciate how complex my queries had been to get the sponsor name, just by misunderstanding the DAL! I really hope this helps someone -- now I am going off to remove a LOT of table joins in my queries! As an aside, does this mean my Web2Py sessions will be reasonably large, because competition entry rows on their own have a lot of baggage about competition-section-category-sponsors and registered_by info?, ...or is Web2Py smart enough to only call in that extra information if required? (I find it hard to believe it is on-demand, since Web2Py would surely need to launch another query just for these references, yes?) Thanks, Duncan. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: This DAL feature was not obvious to me at first, may help others (avoid overly-complex sql joins)
Sorry, even more complex: On Saturday, 15 February 2014 21:19:06 UTC+11, duncan macneil wrote: I was jumping through hoops because before even attempting the above, I was doing: db(db.competition_entry.registered_by == db.registrant.id).select() Astute readers would understand it was even worse, like: db((db.competition_entry.entrant == db.entrant.id) (db.entrant.registered_by = db.registrant.id)).select() Just for the simpler example. Yes, I was tying myself into knots! -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: This DAL feature was not obvious to me at first, may help others (avoid overly-complex sql joins)
Hi again, Sorry to keep posting here, but I really think this information must be useful as it is to me. Now, you can't just use deep references in the format attribute for a table, so you use a lamda. Something like: format=lambda r: r.schedule.company.name + ': ' + r.schedule.name + ': ' +r .descrip + ' - ' + r.done_by.name Also, when it comes to sorting, you can't just say: select(sortby=db.mytopic.schedule.company.name) nor can you use: select(sortby=db.company.name) (It works, but you get a loose join and too many rows returned. Probably distinct would fix, but, meh.) So instead you can wait until the results are returned as use: rows.sort(lamba row: row.schedule.company.name) and to sort multiple columns: rows.sort(lambda row: row.schedule.company.name + row.schedule.name) Again, hope this helps. It is speeding up my designs very much. Finally, don't use this pattern if you allow nulls in the table, such as having a topic which is not forced to be assigned to schedule/company to begin with, for instance. Or if you do, make sure you check for None type objects in these deep references. Regs. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: This DAL feature was not obvious to me at first, may help others (avoid overly-complex sql joins)
Okay, good to know. I do have one quite data-intensive, high-demand query, and that is luckily one for which I had put all the joins in the original query. So good to know that was not wasted effort. Would be good to have a convenience option like: db(db.event).select(join_related=True) Which would be used to have Web2Py do all the known joins and return the wider column set. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups web2py-users group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: IMPORTANT - NEED HELP: more speed improvements to be tested
OK, I have downloaded the latest trunk from Git and have imported a simple application: A copy of my site at http://www.ec2scripts.com, which is based on Plugin_Wiki and seems to (mostly) work. On the page which is the equivalent of http://www.ec2scripts.com/welcome/plugin_wiki/page/submit-your-recipe instead of Sorry, you need to be registeredhttp://www.ec2scripts.com/welcome/default/user/register and logged in http://www.ec2scripts.com/welcome/default/user/login to submit I just get: system error But there are no errors in the admin app, and the console running web2py.py does not spit out any system.out type stuff. Sorry I cannot be more specific. I can provide the wiki template that generates that page, if required. Not sure of the exact version I just pulled. The download is labelled web2py-web2py-6d78589, and the top of my CHANGELOG file says: ## 2.00.1 ### DAL Improvements - Support for DAL(lazy_tables=True) and db.define_table(on_define=lambda table:), thanks Jonathan - MongoDB support in DAL (experimental), thanks Mark Breedveld - geodal and spatialite, thanks Denes and Fran (experimental) etc Is this the kind of testing you were after, or were you wanting people to change over to lazy tables to see if that works, the way Ron did? --
[web2py] Re: need a simple wiki...
Works well!
Just wondering are we keeping the ability to embed template code with
the new auth.wiki?
It would be handy to be able to {{=include.things}} using nothing more than
the wiki form (for advanced users). Or do we always need to use LOAD now or
something?
The menu makes sense to me. Glad we kept the plugin_wiki style of having it
in one place.
--
