Re: [web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

No, it is just a singleton module, just like Auth or Cache.

Re: [web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

Andrew,

If I'm understanding you properly, the only major difference between the 
business process workflow (the one I am writing) and the data integration 
workflow you mention is that one requires human interaction, while the 
other is handled by automated systems. Is that fairly accurate? If so, then 
I could see my workflow engine being used for both types of workflows. I 
have well-defined the human interaction of workflows, but never touched on 
the automated aspect, though it was a consideration during the design of 
the engine.

Let me give an example of what I understand an automated workflow to be. 
Say we are running an e-commerce site and we want to create an automated 
workflow for the ordering process. This is how the process should go:

   1. Customer orders something
   2. Payment is processed
   3. Order confirmation email sent to customer
   4. Order is shipped
   5. Shipping confirmation email sent to customer

You could certainly just hardcode the "workflow" of each step, but then 
there's not a lot of flexibility. For example, maybe you want to insert an 
extra step later (i.e. quality control before shipping). You could use the 
workflow engine I'm creating to handle this scenario. Let me explain how:

You would have several parts to your e-commerce site:

   - Main web interface
   - Payment processing thread
   - Email thread for order confirmations
   - Email thread for shipment confirmations

I use the term "thread", but this would be any process or web2py scheduler 
instance that runs in the background waiting for something to do. You would 
have a process waiting for payments to process and processes waiting for 
emails to send. This could potentially be the workflow you would use for 
the ordering and shipping process:

# auth_group 50 = Automated payment processor group
# auth_group 51 = Automated order confirmation email group
# auth_group 52 = Shipping group of people that package the order
# auth_group 53 = Automated shipment confirmation email group

workflow_engine.create_template('Process order',
Step('Process the payment', dict(group=50)),
Step('Send order confirmation email', dict(group=51)),
Step('Package and ship order', dict(group=52)),
Step('Send shipment confirmation email', dict(group=53))
)

The trick here being that you can assign workflows to groups. So if you 
create a group for each background process, then those processes can 
continually check to see if there are any workflows waiting on them. Once 
they finish their job, they call:

workflow_engine.step_complete(workflow_id, as_group=50) # payment processor 
is finished

Because it's designed to be as general as possible, you can have the best 
of both worlds which this example demonstrates: mixing both human and 
automated workflows into a single workflow.

Does this address you question?

Thanks

[web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

Just wanted to give an update on this. I'm working on the workflow engine 
nearly every day. Workflow engines are complicated things with lots of 
pitfalls that need to be taken into account. Having said that, I am getting 
closer to completing it.

What works:

   - Templates
   - Creating workflows on-the-fly
   - Serial and parallel steps (wait on more than one user/group at a time)
   - Due date calculation taking schedules, holidays, etc into account
   - Simple flow of a workflow (i.e. every step is completed successfully, 
   without issue)
   - Full delta auditing with support for writing audit logs to another DAL 
   instance
   - Triggers (when someone does something to a workflow, some code can be 
   executed)

What needs to be finished:

   - Step rejections (someone noticed a problem created in a previous step 
   and wants to send it back to be corrected)
   - Workflow monitors (users that aren't currently involved with the 
   workflow, but want to monitor the workflow's progress)
   - Workflow comments (have rejections and other comments on the workflow 
   been addressed)
   - Workflow check-list (high-level workflow objectives, created and 
   managed by users, basically just meta-data)
   - Documentation (API and User Guide)

Most of the functionality in the "needs to be finished" list has been 
written, but not tested. The engine is about 1500 lines of code right now, 
which includes about 350 lines of tests. One of my main goals has been to 
write a test suite that tests as many scenarios as possible. Writing a user 
guide with example usage will come after the engine has been accepted in 
web2py. I want to make sure the API doesn't need changing before I start 
writing actual documentation.

I started writing this engine because I needed to replace a proprietary 
engine I created in C# several years ago that has seen years of production 
use. Unfortunately, the old engine lacks a lot of features (parallel steps, 
monitors, comments, check-lists, auditing, etc) and is tied to a single 
specific object. I wanted a workflow engine that could attach a workflow to 
just about anything. I also wanted it to have better features, cleaner 
code, and be easier to use.

Hopefully, I will continue to make excellent progress on it and have 
something to submit for approval soon. I'll keep you posted!

Re: [web2py] Re: Unable to connect to mssql database

[Ross Peoples]

I have been using MSSQL from Linux for quite some time. I use FreeTDS for 
my driver. This is my connection string:

db = DAL('mssql://user:password@server/db_name?DRIVER={FreeTDS}')

If you need help setting up FreeTDS / unixODBC, let me know. I have some 
notes on how to set this up using a Ubuntu machine.

Re: [web2py] Re: What to expect in web2py 2.0

[Ross Peoples]

Richard,

Thanks for the offer. I *think* I'm nearly done. I'm writing some tests for 
it now and so far so good. If I run into any problems, I'll let you know.

Ross

On Tuesday, May 22, 2012 11:09:31 AM UTC-4, Richard wrote:
>
> Ross,
>
> If you need help just ask!
>
> Richard
>
>

[web2py] Re: What to expect in web2py 2.0

[Ross Peoples]

There may even be an experimental workflow engine in there if I can finish 
it in time.

[web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

Yes, thanks for the links. I was able to add parallel steps that work as 
you describe. I have almost finished my workflow engine. I only have a 
couple more methods to write before I begin writing unit tests.

Thanks for all the input. I am still taking comments and suggestions, so if 
anyone has ideas to make it even better (or thoughts on how to handle 
security), your thoughts are welcome.

[web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

On Thursday, May 17, 2012 3:54:44 PM UTC-4, Cliff wrote:
>
> Let's inject manufacturing into the order processing scenario you were 
> using before.  It's a three step process.  Step one fabricates widgets. 
>  Step 2 attaches the widgets to widget bicarackets, purchased from another 
> vendor.  Step 3 packages the assemblies for shipment.
>
> If the last batch of widgets from the fabricator won't fit the brackets 
> because they are oversize, obviously the guy in assembly wants to send them 
> back for rework.  How would the system handle this?
>

In this case, assembly would reject their step, and enter a short message 
to explain why they rejected it. Assembly would also select a previous step 
in the workflow to send it back to, in this case the fabricator (step 1). 
The message from assembly goes into the workflow_comments table as needing 
to be resolved. Once the fabricator fixes the problem, they mark the 
problem resolved and send the workflow on to the next step (which in this 
case would be to step 2).
 

>
> And how would it avoid forcing the guy in charge of assembly to take 
> action on every batch of good widgets, assuming the system's intent is to 
> manage by exception?
>

Let's say the guy in assembly has 5 work orders to assemble for the day and 
each work order makes 20 assemblies (for a total of 100 things to make for 
the day). He would come in with the parts waiting for him for all 5 work 
orders, and he would have 5 workflows waiting on him. When he completes one 
work order (20 assemblies), he marks that workflow as completed and the 
workflow moves on to the shipping department. If we wanted, we could even 
add a "progress" field that allows him to enter a percentage of how close 
he is finishing the 20 assemblies for the one work order.

The great thing about doing it this way is allows management to track where 
everything thing is. My implementation puts an emphasis on accountability 
and making sure nothing gets lost or forgotten. Most manufacturing places 
have computers on the floor, so asking the assembly guy to click a button 
isn't a stretch for most places. In fact, by making this whole 
implementation generic like this, you could even put a simple keypad out 
there in assembly, shipping, and other places that don't have full 
computers to manage workflows.

Nothing about the implementation forces you to use a GUI. You could program 
a keypad to accept a work order number that marks the associated workflow 
as complete via a REST call or something. And worst case, you could always 
have the assembly supervisor handle the workflows for the guys in assembly. 
That's what supervisors are for, right? :)

[web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

On Thursday, May 17, 2012 9:37:46 AM UTC-4, Cliff wrote:
>
> Ross,
>
> I understand your reasons for attaching approvals to the workflow.  I do 
> it the other way because if a deliverable needs approval by six parties, 
> putting six additional steps in the flow makes things a little cluttered. 
>  If you add six people as performers of an approval step, does that mean 
> that any one of the six can approve or must all of them?  
>

If you were to put all of them in a group and add the group as part of the 
workflow, any one of them can approve. For cases when you need all 6 to 
approve, then you would add all 6 of them as steps in the workflow. I don't 
know of a good, clean, and understandable way to do that otherwise.
 

>
> I very much like your workflow object table.
>
> I'm still debating over whether to use auth to handle the roles.  I think 
> auth needs a table called auth_role, actually a group of groups, to collect 
> permissions.  It would be nice to have a 'has_role' decorator for it.
>
>
Authorization is the only thing in this I'm having trouble wrapping my head 
around. Previously, I had the objects the workflows were attached to handle 
their own security with minimal restrictions on the workflow side. It 
worked out great because the documents were tightly integrated with the 
workflow system. The major problem being that only documents could 
participate in the workflows. So I'm still having trouble figuring out how 
to do this in a more general way without making it so complicated that no 
one would use it. I am definitely open to suggestions on this one.

[web2py] Re: web2py back reference and compute fields problem

[Ross Peoples]

I don't know about the "order.order_item.select()" part. Have you tried 
debugging the "calc" method? Insert this line right above "result = 0":
import pdb; pdb.set_trace()

Run the app from the console (i.e: python web2py.py -a password ...) and 
when the method gets called, the application should freeze. If you look at 
the console, you will have a (Pdb) prompt. Continue to enter "n" to move to 
the next line and type "print result" to show the value of result at any 
time. You should be able to figure out the problem this way.

If the application continues to work without freezing, then the "calc" 
method is not even being called, so you'd need to figure out why. 

[web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

The data model I already have does things a bit differently, but I think it 
accomplishes the same thing. I am in the process of writing all of the 
support methods.

This is my current data model:

# workflow table
db.define_table('workflow',
Field('name', length=50),
Field('is_template', 'boolean', default=False),
Field('created_by', db.auth_user, default=current_user),
Field('created_on', 'datetime', default=self.request.now),
Field('table_name', length=128),
Field('row_id', 'integer'),
Field('order_id', 'integer', default=self.ORDER_ID_STOP, 
comment='Current position of the workflow'),
Field('priority', 'integer', requires=IS_INT_IN_RANGE(1, 9), 
default=self.DEFAULT_PRIORITY)
)

# allow users / groups to "monitor" select workflows without needing to be 
a part of the workflow
# if workflow is template, this list is copied to the new workflow
db.define_table('workflow_monitor',
Field('workflow_id', db.workflow),
Field('user_id', db.auth_user),
Field('group_id', db.auth_group),
Field('viewed', 'boolean', default=False) # once the monitor looks at 
it, mark as viewed until another change happens
)

# comments can be attached to workflows so that users can voice questions 
and concerns
db.define_table('workflow_comment',
Field('workflow_id', db.workflow),
Field('user_id', db.auth_user, default=current_user),
Field('event_date', 'datetime', default=self.request.now),
Field('is_active', 'boolean', default=True, comment='Is the comment 
waiting to be addressed'),
Field('comment', length=512, comment='The question, comment, or 
concern'),
Field('response', length=512, comment='Response to the question, 
comment, or concern')
)

# high-level list of goals for workflows. Users mark items as completed as 
they complete the goals
# if workflow is template, the checklist is copied to new workflow
db.define_table('workflow_checklist',
Field('workflow_id', db.workflow),
Field('name', length=50),
Field('order_id', 'integer', comment='Ordering position of the item'),
Field('is_active', 'boolean', default=True, comment='Is the item 
waiting to be addressed'),
Field('completed_by', db.auth_user),
Field('completed_at', 'datetime'),
Field('completed_note', length=512)
)

# workflow step table
db.define_table('workflow_step',
Field('workflow_id', db.workflow),
Field('order_id', 'integer', comment='Ordering position of the step'),
Field('name', length=50),
Field('user_id', db.auth_user),
Field('group_id', db.auth_group),
Field('hours', 'decimal(10,2)', requires=IS_DECIMAL_IN_RANGE(0), 
default=0, comment='Optional time limit in hours'),
Field('due_date', 'datetime', comment='Optional due date'),
Field('note', length=512, comment='Note on current state of the step') 
# represents a stateful note (i.e. kickback reason). Can change at any time.
)

# audit tables
db.define_table('workflow_event',
Field('workflow_id', 'integer'),
Field('table_name', length=128), # i.e. document, folder, etc
Field('row_id', 'integer'), # i.e. the ID of the document
Field('user_id', db.auth_user, default=current_user),
Field('event_date', 'datetime', default=self.request.now),
Field('action', length=10, default='update'), # could be: create, 
update, delete
Field('field_name', length=128), # i.e order_id, name, hours, etc
Field('value_before', length=128), # None if create or delete
Field('value_after', length=128) # None if delete
)

db.define_table('workflow_step_event',
Field('workflow_id', 'integer'),
Field('step_id', 'integer'), # the ID of the workflow_step modified 
(used to match up fields in batch changes)
Field('user_id', db.auth_user, default=current_user),
Field('event_date', 'datetime', default=self.request.now),
Field('action', length=10, default='update'), # could be: create, 
update, delete
Field('field_name', length=128), # i.e order_id, name, hours, etc
Field('value_before', length=128), # None if create or delete
Field('value_after', length=128) # None if delete
)


As you can see, this model has quite a few of the features we discussed: 
workflows, templates, monitors, comments, checklists, and auditing. It also 
has triggers that applications can register (i.e. before_step_complete, 
after_step_complete, etc) so applications can interact with the workflow 
engine when an event is triggered.

Workflows are copied from templates so that changes can be made to 
templates and workflows without one affecting the other. This has worked 
very well in my previous implementation. Tying monitors to templates 
wouldn't work well like this. Attaching monitors to workflows themselves 
also offers the advantage of giving the monitors a read/unread indicator 
and when a workflow has been changed (i.e step completed), the indicator 
turns back to "unread" so that way they keep up with workflows and not have 
to remember if a workflow has changed

[web2py] Re: SNI or dedicated IP.

[Ross Peoples]

I wouldn't know the first thing about setting up SNI, so I would tell 
WebFaction that if they will do it for me and it works, then sure, I'll use 
SNI, otherwise stick with what you know.

On Wednesday, May 16, 2012 12:14:42 PM UTC-4, Anthony wrote:
>
> I'd never heard of SNI before, but looking at the Wikipedia page for it: 
>> http://en.wikipedia.org/wiki/Server_Name_Indication
>>
>> This will NOT work. The main reason being that Python 2 (which web2py 
>> runs on) does not support SNI. Python 3.2 does, however. So until there is 
>> a web3py, SNI will not be an option unless you use it through Apache (and 
>> figure out how to enable it).
>>
>
> Wouldn't that only be a limitation with the Rocket server? But on 
> WebFaction, the app would probably be served with 
> Apacheor 
> Nginx,
>  
> so should work, no?
>
> Anthony 
>

[web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

Cliff,

Thanks for putting this into a spec. My current code follows most of what 
is in there. I still haven't finished writing (or testing it), but I do 
have some thoughts on implementation for one or two of the items listed at 
the end:

I imagine a "workflow_monitor" table that has the fields: user_id, 
workflow_id. By default, when a workflow is started, the user that started 
gets added to this table along with the workflow they started. Using this, 
an application can design an interface that shows two lists: one listing 
workflows waiting on the user, and another listing workflows being 
monitored by the user. Templates should have to ability to set a default 
list of users or groups that automatically get added as monitoring the 
workflow. This would be great for managers to see a birds-eye view of 
everything going on for every person they manage.

Approvers should be part of the workflow, since it should not continue 
unless approved.

I'm not sure how to handle reviewers, since they don't really have any 
control over the workflow. Maybe reviewers can be marked as "monitoring", 
and when they need to comment on something, they just leave a note. For 
this, I see a "workflow_comment" table that has these fields: workflow_id, 
user_id, event_date, is_active, comment. The is_active field will be set to 
True by default. Once the comment / issue has been addressed, is_active 
gets set to False.

I could also imagine a high-level list of goals that need to be 
accomplished with the workflow, sort of like a checklist, that users mark 
as completed as the workflow moves through the system. This would probably 
satisfy a use case that exists in my current company. They currently use a 
page within a document as a "sign off" page to make sure everyone inspected 
their section of the document. Having this workflow checklist would remove 
the need for this extra page within the document.

Thoughts?

Thanks, Ross

[web2py] Re: https for login

[Ross Peoples]

I only know about it because I submitted it :) We really need to open the 
book up for editing so that we can add these kinds of things.

On Wednesday, May 16, 2012 9:22:28 AM UTC-4, Anthony wrote:
>
>request.requires_https()
>>
>
> Yet another secret feature. :-) 
>

[web2py] Re: https for login

[Ross Peoples]

By the way, in case you are wondering, I stopped using Apache once I tried 
Nginx. Configuration is much easier (in my humble opinion) and it's way 
faster. So I've been using Nginx for all of my stuff for over 6 months now 
and haven't looked back. However, I still try to write my applications to 
be server-agnostic when I can.

[web2py] Re: https for login

[Ross Peoples]

You could force a redirect within web2py. I do this with most of my 
applications:

In a model:

if request.controller == 'default' and request.function == 'user':
   request.requires_https()

This will force all user operations (login, profile, reset_password, etc) 
to use HTTPS. The advantage to doing it through web2py rather than through 
the web server is that if you get sick of Apache, you don't have to worry 
about this again with your new web server.

[web2py] Re: SNI or dedicated IP.

[Ross Peoples]

I'd never heard of SNI before, but looking at the Wikipedia page for 
it: http://en.wikipedia.org/wiki/Server_Name_Indication

This will NOT work. The main reason being that Python 2 (which web2py runs 
on) does not support SNI. Python 3.2 does, however. So until there is a 
web3py, SNI will not be an option unless you use it through Apache (and 
figure out how to enable it). It's still pretty new; they just added 
support for it to Java 7 and Qt 4.8 (which were both just released a few 
months ago). Just because the OS supports it doesn't mean that the 
applications or the frameworks for those applications supports it.

This quote seems to sum up SNI: "As of 2012 there are still many users of 
browsers that do not support SNI"

On Wednesday, May 16, 2012 1:33:48 AM UTC-4, Annet wrote:
>
> To be able to access web2py's admin online I had a dedicated IP address at 
> WebFaction, and referenced admin 
> with a subdomain: admin.mydomain.com. A few weeks ago I changed accounts and 
> when I asked for a dedicated IP 
> address they answered:
>
> The connection using SNI is just as strong as a normal SSL connection
> except that the domain is sent in cleartext.
>
> If you are not using a really old browser/OS combination
> you can just use SNI instead of getting a dedicated IP.
>
>
> Is anybody using SNI to access web2py's admin, how does it work, and how do I 
> set it up?
>
>
> Kind regards,
>
> Annet
>
>

[web2py] Re: Is there anyone who running web2py application on Redhat openshift?

[Ross Peoples]

I don't know anything about openshift, but I would think the normal MySQL 
troubleshooting steps apply:


   1. Make sure MySQL is listening on all interfaces
   2. Make sure your MySQL account accepts connections from the '%' host.


On Tuesday, May 15, 2012 3:55:07 AM UTC-4, JungHyun Kim wrote:
>
> Hello. I'm now trying to use redhat openshift.
>
> I am struggling access database - mysql-5.1 - which is set via openshift 
> cartridge.
> I got admin user(admin), password and database name(we2py).
> So I modified database setting in models/db.py 
>
> as
>
> db = DAL('mysql://admin:passw...@web2py-codingday.rhcloud.com/web2py')
>
> But web2py application can't connect to mysql.
>
> OperationalError: (2003, "Can't connect to MySQL server on '
> web2py-codingday.rhcloud.com' (111)")
>
> I think I couldn't understand enough. I am a noob for both web2py and 
> openshift.
>
> How can I make web2py connect to database right?
>
> Thank you.
>
>
>

[web2py] Re: Can/should {{super}} works even without a parent block?

[Ross Peoples]

Sorry, been switch between languages a lot this morning. If you are still 
getting the error, then I'm not sure. Maybe someone else has a thought.

On Monday, May 14, 2012 11:56:45 AM UTC-4, Ray (a.k.a. Iceberg) wrote:
>
> Guess you mean {{try:}}{{super}}{{except: pass}}
>
> Unfortunately it fails too.
>
> So a fix is a must?
>
> Regards,
> Ray
>
> On Monday, May 14, 2012 11:50:45 PM UTC+8, Ross Peoples wrote:
>>
>> Correction, that should be:
>> {{try:}}{{super}}{{catch:pass}}
>>
>>
>>
>> On Monday, May 14, 2012 11:45:37 AM UTC-4, Ross Peoples wrote:
>>>
>>> While not the prettiest solution, you can always use 
>>> {{try:}}{{super}}{{catch}} in views. I know that sounds like a bandaid, but 
>>> I think an exception is the desired behavior here. Otherwise, returning an 
>>> empty string would make troubleshooting difficult if you accidentally 
>>> mistyped the name of the block and the block keeps showing up as empty.
>>
>>

Re: [web2py] Re: Nginx-uwsgi problem.

[Ross Peoples]

The problem with doing it as a C module is that it would have to be 
compiled. I know that this is something that has been mentioned before but 
was shot down because web2py wouldn't be easily accessible, modified, etc, 
which is the goal of web2py. Alternatively, maybe running web2oy in PyPy 
environment could improve the performance without taking away from the 
flexibility of web2py.

On Monday, May 14, 2012 11:57:49 AM UTC-4, Richard wrote:
>
> Hello,
>
> I wonder if some of the speed problem that web2py has could be address by 
> translate some of the web2py module into C??
>
> There is already many walk around for speed problem, but could there is 
> some major speed improvement by rewrite some of web2py fondation into C?
>
> I am just curious about that possibility since I didn't see popup as 
> possible option to improve speed of app in major scalling process.
>
> Thanks
>
> Richard
>
> On Mon, May 14, 2012 at 11:50 AM, Anthony  wrote:
>
>>
>>>1. Your "adviewer/viewads" makes 10 calls to the database. Try to 
>>>optimize this either by writing fewer queries, creating a view, and/or 
>>> only 
>>>selecting fields that you need. Also make sure you get the criteria 
>>> right 
>>>so that you (ideally) don't have any extra, unneeded rows. 
>>>
>>> If it's feasible, also consider caching the queries for some amount of 
>> time (assuming the results don't change too frequently). 
>>
>>>
>>>1. When you absolutely have to load a few thousand rows (or more) in 
>>>a query (you should avoid this whenever possible), then try using 
>>>"db.executesql(query)" to manually execute a hand-crafted SQL query. 
>>> This 
>>>will always be faster than using the DAL directly. 
>>>
>>> Note, the difference in speed is due to the fact that the DAL won't be 
>> converting the results set to a Rows object -- so you won't have the 
>> convenience of dealing with DAL Rows and Row objects. If you do 
>> db.executesql(query, 
>> as_dict=True), it will convert to a list of dictionaries (which is still 
>> faster than converting to a Rows object). 
>>
>>>
>>>1. Another point about executesql: The obvious issue is reduced 
>>>portability, but if you are only planning on using PostgreSQL, then you 
>>> can 
>>> hand-craft a SQL query and profile it against PostgreSQL for maximum 
>>>performance. Once you've got it giving only the data you want, then you 
>>> can 
>>>copy and paste that query into executesql. 
>>>
>>> If you want to use db.executesql() but remain portable, you can still 
>> have the DAL generate the SQL for you by using the ._select() method:
>>
>> db.executesql(db(query)._select(...))
>>
>> Obviously in that case you don't get to hand optimize the SQL, but you 
>> still get the speed advantage of not converting the results to a Rows 
>> object (which is only significant for large results sets).
>>
>> Anthony
>>
>>
>

Re: [web2py] Re: Nginx-uwsgi problem.

[Ross Peoples]

On Monday, May 14, 2012 11:50:06 AM UTC-4, Anthony wrote:
>
>
>>1. Your "adviewer/viewads" makes 10 calls to the database. Try to 
>>optimize this either by writing fewer queries, creating a view, and/or 
>> only 
>>selecting fields that you need. Also make sure you get the criteria right 
>>so that you (ideally) don't have any extra, unneeded rows.
>>
>> If it's feasible, also consider caching the queries for some amount of 
> time (assuming the results don't change too frequently). 
>

This is a good point. For things that may not change a whole lot, I usually 
set the time_expires=3600 or even None. Give the key a well-defined name, 
and then when something changes, run cache.ram.clear(key) so that the next 
query will get the changes. 

>
>>1. When you absolutely have to load a few thousand rows (or more) in 
>>a query (you should avoid this whenever possible), then try using 
>>"db.executesql(query)" to manually execute a hand-crafted SQL query. This 
>>will always be faster than using the DAL directly.
>>
>> Note, the difference in speed is due to the fact that the DAL won't be 
> converting the results set to a Rows object -- so you won't have the 
> convenience of dealing with DAL Rows and Row objects. If you do 
> db.executesql(query, 
> as_dict=True), it will convert to a list of dictionaries (which is still 
> faster than converting to a Rows object). 
>
>>
>>1. Another point about executesql: The obvious issue is reduced 
>>portability, but if you are only planning on using PostgreSQL, then you 
>> can 
>> hand-craft a SQL query and profile it against PostgreSQL for maximum 
>>performance. Once you've got it giving only the data you want, then you 
>> can 
>>copy and paste that query into executesql.
>>
>> If you want to use db.executesql() but remain portable, you can still 
> have the DAL generate the SQL for you by using the ._select() method:
>
> db.executesql(db(query)._select(...))
>
> Obviously in that case you don't get to hand optimize the SQL, but you 
> still get the speed advantage of not converting the results to a Rows 
> object (which is only significant for large results sets).
>

While true, since he is going for performance in a high-traffic environment 
that requires low-latency, such as a site that serves ads, he 
would definitely want to hand-craft the SQL for complex and large queries 
that slow things down. I wouldn't recommend doing it for every query, just 
the slow ones.
 

>
> Anthony
>
>

[web2py] Re: Can/should {{super}} works even without a parent block?

[Ross Peoples]

Correction, that should be:
{{try:}}{{super}}{{catch:pass}}



On Monday, May 14, 2012 11:45:37 AM UTC-4, Ross Peoples wrote:
>
> While not the prettiest solution, you can always use 
> {{try:}}{{super}}{{catch}} in views. I know that sounds like a bandaid, but 
> I think an exception is the desired behavior here. Otherwise, returning an 
> empty string would make troubleshooting difficult if you accidentally 
> mistyped the name of the block and the block keeps showing up as empty.

[web2py] Re: Can/should {{super}} works even without a parent block?

[Ross Peoples]

While not the prettiest solution, you can always use 
{{try:}}{{super}}{{catch}} in views. I know that sounds like a bandaid, but 
I think an exception is the desired behavior here. Otherwise, returning an 
empty string would make troubleshooting difficult if you accidentally 
mistyped the name of the block and the block keeps showing up as empty.

Re: [web2py] Re: Nginx-uwsgi problem.

[Ross Peoples]

I'm reading through the profile log, not that I'm an expert on profiling or 
anything, but I do have a few possible tips:

   1. Most of your calls are done in ~86 ms. That's pretty respectable. 
   2. Most of your time is spent on database calls, with compiling being 
   the second biggest time consumer. So compiling your code should provide you 
   with a quick speed boost without having to change any code.
   3. The biggest bottleneck is almost always the queries to the database, 
   regardless of using the DAL or not. So spend more time optimizing your 
   queries before optimizing your code.
   4. Your "adviewer/viewads" makes 10 calls to the database. Try to 
   optimize this either by writing fewer queries, creating a view, and/or only 
   selecting fields that you need. Also make sure you get the criteria right 
   so that you (ideally) don't have any extra, unneeded rows.
   5. The "trees/index", "trees/get_binary_tree.json", 
   "trees/team_members", etc. I assume are supposed to be showing a 
   parent-child type of relationship. I've done this with folders in a 
   document management system before and trees are tough. The way I did it was 
   to load only the top-level and one level below in a single database query 
   (so you would know if the folder had subfolders), and then loaded the rest 
   whenever the user "opened" or "expanded" a folder. If your application 
   resembles this type of tree, then maybe try that. Otherwise, try to load 
   two or three levels per query. Even using a sub-select will be faster than 
   repeated queries.
   6. Moving models to modules and only loading the tables you need has 
   already been mentioned, but it's worth adding to the list of tips.
   7. When you absolutely have to load a few thousand rows (or more) in a 
   query (you should avoid this whenever possible), then try using 
   "db.executesql(query)" to manually execute a hand-crafted SQL query. This 
   will always be faster than using the DAL directly.
   8. Another point about executesql: The obvious issue is reduced 
   portability, but if you are only planning on using PostgreSQL, then you can 
hand-craft a SQL query and profile it against PostgreSQL for maximum 
   performance. Once you've got it giving only the data you want, then you can 
   copy and paste that query into executesql.

I don't know how much, if any, of this applies to your situation, but I'd 
like to think it's good advice in general.

[web2py] Re: when I open a second DAL connection to the same sqlite database it does not detect a new table

[Ross Peoples]

In my experience, "can" doesn't always mean "should". There may be an issue 
with db2 not seeing the table definitions from db1. For testing, do 
something like this to explicitly share the "test" table definition:

def define_tables(db, migrate=False):
   db.define_table('test', Field('testfield'), migrate=migrate)

define_tables(db1)
define_tables(db2)

Re: [web2py] To uuid or not?

[Ross Peoples]

I have created issue 
796: http://code.google.com/p/web2py/issues/detail?id=796

While creating the issue, I noticed that I had made an error with the MSSQL 
part. This was corrected in the issue.

Re: [web2py] To uuid or not?

[Ross Peoples]

Some databases allow you to temporarily turn off auto-generating IDs so 
that you can import a table keeping its original IDs intact. I wrote a 
database abstraction layer a few years back for another project that 
specifically allowed you to make an identical copy of a database from a 
MySQL database to a PostgreSQL database on a different machine and 
preserving the IDs was something I needed to figure out. Here is how you do 
it with MSSQL, MySQL, and PostgreSQL. Maybe Massimo can add this as an 
option to the DAL's import method:

First, import your records, then run these on your database engine:

MSSQL:
db.executesql('SET IDENTITY_INSERT dbo.mytable OFF');
db.executesql('DBCC CHECKIDENT (mytable)');

MySQL:
count = db.executesql('SELECT MAX(id) FROM mytable;')
count += 1
db.executesql('ALTER TABLE mytable AUTO_INCREMENT=%s;' % count)

PostgreSQL:
count = db.executesql('SELECT MAX(id) FROM mytable;')
count += 1
db.executesql('ALTER SEQUENCE mytable_sequence RESTART WITH %s;' % count)

For all 3 database engines, this sets the auto-increment counter to the max 
ID it finds in the table (AFTER the import) +1 so that new rows will have 
the proper IDs.

[web2py] Re: when I open a second DAL connection to the same sqlite database it does not detect a new table

[Ross Peoples]

I don't think you are supposed to open more than one connection to a SQLite 
database at a time. It is file-based and doesn't have multi-connection 
abilities like other databases like MySQL and PostgreSQL.

On Monday, May 14, 2012 7:29:44 AM UTC-4, simon wrote:
>
> *In the following code I create a new table in db1. However when I open a 
> second DAL connection to the same database the new table is not shown. Why 
> is that?*
> *
> *
> *db1 = DAL('sqlite://storage.sqlite', folder=dbfolder, auto_import=True)*
> *print(db1.tables)*
> *
> *
> *db1.executesql("DROP TABLE IF EXISTS test;")*
> *print(db1._timings)*
> *try:*
> *os.remove(dbfolder+"/test.table")*
> *except:*
> *pass*
> **
> *db1.define_table('test', Field('testfield'), migrate='test.table')*
> *db1.commit()*
> *print(db1.tables)*
> *
> *
> *db2 = DAL('sqlite://storage.sqlite', folder=dbfolder, auto_import=True)*
> *print(db2.tables)*
>

[web2py] Re: Nginx-uwsgi problem.

[Ross Peoples]

The DB layer is usually the bottleneck. However, moving from models to 
modules should reduce any bottleneck caused by the web server.

[web2py] Re: CMS question

[Ross Peoples]

Looking at the "better" version of that you just added to trunk, this is 
pretty cool!

I also looked at bluePen editor. That is very cool and looks like something 
I did once using jQuery UI components. I can't seem to find the code I 
wrote for it. It was a proof-of-concept that didn't really go anywhere. But 
the way it worked was you would mark DIV tags with a 'editable' class (or 
whatever you want to call it) and when the DIV was double-clicked, it 
opened a CSS editing dialog that gave options to change colors, borders, 
and rounding.

When saved, it would send an AJAX request with the CSS that changed and the 
next time the DIV was rendered, the CSS was injected. This way you could 
still use a style.css to style the whole site, but use the CSS editor to 
override some of the attributes for the one DIV. I don't know if this is 
the behavior you are looking for, but it's totally possible.

On Thursday, May 3, 2012 12:16:57 PM UTC-4, Massimo Di Pierro wrote:
>
> Another piece of the puzzle is in trunk
>
> gluon/contrib/autolinks.py
>

[web2py] Re: RFC: web2py-based workflow engine

[Ross Peoples]

Cliff,

Thanks for the feedback. I added the ability to name / title steps. I also 
added the ability to set priorities for workflows.

It is assumed that when a step is complete, that step is already "done" and 
the next step in workflow is "new". Each step has an order_id to show which 
order the steps are in. The workflow table stores the "order_id" of the 
active workflow step. So in this case if the workflow is on order_id 3, 
then anything below 3 is considered done, while everything else is new. Is 
this what you mean by states to account for?

Active workflows can be modified on the fly as well. This has a limitation 
that any step that has been completed cannot be modified. All future steps 
can be modified. So if the workflow needs to have another step added to it 
or if it has been determined that the next destination user_id / group_id 
needs to be changed, that can be done (i.e. someone accidentally added the 
Engineering group, but it should have been Quality instead).

I will add another method to WorkflowEngine (I was going to do this 
anyways, but forgot to include it in my original post:
workflow = workflow_engine.get_workflow(workflow_id)

# The contents of the workflow object would contain:
#name: the name of the workflow
#is_template: is this workflow a template
#item_type, item_id: what object is the workflow attached to
#order_id: the current position of the workflow
#priority: the priority of the workflow
#steps: list of step as Row objects
#
# There would be a few helper methods as well:
#current_step(): returns the Row object of the current step or None
#due_date(): returns the current step's due date or None
#...

I have not built in any security, as I figured that would be something more 
application-specific. For example, a document management system might want 
security on folders, just like a regular file system, and 
not necessarily on a workflow-by-workflow basis.

However, I am planning on adding auditing support. I almost used the new 
record versioning feature, but decided I wanted to go with delta-based 
changes instead of full copies of everything for better timeline support.

Additional questions, comments, and improvements are welcome!

On Tuesday, May 1, 2012 4:07:05 PM UTC-4, Cliff wrote:
>
> Ross,
>
> I like the on-the-fly ability.
>
> Each step in your template changes the state of the item.  Each one of 
> those states should have a title.
>
> There are two other states to account for, new and done.
>
> Workflows sometimes have more than one path to completion.  
>
> I see an entity called a workflow_item that has the ability to determine 
> its current state and all possible states to which it could transition.  It 
> also knows who can cause it to transition to one of those states as well as 
> which of its attributes should be hidden, visible or editable in any given 
> state, and who should be able to see or edit those attributes while in that 
> state.
>
>
> On Tuesday, May 1, 2012 12:00:42 PM UTC-4, Ross Peoples wrote:
>>
>> In reference to: 
>> https://groups.google.com/forum/#!searchin/web2py/workflow/web2py/osEmmtu9hlg/2MHi_ZCeMBMJ
>>
>> Has anyone done any work on this yet? I was thinking about making a 
>> web2py-based workflow engine.
>>
>> I mentioned previously that I built one of these for an application I 
>> wrote several years ago, but it was built specifically for that app. This 
>> will be my first attempt at making a general-use workflow engine, so let me 
>> know if you find any problems with my design:
>>
>> Make a contrib module that is initialized like the Auth module:
>> from gluon.contrib.workflow_engine import WorkflowEngine, Step
>> workflow_engine = WorkflowEngine()
>> workflow_engine.define_tables()
>>
>>
>> I will use the example that I know best, which is passing around a sales 
>> order throughout a company's departments. This first example would define a 
>> workflow template because every sales order will have the same workflow:
>> workflow_engine.add_template('Sales Order',
>> Step(group_id=2, hours=4), # Engineering gets 4 hours to complete 
>> their step
>> Step(user_id=7, hours=0), # The engineering manager (user) has no 
>> time limit
>> Step(group_id=3, hours=2), # Quality department gets 2 hours to 
>> inspect the order
>> Step(group_id=8, hours=6.5), # Shipping department gets 6.5 hours to 
>> ship order
>> )
>>
>>
>> You would start this workflow like this:
>> workflow_id = workflow_engine.load_template('Sales Order', item_type=
>> 'document', item_id=1)
>> workflow_id = workflow_engine.
>> workflow_engine.star

[web2py] Re: CMS question

[Ross Peoples]

Looks pretty nice, Massimo. I look forward to trying it out and 
contributing to it.

[web2py] RFC: web2py-based workflow engine

[Ross Peoples]

In reference 
to: 
https://groups.google.com/forum/#!searchin/web2py/workflow/web2py/osEmmtu9hlg/2MHi_ZCeMBMJ

Has anyone done any work on this yet? I was thinking about making a 
web2py-based workflow engine.

I mentioned previously that I built one of these for an application I wrote 
several years ago, but it was built specifically for that app. This will be 
my first attempt at making a general-use workflow engine, so let me know if 
you find any problems with my design:

Make a contrib module that is initialized like the Auth module:
from gluon.contrib.workflow_engine import WorkflowEngine, Step
workflow_engine = WorkflowEngine()
workflow_engine.define_tables()


I will use the example that I know best, which is passing around a sales 
order throughout a company's departments. This first example would define a 
workflow template because every sales order will have the same workflow:
workflow_engine.add_template('Sales Order',
Step(group_id=2, hours=4), # Engineering gets 4 hours to complete their 
step
Step(user_id=7, hours=0), # The engineering manager (user) has no time 
limit
Step(group_id=3, hours=2), # Quality department gets 2 hours to inspect 
the order
Step(group_id=8, hours=6.5), # Shipping department gets 6.5 hours to 
ship order
)


You would start this workflow like this:
workflow_id = workflow_engine.load_template('Sales Order', item_type=
'document', item_id=1)
workflow_id = workflow_engine.
workflow_engine.start(workflow_id)


Optionally, a workflow can be created on the fly (if the workflow will only 
be used once):
workflow_id = workflow_engine.create_workflow('One-time Workflow', 
'document', 1 # same as item_type and item_id used in load_template()
Step(group_id=2, hours=4)
Step(group_id=3, due_date=request.now + datetime.timedelta(days=1)) # 
set time limit to an exact datetime
)

workflow_engine.start(workflow_id) # start the workflow we just created


We assume that we are going to associate this workflow with another object. 
In this case, we will assume there is a table called "document" and that 
the document we want to pass around has an id of 1. The "item_type" 
argument allows you to pass around any type of database object. In this 
case, we will call it the same thing as our table: "document".

These are some common operations that could be done:
workflow_engine.active_workflows() # returns a list of all active workflows
workflow_engine.active_workflows(user_id=1) # all active workflows for the 
user_id
workflow_engine.active_workflows(user_id=1, include_groups=True) # same as 
above, but includes groups the user is a member of
workflow_engine.active_workflows(group_id=2) # all active workflows for the 
group_id
workflow_engine.late_workflows() # returns a list of all late/overdue 
workflows
workflow_engine.step_complete(workflow_id, notes='General info about 
completed task') # moves workflow to the next step
workflow_engine.step_reject(workflow_id, to_step=2, notes='Why step was 
rejected') # moves workflow back to step 2 incase there was a problem with 
one of the previously completed steps


Workflow triggers:
workflow_engine.before_start = function(workflow, step)
workflow_engine.after_start = function(workflow, step)
workflow_engine.before_step_complete = function(workflow, step)
workflow_engine.after_step_complete = function(workflow, step)
workflow_engine.before_step_reject = function(workflow, step)
workflow_engine.after_step_reject = function(workflow, step)
workflow_engine.before_finish = function(workflow, step)
workflow_engine.after_finish = function(workflow, step)


Finally, (and I MIGHT do this) since we are using time limits in hours, we 
should set some time ranges where users are available. For example, if the 
company is only open from 8 AM to 5 PM, you wouldn't want something to be 
late at 7 PM. You would want to roll over the extra 2 hours so that it 
becomes late at 10 AM the next business day. A list of time ranges would be 
created, and a user would be assigned to one of the time ranges. This would 
accommodate users in different time zones or with different "work" hours. 
Again, this last part I MIGHT do if I have enough time. I've done it 
before, but I'm sure you can imagine how complicated this part is.


So any questions, comments, improvements? Thanks!

Re: [web2py] Re: Ubuntu and Python3

[Ross Peoples]

I would imagine that you would still have the same problem unless shared 
hosts figure out a way to make uWSGI easier to use.

I have given up on shared hosting. The price of a low-end VPS is just as 
low (and sometimes lower) as most shared hosting plans with MUCH better 
performance. I have played with shared hosting from multiple providers, and 
success really varies. I found myself spending hours getting it all set up, 
when I can set up a VPS the right way in about 15 minutes.

You can find really good deals on VPS providers on webhostingtalk.com, but 
I keep up with deals posted to lowendbox.com because they make sure the 
providers are reputable before posting their promotions. I personally have 
VPS instances from both ChicagoVPS and RethinkVPS and have been pretty 
happy. Basically, you can get root access to a web server for $7/month or 
less. They seem to handle traffic pretty well too for the price, but if you 
have a high traffic site, you would want something more expensive (and you 
wouldn't be using shared hosting to begin with).

On Thursday, April 26, 2012 12:22:09 PM UTC-4, sebastian wrote:
>
> just wondering if having web3py in python 3 will give us some problem with 
> some share hosting
>
> On Thu, Apr 26, 2012 at 2:52 PM, Johann Spies wrote:
>
>> That article says that Python 2.7 will still be part of Ubuntu so there 
>> should not be a problem.
>>
>> Regards
>> Johann
>>
>>
>> -- 
>> Because experiencing your loyal love is better than life itself, 
>> my lips will praise you.  (Psalm 63:3)
>>
>>  
>
>
> -- 
> Sebastian E. Ovide
>
>
>
>  

[web2py] Re: crud.create causes an id problem

[Ross Peoples]

I have created Issue 
777: http://code.google.com/p/web2py/issues/detail?id=777

On Friday, April 27, 2012 9:15:07 AM UTC-4, Massimo Di Pierro wrote:
>
> Please open a ticket about this.
>
> On Thursday, 26 April 2012 13:21:34 UTC-5, villas wrote:
>>
>> Just a thought...  if you are trying to create a new table which has a 
>> 'bigint' id,  this may be incompatible if it references an existing table 
>> which has an 'integer' id. 
>>
>> Regards,
>> David
>>
>

[web2py] Add RESTful Client to Contrib

[Ross Peoples]

Should we add a RESTful client to web2py's contrib folder? We already have 
clients for XML-RPC and JSON-RPC, so maybe we should include one for REST 
that can handle both simple a more complex cases (like file uploads, 
unicode, etc). I found one that I was just about to add to a project and 
thought it would be good to include into web2py 
contrib: 
https://github.com/thraxil/restclient/blob/master/restclient/__init__.py

[web2py] Re: crud.create causes an id problem

[Ross Peoples]

I'm having a similar issue using SQLFORM. I am using a table that stores 
extra data for users, and I call it auth_user_extended:

db.define_table('auth_user_extended',
Field('auth_user', db.auth_user, readable=False, writable=False),
Field('supervisor', 'boolean', label='Is Supervisor?', default=False),
Field('status', 'integer', 
requires=IS_IN_SET(db.auth_user_extended_status_types), default=1, 
notnull=True),
Field('legal_name', length=255, notnull=True),
Field('secondary_email', length=255),
Field('primary_phone', length=20),
Field('secondary_phone', length=20),
Field('start_date', 'datetime'),
Field('end_date', 'datetime'),
Field('barcode_id', length=12, notnull=True, unique=True), 
#requires=IS_NOT_IN_DB(db, 'auth_user_extended.barcode_id')),
Field('alliance_id', length=10, default=None),  # this is the user's 
Alliance ID. If this is None, it defaults to auth_user.username.
Field('supervisor_auth_user', db.auth_user, label='User\'s Supervisor', 
ondelete='NO ACTION'),
Field('assistant_to_auth_user', db.auth_user, label='Assistant To 
User', ondelete='NO ACTION'),
Field('vacation_minutes', 'decimal(10,2)', default=0, 
requires=IS_NOT_EMPTY(), notnull=True),
Field('personal_minutes', 'decimal(10,2)', default=0, 
requires=IS_NOT_EMPTY(), notnull=True),
Field('sick_minutes', 'decimal(10,2)', default=0, 
requires=IS_NOT_EMPTY(), notnull=True),
Field('holiday_minutes', 'decimal(10,2)', default=None),   # setting 
this field to None makes it default to the holiday's holiday_minutes setting
Field('picture', 'upload', 
uploadfolder=os.path.join(request.folder,'static/images/user-pictures'))
)

I'm trying to use this code:

form = SQLFORM(db.auth_user, row.auth_user)
form.append(SQLFORM(db.auth_user_extended, row.auth_user_extended))

But I've also tried making a form for auth_user_extended by itself and I 
still get the same problem:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.

Traceback (most recent call last):
  File "/var/web2py/gluon/restricted.py", line 205, in restricted
exec ccode in environment
  File "/var/web2py/applications/bonfire/controllers/administration.py" 
, line 
292, in 
  File "/var/web2py/gluon/globals.py", line 175, in 
self._caller = lambda f: f()
  File "applications/bonfire/modules/core.py", line 473, in f
return action(*a, **b)
  File "/var/web2py/applications/bonfire/controllers/administration.py" 
, line 
286, in users
return update()
  File "/var/web2py/applications/bonfire/controllers/administration.py" 
, line 
232, in update
form.append(SQLFORM(db.auth_user_extended, row.auth_user_extended))
  File "/var/web2py/gluon/sqlhtml.py", line 888, in __init__
inp = self.widgets.options.widget(field, default)
  File "/var/web2py/gluon/sqlhtml.py", line 218, in widget
options = requires[0].options()
  File "/var/web2py/gluon/validators.py", line 471, in options
self.build_set()
  File "/var/web2py/gluon/validators.py", line 458, in build_set
records = self.dbset(table).select(*fields, **dd)
  File "/var/web2py/gluon/dal.py", line 8004, in __call__
query = query._id>0
  File "/var/web2py/gluon/dal.py", line 7165, in __getattr__
return self[key]
  File "/var/web2py/gluon/dal.py", line 7105, in __getitem__
return dict.__getitem__(self, str(key))
KeyError: '_id'


I just updated to the latest trunk and started having this issue.

[web2py] Re: Free e-book abhout Rest API

[Ross Peoples]

Thanks for the link! I've been looking for good API design resources.

[web2py] Re: Cannot start web2py - Runtime error: Unable to import driver

[Ross Peoples]

Can you paste the traceback of the ticket? 

On Monday, April 23, 2012 11:10:55 PM UTC-4, Goronmon wrote:
>
> I'm trying to use web2py on a Ubuntu install. I can start it up from the 
> console, but as soon as I navigate to it in the browser I the internal 
> error about not being able to import the driver.
>
> I haven't touched anything or done any type of special configuration. I've 
> gotten it to work in WIndows just fine, but for some reason I just can't 
> get it working on Ubuntu. I've spent all evening trying to get around the 
> issue but have had no luck so far.
>
> I would really appreciate any help anyone can give me in getting around 
> this error.
>

[web2py] Re: CMS question

[Ross Peoples]

I remember seeing this once before. This is great if you find a theme you 
like online and want to make a prototype of your site in that theme and 
this is a very cool tool in general. The question is, do we really want to 
go in this direction for a CMS? The content is basically imprisoned in 
hard-coded HTML, making theme changes difficult at best. I could see making 
static HTML pages for sites with this (i.e. like Dreamwaver), but for a 
CMS, I'm not sure.

On Monday, April 23, 2012 2:07:27 PM UTC-4, Massimo Di Pierro wrote:
>
> P.S. You have to register else there are copyright issues. This is a 
> complete rewrite of a previous experiment with the same name.
>
> On Monday, 23 April 2012 12:46:13 UTC-5, Massimo Di Pierro wrote:
>>
>> Here is a just a piece of the puzzle for you to try:
>>
>>http://tests.web2py.com/plasmid/
>>
>> It may be useful to build web2py layouts or CMS themes. Or it could be 
>> useful to edit the pages themselves.
>> Nothing prevents from inserting @{...} tags that embed web2py 
>> (plugin_wiki?) components.
>>
>> My server is slow so be patient and try not to kill it or I will have to 
>> take the app down.
>>
>> massimo
>>
>

[web2py] Re: CMS question

[Ross Peoples]

I guess it comes down to figuring out what kind of CMS we want to build. Do 
you want a CMS that is easy for web designers (like WordPress), or do you 
want a CMS that is easy for end users? My feeling is that there are already 
SO MANY WordPress clones out there that if we are going to build a 
"kickass" CMS, it should be easy for end users. I started to design a CMS 
for just this reason, but I haven't had a lot of time to mess with it 
recently.

My thought process when I started to design the CMS was to make it dead 
simple. If you know how to check your email and/or Facebook, then you can 
build a website. I had a working prototype of a CSS designer that was used 
to make themes. I had also come up with the idea for blocks (this was 
before I knew about Concrete5) where you select what type of content you 
want for the block (Markmin, HTML, RSS feed, tag cloud, nivo slider, other 
widget, etc). Then when in design mode, you could drag blocks around, 
resize them, etc.

The result of this design was a CMS that allowed users to create entire 
websites using pre-created components and widgets without knowing anything 
about HTML or CSS. Adding new widgets, plugins, and components would be 
simple for developers. And dropping down to regular HTML for a block was 
always an option incase something complicated needed to be done that the 
plugin system didn't cover.

This is the kind of CMS that I would love to see, and I know for a fact it 
is in high demand. But, it's difficult to develop which is why no one has 
really done it before (Concrete5 got close, but you still have to build 
themes/templates using HTML). So if we are going to build a CMS, we might 
as well make it "kickass".

[web2py] Re: CMS question

[Ross Peoples]

I don't know about that. I've started installing Concrete5 for people 
because it's so easy to use and they can create "blocks" of content that 
they can move around. I think a good CMS should be as dynamic as possible, 
without being overly complicated.

On Monday, April 23, 2012 1:30:12 AM UTC-4, Ramkrishan Bhatt wrote:
>
> 3) using fully editable html with no limitation on themes (any existing 
> page would be a theme without need for tweaking) yet one would not be able 
> to swap a theme on a page without loss of content, any more you can swap 
> the theme on a msworld document.  
> Option3 is better and most in demand now a days. 
>
> On Sunday, 22 April 2012 21:54:50 UTC+5:30, Massimo Di Pierro wrote:
>>
>> Let's say we want to build a new kick-ass CMS.
>>
>> My technical side tells me that the best way it to use markup language 
>> and separate data from presentation (which allows swapping of themes).
>>
>> My practical side tells is is better to allow users to edit html.
>>
>> Everytime I has worked with end-users I had a hard time explaining this 
>> concept of separation of data from presentation. They usually want a page 
>> tat looks like "that page" but the ability to edit all text and images in 
>> it.
>>
>> Most CMS's (like concrete CMS) solve the problem by a compromise. You can 
>> only edit specific parts of  a page (and they must be clearly tag in the 
>> HTML). This allows some separation because as long as two themes have the 
>> same editable tags, the content it portable between the themes. Yet if they 
>> use a wysiwyg the editable blocks are stored as HTML. Moreover creating 
>> themes requires some programming skills and make the themes CMS specific. 
>> In the case of Concrete5 or Joomla for example, this tagging is done in PHP.
>>
>> So what is better?
>> 1) using a markup language with limited choice of themes (like wikipedia)
>> 2) using wysiwyg to edit fixed sections in themes (like joomla and 
>> concrete5)
>> 3) using fully editable html with no limitation on themes (any existing 
>> page would be a theme without need for tweaking) yet one would not be able 
>> to swap a theme on a page without loss of content, any more you can swap 
>> the theme on a msworld document.
>>
>>
>> Massimo
>>
>

[web2py] Re: Getting https to work with rocket

[Ross Peoples]

You can't connect to https using http as the protocol. Getting a "bad 
request" message is exactly what happens when you try. So in your browser, 
you need to specifically type https://:3.

On Monday, April 16, 2012 1:42:06 PM UTC-4, Shivakumar GN wrote:
>
> Hi,
>
> I am using python 2.5 on Solaris and web2py 1.99.7
>
> I am trying to get https to work with in built web server as per recipe at 
> below link.
> http://www.web2py.com/AlterEgo/default/show/140
>
> I built ssl module built from source since it was missing in python 2.5
>
> > python web2py.py --nogui -i  -p 3 -c server.crt -k 
> server.key
>
> After this, there is no service available via https. On using http, a "bad 
> request" message appears in the browser.
>
> This seems similar to issue reported long back, the discussion thread for 
> which doesn't seem to have reached a closure.
> http://comments.gmane.org/gmane.comp.python.web2py/46045
>
> How to resolve this?
>
> thanks & best regards
> Shivakumar GN
>
>

[web2py] Re: priority of web2py CMS

[Ross Peoples]

No, I've been tied up doing other (non-web2py) projects lately. I hope to 
get back to being more active here in a couple of weeks.

On Sunday, April 15, 2012 2:54:45 AM UTC-4, Gour wrote:
>
> On Mon, 16 Jan 2012 12:41:32 -0800 (PST)
> Ross Peoples 
> wrote:
>
> Hello Ross,
>
> > The good news is I've already figured out how to do it, I just have to
> > put the work in.
>
> Have you done some work on your web2py-powered-CMS-ala-C5?
>
>
> Sincerely,
> Gour
>
> -- 
> But those who, out of envy, disregard these teachings and do not 
> follow them are to be considered bereft of all knowledge, befooled, 
> and ruined in their endeavors for perfection.
>
> http://atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
>
>

[web2py] Re: new web2py cheatsheet

[Ross Peoples]

This new cheat sheet is great! I saw the one page version, but I agree 
there is too much information to fit on to one page. I'm glad a second page 
was added.

On Monday, April 9, 2012 4:44:22 PM UTC-4, Massimo Di Pierro wrote:
>
> http://dl.dropbox.com/u/18065445/Tmp/web2py_cheatsheet.pdf
>

[web2py] Re: How to best contribute?

[Ross Peoples]

Most of my contributions come from code written while working on a project. 
While working on a project, I will find a bug or notice something that 
could be much better, then I correct it and send Massimo a patch. The 
mobile detection feature currently in web2py started as some code that I 
wrote for a project that wanted specific interfaces for mobile devices and 
desktops. Once I had it working, I submitted it. Eventually, others from 
the community have submitted further improvements to it, and now it's an 
excellent feature of web2py that is even used in the admin interface.

So I guess to answer your question, there are many projects and features 
that are on the wanted list from conversations over the last 6 months on 
this group. But probably the best way is to pick something you personally 
want to attack. Submit your changes and study how Massimo and the community 
implement those changes. Before long, you will be overloaded with different 
ideas for things.

One thing that I would personally love to see is a JSON-RPC adapter for the 
DAL so that you can use SQLFORM.grid with other data sources. 
Unfortunately, that is a HUGE undertaking with lots of problems to solve. I 
started to come up with an implementation plan, but I've gotten really busy 
over the last few months and haven't really touched it.

On Monday, March 26, 2012 12:30:00 PM UTC-4, Simon Bushell wrote:
>
> Hi all,
>
> So I am a Web2Py convert, and am now using it for pretty much all of my 
> Pythonic WebDev projects. 
>
> As well as being an excellent framework, I am equally impressed with the 
> passion and friendliness of the Web2Py community and I was wondering what I 
> can I do to help contribute to it. 
>
> *Lurking around this group to help people out?* Easily done
>
> *Helping better improve user documentation?* There seems to be a general 
> mood that it could be improved (though I personally haven't run into to 
> many issues)
>
> *Submitting bug fixes/plugins/apps?* My coding skills are largely 
> self-taught, but I think I am competent enough to contribute if needed. 
>
> Basically - where does the community need help the most and what can I do 
> to assist?
>
> cheers
>
> Simon
>
>
>

[web2py] Re: trouble reading this list

[Ross Peoples]

I would suggest clearing browser cache and restarting it.

On Monday, March 19, 2012 1:20:01 PM UTC-4, Massimo Di Pierro wrote:
>
> Is it just me?
>
> I am having trouble accessing some recent posts on this list. I see the 
> title, author, and content says "loading...".
> I apologize if I cannot answer but I do not know what to do but wait.
>
> massimo
>

Re: [web2py] Re: TEXT data type deprecated in future MSSQL?

[Ross Peoples]

I don't think we need to worry about this for a while. It doesn't say 
"deprecated", but that it will be removed in a future version (semantics, I 
know). But, they have not given any indication of what version, which means 
they probably won't remove it until the version AFTER 2012 (2014, maybe). 
And a lot of people are still using MSSQL 2000 and 2005. When the time 
comes, it would probably be just as easy to make a MSSQL2014 (or whatever 
you want to call it) adapter that subclasses MSSQL2, but uses varchar(max) 
instead of text. Or even an option in MSSQL2 to switch between the two 
names.

On Friday, March 16, 2012 5:43:48 PM UTC-4, Joaquin Orbe wrote:
>
> On Fri, Mar 16, 2012 at 5:45 PM, Massimo Di Pierro
>  wrote:
> > P.S. Please open a google code ticket about it.
>
> Done:
> http://code.google.com/p/web2py/issues/detail?id=722
>
> Regards,
> Joaco.
>
>

Re: [web2py] web2py instance accessible only from localhost

[Ross Peoples]

The -i argument should be 0.0.0.0 instead of 127.0.0.1 if you want other 
machines on the network to access the web2py instance.

On Wednesday, March 7, 2012 3:07:20 PM UTC-5, Richard wrote:
>
> Hello,
>
> I am pretty mistify by that issue. My ssh port forwarding is running just 
> fine, I can start second web2py instance under 8002 port like this :
>
> sudo nohup python web2py.py --nogui -p 8002 -i 127.0.0.1 -a 'PWD' &
>
> It just work.
>
> But when I transfert this command into an init daemon, my file look like 
> this :
>
> #! /bin/sh
> ### BEGIN INIT INFO
> # startup script for Ubuntu and Debian Linux servers
> #
> # To use this file
> # cp ubuntu.sh /etc/init.d/web2py
> #
> # To automatitcally start at reboot
> # sudo update-rc.d web2py defaults
> #
> # Provides:  web2py
> # Required-Start:$local_fs $remote_fs
> # Required-Stop: $local_fs $remote_fs
> # Default-Start: 2 3 4 5
> # Default-Stop:  S 0 1 6
> # Short-Description: web2py initscript
> # Description:   This file starts up the web2py server.
> ### END INIT INFO
>
> # Author: Mark Moore 
>
> PATH=/usr/sbin:/usr/bin:/sbin:/bin
> DESC="Web Framework"
> NAME=web2py
> PIDDIR=/var/run/$NAME
> PIDFILE=$PIDDIR/$NAME.pid
> SCRIPTNAME=/etc/init.d/$NAME
> DAEMON=/usr/bin/python
> DAEMON_DIR=/home/www-data/$NAME
> *DAEMON_ARGS="web2py.py --nogui -p 8002 -i 127.0.0.1 -a 
> 'PWD'--pid_filename=$PIDFILE"
> *
> DAEMON_USER=root
>
> # Exit if the package is not installed
> [ -x "$DAEMON" ] || exit 0
>
> # Read configuration variable file if it is present
> [ -r /etc/default/$NAME ] && . /etc/default/$NAME
>
> # Load the VERBOSE setting and other rcS variables
> [ -f /etc/default/rcS ] && . /etc/default/rcS
>
> # Define LSB log_* functions.
> # Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
> . /lib/lsb/init-functions
>
> #
> # Function that starts the daemon/service
> #
> do_start()
> {
> # Return
> #   0 if daemon has been started
> #   1 if daemon was already running
> #   2 if daemon could not be started
>
> # The PIDDIR should normally be created during installation. This
> # fixes things just in case.
> [ -d $PIDDIR ] || mkdir -p $PIDDIR
> [ -n "$DAEMON_USER" ] && chown --recursive $DAEMON_USER $PIDDIR
>
> # Check to see if the daemon is already running.
> start-stop-daemon --stop --test --quiet --pidfile $PIDFILE \
> && return 1
>
> start-stop-daemon --start --quiet --pidfile $PIDFILE \
> ${DAEMON_USER:+--chuid $DAEMON_USER} --chdir $DAEMON_DIR \
> --background --exec $DAEMON -- $DAEMON_ARGS \
> || return 2
>
> return 0;
> }
>
> #
> # Function that stops the daemon/service
> #
> do_stop()
> {
> # Return
> #   0 if daemon has been stopped
> #   1 if daemon was already stopped
> #   2 if daemon could not be stopped
> #   other if a failure occurred
>
> start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE
> RETVAL=$?
> # Many daemons don't delete their pidfiles when they exit.
> rm -f $PIDFILE
> return "$RETVAL"
> }
>
> #
> # Function that restarts the daemon/service
> #
> do_restart()
> {
> # Return
> #   0 if daemon was (re-)started
> #   1 if daemon was not strated or re-started
>
> do_stop
> case "$?" in
> 0|1)
> do_start
> case "$?" in
> 0) RETVAL=0 ;;
> 1) RETVAL=1 ;; # Old process is still running
> *) RETVAL=1 ;; # Failed to start
> esac
> ;;
> *) RETVAL=1 ;; # Failed to stop
> esac
>
> return "$RETVAL"
> }
>
> #
> # Function that sends a SIGHUP to the daemon/service
> #
> do_reload() {
> #
> # If the daemon can reload its configuration without
> # restarting (for example, when it is sent a SIGHUP),
> # then implement that here.
> #
> start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE
> return 0
> }
>
> #
> # Function that queries the status of the daemon/service
> #
> do_status()
> {
> # Return
> #   0 if daemon is responding and OK
> #   1 if daemon is not responding, but PIDFILE exists
> #   2 if daemon is not responding, but LOCKFILE exists
> #   3 if deamon is not running
> #   4 if daemon status is unknown
>
> # Check to see if the daemon is already running.
> start-stop-daemon --stop --test --quiet --pidfile $PIDFILE \
> && return 0
> [ -f $PIDFILE ] && return 1
> return 3
> }
>
> case "$1" in
>   start)
> [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
> do_start
> RETVAL=$?
> [ "$VERBOSE" != no ] &&
> case "$RETVAL" in
> 0|1) log_end_msg 0 ;;
> *)   log_end_msg 1 ;;
> esac
> exit "$RETVAL"
> ;;
>   stop)
> [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
> do_stop
> RETVAL=$?
> [ "$VERBOSE" != no ] &&
> case "$RETVAL" in
> 0|1) log_end_msg 0 ;;
> *)   log_end_msg 1 ;;
> esac
> exit "$RETVAL"
> ;;
>   #reload|force-reload)
> #
> # If do_reload() is not implemented then leave this commented out
> # and leave 'force-reload' as an alias for 'restart'.
> #
> #[ "$VERBOSE" != no ] && log_daemon_msg "Reloading $DESC" "$NAME"
> #do_reload
> #RETVAL=$?
> #[ "$VERBOSE" != no ] && log_end_msg $?
> #exit "$RETVAL"
> #;;
>   restart|f

Re: [web2py] Get the IP of the System

[Ross Peoples]

Alternatively, you can run ipconfig/ifconfig depending on the operating 
system and parse the results. I have been using this method for a year now 
and it works great. I would much rather have an API solution, but this 
method seemed to be the easiest with the fewest dependencies on external 
libraries, as you only need the subprocess module. It is also able to give 
you ALL of the IP addresses on the machine.

On Tuesday, February 28, 2012 10:14:04 AM UTC-5, José Luis Redrejo 
Rodríguez wrote:
>
> You can use netifaces, http://alastairs-place.net/projects/netifaces/
> El 28/02/2012 11:39, "Sanjeet Kumar"  escribió:
>
>> How we can get the IP address of the system
>>
>
On Tuesday, February 28, 2012 10:14:04 AM UTC-5, José Luis Redrejo 
Rodríguez wrote:
>
> You can use netifaces, http://alastairs-place.net/projects/netifaces/
> El 28/02/2012 11:39, "Sanjeet Kumar"  escribió:
>
>> How we can get the IP address of the system
>>
>
On Tuesday, February 28, 2012 10:14:04 AM UTC-5, José Luis Redrejo 
Rodríguez wrote:
>
> You can use netifaces, http://alastairs-place.net/projects/netifaces/
> El 28/02/2012 11:39, "Sanjeet Kumar"  escribió:
>
>> How we can get the IP address of the system
>>
>
On Tuesday, February 28, 2012 10:14:04 AM UTC-5, José Luis Redrejo 
Rodríguez wrote:
>
> You can use netifaces, http://alastairs-place.net/projects/netifaces/
> El 28/02/2012 11:39, "Sanjeet Kumar"  escribió:
>
>> How we can get the IP address of the system
>>
>

[web2py] Re: Time Substraction

[Ross Peoples]

Well, one of the issues is going to be that you are using 'date' for your 
fields instead of 'datetime'. When using 'datetime', it becomes easy to do 
something like this:

delta = row.employee_logout_date - row.employee_login_date
minutes = delta.minutes + (delta.hours * 60)

On Monday, February 27, 2012 11:54:56 AM UTC-5, Sanjeet Kumar wrote:
>
> I am going to develop the time management sheet for that i have the one 
> table name employee table and i am allready taken the login and logout time 
> and i want to substract the time when i substract the time it show me in 
> nanoseconds when i convert this nanosecond to the minute it totally 
> confused me so if any one have solution please help me following is my 
> controller and database :-
> *
> Table:-*
>
> db.define_table('employee_detail',
> Field('employee_id'),
> Field('employee_name'),
> Field('employee_login_date','date'),
> Field('employee_logout_date','date'),
> Field('employee_login_time'),
> Field('employee_logout_time'),
> Field('total_time'))*
>
> Controller:-*
>
> def lgin(form):
> import datetime
> import time
> currentdate=datetime.date.today()
> now = time.localtime(time.time())
> session.sttime=time.time()
> currenttime = time.strftime("%H:%M:%S", now)
> session.time = currenttime
> #response.flash=currentdate
> for row in db(db.auth_user.email == 
> auth.user.email).select(db.auth_user.first_name):
> firstname=row.first_name
> db.employee_detail.insert(employee_id = auth.user.email, 
> employee_name=firstname, employee_login_date=currentdate, 
> employee_login_time=currenttime)
> return '' 
>  
> def lgout(usr): 
> import datetime
> import time
> currentdate=datetime.date.today()
> now = time.localtime(time.time())
> ettime=time.time()
> duration = ettime-session.sttime
> du = str(duration / 600)
> currenttime = time.strftime("%H:%M:%S", now) 
> db((db.employee_detail.employee_id == auth.user.email) & 
> (db.employee_detail.employee_login_date == currentdate) & 
> (db.employee_detail.employee_login_time == 
> session.time)).update(employee_logout_date=currentdate, 
> employee_logout_time=currenttime, total_time=du)  
> return '' 
>
>
>
On Monday, February 27, 2012 11:54:56 AM UTC-5, Sanjeet Kumar wrote:
>
> I am going to develop the time management sheet for that i have the one 
> table name employee table and i am allready taken the login and logout time 
> and i want to substract the time when i substract the time it show me in 
> nanoseconds when i convert this nanosecond to the minute it totally 
> confused me so if any one have solution please help me following is my 
> controller and database :-
> *
> Table:-*
>
> db.define_table('employee_detail',
> Field('employee_id'),
> Field('employee_name'),
> Field('employee_login_date','date'),
> Field('employee_logout_date','date'),
> Field('employee_login_time'),
> Field('employee_logout_time'),
> Field('total_time'))*
>
> Controller:-*
>
> def lgin(form):
> import datetime
> import time
> currentdate=datetime.date.today()
> now = time.localtime(time.time())
> session.sttime=time.time()
> currenttime = time.strftime("%H:%M:%S", now)
> session.time = currenttime
> #response.flash=currentdate
> for row in db(db.auth_user.email == 
> auth.user.email).select(db.auth_user.first_name):
> firstname=row.first_name
> db.employee_detail.insert(employee_id = auth.user.email, 
> employee_name=firstname, employee_login_date=currentdate, 
> employee_login_time=currenttime)
> return '' 
>  
> def lgout(usr): 
> import datetime
> import time
> currentdate=datetime.date.today()
> now = time.localtime(time.time())
> ettime=time.time()
> duration = ettime-session.sttime
> du = str(duration / 600)
> currenttime = time.strftime("%H:%M:%S", now) 
> db((db.employee_detail.employee_id == auth.user.email) & 
> (db.employee_detail.employee_login_date == currentdate) & 
> (db.employee_detail.employee_login_time == 
> session.time)).update(employee_logout_date=currentdate, 
> employee_logout_time=currenttime, total_time=du)  
> return '' 
>
>
>
On Monday, February 27, 2012 11:54:56 AM UTC-5, Sanjeet Kumar wrote:
>
> I am going to develop the time management sheet for that i have the one 
> table name employee table and i am allready taken the login and logout time 
> and i want to substract the time when i substract the time it show me in 
> nanoseconds when i convert this nanosecond to the minute it totally 
> confused me so if any one have solution please help me following is my 
> controller and database :-
> *
> Table:-*
>
> db.define_table('employee_detail',
> Field('e

[web2py] Re: Web2py and SSLH

[Ross Peoples]

I would be interested to see if SSH can actually be forwarded without 
triggering a main-in-the-middle error.

I'm not sure on the first question, but I would guess that you would want 
to disable everything except your app.

At the bottom of the db.py model, just put "session.forget(request)". This 
will still create cookies, I think but will not actually use them. Not sure 
on this one. Maybe someone else has a better answer for turning cookies off 
completely.

In your model, I would also disable anything you don't need: db, mail, 
auth, etc.

On Sunday, February 26, 2012 1:09:21 PM UTC-5, t13one wrote:
>
> I'm thinking about setting up SSLH on my personal server.
>
> From http://freecode.com/projects/sslh:
> 
>
> > sslh accepts HTTPS, SSH, OpenVPN, tinc, and XMPP connections on the
> > same port. This makes it possible to connect to any of these servers
> > on port 443 (e.g., from inside a corporate firewall, which almost
> > never blocks port 443) while still serving HTTPS on that port.
>
> In short summary (and to my limited understanding), SSLH works by
> forwarding the connection from the sslh daemon to either the ssh server
> or the web-server (among other options). This means all SSL connections
> will ultimately appear to be connecting to apache/web2py via 127.0.0.1.
>
> Are there any security concerns with this? Should I disable admin and
> appadmin completely?
>
> How are session cookies affected?
>
> Would any other functionality be affected?
>
>
On Sunday, February 26, 2012 1:09:21 PM UTC-5, t13one wrote:
>
> I'm thinking about setting up SSLH on my personal server.
>
> From http://freecode.com/projects/sslh:
> 
>
> > sslh accepts HTTPS, SSH, OpenVPN, tinc, and XMPP connections on the
> > same port. This makes it possible to connect to any of these servers
> > on port 443 (e.g., from inside a corporate firewall, which almost
> > never blocks port 443) while still serving HTTPS on that port.
>
> In short summary (and to my limited understanding), SSLH works by
> forwarding the connection from the sslh daemon to either the ssh server
> or the web-server (among other options). This means all SSL connections
> will ultimately appear to be connecting to apache/web2py via 127.0.0.1.
>
> Are there any security concerns with this? Should I disable admin and
> appadmin completely?
>
> How are session cookies affected?
>
> Would any other functionality be affected?
>
>

[web2py] Re: request.user_agent() dumping

[Ross Peoples]

Glad you like the feature! I don't think dimensions are available in the 
user-agent string that the browser passes to the server. So your only 
option is to use jQuery or CSS media queries.

On Sunday, February 26, 2012 11:13:42 PM UTC-5, weheh wrote:
>
> And, I might add, this is simply awesome. It would also be very cool 
> if it could give me the dimensions of the browser window. I know I can 
> get it via jQuery, but it would save a step. 
>
> On Feb 22, 10:15 pm, Massimo Di Pierro  
> wrote: 
> > I think this was fixed in a later version. 
> > 
> > On Feb 21, 9:52 pm, weheh  wrote: 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > > I'm trying to detect the browser being used to access my web2py app. 
> > > The literature seems to recommend reading request.env.http_user_agent. 
> > > But the results are a little confusing, so I'm trying 
> > > request.user_agent(), which uses "gluon/contrib/ 
> > > user_agent_parser.py" (as per the doc). This throws the following 
> > > ticket: 
> > 
> > > Traceback (most recent call last): 
> > >   File "N:\web2py\gluon\main.py", line 518, in wsgibase 
> > > session._try_store_on_disk(request, response) 
> > >   File "N:\web2py\gluon\globals.py", line 528, in _try_store_on_disk 
> > > cPickle.dump(dict(self), response.session_file) 
> > > PicklingError: Can't pickle : it's not the 
> > > same object as storage.Storage 
> > 
> > > I'm on web2py v1.99.2 
> > 
> > > Thanks.


On Sunday, February 26, 2012 11:13:42 PM UTC-5, weheh wrote:
>
> And, I might add, this is simply awesome. It would also be very cool 
> if it could give me the dimensions of the browser window. I know I can 
> get it via jQuery, but it would save a step. 
>
> On Feb 22, 10:15 pm, Massimo Di Pierro  
> wrote: 
> > I think this was fixed in a later version. 
> > 
> > On Feb 21, 9:52 pm, weheh  wrote: 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > > I'm trying to detect the browser being used to access my web2py app. 
> > > The literature seems to recommend reading request.env.http_user_agent. 
> > > But the results are a little confusing, so I'm trying 
> > > request.user_agent(), which uses "gluon/contrib/ 
> > > user_agent_parser.py" (as per the doc). This throws the following 
> > > ticket: 
> > 
> > > Traceback (most recent call last): 
> > >   File "N:\web2py\gluon\main.py", line 518, in wsgibase 
> > > session._try_store_on_disk(request, response) 
> > >   File "N:\web2py\gluon\globals.py", line 528, in _try_store_on_disk 
> > > cPickle.dump(dict(self), response.session_file) 
> > > PicklingError: Can't pickle : it's not the 
> > > same object as storage.Storage 
> > 
> > > I'm on web2py v1.99.2 
> > 
> > > Thanks.

[web2py] Re: using a guid in web2py

[Ross Peoples]

You can also use web2py's more robust UUID-generation method:

from gluon.utils import web2py_uuid
myuuid = web2py_uuid()

[web2py] Re: Why admin|examples|welcome in hgignore?

[Ross Peoples]

I'm not great with regular expressions, but I think it's saying ignore 
everything in the applications folder EXCEPT admin, examples, welcome, and 
__init__. I think that's what the '?!' means.

Re: [web2py] Re: How to Generate the effective report by using web2py

[Ross Peoples]

Sanjeet.

I sent some code to Mariano last night. So we will have to wait and see if 
it works for him.

[web2py] Re: Forms not working with web2py in Chrome

[Ross Peoples]

I don't know if it helps, but I spotted a typo:

Replace the dollar sign in:

'image,
format = '$(title)s'

with

format ='%(title)s'

[web2py] Re: Associative Table

[Ross Peoples]

Good practice says you should ALWAYS have an identity (ID) field, even if 
you don't use it.

Re: [web2py] Re: How to Generate the effective report by using web2py

[Ross Peoples]

I had to write a lot of code to get FPDF to do proper headers, footers, and 
page numbers. Then about another 100 lines of code to get it to wrap long 
lines of text inside of a table. I'm also not using HTML to PDF conversion. 
I am using FPDF's methods to create PDFs. If I recall correctly, I tried to 
use paragraphs and even new line characters to force the line to break, but 
ended up having to use absolute positioning for the lines. It sounds 
terrible, I know, but I have been using this successfully for a little 
while. I basically wrote a module that takes a query (just like 
SQLFORM.grid) and generates a PDF with headers, footers, a logo, and can 
even group data into multiple tables, all while wrapping long text and 
repeating column headers on each new page. It was a lot of work, but the 
reports look pretty good.

Mariano, what about FPDF are you planning on updating? And are you planning 
to break backwards-compatibility?

Re: [web2py] OFF Topic - Business inteligence in Python, web2py?

[Ross Peoples]

I think he means reporting functions like Crystal Reports or SQL Server 
Reports.

[web2py] Re: can we use JSONRPC to upload?

[Ross Peoples]

The best way might be to use JSONRPC to get an authorization code, which is 
then POSTed along with the binary contents in a separate HTTP request.

[web2py] Re: can we use JSONRPC to upload?

[Ross Peoples]

I've never tried before, but traditionally, you would encode the binary 
file into a base64 string, then decode it on the other end. I don't know if 
JSONRPC does this for you, or has a better way.

[web2py] Re: css before js advice

[Ross Peoples]

I've always read that CSS should go in the  and JS should go right 
before the closing  tag.

[web2py] Re: digitally sign static files?

[Ross Peoples]

I don't think you can do anything with the static folder. You would have to 
put all the files in "uploaded" or another custom folder, then do something 
like {{=URL('default', 'download', user_signature=True)}}.

[web2py] Re: * DATE FIELDS REPRESENT *

[Ross Peoples]

I don't know if this is the correct way, but I have done this before:

{{=db.contracts.date_start.represent(row.date_start)}}

[web2py] Re: Memcache ??

[Ross Peoples]

http://web2py.com/books/default/chapter/29/4?search=cache#cache

When it takes a while to figure something out, like a complex query, you 
typically want to hold on to the result for a little while so that the next 
few requests don't have to do through the same process every time. It is 
used to increase performance.

memcached is an external process that web2py can use to do caching. In 
large deployments, memcached has a performance increase over the standard 
web2py cache methods.

[web2py] New Plugin: plugin_markitup

[Ross Peoples]

I have created another plugin for the MarkItUp widget. I know that 
plugin_wiki has this currently, but I wanted a dedicated plugin for this 
for an upcoming project I'm working on.

You can get the plugin from 
Bitbucket: https://bitbucket.org/PhreeStyle/web2py_markitup

There is no documentation yet, as I just pushed the code. But I wanted to 
announce in case anyone wants to play with this. It has not been thoroughly 
tested and is only an alpha at this point. It can be used with several 
different markup languages:

   - BBCode
   - HTML
   - Markdown
   - Markmin
   - reStructuredText
   - Textile
   - Wiki

The plugin can be used standalone, or as a form field widget. It currently 
supports live previews for 4 markup languages: html, markmin, markdown, 
textile, and bbcode. Code blocks for markmin and markdown markup are 
highlighted by pygments, which is included as part of the plugin. No 
external dependencies. Using pygments for code highlighting allows over 100 
languages to be highlighted.

Code highlighting in markmin:

``
def testing():
print 'Testing'
``:python

Code highlighting in markdown:

:::python
def testing():
print 'Testing'

Markdown requires four spaces at the beginning of each line for code blocks.

Example for basic usage:

from plugin_markitup.markitup import MarkItUp

def test():
widget = MarkItUp(set_name='markmin').markitup()
return dict(widget=widget)


Example model for form fields:

from plugin_markitup.markitup import MarkItUp

db.define_table('content',
Field('name', length=20),
Field('description', 'text')
)
db.content.description.widget = MarkItUp().widget


Example controller:
def test():
form = SQLFORM(db.content)
if form.accepts(request, session):
redirect(URL())

return dict(form=form)


This plugin is more of an alpha preview, so I expect there to be problems 
and inconsistencies. Please try it out and let me know if you have any 
questions, comments, or problems!

[web2py] Re: Markmin Output Questions

[Ross Peoples]

Massimo,

How do you generate the PDF book now?

Re: [web2py] Dedicated IDE

[Ross Peoples]

I have been following ide/rad2py for a little while now. I am really 
looking forward to trying this out and I appreciate all the work you have 
put into it Mariano!

Depending on the type of project, I will use either TextMate or Komodo 
Edit. For larger projects, I use Komodo for the code completion and 
organization. I also use SourceTree for source code management. I tried 
Eclipse + PyDev / Aptana, but it seems way too heavy and doesn't really 
really offer much more than I have already. I didn't play with it too long, 
as it reminded me of Java development too much, so I could be wrong and 
could totally be missing out.

I am looking forward to playing with ide/rad2py to see if it can unseat 
Komodo as my IDE of choice.

[web2py] Re: Image from Video

[Ross Peoples]

You would have to compile it yourself in that case. I don't know how to do 
that on Windows, so maybe someone else would have some tips for that.

[web2py] Re: Markmin Output Questions

[Ross Peoples]

Anyone have any thoughts on this?

[web2py] Re: Image from Video

[Ross Peoples]

ffmpeg is video encoder / decoder. You might be able to use the Python 
bindings for it to grab a single frame of video from the video file, and 
save that as a PNG or JPG. 

This is the project page for the ffmpeg bindings:
http://code.google.com/p/pyffmpeg/

I've never done this before myself, but looking at the documentation (at 
least for version 1), you should be able to do something like this once 
you've got pyffmpeg installed:

import pyffmpeg

stream = pyffmpeg.VideoStream()
stream.open('myvideo.avi')
image = stream.GetFrameNo(0)
image.save('firstframe.png')

Re: [web2py] Re: [w2py-dev] Re: Movuca - Social CMS beta 0.1

[Ross Peoples]

CKEditor is licensed under LGPL, so you're good there. As for 
plugin_ckeditor. I wrote it, but I haven't given it a license yet (on my 
todo list). However, I will probably go with LGPL as well. It only makes 
sense as web2py and CKEditor are both LGPL. So you are all set with the 
CKEditor stuff.

Everything that I release and publish to the public, such as 
plugin_ckeditor, I do so with the intent to allow people to use it for 
their own purposes, whatever they may be, so long as they contribute 
changes make to my work back to the public. This is pretty much what the 
LGPL is for (from my understanding).

[web2py] Markmin Output Questions

[Ross Peoples]

I am looking into writing lots of documentation and have been looking 
around at the different solutions. I know that the web2py book was written 
with Markin, and seeing as how it is able to output to multiple different 
formats and supports things like tables and image alignment, I figure it's 
probably the best way to go.

I want the documentation to be viewable (like the web2py book online) and 
be able to generate a PDF book, just like the web2py book. How do I do 
this? And here are a few more specific questions I have about this process:

   - Should each chapter (or topic) be in it's own file?
   - How do you insert page numbers, pages breaks, etc into the resulting 
   PDF file?
   - How do I generate the table of contents and the index?

This is the first time I've done anything remotely professional with 
documentation, so any other tips would be helpful. Thanks!

[web2py] Re: security check in web2py

[Ross Peoples]

Another option, which I use is to generate SSL certs and run web2py with -k 
and -c so that web2py runs on SSL, enabling admin from other computers.

[web2py] Re: Folder inside controller dosen't work.

[Ross Peoples]

Not sure on the subfolder thing, but is it possible for you to put most of 
your code into modules and just use controllers as the gateway to your 
modules?

[web2py] Re: xml-rpc code causes compile error in mywiki default.py

[Ross Peoples]

You have a syntax error in fony_by(keyword):

return db(db.page.title.contains(keyword).select().as_list()

Should be:

return db(db.page.title.contains(keyword)).select().as_list()

You forgot the closing parenthesis before .select().

[web2py] Re: option to move smartgrid buttons to the left side

[Ross Peoples]

I was wondering about the previously. I haven't tried this yet, but as a 
temporary workaround, are you able to insert buttons using JavaScript?

Re: [web2py] Re: keep shared objects permanently in RAM: Is it possible ?

[Ross Peoples]

Thanks for pointing that out Michele. I may need to use something like this 
in the future.

[web2py] Re: Modules: how to access db

[Ross Peoples]

from gluon import *

This should give you the "current" object which has these:

current.request
current.response
current.cache

You must initialize your modules with a db instance though. So I usually do 
this:

class MyModule(object):
def __init__(self, db, ...)

Re: [web2py] Re: keep shared objects permanently in RAM: Is it possible ?

[Ross Peoples]

That is a VERY good question. If you already have a setup like this, give 
it a shot and see if it worksI would be interested to know the result.

[web2py] Re: is_https

[Ross Peoples]

Using nginx, you have to have a line somewhere for "fastcgi_pass". If this 
line isn't:

fastcgi_pass http://127.0.0.1:9000

Then you will have problems. When you connect to web2py through nginx, 
web2py only sees nginx. So if nginx is passing fastcgi to another computer 
on the network, then web2py thinks that the nginx server is the client, on 
another machine, not using SSL.

I have had to deal with this issue myself, but maybe for a different reason 
than you. I have one gateway server that my router passes all port 80 and 
443 traffic to. However, I run several different web servers on different 
computers inside the network, so I use the gateway (running nginx) to proxy 
to the inside servers based on domain name. The inside servers are also 
running nginx, and running web2py locally. Therefore, I am using 
fastcgi_pass http://127.0.0.1:9000 and web2py assumes that you are 
accessing it from the local host, allowing admin. My gateway server also 
handles the SSL connections, as there is no need to SSL from one inside 
server to another.

Re: [web2py] Re: keep shared objects permanently in RAM: Is it possible ?

[Ross Peoples]

Sorry Mariano, I misspelled your name in my last post :)

To give you a better example of what you might want to do here is a more 
advanced module that makes a static class and better resembles the 
singleton pattern:

class Counter(object):
instance = None

@classmethod
def get(cls):
if cls.instance == None:
cls.instance = Counter()

return cls.instance

def __init__(self, message='Hello World!'):
self.message = message
self.count = 0

def get_count(self):
self.count += 1
return self.count

def get_message(self):
return self.message


Then your controller would look something like this now:

from mymodule import Counter
counter = Counter.get()
count = counter.get_count()

return dict(count=count)

By calling Counter.get() instead of Counter(), we ensure that there is only 
ever once instance of the Counter object, and that the object will last for 
the lifetime of the web2py instance.

Re: [web2py] Re: keep shared objects permanently in RAM: Is it possible ?

[Ross Peoples]

Marino is referring to the method I mentioned earlier of using modules and 
the singleton pattern. You could use this in combination with Michele's 
mmap suggestion to hold the data to do what you want.

I've never used mmap before, but I can get you on the right track with 
modules. This is a very (very) simple example, but it works:

Create a module called "mymodule.py" and put the following line in it:

count = 0

Then in your controller, put the following code in:

import mymodule

count = mymodule.count
mymodule.count += 1

return dict(count=count)

Then in your view, put this somewhere:

{{=count}}

Every time you visit that page, the number will increment. The purpose of 
this is to show you that module (not models) stick around once you import 
them. The only way to update the module (like if you changed some code in 
it) is to restart web2py. So once you import this module, it will continue 
to increment the number. The only way to make it "forget" the number is to 
restart web2py.

Re: [web2py] default layout.html error (or just me)

[Ross Peoples]

Glad you figured it out!

Re: [web2py] Workflow engine for web2py

[Ross Peoples]

I don't know workflow engines in the general sense that well. I once 
created a full documentation management system where each document couple 
have a workflow assigned to it. The workflow (predefined, or created 
on-the-fly) would push the document around from person to person, ensuring 
that each person completed their step in the time allotted, otherwise an 
email would get sent to their manager.

Would a workflow engine allow me to do the same thing, or is this a 
different type of workflow engine? 

[web2py] Re: [w2py-dev] Re: Movuca - Social CMS beta 0.1

[Ross Peoples]

I'm not sure how much my opinion matters here, but a lot of times, I am not 
allowed to touch GPL code, especially AGPL code for a business project. The 
legal department avoids (A)GPL like the plague. There are just too many 
gotchas with it, whether real or imaginary. They much prefer I use MIT or 
BSD, and have started to come around to LGPL. But there is no way they will 
let me use anything more restrictive. Our legal department can't be the 
only one in the corporate world that feels the same way.

So if you want real businesses to touch code, it has to be LGPL or better 
(less restrictive). I believe this was one of the reasons web2py is using 
LGPL now. But this is your project, and a great one at that! So feel free 
to license it however you like, just be aware of the adoption issues.

[web2py] Re: Set username as default

[Ross Peoples]

I usually do this:

default = auth.user.username.upper() if auth.user else None

[web2py] Re: default layout.html error (or just me)

[Ross Peoples]

Sorry, I meant views/default/index.html, not controllers/default/index.html.

[web2py] Re: default layout.html error (or just me)

[Ross Peoples]

Can you paste the code in your controllers/default/index.html? Also, you 
should be able to scroll down near the bottom of the ticket to see the line 
of code in the view that triggers the issue.

[web2py] Re: VPScolo.com Five dolar VPS, anybody knows it?

[Ross Peoples]

I haven't heard of them before, but I have learned a quick lesson about VPS 
providers: most of them are one-man operations that only last for a few 
months. I have done a LOT of research on VPS providers over the last couple 
of weeks and the one site I always end up turning to for advice on 
providers is Low End Box: http://www.lowendbox.com/

I did a search for VPScolo on LEB and found an ad from 2010. So the fact 
that they've been around for at least 2 years is a good sign. You don't get 
a whole lot of resource though. So far, I have only found two VPS providers 
that give you the most resources for your money. If you need bandwidth, go 
for RethinkVPS. They offer unlimited transfer. If you need more memory, 
look at ChicagoVPS. I got in on a good deal with them: 2GB of RAM, 2TB of 
transfer for $7/month.

As for the burst, you have 256 MB of RAM. The burst is like swap space. It 
is there for temporary usage only. Many providers provide you with burst 
memory in case your site is listed on Reddit or Slashdot and experiences a 
sudden spike in traffic. If you are eating into burst memory often, your 
provider will make you upgrade to a better plan, or ask you to leave. So 
make sure that 256 MB is more than sufficient for your needs.

As a quick note, I am not associated with VPS providers. I was looking for 
one recently, so I started doing my homework for a provider that met my 
needs. I should also mention that these are cheap plans, and are a step up 
from shared hosting, but they are not going to be as high quality as a 
provider such as VPS.net or Rackspace, etc. So there will be some downtime 
every now and then.

Re: [web2py] Re: keep shared objects permanently in RAM: Is it possible ?

[Ross Peoples]

I can't imagine that every request would need to access all 16G every time. 
I use a method for caching data between requests using modules. I don't 
know if it would be truly shared between requests, or like Niphlod said, 
would be duplicated for each request. I have had a lot of success using 
modules + singleton pattern to hold small data in memory, but never used it 
with anything large. So maybe you should play with this idea and see how it 
responds.

[web2py] Re: lungo.js

[Ross Peoples]

GPL licensing would be an issue for me.

[web2py] Re: Debugging in web2py

[Ross Peoples]

Forgot to mention that when the debugger pauses on a line of code, besides 
using the key commands mentioned before, you can run single-line Python 
commands as if you were in an interactive session.

This allows you to inspect variables and such. For example, once the 
debugger pauses and displays the (Pdb) prompt, you can type "print request" 
to print the entire contents of the request object to the console.

[web2py] Re: Debugging in web2py

[Ross Peoples]

Run web2py from console, and put "import pdb; pdb.set_trace()" right above 
the line of code where you want to start debugging. From there, use "n" to 
go to the next line, "s" to "step into" the next line, and "c" to continue 
execution.

Good luck!

[web2py] Re: Opinions, please: best way to set up multiple apps on one site?

[Ross Peoples]

If they are only sharing the auth table, then it's not a big deal, since 
the auth tables are there by default. But plugins are meant to be 
completely isolated from each other, completely unaware that other plugins 
exist.

Putting core logic (accessing data) into modules and using more generic 
plugins to handle the views might be the best way to go. I use modules 
extensively, and only use models to set up the module instances as global 
variables. I use the singleton pattern for my modules, which works well for 
me, but Bruno's approach is also pretty good, but a little more complicated.

[web2py] Re: Opinions, please: best way to set up multiple apps on one site?

[Ross Peoples]

I have used the plugin approach before with success on a medium-sized app. 
You just have to design them properly or the plugins will rely on each 
other, which kind of defeats the purpose.

Re: [web2py] Re: help with js issue

[Ross Peoples]

In reference to the cron error, I posted this a couple days ago:

https://groups.google.com/forum/#!searchin/web2py/ross$20peoples$20cron%7Csort:date/web2py/q4qoTTuu6zw/jo-H1LPjWB4J


Also, I should mention that for the stopping problem: It's not just Mac. 
The previous message with instructions on how to reproduce was done using 
Ubuntu 10.04.

Re: [web2py] Re: help with js issue

[Ross Peoples]

I was able to reproduce this a few times by doing this:

Start web2py:
python web2py.py -a password -i 0.0.0.0 -N

Navigate to the admin app from another computer (important):
http://192.168.1.10:8000/admin

You will get the "admin is disabled" message. Now try to Control + C 
web2py. For me, it took about 30 seconds to quit.

[web2py] Re: [w2py-dev] model less apps (a blog example)

[Ross Peoples]

Bruno,

This is a good article. I have done something like this before. My approach 
was a bit different. I was using the singleton pattern, but I think it 
accomplishes the same goal.

I would for example have a module like this:

from gluon import *

class MyModel(object):
instance = None

def get(db):
if instance is None:
instance = MyModel(db)

return instance

def __init__(self, db):
db.define_table()


Then I would still have a db.py for a model, but I would replace the usual 
db.define_table() with this:

from mymodel import MyModel
MyModel.get(db)

This ensures that the tables only get defined once on the first request 
after starting web2py. I have used this approach on other code that I 
needed to keep alive for other requests.