Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Chris Travers
On Tue, Jul 26, 2011 at 8:13 PM, Karl Nack karln...@futurityinc.com wrote: The current svn trunk (to be 1.3) does. So how far do you take this? I've been playing around with plpgsql a bit and am pretty encouraged by what I'm discovering; now I'm at a point where I'm thinking, how far do/can I

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Vincent Veyron
Le mardi 26 juillet 2011 à 11:23 -0500, Merlin Moncure a écrit : I think I've been cursed due to have discovered the secret to fast, efficient programming while continually being constrained from developing that way. Then again, most programmers probably feel like that :-). I think

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Karl Nack
The current svn trunk (to be 1.3) does. So how far do you take this? I've been playing around with plpgsql a bit and am pretty encouraged by what I'm discovering; now I'm at a point where I'm thinking, how far do/can I go with this? Probably the best example is input validation. Constraints and

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Merlin Moncure
On Wed, Jul 27, 2011 at 1:41 AM, Chris Travers chris.trav...@gmail.com wrote: On Tue, Jul 26, 2011 at 8:13 PM, Karl Nack karln...@futurityinc.com wrote: The current svn trunk (to be 1.3) does. So how far do you take this? I've been playing around with plpgsql a bit and am pretty encouraged by

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Chris Travers
On Wed, Jul 27, 2011 at 7:01 AM, Merlin Moncure mmonc...@gmail.com wrote: Note, we wrote libpqtypes (http://libpqtypes.esilo.com/) precisely to deal with this problem -- first class handling of arrays and composites in the client.  It's not much help for a perl client, but I think similar

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Karsten Hilbert
I wonder which other languages have first class support for these areas of Pg? While already supporting most if not all standard PG datatypes the psycopg2 interface lets you write in/out wrappers of arbitray complexity mapping PG - Python datatypes and insert them into the driver at runtime.

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Karl Nack
I wonder which other languages have first class support for these areas of Pg? It'd be nice if PHP could get there. :p Maybe it's time to look at some of these other languages. Or finally learn C and try hacking on the extension myself. Karl Nack Futurity, Inc 5121 N Ravenswood Ave Chicago,

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Peter Bex
On Wed, Jul 27, 2011 at 04:35:45PM +0200, Karsten Hilbert wrote: I wonder which other languages have first class support for these areas of Pg? While already supporting most if not all standard PG datatypes the psycopg2 interface lets you write in/out wrappers of arbitray complexity

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Karl Nack
The best option is to use exceptions to communicate to the application what went wrong and then allow the application to handle those exceptions in many cases. In other cases, the application may need to know which inputs are mandatory. So other than sanitizing input and making sure it's an

Re: [GENERAL] Implementing thick/fat databases

2011-07-27 Thread Sim Zacks
On 07/27/2011 07:18 PM, Karl Nack wrote: The best option is to use exceptions to communicate to the application what went wrong and then allow the application to handle those exceptions in many cases. In other cases, the application may need to know which inputs are mandatory. So other than

Re: [GENERAL] Implementing thick/fat databases

2011-07-26 Thread Sim Zacks
On 07/25/2011 06:24 PM, Chris Travers wrote: On Sun, Jul 24, 2011 at 11:53 PM, Sim Zackss...@compulab.co.il wrote: The goal is to make our system client agnostic, Most of our GUI is written in wxpython, we also have some web functions and even a barcode terminal function, written in C#. We

Re: [GENERAL] Implementing thick/fat databases

2011-07-26 Thread Chris Travers
On Tue, Jul 26, 2011 at 1:04 AM, Sim Zacks s...@compulab.co.il wrote: No need for PL/Mono or any other client specific language. The GUI should be dumb, so all I really need to program design is the interface and input output methods. When you push a button, it should call the appropriate

Re: [GENERAL] Implementing thick/fat databases

2011-07-26 Thread Merlin Moncure
On Mon, Jul 25, 2011 at 5:21 PM, Chris Travers chris.trav...@gmail.com wrote: On Mon, Jul 25, 2011 at 12:33 PM, Merlin Moncure mmonc...@gmail.com wrote: exactly. procedural middlewares written in languages like java tend to be bug factories: *) over-(mis-)use of threads *) performance wins

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Sim Zacks
On 07/24/2011 06:58 PM, Chris Travers wrote: On Sat, Jul 23, 2011 at 11:44 PM, Sim Zackss...@compulab.co.il wrote: I gave a talk on using postgresql as an application server at PG East in March. Basically, we try to implement all business logic using functions, using plpythonu when

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Frank Lanitz
Am 22.07.2011 21:15, schrieb Karl Nack: to move as much business/transactional logic as possible into the database, so that client applications become little more than moving data into and out of the database using a well-defined API, most commonly (but not necessarily) through the use of stored

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Pavel Stehule
2011/7/25 Frank Lanitz fr...@frank.uvena.de: Am 22.07.2011 21:15, schrieb Karl Nack: to move as much business/transactional logic as possible into the database, so that client applications become little more than moving data into and out of the database using a well-defined API, most

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Frank Lanitz
Am 25.07.2011 10:12, schrieb Pavel Stehule: 2011/7/25 Frank Lanitzfr...@frank.uvena.de: Am 22.07.2011 21:15, schrieb Karl Nack: to move as much business/transactional logic as possible into the database, so that client applications become little more than moving data into and out of the

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Sim Zacks
On 07/25/2011 11:06 AM, Frank Lanitz wrote: Am 22.07.2011 21:15, schrieb Karl Nack: to move as much business/transactional logic as possible into the database, so that client applications become little more than moving data into and out of the database using a well-defined API, most commonly

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Frank Lanitz
Am 25.07.2011 10:24, schrieb Sim Zacks: On 07/25/2011 11:06 AM, Frank Lanitz wrote: Am 22.07.2011 21:15, schrieb Karl Nack: to move as much business/transactional logic as possible into the database, so that client applications become little more than moving data into and out of the database

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Chris Travers
On Sun, Jul 24, 2011 at 11:53 PM, Sim Zacks s...@compulab.co.il wrote: The goal is to make our system client agnostic, Most of our GUI is written in wxpython, we also have some web functions and even a barcode terminal function, written in C#. We would like to use an application server, so

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Chris Travers
On Mon, Jul 25, 2011 at 1:24 AM, Sim Zacks s...@compulab.co.il wrote: If I understand you correctly, you are saying that to handle business logic processing, I may require X servers. Only a percentage of that traffic actually requires database processing. if I use a cluster of application

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Merlin Moncure
On Sun, Jul 24, 2011 at 12:51 AM, Chris Travers chris.trav...@gmail.com wrote: I was thinking similar thoughts, but you not only beat me to it, you made some good points I had not thought of! The only thing I can think of adding: is that it would be good to lock down the database so that only

Re: [GENERAL] Implementing thick/fat databases

2011-07-25 Thread Chris Travers
On Mon, Jul 25, 2011 at 12:33 PM, Merlin Moncure mmonc...@gmail.com wrote: exactly. procedural middlewares written in languages like java tend to be bug factories: *) over-(mis-)use of threads *) performance wins moving logic outside the database to scale it are balanced out by the extra

Re: [GENERAL] Implementing thick/fat databases

2011-07-24 Thread Sim Zacks
I gave a talk on using postgresql as an application server at PG East in March. Basically, we try to implement all business logic using functions, using plpythonu when necessary. For example, we have functions that send email, ftp files, sync remote databases, etc. It is important to

Re: [GENERAL] Implementing thick/fat databases

2011-07-24 Thread Karsten Hilbert
Hello Chris, In LedgerSMB, we take this a step further by making the procedures into discoverable interfaces, how do you do that ? Karsten -- NEU: FreePhone - 0ct/min Handyspartarif mit Geld-zurück-Garantie! Jetzt informieren: http://www.gmx.net/de/go/freephone -- Sent

Re: [GENERAL] Implementing thick/fat databases

2011-07-24 Thread Gavin Flower
On 24/07/11 17:51, Chris Travers wrote: I was thinking similar thoughts, but you not only beat me to it, you made some good points I had not thought of! The only thing I can think of adding: is that it would be good to lock down the database so that only the middleware can access it, everything

Re: [GENERAL] Implementing thick/fat databases

2011-07-24 Thread Chris Travers
On Sun, Jul 24, 2011 at 12:34 AM, Karsten Hilbert karsten.hilb...@gmx.net wrote: Hello Chris, In LedgerSMB, we take this a step further by making the procedures into discoverable interfaces, how do you do that ? The way we do things in 1.3 beta is relatively free-form and not idea. In

Re: [GENERAL] Implementing thick/fat databases

2011-07-24 Thread Chris Travers
On Sat, Jul 23, 2011 at 11:44 PM, Sim Zacks s...@compulab.co.il wrote: I gave a talk on using postgresql as an application server  at PG East in March. Basically, we try to implement all business logic using functions, using plpythonu when necessary. For example, we have functions that send

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Alban Hertroys
On 22 Jul 2011, at 21:15, Karl Nack wrote: But this still falls short, since we're still basically managing the transaction in the application layer. The problem you're facing here is that database statements work with records, while your example has a need to handle a set of (different types

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Chris Travers
On Fri, Jul 22, 2011 at 12:15 PM, Karl Nack karln...@futurityinc.com wrote: I've been following a few blogs (http://database-programmer.blogspot.com/, http://thehelsinkideclaration.blogspot.com/) that make a very compelling argument, in my opinion, to move as much business/transactional logic

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread David Johnston
Now, if you are doing double-entry bookkeeping this doesn't provide enough consistency, IMO. You can't check inside the function to ensure that the transaction is balanced. it would be better to: BEGIN; SELECT create_transaction(1, current_date, 'Transaction 1', '{{1, 1,

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread David Johnston
Now for the application, you can create an API that is semantically clearer. But PostgreSQL doesn't provide an easy way of calling procedures of this sort out of select/update/insert statements and select is the only way to do this. A semantically accurate way to run procedures is:

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Chris Travers
On Sat, Jul 23, 2011 at 1:32 PM, David Johnston pol...@yahoo.com wrote: Now for the application, you can create an API that is semantically clearer.  But PostgreSQL doesn't provide an easy way of calling procedures of this sort out of select/update/insert statements and select is the only

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Darren Duncan
Karl Nack wrote: SELECT create_transaction(1, current_date, 'Transaction 1', ((1, 50), (2, -50))); Well, not quite, because specifying the number 50 twice would be ridiculous for such a non-generic function; you can calculate the - 50 from it in the function. Not if there were more than two

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Karl Nack
Now, if you are doing double-entry bookkeeping this doesn't provide enough consistency, IMO. You can't check inside the function to ensure that the transaction is balanced. One option to consider is restricting final tables but making staging tables available. I would implement this

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread David Johnston
I was assuming here that we were dealing with a double-entry accounting system where every line item in one account had a corresponding line item in another account of equal magnitude, and so all line items were in pairs, because you showed what looked like 1 pair, hence specifying the

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Chris Travers
On Sat, Jul 23, 2011 at 3:51 PM, Karl Nack karln...@futurityinc.com wrote: In LedgerSMB, we take this a step further by making the procedures into discoverable interfaces, so the application logic itself is a sort of thin glue between a UI layer and the database procedure layer.  One thing I

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Chris Travers
CREATE OR REPLACE FUNCTION payment_bulk_post (in_transactions numeric[], in_batch_id int, in_source text, in_total numeric,        in_ar_ap_accno text, in_cash_accno text,        in_payment_date date, in_account_class int, in_payment_type int,        in_exchangerate numeric, in_curr text) Oh

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Karl Nack
For example, you could use XML to describe the data (I'm no fan of XML, but it does suit a need here and allows validation of the provided data), especially as Postgres has XML parsing functionality. Would you go so far as to suggest making the entire parameter an XML statement, something

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Karl Nack
In LedgerSMB, we take this a step further by making the procedures into discoverable interfaces, so the application logic itself is a sort of thin glue between a UI layer and the database procedure layer. One thing I would suggest is to try to keep API calls as atomic as possible. You want

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Karl Nack
SELECT create_transaction(1, current_date, 'Transaction 1', ((1, 50), (2, -50))); Well, not quite, because specifying the number 50 twice would be ridiculous for such a non-generic function; you can calculate the - 50 from it in the function. Not if there were more than two line-items per

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Gavin Flower
On 23/07/11 12:05, John R Pierce wrote: On 07/22/11 4:11 PM, Darren Duncan wrote: Karl Nack wrote: I've been following a few blogs (http://database-programmer.blogspot.com/, http://thehelsinkideclaration.blogspot.com/) that make a very compelling argument, in my opinion, to move as much

Re: [GENERAL] Implementing thick/fat databases

2011-07-23 Thread Chris Travers
I was thinking similar thoughts, but you not only beat me to it, you made some good points I had not thought of! The only thing I can think of adding: is that it would be good to lock down the database so that only the middleware can access it, everything else accesses the database via the

[GENERAL] Implementing thick/fat databases

2011-07-22 Thread Karl Nack
I've been following a few blogs (http://database-programmer.blogspot.com/, http://thehelsinkideclaration.blogspot.com/) that make a very compelling argument, in my opinion, to move as much business/transactional logic as possible into the database, so that client applications become little more

Re: [GENERAL] Implementing thick/fat databases

2011-07-22 Thread Darren Duncan
Karl Nack wrote: I've been following a few blogs (http://database-programmer.blogspot.com/, http://thehelsinkideclaration.blogspot.com/) that make a very compelling argument, in my opinion, to move as much business/transactional logic as possible into the database, so that client applications

Re: [GENERAL] Implementing thick/fat databases

2011-07-22 Thread John R Pierce
On 07/22/11 4:11 PM, Darren Duncan wrote: Karl Nack wrote: I've been following a few blogs (http://database-programmer.blogspot.com/, http://thehelsinkideclaration.blogspot.com/) that make a very compelling argument, in my opinion, to move as much business/transactional logic as possible into

Re: [GENERAL] Implementing thick/fat databases

2011-07-22 Thread Darren Duncan
John R Pierce wrote: On 07/22/11 4:11 PM, Darren Duncan wrote: Karl Nack wrote: I've been following a few blogs (http://database-programmer.blogspot.com/, http://thehelsinkideclaration.blogspot.com/) that make a very compelling argument, in my opinion, to move as much business/transactional