Re: [HACKERS] I just got it: PostgreSQL Application Server -- a new project.
Jumping on that bandwagon with all 6 feet! Carl |};-) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, June 11, 2004 9:38 PM To: [EMAIL PROTECTED] Subject: [HACKERS] I just got it: PostgreSQL Application Server -- a new project. I have been harping for the last few days (years, actually) about tweaks and changes to PostgreSQL for a number of reasons ranging from session management to static tables. I even had a notion to come up with msession on PostgreSQL. I have been incorporating full text search, recommendations, and a slew of other features into PostgreSQL, but you know what? While it does touch Postgre in a real sense, it is not strictly SQL. It is about how to create applications with PostgreSQL. That's what we're missing, Coneptually, PostgreSQL is strictly a database and the core team (rightly so) is fundimentally happy with that aspect of it. Maybe we need a pgfoundary project called PostgreSQL Application Server. Like Apache Tomcat or regular apache or PHP, PostgreSQL could form the SQL base of a far more intricate and flexable framework that encompases a lot of the various features that could provide application sever features from PostgreSQL. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] I just got it: PostgreSQL Application Server -- a new project.
If you consider an app server as a container for functionality that persists data/control state beyond a single invocation, PostgreSQL (and lots of other DP solutions, of course) falls into the category already, ne? I suppose my def. is too gross, but I agree with [EMAIL PROTECTED]'s conjecture that Postgres COULD provide externalization hooks so that the SQL engine could integrate external services/data-sources without recompiling the backend, postmaster, or any other kind of major (and dangerous) restructuring. I believe that some of the contribs (like PL/Perl) have functionality for controlling external processes from stored-procedures, but this kind of functionality should be in the kernel of the engine, in my mind. If I'm totally offbase, plz correct. Carl |};-) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Thomas Hallgren Sent: Saturday, June 12, 2004 9:47 AM To: [EMAIL PROTECTED] Subject: Re: [HACKERS] I just got it: PostgreSQL Application Server -- a new project. Even if I find the concepts as such very interesting, I think the term Application Server is very misleading. People would get very confused and place PostgreSQL in the same category as JBoss, Jonas, Apache Geronimo, IBM Websphere, BEA Weblogic to name a few well known App-servers. IMHO, you really need some other umbrella name for this. Kind regards, Thomas Hallgren Carl E. McMillin [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Jumping on that bandwagon with all 6 feet! Carl |};-) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, June 11, 2004 9:38 PM To: [EMAIL PROTECTED] Subject: [HACKERS] I just got it: PostgreSQL Application Server -- a new project. I have been harping for the last few days (years, actually) about tweaks and changes to PostgreSQL for a number of reasons ranging from session management to static tables. I even had a notion to come up with msession on PostgreSQL. I have been incorporating full text search, recommendations, and a slew of other features into PostgreSQL, but you know what? While it does touch Postgre in a real sense, it is not strictly SQL. It is about how to create applications with PostgreSQL. That's what we're missing, Coneptually, PostgreSQL is strictly a database and the core team (rightly so) is fundimentally happy with that aspect of it. Maybe we need a pgfoundary project called PostgreSQL Application Server. Like Apache Tomcat or regular apache or PHP, PostgreSQL could form the SQL base of a far more intricate and flexable framework that encompases a lot of the various features that could provide application sever features from PostgreSQL. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] I just got it: PostgreSQL Application Server -- a new project.
Thanks for your indepth and patient response!
My name is Carl E. McMillin and I'm still establishing my balance in this
particular knowledge domain with its nomenclature and entities.
The term App-server is very commonly used to describe
the container where the application logic resides. As such,
an app-server has access to one or several Storages. PostgreSQL
is an implementation of such a storage. The thing you describe,
a container for functionality that persists control/data state
beyond a single invocation is also a Storage.
Essentially, I agree with your assessment: App Servers should do app-stuff
and Storage Servers (RDBMS engines for instance) should do storage stuff.
But Postgres isn't purely a storage solution; it is not just a place to hang
your data. Aren't stored procedures, whether SQL-based or backed by native
libraries, very much essential to application-logic performance and
portability? Ok, portability may suffer, but they do help performance!
Perhaps tiers which include extreme Postgres are not as clearly delineated
in function as a DBA or a systems engineer would like, but the extensible
nature of Postgres does lend flexibility to the developer looking to offload
complexity into the database so that the functionality is as accessible as
the data operated on.
One of my personal interests is hybridizing a strong SQL
execution-environment such as Postgres with an equally strong
process-control framework so that components which would normally be in the
middle tier are directly accessible by way of extensions. For instance,
constructs such as the following would be really useful in some
bioinformatics-related consulting I'm involved in:
SELECT * FROM get_list_of_hsp_from_blastall('ACGGATTAT', 'H_sapiens');
The function get_list_of_hsp_from_blastall takes a primer ('ACGGATTAT')
and an organism ('H_sapiens') and runs an external process called blastall
to locate high-scoring pairs where the primer aligns well with the
organism's nucleotide-sequence (its genome). This would be a relatively
trivial exercise if Postgres had a robust framework for process control -
maybe it does, I haven't gotten many responses indicating yea or nay.
Anyway, I'm on for anything in the way of enhancing this aspect of Postgres
if there is sufficient will for such in the community. The moniker under
which this development takes place I leave to better minds.
Best Regards,
Carl |};-), CarlCo, (Newbie) Computer Engineer.
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Thomas Hallgren
Sent: Saturday, June 12, 2004 1:12 PM
To: [EMAIL PROTECTED]; Carl E. McMillin; [EMAIL PROTECTED]
Subject: Re: [HACKERS] I just got it: PostgreSQL Application Server -- a new
project.
The term App-server is very commonly used to describe the container where
the application logic resides. As such, an app-server has access to one or
several Storages. PostgreSQL is an implementation of such a storage. The
thing you describe, a container for functionality that persists
control/data state beyond a single invocation is also a Storage.
It's very common that you impose a separation of concern that imposes 3 (or
more) layers (3-tier, n-tier). You have the backend tier, a middle tier, and
a client tier. PostgreSQL inherently belongs in the backend tier. An
app-server is more or less always considered to be the thingy that lives in
the middle tier.
The ability to persist the state of a session, efficient handling when
storing HTML/XML, and cluster capabilities, will make PostgreSQL an
excellent backend for many app-servers that can utilize that kind of
functionality. It will not however, make PosgreSQL an app-server in itself.
I really think that [EMAIL PROTECTED] has great ideas (B.T.W. it would be
nice to know your name) and I'd be happy to help out if this project takes
off. But some other name for it would be preferable.
Kind regards,
Thomas Hallgren
- Original Message -
From: Carl E. McMillin [EMAIL PROTECTED]
To: 'Thomas Hallgren' [EMAIL PROTECTED];
[EMAIL PROTECTED]
Sent: Saturday, June 12, 2004 20:24
Subject: RE: [HACKERS] I just got it: PostgreSQL Application Server -- a new
project.
If you consider an app server as a container for functionality that persists
data/control state beyond a single invocation, PostgreSQL (and lots of other
DP solutions, of course) falls into the category already, ne?
I suppose my def. is too gross, but I agree with [EMAIL PROTECTED]'s
conjecture that Postgres COULD provide externalization hooks so that the
SQL engine could integrate external services/data-sources without
recompiling the backend, postmaster, or any other kind of major (and
dangerous) restructuring.
I believe that some of the contribs (like PL/Perl) have functionality for
controlling external processes from stored-procedures, but this kind of
functionality should be in the kernel of the engine, in my mind.
If I'm totally offbase, plz correct.
Carl
Re: [HACKERS] I just got it: PostgreSQL Application Server -- a new project.
...That's one of the reasons I wrote Pl/Java.
More power too you! I'd really like to hear more about this project. Is
http://gborg.postgresql.org/project/pljava/projdisplay.php
your URL?
In essence, I don't think we disagree on anything.
The only thing I'm reacting to is the term app-server which I
think is badly chosen. Stored procedures and functions doesn't
make a database an app-server, no matter what you put in them.
I'm now in complete agreement: app-server doesn't fit. Do you have any
suggestions? Would a postgreslet be out of bounds, do you think?
You can write your own functions in C and thereby get
all the process control you want. Or if you want to make life
easier and get a more portable solution (by my standards that is)
why not use Java?
I admit my almost complete ignorance of how sensitive the postgres backend
is to all the hazards of process-control: is the postgres process REALLY
just another UNIX process? Can I exec on top of it? Can I fork? Can I
have a child-process using IPC wait for 10 mins for its connected process do
its work without hosing the postmaster with its shared memory locks and all
that? I've held off any serious development along these lines since I don't
have the time to do heavy code-trawling, that seeming the only way of
obtaining the level of detail necessary to do the job well.
I would most definitely use embedded java if it could do at-minimum SRF's
and spawn processes. Something similar to SPI for Java would be pretty
useful too, I imagine.
Best Regards,
Carl |};-)
-Original Message-
From: Thomas Hallgren [mailto:[EMAIL PROTECTED]
Sent: Saturday, June 12, 2004 4:24 PM
To: Carl E. McMillin
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; Bob; 'Bill Martin';
'Joe Burks'; [EMAIL PROTECTED]
Subject: Re: [HACKERS] I just got it: PostgreSQL Application Server -- a new
project.
Carl E. McMillin [EMAIL PROTECTED] writes:
My name is Carl E. McMillin and I'm still establishing my balance in
this particular knowledge domain with its nomenclature and entities.
Ok, I was thinking more the name behind [EMAIL PROTECTED] ;-)
But Postgres isn't purely a storage solution; it is not just a place
to
hang
your data. Aren't stored procedures, whether SQL-based or backed by
native
libraries, very much essential to application-logic performance and
portability? Ok, portability may suffer, but they do help
performance!
I agree. Some app logic is best performed in the backend. Especially if the
logic is storage intensitive or deals with advanced storage
constraints/rules. That's one of the reasons I wrote Pl/Java. In essence, I
don't think we disagree on anything. The only thing I'm reacting to is the
term app-server which I think is badly chosen. Stored procedures and
functions doesn't make a database an app-server, no matter what you put in
them.
One of my personal interests is hybridizing a strong SQL
execution-environment such as Postgres with an equally strong
process-control framework so that components which would normally be
in
the
middle tier are directly accessible by way of extensions. For
instance,
constructs such as the following would be really useful in some
bioinformatics-related consulting I'm involved in:
SELECT * FROM get_list_of_hsp_from_blastall('ACGGATTAT', 'H_sapiens');
The function get_list_of_hsp_from_blastall takes a primer
('ACGGATTAT') and an organism ('H_sapiens') and runs an external
process called
blastall
to locate high-scoring pairs where the primer aligns well with the
organism's nucleotide-sequence (its genome). This would be a
relatively trivial exercise if Postgres had a robust framework for
process control - maybe it does, I haven't gotten many responses
indicating yea or nay.
You can write your own functions in C and thereby get all the process
control you want. Or if you want to make life easier and get a more portable
solution (by my standards that is) why not use Java?
Kind regards,
Thomas Hallgren
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Hacking postgres backend process
...Basically if you are in constant fear you are in the right shift of mind to do it ... check every return code, make sure you don't write unassigned memory, make sure the function wears its mithril shirt at all times, etc. Hehe! Thanks for the warning. Do you know of anyone that's managed to successfully work these control-structures in with the C api? I've heard some good words apropos PL/Perl to control external processes, but I've also heard there are notable limitations (say absence) with set-returning functions in PL/Perl (tho perhaps under construction). Carl |};-) -Original Message- From: Alvaro Herrera [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 04, 2004 6:29 AM To: Carl E. McMillin Cc: [EMAIL PROTECTED]; Bob Subject: Re: [HACKERS] Hacking postgres backend process On Wed, Apr 28, 2004 at 08:26:09AM -0700, Carl E. McMillin wrote: I posted this subject on General discussion-list but got no takers. I'll restate my query and be as brief as I possible. What are the issues/dangers involved in putting an external process-execution call in instance of main postgres-backend thread of execution? I'm not sure of all the issues it has, but as you probably already know, a C function has access to anything inside the server process. This means it can corrupt private structures, look memory and data bypassing privileges, etc; and if you get an uncaught SIGSEGV the backend will die and the postmaster will terminate all running backends. Basically if you are in constant fear you are in the right shift of mind to do it ... check every return code, make sure you don't write unassigned memory, make sure the function wears its mithril shirt at all times, etc. -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) If it wasn't for my companion, I believe I'd be having the time of my life (John Dunbar) ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] Hacking postgres backend process
Title: Message Hi All, I posted this subject on General discussion-list but got no takers. I'll restate my query and be asbriefas I possible. "What are the issues/dangers involvedin putting an external process-execution call in instance of main postgres-backend thread of execution?" The operating context will be a Linux/UNIX OS. Here is a typical SQL statement I'm trying to field: "SELECT * FROM f(a)." Where "f" is a stored-procedure stub to a shared library C function, "a" is astring-parameter. "f" will need to - under the proper circumstances - call an external process "p", parse the process-output, and return a set of structured records. "p" may run for a very long time; may cause SIG_*; may leave heap in an inconsistent state; may spawn child-processes. I've already written a number ofstored-procedures backed by shared libraries implemented in C,including set-returning functions, and I know the basics of user-types and arrays (including some custom array extensions). I've written UNIX shell processes in C while in school, so I know a bit about child-process control and signal-handling. It seems that "fork" is clearly out; I'm assuming process execution environment MUST be guaranteed consistent on re-entrance into postgres.Using"exec" is possibly worse with a full image-overlay destroying any hope of reconstructing pre-spawn environment. What are my options here? Thanks for any input, Carl |};-)
