Hello, I'm a little new at this so please bear with me.
I am trying to create a function that loads 100M test records into a
database, however I am having a hard time building the function that
does so.
I'm trying to do this in PGAdmin III for Ubuntu. Is there something
that I have wrong w
On 05/12/2010 01:32 PM, Josh wrote:
Hello, I'm a little new at this so please bear with me.
I am trying to create a function that loads 100M test records into a
database, however I am having a hard time building the function that
does so.
I'm trying to do this in PGAdmin III
ecreating the
table rather than deleting rows). Most of the dependent tables have ON
DELETE CASCADE. The 'unique_records' table is a temp table I got via
something like: SELECT DISTINCT (other_column) id INTO unique_records
FROM records
Thanks very much!
Josh Leder
--
Sent via pgsql-s
Many of the tables do not have indexes on the FK, though a couple of
the biggest ones do. It does seem worth the time to put an index on
each of these tables, considering the few hundred hours I'm already
spending on the DELETE.
I've started the EXPLAIN ANALYZE but it will take a while, no doubt.
I want to limit the number of records returned by an inner join.
Suppose I have a table of Books:
book_id
title
And, a table of authors:
book_id
author_name
Now, suppose I want to get book + author, but I only want one author for
books with multiple authors. Traditionally, I'd do something
ing the program in
December, but you probably can't wait that long.
I'd be interested to hear from anyone who's written a web search
engine. It seems to me that the algorithm for, say, google should be
fairly similar to what I'm doing for HR.
Sr. Siquiera,
> Where can I find a tutorial on PL/SQL for postgres? Is there any
> documentation focused on it on postgres' site?
Try: http://www.postgresql.org/users-lounge/docs/v7.0/postgres/c4091.htm
-Josh
--
__AGL
Mr. Huang,
Seems to me that your GROUP BY line should read:
GROUP BY ltb.v_id, vtb.equip_attr[1], vtb.equip_attr[3],
vtb.equip_attr[4]
Or am I missing the point?
-Josh
> SELECT ltb.v_id,
>count(ltb.v_id) AS num_of_times_bo
Function to create a temporary table or view and
summarizing from that.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (
table
2. a. Drop the Foriegn Key constraint
b. Update both the routes and hosts tables
c. Re-establish the foriegn key constraint
If either of these approaches doesn't work, you have a valid bug
report. COngratulations!
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS
uy" policy, in which
> case all of my work is for naught! ARGH!!
Well, if they don't use it, you can easily re-create your work at home
and GPL it. It also depends on the contract you signed ...
-Josh
--
__AGLIO DATABASE SOLUTIONS___
wed in subselects?
> >
>
> It is a very very sad fact, but, no, they're not.
H ... can't say as I've ever seen an ORDER BY in a subselect before.
Why would you want one?
And if you do want one, Louis-David, you can always use a temporary
table as previously describe
le in MSSQL
rather than using the syntac you're suggesting. I've found
that the order that the MSSQL chooses to execute query
segments in can cause some unpredicatble results ... )
-Josh Berkus
e just use modulo
instead?
Thanks for any advice!
-Josh Berkus
nt-server connection open after
data has stopped transmitting. The last place I'd look
would be the linux server; my experience is that Linux is
worlds better at managing connections than Win32 or cheap
Ethernet switches.
Good Luck!
-Josh
_part('day',haveadate),
date_part('year',haveadate) from test_date
haveadate
2000-04-30 4 30 2000
No problem here. Or on PG-ACCESS.
The problem must be in the OS/2 compile, probably some problem in
accessing the internal clock?
customer-transaction database could easily exceed 2 billion objects
created (and destroyed). Are there plans to expand this to Int8?
-Josh Berkus
P.S. My aplolgies if I've already posted these questions; I never
received them back from the list mailer.
g
aliasing) would be cosmetic.
-Josh
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (415) 436
egant, and using serials instead of the OID not possible.
SO I'm concerned about the problems you mentioned above. pg_dump has a
-o option; are there problems with this? And how liekly are counter
overflow problems?
Josh Berkus
--
__AGLIO D
ael for the
suggestion; I'll use it and send you all back notes on how
it affects performance.
-Josh
Folks,
These JDBC issues belong on the pgsql-interfaces list, where
you'll find a community of JDBC bug-finders.
Have fun!
-Josh Berkus
. Is that an accurate summary assessment?
-Josh berkus
Folks,
Is the TEXT data type automatically a BLOB (or TLOB?), or
does it only become so if huge amounts of text are saved to
the TEXT field?
-Josh Berkus
supports UNION; the relational calculus (I'm told) is
impossible otherwise.
So ... we keep hearing about all the fantastic fixes in 7.1.
When will a stable build show up? :-)
-Josh
tart getting re-used regardless if they are
already
present, then, like Tom says, it's Ragnarok. But it
seems
like somebody would have increased the OID to INT8 if
that
were a prospect.
-Josh Berkus
P.S. Bruce, I'm sorry about not sending my comments on
your
book. Do
> > OIDs as a primary index. None of my OIDs still in use
> will
> > be touched.
>
>
> No, it uses all oids, and can create duplicates.
Does this mean that Tom's "Wraparound Ragnarok" is the
accurate scenario?
-Josh Berkus
Tom, Bruce,
Thanks. I think that gives me a pretty clear picture. How can we
submit this whole OID thing to the PGSQL FAQ? Want me to write it up?
-Josh Berkus
P.S. BTW, my conclusion based on this discussion is that I will not use
the OIDs
Mr. Vadsholt,
This is a Postgre-SQL list. If you are using Microsoft SQL Server you
should proceed to http://msdn.microsoft.com/
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Please see:
... H. The online docs appear to be down. When they're back up,
please check the sections on: Date/Time data types, and Date/Time
manipulation functions.
-Josh Berkus
P.S. Brian, a general tutorial on writing SQL, such as O'Reilly
Franz,
You'd better stay away from that syntax if you want to make your
applications portable. I can tell you that it won't work on MS SQL
Server or MySQL. I can't speak for Oracle.
-Josh Berkus
P.S. Thanks for the nifty construction ... I wouldn't have thought of
2. If 1. is "no", is there a way to do the above without programming the
type in C?
3. What sort of trouble am I going to get into trying to pull data from
a custom type into an external interface (i.e. PHP4)?
Thanks for your thoughts!
-Josh Be
UPDATE, because it
will space me having to remember to use the function every time I handle
a phone number field. I'll post the PLSQL function after I've written
it.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Be
d phone number for Primary Phone'
RETURN output_string
)
More code
-Josh
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solut
dded in data entry.
Soon.
-Josh
P.S. this makes you the first outside contributor to my open-source
project ... which isn't up on the web yet!
--
__AGLIO DATABASE SOLUTIONS___
jian's excellent PGSQL
book-in-progress, available on the Postgresql.org website. (Chapter 7, I
believe).
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete in
Mr. May,
For discussions of JDBC, please subscribe to the pgsql-interfaces
list. You will find many JDBC users on that list.
-Josh Berkus
P.S. PGSQL folks, is there any way we can clarify this on the web page?
The JDBC users seem to keep ending
Eduardo,
Use the curval(serial) function. For more information, look in the
online docs under the name of that function. nextval(), curval() are
wonderful things!
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
eff or Jan Wieck.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (415) 436-9166
for
ed columns must either
contain and aggregate function (e.g. SUM) or be named in the GROUP BY
clause.
O'Reilly has just come out with a SQL in a Nutshell book. I'd suggest
picking one up.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
ery, possibly using a PL/PGSQL or C function,
so that it works for any number of node levels.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data manage
Mr. May,
The reason you're having trouble is that the problem you've described
does not call for a union query at all. What you want is a simple GROUP
BY query:
SELECT Node_ID, Word, Count(*)
FROM NodeIndex
GROUP BY Node_ID, Word
-Josh Berkus
--
__AGLI
o run.
I've fooled around with drectional joins, views, and temporary tables,
but I can seem to find anything that works faster. Suggestions?
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information
A *lot* faster. Like, 7x as fast. I'd forgotten about EXISTS, since I
so seldom have a use for it ... but this is shy it was created, I guess.
Thanks so much for your help!
-Josh Berkus
--
__AGLIO DATABASE S
erver ... I run two of the damn machines, and they're nothing but
grief.
-Josh
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
in fact, very fast
... just awkward to code and manipulate.
-Josh
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data man
xecution tests
between the two query structures.
Fortunately, this will all soon become moot; Tom says that outer joins
have been stable in the 7.1 build for a while. Speaking of which,
when's the 7.1 "release"? Huh, huh?
IN" on *either* version. Instead, use "NOT EXISTS", which is much, much
faster.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data ma
posted to our mailing list in error. This is a
PostgreSQL SQL-developers mailing list. If you need help with Oracle, I
suggest proceeding to www.oracle.com and looking for appropriate forums;
I'm sure there are many.
-Josh Berkus
--
__
r ')'" in a huge DDL
statement.
4. Use of named in addition to ordinal variables in PL/PGSQL functions
(e.g. $account_type, $period instead of $1, $2).
Thanks so much for your ongoing hard work!
-Josh Berkus
--
__AGLIO DATABASE S
b script to parse it out 25 records at a
time.
Hopefully, someone on this list will have done that before and can
provide less theoretical advice.
-Josh Berkus
P.S. I've also posted this to the pgsql-php list. I;ve quoted the full
text o
I'm intrigued by this. How would I retrieve cursor rows into a web
application? If we could output a cursor to a functon result (we
can't), it would be easy, but I'm not sure otherwise.
-Josh
--
__A
writer about
dressing those up into an educational "chapter". It'll cost
me a little more than $0.05, but is only my fair
contribution. Look for something in february-march.
-Josh Berkus
at
your disposal. Post the text, I'll help clean it up!
-Josh
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (415) 436-9166
fo
Frank,
Please look in the list archives. About 2 months ago this topic came
up and was discussed extensively (including a creative solution by yours
truly).
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
ing to itself on
(hopefully) indexed fields, in my expreience it runs very, very fast.
4. My PHP developer has reprogrammed the easily available PHP Tree
Control to uses this table structure (I don't know if he's giving it
out, but he said it wasn't very difficult).
-Josh Berk
.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (415) 436-9166
for law firms
dirty.
Great. Maybe I'll buy it from you if I ever need to use Java :-)
-Josh
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management s
lly as straightforward
> as one might
> hope. Is there an easier way?
Hmmm. I don't know, Frank. That strikes me as a really
good, straightforward workaround to your problem. I'm not
sure what you could do that would be simpler. This is
practically a textbook example of why triggers are necessary
to retain relational integrity.
-Josh Berkus
string, it's a datetime field.
WHy are you trying to substring a datetime field, anyway?
-Josh
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology
se
> for invoice numbers.
>
> - Sequences are not rollback'able.
> - It seems overkill to have a table just for this.
> - What else?
Given a full set of business rules for invoice numbers, I
could probably throw you a solution. Gods know that I've
devised enough invoicing systems in the past. Please post
more detail.
-Josh Berkus
PostgreSQL, you are out of luck at
this time. However, any number of institutions may offer it soon, and
O'Reilly has a new book pending that covers many SQL databases ("SQL in
a Nutshell").
-Josh
--
__AGLIO DATABASE SOLUTIONS
rom it too for more exotic tasks.
Also, pgAccess, a tcl/tk GUI, is excellent although incomplete. I end
up using a combination of pgAccess, KpgSQL, and command line access.
-Josh Berkus
--
__AGLIO DATABASE SOLU
e, please reply and I'll give you a pointer.
Othervwise, RTFM.
-Josh Berkus
?
For example, if I want to locate the NEXTVAL. do a bunch of
stuff, and then insert the record, does this obligate me to
take the DEFAULT NEXTVAL out of the table and do it the
NEXTVAL insertion in my functions, or is it irrelevant?
-Josh Berkus
mples?
Anybody (Jan?) who can shed some light on the above will
receive my enthusiastic gratitude in ASCII text.
-Josh Berkus
P.S. I'm using Postgres 7.0.2 on SuSE 7.0 and use pgaccess
extensively for function editing.
P.P.S. My most heartfelt gratitude to Jan Wieck for writing
some decen
Folks,
Oh, yes, one more:
4. If I pass a NULL to any of the parameters of a PL/PGSQL
function, any (other) VARCHAR parameters are set to NULL as
well.
Thanks!
-Josh
still has a "not advisable for
production" warning, and I'm being tied up in knots by the
number of things I need in 7.1.
Would it help if I sent more money? :-)
-Josh
Folks,
ALTER TABLE won't work until 7.1. CUrrently, I have a table
that needs one small change, but it's refrenced as a foriegn
key by 7 other tables. Any suggestions on how I can make
the table change without having to drop and re-create 8
tables?
-Josh Berkus
unded a little harsh. I am a bit
desperate, but that's hardly your fault. Thank you so much
for all of your hard work as our "database engine" guru, and
for staying up late to answer our questions!
-Josh
en sent to the client and emitting
> NOTICE's then could confuse the fe/be protocol.
Hey, just the fact that you spit back "Error on Line 38" cuts my
debugging time in half over the SQL handler's "Error at or near ';'"
Of course, running a tail on
: Why do you want a Bit type? What
purpose does it serve that INT2 and BOOLEAN do not?
I'm not being sarcastic -- this is an important question to
ask before creating *any* custom type on any RDBMS.
-Josh Berkus
re a comprehensive list of built-in functions.
You know, stuff like CURRVAL() and NOW(). Can anyone point
me to such a list? Guessing parameters is getting
frustrating!
-Josh
ly; thus their
implementation is entirely proprietary to the RDBMS. The
solution is not to use BLOBs.
> Distributing schema patches is proving troublesome across
> multiple
> platforms.
Yup. Yer in for a world of pain, sonny. Hope you get paid
hourly.
-Josh
Josh Berkus wrote:
> Well, yes. This is beacause BLOBs are NOT part of the SQL
> standard and IMHO a bad idea relationally; thus their
> implementation is entirely proprietary to the RDBMS. The
> solution is not to use BLOBs.
Ooops. Let me re-state: This is because the *implem
27;d also need to see if our functions survive the
dump ...
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (415) 436-9166
...
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (415) 436-9166
for law firms, small businesses
.id = t2.id
Although this is a subselect in the FROM clause, something
we've had trouble with as I recall.
-Josh Berkus
Folks-
Answered my own question about the backslashes before
spaces, in text returned as results from functions: it's a
bug in kpsql, one of the interface tools I was using.
Somebody might want to forward this to the Interfaces list.
-Josh Berkus
Najm
CURRVAL('sequence_name')
For this and other sequence and serial functions, please see the online
version of Bruce's book.
BTW, O'Reilly's "SQL in at Nutshell" also catalogs all PGSQL functions
from ver. 6.5.
-:).
> > Regards, Najm
>
> select nextval('nameofseq');
Sorry, no. You want SELECT CURRVAL('sequence_name'). NEXTVAL select
the NEXT value, and increments the sequence in the process.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS____
;t even *have* your book yet, it's better than nothing! :-)
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
Folks,
Fascinating as this thread is, is the SQL Developers list really the
appropriate place for it? Don't we have a Policy list or something?
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS__
)
> ERROR: referential integrity violation - key
> referenced from
> users not found in age_list
Simple ... you have values in the AGE column that are not in
the age_list table. Thus you're in violation of the foriegn
key you're trying to establish.
-Josh Berkus
databases gone bad,
composite data types have not earned a warm place in my
heart ...
-Josh Berkus
sion key.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (415) 436-9166
for law firm
tionally
compliant is not achievable.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions (415) 436-9166
for law firms, small bu
e my code easier to read
and maintain. And, of course, you may have already implemented one or
the other in 7.1 (which I have not yet got to run on an alternate port).
Thanks for your hard work and consideration towards us users.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIO
e 'plpgsql';
> CREATE
Hmmm? When I've tried creating similar functions, I got from the
compiler:
Error at or near 'END'
I'll try your code above as a test, then try re-modifying some of my own
functions.
-Josh Berkus
--
______A
eks :-) Is this the
> appropriate list, or
> should I move over to hackers?
You should probably cross-post. This list is the place to
see if a number of other developers are interested in the
functionality you propose (yes), hackers is probably the
place to ask how to make the actual changes.
I can't help. Heck, I can't even get 7.1 beta to run on an
alternate port.
-Josh Berkus
P.S. BTW, John, I'm thrilled to get a discussion of issues,
going here in addition to the how-tos!
re there plans to improve this, or has somebody written a script that
handles the steps involved?
4. I'm not trying 7.1 beta 3. I noticed that for this version,
Theodescu's PGAccess lists all builtin functions along with the
user-defined functions in the functions window. Anyb
ocess running in
the foreground, you see all sorts of useful debugging info
as execution takes place. Not quite as complete, but a
*lot* faster than checking the log manually.
-Josh
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete in
Tom,
What's the syntax for directional joins in 7.1 beta?
Thanks!
-Josh Berkus
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions
Tom,
Sorry! Never mind, I found it in the Development Docs.
Grazie!
-Josh
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and
;
And I get:
ERROR: JOIN/ON CLAUSE REFERS TO 'c' WHICH IS NOT PART OF
JOIN.
What's wrong here?
-Josh Berkus
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED
an, please tall me if I'm wrong in this; I could really
use the functionality if it *is* possible.)
You could use some creative manipulation of the system
tables to achieve the same result, however.
-Josh Berkus
__AGLIO DATABASE SOLUTIONS___
or something similar (it may be EXECUTE), which uses the dynamic sql
> elements of plpgsql. I think.
Is this true, Jan? Does anyone have more specific documentation?
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
luenced by whatever you guys are planning
to do about Stored Procedures.
-Josh Berkus
--
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Complete information technology [EMAIL PROTECTED]
and data management solutions
TE TRUSTED PROCEDURAL LANGUAGE 'plpgsql' HANDLER
"plpgsql_call_handler" LANCOMPILER 'PL/pgSQL';
-Josh Berkus
(Instructions courtesy of Jeff at PGSQL Inc.)
--
__AGLIO DATABASE SOLUTIONS___
ault for keying off a non-unique value?
And, if the latter, is there a way I can construct a foreign
key constraint that keys onto a view or query?
Grazie!
-Josh Berkus
__AGLIO DATABASE SOLUTIONS___
Josh Berkus
Comple
1 - 100 of 864 matches
Mail list logo