On Fri, Sep 12, 2008 at 04:48:43AM -0700, sylver_b wrote: > Hi, > > When i run the COPY command from a batch script and after few minutes of > processing (2,3minutes), and few thousand lines inserted in the DB, im > getting : > > Connection terminated > > > merovingian log is showing: > > MSG demo[5084]: # MonetDB server v5.6.0, based on kernel v1.24.0 > MSG demo[5084]: # Serving database 'demo' > MSG demo[5084]: # Compiled for i686-pc-linux-gnu/32bit with 32bit OIDs > dynamically linked > MSG demo[5084]: # Copyright (c) 1993-2008 CWI, all rights reserved > MSG demo[5084]: # Visit http://monetdb.cwi.nl/ for further information > MSG demo[5084]: # Listening for connection requests on > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: proxying client 127.0.0.1:41420 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41420 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41422 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41422 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41424 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41424 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41426 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41426 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41428 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41428 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41430 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41430 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41432 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41434 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41432 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41436 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41434 disconnected from proxy > MSG merovingian[3981]: client has 127.0.0.1:41436 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41438 for database 'demo' to > mapi:monetdb://ulys:50001/ > MSG merovingian[3981]: client has 127.0.0.1:41438 disconnected from proxy > MSG merovingian[3981]: database 'demo' already running since 2008-09-12 > 11:44:44, up min/avg/max: 30/37441/213742, crash average: 0.00 0.10 0.03 > (8-6=1) > MSG merovingian[3981]: proxying client 127.0.0.1:41440 for database 'demo' to > mapi:monetdb://ulys:50001/ > TME 2008-09-12 11:56:46 > MSG merovingian[3981]: client has 127.0.0.1:41440 disconnected from proxy > > > This is what my script does : > > #!/bin/sh > > FILE=$1 > awk -F "\"*;\"*" '{print > ";",$1,";",$2,";",$3,";",$4,";",$5,";",$6,";",$8,";",$9,";",$10,";",$13,";",$15,";",$18,";",$22,";",$24,";",$25,";",$26,";",$27,";",$28,";",$29,";",$30,";",$31,";",$32,";",$33,";",$35,";",$36,";",$38,";",$39,";",$42,";",$47,";",$48,";",$49,";",$53,";",$54,";",$55,";","FILE"}' > $1 | sed -e "s/ ; /;/g; s/; /;/g" | mclient -lsql -d demo -i -s "COPY INTO > cdrwholesaler FROM STDIN USING DELIMITERS ';' NULL AS ''" > > and this is how i call it : > > find /usr/ftp -name '*.CDR' -exec > /var/www/consolecdr/batch/test_monetdb/awk_csv_generator.sh {} \; > > > Is there anything im doing wrong ? I have the feeling there must be too many > connections crashing the system .. coz after that i can't use mclient > anymore, i have to restart the DB.
How many (sequential, not concurrent, right?) connections are we talking about, i.e., how many *.CDR files do you have? How many rows do these *.CDR files hold in total? Are all attributes of the cdrwholesaler table string attributes? ... "too many connections" *might* be the problem (though it should of course not) --- strange, though, that the merovingian log do not give any indication of whether, and if so why the server might have crashed ... Stefan > Please advise, > Thanks > SB > > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-developers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- | Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 | ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Monetdb-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-developers
