date:20220331

Re: [sqlalchemy] Re: create database name lowcase ?

2022-03-31 Thread Massimiliano della Rovere

Postgresql is case case insensitive unless you impose a specific casing
using " (double quotes): they are valid everywhere postgresql expects an
identifier (schema name, table name, column name, cte name, after AS, etc.

So it's all correct.
See
https://docs.sqlalchemy.org/en/14/core/sqlelement.html#sqlalchemy.sql.expression.quoted_name



Il ven 1 apr 2022, 04:01 oislone  ha scritto:

> I see.
> I didn't expect it to be so simple.
> I tried to debug sqlalchemy, but didn't find character conversion
> It turned out to be the default way of postgreql.
> Thanks for your instruction.
> 'Jonathan Vanasco' via sqlalchemy 於 2022/4/1 04:47 寫道:
>
> I'm not aware of any recent changes in the libraries that would cause that
> behavior.
>
> It may be how you are using the libraries or raw sql.
>
> PostgreSQL will convert database names to lowercase UNLESS the database
> name is in quotes.
>
> These will all create `abc`:
>
> CREATE DATABASE abc;
> CREATE DATABASE Abc;
> CREATE DATABASE ABc;
> CREATE DATABASE ABC;
> CREATE DATABASE aBc;
> CREATE DATABASE aBC;
> CREATE DATABASE abC;
>
> These will create two different databases:
>
> CREATE DATABASE "abc";
> CREATE DATABASE "Abc";
> CREATE DATABASE "ABc";
> CREATE DATABASE "ABC";
> .. etc..
>
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sqlalchemy/89e4aa92-c9c2-767d-6caf-dc0b77477a2a%40gmail.com
> 
> .
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/CADKhPGTYxvQRrXeYx%2BqvQEAiSByeanHaVJFgzofEOkdhASUvLg%40mail.gmail.com.

Re: [sqlalchemy] Re: create database name lowcase ?

2022-03-31 Thread oislone


I see.
I didn't expect it to be so simple.
I tried to debug sqlalchemy, but didn't find character conversion
It turned out to be the default way of postgreql.
Thanks for your instruction.

'Jonathan Vanasco' via sqlalchemy 於 2022/4/1 04:47 寫道:
I'm not aware of any recent changes in the libraries that would cause 
that behavior.


It may be how you are using the libraries or raw sql.

PostgreSQL will convert database names to lowercase UNLESS the 
database name is in quotes.


These will all create `abc`:

    CREATE DATABASE abc;
    CREATE DATABASE Abc;
    CREATE DATABASE ABc;
    CREATE DATABASE ABC;
    CREATE DATABASE aBc;
    CREATE DATABASE aBC;
    CREATE DATABASE abC;

These will create two different databases:

    CREATE DATABASE "abc";
    CREATE DATABASE "Abc";
    CREATE DATABASE "ABc";
    CREATE DATABASE "ABC";
    .. etc..




--
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper


http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/89e4aa92-c9c2-767d-6caf-dc0b77477a2a%40gmail.com.

[sqlalchemy] Re: create database name lowcase ?

2022-03-31 Thread 'Jonathan Vanasco' via sqlalchemy

I'm not aware of any recent changes in the libraries that would cause that 
behavior.

It may be how you are using the libraries or raw sql.

PostgreSQL will convert database names to lowercase UNLESS the database 
name is in quotes.

These will all create `abc`:

CREATE DATABASE abc;
CREATE DATABASE Abc;
CREATE DATABASE ABc;
CREATE DATABASE ABC;
CREATE DATABASE aBc;
CREATE DATABASE aBC;
CREATE DATABASE abC;

These will create two different databases:

CREATE DATABASE "abc";
CREATE DATABASE "Abc";
CREATE DATABASE "ABc";
CREATE DATABASE "ABC";
.. etc.. 


On Thursday, March 31, 2022 at 2:39:32 PM UTC-4 ois...@gmail.com wrote:

> Hi everyone, I have a question
>
> I use Postgresql
> Before creating a database, the name is uppercase and lowercase, and there 
> is no problem.
>
> Later SQLAlchemy was updated to version 1.4
> Don't know when the version started,
> When creating a database again, use uppercase and lowercase names, which 
> will always be lowercase database names.
> As a result, using drop database will fail.
>
> I am currently using:
> Arch-linux
> postgresql  V13.6-1
> sqlalcgemy V1.4.33
> pyscopg2V2.93
> dictalchemy3 V1.0.0
>
> E.g :
> engine = sqlalchemy.create_engine(
> "postgresql://xxx:yyy@localhost/postgres"
> )
> conn = engine.connect()
> conn.execute( "commit" )
> stt = "CREATE DATABASE ABCDEF"
> conn.execute(stt)
> conn.close()
>
> ===
> The database name will become abcdef
>
> I'm not sure if this is the reason for sqlalchemy or pyscopg2 ?
>
> Thank you everyone.
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/25fa8667-d4dd-43f8-a137-0c6a9125ccbbn%40googlegroups.com.

[sqlalchemy] create database name lowcase ?

2022-03-31 Thread lone ois

Hi everyone, I have a question

I use Postgresql
Before creating a database, the name is uppercase and lowercase, and there 
is no problem.

Later SQLAlchemy was updated to version 1.4
Don't know when the version started,
When creating a database again, use uppercase and lowercase names, which 
will always be lowercase database names.
As a result, using drop database will fail.

I am currently using:
Arch-linux
postgresql  V13.6-1
sqlalcgemy V1.4.33
pyscopg2V2.93
dictalchemy3 V1.0.0

E.g :
engine = sqlalchemy.create_engine(
"postgresql://xxx:yyy@localhost/postgres"
)
conn = engine.connect()
conn.execute( "commit" )
stt = "CREATE DATABASE ABCDEF"
conn.execute(stt)
conn.close()

===
The database name will become abcdef

I'm not sure if this is the reason for sqlalchemy or pyscopg2 ?

Thank you everyone.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/ca3b5777-bab9-469e-ad92-95bdda3c55c3n%40googlegroups.com.

Re: [sqlalchemy] SQLAlchemy + multiprocessing problems

2022-03-31 Thread Evgenii

Works! Thank you!

четверг, 31 марта 2022 г. в 15:42:25 UTC+3, Mike Bayer: 

> when using multiprocessing, the connection pool in the new process must be 
> replaced with a new one. This is usually accomplished by calling 
> engine.dispose().  However, to maintain the pool in the parent process as 
> well, replace the connection pool alone without disposing the old one:
>
> engine = create_engine(...)
>
> Session = sessionmaker(bind=engine)
>
> def job_update_rd(data_list):
> updated = []
> with Session() as session:
> for t, ts in data_list:
> rd = session.query(RawDataTable).filter(and_(
> RawDataTable.timestamp == t,
> RawDataTable.ts == ts)).one()
>
> rd.ts = updated_ts[ts]
> session.commit()
>
> updated.append(rd)
>
> return updated
>
>
> def initializer():
> engine.pool = engine.pool.recreate()
>
> with Pool(10, initializer=initializer) as p:
> upd_list = p.map(job_update_rd, chunks)
>
>
> For many years we've advised calling engine.dispose() here as documented 
> at 
> https://docs.sqlalchemy.org/en/14/core/pooling.html#using-connection-pools-with-multiprocessing-or-os-fork
>  
> .It was recently pointed out that this closes out the parent process' 
> connections, so in SQLAlchemy 1.4.33 there will be a parameter so you can 
> change the above code to engine.dispose(close=False).   
>
>
>
> On Thu, Mar 31, 2022, at 7:06 AM, Evgenii wrote:
>
>
> Hello!
> From time to time, I need to update data in tables and multiprocessing can
> speed up this process. Last example: I’m trying to update data 7M rows in 
> table
>
>
> SQLAlchemy 1.4.31, psycopg2 2.8.6, PostgreSQL
>
> *def* *job_update_rd*(data_list):
> updated = []
> *with* Session() *as* session:
> *for* t, ts *in* data_list:
> rd = session.query(RawDataTable).filter(and_(
> RawDataTable.timestamp == t, 
> RawDataTable.ts == ts)).one()
>
> rd.ts = updated_ts[ts]
> session.commit()
>
> updated.append(rd)
>
> *return* updated
> *with* Pool(10) *as* p:
> upd_list = p.map(job_update_rd, chunks)
>
> Code is very simple, but it does not work. I get these errors randomly:
>
>- psycopg2.OperationalError: SSL error: sslv3 alert bad record mac
>- sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL 
>SYSCALL error: EOF detected
>
> But this example works fine:
>
> *def* *other_job*(data_list):
> *with* Session() *as* s:
>  *return* [s.query(RawDataTable).filter(and_(
>RawDataTable.timestamp == t, 
>RawDataTable.ts == ts)).all() *for* t, ts *in* data_list]
> *with* Pool(10) *as* p:
> res = p.map(other_job, chunks)
>
>
> Please, help to solve this problem.
> Some people is our team also uses multiprocessing, and 1 time a week get 
> these errors.
>
>
>
>
> -- 
> SQLAlchemy - 
> The Python SQL Toolkit and Object Relational Mapper
>  
> http://www.sqlalchemy.org/
>  
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> --- 
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/4d61a93d-f83f-455c-ab2a-1cb28e154af8n%40googlegroups.com
>  
> 
> .
>
>
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/26fb15e2-7ab8-420f-a48a-4bcc42ddd906n%40googlegroups.com.

Re: [sqlalchemy] SQLAlchemy + multiprocessing problems

2022-03-31 Thread Mike Bayer

when using multiprocessing, the connection pool in the new process must be 
replaced with a new one. This is usually accomplished by calling 
engine.dispose().  However, to maintain the pool in the parent process as well, 
replace the connection pool alone without disposing the old one:

engine = create_engine(...)

Session = sessionmaker(bind=engine)

def job_update_rd(data_list):
updated = []
with Session() as session:
for t, ts in data_list:
rd = session.query(RawDataTable).filter(and_(
RawDataTable.timestamp == t,
RawDataTable.ts == ts)).one()

rd.ts = updated_ts[ts]
session.commit()

updated.append(rd)

return updated


def initializer():
engine.pool = engine.pool.recreate()

with Pool(10, initializer=initializer) as p:
upd_list = p.map(job_update_rd, chunks)


For many years we've advised calling engine.dispose() here as documented at 
https://docs.sqlalchemy.org/en/14/core/pooling.html#using-connection-pools-with-multiprocessing-or-os-fork
 .It was recently pointed out that this closes out the parent process' 
connections, so in SQLAlchemy 1.4.33 there will be a parameter so you can 
change the above code to engine.dispose(close=False).   



On Thu, Mar 31, 2022, at 7:06 AM, Evgenii wrote:
> 
> 
> Hello!
> From time to time, I need to update data in tables and multiprocessing can
> speed up this process. Last example: I’m trying to update data 7M rows in 
> table
> 
> 
> `SQLAlchemy 1.4.31`, `psycopg2 2.8.6`, `PostgreSQL`
> 
> `*def* *job_update_rd*(data_list):
> updated = []
> *with* Session() *as* session:
> *for* t, ts *in* data_list:
> rd = session.query(RawDataTable).filter(and_(
> RawDataTable.timestamp == t, 
> RawDataTable.ts == ts)).one()
> 
> rd.ts = updated_ts[ts]
> session.commit()
> 
> updated.append(rd)
> 
> *return* updated
> 
> *with* Pool(10) *as* p:
> upd_list = p.map(job_update_rd, chunks)
`
> Code is very simple, but it does not work. I get these errors randomly:
> 
>  * `psycopg2.OperationalError: SSL error: sslv3 alert bad record mac`
>  * `sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL SYSCALL 
> error: EOF detected`
> But this example works fine:
> 
> `*def* *other_job*(data_list):
> *with* Session() *as* s:
>  *return* [s.query(RawDataTable).filter(and_(
>RawDataTable.timestamp == t, 
>RawDataTable.ts == ts)).all() *for* t, ts *in* data_list]
> 
> *with* Pool(10) *as* p:
> res = p.map(other_job, chunks)
`
> 
> 
> Please, help to solve this problem.
> Some people is our team also uses multiprocessing, and 1 time a week get 
> these errors.
> 
> 
> 
> 
> 
> -- 
> SQLAlchemy - 
> The Python SQL Toolkit and Object Relational Mapper
>  
> http://www.sqlalchemy.org/
>  
> To post example code, please provide an MCVE: Minimal, Complete, and 
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> description.
> --- 
> You received this message because you are subscribed to the Google Groups 
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/sqlalchemy/4d61a93d-f83f-455c-ab2a-1cb28e154af8n%40googlegroups.com
>  
> .

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/d0a631b1-8f44-448a-93a1-16c01acfaab4%40www.fastmail.com.

[sqlalchemy] SQLAlchemy + multiprocessing problems

2022-03-31 Thread Evgenii



Hello!
>From time to time, I need to update data in tables and multiprocessing can
speed up this process. Last example: I’m trying to update data 7M rows in 
table

SQLAlchemy 1.4.31, psycopg2 2.8.6, PostgreSQL

def job_update_rd(data_list):
updated = []
with Session() as session:
for t, ts in data_list:
rd = session.query(RawDataTable).filter(and_(
RawDataTable.timestamp == t, 
RawDataTable.ts == ts)).one()

rd.ts = updated_ts[ts]
session.commit()

updated.append(rd)

return updated
with Pool(10) as p:
upd_list = p.map(job_update_rd, chunks)

Code is very simple, but it does not work. I get these errors randomly:

   - psycopg2.OperationalError: SSL error: sslv3 alert bad record mac 
   - sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL 
   SYSCALL error: EOF detected 

But this example works fine:

def other_job(data_list):
with Session() as s:
 return [s.query(RawDataTable).filter(and_(
   RawDataTable.timestamp == t, 
   RawDataTable.ts == ts)).all() for t, ts in data_list]
with Pool(10) as p:
res = p.map(other_job, chunks)

Please, help to solve this problem.
Some people is our team also uses multiprocessing, and 1 time a week get 
these errors.


-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sqlalchemy/4d61a93d-f83f-455c-ab2a-1cb28e154af8n%40googlegroups.com.

Re: [sqlalchemy] Re: create database name lowcase ?

Re: [sqlalchemy] Re: create database name lowcase ?

[sqlalchemy] Re: create database name lowcase ?

[sqlalchemy] create database name lowcase ?

Re: [sqlalchemy] SQLAlchemy + multiprocessing problems

Re: [sqlalchemy] SQLAlchemy + multiprocessing problems

[sqlalchemy] SQLAlchemy + multiprocessing problems

7 matches

Site Navigation

Mail list logo

Footer information