Re: [sqlalchemy] Problem with binary field
Please use mysqlclient or pymysql, don't use mysqlclient connector. Also
please define "crash". complete stack trace please ?
On Thu, Jan 31, 2019, 4:44 PM Teymur Agayev Hi All,
>
> After upgrade on my VPS the one of the scripts started periodical crash.
> After debugging I found that the problem in the filed that stores hashes.
> for some reason the script started crash on some particular hash values.
> Below is the test script and attached is the database dump.
>
> when ID is 380059 script crashs, other ids, 380058 or 380060 for example,
> work ok.
>
> from sqlalchemy import MetaData, Column, Integer, LargeBinary
> from sqlalchemy.ext.declarative import declarative_base
> from sqlalchemy import create_engine
> from sqlalchemy.orm import sessionmaker
>
> metadata = MetaData()
> Base = declarative_base(metadata=metadata)
>
> class Test(Base):
> __tablename__ = "test"
> id = Column(Integer, primary_key=True)
> hash = Column(LargeBinary(16))
>
> engine = create_engine('mysql+mysqlconnector://root:q2w3e4r5@localhost/
> ru.feeds.az')
> Session = sessionmaker(bind=engine)
> session = Session()
>
> t = session.query(Test.hash).filter(Test.id == 380059).all()
> print(t)
>
>
> On this system doesn't work.
> Python 3.6, SQLAlchemy 1.2.17
>
> On this systems works fine.
> Python 3.5, SQLAlchemy 1.1.14
>
> Is it a bug or I just doing something wrong.
>
> Thanks,
> Teymur
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
>
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Problem with binary field
Hi All,
After upgrade on my VPS the one of the scripts started periodical crash.
After debugging I found that the problem in the filed that stores hashes.
for some reason the script started crash on some particular hash values.
Below is the test script and attached is the database dump.
when ID is 380059 script crash, other ids, 380058 or 380060 for example,
work OK.
from sqlalchemy import MetaData, Column, Integer, LargeBinary
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
metadata = MetaData()
Base = declarative_base(metadata=metadata)
class Test(Base):
__tablename__ = "test"
id = Column(Integer, primary_key=True)
hash = Column(LargeBinary(16))
engine = create_engine('')
Session = sessionmaker(bind=engine)
session = Session()
t = session.query(Test.hash).filter(Test.id == 380059).all()
print(t)
On this system doesn't work.
Python 3.6, SQLAlchemy 1.2.17
On this systems works fine.
Python 3.5, SQLAlchemy 1.1.14
Is it a bug or I just doing something wrong.
Thanks,
Teymur
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
[sqlalchemy] Problem with binary field
Hi All,
After upgrade on my VPS the one of the scripts started periodical crash.
After debugging I found that the problem in the filed that stores hashes.
for some reason the script started crash on some particular hash values.
Below is the test script and attached is the database dump.
when ID is 380059 script crashs, other ids, 380058 or 380060 for example,
work ok.
from sqlalchemy import MetaData, Column, Integer, LargeBinary
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
metadata = MetaData()
Base = declarative_base(metadata=metadata)
class Test(Base):
__tablename__ = "test"
id = Column(Integer, primary_key=True)
hash = Column(LargeBinary(16))
engine =
create_engine('mysql+mysqlconnector://root:q2w3e4r5@localhost/ru.feeds.az')
Session = sessionmaker(bind=engine)
session = Session()
t = session.query(Test.hash).filter(Test.id == 380059).all()
print(t)
On this system doesn't work.
Python 3.6, SQLAlchemy 1.2.17
On this systems works fine.
Python 3.5, SQLAlchemy 1.1.14
Is it a bug or I just doing something wrong.
Thanks,
Teymur
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
-- MySQL dump 10.13 Distrib 5.7.25, for Linux (x86_64)
--
-- Host: localhostDatabase: ru.feeds.az
-- --
-- Server version 5.7.25-0ubuntu0.18.04.2-log
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table `test`
--
DROP TABLE IF EXISTS `test`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `test` (
`id` int(10) unsigned zerofill NOT NULL AUTO_INCREMENT,
`hash` binary(16) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `url_hash_UNIQUE` (`hash`)
) ENGINE=InnoDB AUTO_INCREMENT=380241 DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `test`
--
LOCK TABLES `test` WRITE;
/*!4 ALTER TABLE `test` DISABLE KEYS */;
INSERT INTO `test` VALUES (380058,0x21B11D9B9D2168E0D35E1BE6AF55A096),(380059,0x6D3BCAA5241A62480DC8AECDA5625863),(380060,0x7E3541D9BDFE8473A0DA0EB5ACAB832E),(380057,0x9F1B213C50F645F4D7BB02CBAB021BBA);
/*!4 ALTER TABLE `test` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
-- Dump completed on 2019-02-01 0:30:14
Re: [sqlalchemy] Re: pandas to MS SQL DataWarehouse (to_sql)
No the error related to this:
sqlalchemy ProgrammingError: (pyodbc.ProgrammingError) ('42000', '[42000]
[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]111214;An attempt to
complete a transaction has failed. No corresponding transaction found.
(111214) (SQLEndTran)') (Background on this error at:
http://sqlalche.me/e/f405)
On Thu, Jan 31, 2019 at 11:02 AM Mike Bayer
wrote:
> for "dm_exec_sessions" ? that's an old SQLAlchemy bug that was fixed
> long ago. see https://github.com/sqlalchemy/sqlalchemy/issues/3994
> please upgrade.
>
> On Wed, Jan 30, 2019 at 10:52 PM wrote:
> >
> > Any solution for this?
> >
> > On Monday, September 11, 2017 at 6:34:47 PM UTC-4, dirk.biesinger wrote:
> >>
> >> I am encountering errors when trying to use the pd.to_sql function to
> write a dataframe to MS SQL Data Warehouse.
> >> The connection works when NOT using sqlalchemy engines.
> >> I can read dataframes as well as row-by-row via select statements when
> I use pyodbc connections
> >> I can write data via insert statements (as well as delete data) when
> using pyodbc.
> >> However, when I try to connect using a sqlalchemy engine I run into a
> string of error messages starting with:
> >>
> >> ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000]
> [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Catalog view
> 'dm_exec_sessions' is not supported in this version. (104385)
> (SQLExecDirectW)")
> >>
> >>
> >> I have searched online, and this exact error seems to have been
> reported / evaluated in May of this year as issue #3994:
> >>
> >>
> >>
> https://bitbucket.org/zzzeek/sqlalchemy/issues/3994/azure-sql-datawarehouse-basic
> >>
> >>
> >> I could not find a solution to this, and I'd really dislike to do a
> line-wise or blob insert statement (I'm working with multiple datasets that
> each has a few million rows, so execution time is a consideration, although
> the result sets I'm getting are more like in the 100k lines area each.)
> >>
> >>
> >> I get the same error messages even when I replace the pd.to_sql command
> with a simple engine.connect()
> >>
> >>
> >> Enclosed my installed packages (packages.list)
> >>
> >> Enclosed the full traceback (traceback.txt)
> >>
> >>
> >> This is the code I'm using:
> >>
> >> connection_string = "mssql+pyodbc://:@.
> database.windows.net
> :/?driver=ODBC+Driver+13+for+SQL+Server"
> >> engn = sqlalchemy.engine.create_engine(connection_string, echo=True)
> >> engn.connect()
> >>
> >>
> >> I'm very well aware that MS SQL DataWarehouse behaves a bit different,
> so I'm open for some experimenting to get this issue narrowed down.
> >>
> >> In case it matters: I'm running an ubuntu 16.04 VM on azure with
> jupyter notebook server and python 3.6.1.
> >>
> >> Best,
> >>
> >> DB
> >
> >
> > Confidentiality Note: This email may contain confidential and/or private
> information.
> > If you received this email in error please delete and notify sender.
> >
> > --
> > SQLAlchemy -
> > The Python SQL Toolkit and Object Relational Mapper
> >
> > http://www.sqlalchemy.org/
> >
> > To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> > ---
> > You received this message because you are subscribed to the Google
> Groups "sqlalchemy" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to sqlalchemy+unsubscr...@googlegroups.com.
> > To post to this group, send email to sqlalchemy@googlegroups.com.
> > Visit this group at https://groups.google.com/group/sqlalchemy.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
>
--
Mark Pearl
Senior Data Developer | Data & New Markets
613.722.0688
mjpe...@lixar.com
[image: Lixat IT] [image: YouTube]
Re: [sqlalchemy] Re: pandas to MS SQL DataWarehouse (to_sql)
for "dm_exec_sessions" ? that's an old SQLAlchemy bug that was fixed
long ago. see https://github.com/sqlalchemy/sqlalchemy/issues/3994
please upgrade.
On Wed, Jan 30, 2019 at 10:52 PM wrote:
>
> Any solution for this?
>
> On Monday, September 11, 2017 at 6:34:47 PM UTC-4, dirk.biesinger wrote:
>>
>> I am encountering errors when trying to use the pd.to_sql function to write
>> a dataframe to MS SQL Data Warehouse.
>> The connection works when NOT using sqlalchemy engines.
>> I can read dataframes as well as row-by-row via select statements when I use
>> pyodbc connections
>> I can write data via insert statements (as well as delete data) when using
>> pyodbc.
>> However, when I try to connect using a sqlalchemy engine I run into a string
>> of error messages starting with:
>>
>> ProgrammingError: (pyodbc.ProgrammingError) ('42000', "[42000]
>> [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Catalog view
>> 'dm_exec_sessions' is not supported in this version. (104385)
>> (SQLExecDirectW)")
>>
>>
>> I have searched online, and this exact error seems to have been reported /
>> evaluated in May of this year as issue #3994:
>>
>>
>> https://bitbucket.org/zzzeek/sqlalchemy/issues/3994/azure-sql-datawarehouse-basic
>>
>>
>> I could not find a solution to this, and I'd really dislike to do a
>> line-wise or blob insert statement (I'm working with multiple datasets that
>> each has a few million rows, so execution time is a consideration, although
>> the result sets I'm getting are more like in the 100k lines area each.)
>>
>>
>> I get the same error messages even when I replace the pd.to_sql command with
>> a simple engine.connect()
>>
>>
>> Enclosed my installed packages (packages.list)
>>
>> Enclosed the full traceback (traceback.txt)
>>
>>
>> This is the code I'm using:
>>
>> connection_string =
>> "mssql+pyodbc://:@.database.windows.net:/?driver=ODBC+Driver+13+for+SQL+Server"
>> engn = sqlalchemy.engine.create_engine(connection_string, echo=True)
>> engn.connect()
>>
>>
>> I'm very well aware that MS SQL DataWarehouse behaves a bit different, so
>> I'm open for some experimenting to get this issue narrowed down.
>>
>> In case it matters: I'm running an ubuntu 16.04 VM on azure with jupyter
>> notebook server and python 3.6.1.
>>
>> Best,
>>
>> DB
>
>
> Confidentiality Note: This email may contain confidential and/or private
> information.
> If you received this email in error please delete and notify sender.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
