Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread Chris Angelico
On Fri, Oct 23, 2020 at 3:35 AM Grant Edwards wrote: > Moving from 2.x to 3.x isn't too bad, but trying to maintain > compatiblity with both is painful. At this point, I would probably > just abandon 2.x. > Definitely. No point trying to support both when you're starting with code from a Py3

Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread Grant Edwards
On 2020-10-22, Chris Angelico wrote: > On Fri, Oct 23, 2020 at 12:15 AM Shaozhong SHI wrote: >> What should I know or watch out if I decide to move from Python 2.7 >> to Python 3? > > Key issues? Well, for starters, you don't have to worry about whether > your strings are Unicode or not. They

Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread Chris Angelico
On Fri, Oct 23, 2020 at 12:15 AM Shaozhong SHI wrote: > > Thanks, Chris. > > What should I know or watch out if I decide to move from Python 2.7 to Python > 3? > > What are the key issues? Syntax? > Keep it on-list please :) Key issues? Well, for starters, you don't have to worry about

Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread Marco Sulla
Never worked with ZFS, it sounds interesting. Anyway, IMHO it's much more simple to save to disk also for debugging: you have not to extract the data from the db if you need to inspect them. On Thu, 22 Oct 2020 at 14:39, D'Arcy Cain wrote: > On 10/22/20 7:23 AM, Marco Sulla wrote: > > I would

Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread D'Arcy Cain
On 10/22/20 7:23 AM, Marco Sulla wrote: I would add that usually I do not recommend saving files on databases. I usually save the file on the disk and the path and mime on a dedicated table. I used to do that because backing up the database became huge. Now I use ZFS snapshots with

Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread Chris Angelico
On Thu, Oct 22, 2020 at 8:28 PM Shaozhong SHI wrote: > > I found this last option is very interesting. > > Saving the dataframe to memory using StringIO > > https://naysan.ca/2020/06/21/pandas-to-postgresql-using-psycopg2-copy_from/ > > But, testing shows > unicode argument expected, got 'str' >

Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread Marco Sulla
I would add that usually I do not recommend saving files on databases. I usually save the file on the disk and the path and mime on a dedicated table. -- https://mail.python.org/mailman/listinfo/python-list

Re: Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread Marco Sulla
Try to save it in a binary field on PG using hdf5: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_hdf.html On Thu, 22 Oct 2020 at 11:29, Shaozhong SHI wrote: > I found this last option is very interesting. > > Saving the dataframe to memory using StringIO > >

Dataframe to postgresql - Saving the dataframe to memory using StringIO

2020-10-22 Thread Shaozhong SHI
I found this last option is very interesting. Saving the dataframe to memory using StringIO https://naysan.ca/2020/06/21/pandas-to-postgresql-using-psycopg2-copy_from/ But, testing shows unicode argument expected, got 'str' Any working example for getting DataFrame into a PostgreSQL table