[sqlalchemy] Re: MapperExtension.[before|after]_update problem

2010-10-14 Thread Christophe de Vienne
Hi Connor,

On 13 oct, 18:23, Conor conor.edward.da...@gmail.com wrote:
   On 10/13/2010 10:55 AM, Christophe de Vienne wrote:



  Hi all,

  I am running into an issue with MapperExtension.[before|after]_update.

  [...]

 AFAIK SQLAlchemy does not support the following in MapperExtensions:

 * lazy-loading related objects (maybe?)

It looks like this. Because if I make sure the related objects are
loaded before flushing the problem.

 * changing the flush plan, which I believe means changing which
   objects are considered new, dirty, or deleted

Which is why I do a double flush.

 Your code is possibly trying to do both.

Yep, but the disturbing thing is that I could not reproduce the issue
in a small example, hence my question.

 You need to instead create a
 SessionExtension and override before_flush, which allows you to modify
 the session however you want, e.g. (untested):

 class MySessionExtension(object):
  def before_flush(self, session, flush_context, instances):
  for obj in session.dirty:
  if isinstance(obj, Parent):
  for child in obj.children:
  child.name = 'another name'

This is a very interesting idea, which beside solving my issue
(hopefully) would also avoid a double-flush.

I guess I should also have a look to AttributeExtension which could be
a solution in my case.


Thanks,

Christophe

-- 
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en.



[sqlalchemy] MapperExtension.[before|after]_update problem

2010-10-13 Thread Christophe de Vienne
Hi all,

I am running into an issue with MapperExtension.[before|after]_update.

Context
---

SQLAlchemy 0.5.8
TurboGears 1.1.1

Description
---

I will attempt a simple description first, as I don't think my actual
code will help (I know it is not a thing to say, but really).

We have 2 mapped classes, Parent and Child.

Child is mapped this way :

mapper(Child, child_table, properties={
  parent=relation(Parent, backref='children')
})

And parent has a mapperextension that defines a after_update :

def after_update(self, mapper, connection, instance):
for child in instance.children:
child.name = 'another name'


If I do 2 session flush() after modifying a Parent instance, the
modifications on the children should be reflected to the database.

It is the case in my unit tests if I use directly the DBSession and
manipulate the objects 'myself'.

BUT, if I go through the complete TG stack, in the unittests or in
real-life, the modifications done on child are never sent to the database.

One subtle thing though : if, before the first flush(), I access the
children attribute, the problem goes away.

Example :

parent = DBSession.query(Parent).get('myid')
parent.name = 'test'
# parent.children # Un-commenting this line solve the issue, but of
  # course it is not an acceptable solution
DBSession.flush()
DBSession.flush()

I could not reproduce in a simpler context, and don't know how to go
further in my investigation.

Help ?

Thanks

Christophe

-- 
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en.



[sqlalchemy] Re: @synonym_for lose the docstrings

2009-06-25 Thread Christophe de VIENNE
Hi Michael,

Do you want me to create a ticket on this issue so it does not get lost ?

Thanks,

Christophe

2009/6/15 Christophe de VIENNE cdevie...@gmail.com

 Hi,

 I tried the patch, but got the following error :

 /home/cdevienne/prog/sqlalchemy/lib/sqlalchemy/ext/declarative.pyc in
 decorate(fn)
 645 
 646 def decorate(fn):
 -- 647 return util.update_wrapper(_orm_synonym(name,
 map_column=map_column, descriptor=fn), fn)
 648 return decorate
 649

 /usr/lib/python2.5/functools.pyc in update_wrapper(wrapper, wrapped,
 assigned, updated)
  31 
  32 for attr in assigned:
 --- 33 setattr(wrapper, attr, getattr(wrapped, attr))
  34 for attr in updated:
  35 getattr(wrapper, attr).update(getattr(wrapped, attr, {}))

 AttributeError: 'property' object has no attribute '__module__'

 I think the problem is, in that case, that the decorator is applied on
 property, not a function.

 I also tried to do the following :

   def decorate(fn):
   w = _orm_synonym(name, map_column=map_column, descriptor=fn)
   w.__doc__ = fn.__doc__
   return w
   return decorate

 But in my documentation, I get A combination of InsturmentedAttribute and
 a regular descriptor., which is the docstring of a Proxy class defined in
 proxied_attribute_factory, in attributes.py.

 From what I could understand, at one moment the
 attributes.register_descriptor function replace my property on the class,
 and it is seems to be at that moment that the docstring is lost. I might be
 completely wrong though, it is just a guess.

 Thanks,

 Chrisotphe

 2009/6/12 Michael Bayer mike...@zzzcomputing.com


 try this patch:

 Index: lib/sqlalchemy/ext/declarative.py
 ===
 --- lib/sqlalchemy/ext/declarative.py   (revision 6051)
 +++ lib/sqlalchemy/ext/declarative.py   (working copy)
 @@ -639,8 +639,9 @@
   prop = synonym('col', descriptor=property(_read_prop, _write_prop))

 
 +
 def decorate(fn):
 -return _orm_synonym(name, map_column=map_column, descriptor=fn)
 +return util.update_wrapper(_orm_synonym(name,
 map_column=map_column, descriptor=fn), fn)
 return decorate

  def comparable_using(comparator_factory):
 @@ -661,7 +662,7 @@

 
 def decorate(fn):
 -return comparable_property(comparator_factory, fn)
 +return
 util.update_wrapper(comparable_property(comparator_factory, fn), fn)
 return decorate

  def _declarative_constructor(self, **kwargs):



 Angri wrote:
 
  I think that patch which you would like to provide could add
  functools.wraps decorator to the decorator defined in synonym_for().
  See http://docs.python.org/library/functools.html#functools.wraps
 
  --
  Anton Gritsay
  http://angri.ru
 
  On 12 ÉÀÎ, 22:20, Christophe de VIENNE cdevie...@gmail.com wrote:
  Hi,
 
  I noticed that when I use the @synonym_for decorator, my function
  docstring
  get lost.
 
  I got lost in SA code around the attributes.register_descriptor
  function,
  which is one of the steps I guess the doc is not copied (along with the
  decorator function itself), and cannot propose a patch.
 
  I workaround the issue by doing the plain synonym declaration and using
  the
  sphinx specific docstrings for class attributes, but it would be really
  great if it could just work.
 
  And thanks for the awesome tool SqlAlchemy is : it is a constant source
  of
  amazement to me.
 
  Regards,
 
  Christophe
  
 


 



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: @synonym_for lose the docstrings

2009-06-15 Thread Christophe de VIENNE
Hi,

I tried the patch, but got the following error :

/home/cdevienne/prog/sqlalchemy/lib/sqlalchemy/ext/declarative.pyc in
decorate(fn)
645 
646 def decorate(fn):
-- 647 return util.update_wrapper(_orm_synonym(name,
map_column=map_column, descriptor=fn), fn)
648 return decorate
649

/usr/lib/python2.5/functools.pyc in update_wrapper(wrapper, wrapped,
assigned, updated)
 31 
 32 for attr in assigned:
--- 33 setattr(wrapper, attr, getattr(wrapped, attr))
 34 for attr in updated:
 35 getattr(wrapper, attr).update(getattr(wrapped, attr, {}))

AttributeError: 'property' object has no attribute '__module__'

I think the problem is, in that case, that the decorator is applied on
property, not a function.

I also tried to do the following :

  def decorate(fn):
  w = _orm_synonym(name, map_column=map_column, descriptor=fn)
  w.__doc__ = fn.__doc__
  return w
  return decorate

But in my documentation, I get A combination of InsturmentedAttribute and a
regular descriptor., which is the docstring of a Proxy class defined in
proxied_attribute_factory, in attributes.py.

From what I could understand, at one moment the
attributes.register_descriptor function replace my property on the class,
and it is seems to be at that moment that the docstring is lost. I might be
completely wrong though, it is just a guess.

Thanks,

Chrisotphe

2009/6/12 Michael Bayer mike...@zzzcomputing.com


 try this patch:

 Index: lib/sqlalchemy/ext/declarative.py
 ===
 --- lib/sqlalchemy/ext/declarative.py   (revision 6051)
 +++ lib/sqlalchemy/ext/declarative.py   (working copy)
 @@ -639,8 +639,9 @@
   prop = synonym('col', descriptor=property(_read_prop, _write_prop))

 
 +
 def decorate(fn):
 -return _orm_synonym(name, map_column=map_column, descriptor=fn)
 +return util.update_wrapper(_orm_synonym(name,
 map_column=map_column, descriptor=fn), fn)
 return decorate

  def comparable_using(comparator_factory):
 @@ -661,7 +662,7 @@

 
 def decorate(fn):
 -return comparable_property(comparator_factory, fn)
 +return
 util.update_wrapper(comparable_property(comparator_factory, fn), fn)
 return decorate

  def _declarative_constructor(self, **kwargs):



 Angri wrote:
 
  I think that patch which you would like to provide could add
  functools.wraps decorator to the decorator defined in synonym_for().
  See http://docs.python.org/library/functools.html#functools.wraps
 
  --
  Anton Gritsay
  http://angri.ru
 
  On 12 ÉÀÎ, 22:20, Christophe de VIENNE cdevie...@gmail.com wrote:
  Hi,
 
  I noticed that when I use the @synonym_for decorator, my function
  docstring
  get lost.
 
  I got lost in SA code around the attributes.register_descriptor
  function,
  which is one of the steps I guess the doc is not copied (along with the
  decorator function itself), and cannot propose a patch.
 
  I workaround the issue by doing the plain synonym declaration and using
  the
  sphinx specific docstrings for class attributes, but it would be really
  great if it could just work.
 
  And thanks for the awesome tool SqlAlchemy is : it is a constant source
  of
  amazement to me.
 
  Regards,
 
  Christophe
  
 


 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] @synonym_for lose the docstrings

2009-06-12 Thread Christophe de VIENNE
Hi,

I noticed that when I use the @synonym_for decorator, my function docstring
get lost.

I got lost in SA code around the attributes.register_descriptor function,
which is one of the steps I guess the doc is not copied (along with the
decorator function itself), and cannot propose a patch.

I workaround the issue by doing the plain synonym declaration and using the
sphinx specific docstrings for class attributes, but it would be really
great if it could just work.

And thanks for the awesome tool SqlAlchemy is : it is a constant source of
amazement to me.

Regards,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: query.filter and entity_name

2008-07-10 Thread Christophe de VIENNE

Thank you Michael, I will use subclassing instead.

A mention of this technique in the documentation (in the Multiple
Mappers for One Class) would be great, as it is the place I was
looking for a solution to my problem when I found entity_name.

Best regards,

Christophe

2008/7/7 Michael Bayer [EMAIL PROTECTED]:

 It's currently preferred if you didn't use entity_name since it works
 quite poorly and even worse in 0.5, and we'd like to remove it - it
 has built-in undefined behavior in that its not determined which set
 of attribute instrumentation gets applied to the class.

 This feature is an artifact of Hibernate which we copied at some point
 but doesn't apply well to Python where subclassing does not place a
 significant structural burden on code (and multiple inheritance makes
 it even less burdensome).   Mapping to individual subclases is much
 more straightforward - its the difference between your instance which
 is of class A plus magic entity name attribue B, versus, your instance
 is of class B subclassing A.   The object has state which represents
 the entity_name in either case.

 We haven't yet removed entity_name from 0.5 because I'm waiting for
 someone to have a truly compelling argument for it.


 On Jul 7, 2008, at 6:45 AM, Christophe de VIENNE wrote:


 Hi,

 I'm having trouble using entity_name.

 I have two mappers for the same class, one of them having an
 entity_name=legacy.

 If I do a query with the legacy mapper, I cannot figure how to
 filter on properties. Ex:

 session.query(MyClass, entity_name='legacy').filter(
MyClass.arelationprop.has( criterions ))

 In this case, the table from the default mapper is always getting in
 the way.

 I think I am not using properly the alternate mapper, but cannot find
 any example.

 Thanks a lot,

 Christophe

 


 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] query.filter and entity_name

2008-07-07 Thread Christophe de VIENNE

Hi,

I'm having trouble using entity_name.

I have two mappers for the same class, one of them having an
entity_name=legacy.

If I do a query with the legacy mapper, I cannot figure how to
filter on properties. Ex:

session.query(MyClass, entity_name='legacy').filter(
MyClass.arelationprop.has( criterions ))

In this case, the table from the default mapper is always getting in the way.

I think I am not using properly the alternate mapper, but cannot find
any example.

Thanks a lot,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] SchemaGenerator.get_column_default_string

2007-12-21 Thread Christophe de VIENNE

Hi all,

In the HEAD version, the table creation does not set the default
values on the columns (tested on mysql and mssql)
It seems that the function SchemaGenerator.get_column_default_string
(in compiler.py) always returns None, even if a default value is
specified.

 801 def get_column_default_string(self, column):
 802 if isinstance(column.default, schema.PassiveDefault):
 803 if isinstance(column.default.arg, basestring):
 804 return '%s' % column.default.arg
 805 else:
 806 return unicode(self._compile(column.default.arg, None))
 807 else:
 808 return None

On line 803, the test is always false because column.default is always
a ColumnDefault.

I'm not sure how this should be fixed, any help is very welcome.

Regards,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: SchemaGenerator.get_column_default_string

2007-12-21 Thread Christophe de VIENNE

2007/12/21, Michael Bayer [EMAIL PROTECTED]:
  In the HEAD version, the table creation does not set the default
  values on the columns (tested on mysql and mssql)
  It seems that the function SchemaGenerator.get_column_default_string
  (in compiler.py) always returns None, even if a default value is
  specified.
 
  801 def get_column_default_string(self, column):
  802 if isinstance(column.default, schema.PassiveDefault):
  803 if isinstance(column.default.arg, basestring):
  804 return '%s' % column.default.arg
  805 else:
  806 return unicode(self._compile(column.default.arg,
  None))
  807 else:
  808 return None
 

 ColumnDefaults are in-python default generators, they dont get
 rendered into DDL.  PassiveDefault is used to produce a DDL-rendered
 default:

 http://www.sqlalchemy.org/docs/04/metadata.html#metadata_defaults_passive


What is really disturbing is that Column(..., PassiveDefault(u'N'))
and Column(..., default=PassiveDefault(u'N')) don't have the same
behavior.
Anyway, now I understand what was my mistake.

Thank you !

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] SA 0.4.1 and MS-SQL problem at create time

2007-12-06 Thread Christophe de Vienne

Hi all,

Since SQLAlchemy 0.4.1, I cannot create my database anymore on mssql
only (mysql and sqlite are fine).
The error is the following :

  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/databases/mssql.py, line 945, in get_column_specification
if (not getattr(column.table, 'has_sequence', False)) and
column.primary_key and \
AttributeError: 'Column' object has no attribute 'foreign_key'

It seems that the mssql backend use a 'foreign_key' attribute on
Column which does not exist anymore.

How should this information be checked on a Column ?

Regards,

Christophe de Vienne


PS : More complete trace :
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/schema.py, line 303, in create
self.metadata.create_all(bind=bind, checkfirst=checkfirst,
tables=[self])
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/schema.py, line 1232, in create_all
bind.create(self, checkfirst=checkfirst, tables=tables)
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/engine/base.py, line 1052, in create
self._run_visitor(self.dialect.schemagenerator, entity,
connection=connection, **kwargs)
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/engine/base.py, line 1082, in _run_visitor
visitorcallable(self.dialect, conn, **kwargs).traverse(element)
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/sql/visitors.py, line 79, in traverse
meth(target)
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/sql/compiler.py, line 761, in visit_metadata
self.traverse_single(table)
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/sql/visitors.py, line 30, in traverse_single
return meth(obj, **kwargs)
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/sql/compiler.py, line 780, in visit_table
self.append(\t + self.get_column_specification(column,
first_pk=column.primary_key and not first_pk))
  File /usr/lib/python2.4/site-packages/SQLAlchemy-0.4.1-py2.4.egg/
sqlalchemy/databases/mssql.py, line 945, in get_column_specification
if (not getattr(column.table, 'has_sequence', False)) and
column.primary_key and \
AttributeError: 'Column' object has no attribute 'foreign_key'

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: SA 0.4.1 and MS-SQL problem at create time

2007-12-06 Thread Christophe de Vienne

Hi Paul,

On 6 déc, 19:31, Paul Johnston [EMAIL PROTECTED] wrote:
 Hi,

 It seems that the mssql backend use a 'foreign_key' attribute on
 Column which does not exist anymore.

 Yes, it's now foreign_keys. This is fixed in the svn trunk.

Thanks !


 I still need to sort out a way to have MSSQL unit tests run
 periodically, so we can pick up this kind of issue before releases.

I may be able to help, I'm sending you a mail privately.

Regards,

Christophe
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: FYI: AutoCode moved to a new repository

2007-10-12 Thread Christophe de VIENNE

2007/10/12, John M Camara [EMAIL PROTECTED]:

  I performed a release under LGPL. Hope that this is ok and fits into
  the sqlalchemy environment.
 

 Why not just release it under MIT like SQLAlchemy?  The project will
 likely receive wider use under MIT rather than LGPL.

+1 on the MIT licence

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: FYI: AutoCode moved to a new repository

2007-10-12 Thread Christophe de VIENNE

2007/10/12, Simon Pamies [EMAIL PROTECTED]:
 On Oct 12, 2:48 am, John M Camara [EMAIL PROTECTED] wrote:
   I performed a release under LGPL. Hope that this is ok and fits into
   the sqlalchemy environment.
 
  Why not just release it under MIT like SQLAlchemy?  The project will
  likely receive wider use under MIT rather than LGPL.

 Can you give me a short explanation of the MIT license model?

Wikipedia is clearer than I would be :
http://en.wikipedia.org/wiki/MIT_License

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: sqlalchemy 0.4 beta3 released

2007-08-17 Thread Christophe de VIENNE

Hi,

Any chance Ticket #731 get into 0.4 ? I'd prefer not relying on a
patch if possible.

Thanks a lot,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: sqlalchemy 0.4 beta3 released

2007-08-17 Thread Christophe de VIENNE

2007/8/17, Paul Johnston [EMAIL PROTECTED]:

 Hi,

 Any chance Ticket #731 get into 0.4 ? I'd prefer not relying on a
 patch if possible.
 
 
 I'd applied this to trunk and rel_0_3. I don't have alert mails setup
 from trac (maybe I should) so if you drop me a mail when you want an
 MSSQL patch looking at, I'll do my best.

No problem, I'll do that next time :-)

Thanks !

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] empty unicode strings are not encoded

2007-08-16 Thread Christophe de VIENNE

Hi,

I have a strange behavior (well, a blocking issue) on both sqlite and
mssql with unicode strings. It inserting a empty unicode string, it's
not encoded before being sent, while a non-empty string get encoded
correctly.
With sqlite it's not a problem, but with mssql it leads to a Invalid
data type (0)

The following program illustrate it :

from sqlalchemy import *

e = create_engine(sqlite:///, echo=True)
m = MetaData(e)

t = Table(test, m,
Column('test', String(convert_unicode=True)))
t.create()

m.bind.execute(t.insert(), dict(test=u'hello'))
m.bind.execute(t.insert(), dict(test=u''))


Running it output this :

2007-08-16 19:21:22,624 INFO sqlalchemy.engine.base.Engine.0x..34
CREATE TABLE test (
test TEXT
)


2007-08-16 19:21:22,625 INFO sqlalchemy.engine.base.Engine.0x..34 None
2007-08-16 19:21:22,626 INFO sqlalchemy.engine.base.Engine.0x..34 COMMIT
2007-08-16 19:21:22,628 INFO sqlalchemy.engine.base.Engine.0x..34
INSERT INTO test (test) VALUES (?)
2007-08-16 19:21:22,629 INFO sqlalchemy.engine.base.Engine.0x..34 ['hello']
2007-08-16 19:21:22,629 INFO sqlalchemy.engine.base.Engine.0x..34 COMMIT
2007-08-16 19:21:22,631 INFO sqlalchemy.engine.base.Engine.0x..34
INSERT INTO test (test) VALUES (?)
2007-08-16 19:21:22,632 INFO sqlalchemy.engine.base.Engine.0x..34 [u'']
2007-08-16 19:21:22,632 INFO sqlalchemy.engine.base.Engine.0x..34 COMMIT

I had a quick look at the code and I cannot see anything leading to
this behavior.

Any idea ?

Regards,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: empty unicode strings are not encoded

2007-08-16 Thread Christophe de VIENNE

I forgot to mention that I'm using the latest svn trunk version, and
that yesterday I was not having the problem, and although I did some
upgrade of my system, I don't think they are related (yet?).

2007/8/16, Christophe de VIENNE [EMAIL PROTECTED]:
 Hi,

 I have a strange behavior (well, a blocking issue) on both sqlite and
 mssql with unicode strings. It inserting a empty unicode string, it's
 not encoded before being sent, while a non-empty string get encoded
 correctly.

[...]

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: empty unicode strings are not encoded

2007-08-16 Thread Christophe de VIENNE

2007/8/16, Michael Bayer [EMAIL PROTECTED]:
 ok thats fixed in r3334.

That was fast ! Thanks a lot.

 pushes my nice speed test back over the 1M function call mark  :(

Sorry about that... ;-)

Cheers

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: MSSQL default_schema

2007-08-14 Thread Christophe de VIENNE

2007/8/13, Rick Morrison [EMAIL PROTECTED]:
 That SQL log is from the table existence check.

 Although it's unclear from the trace and log as to
 whether the check is for the table create
 or for the table drop, it is correctly using
 the default schema, which is 'dbo' on all
 MSSQL platforms.

It's for the delete (which then does not happen because the table is not found)

 So, the table check and the drop are working correctly. It's the table
 create that is incorrectly creating the table in the old owner schema,
 rather than using the default schema.

 I believe that's an pyodbc-ism, as I routinely use implicit (default) schema
 on pymssql, and all tables are created in the default schema correctly.
 Paul's workaround for this will work fine for now.

 Christophe, Are you using pyodbc, or some other DBAPI?

pyodbc on a linux platform

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: MSSQL default_schema

2007-08-14 Thread Christophe de VIENNE

2007/8/14, Rick Morrison [EMAIL PROTECTED]:
  It's for the delete (which then does not happen because the table is not
 found)

 Sure, but the drop is being issued in the correct default schema (dbo).

No it's not. If I don't enable checkfirst the table is dropped, which
means both statements are issued on the wrong schema (considering that
the check is right).

 The
 error is not that the drop is being issued in the wrong schema, it is that
 the table was *created* in the wrong schema, and so is not where it ought to
 be.

  pyodbc on a linux platform

 That is a problematic combination right now, currently pymssql is a better
 bet on Linux unless you're working with unicode columns. We're working on
 better support for pyodbc + *nix, but we first need to get through the 0.4
 beta cycle, as MSSQL is currently broken on the 0.4 trunk. A stable
 short-term platform for you would be the 0.3.10 release or the 0.3 branch
 tip + pymssql.

I'm going back and forth between both implementations, and am willing
to help a bit on mssql support. From time to time I try to see why a
unittest wouldn't pass and if I can see why I propose a patch.


Regards,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] MSSQL, pyodbc linux

2007-08-13 Thread Christophe de VIENNE

Hi all,

I have checked out the trunk and am trying to run the unittests under
linux using pyodbc.
So far, the connection to the database is working, although I had to
name the freetds odbc driver {SQL Server}, including the {}, so the
driver manager nows which one to use.

My problem is the following, and I have to idea where it comes from :
Any test from engine/bind fail with the following error (only the
query changes) :

DBAPIError: (ProgrammingError) ('42000', [42000] [FreeTDS][SQL
Server]Could not find stored procedure 'S'. (2812); [42000]
[FreeTDS][SQL Server]Could not find stored procedure 'S'. (2812))
u'SELECT [COLUMNS_1].[TABLE_SCHEMA], [COLUMNS_1].[TABLE_NAME],
[COLUMNS_1].[COLUMN_NAME], [COLUMNS_1].[IS_NULLABLE],
[COLUMNS_1].[DATA_TYPE], [COLUMNS_1].[ORDINAL_POSITION],
[COLUMNS_1].[CHARACTER_MAXIMUM_LENGTH],
[COLUMNS_1].[NUMERIC_PRECISION], [COLUMNS_1].[NUMERIC_SCALE],
[COLUMNS_1].[COLUMN_DEFAULT] \nFROM [INFORMATION_SCHEMA].[COLUMNS] AS
[COLUMNS_1] \nWHERE [COLUMNS_1].[TABLE_NAME] = ? AND
[COLUMNS_1].[TABLE_SCHEMA] = ?' ['test_table', 'dbo']


Any hint ?

Thanks

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: MSSQL, pyodbc linux

2007-08-13 Thread Christophe de VIENNE

Hi again,

I found something :
Using ipython, I could reprodure the error, by calling a
[engine.connect].execute(q, r), when q is a... unicode string. writing
...execute(q.encode('utf-8'), r) solved the problem.

I am looking at pyodbc internals to see if the problem comes from there.

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] MSSQL default_schema

2007-08-13 Thread Christophe de VIENNE

Hi all,

I still have problems with the default_schema and the way it's
handled in the mssql backend.

The following little program fails. If I run it with a simple user
(let's name it tester), I end up with a table  tester.t1, while
the function drop() look for a table dbo.t1.
I can't figure if the misbehaving part is the table creation which is
not explicit, the drop which is too explicit, or the
dialect.default_schema which doesn't return the correct value.

The program:
  1 from sqlalchemy import *
  2
  3 import sys
  4
  5 engine = create_engine(sys.argv[1], echo=True)
  6 meta = MetaData(engine)
  7
  8 t = Table('t1', meta, Column('id', Integer()))
  9
 10 t.create()
 11 t.drop(checkfirst=True)

Here is the echoed logs :

2007-08-13 17:48:42,550 INFO sqlalchemy.engine.base.Engine.0x..b4
CREATE TABLE t1 (
id INTEGER
)


2007-08-13 17:48:42,550 INFO sqlalchemy.engine.base.Engine.0x..b4 None
2007-08-13 17:48:42,551 INFO sqlalchemy.engine.base.Engine.0x..b4 COMMIT
2007-08-13 17:48:42,566 INFO sqlalchemy.engine.base.Engine.0x..b4 SET nocount ON
2007-08-13 17:48:42,566 INFO sqlalchemy.engine.base.Engine.0x..b4 None
2007-08-13 17:48:42,567 INFO sqlalchemy.engine.base.Engine.0x..b4
SELECT [COLUMNS_1].[TABLE_SCHEMA], [COLUMNS_1].[TABLE_NAME],
[COLUMNS_1].[COLUMN_NAME], [COLUMNS_1].[IS_NULLABLE],
[COLUMNS_1].[DATA_TYPE], [COLUMNS_1].[ORDINAL_POSITION],
[COLUMNS_1].[CHARACTER_MAXIMUM_LENGTH],
[COLUMNS_1].[NUMERIC_PRECISION], [COLUMNS_1].[NUMERIC_SCALE],
[COLUMNS_1].[COLUMN_DEFAULT]
FROM [INFORMATION_SCHEMA].[COLUMNS] AS [COLUMNS_1]
WHERE [COLUMNS_1].[TABLE_NAME] = ? AND [COLUMNS_1].[TABLE_SCHEMA] = ?
2007-08-13 17:48:42,567 INFO sqlalchemy.engine.base.Engine.0x..b4 ['t1', 'dbo']
c

Regards,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: autoload'ing metadata

2007-07-27 Thread Christophe de VIENNE

Hi svil,

Still no luck. I don't know if the information_schema module is
supposed to work well with pymssql. Anyway :

Traceback (most recent call last):
  File autoload.py, line 233, in ?
autoloader = AutoLoader( engine)
  File autoload.py, line 100, in __init__
me.table_names = engine.execute( sqltext)
  File /home/cdevienne/prog/sqlalchemy/lib/sqlalchemy/engine/base.py,
line 773, in execute
return connection.execute(statement, *multiparams, **params)
  File /home/cdevienne/prog/sqlalchemy/lib/sqlalchemy/engine/base.py,
line 517, in execute
return Connection.executors[c](self, object, *multiparams, **params)
  File /home/cdevienne/prog/sqlalchemy/lib/sqlalchemy/engine/base.py,
line 557, in execute_clauseelement
return self.execute_compiled(elem.compile(dialect=self.dialect,
parameters=param), *multiparams, **params)
  File /home/cdevienne/prog/sqlalchemy/lib/sqlalchemy/engine/base.py,
line 568, in execute_compiled
self._execute_raw(context)
  File /home/cdevienne/prog/sqlalchemy/lib/sqlalchemy/engine/base.py,
line 581, in _execute_raw
self._execute(context)
  File /home/cdevienne/prog/sqlalchemy/lib/sqlalchemy/engine/base.py,
line 599, in _execute
raise exceptions.SQLError(context.statement, context.parameters, e)
sqlalchemy.exceptions.SQLError: (DatabaseError) internal error (SQL
Server message 208, severity 16, state 1, line 1:
Invalid object name 'information_schema.tables'.
DB-Lib error message 20018, severity 5:
General SQL Server error: Check messages from the SQL Server.
): SQL Server message 208, severity 16, state 1, line 1:
Invalid object name 'information_schema.tables'.
DB-Lib error message 20018, severity 5:
General SQL Server error: Check messages from the SQL Server.
 'SELECT tables_a3c4.table_name, tables_a3c4.table_schema \nFROM
information_schema.tables AS tables_a3c4 \nWHERE
tables_a3c4.table_schema = %(tables_table_schema)s'
{'tables_table_schema': 'sf_tmp'}

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: Dialect default schema

2007-07-26 Thread Christophe de VIENNE

2007/7/26, Christophe de VIENNE [EMAIL PROTECTED]:
 Last thing : could someone tell me if the patches I posted for mssql
 looks 'commitable', because I'd prefer fixing those if needed before
 continuing my failing unit-tests slow (one per day) review.

Forgot the ticket numbers, sorry :
http://www.sqlalchemy.org/trac/ticket/679
http://www.sqlalchemy.org/trac/ticket/684
http://www.sqlalchemy.org/trac/ticket/685

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: Dialect default schema

2007-07-26 Thread Christophe de VIENNE

Hi Rick,

Thanks for the explanations. My confusion between database and schema
comes from how there words are inter-changeable in mysql.
I'll rethink all of that later, and make sure I understand it
correctly before proposing a patch.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: MSSQL Time

2007-07-26 Thread Christophe de VIENNE

2007/7/26, Paul Johnston [EMAIL PROTECTED]:
 One thing PyMSSQL will never support is Unicode - the MSSQL interface is
 uses is deprecated and just doesn't do unicode. For that reason, you may
 do better to focus your efforts on getting PyODBC working better on
 Unix. Right at the minute though it's more problematic than PyMSSQL.

I could see that...
One thing though :
 - pymssql, on unix, is based on freetds
 - pyODBC, at least on my system, use an ODBC driver based on freetds
The question is : if pymssql does not support unicode, does it mean
that freetds doesn't ?
Sub-Question : if FreeTDS supports unicode, is it realistic to try and
modify pymssql to make it more specific to freetds and support unicode
?

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Dialect default schema

2007-07-26 Thread Christophe de VIENNE

Hi,

I'm looking why the unittest SchemaTest.test_create_with_defaultschema
does not pass with MS-SQL.

The reason is that MSSQLDialect.get_default_schema_name takes no
argument, while the unittext gives it an engine. The Mssql dialect,
for example, does take an argument.

Having a closer look at MSSQLDialect.get_default_schema_name, it
simply returns self.schema_name, which is initialised in __init__ with
the value dbo. This is were I don't know what to think. Isn't the
default schema depending on the connection ? If so why is it
initialized to dbo ?

Another strange thing : MSSQLDialect defines a
set_default_schema_name, and it's the only dialect to do that.

Before patching this in the wrong way, I'd like to have a little bit
more details on how the default_shema stuffs are supposed to work.
Sub-question : shouldn't the set_default_schema_name issue a USE name ?

Last thing : could someone tell me if the patches I posted for mssql
looks 'commitable', because I'd prefer fixing those if needed before
continuing my failing unit-tests slow (one per day) review.

Thanks,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: Dialect default schema

2007-07-26 Thread Christophe de VIENNE

Thanks for the commits

I didn't say my last word for #685... I'll be back !

Christophe

2007/7/26, Rick Morrison [EMAIL PROTECTED]:
 I'm going to reply here, as I can't seem to login to trac again. I did
 manage to get comments in for #685

 #679 - committed in r3050
 #684 - committed in r3051
 #685 - needs more discussion see the Trac comments

 Thanks for the patches!
 Rick




 On 7/26/07, Christophe de VIENNE [EMAIL PROTECTED] wrote:
 
  2007/7/26, Christophe de VIENNE [EMAIL PROTECTED] :
   Last thing : could someone tell me if the patches I posted for mssql
   looks 'commitable', because I'd prefer fixing those if needed before
   continuing my failing unit-tests slow (one per day) review.
 
  Forgot the ticket numbers, sorry :
  http://www.sqlalchemy.org/trac/ticket/679
  http://www.sqlalchemy.org/trac/ticket/684
  http://www.sqlalchemy.org/trac/ticket/685
 
 
 
 


  


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] MSSQL Time

2007-07-24 Thread Christophe de VIENNE

Hi,

The mssql database does not provide a Time or a Date type, but
only datetime and smalldatetime.
In SQLAlchemy, the smalldatetime is used to implement the generic type
Date, but the Time type is not implemented.
Is there a particular reason for that ? could it inherit the DateTime
type, with a date set to 0 ?

Regards,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: MSSQL Time

2007-07-24 Thread Christophe de VIENNE

Hi Paul,

Thanks, I will give it a try. Stay tuned :-)

Christophe

2007/7/24, Paul Johnston [EMAIL PROTECTED]:

 Hi Christophe,

 No particular reason, just no-one has needed it yet, and the unit tests
 don't cover it.

 If you want to have a go at a patch, that'd be great. Otherwise I'll
 take a look at this in the next few days.

 Paul


 Christophe de VIENNE wrote:

 Hi,
 
 The mssql database does not provide a Time or a Date type, but
 only datetime and smalldatetime.
 In SQLAlchemy, the smalldatetime is used to implement the generic type
 Date, but the Time type is not implemented.
 Is there a particular reason for that ? could it inherit the DateTime
 type, with a date set to 0 ?
 
 Regards,
 
 Christophe
 
 
 
 
 


 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: MSSQL Time

2007-07-24 Thread Christophe de VIENNE

2007/7/24, Paul Johnston [EMAIL PROTECTED]:

 Hi,

 Is there any special condition for the unittests to run on mssql ?
 
 
 A few... you really need to be running on Windows and using PyODBC. And
 then append ?text_as_varchar=1 to your DBURI.

I'll try that :-)

 Looking at your command line, it looks like you're running from Unix.
 You can use PyMSSQL from Unix, and the basic features work fine, but a
 number of edge cases don't, so you get quite a lot of unit test failures.

I see. Are the reasons for thoses failures well known ? fixable ? If
it's not too tricky I could spend a bit of time on it in a little
while.

Thanks,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: Using SA to move data between databases

2007-07-24 Thread Christophe de VIENNE

Hi,

I'm in the same process, and very interested in the answer !

One idea I had is to define an universal dump format, (based for
example on pytables), which could be used to backup and restore datas
from/to various databases.
If this way is a good one and a good implementation proposed, it could
become an interesting addon to SA.

Regards,

Christophe

2007/7/24, Paul Johnston [EMAIL PROTECTED]:

 Hi,

 I am migrating an Access database to MSSQL server. I have coded up basic
 Access support for SQLAlchemy, which I'll be commiting to the 0.4 branch
 shortly.

 Using autocode, I now have a set of SQLAlchemy table definitions. My
 question is: what's a good way to move all the data across?

 Thanks for any help,

 Paul

 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Re: Using SA to move data between databases

2007-07-24 Thread Christophe de VIENNE

Hi

2007/7/24, michael [EMAIL PROTECTED]:

 On Tue, 24 Jul 2007 15:14:52 +0200
 Christophe de VIENNE [EMAIL PROTECTED] wrote:

 
  Hi,
 
  I'm in the same process, and very interested in the answer !
 
  One idea I had is to define an universal dump format, (based for
  example on pytables), which could be used to backup and restore datas
  from/to various databases.
  If this way is a good one and a good implementation proposed, it could
  become an interesting addon to SA.
 
  Regards,
 
  Christophe
 
  2007/7/24, Paul Johnston [EMAIL PROTECTED]:
  
   Hi,
  
   I am migrating an Access database to MSSQL server. I have coded up
   basic Access support for SQLAlchemy, which I'll be commiting to the
   0.4 branch shortly.
  
   Using autocode, I now have a set of SQLAlchemy table definitions. My
   question is: what's a good way to move all the data across?
  
   Thanks for any help,
  
   Paul
  
   
  

 With all due respect for the brilliance of SQLAlchemy.. it is not an
 operating system and not a database.

 Maybe I am missing the point here, but, in the two hours it took to get
 a reply to the OP, one could have output from one db (to csv) and
 import to the other one.  Another alternative is to actually use the db
 functionality.  MSAccess and MSSQL both start with 'MS'.  If I am not
 mistaken, those are interoperable.  One can set up a 'link' and
 transfer the data, no?  It has been years, but I remember doing that.

 Moving data in/out of disparate data sources is a pretty common data
 wharehouse process.  And if they are large datasets, native 'bulk'
 transfers are fastest.  All of which can be automated... without
 intervention from the application layer.  (was that blasphemy?)

I see no blasphemy, but that does not exactly address my personal
issue (which is not exactly the same as Paul it seems).
I will have, in a few months, clients running my software on mysql,
other on mssql. I want to have a common backup format, so I can
restore any backup on any supported db, and all that should be doable
by a Toto User (toto=dummy).
Having it in the application layer allow me do to that. And since I
hate to re-do things, my approach will most probably to use SA to dump
and restore the datas, even if it's a bit slow (the databases are not
very big), and it will always be possible to optimize the process by
doing db-specific operations.
The pytables format looks attractive for this use because it's fast,
scalable, compresses the datas, and have generic viewer.

My experience with SA is still a bit light, and I might say stupid
things without seeing it, but that's the general idea.

My two cents :-)

Regards,

Christophe

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---



[sqlalchemy] Improved autocode.py

2007-07-17 Thread Christophe de VIENNE
Hi all,

I'm Christophe, and this is my first post here. As such, I must start
it with a big : Thanks for sqlalchemy, it definitely rocks !.

Now to my main point.

I modified, for my needs, the autocode.py script which is found here :
http://www.sqlalchemy.org/trac/wiki/UsageRecipes/AutoCode

Before uploading my script onto the wiki page, I want to make sure
it's worth it, and the 'paj', the initial author, wouldn't mind.

Please find the script attached.

Regards,

Christophe de Vienne

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---

# autocode.py
#
# Author(s): Christophe de Vienne [EMAIL PROTECTED]
#'paj'
#
# Based on autocode.py by 'paj'
# (http://www.sqlalchemy.org/trac/wiki/UsageRecipes/AutoCode)
#
# Improvements over the original autocode.py:
#   * Takes arguments on the command line to select the dburl and
#   the output destination
#   * Replace a bunch of database specific types by generic ones.
#   This is incomplete as it feats only my needs for a mysql to mssql
#   database conversion.
#   * Output the indexes and ForeignKeyConstraints (including multi-columns
#   ones) correctly
#
# The resulting script is directly usable (ie import and create/use the tables)
# with my testing database (a legacy mysql db with about 140+ tables, 140+
# foreign keys, 170+ indexes), after applying patches
# http://www.sqlalchemy.org/trac/ticket/662 and
# http://www.sqlalchemy.org/trac/ticket/663 on a 0.3.9 release.
#

from sqlalchemy import *
from sqlalchemy.databases import information_schema
import string
import sys

from optparse import OptionParser

parser = OptionParser(usage: %prog [options] dburl)
parser.add_option('--output', '-o', action='store', dest='output',
metavar='FILE', default='stdout',
help='Write the result into FILE (default stdout)')

(options, args) = parser.parse_args()

if len(args) != 1:
parser.error('Wrong number or arguments')

dburl = engine.url.make_url(args[0])
db = create_engine(dburl)
metadata = BoundMetaData(db)

if options.output == 'stdout':
output = sys.stdout
else:
output = open(options.output, 'w')

def textclause_repr(self):
return 'text(%s)' % repr(self.text)

def table_repr(self):
return Table(%s) % ,\n.join(
[repr(self.name)] + [repr(self.metadata)] +
[repr(x) for x in self.columns] +
[repr(x) for x in self.constraints
if not isinstance(x, PrimaryKeyConstraint)]
)

def column_repr(self):
kwarg = []
if self.key != self.name:
kwarg.append('key')
if self._primary_key:
kwarg.append('primary_key')
if not self.nullable:
kwarg.append('nullable')
if self.onupdate:
kwarg.append('onupdate')
if self.default: 
kwarg.append('default')
return Column(%s) % ', '.join(
[repr(self.name)] + [repr(self.type)] +
[repr(x) for x in self.constraints] +
[%s=%s % (k, repr(getattr(self, k))) for k in kwarg]
)

def foreignkeyconstraint_repr(self):
return ForeignKeyConstraint(%s) % ', '.join(
[
repr([x.parent.name for x in self.elements]),
repr([x._get_colspec() for x in self.elements]),
'name=' + repr(self.name)
]
)

def repr_index(index, tvarname):
return Index(%s) % , .join(
[repr(index.name)] +
[%s.c.%s % (tvarname, c.name) for c in index.columns] +
['unique=' + repr(index.unique)])


sql._TextClause.__repr__ = textclause_repr
schema.Table.__repr__ = table_repr
schema.Column.__repr__ = column_repr
schema.ForeignKeyConstraint.__repr__ = foreignkeyconstraint_repr

sql = select([information_schema.tables.c.table_name,
  information_schema.tables.c.table_schema],
  information_schema.tables.c.table_schema==dburl.database)

output.write(from sqlalchemy import *
metadata = MetaData()

)

tname_list = []

for tname,schema in db.execute(sql):
if schema != dburl.database:
continue
tname_list.append(tname)
tbl = Table(tname, metadata, schema=schema, autoload=True)
code = repr(tbl)
code = code.replace('BoundMetaData()', 'metadata')
code = code.replace('MSChar', 'CHAR')
code = code.replace('MSSmallInteger(length=1)', 'Boolean()')
code = code.replace('MSSmallInteger', 'SmallInteger')
code = code.replace('MSDateTime', 'DateTime')
code = code.replace('MSMediumText', 'TEXT')
code = code.replace('MSDouble', 'Numeric')
code = code.replace('MSMediumText', 'TEXT')
code = code.replace('MSLongBlob', 'TEXT')
code = code.replace('MSString

[sqlalchemy] Re: Improved autocode.py

2007-07-17 Thread Christophe de VIENNE

done.

2007/7/17, Paul Johnston [EMAIL PROTECTED]:
 Hi Christophe,

 I'm glad SA works for you. We should all pay respect to Mike for creating
 the best ORM in town.

 I am paj on the wiki, and go ahead - add your updates to autocode.py. I am
 planning to do a somewhat more comprehensive fixup of the script at some
 point, which will probably take on board some of your changes.

 Happy coding!

 Paul




 On 7/17/07, Christophe de VIENNE [EMAIL PROTECTED] wrote:
  Hi all,
 
  I'm Christophe, and this is my first post here. As such, I must start
  it with a big : Thanks for sqlalchemy, it definitely rocks !.
 
  Now to my main point.
 
  I modified, for my needs, the autocode.py script which is found here :
  http://www.sqlalchemy.org/trac/wiki/UsageRecipes/AutoCode
 
  Before uploading my script onto the wiki page, I want to make sure
  it's worth it, and the 'paj', the initial author, wouldn't mind.
 
  Please find the script attached.
 
  Regards,
 
  Christophe de Vienne
 
 
 
 
 


  


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---