Re: [sqlalchemy] SQLAlchemy enum in external file

[Jonathan Vanasco]

sorry. i'm getting over a code and in my fever-dream state thought I was 
making something clear with an analogy.

looking at this, and literally everything else I wrote yesterday, I'm 
amazed it's at least in english.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SQLAlchemy enum in external file

[TazMainiac]

On Fri, Oct 20, 2017 at 3:58 AM, Simon King  wrote:

> This is a subtle python gotcha based on how the import system works.
> It doesn't have anything to do with SQLAlchemy.
>
>  latest/python_concepts/import_traps.html#executing-the-main-module-twice>
>
> When you import a module, it gets cached in sys.modules under its
> "fully qualified module name", so it doesn't matter how many times you
> run "import foo", the module is only loaded once.
>
> However, the script that you initially *execute* gets the special name
> "__main__". If it gets subsequently imported via an "import"
> statement, it is not found in the cache, so it is executed again,
> meaning you get duplicate definitions of everything in the module.
>
> Hope that helps,
>
> Simon
>

Thank you.  This was the explanation I'd been looking for.

So - trying to work around this - the following works (after moving exiting
__main__ code into main() function):

$ python -c "from myoper import main; import sys; main(*sys.argv[1:])"
ext_check 1

But it seems like the best solution to this is to probably move any test
code that calls out to an external module OUT of myoper (probably into
myoper_test itself, which is what I'm going to end up doing).  Are there
any other common solutions?

Thanks again,
Taz

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SQLAlchemy enum in external file

[Simon King]

On Thu, Oct 19, 2017 at 9:10 PM, TazMainiac  wrote:
>
>
> On Thu, Oct 19, 2017 at 2:25 PM, Jonathan Vanasco 
> wrote:
>>
>> On Thursday, October 19, 2017 at 11:55:49 AM UTC-4, Taz Mainiac wrote:
>>>
>>> So - a morning spent googling does not turn up any information about
>>> Python classes having a different id depending on context (script vs
>>> module).  I'm probably not googling for the right thing?  Can anyone point
>>> me to information?
>>>
>>
>> The Python classes have a different id on each run not because of the
>> context (script vs module), but because they are run in different processes.
>> The "id" , like the object, is only local to the process.
>>
>> The attributes of your custom enum class are instances of enum objects.
>> which have attributes like name and value.
>
>
> I understand this.  The id is the address of the object under CPython (so it
> says in the docs).
>
>> If you ran two scripts or two modules, it would also almost always fail
>> (IIRC, it is remotely possible but highly unlikely, to generate the same id
>> in another process).
>
>
> I am not doing that.
>
>>
>> You should be storing/comparing the enum's 'name' or 'value' attributes --
>> not the enum object itself.  The enum object will always be different.
>
>
> I'm following the established documentation on how to use an enum with
> SQLAlchemy (with Postgres DB backend) - almost exactly as described in the
> doc under the Enum type:
>
>   http://docs.sqlalchemy.org/en/rel_1_1/core/type_basics.html
>
> (except I'm using declarative style).
>
> The problem is apparently that the enumerated type gets a different id
> depending on the context in which it is used (from a script vs from a module
> import).  This is very surprising to me, and I have not seen anything like
> it before (hence my question above asking for any references to this
> behavior).
>
> In any case, this causes SQL Alchemy to not recognize that the enumerated
> value assigned to a field (in another file) is not the same type as what is
> expected.  Clearly the ' is ' relationship is failing (since the id's are
> different).
>
> So - need to reorganize code a bit to avoid this...
>
> Taz

This is a subtle python gotcha based on how the import system works.
It doesn't have anything to do with SQLAlchemy.



When you import a module, it gets cached in sys.modules under its
"fully qualified module name", so it doesn't matter how many times you
run "import foo", the module is only loaded once.

However, the script that you initially *execute* gets the special name
"__main__". If it gets subsequently imported via an "import"
statement, it is not found in the cache, so it is executed again,
meaning you get duplicate definitions of everything in the module.

Hope that helps,

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SQLAlchemy enum in external file

[Mike Bayer]

It's not different "processes".   It's the difference between running
myfile.py from the python interpreter, making the module effectively
__main__, vs running myfile.py via "import myfile".   Same filesystem file
but two totally different modules in the same process.

On Oct 19, 2017 4:29 PM, "Jonathan Vanasco"  wrote:

>
>
> On Thursday, October 19, 2017 at 4:10:11 PM UTC-4, Taz Mainiac wrote:
>>
>>
>> If you ran two scripts or two modules, it would also almost always fail
>>> (IIRC, it is remotely possible but highly unlikely, to generate the same id
>>> in another process).
>>>
>>
>> I am not doing that.
>>
>>
> re-read Mike's explanation of how your implementation is effectively doing
> that.
>
> --
> SQLAlchemy -
> The Python SQL Toolkit and Object Relational Mapper
>
> http://www.sqlalchemy.org/
>
> To post example code, please provide an MCVE: Minimal, Complete, and
> Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> description.
> ---
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SQLAlchemy enum in external file

[Jonathan Vanasco]

On Thursday, October 19, 2017 at 4:10:11 PM UTC-4, Taz Mainiac wrote:
>
>
> If you ran two scripts or two modules, it would also almost always fail 
>> (IIRC, it is remotely possible but highly unlikely, to generate the same id 
>> in another process).
>>
>
> I am not doing that.
>
>
re-read Mike's explanation of how your implementation is effectively doing 
that.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SQLAlchemy enum in external file

[TazMainiac]

On Thu, Oct 19, 2017 at 2:25 PM, Jonathan Vanasco 
wrote:

> On Thursday, October 19, 2017 at 11:55:49 AM UTC-4, Taz Mainiac wrote:
>>
>> So - a morning spent googling does not turn up any information about
>> Python classes having a different id depending on context (script vs
>> module).  I'm probably not googling for the right thing?  Can anyone point
>> me to information?
>>
>>
> The Python classes have a different id on each run not because of the
> context (script vs module), but because they are run in different
> processes.  The "id" , like the object, is only local to the process.
>
> The attributes of your custom enum class are instances of enum objects.
> which have attributes like name and value.
>

I understand this.  The id is the address of the object under CPython (so
it says in the docs).

If you ran two scripts or two modules, it would also almost always fail
> (IIRC, it is remotely possible but highly unlikely, to generate the same id
> in another process).
>

I am not doing that.


> You should be storing/comparing the enum's 'name' or 'value' attributes --
> not the enum object itself.  The enum object will always be different.
>

I'm following the established documentation on how to use an enum with
SQLAlchemy (with Postgres DB backend) - almost exactly as described in the
doc under the Enum type:

  http://docs.sqlalchemy.org/en/rel_1_1/core/type_basics.html

(except I'm using declarative style).

The problem is apparently that the enumerated type gets a different id
depending on the context in which it is used (from a script vs from a
module import).  This is very surprising to me, and I have not seen
anything like it before (hence my question above asking for any references
to this behavior).

In any case, this causes SQL Alchemy to not recognize that the enumerated
value assigned to a field (in another file) is not the same type as what is
expected.  Clearly the ' is ' relationship is failing (since the id's are
different).

So - need to reorganize code a bit to avoid this...

Taz

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SQLAlchemy enum in external file

[Jonathan Vanasco]

On Thursday, October 19, 2017 at 11:55:49 AM UTC-4, Taz Mainiac wrote:
>
> So - a morning spent googling does not turn up any information about 
> Python classes having a different id depending on context (script vs 
> module).  I'm probably not googling for the right thing?  Can anyone point 
> me to information?
>
>
The Python classes have a different id on each run not because of the 
context (script vs module), but because they are run in different 
processes.  The "id" , like the object, is only local to the process.  

The attributes of your custom enum class are instances of enum objects. 
which have attributes like name and value.

If you ran two scripts or two modules, it would also almost always fail 
(IIRC, it is remotely possible but highly unlikely, to generate the same id 
in another process).

You should be storing/comparing the enum's 'name' or 'value' attributes -- 
not the enum object itself.  The enum object will always be different.  

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SQLAlchemy enum in external file

[TazMainiac]

On Wed, Oct 18, 2017 at 9:08 PM, Mike Bayer 
wrote:

> the example calls the "myoper" python file in two different contexts,
> resulting in two different MyStatus enumeration objects.   Calling
> "python myoper.py" invokes "myoper" as a plain Python script, sets up
> MyStatus and the mapping, then runs some database operations.   When
> it imports myoper_test, myoper_test imports myoper, which runs myoper
> all over again as a Python module since "myoper" is not in
> sys.modules.  It makes a new MyStatus and then is used against the
> mappings from the "python myoper.py" script, and fails.
>
> Place a print statement below MyStatus like this and you'll see:
>
> print "MAKING A MYSTATUS!!!  IDENTIFIER IS!! %s" % id(MyStatus)
>

I added:

print "MAKING A MYSTATUS!!!  IDENTIFIER IS!! %s" % id(MyStatus)
print "id(INITIAL) %s" % id(MyStatus.INITIAL)
print "type(INITIAL) %s" % type(MyStatus.INITIAL)
print "id(type(INITIAL)) %s" % id(type(MyStatus.INITIAL))

Immediately after he definition of MyStatus.

When I run it:

[~/test] ./myoper.py ext_check 1
MAKING A MYSTATUS!!!  IDENTIFIER IS!! 15407552
id(INITIAL) 140013357970512
type(INITIAL) 
id(type(INITIAL)) 15407552
test for opid 1
found oper: 
MAKING A MYSTATUS!!!  IDENTIFIER IS!! 17572848
id(INITIAL) 140013306205456
type(INITIAL) 
id(type(INITIAL)) 17572848
after compliance check: 
status: authorized check (external)
Traceback (most recent call last):


So - a morning spent googling does not turn up any information about Python
classes having a different id depending on context (script vs module).  I'm
probably not googling for the right thing?  Can anyone point me to
information?

So the obvious solution is to move the test code in __main__ to the
myoper_test.py script.  And that does indeed work.  Are there other obvious
workarounds for this kind of thing?

Thanks,
Taz

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.