Re: [Twisted-Python] advice sought - two database connections in a RunInteraction (adbapi.ConnectionPool)

[Glyph]

On Sep 22, 2014, at 9:00 AM, Jonathan Vanasco  wrote:

> has anyone had a need like this before?  if so, what did you do?  

Yes.  I re-wrote twisted.enterprise entirely, with a better design.

You can see the result here:



And you can 'pip install' it from here:



Hopefully it will see a non-alpha release at some point.

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] GPRS/Modem protocol through USB support

[Werner Thie]

Aloha

maybe you can get some clues from here

https://gist.github.com/claws/2464017

which is pretty close to what your looking for.

HTH, Werner

On 9/22/14 7:34 AM, Bram Van Steenlandt wrote:


op 22-09-14 18:03, Laurens Van Houtven schreef:

On 22 Sep 2014, at 17:31, Bram Van Steenlandt  wrote:


No specific reason actually, I didn't even know twisted had support for a 
serial port.

This being said:
ftdio_sio can be a bit buggy at times, esp when using multiple devices on the same 
pc, ftdi on freebsd & osx isn't very stable either.
I use this interface selector system where the serial connection can be setup with 
pyserial,libftdi (which works much better for ftdi adapters) or a tcp socket (which uses 
a so called "device server" with ethernet and rs232 onboard).
Doing it with twisted seems more complicated to me (but I could be wrong), in 
my case it makes a lot of sense to wait for the data, I poll plc's over rs485 
and those kind of things.

Use cases vary :-) In a wait-dosomething-wait scenario, I find @inlineCallbacks 
to typically yield virtually the same code you ended up with.

When I was doing this, there was some pretty complex chatter back and forth, 
and I was multiplexing several connections over a single RS232 device, so 
Twisted certainly came in handy. In your case: don’t fix what ain’t broken, of 
course :)

Wouldn't it be difficult to combine the twisted method with external
libaries like MinimalModbus, libftdi , libartnet etc ?
Or is there some magic that I'm not aware of ?
I there a disadvantage to using a thread ?



Also as I've said, there are sometimes a lot of hardware issues (usb bus resets 
and stuff, users that disconnect usb-ports and plug them back in) which can be 
difficult to deal with.

How long does twisted has this serial port support actually ?

Well over a decade: Mon Dec 16 04:04:25 2002 to be precise. See svn rev 
trunk@4528.

hth
lvh


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python




___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python




___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] GPRS/Modem protocol through USB support

[Bram Van Steenlandt]

op 22-09-14 18:03, Laurens Van Houtven schreef:

On 22 Sep 2014, at 17:31, Bram Van Steenlandt  wrote:


No specific reason actually, I didn't even know twisted had support for a 
serial port.

This being said:
ftdio_sio can be a bit buggy at times, esp when using multiple devices on the same 
pc, ftdi on freebsd & osx isn't very stable either.
I use this interface selector system where the serial connection can be setup with 
pyserial,libftdi (which works much better for ftdi adapters) or a tcp socket (which uses 
a so called "device server" with ethernet and rs232 onboard).
Doing it with twisted seems more complicated to me (but I could be wrong), in 
my case it makes a lot of sense to wait for the data, I poll plc's over rs485 
and those kind of things.

Use cases vary :-) In a wait-dosomething-wait scenario, I find @inlineCallbacks 
to typically yield virtually the same code you ended up with.

When I was doing this, there was some pretty complex chatter back and forth, 
and I was multiplexing several connections over a single RS232 device, so 
Twisted certainly came in handy. In your case: don't fix what ain't broken, of 
course :)
Wouldn't it be difficult to combine the twisted method with external 
libaries like MinimalModbus, libftdi , libartnet etc ?

Or is there some magic that I'm not aware of ?
I there a disadvantage to using a thread ?



Also as I've said, there are sometimes a lot of hardware issues (usb bus resets 
and stuff, users that disconnect usb-ports and plug them back in) which can be 
difficult to deal with.

How long does twisted has this serial port support actually ?

Well over a decade: Mon Dec 16 04:04:25 2002 to be precise. See svn rev 
trunk@4528.

hth
lvh


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] GPRS/Modem protocol through USB support

[Laurens Van Houtven]

On 22 Sep 2014, at 17:31, Bram Van Steenlandt  wrote:

> No specific reason actually, I didn't even know twisted had support for a 
> serial port.
> 
> This being said:
> ftdio_sio can be a bit buggy at times, esp when using multiple devices on the 
> same pc, ftdi on freebsd & osx isn't very stable either.
> I use this interface selector system where the serial connection can be setup 
> with pyserial,libftdi (which works much better for ftdi adapters) or a tcp 
> socket (which uses a so called "device server" with ethernet and rs232 
> onboard).
> Doing it with twisted seems more complicated to me (but I could be wrong), in 
> my case it makes a lot of sense to wait for the data, I poll plc's over rs485 
> and those kind of things.

Use cases vary :-) In a wait-dosomething-wait scenario, I find @inlineCallbacks 
to typically yield virtually the same code you ended up with. 

When I was doing this, there was some pretty complex chatter back and forth, 
and I was multiplexing several connections over a single RS232 device, so 
Twisted certainly came in handy. In your case: don’t fix what ain’t broken, of 
course :)

> Also as I've said, there are sometimes a lot of hardware issues (usb bus 
> resets and stuff, users that disconnect usb-ports and plug them back in) 
> which can be difficult to deal with.
> 
> How long does twisted has this serial port support actually ?

Well over a decade: Mon Dec 16 04:04:25 2002 to be precise. See svn rev 
trunk@4528.

hth
lvh


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] advice sought - two database connections in a RunInteraction (adbapi.ConnectionPool)

[Jonathan Vanasco]

I've got some database transactional code running within a thread via 
RunInteraction (it needs to rollback on error).

One of the features on the logic is that it interacts with a 3rd party API, 
which may or may not have a ratelimit in effect.  Most 3rd party APIs i consume 
have a minute/hourly based limit -- so I just do a graceful fail on the 
transaction to requeue it, and set a local timeout.  

This particular API has a per-second meter in effect.  It is considerably more 
efficient to just sleep for a second and "try again" a second later.  I end up 
being able to process over twice as many API calls in a given period with that 
strategy.

The problem I ran into, is that I need a second database connection to note 
that a ratelimit has been created in the database -- and i'd like to pull that 
db connection out of the twisted connection pool to simplify my connection 
management.

- I can't do it within the existing connection, because I am in a transaction 
and the limit won't appear to other twisted jobs.  
- I can't use runInteraction() off the pool, because that will defer to a 
thread and I am in a thread, and the last time I did something like that... 
everything broke.
- the stopgap I have right now is just creating/closing a raw pscycopg2 
connection and using that.  it works, but it requires a bit more management of 
postgres to ensure I can balance the right number of connections.

has anyone had a need like this before?  if so, what did you do?  

i basically just need to make two "INSERT" statements from a second connection.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] GPRS/Modem protocol through USB support

[Bram Van Steenlandt]

No specific reason actually, I didn't even know twisted had support for 
a serial port.


This being said:
ftdio_sio can be a bit buggy at times, esp when using multiple devices 
on the same pc, ftdi on freebsd & osx isn't very stable either.
I use this interface selector system where the serial connection can be 
setup with pyserial,libftdi (which works much better for ftdi adapters) 
or a tcp socket (which uses a so called "device server" with ethernet 
and rs232 onboard).
Doing it with twisted seems more complicated to me (but I could be 
wrong), in my case it makes a lot of sense to wait for the data, I poll 
plc's over rs485 and those kind of things.
Also as I've said, there are sometimes a lot of hardware issues (usb bus 
resets and stuff, users that disconnect usb-ports and plug them back in) 
which can be difficult to deal with.


How long does twisted has this serial port support actually ?

Bram

op 22-09-14 17:05, Laurens Van Houtven schreef:

Hi Bram & Simon,

I'll answer both e-mails in one go :-)

On 22 Sep 2014, at 16:57, Bram Van Steenlandt  wrote:


I'm using a gsm modem for sending sms messages, the modem is rs232 so I have to 
use and ftdi adapter.
You can use these with pyserial in a thread in twisted.

Huh! Interesting; what drove that choice, given twisted's serial support?


op 22-09-14 16:01, Simon Andrieu schreef:

Could you help me to know if GPRS/3G/Modem protocols through USB are supported 
with Twisted please? And how it is possible to use this interface?

Depends what that thing exposes. Usually it's a character device, so you end up 
whistling some bytes. However, if you are *just* interested in a very standard 
Internet connection setup, it is probably much easier to use the existing OS 
tools for connecting to the internet using that model, than to write something 
using Twisted. Twisted would be interesting if you want to do interesting 
things beyond that; e.g. talk to the device and ask it for all sorts of e.g. 
telemetry data, data that is not already exposed in some standard, easy to 
consume way.

hth
lvh


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] GPRS/Modem protocol through USB support

[Laurens Van Houtven]

Hi Bram & Simon,

I’ll answer both e-mails in one go :-)

On 22 Sep 2014, at 16:57, Bram Van Steenlandt  wrote:

> I'm using a gsm modem for sending sms messages, the modem is rs232 so I have 
> to use and ftdi adapter.
> You can use these with pyserial in a thread in twisted.

Huh! Interesting; what drove that choice, given twisted’s serial support?

> op 22-09-14 16:01, Simon Andrieu schreef: 
>> 
>> Could you help me to know if GPRS/3G/Modem protocols through USB are 
>> supported with Twisted please? And how it is possible to use this interface?

Depends what that thing exposes. Usually it’s a character device, so you end up 
whistling some bytes. However, if you are *just* interested in a very standard 
Internet connection setup, it is probably much easier to use the existing OS 
tools for connecting to the internet using that model, than to write something 
using Twisted. Twisted would be interesting if you want to do interesting 
things beyond that; e.g. talk to the device and ask it for all sorts of e.g. 
telemetry data, data that is not already exposed in some standard, easy to 
consume way.

hth
lvh


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] GPRS/Modem protocol through USB support

[Bram Van Steenlandt]

Hi,

I'm using a gsm modem for sending sms messages, the modem is rs232 so I 
have to use and ftdi adapter.

You can use these with pyserial in a thread in twisted.
It uses AT commands, I have one from sierra wireless, some phones can do 
the same.


3G/internet is something else, here you just need something the 
operating system will support, twisted doesn't care how you are connected.


Bram

op 22-09-14 16:01, Simon Andrieu schreef:


Hi,

I work for a company that develops telemetry boxes, and that uses the 
Twisted framework running on the main board.


It is planned to integrate a cellular module connected via USB or 
serial link to the main board.


Could you help me to know if GPRS/3G/Modem protocols through USB are 
supported with Twisted please? And how it is possible to use this 
interface?


Thanks for your help.

Best Regards,

Simon Andrieu



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] GPRS/Modem protocol through USB support

[Simon Andrieu]

Hi,

I work for a company  that develops telemetry boxes, and that uses the Twisted 
framework running on the main board.
It is planned to integrate a cellular module connected via USB or serial link 
to the main board.

Could you help me to know if GPRS/3G/Modem protocols through USB are supported 
with Twisted please? And how it is possible to use this interface?

Thanks for your help.

Best Regards,
Simon Andrieu

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python