[
https://issues.apache.org/jira/browse/QPID-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Bouterse updated QPID-5637:
---------------------------------
Description:
qpid.messaging has an issue with forking in the following situation.
1. A parent Python process imports and uses qpid.messaging to connect to a a
Qpid broker
2. The parent process forks a child process
3. The child process imports qpid.messaging and tries to connect to a Qpid
broker.
I expected to see the child process use qpid.messaging normally as it would if
it weren't forked in the way described above. Instead, the server receives the
opening of a TCP socket, but the client never sends the AMQP protocol
announcement.
[Forking bring child descriptors with
it|http://man7.org/linux/man-pages/man2/fork.2.html] Because file descriptors
on the parent and the child are the same, I expect qpid.messaging to have the
parent and child process share the file descriptors and socket and work without
any modification.
There is at least one place where I do understand how this can be avoided. One
of the issues is that the file descriptors registered by the Selector object
inside of qpid.messaging are stale after the fork. The Selector object uses a
singleton pattern to provide a reference to the same Selector object no matter
how many times you call it. This selector object already has registered file
descriptors with the filesystem, which allow the selector to read/write data in
an I/O efficient manner. See the attached patch for an example of this.
This is not the only issue though, even with this patch in place, communication
flows, but queue creation, and deletion seem to fail. Here is the overall way
to reproduce this issue:
1. clone our fork of kombu: `git clone [email protected]:pulp/kombu.git`
2. Change into the kombu folder `cd kombu`
3. Switch to the branch containing the qpid code: `git checkout
pulp-dep-3.0.15-with-qpid`
4. Install kombu onto your system or virtualenv (I do it systemwide using
sudo): `sudo python setup.py develop`
5. install celery version 3.1.11. I do it using pip. `sudo pip install
celery==3.1.11`
6. Install qpid.messaging and qpidtoollibs. One way I do it is systemwide
using pip. `sudo pip install qpid-tools qpid-python`
7. Start up qpidd (We've been testing with 0.24 and auth off). `sudo -u
qpidd qpidd --auth=no`
8. Put the attached file tasks.py into a directory
9. Open two terminals and change their working directory to be the same as
step 8.
10. In one one terminal start the celery worker `celery worker -A tasks
--loglevel=INFO -c1`
11. In the other terminal dispatch 10 tasks `python tasks.py`
You should see exceptions raised similar to those in the attached file
[celery_worker_output.txt]
was:
qpid.messaging has an issue with forking in the following situation.
1. A parent Python process imports and uses qpid.messaging to connect to a a
Qpid broker
2. The parent process forks a child process
3. The child process imports qpid.messaging and tries to connect to a Qpid
broker.
I expected to see the child process use qpid.messaging normally as it would if
it weren't forked in the way described above. Instead, the server receives the
opening of a TCP socket, but the client never sends the AMQP protocol
announcement.
[Forking bring child descriptors with
it|http://man7.org/linux/man-pages/man2/fork.2.html] Because file descriptors
on the parent and the child are the same, I expect qpid.messaging to have the
parent and child process share the file descriptors and socket and work without
any modification.
There is at least one place where I do understand how this can be avoided. One
of the issues is that the file descriptors registered by the Selector object
inside of qpid.messaging are stale after the fork. The Selector object uses a
singleton pattern to provide a reference to the same Selector object no matter
how many times you call it. This selector object already has registered file
descriptors with the filesystem, which allow the selector to read/write data in
an I/O efficient manner. See the attached patch for an example of this.
This is not the only issue though, even with this patch in place, communication
flows, but queue creation, and deletion seem to fail. Here is the overall way
to reproduce this issue:
1. clone our fork of kombu: `git clone [email protected]:pulp/kombu.git`
2. Change into the kombu folder `cd kombu`
3. Switch to the branch containing the qpid code: `git checkout
pulp-dep-3.0.15-with-qpid`
4. Install kombu onto your system or virtualenv (I do it systemwide using
sudo): `sudo python setup.py develop`
5. install celery version 3.1.11. I do it using pip. `sudo pip install
celery==3.1.11`
6. Install qpid.messaging and qpidtoollibs. One way I do it is systemwide
using pip. `sudo pip install qpid-tools qpid-python`
7. Start up qpidd (We've been testing with 0.24 and auth off). `sudo -u
qpidd qpidd --auth=no`
8. Put the attached file tasks.py into a directory
9. Open two terminals and change their working directory to be the same as
step 8.
10. In one one terminal start the celery worker `celery worker -A tasks
--loglevel=INFO -c1`
11. In the other terminal dispatch 10 tasks `python tasks.py`
> qpid.messaging Issues With Forking
> ----------------------------------
>
> Key: QPID-5637
> URL: https://issues.apache.org/jira/browse/QPID-5637
> Project: Qpid
> Issue Type: Bug
> Components: Python Client
> Affects Versions: 0.24
> Reporter: Brian Bouterse
> Fix For: 0.18, 0.22, 0.24, 0.26, 0.27
>
> Attachments: celery_worker_output.txt, pid_aware_selector.patch
>
>
> qpid.messaging has an issue with forking in the following situation.
> 1. A parent Python process imports and uses qpid.messaging to connect to a a
> Qpid broker
> 2. The parent process forks a child process
> 3. The child process imports qpid.messaging and tries to connect to a Qpid
> broker.
> I expected to see the child process use qpid.messaging normally as it would
> if it weren't forked in the way described above. Instead, the server
> receives the opening of a TCP socket, but the client never sends the AMQP
> protocol announcement.
> [Forking bring child descriptors with
> it|http://man7.org/linux/man-pages/man2/fork.2.html] Because file
> descriptors on the parent and the child are the same, I expect qpid.messaging
> to have the parent and child process share the file descriptors and socket
> and work without any modification.
> There is at least one place where I do understand how this can be avoided.
> One of the issues is that the file descriptors registered by the Selector
> object inside of qpid.messaging are stale after the fork. The Selector
> object uses a singleton pattern to provide a reference to the same Selector
> object no matter how many times you call it. This selector object already
> has registered file descriptors with the filesystem, which allow the selector
> to read/write data in an I/O efficient manner. See the attached patch for an
> example of this.
> This is not the only issue though, even with this patch in place,
> communication flows, but queue creation, and deletion seem to fail. Here is
> the overall way to reproduce this issue:
> 1. clone our fork of kombu: `git clone [email protected]:pulp/kombu.git`
> 2. Change into the kombu folder `cd kombu`
> 3. Switch to the branch containing the qpid code: `git checkout
> pulp-dep-3.0.15-with-qpid`
> 4. Install kombu onto your system or virtualenv (I do it systemwide using
> sudo): `sudo python setup.py develop`
> 5. install celery version 3.1.11. I do it using pip. `sudo pip install
> celery==3.1.11`
> 6. Install qpid.messaging and qpidtoollibs. One way I do it is systemwide
> using pip. `sudo pip install qpid-tools qpid-python`
> 7. Start up qpidd (We've been testing with 0.24 and auth off). `sudo -u
> qpidd qpidd --auth=no`
> 8. Put the attached file tasks.py into a directory
> 9. Open two terminals and change their working directory to be the same as
> step 8.
> 10. In one one terminal start the celery worker `celery worker -A
> tasks --loglevel=INFO -c1`
> 11. In the other terminal dispatch 10 tasks `python tasks.py`
> You should see exceptions raised similar to those in the attached file
> [celery_worker_output.txt]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
