[jira] [Commented] (PROTON-784) Settled Deliveries keep piling up, and do not seem to get freed from heap
[
https://issues.apache.org/jira/browse/PROTON-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265271#comment-14265271
]
Chuck Rolke commented on PROTON-784:
I'm sure there is an issue. It may not be this issue.
> Settled Deliveries keep piling up, and do not seem to get freed from heap
> -
>
> Key: PROTON-784
> URL: https://issues.apache.org/jira/browse/PROTON-784
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.9
>Reporter: German Shepherd (PrE)
> Fix For: 0.9
>
>
> When running the ProtonC 0.8, this issue was not present.
> Then, in ProtonC 0.9 (Git trunk of 12/19/2014), there are changes in a way
> the deliveries are handled (engine.c).
> Correct me if I'm wrong, but given the behavior in 0.8, and keeping the same
> application side approach to ProtonC, while running 0.9, I would expect the
> Deliveries to be eventually freed from heap. Or is there a new plan on how to
> approach this ?
> I'm calling pn_messenger_settle with PN_CUMMULATIVE for both input and output
> trackers, and the delivery->local.settled gets set to true (verified).
> Here is the dump of Deliveries, after there is one message received:
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> Then my application code sends a message to broker. The previous, now fully
> settled Delivery, is not freed - here is the dump (there are now two
> Deliveries on heap):
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> Continuing, by received one more message and also sending one to broker,
> there are now four Deliveries on the heap.
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x01\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> And so on, until it eventually runs out of heap.
> I have done an extensive debug session, looking at the refcounts, behavior
> inside the pn_delivery_finalize(), places this gets called from, etc.
> One thing that hits the eye is the missing pn_full_settle() function, which
> is now replaced somehow differently by other code. Also, the tpwork-ers used
> to decref the Disposions, this is also modified for 0.9 (compared to 0.8).
> I'm not sure (yet) what is going on in there, it also might just be some
> error of mine (so I'm not immediately describing this as a bug, and I might
> provide more information as the situation unravels).
> At this moment, I would appreciate some hints on this topic, and also a
> review of the particular functionality (dispositon of Deliveries) in ProtonC
> 0.9.
> Thanks a lot !
> (minor edit - text formatting - on 12/21/2014)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-784) Settled Deliveries keep piling up, and do not seem to get freed from heap
[
https://issues.apache.org/jira/browse/PROTON-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265176#comment-14265176
]
Rafael H. Schloming commented on PROTON-784:
I'm a bit confused as to how the fix could be platform specific. Are you sure
the issue was there on linux prior to my fix?
> Settled Deliveries keep piling up, and do not seem to get freed from heap
> -
>
> Key: PROTON-784
> URL: https://issues.apache.org/jira/browse/PROTON-784
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.9
>Reporter: German Shepherd (PrE)
> Fix For: 0.9
>
>
> When running the ProtonC 0.8, this issue was not present.
> Then, in ProtonC 0.9 (Git trunk of 12/19/2014), there are changes in a way
> the deliveries are handled (engine.c).
> Correct me if I'm wrong, but given the behavior in 0.8, and keeping the same
> application side approach to ProtonC, while running 0.9, I would expect the
> Deliveries to be eventually freed from heap. Or is there a new plan on how to
> approach this ?
> I'm calling pn_messenger_settle with PN_CUMMULATIVE for both input and output
> trackers, and the delivery->local.settled gets set to true (verified).
> Here is the dump of Deliveries, after there is one message received:
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> Then my application code sends a message to broker. The previous, now fully
> settled Delivery, is not freed - here is the dump (there are now two
> Deliveries on heap):
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> Continuing, by received one more message and also sending one to broker,
> there are now four Deliveries on the heap.
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x01\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> And so on, until it eventually runs out of heap.
> I have done an extensive debug session, looking at the refcounts, behavior
> inside the pn_delivery_finalize(), places this gets called from, etc.
> One thing that hits the eye is the missing pn_full_settle() function, which
> is now replaced somehow differently by other code. Also, the tpwork-ers used
> to decref the Disposions, this is also modified for 0.9 (compared to 0.8).
> I'm not sure (yet) what is going on in there, it also might just be some
> error of mine (so I'm not immediately describing this as a bug, and I might
> provide more information as the situation unravels).
> At this moment, I would appreciate some hints on this topic, and also a
> review of the particular functionality (dispositon of Deliveries) in ProtonC
> 0.9.
> Thanks a lot !
> (minor edit - text formatting - on 12/21/2014)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PROTON-791) c-reactor-tests fails because it listens to an IPv4 socket but connects to an IPv6 socket
Andrew Stitcher created PROTON-791:
--
Summary: c-reactor-tests fails because it listens to an IPv4
socket but connects to an IPv6 socket
Key: PROTON-791
URL: https://issues.apache.org/jira/browse/PROTON-791
Project: Qpid Proton
Issue Type: Bug
Components: proton-c
Reporter: Andrew Stitcher
Assignee: Rafael H. Schloming
strace of relevant bits of failing run.
{noformat}
...
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
bind(3, {sa_family=AF_INET, sin_port=htons(5672),
sin_addr=inet_addr("0.0.0.0")}, 16) = 0
listen(3, 50) = 0
...
socket(PF_INET6, SOCK_STREAM, IPPROTO_TCP) = 4
fcntl(4, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(4, F_SETFL, O_RDWR|O_NONBLOCK)= 0
setsockopt(4, SOL_TCP, TCP_NODELAY, [1], 4) = 0
connect(4, {sa_family=AF_INET6, sin6_port=htons(5672), inet_pton(AF_INET6,
"::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EINPROGRESS
(Operation now in progress)
poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN|POLLOUT}], 2, 1000) = 1
([{fd=4, revents=POLLIN|POLLOUT|POLLERR|POLLHUP}])
recvfrom(4, 0x1b19200, 16384, 0, 0, 0) = -1 ECONNREFUSED (Connection refused)
...
sendto(4, "AMQP\3\1\0\0\0\0\0!\2\1\0\0\0SA\320\0\0\0\21\0\0\0\2\243\tAN"...,
224, MSG_NOSIGNAL, NULL, 0) = -1 EPIPE (Broken pipe)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-784) Settled Deliveries keep piling up, and do not seem to get freed from heap
[
https://issues.apache.org/jira/browse/PROTON-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265090#comment-14265090
]
Chuck Rolke commented on PROTON-784:
On Linux this issue appears resolved. On Windows it's still an issue and I'm
investigating further from my end.
-Cheers
> Settled Deliveries keep piling up, and do not seem to get freed from heap
> -
>
> Key: PROTON-784
> URL: https://issues.apache.org/jira/browse/PROTON-784
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.9
>Reporter: German Shepherd (PrE)
> Fix For: 0.9
>
>
> When running the ProtonC 0.8, this issue was not present.
> Then, in ProtonC 0.9 (Git trunk of 12/19/2014), there are changes in a way
> the deliveries are handled (engine.c).
> Correct me if I'm wrong, but given the behavior in 0.8, and keeping the same
> application side approach to ProtonC, while running 0.9, I would expect the
> Deliveries to be eventually freed from heap. Or is there a new plan on how to
> approach this ?
> I'm calling pn_messenger_settle with PN_CUMMULATIVE for both input and output
> trackers, and the delivery->local.settled gets set to true (verified).
> Here is the dump of Deliveries, after there is one message received:
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> Then my application code sends a message to broker. The previous, now fully
> settled Delivery, is not freed - here is the dump (there are now two
> Deliveries on heap):
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> Continuing, by received one more message and also sending one to broker,
> there are now four Deliveries on the heap.
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x01\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> And so on, until it eventually runs out of heap.
> I have done an extensive debug session, looking at the refcounts, behavior
> inside the pn_delivery_finalize(), places this gets called from, etc.
> One thing that hits the eye is the missing pn_full_settle() function, which
> is now replaced somehow differently by other code. Also, the tpwork-ers used
> to decref the Disposions, this is also modified for 0.9 (compared to 0.8).
> I'm not sure (yet) what is going on in there, it also might just be some
> error of mine (so I'm not immediately describing this as a bug, and I might
> provide more information as the situation unravels).
> At this moment, I would appreciate some hints on this topic, and also a
> review of the particular functionality (dispositon of Deliveries) in ProtonC
> 0.9.
> Thanks a lot !
> (minor edit - text formatting - on 12/21/2014)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-784) Settled Deliveries keep piling up, and do not seem to get freed from heap
[
https://issues.apache.org/jira/browse/PROTON-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14264501#comment-14264501
]
Rafael H. Schloming commented on PROTON-784:
Any chance you can come up with a minimal reproducer in terms of the proton
API? I have no clue how messaging is using proton internally, so a minimal
reproducer in terms of the messaging API doesn't tell me a lot.
> Settled Deliveries keep piling up, and do not seem to get freed from heap
> -
>
> Key: PROTON-784
> URL: https://issues.apache.org/jira/browse/PROTON-784
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.9
>Reporter: German Shepherd (PrE)
> Fix For: 0.9
>
>
> When running the ProtonC 0.8, this issue was not present.
> Then, in ProtonC 0.9 (Git trunk of 12/19/2014), there are changes in a way
> the deliveries are handled (engine.c).
> Correct me if I'm wrong, but given the behavior in 0.8, and keeping the same
> application side approach to ProtonC, while running 0.9, I would expect the
> Deliveries to be eventually freed from heap. Or is there a new plan on how to
> approach this ?
> I'm calling pn_messenger_settle with PN_CUMMULATIVE for both input and output
> trackers, and the delivery->local.settled gets set to true (verified).
> Here is the dump of Deliveries, after there is one message received:
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> Then my application code sends a message to broker. The previous, now fully
> settled Delivery, is not freed - here is the dump (there are now two
> Deliveries on heap):
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> Continuing, by received one more message and also sending one to broker,
> there are now four Deliveries on the heap.
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x01\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> And so on, until it eventually runs out of heap.
> I have done an extensive debug session, looking at the refcounts, behavior
> inside the pn_delivery_finalize(), places this gets called from, etc.
> One thing that hits the eye is the missing pn_full_settle() function, which
> is now replaced somehow differently by other code. Also, the tpwork-ers used
> to decref the Disposions, this is also modified for 0.9 (compared to 0.8).
> I'm not sure (yet) what is going on in there, it also might just be some
> error of mine (so I'm not immediately describing this as a bug, and I might
> provide more information as the situation unravels).
> At this moment, I would appreciate some hints on this topic, and also a
> review of the particular functionality (dispositon of Deliveries) in ProtonC
> 0.9.
> Thanks a lot !
> (minor edit - text formatting - on 12/21/2014)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-784) Settled Deliveries keep piling up, and do not seem to get freed from heap
[
https://issues.apache.org/jira/browse/PROTON-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14264322#comment-14264322
]
German Shepherd (PrE) commented on PROTON-784:
--
I'm back at work. And - so I got to compile the 0.9 with the Rafael's patch
(BTW. thanks for that !):
it is stable and it does work well, the deliveries are removed from heap (or
put onto the list for reuse) now.
Nice.
I will let it work over night, sending and receiving messages, to see the
longer-term outcome.
Note. I'm testing the plain ProtonC (the Messenger), not the Qpid additions to
it,
so I can't comment on whether it fixed the situation Chuck has reported.
> Settled Deliveries keep piling up, and do not seem to get freed from heap
> -
>
> Key: PROTON-784
> URL: https://issues.apache.org/jira/browse/PROTON-784
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
>Affects Versions: 0.9
>Reporter: German Shepherd (PrE)
> Fix For: 0.9
>
>
> When running the ProtonC 0.8, this issue was not present.
> Then, in ProtonC 0.9 (Git trunk of 12/19/2014), there are changes in a way
> the deliveries are handled (engine.c).
> Correct me if I'm wrong, but given the behavior in 0.8, and keeping the same
> application side approach to ProtonC, while running 0.9, I would expect the
> Deliveries to be eventually freed from heap. Or is there a new plan on how to
> approach this ?
> I'm calling pn_messenger_settle with PN_CUMMULATIVE for both input and output
> trackers, and the delivery->local.settled gets set to true (verified).
> Here is the dump of Deliveries, after there is one message received:
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> Then my application code sends a message to broker. The previous, now fully
> settled Delivery, is not freed - here is the dump (there are now two
> Deliveries on heap):
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> Continuing, by received one more message and also sending one to broker,
> there are now four Deliveries on the heap.
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x00\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> {{tag=, local.type=36, remote.type=0, local.settled=1, remote.settled=1,
> updated=0, current=0, writable=0, readable=0, work=0}}
> {{tag=\x01\x00\x00\x00\x00\x00\x00\x00, local.type=36, remote.type=36,
> local.settled=1, remote.settled=1, updated=0, current=0, writable=0,
> readable=0, work=0}}
> And so on, until it eventually runs out of heap.
> I have done an extensive debug session, looking at the refcounts, behavior
> inside the pn_delivery_finalize(), places this gets called from, etc.
> One thing that hits the eye is the missing pn_full_settle() function, which
> is now replaced somehow differently by other code. Also, the tpwork-ers used
> to decref the Disposions, this is also modified for 0.9 (compared to 0.8).
> I'm not sure (yet) what is going on in there, it also might just be some
> error of mine (so I'm not immediately describing this as a bug, and I might
> provide more information as the situation unravels).
> At this moment, I would appreciate some hints on this topic, and also a
> review of the particular functionality (dispositon of Deliveries) in ProtonC
> 0.9.
> Thanks a lot !
> (minor edit - text formatting - on 12/21/2014)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
