[jira] [Created] (PROTON-1028) connect failue when creating BlockingConnection result in memory leak
Gordon Sim created PROTON-1028:
--
Summary: connect failue when creating BlockingConnection result in
memory leak
Key: PROTON-1028
URL: https://issues.apache.org/jira/browse/PROTON-1028
Project: Qpid Proton
Issue Type: Bug
Components: python-binding
Affects Versions: 0.10
Reporter: Gordon Sim
Assignee: Gordon Sim
Fix For: 0.11
E.g. assuming there is nothing listening on port 12345, run the following:
{noformat}
while True:
sleep(0.1)
try:
conn = BlockingConnection("amqp://localhost:12345", heartbeat=2)
except ConnectionException, e:
print e
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-1025) CLOSE_WAIT leak following reproducer for PROTON-1023 / PROTON-1024
[
https://issues.apache.org/jira/browse/PROTON-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966806#comment-14966806
]
Gordon Sim commented on PROTON-1025:
The example above 'leaks' BlockingConnection instances. I.e. if rec2 is created
successfully, the while loop will proceed and the BlockingConnection instance
will not have been closed when a new instance is assigned to the conn variable.
Perhaps the BlockingConnection could close itself when it becomes unreferenced,
but at present the expectation is that it should be explicitly closed.
> CLOSE_WAIT leak following reproducer for PROTON-1023 / PROTON-1024
> --
>
> Key: PROTON-1025
> URL: https://issues.apache.org/jira/browse/PROTON-1025
> Project: Qpid Proton
> Issue Type: Bug
> Components: python-binding
>Affects Versions: 0.10
>Reporter: Pavel Moravec
>Priority: Minor
>
> Following reproducer for PROTON-1023 or PROTON-1024 (attached at the botton),
> client leaves some sockets in CLOSE_WAIT state forever.
> I tested the reproducer before & after those two fixes and it is present in
> both. I.e. this bug is not a regression caused by PROTON-1023 or PROTON-1024.
> Reproducer:
> (assuming localhost runs qdrouterd that is restarted every 5 seconds in a
> loop):
> {code}
> #!/usr/bin/python
> from time import sleep
> from uuid import uuid4
> from proton import ConnectionException
> from proton.utils import BlockingConnection
> import traceback
> import random
> while True:
> sleep(random.uniform(0.3,3))
> try:
> conn = BlockingConnection("proton+amqp://localhost:5672",
> ssl_domain=None, heartbeat=2)
> rec = conn.create_receiver("another_address", name=str(uuid4()),
> dynamic=False, options=None)
> print "sleeping.."
> sleep(random.uniform(0.3,3))
> rec2 = conn.create_receiver("some_address", name=str(uuid4()),
> dynamic=False, options=None)
> except ConnectionException:
> try:
> if conn:
> conn.close()
> except Exception, e:
> print e
> print(traceback.format_exc())
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PROTON-1029) Do not fail hard if strerror_r fails.
Ken Giusti created PROTON-1029: -- Summary: Do not fail hard if strerror_r fails. Key: PROTON-1029 URL: https://issues.apache.org/jira/browse/PROTON-1029 Project: Qpid Proton Issue Type: Bug Components: proton-c Affects Versions: 0.10 Reporter: Ken Giusti Assignee: Ken Giusti Fix For: 0.11 The return type from strerror_r differs depending on the compiler's feature set. if _GNU_SOURCE is defined, strerror_r returns a char * if _POSIX_C_SOURCE is defined to a certain set of values, strerror_r returns an int. proton expects int, and fails hard (aborts) when a string ptr is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: PN_REACTOR_QUIESCED
On Wed, 2015-10-14 at 10:31 -0400, Rafael Schloming wrote: > It wasn't actually an accidental commit. If I recall correctly I > ended up > using it more like a 0xDEADBEEF value. It makes it easy to > distinguish > between the failure mode of an actual hang (e.g. infinite loop or > blocking > call inside a handler) vs reaching a state where there are simply no > more > events to process. I guess you can think of it like a heartbeat in a > way. > It seems like an odd default. Can we make it 0 in the code and set it non-0 in tests if that is needed? Randomly waking up a user app that hasn't asked for heartbeats every sort-of-3-seconds is a surprising behavior. > --Rafael > > On Tue, Oct 13, 2015 at 10:56 AM, Michael Goulish < > mgoul...@redhat.com> > wrote: > > > > > But it's obvious how this constant was chosen. > > > > With circular reasoning. > > > > > > > > > > > > - Original Message - > > > On Mon, 2015-10-12 at 16:05 -0400, aconway wrote: > > > > ... > > > > +1, that looks like the right fix. 3141 is an odd choice of > > > > default, > > > > even for a mathematician. > > > > > > > > > > At this point, I'm desperately trying to find an appropriate pi > > > joke : > > > -) > > > > > > Andrew > > > > > > > >
[jira] [Commented] (PROTON-1030) Reactor never freed if handler/global_handler set
[
https://issues.apache.org/jira/browse/PROTON-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967752#comment-14967752
]
Gordon Sim commented on PROTON-1030:
I've committed a fix for this in the python binding. It feels like this should
really be handled more cleanly in the c code, but the reference counting there
is still something of a mystery to me.
> Reactor never freed if handler/global_handler set
> -
>
> Key: PROTON-1030
> URL: https://issues.apache.org/jira/browse/PROTON-1030
> Project: Qpid Proton
> Issue Type: Bug
> Components: python-binding
>Affects Versions: 0.10
>Reporter: Gordon Sim
> Fix For: 0.11
>
>
> E.g.
> {noformat}
> from proton.reactor import Reactor
> class Print(object):
> def on_unhandled(self, name, event):
> print("%s %s" % (name, event))
> while True:
> reactor = Reactor()
> reactor.handler = Print()
> reactor.start()
> reactor.stop()
> {noformat}
> Will grow in memory and eventually crash. On debugging the reactor is never
> finalised.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PROTON-1030) Reactor never freed if handler/global_handler set
[
https://issues.apache.org/jira/browse/PROTON-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967748#comment-14967748
]
ASF subversion and git services commented on PROTON-1030:
-
Commit bbba61afef7c2c67cd6be6a0393dd5233d0d473c in qpid-proton's branch
refs/heads/master from [~gsim]
[ https://git-wip-us.apache.org/repos/asf?p=qpid-proton.git;h=bbba61a ]
PROTON-1030: workaround that resets reactors handlers after stopping to allow
it to be freed
> Reactor never freed if handler/global_handler set
> -
>
> Key: PROTON-1030
> URL: https://issues.apache.org/jira/browse/PROTON-1030
> Project: Qpid Proton
> Issue Type: Bug
> Components: python-binding
>Affects Versions: 0.10
>Reporter: Gordon Sim
> Fix For: 0.11
>
>
> E.g.
> {noformat}
> from proton.reactor import Container, Reactor
> class Print(object):
> def on_unhandled(self, name, event):
> print("%s %s" % (name, event))
> while True:
> container = Reactor()
> container.handler = Print()
> container.start()
> container.stop()
> {noformat}
> Will grow in memory and eventually crash. On debugging the reactor is never
> finalised.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PROTON-1030) Reactor never freed if handler/global_handler set
[
https://issues.apache.org/jira/browse/PROTON-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gordon Sim updated PROTON-1030:
---
Description:
E.g.
{noformat}
from proton.reactor import Reactor
class Print(object):
def on_unhandled(self, name, event):
print("%s %s" % (name, event))
while True:
reactor = Reactor()
reactor.handler = Print()
reactor.start()
reactor.stop()
{noformat}
Will grow in memory and eventually crash. On debugging the reactor is never
finalised.
was:
E.g.
{noformat}
from proton.reactor import Container, Reactor
class Print(object):
def on_unhandled(self, name, event):
print("%s %s" % (name, event))
while True:
container = Reactor()
container.handler = Print()
container.start()
container.stop()
{noformat}
Will grow in memory and eventually crash. On debugging the reactor is never
finalised.
> Reactor never freed if handler/global_handler set
> -
>
> Key: PROTON-1030
> URL: https://issues.apache.org/jira/browse/PROTON-1030
> Project: Qpid Proton
> Issue Type: Bug
> Components: python-binding
>Affects Versions: 0.10
>Reporter: Gordon Sim
> Fix For: 0.11
>
>
> E.g.
> {noformat}
> from proton.reactor import Reactor
> class Print(object):
> def on_unhandled(self, name, event):
> print("%s %s" % (name, event))
> while True:
> reactor = Reactor()
> reactor.handler = Print()
> reactor.start()
> reactor.stop()
> {noformat}
> Will grow in memory and eventually crash. On debugging the reactor is never
> finalised.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (PROTON-1028) BlockingConnection leaks due to cyclical reference
[
https://issues.apache.org/jira/browse/PROTON-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gordon Sim resolved PROTON-1028.
Resolution: Fixed
> BlockingConnection leaks due to cyclical reference
> --
>
> Key: PROTON-1028
> URL: https://issues.apache.org/jira/browse/PROTON-1028
> Project: Qpid Proton
> Issue Type: Bug
> Components: python-binding
>Affects Versions: 0.10
>Reporter: Gordon Sim
>Assignee: Gordon Sim
> Fix For: 0.11
>
>
> E.g. (whether or not a server is listening):
> {noformat}
> while True:
> sleep(0.1)
> try:
> conn = BlockingConnection("amqp://localhost")
> except ConnectionException, e:
> print e
> {noformat}
> This keeps increasing memory until eventually it core dumps.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PROTON-1026) Invalid queue/destination causes a segmentation fault
Otavio Rodolfo Piske created PROTON-1026: Summary: Invalid queue/destination causes a segmentation fault Key: PROTON-1026 URL: https://issues.apache.org/jira/browse/PROTON-1026 Project: Qpid Proton Issue Type: Bug Components: cpp-binding Affects Versions: 0.11 Environment: Fedora Linux 22, 64bit. Using built-in specs. COLLECT_GCC=/usr/bin/gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/5.1.1/lto-wrapper Target: x86_64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --disable-libgcj --with-default-libstdcxx-abi=c++98 --with-isl --enable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux Thread model: posix gcc version 5.1.1 20150618 (Red Hat 5.1.1-4) (GCC) Compiled with: cmake -DCMAKE_INSTALL_PREFIX=/opt/devel/qpid-proton-0.11-SNAPSHOT -DSYSINSTALL_BINDINGS=ON -DBUILD_PYTHON=OFF -DBUILD_PERL=OFF .. --- Reporter: Otavio Rodolfo Piske Assignee: Cliff Jansen Fix For: 0.11 Using an invalid path/destination in the address causes the code to crash with SIGSEGV. Having the QPid Proton code compiled, please use these teps to reproduce: 1. Configure a broker 2. Use the server example to send a message to a non-existent queue: ./server -a server_address:5672/this_destination_does_no_exist -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PROTON-1026) Invalid queue/destination causes a segmentation fault
[ https://issues.apache.org/jira/browse/PROTON-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otavio Rodolfo Piske updated PROTON-1026: - Attachment: proton-1026-backtrace.txt This attachment contains the back trace for the problem. > Invalid queue/destination causes a segmentation fault > - > > Key: PROTON-1026 > URL: https://issues.apache.org/jira/browse/PROTON-1026 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.11 > Environment: Fedora Linux 22, 64bit. > > Using built-in specs. > COLLECT_GCC=/usr/bin/gcc > COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/5.1.1/lto-wrapper > Target: x86_64-redhat-linux > Configured with: ../configure --enable-bootstrap > --enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto --prefix=/usr > --mandir=/usr/share/man --infodir=/usr/share/info > --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared > --enable-threads=posix --enable-checking=release --enable-multilib > --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions > --enable-gnu-unique-object --enable-linker-build-id > --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array > --disable-libgcj --with-default-libstdcxx-abi=c++98 --with-isl > --enable-libmpx --enable-gnu-indirect-function --with-tune=generic > --with-arch_32=i686 --build=x86_64-redhat-linux > Thread model: posix > gcc version 5.1.1 20150618 (Red Hat 5.1.1-4) (GCC) > > Compiled with: > cmake -DCMAKE_INSTALL_PREFIX=/opt/devel/qpid-proton-0.11-SNAPSHOT > -DSYSINSTALL_BINDINGS=ON -DBUILD_PYTHON=OFF -DBUILD_PERL=OFF .. > --- >Reporter: Otavio Rodolfo Piske >Assignee: Cliff Jansen > Fix For: 0.11 > > Attachments: proton-1026-backtrace.txt > > > Using an invalid path/destination in the address causes the code to crash > with SIGSEGV. > Having the QPid Proton code compiled, please use these teps to reproduce: > 1. Configure a broker > 2. Use the server example to send a message to a non-existent queue: > ./server -a server_address:5672/this_destination_does_no_exist -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PROTON-1021) ProtonE - the Deeply Embedded ProtonC variant
[
https://issues.apache.org/jira/browse/PROTON-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14966682#comment-14966682
]
German Shepherd (PrE) commented on PROTON-1021:
---
Got a bunch of responses and quite a positive feedback.
*To provide all the answers at once:*
1) The code will be delivered through a set of patches, attached to this JIRA
ticket.
2) The patches will directly apply to the latest ProtonC version, and the style
of the patching will include the "PROTONE" define.
{noformat}
#ifdef PROTON_E
specific variant of a piece of code (Proton E)
#else
original ProtonC code
#endif
{noformat}
3) To repeat the approach here - the ProtonE is a patched ProtonC. The core is
in the contents of the patches.
4) Is there a possibility to further squeeze the ProtonE on the RAM usage ?
Yes, you can save a few kB just by limiting the maximal message (payload) size.
The 32kB is the typical usage for one message in flight both ways, with 1.5kB
of the total payload size (~1.8kB of AMQP-on-wire size). To go even further -
this would require a fresh library to be written, which is possible, but this
won't be based on ProtonC any more. Such a library can be done, if the
optimizations are kept on ones mind, and I can image about 20kB of total RAM
usage in such a specific case (whole new own library). To be clear here now - I
do not have time to do anything like that (so I will not), since the ProtonE is
perfectly enough for most of the folks.
5) Microsoft did, very recently, release the IoT Hub C-code *wrapper* for
ProtonC, which works fine, but has a few drawbacks. The ProtonE is not here to
replace this piece of code, as you will need a wrapper (to call the Messenger
functions) anyway, regardless of your use of ProtonE or ProtonC. Obviously,
there is no requirement for deeply embedded systems applied (that is, the RAM
savings) - so should you want to use this wrapper, the need for ProtonE is yet
there, as well as the need to rewrite that very wrapper to actually be memory
savvy.
6) The license - I'm not going to add any licenses to any of the patches I'm
going to release. The ProtonC itself is open source under a specific license -
and I do not have any rights (and will not even attempt) to modify this
specific approach.
7) When it all will be available ? I will start posting the stuff slowly, and
hopefully fully finish on the very start of Q1 2016.
> ProtonE - the Deeply Embedded ProtonC variant
> -
>
> Key: PROTON-1021
> URL: https://issues.apache.org/jira/browse/PROTON-1021
> Project: Qpid Proton
> Issue Type: Improvement
> Components: proton-c
> Environment: Embedded C - IAR or GCC.
>Reporter: German Shepherd (PrE)
>Priority: Minor
> Labels: features
>
> Let me introduce a project I've started on 2013, when I've been tasked with a
> solution on how to connect an deeply embedded Wi-Fi enabled device to MS
> Azure Cloud (the plan is to connect millions of such devices).
> The obvious solution was to go with ServiceBus and AMQP over TLS. Since the
> embedded stuff is always written in plain C, the ProtonC is choice was
> obvious.
> The Proton-C typically requires a 230kB of RAM by default (most of it
> dynamically allocated), which is way too much of what is physically available
> in any embedded CPU of the general class (those typically end up with RAM
> sizes in 192kB area) - so there is no way on how to directly use the ProtonC
> on such CPUs.
> *Solution?*
> The outcome of my work is the *Proton-E*, based off the Proton-C library,
> squeezed to run on 32kB of total RAM usage (without sacrificing any
> functionality) and with a TLS on side taking addition approx. 11kB.
> This allows you to fit the AMQPS into any connected embedded CPU with 128kB
> of RAM (or more) - which most of the commonly available CPUs on market
> provide (for a great price).
> The "embedded" modifications are backwards applicable to the ProtonC, and
> this is what I would like to introduce through this JIRA Ticket.
> *Note*
> This work was picked up by Microsoft in mid-2014, and I have personally
> cooperated with them; but I have not seen any result being provided to
> community, as it seems the whole effort on their side vanished (and if not,
> they stopped the cooperation - so they won't have my latest sources).
> Never mind, as I have the most actual source code (as I've never stopped the
> development, and I do follow every commit the ProtonC community does), none
> is lost.
> At this moment, before the ProtonE source gets out, let me use this ticket to
> provide an assurance, that indeed the ProtonE (resp. modified ProtonC) can be
> used on the constrained system, and, most importantly, it is production ready
> for Azure Cloud.
> *ProtonE
