Re: After Autoreleasing Still Getting Leaked

[Marcus Karlsson]

On Jun 16, 2011, at 12:33 PM, Bing Li wrote:

> Dear Conrad, Jens, Tony, Scott, Wade and all,
> 
> I appreciate so much for your replies. I learn a lot from the interactions
> with you. Since I am new, your patience is so valuable to me!
> 
> I just got the problem. At least, right now, there is no the leak. When
> receiving messages, I put them into a queue. However, I forgot implementing
> a dealloc for it and I also did not release it when the connection was
> disconnected. After the bug was fixed, the leaking issue was solved.

Well done.

> But I still have a question. The queue is actually derived from NSArray. It
> should be empty after when the connection was disconnected. Why was there
> such a huge leak?

In your snippet that you sent earlier you allocated memory in an infinite loop. 
It's not uncommon that such loops keeps going for a large number of iterations 
when things goes wrong. Even if you leak just a little bit of data for each 
iteration, the sum of leaked memory can become very big.

> For the TCP issues, I planned to change my current solution. I will send the
> length of the buffer first and then send the real data. I have ever done
> that on .NET. The difference is that this time I need to exchange messages
> among different OSs (Java/Linux and iOS). So XML is used. On .NET, I just
> use its serialization technique. I am not sure if I need to consider little
> endian/big endian issues? My iMac is an Intel CPU. I don't need to do that,
> right? What about iPad/iPhone?

It's not uncommon that network protocols have the length included somewhere in 
the beginning. But it's not a bullet-proof solution and you still have to 
handle the possibility of getting fragmented data. If you're not dealing with 
persistent connections then maybe an even simpler approach could be to just 
keep reading until he remote socket is closed.

XML is already encoded so you don't need to convert it according to endianness. 
It's only when you're transmitting raw bytes directly over the network when you 
need to do that. As long as you have defined your protocol and how you encode 
and decode the transmitted data it doesn't matter which operating system or CPU 
architecture is on each end. The serialization you've used earlier is just one 
way of doing that. In the end it's just bits that goes over the wire, whether 
it's XML or your own binary format doesn't matter.

> When implementing multi-threading, NSOperationQueue is used in my system. I
> don't create threads explicitly unless in some specific cases, for example,
> a streaming control thread pool.
> 
> The major reason I intend to use sockets is that I attempt to design a P2P
> communication protocol when transmitting data over the Internet. I also did
> that successfully on .NET. I think a controllable socket is more flexible
> although its programming is difficult a little bit.

I'm not particularly familiar with NSOperationQueue and don't know how well it 
works for network operations. I often use the CFNetwork API:s which I recommend 
that you take a look at.

> I will learn Cocoa programming continually and carefully. Thanks so much for
> your help again!
> 
> Best regards,
> Bing
> 
> 
> 
> On Thu, Jun 16, 2011 at 4:10 AM, Conrad Shultz <
> con...@synthetiqsolutions.com> wrote:
> 
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>> 
>> On 6/15/11 12:50 PM, Bing Li wrote:
>>> Jens,
>>> 
>>> Thanks so much for your suggestions!
>>> 
>>> I wonder why it works fine according to Activity Monitor if such a huge
>> leak
>>> exists. The consumed memory in the Activity Monitor is stable and much
>>> smaller unless some threads are created at a high concurrent moment.
>> After
>>> the threads are dead, the consumed memory becomes stable and small. I
>> feel
>>> weird for this.
>> 
>> Honestly, I feel like you are not listening to the excellent responses
>> people are giving you.
>> 
>> 1) Activity Monitor is not a profiling tool.  Don't use it as such.  Use
>> Leaks/Allocations, maybe with some heapshot analysis thrown in as I
>> believe I mentioned a while back.
>> 
>> 2) Analyze your code with the Clang static analyzer ("Build & Analyze").
>> This will shake out many common memory issues (and more).
>> 
>> 3) Recognize that posting snippets of the code that you THINK might be
>> responsible for a leak does not mean that people on the list can
>> actually help you find it.  For example, even if you are doing
>> everything completely properly inside a function, if that function
>> returns some object, the calling code can still leak that object.
>> 
>> 4) It seems as if much of your code is multi-threaded.  All else being
>> equal, this makes such problems even harder to debug.  If I were having
>> such serious issues, I would probably spend some time trying to get the
>> task to work on the main thread and only once that is thoroughly
>> debugged would I break it up across threads.  (This won't work for every
>> type of problem, but if it can work 

Re: After Autoreleasing Still Getting Leaked

[Tony Romano]

No,  I meant to use packet since I was speaking in terms at the protocol
level.  In either case, your explanation is helpful to Bing but I fear he
is new to socket level programming and will stumble with other issues as
well.  Your advice to use some framework is what he should follow.

Tony Romano


On 6/16/11 12:29 PM, "Jens Alfke"  wrote:

>
>On Jun 15, 2011, at 7:01 PM, Tony Romano wrote:
>
>> TCP does NOT guarantee you will get the WHOLE PACKET on one receive
>>call.
>> BEFORE you PROCESS any data, you need to know that you have ALL the
>>data.
>> It may work MOST of the TIME, but there are times when it won't and your
>> code WILL FAIL.
>
>You¹re correct, but your point is a little confusing because you¹re using
>the word ³packet² to mean an application-level message. Since Œpacket¹
>already has a very specific meaning in IP networking, it¹s best to use a
>different word like ³message² or ³chunk² or ³blob² or something.
>
>So to rephrase: Just because you sent a chunk of bytes together in one
>TCP write, the recipient is not guaranteed to receive the same chunk of
>bytes in a single read. It¹s quite likely that this chunk will be
>combined with bytes written in other write calls, or split in half across
>two reads. TCP is taking the data you give it and shoving it into buffers
>in the kernel, and those buffers get split up and sent out as IP packets
>according to the kernel¹s whim and some complicated windowing algorithms.
>There¹s absolutely no relation between the groups of bytes you write, the
>IP packets that get sent, and the groups of bytes the client reads.
>
>If I can once again plug my MYNetwork framework
>, it includes a protocol called BLIP
>that supports sending application-level messages, which can also include
>metadata (key/value pairs like HTTP headers). It even supports
>multiplexing multiple messages at the same time, so one huge message
>won¹t clog up the socket and block others till it finishes. This stuff is
>a pain in the ass to get right and I recommend using someone¹s
>already-tested implementation over writing your own. (Or in other words,
>³I suffered for my art so you wouldn¹t have to² :)
>
>‹Jens


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Jens Alfke]

On Jun 15, 2011, at 7:01 PM, Tony Romano wrote:

> TCP does NOT guarantee you will get the WHOLE PACKET on one receive call.
> BEFORE you PROCESS any data, you need to know that you have ALL the data.
> It may work MOST of the TIME, but there are times when it won't and your
> code WILL FAIL.

You’re correct, but your point is a little confusing because you’re using the 
word “packet” to mean an application-level message. Since ‘packet’ already has 
a very specific meaning in IP networking, it’s best to use a different word 
like “message” or “chunk” or “blob” or something.

So to rephrase: Just because you sent a chunk of bytes together in one TCP 
write, the recipient is not guaranteed to receive the same chunk of bytes in a 
single read. It’s quite likely that this chunk will be combined with bytes 
written in other write calls, or split in half across two reads. TCP is taking 
the data you give it and shoving it into buffers in the kernel, and those 
buffers get split up and sent out as IP packets according to the kernel’s whim 
and some complicated windowing algorithms. There’s absolutely no relation 
between the groups of bytes you write, the IP packets that get sent, and the 
groups of bytes the client reads.

If I can once again plug my MYNetwork framework 
, it includes a protocol called BLIP that 
supports sending application-level messages, which can also include metadata 
(key/value pairs like HTTP headers). It even supports multiplexing multiple 
messages at the same time, so one huge message won’t clog up the socket and 
block others till it finishes. This stuff is a pain in the ass to get right and 
I recommend using someone’s already-tested implementation over writing your 
own. (Or in other words, “I suffered for my art so you wouldn’t have to” :)

—Jens

smime.p7s
Description: S/MIME cryptographic signature
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

Dear Marcus,

Thanks so much for your reply!

Your experiences on encoding on network helps me a lot!

NSOperationQueue is not a technique particularly for networking. It is an
asynchronous programming solution. Using it, developers are not required to
take care of a lot of details of threading.

Best regards,
Bing

On Thu, Jun 16, 2011 at 7:49 PM, Marcus Karlsson  wrote:

>
> On Jun 16, 2011, at 12:33 PM, Bing Li wrote:
>
> > Dear Conrad, Jens, Tony, Scott, Wade and all,
> >
> > I appreciate so much for your replies. I learn a lot from the
> interactions
> > with you. Since I am new, your patience is so valuable to me!
> >
> > I just got the problem. At least, right now, there is no the leak. When
> > receiving messages, I put them into a queue. However, I forgot
> implementing
> > a dealloc for it and I also did not release it when the connection was
> > disconnected. After the bug was fixed, the leaking issue was solved.
>
> Well done.
>
> > But I still have a question. The queue is actually derived from NSArray.
> It
> > should be empty after when the connection was disconnected. Why was there
> > such a huge leak?
>
> In your snippet that you sent earlier you allocated memory in an infinite
> loop. It's not uncommon that such loops keeps going for a large number of
> iterations when things goes wrong. Even if you leak just a little bit of
> data for each iteration, the sum of leaked memory can become very big.
>
> > For the TCP issues, I planned to change my current solution. I will send
> the
> > length of the buffer first and then send the real data. I have ever done
> > that on .NET. The difference is that this time I need to exchange
> messages
> > among different OSs (Java/Linux and iOS). So XML is used. On .NET, I just
> > use its serialization technique. I am not sure if I need to consider
> little
> > endian/big endian issues? My iMac is an Intel CPU. I don't need to do
> that,
> > right? What about iPad/iPhone?
>
> It's not uncommon that network protocols have the length included somewhere
> in the beginning. But it's not a bullet-proof solution and you still have to
> handle the possibility of getting fragmented data. If you're not dealing
> with persistent connections then maybe an even simpler approach could be to
> just keep reading until he remote socket is closed.
>
> XML is already encoded so you don't need to convert it according to
> endianness. It's only when you're transmitting raw bytes directly over the
> network when you need to do that. As long as you have defined your protocol
> and how you encode and decode the transmitted data it doesn't matter which
> operating system or CPU architecture is on each end. The serialization
> you've used earlier is just one way of doing that. In the end it's just bits
> that goes over the wire, whether it's XML or your own binary format doesn't
> matter.
>
> > When implementing multi-threading, NSOperationQueue is used in my system.
> I
> > don't create threads explicitly unless in some specific cases, for
> example,
> > a streaming control thread pool.
> >
> > The major reason I intend to use sockets is that I attempt to design a
> P2P
> > communication protocol when transmitting data over the Internet. I also
> did
> > that successfully on .NET. I think a controllable socket is more flexible
> > although its programming is difficult a little bit.
>
> I'm not particularly familiar with NSOperationQueue and don't know how well
> it works for network operations. I often use the CFNetwork API:s which I
> recommend that you take a look at.
>
> > I will learn Cocoa programming continually and carefully. Thanks so much
> for
> > your help again!
> >
> > Best regards,
> > Bing
> >
> >
> >
> > On Thu, Jun 16, 2011 at 4:10 AM, Conrad Shultz <
> > con...@synthetiqsolutions.com> wrote:
> >
> >> -BEGIN PGP SIGNED MESSAGE-
> >> Hash: SHA1
> >>
> >> On 6/15/11 12:50 PM, Bing Li wrote:
> >>> Jens,
> >>>
> >>> Thanks so much for your suggestions!
> >>>
> >>> I wonder why it works fine according to Activity Monitor if such a huge
> >> leak
> >>> exists. The consumed memory in the Activity Monitor is stable and much
> >>> smaller unless some threads are created at a high concurrent moment.
> >> After
> >>> the threads are dead, the consumed memory becomes stable and small. I
> >> feel
> >>> weird for this.
> >>
> >> Honestly, I feel like you are not listening to the excellent responses
> >> people are giving you.
> >>
> >> 1) Activity Monitor is not a profiling tool.  Don't use it as such.  Use
> >> Leaks/Allocations, maybe with some heapshot analysis thrown in as I
> >> believe I mentioned a while back.
> >>
> >> 2) Analyze your code with the Clang static analyzer ("Build & Analyze").
> >> This will shake out many common memory issues (and more).
> >>
> >> 3) Recognize that posting snippets of the code that you THINK might be
> >> responsible for a leak does not mean that people on the list can
> >> actually help you find it.  For example, even if you

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

Dear Conrad, Jens, Tony, Scott, Wade and all,

I appreciate so much for your replies. I learn a lot from the interactions
with you. Since I am new, your patience is so valuable to me!

I just got the problem. At least, right now, there is no the leak. When
receiving messages, I put them into a queue. However, I forgot implementing
a dealloc for it and I also did not release it when the connection was
disconnected. After the bug was fixed, the leaking issue was solved.

But I still have a question. The queue is actually derived from NSArray. It
should be empty after when the connection was disconnected. Why was there
such a huge leak?

For the TCP issues, I planned to change my current solution. I will send the
length of the buffer first and then send the real data. I have ever done
that on .NET. The difference is that this time I need to exchange messages
among different OSs (Java/Linux and iOS). So XML is used. On .NET, I just
use its serialization technique. I am not sure if I need to consider little
endian/big endian issues? My iMac is an Intel CPU. I don't need to do that,
right? What about iPad/iPhone?

When implementing multi-threading, NSOperationQueue is used in my system. I
don't create threads explicitly unless in some specific cases, for example,
a streaming control thread pool.

The major reason I intend to use sockets is that I attempt to design a P2P
communication protocol when transmitting data over the Internet. I also did
that successfully on .NET. I think a controllable socket is more flexible
although its programming is difficult a little bit.

I will learn Cocoa programming continually and carefully. Thanks so much for
your help again!

Best regards,
Bing



On Thu, Jun 16, 2011 at 4:10 AM, Conrad Shultz <
con...@synthetiqsolutions.com> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 6/15/11 12:50 PM, Bing Li wrote:
> > Jens,
> >
> > Thanks so much for your suggestions!
> >
> > I wonder why it works fine according to Activity Monitor if such a huge
> leak
> > exists. The consumed memory in the Activity Monitor is stable and much
> > smaller unless some threads are created at a high concurrent moment.
> After
> > the threads are dead, the consumed memory becomes stable and small. I
> feel
> > weird for this.
>
> Honestly, I feel like you are not listening to the excellent responses
> people are giving you.
>
> 1) Activity Monitor is not a profiling tool.  Don't use it as such.  Use
> Leaks/Allocations, maybe with some heapshot analysis thrown in as I
> believe I mentioned a while back.
>
> 2) Analyze your code with the Clang static analyzer ("Build & Analyze").
>  This will shake out many common memory issues (and more).
>
> 3) Recognize that posting snippets of the code that you THINK might be
> responsible for a leak does not mean that people on the list can
> actually help you find it.  For example, even if you are doing
> everything completely properly inside a function, if that function
> returns some object, the calling code can still leak that object.
>
> 4) It seems as if much of your code is multi-threaded.  All else being
> equal, this makes such problems even harder to debug.  If I were having
> such serious issues, I would probably spend some time trying to get the
> task to work on the main thread and only once that is thoroughly
> debugged would I break it up across threads.  (This won't work for every
> type of problem, but if it can work for you, I'd do it.)
>
> 5) As others have commented, there is substantial evidence that your
> code probably has issues other than memory management.  If you are, for
> example, smashing the stack in your C code, all bets are off and you
> really need to fix that first.
>
> It's been so long since you first started posting that I've forgotten
> what you are trying to do.  You are communicating in a high level format
> (XML), yet all your code seems to concern very low level network
> behaviors (opening sockets, reading bytes, et cetera).  I wonder whether
> you are over-engineering all this.  What is your goal?
>
> - --
> Conrad Shultz
>
> Synthetiq Solutions
> www.synthetiqsolutions.com
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.7 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iD8DBQFN+RG8aOlrz5+0JdURAjRNAJ9YCKwfHbB6iICdmKARupttOoJkMQCdG5TT
> ZAaJuogqgVGHjKxS4sqA8pw=
> =RkC+
> -END PGP SIGNATURE-
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Scott Ribe]

On Jun 15, 2011, at 8:01 PM, Tony Romano wrote:

> 2. You need to know how many bytes any particular packet contains.
> Usually there is some operation code pair with the number of byte to
> expect for the operation.

That's certainly the way I prefer, but it is also possible to use a marker at 
the end of the data, and collect until you hit the marker.

-- 
Scott Ribe
scott_r...@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Tony Romano]

I state a point slightly different to Bing about the TCP issue.

TCP does NOT guarantee you will get the WHOLE PACKET on one receive call.
BEFORE you PROCESS any data, you need to know that you have ALL the data.
It may work MOST of the TIME, but there are times when it won't and your
code WILL FAIL.

When you code at the socket level, you need to be robust and know 2 Major
pieces of data.

1. Is this packet for you?  Anyone can send you data on that socket but
you need to make sure the packet is something you understand.  The classic
way is to wrap the data with a header with some magic number unique to
you.  Checking for the magic number helps signal your code, this is
something you can understand.

2. You need to know how many bytes any particular packet contains.
Usually there is some operation code pair with the number of byte to
expect for the operation.  These two additional pieces of information tell
you how much more you need to read and what operation to do on that data.

Once you have a good packet, then you can process it correctly.  There are
other things you need to do to make it robust, but these are the basics.

Note, if you have only type of data between your clients, you may not need
the operation code.

I have no idea if your run method is in a thread(you better figure out how
select works). If it is, the method notifyMessage, is a black box and I
have no idea what happens to the string there.  Is it retained, released,
ignored, no clue.  Your leak is not in the run method.  As I and others
have said before, countless times. DON¹T USE LEAKS!  It's like calling the
fire dept and saying there is a fire and NOT telling them where it is.

I'm not trying to be harsh, countless people are trying to help and they
are not convinced you are listening :-).
Tony Romano



On 6/15/11 2:20 PM, "Scott Ribe"  wrote:

>On Jun 15, 2011, at 2:30 PM, Jens Alfke wrote:
>
>> On Jun 15, 2011, at 1:26 PM, Bing Li wrote:
>> 
>>> Actually, I attempt to design P2P system using Cocoa. Meanwhile, the
>>>peer on Mac OS X must communicate with some Java systems. So I need to
>>>use sockets and transmit XML.
>> 
>> Sorry to be blunt, but it¹s clear from this answer (and others) that
>>you¹re not reading the suggestions people are giving you. All you
>>responded to here was one minor question at the end of Conrad¹s message,
>>not the numbered list of five major issues.
>
>Not 100% sure, but I believe he may already have been told that he
>doesn't need to use low-level sockets just because he's communicating
>with Java...
>
>Anyway, to Bing, my last advice:
>
>- You have to follow proper memory management everywhere, not just where
>you're originally allocating objects. As has been pointed out to you
>multiple times now, your leak is not coming in the code snippet you
>posted. It's happening elsewhere, in code that uses the return value, so
>that's what you have to look at, what the consumers of that returned
>value do with it.
>
>- Given that you had two stack-smashing bugs in one small snippet of C,
>and other oddities in the code, I think you really need to read a basic
>introductory text on C.
>
>- You also don't seem to really understand how sockets work. Perhaps I'm
>wrong here, and your receivedMessage method does everything it would need
>to do in terms of assembling message fragments, if only you weren't
>potentially messing up strings because of boundary issues before you ever
>called it. (In other words, if you just appended to NSData or some such
>until you had a whole message.) But from the overall sense of your code
>and your questions, I really doubt it. So an introductory text on network
>programming is also in order (Stevens is the classic on this subject). Or
>you could use higher-level network APIs, to which I believe you were
>referred a long time ago: CFSocket, maybe NSURL depending on the
>structure of the messages you're exchanging with Java.
>
>
>-- 
>Scott Ribe
>scott_r...@elevated-dev.com
>http://www.elevated-dev.com/
>(303) 722-0567 voice
>
>
>
>
>___
>
>Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
>
>Please do not post admin requests or moderator comments to the list.
>Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>
>Help/Unsubscribe/Update your Subscription:
>http://lists.apple.com/mailman/options/cocoa-dev/tonyrom%40hotmail.com
>
>This email sent to tony...@hotmail.com
>


___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Scott Ribe]

On Jun 15, 2011, at 2:30 PM, Jens Alfke wrote:

> On Jun 15, 2011, at 1:26 PM, Bing Li wrote:
> 
>> Actually, I attempt to design P2P system using Cocoa. Meanwhile, the peer on 
>> Mac OS X must communicate with some Java systems. So I need to use sockets 
>> and transmit XML.
> 
> Sorry to be blunt, but it’s clear from this answer (and others) that you’re 
> not reading the suggestions people are giving you. All you responded to here 
> was one minor question at the end of Conrad’s message, not the numbered list 
> of five major issues.

Not 100% sure, but I believe he may already have been told that he doesn't need 
to use low-level sockets just because he's communicating with Java...

Anyway, to Bing, my last advice:

- You have to follow proper memory management everywhere, not just where you're 
originally allocating objects. As has been pointed out to you multiple times 
now, your leak is not coming in the code snippet you posted. It's happening 
elsewhere, in code that uses the return value, so that's what you have to look 
at, what the consumers of that returned value do with it.

- Given that you had two stack-smashing bugs in one small snippet of C, and 
other oddities in the code, I think you really need to read a basic 
introductory text on C.

- You also don't seem to really understand how sockets work. Perhaps I'm wrong 
here, and your receivedMessage method does everything it would need to do in 
terms of assembling message fragments, if only you weren't potentially messing 
up strings because of boundary issues before you ever called it. (In other 
words, if you just appended to NSData or some such until you had a whole 
message.) But from the overall sense of your code and your questions, I really 
doubt it. So an introductory text on network programming is also in order 
(Stevens is the classic on this subject). Or you could use higher-level network 
APIs, to which I believe you were referred a long time ago: CFSocket, maybe 
NSURL depending on the structure of the messages you're exchanging with Java.


-- 
Scott Ribe
scott_r...@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

Dear Jens and Conrad,

I appreciate so much for your help!

Actually, I am still new to program Cocoa. My past experiences are more high
level languages such as C# and Java. So I am not familiar with memory
management although I have read the documents. I will follow your
suggestions and reread the documents to fix the bug.

Best regards,
Bing

On Thu, Jun 16, 2011 at 4:30 AM, Jens Alfke  wrote:

>
> On Jun 15, 2011, at 1:26 PM, Bing Li wrote:
>
> > Actually, I attempt to design P2P system using Cocoa. Meanwhile, the peer
> on Mac OS X must communicate with some Java systems. So I need to use
> sockets and transmit XML.
>
> Sorry to be blunt, but it’s clear from this answer (and others) that you’re
> not reading the suggestions people are giving you. All you responded to here
> was one minor question at the end of Conrad’s message, not the numbered list
> of five major issues.
>
> This does not make me feel that it’s worthwhile to keep giving more advice.
> Best of luck with your project.
>
> —Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Jens Alfke]

On Jun 15, 2011, at 1:26 PM, Bing Li wrote:

> Actually, I attempt to design P2P system using Cocoa. Meanwhile, the peer on 
> Mac OS X must communicate with some Java systems. So I need to use sockets 
> and transmit XML.

Sorry to be blunt, but it’s clear from this answer (and others) that you’re not 
reading the suggestions people are giving you. All you responded to here was 
one minor question at the end of Conrad’s message, not the numbered list of 
five major issues.

This does not make me feel that it’s worthwhile to keep giving more advice. 
Best of luck with your project.

—Jens

smime.p7s
Description: S/MIME cryptographic signature
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

Dear Conrad,

I appreciate so much for your suggestions!

Actually, I attempt to design P2P system using Cocoa. Meanwhile, the peer on
Mac OS X must communicate with some Java systems. So I need to use sockets
and transmit XML.

Best regards,
Bing

On Thu, Jun 16, 2011 at 4:10 AM, Conrad Shultz <
con...@synthetiqsolutions.com> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 6/15/11 12:50 PM, Bing Li wrote:
> > Jens,
> >
> > Thanks so much for your suggestions!
> >
> > I wonder why it works fine according to Activity Monitor if such a huge
> leak
> > exists. The consumed memory in the Activity Monitor is stable and much
> > smaller unless some threads are created at a high concurrent moment.
> After
> > the threads are dead, the consumed memory becomes stable and small. I
> feel
> > weird for this.
>
> Honestly, I feel like you are not listening to the excellent responses
> people are giving you.
>
> 1) Activity Monitor is not a profiling tool.  Don't use it as such.  Use
> Leaks/Allocations, maybe with some heapshot analysis thrown in as I
> believe I mentioned a while back.
>
> 2) Analyze your code with the Clang static analyzer ("Build & Analyze").
>  This will shake out many common memory issues (and more).
>
> 3) Recognize that posting snippets of the code that you THINK might be
> responsible for a leak does not mean that people on the list can
> actually help you find it.  For example, even if you are doing
> everything completely properly inside a function, if that function
> returns some object, the calling code can still leak that object.
>
> 4) It seems as if much of your code is multi-threaded.  All else being
> equal, this makes such problems even harder to debug.  If I were having
> such serious issues, I would probably spend some time trying to get the
> task to work on the main thread and only once that is thoroughly
> debugged would I break it up across threads.  (This won't work for every
> type of problem, but if it can work for you, I'd do it.)
>
> 5) As others have commented, there is substantial evidence that your
> code probably has issues other than memory management.  If you are, for
> example, smashing the stack in your C code, all bets are off and you
> really need to fix that first.
>
> It's been so long since you first started posting that I've forgotten
> what you are trying to do.  You are communicating in a high level format
> (XML), yet all your code seems to concern very low level network
> behaviors (opening sockets, reading bytes, et cetera).  I wonder whether
> you are over-engineering all this.  What is your goal?
>
> - --
> Conrad Shultz
>
> Synthetiq Solutions
> www.synthetiqsolutions.com
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.7 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iD8DBQFN+RG8aOlrz5+0JdURAjRNAJ9YCKwfHbB6iICdmKARupttOoJkMQCdG5TT
> ZAaJuogqgVGHjKxS4sqA8pw=
> =RkC+
> -END PGP SIGNATURE-
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Conrad Shultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 6/15/11 12:50 PM, Bing Li wrote:
> Jens,
> 
> Thanks so much for your suggestions!
> 
> I wonder why it works fine according to Activity Monitor if such a huge leak
> exists. The consumed memory in the Activity Monitor is stable and much
> smaller unless some threads are created at a high concurrent moment. After
> the threads are dead, the consumed memory becomes stable and small. I feel
> weird for this.

Honestly, I feel like you are not listening to the excellent responses
people are giving you.

1) Activity Monitor is not a profiling tool.  Don't use it as such.  Use
Leaks/Allocations, maybe with some heapshot analysis thrown in as I
believe I mentioned a while back.

2) Analyze your code with the Clang static analyzer ("Build & Analyze").
 This will shake out many common memory issues (and more).

3) Recognize that posting snippets of the code that you THINK might be
responsible for a leak does not mean that people on the list can
actually help you find it.  For example, even if you are doing
everything completely properly inside a function, if that function
returns some object, the calling code can still leak that object.

4) It seems as if much of your code is multi-threaded.  All else being
equal, this makes such problems even harder to debug.  If I were having
such serious issues, I would probably spend some time trying to get the
task to work on the main thread and only once that is thoroughly
debugged would I break it up across threads.  (This won't work for every
type of problem, but if it can work for you, I'd do it.)

5) As others have commented, there is substantial evidence that your
code probably has issues other than memory management.  If you are, for
example, smashing the stack in your C code, all bets are off and you
really need to fix that first.

It's been so long since you first started posting that I've forgotten
what you are trying to do.  You are communicating in a high level format
(XML), yet all your code seems to concern very low level network
behaviors (opening sockets, reading bytes, et cetera).  I wonder whether
you are over-engineering all this.  What is your goal?

- -- 
Conrad Shultz

Synthetiq Solutions
www.synthetiqsolutions.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFN+RG8aOlrz5+0JdURAjRNAJ9YCKwfHbB6iICdmKARupttOoJkMQCdG5TT
ZAaJuogqgVGHjKxS4sqA8pw=
=RkC+
-END PGP SIGNATURE-
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

Jens,

Thanks so much for your suggestions!

I wonder why it works fine according to Activity Monitor if such a huge leak
exists. The consumed memory in the Activity Monitor is stable and much
smaller unless some threads are created at a high concurrent moment. After
the threads are dead, the consumed memory becomes stable and small. I feel
weird for this.

Best,
Bing

On Thu, Jun 16, 2011 at 3:39 AM, Jens Alfke  wrote:

>
> On Jun 15, 2011, at 12:34 PM, Bing Li wrote:
>
> > Yes, TCP has the boundary issue. So I put a "\n" at the end of each XML.
> And there is no "\n" within any XML.
>
> That has nothing to do with the issue of UTF-8 sequences being split across
> reads.
>
> In UTF-8, non-ASCII characters are represented as multi-byte sequences
> (where each byte has the high bit set.) If the message is broken into
> packets such that one packet ends partway through such a sequence and the
> next packet begins with the rest of it, then neither packet’s contents will
> be parseable by itself as UTF-8 and the NSStrings you get will be nil. So
> you’ll lose data.
>
> > However, it should not be the reason to cause memory leaks, right?
>
> No, but it sounds like your code has bigger problems than memory leaks, so
> you probably shouldn’t worry about those yet. (And I believe we’ve already
> told you what you need to know about tracking down the leaks.)
>
> —Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Jens Alfke]

On Jun 15, 2011, at 12:34 PM, Bing Li wrote:

> Yes, TCP has the boundary issue. So I put a "\n" at the end of each XML. And 
> there is no "\n" within any XML.

That has nothing to do with the issue of UTF-8 sequences being split across 
reads.

In UTF-8, non-ASCII characters are represented as multi-byte sequences (where 
each byte has the high bit set.) If the message is broken into packets such 
that one packet ends partway through such a sequence and the next packet begins 
with the rest of it, then neither packet’s contents will be parseable by itself 
as UTF-8 and the NSStrings you get will be nil. So you’ll lose data.

> However, it should not be the reason to cause memory leaks, right?

No, but it sounds like your code has bigger problems than memory leaks, so you 
probably shouldn’t worry about those yet. (And I believe we’ve already told you 
what you need to know about tracking down the leaks.)

—Jens

smime.p7s
Description: S/MIME cryptographic signature
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

OK. That's a good solution to XML transmission.

However, it should not be the reason to cause memory leaks, right?

On Thu, Jun 16, 2011 at 3:33 AM, Jens Alfke  wrote:

>
> On Jun 15, 2011, at 12:24 PM, Bing Li wrote:
>
> > The string can be any length. I put the received strings into a queue and
> > another thread gets the strings out of the queue and parse there.
> Actually,
> > the string is XML. During the test, the string length is always 259.
>
> If it’s XML, you should avoid converting the data into strings yourself,
> and just pass the raw bytes into the XML parser. That will fix the problem
> with UTF-8 sequences being split across reads, and it also allows the XML
> parser to use the metadata in the file to decide what encoding to use.
>
> —Jens
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Jens Alfke]

On Jun 15, 2011, at 12:04 PM, Bing Li wrote:

> I test the program just on a single Mac machine using TCP, i.e., both the 
> client and the server are located on the same machine. So I think it is 
> impossible that the data is corrupted during the transmission.

The UTF-8 issue isn’t something that’s causing the problems you’re seeing; 
we’re bringing it up as a potential problem that could bite you in real 
deployment.

—Jens

smime.p7s
Description: S/MIME cryptographic signature
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

Scott,

Yes, TCP has the boundary issue. So I put a "\n" at the end of each XML. And
there is no "\n" within any XML.

Thanks!
Bing

On Thu, Jun 16, 2011 at 3:13 AM, Scott Ribe wrote:

> On Jun 15, 2011, at 1:04 PM, Bing Li wrote:
>
> > I test the program just on a single Mac machine using TCP, i.e., both the
> client and the server are located on the same machine. So I think it is
> impossible that the data is corrupted during the transmission.
>
> Correction to my prior post: even if all your strings are < 1024 bytes, you
> still have no guarantee that they won't be broken in the middle of a
> multi-byte sequence. On the sending end, multiple short strings might get
> packed into a single packet > 1024 bytes. Your receiving end is not going to
> be reliable unless you only use 7-bit ASCII--and even then you need the
> appropriate logic in receivedMessage to recognize message boundaries, append
> bytes to a buffer until you have a full message, and so on.
>
> --
> Scott Ribe
> scott_r...@elevated-dev.com
> http://www.elevated-dev.com/
> (303) 722-0567 voice
>
>
>
>
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Jens Alfke]

On Jun 15, 2011, at 12:24 PM, Bing Li wrote:

> The string can be any length. I put the received strings into a queue and
> another thread gets the strings out of the queue and parse there. Actually,
> the string is XML. During the test, the string length is always 259.

If it’s XML, you should avoid converting the data into strings yourself, and 
just pass the raw bytes into the XML parser. That will fix the problem with 
UTF-8 sequences being split across reads, and it also allows the XML parser to 
use the metadata in the file to decide what encoding to use.

—Jens

smime.p7s
Description: S/MIME cryptographic signature
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

The string can be any length. I put the received strings into a queue and
another thread gets the strings out of the queue and parse there. Actually,
the string is XML. During the test, the string length is always 259.

I also wonder why it works fine according to Activity Monitor if such a huge
leak exists. The consumed memory in the Activity Monitor is stable and much
smaller unless some threads are created.

Thanks,
Bing

On Thu, Jun 16, 2011 at 3:07 AM, Scott Ribe wrote:

> On Jun 15, 2011, at 1:04 PM, Bing Li wrote:
>
> > I test the program just on a single Mac machine using TCP, i.e., both the
> client and the server are located on the same machine. So I think it is
> impossible that the data is corrupted during the transmission.
>
> Yes, but it is possible to get a buffer that contains a break in the middle
> of a multi-byte sequence, and is therefore invalid UTF-8, unless your
> strings are never > 1024 bytes in length.
>
> --
> Scott Ribe
> scott_r...@elevated-dev.com
> http://www.elevated-dev.com/
> (303) 722-0567 voice
>
>
>
>
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Wade Williams]

Bing,

That's not the point.  The point is a program has to be ready to handle the 
possibility of corrupted, truncated or fragmented data.

Writing a client-server program, particularly at the socket level, is not easy 
for that very reason.  If you fail to take the possibilities into account, you 
may well be coming back to the list saying "my program is crashing and I don't 
know why."

Wade
On Jun 15, 2011, at 2:04 PM, Bing Li wrote:

> I test the program just on a single Mac machine using TCP, i.e., both the
> client and the server are located on the same machine. So I think it is
> impossible that the data is corrupted during the transmission.
> 
> Thanks so much!
> Bing
> 
> On Thu, Jun 16, 2011 at 2:57 AM, Scott Ribe 
> wrote:
> 
>> On Jun 15, 2011, at 12:38 PM, Jens Alfke wrote:
>> 
>>> Also also, be aware that creating an NSString from UTF-8 data can result
>> in a nil string if the data is not valid UTF-8. This can happen if the
>> packet gets corrupted in transit or if something else is accidentally or
>> maliciously sending unexpected data to your port. To be robust, your code
>> should detect this and (probably) just ignore the packet.
>> 
>> So, come to think of it, what if the packet breaks in the middle of a
>> multi-byte sequence?
>> 
>> --
>> Scott Ribe
>> scott_r...@elevated-dev.com
>> http://www.elevated-dev.com/
>> (303) 722-0567 voice
>> 
>> 
>> 
>> 
>> 
> ___
> 
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
> 
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
> 
> Help/Unsubscribe/Update your Subscription:
> http://lists.apple.com/mailman/options/cocoa-dev/wadesworld%40mac.com
> 
> This email sent to wadeswo...@mac.com

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Scott Ribe]

On Jun 15, 2011, at 1:04 PM, Bing Li wrote:

> I test the program just on a single Mac machine using TCP, i.e., both the 
> client and the server are located on the same machine. So I think it is 
> impossible that the data is corrupted during the transmission.

Correction to my prior post: even if all your strings are < 1024 bytes, you 
still have no guarantee that they won't be broken in the middle of a multi-byte 
sequence. On the sending end, multiple short strings might get packed into a 
single packet > 1024 bytes. Your receiving end is not going to be reliable 
unless you only use 7-bit ASCII--and even then you need the appropriate logic 
in receivedMessage to recognize message boundaries, append bytes to a buffer 
until you have a full message, and so on.

-- 
Scott Ribe
scott_r...@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

I test the program just on a single Mac machine using TCP, i.e., both the
client and the server are located on the same machine. So I think it is
impossible that the data is corrupted during the transmission.

Thanks so much!
Bing

On Thu, Jun 16, 2011 at 2:57 AM, Scott Ribe wrote:

> On Jun 15, 2011, at 12:38 PM, Jens Alfke wrote:
>
> > Also also, be aware that creating an NSString from UTF-8 data can result
> in a nil string if the data is not valid UTF-8. This can happen if the
> packet gets corrupted in transit or if something else is accidentally or
> maliciously sending unexpected data to your port. To be robust, your code
> should detect this and (probably) just ignore the packet.
>
> So, come to think of it, what if the packet breaks in the middle of a
> multi-byte sequence?
>
> --
> Scott Ribe
> scott_r...@elevated-dev.com
> http://www.elevated-dev.com/
> (303) 722-0567 voice
>
>
>
>
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Scott Ribe]

On Jun 15, 2011, at 12:38 PM, Jens Alfke wrote:

> Also also, be aware that creating an NSString from UTF-8 data can result in a 
> nil string if the data is not valid UTF-8. This can happen if the packet gets 
> corrupted in transit or if something else is accidentally or maliciously 
> sending unexpected data to your port. To be robust, your code should detect 
> this and (probably) just ignore the packet.

So, come to think of it, what if the packet breaks in the middle of a 
multi-byte sequence?

-- 
Scott Ribe
scott_r...@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Jens Alfke]

What Scott said.

Also, have you tried the "Build & Analyze" command in Xcode 3 (“Analyze” in 
Xcode 4)? It can detect a lot of common refcounting errors at compile time.

Also also, be aware that creating an NSString from UTF-8 data can result in a 
nil string if the data is not valid UTF-8. This can happen if the packet gets 
corrupted in transit or if something else is accidentally or maliciously 
sending unexpected data to your port. To be robust, your code should detect 
this and (probably) just ignore the packet.

—Jens

smime.p7s
Description: S/MIME cryptographic signature
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Scott Ribe]

On Jun 15, 2011, at 11:38 AM, Bing Li wrote:

> According to your suggestions, the code is changed as follows. But the leaks 
> are still notified by Instruments. But in Activity Monitor, the program works 
> fine. I feel weird about that. Could you figure out the problem? Thanks!

OK, 1st, your buffer length logic is now correct. But I don't think you need 
the 0 at the end of the buffer for any reason. You can just have a 1024 byte 
buffer, and read up to 1024 bytes into it.

2nd, this method looks correct to me. So in order to leak the message, you're 
leaking it somewhere else. As with your prior question and its answers, 
instruments only tells you where the leaked object was allocated. It cannot 
tell you where the bug is. I would guess that somewhere early in 
notifyMessageReceived: you retain the string, and then you do not ever release 
it. If you're going to keep it around (especially if you're going to send it to 
another thread), you need to retain it, so that part is likely correct. 
(Although if you happen to process it and are done with it before 
notifyMessageReceived: returns then you don't need to retain it.) So then the 
issue is that you don't release/autorelease it when you're done with it. But 
that's just my guess. It could just as well be that where you should retain it, 
you do so inadvertently in two different places, then only release once. Or all 
sorts of other combinations of errors. The point is, somewhere later on you're 
not following the memory management rules with regard to ownership of the 
receivedMessage string.

-- 
Scott Ribe
scott_r...@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Bing Li]

Dear Scott, Gary and all,

I appreciate so much for your replies! I am not a good C programmer. So I
made some mistakes in the code. C is used only when programming with BSD
sockets.

According to your suggestions, the code is changed as follows. But the leaks
are still notified by Instruments. But in Activity Monitor, the program
works fine. I feel weird about that. Could you figure out the problem?
Thanks!

Best regards,
Bing

- (void) run
{
char buffer[1025];
ssize_t numberBytesReceived;
while (true)
{
NSAutoreleasePool *loopPool = [[NSAutoreleasePool alloc]
init];
numberBytesReceived = recv(clientSocket, buffer, 1024, 0);
if (numberBytesReceived > 0)
{
buffer[numberBytesReceived] = '\0';
NSAutoreleasePool *subLoopPool = [[NSAutoreleasePool
alloc] init];

// This line sometimes gets huge leaks (about
167.80MB, 226933 leaks). But it does not always happen
// I don't understand why it gets leaked!
NSMutableString *receivedMessage =
[[[NSMutableString alloc] initWithBytes:buffer length:numberBytesReceived
encoding:NSUTF8StringEncoding] autorelease];

[self notifyMessageReceived:receivedMessage];
[subLoopPool drain];
}
else
{
[loopPool drain];
break;
}
[loopPool drain];
}
}



On Thu, Jun 16, 2011 at 12:21 AM, Scott Ribe wrote:

>
> On Jun 15, 2011, at 10:02 AM, Bing Li wrote:
>
> >char buffer[1024];
> >ssize_t numberBytesReceived;
> >while (true)
> >{
> >NSAutoreleasePool *loopPool = [[NSAutoreleasePool alloc]
> > init];
> >numberBytesReceived = recv(clientSocket, buffer, 1024, 0);
> >if (numberBytesReceived > 0)
> >{
> >buffer[numberBytesReceived] = '\0';
>
> So, if you receive 1024 bytes, you write past the end of the buffer. Now
> granted, you're only writing a 0 onto the least significant (assuming Intel)
> byte of numberBytesReceived, so it's not the source of your leak, but it's a
> pretty bad bug that's going to mess up your buffer contents for any buffer
> that's not a multiple of 256 bytes in length.
>
> >for (int i = 0; i < 1024; i ++)
> >{
> >buffer[i] = '\0';
> >}
>
> That's pretty pointless isn't it?
>
> >if (buffer[0] != '\0')
> >{
> >free(buffer);
> >}
>
>
> Seriously??? Freeing a stack buffer? Ouch. And the test? Why would you
> decide whether to free the block based on its contents?
>
> I don't see offhand anything in this method that would cause a leak. But if
> this code is typical of the rest of the program, then you have bugs all over
> the place that corrupt memory which can lead to all sorts of things
> failing--including memory reclamation.
>
> --
> Scott Ribe
> scott_r...@elevated-dev.com
> http://www.elevated-dev.com/
> (303) 722-0567 voice
>
>
>
>
>
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Scott Ribe]

On Jun 15, 2011, at 10:22 AM, Bing Li wrote:

> In addition, according to Activity Monitor, the memory consumed by the
> program increased only when some new threads are added. After the threads
> were dead, the memory was lowered to a fixed amount. Is it fine?

Each thread has data structures associated with it, besides any memory you may 
allocate within the thread, which cannot be freed until the thread terminates 
(the thread's stack for instance). So it's a question of how much the memory 
went up. If you want to find out if it's system overhead or your allocations, 
you could experiment with creating some "empty" threads that just sleep() long 
enough to give you a chance to watch them, and compare the memory usage to what 
you see with your real threads.

-- 
Scott Ribe
scott_r...@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Gary L. Wade]

There are so many fundamental issues with your coding. Rather than just 
throwing solutions at perceived problems, I would recommend you step back and 
document your justification for every line of code you write. In order to 
learn, you should consider this exercise to be descriptive enough to explain to 
a non-programmer.

- Gary L. Wade (Sent from my iPhone)

On Jun 15, 2011, at 9:02 AM, Bing Li  wrote:

> Dear all,
> 
> I got a problem which confused me. The following method is put into
> NSOperationQueue so that it can run asynchronously. The method received data
> from a remote node and then converted the bytes to NSMutableString. This is
> a loop so that I set up an autorelease pool. I even set up an additional
> autorelease pool in case that the loop is blocked at the recv() function.
> However, it occasionally gets huge leaks.
> 
> Another question is that what are the states, according to which I can
> believe the program gets robust. I don't worry about leaks or any other
> issues? Most time, the current program runs well. I am even not aware that
> the line has a so big problem without Instruments. And, the problem does not
> always happen even with Instruments.
> 
> Thanks so much!
> Bing
> 
> - (void) run
> {
>char buffer[1024];
>ssize_t numberBytesReceived;
>while (true)
>{
>NSAutoreleasePool *loopPool = [[NSAutoreleasePool alloc]
> init];
>numberBytesReceived = recv(clientSocket, buffer, 1024, 0);
>if (numberBytesReceived > 0)
>{
>buffer[numberBytesReceived] = '\0';
>NSAutoreleasePool *subLoopPool = [[NSAutoreleasePool
> alloc] init];
> 
>// This line sometimes gets huge leaks (about
> 167.80MB, 226933 leaks). But it does not always happen
>// I don't understand why it gets leaked!
>NSMutableString *receivedMessage =
> [[[NSMutableString alloc] initWithBytes:buffer length:numberBytesReceived
> encoding:NSUTF8StringEncoding] autorelease];
> 
>[self notifyMessageReceived:receivedMessage];
>[subLoopPool drain];
>}
>else
>{
>[loopPool drain];
>break;
>}
>for (int i = 0; i < 1024; i ++)
>{
>buffer[i] = '\0';
>}
>[loopPool drain];
>}
>if (buffer[0] != '\0')
>{
>free(buffer);
>}
> }

___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

After Autoreleasing Still Getting Leaked

[Bing Li]

Dear all,

I got a problem which confused me. The following method is put into
NSOperationQueue so that it can run asynchronously. The method received data
from a remote node and then converted the bytes to NSMutableString. This is
a loop so that I set up an autorelease pool. I even set up an additional
autorelease pool in case that the loop is blocked at the recv() function.
However, it occasionally gets huge leaks.

Another question is that what are the states, according to which I can
believe the program gets robust. I don't worry about leaks or any other
issues? Most time, the current program runs well. I am even not aware that
the line has a so big problem without Instruments. And, the problem does not
always happen even with Instruments.

In addition, according to Activity Monitor, the memory consumed by the
program increased only when some new threads are added. After the threads
were dead, the memory was lowered to a fixed amount. Is it fine?

Thanks so much!
Bing

- (void) run
{
char buffer[1024];
ssize_t numberBytesReceived;
while (true)
{
NSAutoreleasePool *loopPool = [[NSAutoreleasePool alloc]
init];
numberBytesReceived = recv(clientSocket, buffer, 1024, 0);
if (numberBytesReceived > 0)
{
buffer[numberBytesReceived] = '\0';
NSAutoreleasePool *subLoopPool = [[NSAutoreleasePool
alloc] init];

// This line sometimes gets huge leaks (about
167.80MB, 226933 leaks). But it does not always happen
// I don't understand why it gets leaked!
NSMutableString *receivedMessage =
[[[NSMutableString alloc] initWithBytes:buffer length:numberBytesReceived
encoding:NSUTF8StringEncoding] autorelease];

[self notifyMessageReceived:receivedMessage];
[subLoopPool drain];
}
else
{
[loopPool drain];
break;
}
for (int i = 0; i < 1024; i ++)
{
buffer[i] = '\0';
}
[loopPool drain];
}
if (buffer[0] != '\0')
{
free(buffer);
}
}
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: After Autoreleasing Still Getting Leaked

[Scott Ribe]

On Jun 15, 2011, at 10:02 AM, Bing Li wrote:

>char buffer[1024];
>ssize_t numberBytesReceived;
>while (true)
>{
>NSAutoreleasePool *loopPool = [[NSAutoreleasePool alloc]
> init];
>numberBytesReceived = recv(clientSocket, buffer, 1024, 0);
>if (numberBytesReceived > 0)
>{
>buffer[numberBytesReceived] = '\0';

So, if you receive 1024 bytes, you write past the end of the buffer. Now 
granted, you're only writing a 0 onto the least significant (assuming Intel) 
byte of numberBytesReceived, so it's not the source of your leak, but it's a 
pretty bad bug that's going to mess up your buffer contents for any buffer 
that's not a multiple of 256 bytes in length.

>for (int i = 0; i < 1024; i ++)
>{
>buffer[i] = '\0';
>}

That's pretty pointless isn't it?

>if (buffer[0] != '\0')
>{
>free(buffer);
>}


Seriously??? Freeing a stack buffer? Ouch. And the test? Why would you decide 
whether to free the block based on its contents?

I don't see offhand anything in this method that would cause a leak. But if 
this code is typical of the rest of the program, then you have bugs all over 
the place that corrupt memory which can lead to all sorts of things 
failing--including memory reclamation.

-- 
Scott Ribe
scott_r...@elevated-dev.com
http://www.elevated-dev.com/
(303) 722-0567 voice




___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

After Autoreleasing Still Getting Leaked

[Bing Li]

Dear all,

I got a problem which confused me. The following method is put into
NSOperationQueue so that it can run asynchronously. The method received data
from a remote node and then converted the bytes to NSMutableString. This is
a loop so that I set up an autorelease pool. I even set up an additional
autorelease pool in case that the loop is blocked at the recv() function.
However, it occasionally gets huge leaks.

Another question is that what are the states, according to which I can
believe the program gets robust. I don't worry about leaks or any other
issues? Most time, the current program runs well. I am even not aware that
the line has a so big problem without Instruments. And, the problem does not
always happen even with Instruments.

Thanks so much!
Bing

- (void) run
{
char buffer[1024];
ssize_t numberBytesReceived;
while (true)
{
NSAutoreleasePool *loopPool = [[NSAutoreleasePool alloc]
init];
numberBytesReceived = recv(clientSocket, buffer, 1024, 0);
if (numberBytesReceived > 0)
{
buffer[numberBytesReceived] = '\0';
NSAutoreleasePool *subLoopPool = [[NSAutoreleasePool
alloc] init];

// This line sometimes gets huge leaks (about
167.80MB, 226933 leaks). But it does not always happen
// I don't understand why it gets leaked!
NSMutableString *receivedMessage =
[[[NSMutableString alloc] initWithBytes:buffer length:numberBytesReceived
encoding:NSUTF8StringEncoding] autorelease];

[self notifyMessageReceived:receivedMessage];
[subLoopPool drain];
}
else
{
[loopPool drain];
break;
}
for (int i = 0; i < 1024; i ++)
{
buffer[i] = '\0';
}
[loopPool drain];
}
if (buffer[0] != '\0')
{
free(buffer);
}
}
___

Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com