How are you delivering the eventual result, to whatever is getting it? I
assume you'll be passing it to some other software, and will need it all to be
in a single contiguous memory block (SCMB) at that point.
The only way to avoid copying the data into the result SCMB would be to
allocate mo
"You could calculate locations rather easily .. chunk = n / chunksize then
offset = a modulus n % chunksize. This would allow you to continue adding
items without ever having full copy operations. I think someone has already
mentioned a similar strategy but this is the way to go. If you ran out of
This is exactly what I was thinking, then you can construct the final array as
the final step, or, if you don't have something that is going to consume the
array, you could probably just use the ArrayList with a mod to figure out which
index to use.
Erick
> -Original Message-
> From:
Right, I understand, but List copies the inner list on a grow, but does not
do a deep copy.
List list = new List();
As list grows, the inner list that contains the byte arrays will get copied,
but the underlying bytes will not. Arrays are objects, not structs, so a copy
is always a copy of th
hmmm.. I think you may have mis-understood me (or maybe I'm
mis-understanding you).
I was thinking of prototyping:
ArrayList items = new ArrayList();
byte[] data = new byte[ bufferSize ];
while( socket.Read( data ) ){
items.Add( data );
}
Obviously this is rough pseduo code, but I'm looking at
Well, it's relatively simple. All I need to-do, is return the byte array of
read data.
internal byte[] GetData( int index, byte[] terminatingData ){
MemoryStream ms = new MemoryStream();
while( !eof ){
countRead = ReadSocket(data, 0, data.Length);
//store byte array
ms.Write( data, 0, countRead
Gregory was talking about the implementation details of List<> which
copies the existing items when it grows. On a side note, List<*byte[]>
is an invalid construct and won't even compile.
On 7/7/06, Erick Thompson <[EMAIL PROTECTED]> wrote:
If you have an List and the list needs to expand, it sh
If you have an List and the list needs to expand, it should only copy
the pointer to the byte[] object, which is cheap. I see no reason why it would
do a deep copy. If it does do a deep copy, make the pointer explicit, e.g.,
List<*byte[]>. However, I don't believe you need to do this.
Erick
>
Hi John,
I don't have time to take a look at the code, but take a look at
System.Runtime.InteropServices.HandleRef and GC.KeepAlive and see if
you're having a premature GC problem.
Cheers,
Stoyan
On 7/7/06, John Whattam <[EMAIL PROTECTED]> wrote:
I have written a Combobox-based control for dis
I have written a Combobox-based control for displaying all the raster and
truetype fonts available on the users system. It makes extensive use of
the EnumFontFamiliesEx API for two purposes:
1.When InitialiseFontCombo is called in the dialogs load event,
EnumFontFamiliesEx is called to enumerate
An arraylist containing bytes would have far worse performance (it also does
doubling internally except now you would be dealing with references instead
of of single bytes). A List also will do doubling internally (if you
don't set it large enough initially) but will do it dealing with a byte []
i
Has the frequency of the LOH scan changed in .Net 2.0 or is it still only
occurring with a Gen 2 collection?
-Original Message-
From: Discussion of advanced .NET topics.
[mailto:[EMAIL PROTECTED] On Behalf Of Barry Kelly
Sent: Friday, July 07, 2006 6:54 AM
To: ADVANCED-DOTNET@DISCUSS.DEVEL
Dave,
Are you willing to share with us the interface(s) of your component.
If we see how your component's clients will use it, we might be able
to come up with an idea.
Cheers,
Stoyan
On 7/6/06, dave wanta <[EMAIL PROTECTED]> wrote:
I wish I could. But basically these are large blobs (usually
Hi Dave,
Do you want the byte[] to be trimmed or the stream iself?
In case of the latter, you can read the data in small chuncks, and do
special handling for the 0x0 character.
1) If no other data was read before, you can skip all 0x0 bytes
2) Otherwise count the # contigious 0x0 bytes in an integ
Saad was suggesting that you either build an unmanaged DLL or a
managed C++ wrapper, pass chunks of data to it and get the result when
finished. This however has the following drawbacks, which you might
want to consider before moving to implementation:
- you'll essentially copy all data three tim
I wish I could. But basically these are large blobs (usually GIS images, but
can be regular images and sometimes large zips).
Unfortunately I don't have control over the project, and only have this
little box I can work in (read data from a socket, and optimize the
memory ).
I was using a MemoryS
Hi Erick,
hmm... this is interesting.
I wouldn't have to worry about constantly expanding the byte array, and in
this case I don't need to worry about accessing any internal bytes, just the
end of the overall byte array to check for terminating characters.
(thinking out loud)
I wonder if even som
dave wanta <[EMAIL PROTECTED]> wrote:
> That would be a good work-around. But, I don't have write access (gotta love
> all of these constraints. ;-) )
The solution I would use in this scenario is a class I've developed for
a similar purpose. As you may be aware, large objects (i.e. greater than
8
Dave,
Do you really need to keep all the data around? Could you read a few meg,
then process that data, then read more data and process that, etc... ?
I've found it rare to "have to" deal with such large collections of data all
at once.
Basically, I'm saying that maybe you could look at the pro
That would be a good work-around. But, I don't have write access (gotta love
all of these constraints. ;-) )
Cheers!
Dave
- Original Message -
From: "Barry Kelly" <[EMAIL PROTECTED]>
To:
Sent: Thursday, July 06, 2006 3:20 PM
Subject: Re: [ADVANCED-DOTNET] trim byte array
dave wanta <[EM
It's the copy that gets pricey, which is going to be hard to avoid in this
situation. A list just won't cut it with this size. I would suggest a new class
that uses a list of byte[], and then uses the length of the internal byte array
to determine the right array to touch in the indexer. I have
dave wanta <[EMAIL PROTECTED]> wrote:
> It's from a socket, so I don't know the size until i get the terminating
> characters.
You could probably consider streaming it to a temporary file, then,
since I expect a download of many 100s of MB wouldn't be immediate. You
wouldn't want many of those ar
Dave this is how list would work as well ... you can however set an
initial size if you know it to avoid these doublings.
Cheers,
Greg
On 7/6/06, dave wanta <[EMAIL PROTECTED]> wrote:
Hi all,
btw, just fyi...if you Reflector a MemoryStream, it doubles it's internal
buffer every time it exce
Hi Saad,
hmm...this sounds interesting.
Do you have some links you could point me to? or a phrase or 2 I could
goole?
Also, will there be any issues (perhaps permissions?) if I run this in an
ASP.NET app?
Cheers!
Dave
- Original Message -
From: "Saad Rehmani" <[EMAIL PROTECTED]>
To:
S
Hi Chris,
I was using a list of primes to dynamically grow the byte array, and keep
track of the length of read data.
Cheers!
Dave
- Original Message -
From: "Chris Anderson" <[EMAIL PROTECTED]>
To:
Sent: Thursday, July 06, 2006 1:21 PM
Subject: Re: [ADVANCED-DOTNET] trim byte array
H
Hi Stoyan,
It's from a socket, so I don't know the size until i get the terminating
characters.
Cheers!
Dave
- Original Message -
From: "Stoyan Damov" <[EMAIL PROTECTED]>
To:
Sent: Thursday, July 06, 2006 2:43 PM
Subject: Re: [ADVANCED-DOTNET] trim byte array
> Hi Dave,
>
> Are you rea
Hi all,
btw, just fyi...if you Reflector a MemoryStream, it doubles it's internal
buffer every time it exceeds it's length.
The MemoryStream is really clean from a programmatic standpoint, it's just
not the performance I really want.
Thanks for all the suggestions guys! I really appreciate it.
Hi Adam,
Unfortunately, I'm already using a memory stream, and it is killing the
performance. Under
the covers the memory stream also copies the byte array, and returns a copy.
So a 100Meg array bloats to 200Meg
Cheers!
Dave
- Original Message -
From: "Adam Sills" <[EMAIL PROTECTED]>
To:
Hi Dave,
Are you reading your data from a fixed storage (e.g. a file, and not
from a socket) and do you know how much bytes you'd need upfront? If
you do then you can preallocate the buffer w/ the exact size you need,
but I guess you don't know the size upfront. If you don't, then I'd
suggest wha
You are right, you cannot resize an array. You can copy a range of bytes
from one array into a new array (see Array.Copy()), but that requires 2x the
memory.
Adam may be right - depending on your needs - a MemoryStream may be the best
choice for size and perf. The downside of a MemoryStream is t
I don't think you can resize managed arrays.
If its super important, you can keep the byte array in unmanaged space and call
a realloc (or equivalent) on it.
-s
--
Saad Rehmani / Prodika / Dallas / TX / USA
-Original Message-
From: dave wanta <[EMAIL PROTECTED]>
Date:
Actually Adam is right .. a memorystream would probably be better than a
list byte in this case.
Cheers,
Greg
On 7/6/06, gregory young <[EMAIL PROTECTED]> wrote:
A List would provide this exact behavior for you. Internally it
deals with a buffer but encapsulates a seperate length than the l
A List would provide this exact behavior for you. Internally it deals
with a buffer but encapsulates a seperate length than the length on the
buffer.
Cheers,
Greg
On 7/6/06, Chris Anderson <[EMAIL PROTECTED]> wrote:
How about reading the data into a small intermediate buffer array first.
The
Use a MemoryStream.
Adam..
> -Original Message-
> From: Discussion of advanced .NET topics. [mailto:ADVANCED-
> [EMAIL PROTECTED] On Behalf Of dave wanta
> Sent: Thursday, July 06, 2006 12:51 PM
> To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM
> Subject: [ADVANCED-DOTNET] trim byte array
>
> hi
you have two options I believe.
1. use a seaprate index to track location in byte array.
2. change the code for the dynamically expanding array (or create your own) to
internally track this.
unless you use a method of tracking the length, any other method you use will
prob copy. I believe for in
How about reading the data into a small intermediate buffer array first.
Then when that's full, or you've finished reading, copy the appropriate number
of bytes from that buffer into your large array.
Also, out of interest, how are you dynamically expanding your array?
-Original Message
hi all,
does anyone know of an efficient way to trim a byte array?
Here is what is happening. I'm reading in some binary data into a
dynamically expanding byte array, until all of the data has been read into
memory. I need to trim the byte array to remove the trailing nulls. Because
the amount of
Thanks everyone for offering your solutions. I will pass all this
information to my guy, hoping he will know what to do next.
-Original Message-
From: Discussion of advanced .NET topics.
[mailto:[EMAIL PROTECTED] Behalf Of Peter Suter
Sent: Wednesday, July 05, 2006 5:21 PM
To: ADVANCED-DOT
Wel, skip the process bit. I figured out that even if the process name is
changed it still opens using an open excel instance because I renamed
excel.exe to foobar.exe, opened it and then opened a xls with explorer and
it opened in foobar.exe!
Then I had a look at the explorer file associations an
Is it possible to change the name of a process?
I have a vb.net app that encapsulates an EXCEL application object that I
only use to host an instance of the VBE. Which I basically use to create VBA
applications. The problems is that if I open an XLS file from the explorer
then it opens up in my en
40 matches
Mail list logo
