Re: [PHP-DEV] Array max size
On Wed, Aug 1, 2018 at 2:41 PM, Marcos Passos wrote: > Whenever you look for more information about the maximum size of an array, > you find someone saying that "PHP arrays do not have a maximum size, but > the amount of memory available". However, I could not find any excerpt in > PHP documentation that supports that. > I would say that *AS A LANGUAGE*, PHP does in fact have no limit on array sizes. Period. Beyond that definition in the language specification, there come two limiting factors: 1. The hardware limitation of the machine running PHP. i.e. How much memory it has. This is the most practical answer for most users. 2. Implementation details of the PHP runtime. The reference implementation of PHP uses a uint32_t to keep track of the number of elements for quick and easy O(1) access. This may or may not limit the array to (2^32-1) elements, but it does certainly limit the functionality of count() when operating on arrays. I think documenting in the php.net manual (which refers solely to the reference implementation) a limit of (2^32-1) elements as an implementation detail is not unreasonable, but it's worthy of nothing more than a footnote on the arrays reference page. The answer in #1 about memory limits is FAR more salient to typical uses. -Sara -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Array max size
If you are arguing strictly about not knowing what to write on the documentation, let's move the discussion towards whether it should be defined or not. 2018-08-01 16:51 GMT-03:00 Sherif Ramadan : > Right, and therein lies the problem. No one has ever defined this > behavior. As such, one cannot document what has never been defined. > > What you're describing is UNdefined. Undefined things cannot be documented. > > > On Wed, Aug 1, 2018, 3:46 PM Marcos Passos > wrote: > >> *The point is not about the possibility of crossing the limit of the >> array, but the need for a definition so one can design a method or function >> whose behavior aligns with the array capabilities. This need, in fact, >> became more noticeable from a design point of view since the introduction >> of the iterable pseudo-type. >> >> >> >> 2018-08-01 16:44 GMT-03:00 Marcos Passos : >> >>> It looks like the limit I mentioned, used by some functions, is >>> architecture-dependent: >>> https://github.com/php/php-src/blob/master/Zend/zend_types.h#L288 >>> >>> Also, that's such a ridiculously large number for the vast majority of people using PHP that hardly anyone ever runs into this limit. >>> >>> >>> The point is not about the possibility of crossing the limit of the >>> array, but the need for a definition is one to design a method or function >>> whose behavior aligns with the array capabilities. This need, in fact, >>> became more noticeable from a design point of view since the introduction >>> of the iterable pseudo-type. >>> >>> No one ever said "this is what will happen if a PHP array exceeds size". Until someone does that, I cannot document it. >>> >>> >>> I cannot disagree more with this statement. >>> >>> - Marcos >>> >>> 2018-08-01 16:29 GMT-03:00 Sherif Ramadan : >>> It's undocumented, because it's considered undefined behavior. PHP arrays implicitly store the number of elements internally as an unsigned 32 bit integer (regardless of architecture). This means that (technically) you can't create an array with more than ((2**31) - 1) elements (or 2,147,483,647 elements). However, PHP won't actually attempt to stop you from doing this! The problem is, once you exceed these number of elements, the integer will overflow, causing undefined behavior (all kinds weird bugs). So we cannot document behavior that was never defined. No one ever said "this is what will happen if a PHP array exceeds size". Until someone does that, I cannot document it. Also, that's such a ridiculously large number for the vast majority of people using PHP that hardly anyone ever runs into this limit. On Wed, Aug 1, 2018 at 2:42 PM Marcos Passos < marcospassos@gmail.com> wrote: > Whenever you look for more information about the maximum size of an > array, > you find someone saying that "PHP arrays do not have a maximum size, > but > the amount of memory available". However, I could not find any excerpt > in > PHP documentation that supports that. > > Even if the engine imposes no hard limit on the size of an array, in > fact, > there an inherent limit that is assumed by some functions, and that is > what > I would like to suggest making explicit on the documentation. The lack > of > this definition leads to inconsistencies and leaves several open > questions, > including: > >- If no limit exists, then it's conceptually possible to have an > array >with *PHP_INT_MAX + 1* elements. In that sense, what would be the > return >of the *\count()*? >- The function *range* validates the size of the resulting range > against >the maximum size of the hash table (defined internally as > *HT_MAX_SIZE*), >and throw an error if it exceeds that value. Is that the limit? >- he function *array_fill*, in contrast, does not rely on > *HT_MAX_SIZE* >for validating the size of the result. But, why? >- The documentation says that omitting the third parameter of >array_split is equivalent to \count($array). If the maximum number >representable in PHP is *PHP_INT_MAX*, is the max size the same as >*PHP_INT_MAX*? > > There are other examples, but I think these are enough to make the > point. > > My understanding is that the conceptual limit is *PHP_INT_MAX*, as > there is > no way to represent the size above this value. If so, the interval that > represents all possibles indexes for an array is defined as *0 <= > index <= > PHP_INT_MAX -1*. That definition aligns with all functions that support > negative length as *-PHP_INT_MAX* is equivalent to the start of the > array. > > Could you guys please share your thoughts on this topic? > >>> >>
Re: [PHP-DEV] Array max size
Right, and therein lies the problem. No one has ever defined this behavior. As such, one cannot document what has never been defined. What you're describing is UNdefined. Undefined things cannot be documented. On Wed, Aug 1, 2018, 3:46 PM Marcos Passos wrote: > *The point is not about the possibility of crossing the limit of the > array, but the need for a definition so one can design a method or function > whose behavior aligns with the array capabilities. This need, in fact, > became more noticeable from a design point of view since the introduction > of the iterable pseudo-type. > > > > 2018-08-01 16:44 GMT-03:00 Marcos Passos : > >> It looks like the limit I mentioned, used by some functions, is >> architecture-dependent: >> https://github.com/php/php-src/blob/master/Zend/zend_types.h#L288 >> >> Also, that's such a ridiculously large number for the vast majority of >>> people using PHP that hardly anyone ever runs into this limit. >> >> >> The point is not about the possibility of crossing the limit of the >> array, but the need for a definition is one to design a method or function >> whose behavior aligns with the array capabilities. This need, in fact, >> became more noticeable from a design point of view since the introduction >> of the iterable pseudo-type. >> >> No one ever said "this is what will happen if a PHP array exceeds >>> size". Until someone does that, I cannot document it. >> >> >> I cannot disagree more with this statement. >> >> - Marcos >> >> 2018-08-01 16:29 GMT-03:00 Sherif Ramadan : >> >>> It's undocumented, because it's considered undefined behavior. PHP >>> arrays implicitly store the number of elements internally as an unsigned 32 >>> bit integer (regardless of architecture). This means that (technically) you >>> can't create an array with more than ((2**31) - 1) elements (or >>> 2,147,483,647 elements). However, PHP won't actually attempt to stop you >>> from doing this! The problem is, once you exceed these number of elements, >>> the integer will overflow, causing undefined behavior (all kinds weird >>> bugs). So we cannot document behavior that was never defined. No one ever >>> said "this is what will happen if a PHP array exceeds size". Until >>> someone does that, I cannot document it. Also, that's such a ridiculously >>> large number for the vast majority of people using PHP that hardly anyone >>> ever runs into this limit. >>> >>> >>> >>> >>> On Wed, Aug 1, 2018 at 2:42 PM Marcos Passos >>> wrote: >>> Whenever you look for more information about the maximum size of an array, you find someone saying that "PHP arrays do not have a maximum size, but the amount of memory available". However, I could not find any excerpt in PHP documentation that supports that. Even if the engine imposes no hard limit on the size of an array, in fact, there an inherent limit that is assumed by some functions, and that is what I would like to suggest making explicit on the documentation. The lack of this definition leads to inconsistencies and leaves several open questions, including: - If no limit exists, then it's conceptually possible to have an array with *PHP_INT_MAX + 1* elements. In that sense, what would be the return of the *\count()*? - The function *range* validates the size of the resulting range against the maximum size of the hash table (defined internally as *HT_MAX_SIZE*), and throw an error if it exceeds that value. Is that the limit? - he function *array_fill*, in contrast, does not rely on *HT_MAX_SIZE* for validating the size of the result. But, why? - The documentation says that omitting the third parameter of array_split is equivalent to \count($array). If the maximum number representable in PHP is *PHP_INT_MAX*, is the max size the same as *PHP_INT_MAX*? There are other examples, but I think these are enough to make the point. My understanding is that the conceptual limit is *PHP_INT_MAX*, as there is no way to represent the size above this value. If so, the interval that represents all possibles indexes for an array is defined as *0 <= index <= PHP_INT_MAX -1*. That definition aligns with all functions that support negative length as *-PHP_INT_MAX* is equivalent to the start of the array. Could you guys please share your thoughts on this topic? >>> >> >
Re: [PHP-DEV] Array max size
*The point is not about the possibility of crossing the limit of the array, but the need for a definition so one can design a method or function whose behavior aligns with the array capabilities. This need, in fact, became more noticeable from a design point of view since the introduction of the iterable pseudo-type. 2018-08-01 16:44 GMT-03:00 Marcos Passos : > It looks like the limit I mentioned, used by some functions, is > architecture-dependent: > https://github.com/php/php-src/blob/master/Zend/zend_types.h#L288 > > Also, that's such a ridiculously large number for the vast majority of >> people using PHP that hardly anyone ever runs into this limit. > > > The point is not about the possibility of crossing the limit of the array, > but the need for a definition is one to design a method or function whose > behavior aligns with the array capabilities. This need, in fact, became > more noticeable from a design point of view since the introduction of the > iterable pseudo-type. > > No one ever said "this is what will happen if a PHP array exceeds >> size". Until someone does that, I cannot document it. > > > I cannot disagree more with this statement. > > - Marcos > > 2018-08-01 16:29 GMT-03:00 Sherif Ramadan : > >> It's undocumented, because it's considered undefined behavior. PHP arrays >> implicitly store the number of elements internally as an unsigned 32 bit >> integer (regardless of architecture). This means that (technically) you >> can't create an array with more than ((2**31) - 1) elements (or >> 2,147,483,647 elements). However, PHP won't actually attempt to stop you >> from doing this! The problem is, once you exceed these number of elements, >> the integer will overflow, causing undefined behavior (all kinds weird >> bugs). So we cannot document behavior that was never defined. No one ever >> said "this is what will happen if a PHP array exceeds size". Until >> someone does that, I cannot document it. Also, that's such a ridiculously >> large number for the vast majority of people using PHP that hardly anyone >> ever runs into this limit. >> >> >> >> >> On Wed, Aug 1, 2018 at 2:42 PM Marcos Passos >> wrote: >> >>> Whenever you look for more information about the maximum size of an >>> array, >>> you find someone saying that "PHP arrays do not have a maximum size, but >>> the amount of memory available". However, I could not find any excerpt in >>> PHP documentation that supports that. >>> >>> Even if the engine imposes no hard limit on the size of an array, in >>> fact, >>> there an inherent limit that is assumed by some functions, and that is >>> what >>> I would like to suggest making explicit on the documentation. The lack of >>> this definition leads to inconsistencies and leaves several open >>> questions, >>> including: >>> >>>- If no limit exists, then it's conceptually possible to have an array >>>with *PHP_INT_MAX + 1* elements. In that sense, what would be the >>> return >>>of the *\count()*? >>>- The function *range* validates the size of the resulting range >>> against >>>the maximum size of the hash table (defined internally as >>> *HT_MAX_SIZE*), >>>and throw an error if it exceeds that value. Is that the limit? >>>- he function *array_fill*, in contrast, does not rely on >>> *HT_MAX_SIZE* >>>for validating the size of the result. But, why? >>>- The documentation says that omitting the third parameter of >>>array_split is equivalent to \count($array). If the maximum number >>>representable in PHP is *PHP_INT_MAX*, is the max size the same as >>>*PHP_INT_MAX*? >>> >>> There are other examples, but I think these are enough to make the point. >>> >>> My understanding is that the conceptual limit is *PHP_INT_MAX*, as there >>> is >>> no way to represent the size above this value. If so, the interval that >>> represents all possibles indexes for an array is defined as *0 <= index >>> <= >>> PHP_INT_MAX -1*. That definition aligns with all functions that support >>> negative length as *-PHP_INT_MAX* is equivalent to the start of the >>> array. >>> >>> Could you guys please share your thoughts on this topic? >>> >> >
Re: [PHP-DEV] Array max size
It looks like the limit I mentioned, used by some functions, is architecture-dependent: https://github.com/php/php-src/blob/master/Zend/zend_types.h#L288 Also, that's such a ridiculously large number for the vast majority of > people using PHP that hardly anyone ever runs into this limit. The point is not about the possibility of crossing the limit of the array, but the need for a definition is one to design a method or function whose behavior aligns with the array capabilities. This need, in fact, became more noticeable from a design point of view since the introduction of the iterable pseudo-type. No one ever said "this is what will happen if a PHP array exceeds > size". Until someone does that, I cannot document it. I cannot disagree more with this statement. - Marcos 2018-08-01 16:29 GMT-03:00 Sherif Ramadan : > It's undocumented, because it's considered undefined behavior. PHP arrays > implicitly store the number of elements internally as an unsigned 32 bit > integer (regardless of architecture). This means that (technically) you > can't create an array with more than ((2**31) - 1) elements (or > 2,147,483,647 elements). However, PHP won't actually attempt to stop you > from doing this! The problem is, once you exceed these number of elements, > the integer will overflow, causing undefined behavior (all kinds weird > bugs). So we cannot document behavior that was never defined. No one ever > said "this is what will happen if a PHP array exceeds size". Until > someone does that, I cannot document it. Also, that's such a ridiculously > large number for the vast majority of people using PHP that hardly anyone > ever runs into this limit. > > > > > On Wed, Aug 1, 2018 at 2:42 PM Marcos Passos > wrote: > >> Whenever you look for more information about the maximum size of an array, >> you find someone saying that "PHP arrays do not have a maximum size, but >> the amount of memory available". However, I could not find any excerpt in >> PHP documentation that supports that. >> >> Even if the engine imposes no hard limit on the size of an array, in fact, >> there an inherent limit that is assumed by some functions, and that is >> what >> I would like to suggest making explicit on the documentation. The lack of >> this definition leads to inconsistencies and leaves several open >> questions, >> including: >> >>- If no limit exists, then it's conceptually possible to have an array >>with *PHP_INT_MAX + 1* elements. In that sense, what would be the >> return >>of the *\count()*? >>- The function *range* validates the size of the resulting range >> against >>the maximum size of the hash table (defined internally as >> *HT_MAX_SIZE*), >>and throw an error if it exceeds that value. Is that the limit? >>- he function *array_fill*, in contrast, does not rely on *HT_MAX_SIZE* >>for validating the size of the result. But, why? >>- The documentation says that omitting the third parameter of >>array_split is equivalent to \count($array). If the maximum number >>representable in PHP is *PHP_INT_MAX*, is the max size the same as >>*PHP_INT_MAX*? >> >> There are other examples, but I think these are enough to make the point. >> >> My understanding is that the conceptual limit is *PHP_INT_MAX*, as there >> is >> no way to represent the size above this value. If so, the interval that >> represents all possibles indexes for an array is defined as *0 <= index <= >> PHP_INT_MAX -1*. That definition aligns with all functions that support >> negative length as *-PHP_INT_MAX* is equivalent to the start of the array. >> >> Could you guys please share your thoughts on this topic? >> >
Re: [PHP-DEV] Array max size
It's undocumented, because it's considered undefined behavior. PHP arrays implicitly store the number of elements internally as an unsigned 32 bit integer (regardless of architecture). This means that (technically) you can't create an array with more than ((2**31) - 1) elements (or 2,147,483,647 elements). However, PHP won't actually attempt to stop you from doing this! The problem is, once you exceed these number of elements, the integer will overflow, causing undefined behavior (all kinds weird bugs). So we cannot document behavior that was never defined. No one ever said "this is what will happen if a PHP array exceeds size". Until someone does that, I cannot document it. Also, that's such a ridiculously large number for the vast majority of people using PHP that hardly anyone ever runs into this limit. On Wed, Aug 1, 2018 at 2:42 PM Marcos Passos wrote: > Whenever you look for more information about the maximum size of an array, > you find someone saying that "PHP arrays do not have a maximum size, but > the amount of memory available". However, I could not find any excerpt in > PHP documentation that supports that. > > Even if the engine imposes no hard limit on the size of an array, in fact, > there an inherent limit that is assumed by some functions, and that is what > I would like to suggest making explicit on the documentation. The lack of > this definition leads to inconsistencies and leaves several open questions, > including: > >- If no limit exists, then it's conceptually possible to have an array >with *PHP_INT_MAX + 1* elements. In that sense, what would be the return >of the *\count()*? >- The function *range* validates the size of the resulting range against >the maximum size of the hash table (defined internally as > *HT_MAX_SIZE*), >and throw an error if it exceeds that value. Is that the limit? >- he function *array_fill*, in contrast, does not rely on *HT_MAX_SIZE* >for validating the size of the result. But, why? >- The documentation says that omitting the third parameter of >array_split is equivalent to \count($array). If the maximum number >representable in PHP is *PHP_INT_MAX*, is the max size the same as >*PHP_INT_MAX*? > > There are other examples, but I think these are enough to make the point. > > My understanding is that the conceptual limit is *PHP_INT_MAX*, as there is > no way to represent the size above this value. If so, the interval that > represents all possibles indexes for an array is defined as *0 <= index <= > PHP_INT_MAX -1*. That definition aligns with all functions that support > negative length as *-PHP_INT_MAX* is equivalent to the start of the array. > > Could you guys please share your thoughts on this topic? >
Re: [PHP-DEV] Array max size
On Wed, Aug 1, 2018 at 12:42 PM Marcos Passos wrote: > > Whenever you look for more information about the maximum size of an array, > you find someone saying that "PHP arrays do not have a maximum size, but > the amount of memory available". However, I could not find any excerpt in > PHP documentation that supports that. > > Even if the engine imposes no hard limit on the size of an array, in fact, > there an inherent limit that is assumed by some functions, and that is what > I would like to suggest making explicit on the documentation. The lack of > this definition leads to inconsistencies and leaves several open questions, > including: > >- If no limit exists, then it's conceptually possible to have an array >with *PHP_INT_MAX + 1* elements. In that sense, what would be the return >of the *\count()*? >- The function *range* validates the size of the resulting range against >the maximum size of the hash table (defined internally as *HT_MAX_SIZE*), >and throw an error if it exceeds that value. Is that the limit? >- he function *array_fill*, in contrast, does not rely on *HT_MAX_SIZE* >for validating the size of the result. But, why? >- The documentation says that omitting the third parameter of >array_split is equivalent to \count($array). If the maximum number >representable in PHP is *PHP_INT_MAX*, is the max size the same as >*PHP_INT_MAX*? > > There are other examples, but I think these are enough to make the point. > > My understanding is that the conceptual limit is *PHP_INT_MAX*, as there is > no way to represent the size above this value. If so, the interval that > represents all possibles indexes for an array is defined as *0 <= index <= > PHP_INT_MAX -1*. That definition aligns with all functions that support > negative length as *-PHP_INT_MAX* is equivalent to the start of the array. > > Could you guys please share your thoughts on this topic? If we look at the definition of _zend_array: https://github.com/php/php-src/blob/1a303f1a2b6ed8853fea3a207eef7f462d86/Zend/zend_types.h#L237-L257 It stores element counts, num used, etc, in uint32_t. It seems to me that therefore they can't hold more than UINT32_MAX number of elements. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Array max size
On Wed, Aug 1, 2018 at 12:42 PM Marcos Passos wrote: > > Whenever you look for more information about the maximum size of an array, > you find someone saying that "PHP arrays do not have a maximum size, but > the amount of memory available". However, I could not find any excerpt in > PHP documentation that supports that. > > Even if the engine imposes no hard limit on the size of an array, in fact, > there an inherent limit that is assumed by some functions, and that is what > I would like to suggest making explicit on the documentation. The lack of > this definition leads to inconsistencies and leaves several open questions, > including: > >- If no limit exists, then it's conceptually possible to have an array >with *PHP_INT_MAX + 1* elements. In that sense, what would be the return >of the *\count()*? >- The function *range* validates the size of the resulting range against >the maximum size of the hash table (defined internally as *HT_MAX_SIZE*), >and throw an error if it exceeds that value. Is that the limit? >- he function *array_fill*, in contrast, does not rely on *HT_MAX_SIZE* >for validating the size of the result. But, why? >- The documentation says that omitting the third parameter of >array_split is equivalent to \count($array). If the maximum number >representable in PHP is *PHP_INT_MAX*, is the max size the same as >*PHP_INT_MAX*? > > There are other examples, but I think these are enough to make the point. > > My understanding is that the conceptual limit is *PHP_INT_MAX*, as there is > no way to represent the size above this value. If so, the interval that > represents all possibles indexes for an array is defined as *0 <= index <= > PHP_INT_MAX -1*. That definition aligns with all functions that support > negative length as *-PHP_INT_MAX* is equivalent to the start of the array. > > Could you guys please share your thoughts on this topic? -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Array max size
On 01/08/18 19:41, Marcos Passos wrote: My understanding is that the conceptual limit is*PHP_INT_MAX*, as there is no way to represent the size above this value. And on which definition, 32bit versions of PHP crash out before 64bit builds ... It assumes that one HAS to convert variables to binary numbers but one can quite easily retain 64bit numbers on a 32bit version of PHP, so why should it not be possible to to handle even larger numbers ... and text keys for an array can handle 'larger' numbers or other identifiers. Now some recent changes to arrays may be containing an unlimited array size, but it should not prevent it from being able to be handled. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
