[PHP-DEV] Re: Json error - better message
El El mar, 16 jul 2024 a la(s) 1:51 p. m., juan carlos morales < dev.juan.mora...@gmail.com> escribió: > Hello Internals again: > > I created this pull request: > > https://github.com/php/php-src/pull/14672 > > To have better error messages when decoding a JSON in php using the > json_last_error_msg() function. > > With this change I pretend to have a message like: > > "Syntax error, at character 0 near content: blah" > > I would like to heard your comments about it and if you have comments > about implementation please write your comments in the PR. > > I hope we can get this done soon. > > thanks in advance. > > Juan > Hello internals, I kindly ask for help, as the PR is getting some comments. Whoever can participate is welcome. Hopefully we can get this small but (in my opinión) useful change quickly. Thanks in advance
[PHP-DEV] Json error - better message
Hello Internals again: I created this pull request: https://github.com/php/php-src/pull/14672 To have better error messages when decoding a JSON in php using the json_last_error_msg() function. With this change I pretend to have a message like: "Syntax error, at character 0 near content: blah" I would like to heard your comments about it and if you have comments about implementation please write your comments in the PR. I hope we can get this done soon. thanks in advance. Juan
Re: [PHP-DEV] [RFC][Discussion] Enhance JSON error messages
El El dom, 30 de jun. de 2024 a la(s) 11:52, juan carlos morales < dev.juan.mora...@gmail.com> escribió: > > > El El dom, 30 de jun. de 2024 a la(s) 11:15, Tim Düsterhus < > t...@bastelstu.be> escribió: > >> Hi >> >> On 6/30/24 15:43, juan carlos morales wrote: >> > So, what I see here in my shortexperiencie is an RFC with 3 Options >> >> That is not an option. Each RFC needs a clear primary vote that is a >> "Yes/No" vote. >> >> > 1) Enhance the error message we already have >> > >> > 2) keep json_last_error_msg as it is and add a new function >> > json_last_error_info function >> > >> > 3) both >> > >> >> Adjusting error messages to make them better is not something that >> requires an RFC. We do this all the time without an RFC. Here's an >> example PR: https://github.com/php/php-src/pull/14496 >> >> A new json_last_error_info() for easier programmatic access of the >> important bits can still be considered, but would not need an RFC either >> from my point of view. It's a simple and obvious addition. Those can be >> done without an RFC, unless someone complains. >> >> Best regards >> Tim Düsterhus > > >> > Ok , then, i will prepare 2 PRs and lets see what happens then. > > Thanks. > > I will let everybody know When ready > Hello again. Well seems there are different opinions about how to approach this. Everyone agrees that having better error messages is always welcome but at the moment are different postures about having 1) enhanced error message 2) have a new function with better data 3) having both At the moment the mentioned pull request has both But i need to understand WHAT are we going to keep. And in case an RFC is needed then i will make it. Please advice Thanks >
Re: [PHP-DEV] [RFC][Discussion] Enhance JSON error messages
El El dom, 30 de jun. de 2024 a la(s) 11:15, Tim Düsterhus escribió: > Hi > > On 6/30/24 15:43, juan carlos morales wrote: > > So, what I see here in my shortexperiencie is an RFC with 3 Options > > That is not an option. Each RFC needs a clear primary vote that is a > "Yes/No" vote. > > > 1) Enhance the error message we already have > > > > 2) keep json_last_error_msg as it is and add a new function > > json_last_error_info function > > > > 3) both > > > > Adjusting error messages to make them better is not something that > requires an RFC. We do this all the time without an RFC. Here's an > example PR: https://github.com/php/php-src/pull/14496 > > A new json_last_error_info() for easier programmatic access of the > important bits can still be considered, but would not need an RFC either > from my point of view. It's a simple and obvious addition. Those can be > done without an RFC, unless someone complains. > > Best regards > Tim Düsterhus > Ok , then, i will prepare 2 PRs and lets see what happens then. Thanks. I will let everybody know When ready
Re: [PHP-DEV] [RFC][Discussion] Enhance JSON error messages
El El dom, 30 de jun. de 2024 a la(s) 09:02, Tim Düsterhus escribió: > Hi > > On 6/28/24 15:49, juan carlos morales wrote: > > Suggested by Jakub Zelenka (the main arquitect of the JSON part in php > > basically), in the PULL REQUEST is better to create a brand new > > function to have detailed error messages when json encoding, decoding, > > validating, fails. > > For the folks following along, I've commented on the PR. I think that > improving the existing messages is the right choice, not adding new > functions that users will need to remember to use. > > Best regards > Tim Düsterhus > Thanks for writting Tim. So, what I see here in my shortexperiencie is an RFC with 3 Options 1) Enhance the error message we already have 2) keep json_last_error_msg as it is and add a new function json_last_error_info function 3) both And let the voting to decide That is what I see so far Is evident that everyone agrees is a helpful feature to have better error understandind
Re: [PHP-DEV] [RFC][Discussion] Enhance JSON error messages
El jue, 27 jun 2024 a las 1:10, Gina P. Banyard () escribió: > > On Wednesday, 26 June 2024 at 15:11, juan carlos morales > wrote: > > > I also think it would be very useful to have better error messages. > > > > At the moment, we get things like "Syntax Error" ... period, no clue > > where the JSON is corrupt or anything else. With big JSONs is very > > cumberstone to get to the line and character position where the JSON > > is corrupt for example. > > > > I am already working in a change that would improve error messages > > while working with JSONs. > > > > What I am very close to achieve is to have error messages like "Syntax > > Error - at character position X near content Y" something like > > that. > > > > I already know that I can do it, actually there is a Work In Progress > > Pull Request for this : > > > > https://github.com/php/php-src/pull/14672 > > > > > > I repeat, is work in progress. BUT it can be done. Maybe my approach > > is not the best ... but one way or another it can be done. > > > > Please let me know your ideas, thoughts about this. > > > > Please, dont kill me for the actual code, is WIP. > > > > Thanks in advance to everyone. > > > > Juan > > Hello Juan, > > I'm in favour of this, but also I do not think this needs an RFC. > This is just an enhancement to our current crappy parsing errors, > and an RFC feels very much overkill for this. > > Best regards, > > Gina P. Banyard > Suggested by Jakub Zelenka (the main arquitect of the JSON part in php basically), in the PULL REQUEST is better to create a brand new function to have detailed error messages when json encoding, decoding, validating, fails. So, seems a new RFC will be needed. I am working on this at the moment. Will be ready next week or I will try to make it for next week. I am working
[PHP-DEV] [RFC][Discussion] Enhance JSON error messages
Hello Internals. I'm here to discuss an improvement in error messages when dealing with JSON in PHP. After watching Derick Rethans speak at the UK PHP Conference: https://youtu.be/3U0DGhzSH2U?feature=shared&t=2483 I also think it would be very useful to have better error messages. At the moment, we get things like "Syntax Error" ... period, no clue where the JSON is corrupt or anything else. With big JSONs is very cumberstone to get to the line and character position where the JSON is corrupt for example. I am already working in a change that would improve error messages while working with JSONs. What I am very close to achieve is to have error messages like "Syntax Error - at character position X near content Y" something like that. I already know that I can do it, actually there is a Work In Progress Pull Request for this : https://github.com/php/php-src/pull/14672 I repeat, is work in progress. BUT it can be done. Maybe my approach is not the best ... but one way or another it can be done. Please let me know your ideas, thoughts about this. Please, dont kill me for the actual code, is WIP. Thanks in advance to everyone. Juan
Re: [PHP-DEV] Discussion - PHP with distributed programming features.
El El sáb, 28 de oct. de 2023 a la(s) 19:25, Jordan LeDoux < jordan.led...@gmail.com> escribió: > > > On Sat, Oct 28, 2023 at 10:54 AM juan carlos morales < > dev.juan.mora...@gmail.com> wrote: > >> >> >> What I mean is more about … migrating a running php instance to another >> node or another php instance, in fact your php code is running, suddenly >> we >> Need to move to another node, how to do it? >> > > This seems less like a discussion about a PHP feature and more like you > asking internals for tech support at work, honestly. As far as I know, > there isn't a way to do what you're asking, because you can accomplish the > same thing much easier by designing your application better to make API > calls to services, which is what people have already suggested. The PHP way > to handle the root cause of your problem (too many resources being used on > a single machine) is to divide the application into services and use APIs, > often through sockets, the delegate work to those services. > > There are also other ways of handling this common problem in web > application architecture. You can break up the request into multiple > requests on the client side so that the user sees progress happen in > chunks. You can use tools like RabbitMQ or other queuing services to > perform certain long-running tasks asynchronously. You can use more > powerful hardware or provision more capable nodes. > > In fact, several of these actually fall under the Actor/Object or Dataflow > model of implementing Distributed Programming. What PHP does not support is > the Distributed Shared Memory model of implementing Distributed Programming > (which is what you are asking about apparently) because doing so would > almost certainly make PHP worse at the things it is well suited for, would > massively complicate the work of maintaining and developing the language > for the contributors to the language, and would be a massive undertaking to > implement in the first place. > > PHP has distributed programming features. In fact, all of the suggestions > you have received so far ARE ways of doing distributed programming > features. But internals is not your, or my, or anyone else's personal tech > support channel, and personally it feels like you haven't explained what it > is you want to discuss with internals about improving or changing PHP's > support for distributed programming. Are you interested in working on an > RFC to clone a PHP process onto a remote box? I can't imagine that would > get much support here, or be something that is very simple to do. > > Jordan > > Hello Jordan, thanks for the reply. When I read you, I have the feeling that you are a little angry about my question, please dont. It is a very honest question, that do belong to the internals, because if PHP does not have a way to do it, I would like to think a way to do it (despite the fact someone believes is useful or not, it is just me). So again, thanks!, dont get mad :D, and I will come back later on this topic, in a more clear way, maybe I am not expressing myself clearly. Regards
Re: [PHP-DEV] Discussion - PHP with distributed programming features.
El El vie, 27 de oct. de 2023 a la(s) 10:59, Kirill Nesmeyanov < n...@xakep.ru> escribió: > > >Пятница, 27 октября 2023, 14:38 +03:00 от juan carlos morales < > dev.juan.mora...@gmail.com>: > > > >Imagine a request comes to NgInx, then we process the request in PHP > > suddenly while execution of PHP is running, we dont have more > >resources in the server , so we need to process that PHP code on other > >server with more resources. > > > >How can we "move" the complete PHP process to another server with more > >resources and then send back the result ? > > > >this is like having a distributed php infrastructure, behaving in a > dynamic way. > > > >how can we achieve something like this? > > > >or maybe one piece of code needs to run in another server because of > >resources or something like that, like a function for example, only > >running one function in a different host and then come back to the > >mother one. > > > >My question is what do we need to provide PHP some functionality > >like the distributed programming languages? > > > >It could be achieve with long running PHP instances and using MPI for > >example (I imagine) but ... ideas on this ? Maybe I am crazy > > > >-- > >PHP Internals - PHP Runtime Development Mailing List > >To unsubscribe, visit: https://www.php.net/unsub.php > > Hello! > > nginx connects to fpm using the fcgi protocol via sockets. All you need to > do is create a socket with roundrobin (or like that), which will delegate > the request to a specific fpm socket. > > Or don’t reinvent the wheel, since nginx itself copes with this (with > balancing) perfectly in reverse proxy mode =)) > > -- > Kirill Nesmeyanov > What I mean is more about … migrating a running php instance to another node or another php instance, in fact your php code is running, suddenly we Need to move to another node, how to do it?
[PHP-DEV] Discussion - PHP with distributed programming features.
Imagine a request comes to NgInx, then we process the request in PHP suddenly while execution of PHP is running, we dont have more resources in the server , so we need to process that PHP code on other server with more resources. How can we "move" the complete PHP process to another server with more resources and then send back the result ? this is like having a distributed php infrastructure, behaving in a dynamic way. how can we achieve something like this? or maybe one piece of code needs to run in another server because of resources or something like that, like a function for example, only running one function in a different host and then come back to the mother one. My question is what do we need to provide PHP some functionality like the distributed programming languages? It could be achieve with long running PHP instances and using MPI for example (I imagine) but ... ideas on this ? Maybe I am crazy -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
[PHP-DEV][Discussion] PHP distributed programming
Imagine a request comes to NgInx, then we process the request in PHP suddenly while execution of PHP is running, we dont have more resources in the server , so we need to process that PHP code on other server with more resources. How can we "move" the complete PHP process to another server with more resources and then send back the result ? this is like having a distributed php infrastructure, behaving in a dynamic way. how can we achieve something like this? or maybe one piece of code needs to run in another server because of resources or something like that, like a function for example, only running one function in a different host and then come back to the mother one. My question is what do we need to provide PHP some functionality like the distributed programming languages? It could be achieve with long running PHP instances and using MPI for example (I imagine) but ... ideas on this ? Maybe I am crazy -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC Idea - json_validate() validate schema
Excellent Jakub, amazing, happy to know all this. Lets wait for your proposal. Good luck, make it happen! El mié, 1 mar 2023 a las 10:45, Jakub Zelenka () escribió: > > Hi, > > On Wed, Mar 1, 2023 at 1:36 PM Michał Marcin Brzuchalski > wrote: >> >> Hi Jakub >> >> śr., 1 mar 2023, 14:09 użytkownik Jakub Zelenka napisał: >>> >>> > >>> > Question ... are you planning to incorporate this by enhancing >>> > json_validate() ??? >>> > >>> >>> Yes the plan is to initially enhance json_decode and json_validate that >>> would get a new $schema argument . I plan to create a class for the >>> actually schema as it needs to be parsed to its own representation so it is >>> convenient to have it in the object. It could be also later created from >>> the different sources than just JSON string (e.g. assoc array / stdClass or >>> automatic generation from the class that I mentioned before) so it will be >>> better to have it in the class. >>> >>> Regards >>> >>> Jakub >> >> >> Do we really need this in core? What makes it less usable as an extension? >> > > The primary motivation is that this allow stopping decoding / validation once > first invalid part is found - basically this is going to be validated as > parsed. It means this will eliminate all currently possible DOS attacks on > the actual JSON parsing. There are other reasons that we can discuss in more > details once proposed like better availability for users but those are just > secondary reasons and sort of side effects. > > Regards > > Jakub -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC Idea - json_validate() validate schema
Jakub, wow, great to know this. thanks for writting. Ok, then I will assume that this feature will come from you sometime in future. Since json_validate() was announced ... I have being receiving messages (a lot) about providing the ability to use JSON SCHEMAS as well, hopefully we will have this in PHP in 8.4 (this will be a shock!) Question ... are you planning to incorporate this by enhancing json_validate() ??? El mié, 1 mar 2023 a las 9:07, Jakub Zelenka () escribió: > > On Wed, Mar 1, 2023 at 11:44 AM juan carlos morales > wrote: >> >> Hello Internals. >> >> I am thinking about improving the json_validate() function developed >> for php 8.3. >> >> The actual descriptions goes like this: >> >> json_validate(string $json, int $depth = 512, int $flags = 0): bool >> >> I am thinking about enhancing this function to also be able to >> validate against a JSON SCHEMA, giving us something like this: >> >> json_validate(string $json, int $depth = 512, int $flags = 0, string >> $json_schema = null): bool >> >> so, if the string is a valid JSON and also respects the schema ... then TRUE. >> >> What do you think ? >> > > I'm actually working on this. Currently developing the schema parsing in pure > C implementation in my play C tool called jso. You can see progress here: > https://github.com/bukka/jso/commits/next . The plan is to develop it inside > jso and then port it to jsond and then propose it for json ext inclusion > (that's how I developed the current parser). There is a lot of to do as > JsonSchema is quite complex (composition, JSON pointers, stream integration > for external pointers and more tricky bits) so this won't likely be ready for > 8.3 but should be ready for 8.4. I plan to introduce some smaller things for > 8.3 like better error reporting (error location which I have already working > in jso) and some other small additions. By the way, the schema support won't > be useful just for validation but also for decoding and possibly encoding > (sort of replacement for JsonSerializable). Especially for decoding it can be > further extended to allow class mapping. We could also provide automatic > generation of schema from class and support attributes. I plan to propose all > of this later as well but that might take some time. > > Regards > > Jakub -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
[PHP-DEV] RFC Idea - json_validate() validate schema
Hello Internals. I am thinking about improving the json_validate() function developed for php 8.3. The actual descriptions goes like this: json_validate(string $json, int $depth = 512, int $flags = 0): bool I am thinking about enhancing this function to also be able to validate against a JSON SCHEMA, giving us something like this: json_validate(string $json, int $depth = 512, int $flags = 0, string $json_schema = null): bool so, if the string is a valid JSON and also respects the schema ... then TRUE. What do you think ? Thanks in advance, your team mate Juan -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] Feature preview (formely was: experimental features)
As I previously wrote in the old Thread, ... I would like to say that we should keep the release process as it is, until a true, real, use case shows up. Whoever participated in the threads is already aware about the so many possibilities, and adjustments to the release/work process , that have to be done; because of that, such work should be at least justified with a real case. The actual release process is consistent, secure, predictable, not perfect ... sure, but to justify such a radical change, I believe we should have a real use case. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] Experimental features
I am back into PHP internals. After reading again and again, I would like to say that I think we should keep the release process as it is. If there is a feature not currently available in the current release, *at least* a user has the possibility to download and compile the code by itself; not the best practice but ... for sure ... consistent and possible. I think we should keep the release process as it is, until a real scenario shows up, that would make us think about experimental features. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] [VOTE] Improve unserialize() error handling
El sáb., 15 de octubre de 2022 11:02, juan carlos morales < dev.juan.mora...@gmail.com> escribió: > As far as I read... Everyone is right. All agree is BC break change, but > this is being considered to be part of PHP 9.0, so if what We get is a > better PHP then We must move forward, and such steps should be done > in a major version change. > I just checked that the RFC says in the PROPOSED PHP VERSIONS section next PHP 8.X Maybe that should be adjusted for clarification and documentation reasons >
Re: [PHP-DEV] [VOTE] Improve unserialize() error handling
As far as I read... Everyone is right. All agree is BC break change, but this is being considered to be part of PHP 9.0, so if what We get is a better PHP then We must move forward, and such steps should be done in a major version change.
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
The RFC got accepted and merge. Thanks for contributing :)
[PHP-DEV] RFC json_validate Accepted
I am glad to announce that the RFC for json_validate() function has been Accepted, 18 positive votes and 1 negative vote. https://wiki.php.net/rfc/json_validate I still have to do some adjustments to the code that is under review, but all good. Thanks a lot to everyone that participated and helped, thanks a lot!!
Re: [PHP-DEV] Union type casts
There is a difference is declaring a function parameter of different types ... because before hand, when the code calls the function, we might not be sure about what kind if data the function will receive, so the function could expect different types, and work the parameter correctly internally. nice but ... when casting, on purpose, what use case can apply to it? Are you facing a situation where manual union type cast is needed? can you share it with us? thanks -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] Experimental features
My small contribution to this First of all, nice you picked "json_validate()" for your example, thanks for this, somehow, support! :) Second... Apply a PECL strategy to try experimental features might not be the convenient way always, for example, if we create a new ... sensitive ... ini setting that affects the behavior of PHP somehow, then ... maybe applying the PECL strategy might not be the most confortable way to approach it, OR ... what about a new sensitive funtion in a "core extension" like JSON? An Experimental feature is something that WE want it to be in such a way because we want to see how the feature behaves, and what can of feedback can we get out of it. In case this proposal is accepted, it will change the RFC process for sure in order to indicate if its experimental or not. I think Experimental features , no matther what they are, should be backup with a consistent monitoring strategy around them something to keep in mind; This proposal is more about workflows in the PHP project than something else. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] Sanitize filters
My 2 cents on this. We should keep what is web related IMO. It does not make any sense to take things out, that later everyone will write by its own, or end up using a 3rd party package. PHP should have what is web related already to be use. Another different thing is the naming, the implementation code, etc. An RFC documenting each case would be very helpful, to centralize the ideas on each case, instead of scrolling the mailing list. Cheers. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] Re: Alias for `int|float`
Very interesting But I think that the original suggestion had a slimer scope than the rest of the opinions, that is why I also agree that the "scope" should be defined clearly, and provide the examples needed to clarify the suggestion itself. Cheers -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] Specify abstract parent methods for traits
I am also confused about this. Are you trying to generate multiple inheritance with this? If not ... can you provide code examples of the problem and a a hypothetical code of the solution you propose to that problem? Thanks -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
The RFC is in voting phase now. I want to thank everyone involved in the discussions related to this RFC, the ones who reviewed the code in advance, and the ones that with some emojis in github showed their interest and support. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
[PHP-DEV] [VOTE] json_validate
I am glad to announce that the RFC for the "json_validate()" function is in voting phase. Voting started 2022-09-22 and will end on 2022-10-07 at 00:00:00 UTC time zone (2 weeks + 1 day). I kindly ask everyone to consider that is the first time I do an RFC, if there is something wrong with it, I ask for your patience and to communicate with me to solve any issue with it. I want to thank everyone involved in the discussions related to this RFC, the ones who reviewed the code in advance, and the ones that with some emojis in github showed their interest and support. RFC link: https://wiki.php.net/rfc/json_validate RFC vote link: https://wiki.php.net/rfc/json_validate#vote Github: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
> The RFC still contains a non-empty "Open Issues" section. This needs to > be resolved before the vote starts. > > I would also recommend inserting a closed voting widget (or multiple, if > you want to have additional votes for the details), so that it's clear > for everyone how exactly the vote will look like. Tim, how should I proceed with the Open Issues/Questions? is only about the return value of the function. Should I create a poll for that? Should I leave it like this and decide during Voting phase? This question raised during the mailing discussion and never moved forward, can you advice please? Or I should delete the Open Issues/question and move forward with the current approach I choosed? A lot of days passed by already, and I would like to move on with the RFC process. Thanks in advance. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
> It appears the discussion came to a halt. How do you plan to proceed > with your RFC? Thanks for the heads up email Tim. People, following the procedure, this is a "heads up" email to all of you regarding this RFC. If no significant issue/complain raises, then I will move the RFC for voting by Wednesday 21 of September. I also want to thank your support in the github pull request, the hearts and thums up emojis were awesome (https://github.com/php/php-src/pull/9399) Regards Juan. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] Error behaviour for max_input_vars
El mar., 13 de septiembre de 2022 20:01, Derick Rethans escribió: > On 13 September 2022 19:36:15 BST, juan carlos morales < > dev.juan.mora...@gmail.com> wrote: > >El mar., 13 de septiembre de 2022 15:33, juan carlos morales < > >dev.juan.mora...@gmail.com> escribió: > > > >> > >> > >> El mar., 13 de septiembre de 2022 14:58, Mel Dafert > >> escribió: > >> > >>> > >>> In summary, I believe this can only be solved inside of PHP itself, by > >>> allowing to configure a way for `max_input_vars` to abort the request > >>> instead of truncating the input. > >>> The options I see feasible are: > >>> - A new ini setting `max_input_vars_abort` (default to 0), which, if > set > >>> to 1, will abort the request if there are more input variables than > >>> allowed. > >>> - A method to reliably detect whether the input vars were truncated > (eg. > >>> `function has_post_been_truncated(): bool`), so the application can > >>> decide whether to abort or not. > >>> - Deciding that `max_input_vars` is not relevant anymore and should be > >>> handled by the likes of Apache and NGINX, thus changing the default to > >>> `0` and removing the setting > >>> over a deprecation period. > >>> > >>> I am leaning towards the first option, but would be open to either > >>> outcome. > >>> > >> > >> > >> We should not delete the ini setting "max_input_vars"... Is a breaking > >> change very hard. > >> > >> I Am in favour of adding More flexibility about how to handle this > >> situation... And I also think that options 1 and 2 can coexist smoothly. > >> > >> I suggest you write and RFC for this and continue the discussion on this > >> e-mail list but with the RFC already created. > >> > > > > > >Check this out > > > >https://wiki.php.net/rfc/howto > > That's quite a condescending thing to say, considering that Mel has > already successfully passed an RFC ( > https://wiki.php.net/rfc/intldatetimepatterngenerator). > > cheers > Derick > I did not know that :=) You know My intention was to be supportive 100% :=) Thanks for pointing It. Have a good day >
Re: [PHP-DEV] Error behaviour for max_input_vars
El mar., 13 de septiembre de 2022 15:33, juan carlos morales < dev.juan.mora...@gmail.com> escribió: > > > El mar., 13 de septiembre de 2022 14:58, Mel Dafert > escribió: > >> >> In summary, I believe this can only be solved inside of PHP itself, by >> allowing to configure a way for `max_input_vars` to abort the request >> instead of truncating the input. >> The options I see feasible are: >> - A new ini setting `max_input_vars_abort` (default to 0), which, if set >> to 1, will abort the request if there are more input variables than >> allowed. >> - A method to reliably detect whether the input vars were truncated (eg. >> `function has_post_been_truncated(): bool`), so the application can >> decide whether to abort or not. >> - Deciding that `max_input_vars` is not relevant anymore and should be >> handled by the likes of Apache and NGINX, thus changing the default to >> `0` and removing the setting >> over a deprecation period. >> >> I am leaning towards the first option, but would be open to either >> outcome. >> > > > We should not delete the ini setting "max_input_vars"... Is a breaking > change very hard. > > I Am in favour of adding More flexibility about how to handle this > situation... And I also think that options 1 and 2 can coexist smoothly. > > I suggest you write and RFC for this and continue the discussion on this > e-mail list but with the RFC already created. > Check this out https://wiki.php.net/rfc/howto
Re: [PHP-DEV] Error behaviour for max_input_vars
El mar., 13 de septiembre de 2022 14:58, Mel Dafert escribió: > > In summary, I believe this can only be solved inside of PHP itself, by > allowing to configure a way for `max_input_vars` to abort the request > instead of truncating the input. > The options I see feasible are: > - A new ini setting `max_input_vars_abort` (default to 0), which, if set > to 1, will abort the request if there are more input variables than > allowed. > - A method to reliably detect whether the input vars were truncated (eg. > `function has_post_been_truncated(): bool`), so the application can > decide whether to abort or not. > - Deciding that `max_input_vars` is not relevant anymore and should be > handled by the likes of Apache and NGINX, thus changing the default to > `0` and removing the setting > over a deprecation period. > > I am leaning towards the first option, but would be open to either outcome. > We should not delete the ini setting "max_input_vars"... Is a breaking change very hard. I Am in favour of adding More flexibility about how to handle this situation... And I also think that options 1 and 2 can coexist smoothly. I suggest you write and RFC for this and continue the discussion on this e-mail list but with the RFC already created. >
Re: [PHP-DEV] Increase maximum size of an uploaded file to 50Mbyte
I also agree that increasing the size to something bigger than 8M might not be a good idea; I can imagine that a value bigger than 8M (like 50M) will cause an impact in hosting platforms specially, which will be forced to always change the php's default values to a lower one, because of potential DoS Attacks. Default settings should have a reasonable level of security in mind. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] Re: Increase maximum size of an uploaded file to 50Mbyte
> > I can sense we are all feeling like upping the value here. > > If we take it slow and move from 2MB to 8MB that would ease into it, and > see how it goes in our next release ? > > Do we RFC this? Who is volunteering to make it, > There is already a PR in GitHub the author should have Priority on this, otherwise I volunteer myself to help the author and work on this together.
Re: [PHP-DEV] Re: Increase maximum size of an uploaded file to 50Mbyte
I looked for a potential problem out of doing such a change but I could not find anything. But for sure such a change should be well marked in case is merge. I did not take look at the PR as that can wait also. Lets respect the RFC protocol and wait the specified time to see if someone else comes with something. By the Way... This needs an RFC right?
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El lun, 29 ago 2022 a las 11:26, Deleu () escribió:
>
>
>
> On Mon, Aug 29, 2022 at 11:19 AM juan carlos morales
> wrote:
>>
>> El lun, 29 ago 2022 a las 11:06, Deleu () escribió:
>> >
>> > Has the option of returning a Result object been discussed/considered? Can
>> > it be an option? I imagine that if `json_validate(): JsonValidationResult`
>> > always returns a `JsonValidationResult` which contains a `public readonly
>> > bool $valid` and a `public readonly ?string $error` it would be better
>> > than both options on the table right now. The option of returning CLI-like
>> > results means that we will need a `if (! json_validate())` to treat a
>> > valid JSON (really awkward) and the option of using `json_last_error()`
>> > relies on an internal state instead of an immutable structure.
>> > Effectively, what we need is to return a complex structure which can
>> > contain a boolean and a string and that is a class/object.
>> >
>> Interesting, it was not considered.
>>
>> I still think BOOLEAN is my preferred choice, but can you provide an
>> example about how the code would look like with your approach? Imagine
>> you have the feature like in your suggestion ... how a developer would
>> write the code?
>
>
> ```
> $json = '{}';
>
> $result = json_validate($json);
>
> if (! $result->valid) {
> throw new \Exception('Invalid JSON provided: ' . $result->error);
> }
>
> $something->saveUserProvidedJson($json);
> ```
>
> --
> Marco Deleu
Thanks for participating on the discussion and I will take note about
this also, but ...
Quick question: Is in PHP some rule, norma, estandard about "result
objects" already? (like, "should have these methods, and implement
this interface, etc.)
FYI: Just want to clarify that there is nothing bad/wrong about
relying on "internal state" by calling json_last_error(), as this
pattern is used not only by json_decode() but also mysqli extension
for example, among others. Nothing wrong honeslty IMO.
RFC: https://wiki.php.net/rfc/json_validate
Implementation: https://github.com/php/php-src/pull/9399
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El lun, 29 ago 2022 a las 11:06, Deleu () escribió: > > Has the option of returning a Result object been discussed/considered? Can it > be an option? I imagine that if `json_validate(): JsonValidationResult` > always returns a `JsonValidationResult` which contains a `public readonly > bool $valid` and a `public readonly ?string $error` it would be better than > both options on the table right now. The option of returning CLI-like results > means that we will need a `if (! json_validate())` to treat a valid JSON > (really awkward) and the option of using `json_last_error()` relies on an > internal state instead of an immutable structure. Effectively, what we need > is to return a complex structure which can contain a boolean and a string and > that is a class/object. > Interesting, it was not considered. I still think BOOLEAN is my preferred choice, but can you provide an example about how the code would look like with your approach? Imagine you have the feature like in your suggestion ... how a developer would write the code? -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
There is still 1 open question on the RFC, and is about the return value. https://wiki.php.net/rfc/json_validate#open_issuesquestions I would appreciate your feedback on this. Even though I was told the RFC can go with 2 votings, I would like to know your thoughts about that open question, in short, the return value. The same thing was asked during code review of the pull request. I already expressed my opinion in the RFC, I prefer to return boolean and check errors using json_last_error() and I wrote down the details about this; instead of returning INT ... with 0 or JSON_ERROR_NONE representing successful validation. Thanks in advance. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El sáb, 27 ago 2022 a las 2:50, David Gebler () escribió: > > You can always offer two votes on the RFC - one for the function itself, then > one for should it return boolean or return an int representing the > json_last_error constants and let it be decided that way. Thanks for the advice I did not know that. At the moment this is the only open question for this RFC, and I would like to know what the people thinks about it, otherwise, will go with 2 votings. > I think on the whole, I agree with the sentiment that returning boolean and > checking json_last_error() on false is probably the best / least worst option. > So if I could vote, I would vote yes and for the boolean option, with a > secondary preference for returning int if boolean option is rejected. Thanks for the feedback. > And I was unconvinced about the whole idea originally, so a good example of > where positive, robust discussion can change someone's mind. That is why we have these discussions :) thanks a lot for taking your time on this. > Good luck with progressing the RFC, I don't think I have anything else to add. Thanks a lot! -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
OMG, I need to take a rest, sorry for this, here it goes again; the about JSON_INVALID_UTF8_IGNORE opinion is the same, but previous code was wrong Code: string(5) "dummy" } string(8) "No error" Saying so, now ... yes I support and think is NEEDED the usage of the JSON_INVALID_UTF8_IGNORE , as json_validate() result goes in the same direction with json_decode(). I think we need to have this flag. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El sáb, 27 ago 2022 a las 1:31, juan carlos morales
() escribió:
>
> === Open issues/concerns ===
>
>
> @@@ Usage of JSON_INVALID_UTF8_IGNORE @@@
>
> - I have my doubts now, because of this codes:
>
> ```
>
> var_dump(json_validate("\"a\xb0b\""), json_last_error_msg());
> var_dump("");
> var_dump(json_validate("\"a\xb0b\"", 512, JSON_INVALID_UTF8_IGNORE),
> json_last_error_msg());
> ```
>
> Results:
>
> bool(false)
> string(56) "Malformed UTF-8 characters, possibly incorrectly encoded"
> string(12) ""
> bool(true)
> string(8) "No error"
>
>
> Gives different results, but ...
>
> ```
>
> var_dump(json_decode("{ \"a\xb0b\" : \"dummy\" }"), json_last_error_msg());
> var_dump("");
> var_dump(json_decode("{ \"a\xb0b\" : \"dummy\" }", 512,
> JSON_INVALID_UTF8_IGNORE), json_last_error_msg());
> ```
>
> Results in:
> NULL
> string(56) "Malformed UTF-8 characters, possibly incorrectly encoded"
> string(12) ""
> NULL
> string(56) "Malformed UTF-8 characters, possibly incorrectly encoded"
>
>
>
> So at a first look, seems the flag should also be remove, as with
> json_validate() we say its valid, but then on json_decode() we get
> NULL, even with the JSON_INVALID_UTF8_IGNORE provided; does not make
> sense I believe.
>
> What do you think? Is there a use case I am missing here? Is this flag
> worth to have still?
>
I made a huge mistake sorry, long day here, 1:30 am , the code for
json_decode should be as:
var_dump(json_decode("{ \"a\xb0b\" : \"dummy\" }"), true, 512,
json_last_error_msg());
var_dump("");
var_dump(json_decode("{ \"a\xb0b\" : \"dummy\" }", true, 512,
JSON_INVALID_UTF8_IGNORE), json_last_error_msg());
Resulting in
NULL
bool(true)
int(512)
string(56) "Malformed UTF-8 characters, possibly incorrectly encoded"
string(12)""
array(1) { ["ab"]=> string(5) "dummy" }
string(8) "No error"
Saying so, now ... yes I support the usage of the
JSON_INVALID_UTF8_IGNORE , as json_validate() result goes in the same
direction with json_decode(). I think we need to have this flag.
RFC: https://wiki.php.net/rfc/json_validate
Implementation: https://github.com/php/php-src/pull/9399
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
I now provide an update of the discussion. The good, the bads, the open questions, etc. All of this will go into the RFC also, as requested by the procedure in https://wiki.php.net/rfc/howto "Listen to the feedback, and try to answer/resolve all questions. Update your RFC to document all the issues and discussions. Cover both the positive and negative arguments. Put the RFC URL into all your replies." === UPDATES === - Different users have tested the functionality and obtained the promissed results. Also their feedback about it was positive. - Most part of the community in the mailing list showed a positive opinion about this RFC, and looks forward for its integration into PHP. - The ones that checked the code also agree that is small implementation, easy to mantain, and at the same time provides a big benefit for such small implementation. - The community got involve very actively in the discussion of the RFC and provided all kind of useful feedback, and also took the time to test json_validate() by themselves. === Bad reasons for json_validate() provided by the community === - One member of the mailing list expressed that: 1) Incorporating such a small implementation that can be achieve with userland code is not a good idea. Quote: "If we keep the tendency to pollute already bloated standard library with an army of small functions that could have not exists and be replaced with normal PHP counterparts IMHO we'll end with frustration from developers as I believe DX slowly falls down here." 2) json_validate() would only be useful for edge cases. Quote: "A `json_decode()` is a substitute that IMO solves 99% of use cases. If I'd follow your logic and accept every small addition that handles 1% of use cases, somebody will raise another RFC for simplexml_validate_string or yaml_validate and the next PhpToken::validate. All above can be valid if we trust that people normally validate 300MB payloads to do nothing if they DON'T fail and there is nothing strange about that." 3) The user also provided an implementation of a JSON parser written in pure PHP. https://gist.github.com/brzuchal/37e888d9b13937891c3e05fead5042bc === Good reasons for json_validate() provided by the community === @@@ Use cases provided by some members, I quote: - "Yes well-formed JSON from a trusted source tends to be small-ish. But a validation function also needs to deal with non-well-formed JSON, otherwise you would not need to validate it." - "If with a new function (json_validate()) it becomes much easier to defend against a Denial-of-Service attack for some parts of a JSON API, then this can be a good addition just for security reasons." - "fast / efficient validation of a common communication format reduces the attack surface for Denial-of-Service attacks." @@@ Memory usage - During the test of json_validate() from some users, they were happy about the memory usage that was zero in most cases (which is the main benefit out this feature). Someone also did a test with a very large string (75 MB) and only a few bytes were needed as reported by him; also the same user reported an execution speed improvement by a 20-25% over using json_decode(). @@@ Reasons not to depend on userland JSON parsers Even possible to write an excellent JSON parser in PHP like one of the members in the mailing list provided us, there are good reasons for dont relying on userland solutions. # 1 - User Tim Düsterhus provided nice thoughts about this, in favor to json_validate(), ... I quote him: - "Writing a JSON parser is non-trivial as evidenced by: https://github.com/nst/JSONTestSuite. I expect userland implementations to be subtly buggy in edge cases. The JSON parser in PHP 7.0+ is certainly more battle-tested and in fact it appears to pass all of the tests in the linked test suite." - "Even if the userland implementation is written very carefully, it might behave differently than the native implementation used by json_decode() (e.g. because the latter is buggy for some reason or because the correct behavior is undefined). This would imply that an input string that was successfully validated by your userland parser might ultimately fail to parse when passed to json_decode(). This is exactly what you don't want to happen." (Some other members including me, also share this opinion.) # 2 - The JSON parser in PHP follows an special convention, marked in the PHP documentation. # 3 - We already have a JSON parser in PHP, that is used by json_decode(); reusing the existing JSON Parser provides 100% compatibility between the validation of a json-string, and the decoding of it. # 4 - The user Larry Gafield also provided good reason to integrate this implementation into PHP. I quote him: "The heuristic I use is that an API should be "reasonably complete" in one location. Having a half-assed API in C and the rest left to inconsistent and redundant user-space implementations is a terrible API; the same would apply for a user-space li
Re: [PHP-DEV] ksort breaking change
Oh! My bad. Nice work! Cheers El vie., 26 de agosto de 2022 16:51, Tim Düsterhus escribió: > Hi > > On 8/26/22 16:15, juan carlos morales wrote: > > Seems like we Don't Have tests for the function or we should enhance the > > existing ones. If We would Have proper tests the FIX would break it (as I > > Don't see tests in the fix itself).That is why I say so. > > > > Various tests were adjusted in: > https://github.com/php/php-src/pull/9293/files > > Best regards > Tim Düsterhus >
Re: [PHP-DEV] ksort breaking change
Seems like we Don't Have tests for the function or we should enhance the existing ones. If We would Have proper tests the FIX would break it (as I Don't see tests in the fix itself).That is why I say so. I Am not in my computer atm. I Will check this later El vie., 26 de agosto de 2022 15:01, Christian Schneider < cschn...@cschneid.com> escribió: > Am 26.08.2022 um 14:19 schrieb Christoph M. Becker : > > On 26.08.2022 at 05:15, Go Kudo wrote: > > > >> In the actively supported version of PHP, `ksort()` has been modified to > >> include BC Break. > >> > >> https://github.com/php/php-src/issues/9296 > >> > >> This may seem like an appropriate bug fix, but it is a clear BC Break. I > >> think this change should only be introduced in PHP 8.2 and later. > > > > In this case, the functions didn't behave as documented, namely to > > conform to the general conversion rules, which had a relevant change in > > PHP 8.0. Apparently, this case has been overlooked when the change had > > been implemented, and only been noticed recently (what still surprises > > me). Anyway, fixing the issue now is not really introducing a BC break, > > since code relying on the previous behavior did not conform to the > > documentation. > > > I don't really agree with your definition of BC break. > > The behavior of the function with a mix of numeric/non-numeric string keys > changes from 8.1.9 to 8.1.10 which in my world qualifies as a BC break. > > Regards, > - Chris > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: https://www.php.net/unsub.php > >
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie., 26 de agosto de 2022 13:08, juan carlos morales < dev.juan.mora...@gmail.com> escribió: > > Is there a better way to do validation other than json_decode? > Better way than json_decode... Always from the core perspective >
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
Just Want to remind that this discussion is not about "a json parser can be written in PHP or not?". We Have a JSON parser already in the core, ready to be use for validation. Does it make sense to have another parser in User land to do validation if We already have one? Is there a better way to do validation other than json_decode? El vie., 26 de agosto de 2022 12:48, Michał Marcin Brzuchalski < michal.brzuchal...@gmail.com> escribió: > Hi Tim, > > pt., 26 sie 2022 o 12:15 Tim Düsterhus napisał(a): > >> Hi >> >> On 8/26/22 11:14, Hans Henrik Bergan wrote: >> >> you can't efficiently validate JSON in userland >> > >> > Has anyone actually put that claim to the test? Has anyone actually >> made a >> > userland json validator (not just wrap json_decode()/json_last_error()) >> for >> > performance comparison? >> > ( if not, https://www.json.org/JSON_checker/JSON_checker.c would >> probably >> > be a good start) >> > >> >> Worded like "you can't efficiently" the claim is false. Of course you >> can memory-efficiently validate the input by traversing the string byte >> by byte and keeping track of the nesting. >> >> However the points that make a userland implementation infeasible are: >> >> 1. Writing a JSON parser is non-trivial as evidenced by: >> https://github.com/nst/JSONTestSuite. I expect userland implementations >> to be subtly buggy in edge cases. The JSON parser in PHP 7.0+ is >> certainly more battle-tested and in fact it appears to pass all of the >> tests in the linked test suite. >> >> 2. Even if the userland implementation is written very carefully, it >> might behave differently than the native implementation used by >> json_decode() (e.g. because the latter is buggy for some reason or >> because the correct behavior is undefined). This would imply that an >> input string that was successfully validated by your userland parser >> might ultimately fail to parse when passed to json_decode(). This is >> exactly what you don't want to happen. >> > > Now this is an argument I could think of. > But that one is not even mentioned in RFC. > > The JSON_checker.c example delivered by json.org is probably not > something impossible > as it required around 1h of work to port it see working implementation > here https://gist.github.com/brzuchal/37e888d9b13937891c3e05fead5042bc > > Cheers, > Michał Marcin Brzuchalski >
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie, 26 ago 2022 a las 11:55, Peter Bowyer () escribió: > > That's a good point which I had overlooked. Does an exception make most > sense when this happens? > > Peter I just got a code review from Derick Rethans explaining me that a validation routine should not throw an exception. I think is a good idea to take it out, after all , the depth parameter is part of the validation too. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie, 26 ago 2022 a las 11:43, Andreas Leathley () escribió: > > On 26.08.22 11:00, Michał Marcin Brzuchalski wrote: > > There is already a way to validate XML in PHP, and Yaml or PHP is > something within the control of a PHP programmer, while JSON is mostly > used as a format for communication in APIs, so you never know what you > get. If with a new function it becomes much easier to defend against a > Denial-of-Service attack for some parts of a JSON API, then this can be > a good addition just for security reasons. > > But this reason, which most resonates with me, is currently missing in > the RFC, so I would suggest to add that fast / efficient validation of a > common communication format reduces the attack surface for > Denial-of-Service attacks. > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: https://www.php.net/unsub.php > For sure I will add this. Thanks a lot !! That is exactly why we are having this discussion. Once again, Thanks! RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie, 26 ago 2022 a las 11:35, Michał Marcin Brzuchalski () escribió: > Your examples are a couple of functions. > Assuming that they're heavily used is as true as my assumptions. > > Cheers, Is good you clarify that your numbers were assumptions. By the way, I never said anything about how my examples were use in real life, I just provided them as examples of a "need"/requirement to do something. Thanks for participating. Cheers. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie, 26 ago 2022 a las 11:26, juan carlos morales () escribió: > > How can you make such an assertion in those numbers (99% of use cases > and son on, that you mention) ? Can you give us more information about > this assertions? > > I have provide real examples where the need to validate-only a > json-string is actually needed, also the need from our developers > community asking for this. > > RFC: https://wiki.php.net/rfc/json_validate > > Implementation: https://github.com/php/php-src/pull/9399 By the way you dont need to frame this function to handle 300MB of json-string, as I already mentioned a couple of times, that a 2 MB json-sring validation with json_decode() needs a large amount of memory to be validated, possibly hitting the allowed memory limit, which usually is not that high at all. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie, 26 ago 2022 a las 11:00, Michał Marcin Brzuchalski () escribió: > > A `json_decode()` is a substitute that IMO solves 99% of use cases. > If I'd follow your logic and accept every small addition that handles 1% of > use cases, somebody will raise another RFC > for simplexml_validate_string or yaml_validate and the next > PhpToken::validate. > All above can be valid if we trust that people normally validate 300MB > payloads to do nothing if they DON'T fail and there is nothing strange about > that. > > Cheers, > How can you make such an assertion in those numbers (99% of use cases and son on, that you mention) ? Can you give us more information about this assertions? I have provide real examples where the need to validate-only a json-string is actually needed, also the need from our developers community asking for this. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie, 26 ago 2022 a las 10:56, Paul Crovella () escribió: > > I'm for this as well. Though something more than a boolean is useful to > indicate when the max depth has been exceeded as that's not strictly a > problem with the json's validity. Thanks, seems to be good idea to take it out, and I will. Nice observation. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie, 26 ago 2022 a las 10:28, Peter Bowyer () escribió: > > Hi Juan, > > On Thu, 25 Aug 2022 at 17:02, juan carlos morales > wrote: >> >> RFC: https://wiki.php.net/rfc/json_validate > > > Thanks for bringing forward this RFC. I am in favour of this change, as you > can't efficiently validate JSON in userland. > > Like Rowan I'm not convinced JSON_THROW_ON_ERROR belongs here. I can't think > of a case where more than a boolean response is needed. > > Peter Seems that will be a good idea to take out the usage of the flag JSON_THROW_ON_ERROR after all. That was a nice and objective critic and I appreciate it. Thanks! I will take it out of the implementation. Regards RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
El vie, 26 ago 2022 a las 6:47, Michał Marcin Brzuchalski () escribió: > > > I share the same opinion you expressed here even though you admit in recent > email that you changed your mind. > > In recent versions we tend to accept more and more small standard library > functions with IMO questionable argumentation. The same goes here and I'm not > convinced we should introduce next small function that can be simply > implemented in user land. Sorry but I dont think that a JSON parser with memory usage zero (or maybe a few bytes) can be simply done in the userland. This function (json_validate) is small by itself, but gives you access to the JSON parser. Can you please provide an example of what you commented? > Any example testing > 3MB JSON string are for me edge cases that normally > don't happen often to deserve special treatment. I don't agree with your definition of "edge case" here, as edge cases depend/belong on/to the system under analysis. By the way, the test case provided in the PR , "test 005" uses a json-string of about 3 MB (maybe 3.1) , and in order to decode it json_decode() needs something around 109 MB of memory. For me, validation can be done in a better and efficient way, like it has been probe with this proposal. > If we keep the tendency to pollute already bloated standard library with an > army of small functions that could have not exists and be replaced with > normal PHP counterparts IMHO we'll end with frustration from developers as I > believe DX slowly falls down here. Last but not least, I want to say that the function json_validate() is small, easy to maintain and extend if needed, and at the same gives us access to something that is not trivial to write in userland the existing JSON parser. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
> Having actually compiled the branch and tried it out, I have to say > regardless of whether validating arbitrarily large blocks of JSON without > being interested in the contents is a common or more niche use case, the > memory savings ARE highly impressive. I had thought that because the function > was built on top of the existing parser and is still parsing the entire > string (or up until invalid JSON is encountered), the performance saving for > a very large input would be smaller than it is. > > I tested using a 75MB valid JSON input - a string large enough that it's not > going to be very common. The processing time isn't hugely different, the > saving appears to be around maybe 20-25% (and it's not a significant amount > of time using either json_decode or json_validate, even on an input of this > size, about half a second on my machine for both). But the memory saving is > enormous, almost total. Gone from needing ~5x the size of the input to almost > literally just a few extra bytes. > > I'm persuaded now on both that benchmarking and having had a closer look at > the implementation PR, which is clearly a minimal and easily maintainable > change. > > As I've said, my feelings are irrelevant to the extent I'm not a voter, but I > am in principle a +1 thumbs up for including this now. > > David, thanks very much for sharing your results and opinion. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
I also added in the RFC your concern about JSON_THROW_ON_ERROR usage. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
Ok then ... I take it out of the RFC just because we have doubts about it. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
Hey Tim, thanks a lot for participating and taking your time to use the code and making your own tests, I highly appreciate it. Regards. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399 -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
Hello Rowan thanks for participating. My answer for your comments would be: === Regarding JSON_THROW_ON_ERROR === I think is a valid point from your side, and I am open to change the implementation if the maintainers believe so. I leave it there as it existed in json_decode() and wanted to give the developer the option to use it or not. But as I said, you hae a valid point, that I also like. === Regarding the examples you mentioned === The one from humhub and Symfony http-kernel, yes you are right, I will take them out from the RFC right now. Nice catch. But the Magento one, I took a look again, and I still believe is right example, because the only reason is using json_decode() for, is to validate only. Am I missing something here? Regards. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
Thanks for participating. Cheers. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399
Re: [PHP-DEV] RFC json_validate() - status: Under Discussion
Hello David, thanks for your feedback. I believe that the answer to most of your concerns/questions related to other projects that would benefit out of something like this is already answered by me in the RFC. I mentioned major PHP projects that will directly benefit out of something like this. About the demand, well, that is also being written in the RFC. Regarding the question in StackOverflow, was just mentioned to expose the "demand" of validating a json string, and is one of the most ranked questions, is in the top 10 of questions with php && json tags. I believe that such a thing is talking by iself. Providing real use cases wont make a difference, because the number of examples are infinite honestly. Yes there are userland packages that can do this for us, but in big products on big companies, relying on 3rd party packages is not always cool, I have being experiencing that in my company actually. Regarding examples of raising issues ... there are a LOT in github, and the only possible solution is always to increase the memory limit or use a 3rd party package (which as I said, is not always suitable). But ... lets be objective, or try to be at least ... PHP has a parser already, this functionality is just allowing us to use it without the need to consume memory. I see it useful, actually, this RFC is the result of a real need we have in my current company at the moment, and after digging around the internet I found hat we were not the only ones. Thanks for taking our time on this, I appreciate it.
[PHP-DEV] RFC json_validate() - status: Under Discussion
Hello. I am glad to present to you the RFC for json_validate() function. The code/implementation still needs some workout, but seems to be fine enough to be presented to you all. I look forward for feedback Thanks in advance. Juan. RFC: https://wiki.php.net/rfc/json_validate Implementation: https://github.com/php/php-src/pull/9399
[PHP-DEV] RFC karma request
I kindly ask to get RFC Karma, I already have wiki account. RFC: is_json() function I have previously write an email to Internals regarding this feature I would like to include, and feedback was positive (email subject RFC Idea - is_json - looking for feedback). Thanks in advance. Juan
Re: [PHP-DEV] [RFC] Asymmetric visibility
Quite a feature to the language. Nice I wonder if can be done using a different wording. If I can get an idea to contribute I Will write again. Keep it up!
Re: [PHP-DEV] Re: RFC Idea - is_json - looking for feedback
Hello Ollie. Dont be sorry. Actually I appreciate every feedback I got. I promise I Will not dissapoint anyone. I Will prepare de RFC next week. Regards. Juan El dom., 7 de agosto de 2022 14:03, Ollie Read escribió: > Hey, > > While I'm not opposed to the idea, I'm struggling to think of a way to > accurately determine whether a string is true JSON that doesn't involve > some sort of parsing. > > Sure some regexes can be run on it, but I'm not sure that will ever be > 100% accurate. Any parsing of the JSON to determine whether or not it's > valid JSON would, in most situations, lead down a code path that then > parsed the JSON again, essentially repeating the same function for little > benefit. > > I personally like to use the *`JSON_THROW_ON_ERROR`* flag on json_decode. > > Not trying to rain on your parade, so sorry if it comes across that way, > I'm just not sure of a way to do it where the benefit outweighs the cons, > but then again, there are people a lot smarter than me on this list. > > --- > Best Regards, > *Ollie Read* >
Re: [PHP-DEV] RFC Idea - is_json - looking for feedback
Hi Juan, > > Thanks for your efforts, and examples. > > You are solving a common use case, in frameworks, libs, and general > userland. > > This much needed function will see good perf/memory improvements once > adoption of is_json() hits popular libs and frameworks. > > I will be voting in favor of this addition, subject to reviewing the > implementation and full RFC. > > Thanks for the effort. Looking forward to seeing it merged :) > > Many thanks, > Paul Dragoonis > > > >> Hello Paul, and sorry, I totally missed your message, thanks for the kind feedback, you made my day. So, if I get it correctly, I have green light to move on with the RFC process right? so ... I will move on. Once everything is done I will ping again. Regards, Juan PD: I had to re-send the message as it was too long, ... I am using a mailing list for the first time ever :)
Re: [PHP-DEV] RFC Idea - is_json - looking for feedback
(I received an email from mailer-dae...@lists.php.net , I had to cut the message history as the mail server complained because the email got too long) Larry, here is the same benchmark including memory_get_peak_usage as requested (more after the benchmark) - Benchmark including memory_get_peak_usage - # json_decode() Megas used before call: 79.23828125 PEAK Megas used before call: 128.00390625 //< Megas used after call: 3269.23828125 PEAK Megas used after call: 3309.23828125 //< Difference: 3190 PEAK Difference: 3181.234375 //<- Time: 12.109144926071 seconds # is_json() Megas used before call: 79.23828125 PEAK Megas used before call: 128.00390625 //< Megas used after call: 79.23828125 PEAK Megas used after call: 128.00390625 //< Difference: 0 PEAK Difference: 0 //<- Time: 5.4504480361938 seconds I want to clarify, as I previously mentioned, I did not develop any parser at all, the parser already exists in PHP and is use by json_decode(), in short words , what I propose is to create a new function, as an interface to THE EXISTING PARSER, only, and only, to check if a string is a valid json-string or not. I already provided a lot, but a lot of real usages of this, and at the moment the only way to do it (without using any 3rd package) is by using json_decode() , and the drawback is that json_decode() generates an object/array while parsing the string provided affecting the memory associated with the php process; and sometimes you just need to check if an string is valid json-string or not and nothing else; I have provided code from major open-source projects, and also THE question in stack overflow, which is viewed near 500k times, which indicates somehow the number of users interested in something like this. And yes, we can use a 3rd party package, but in companies, using a 3rd party package is not that simple, believe me. Regarding publishing the a PR with my implementation, is out of context at the moment, because we are discussing if a functionality like this should be included or not, and nothing else. Also I am pretty sure that if this gets a YES from the community (so far seems so), and I show my implementation, someone will tell me that it could be done in a different way , and is totally fine, is great actually. But now, at this moment, everything is about the functionality. - I also want you to know that I really, really appreciate the time you took to participate on this and being direct and honest . THANKS A LOT (I really mean it). == People I have made my best effort providing you all with as much information and answers as possible. At the end is up to the community, if YES ... great , if NO ... great anyway, it has being a beautiful trip so far, but I cannot lie to you, I would really like to see whats next. I send you all my best wishes. Juan >
Re: [PHP-DEV] RFC Idea - is_json - looking for feedback
El dom, 31 jul 2022 a las 0:56, Deleu () escribió:
>
>
> On Sat, Jul 30, 2022, 4:48 PM David Gebler wrote:
>
>>
>>
>> What I'm asking is what's the practical use for this proposed function?
>> Where are you likely to need to know if a string is valid JSON but not
>> have
>> to (try to, with error handling) parse it almost immediately afterwards
>> anyway?
>>
>
> I'm still on the fence over the general idea, but I thought I could at
> least address this question in particular.
>
> I can definitely see it's usefulness on public HTTP API ingesting data
> (specially large data) where if the payload is a valid JSON, it gets stored
> and processed by a background queue, making it so that the HTTP layer can
> either reject the request with a 4xx status or accept it and only truly
> decode it on a separate process that may even be hosted by a separate
> server with larger memory size.
>
Before starting, I want to thank all for taking time from your time, to
give me a feedback, I sincerely respect that, so ... thanks!
Sorry for the long message, but I have the feeling that ... this is it, is
now ... or it will not be, at least not now, so ... here my best effort;
even though, I dont put aside the possibility that I might have an
offuscated view/opinion regarding my proposal, so is possible that I might
not see the things right ... you know .. I am human after all.
So, enough of prologue and let's start
# Why would I have code that checks if a json-string is a valid json, if I
will not use the content inside it? (or something like that)
- Some of you , asked me this question.
- That depends on the infinite amout of use cases that the human brain can
imagine, that is why I say that is not the right approach to discuss this
topic.
# Why this change then?
- Is web related, ergo, is PHP related.
- It does not add complexity into PHP, with this functionality we are using
the existing JSON parser that exists in PHP at the moment, the only thing
we do is to create an interface between "userland" and the parser itself,
without the need to use memory as json_decode() is using to check if a
string is a valid json-string or not.
## Proposed functionality (proposed, subject to changes for sure, as now I
have only a working and dirty prototype for this)
function is_json(string $json, int $flags): bool {}
Returns:
TRUE if json-string is valid json, otherwise returns FALSE.
Exceptions:
Optionally set, for the same or subset of exceptions of the actual
json_decode(), like the one Syntax Error for example.
So far this is it, subject to changes for sure.
# Real open-source project over github using json_decode() just to check if
a json-string is valid ... and nothing else than that.
- I provide here use cases from major projects, where they ned to have code
that has to be able to check if an string is a valid JSON and nothing else
than that.
- Please check the link to see full code
- I provide here some small snippets from the link I provide
- We are not discussing here how they implemented the code, what I want to
show here is that there is a need to check if an string is a valid
JSON-string without th need to create an object/array out of it.
- Also, we are not discussing if they are mejor projects or not. They are
listed in github as major projects, with stars and a big community of users
and developers maintaining them.
- On some snippets I wrote some notes too, because on some of them is not
obvious how a funciton like the one I propose could be useful.
## Symfony Framework
https://github.com/symfony/symfony/blob/870eeb975feb1abb4b8a1722e1fd57beeab2b230/src/Symfony/Component/Validator/Constraints/JsonValidator.php
```
class JsonValidator extends ConstraintValidator
...
...
...
```
## Laravel Framework
https://github.com/laravel/framework/blob/302a579f00ebcb2573f481054cbeadad9c970605/src/Illuminate/Validation/Concerns/ValidatesAttributes.php
```
public function validateJson($attribute, $value)
{
if (is_array($value)) {
return false;
}
if (! is_scalar($value) && ! is_null($value) && !
method_exists($value, '__toString')) {
return false;
}
json_decode($value);
return json_last_error() === JSON_ERROR_NONE;
}
```
https://github.com/laravel/framework/blob/61eac9cae4717699ecb3941b16c3d775820d4ca2/src/Illuminate/Support/Str.php
```
public static function isJson($value)
{
```
## Magento
https://github.com/magento/magento2/blob/7c6b6365a3c099509d6f6e6c306cb1821910aab0/app/code/Magento/User/Block/Role/Grid/User.php
```
private function getJSONString($input)
{
$output = json_decode($input);
return $output ? $this->_jsonEncoder->encode($output) : '{}';
}
```
https://github.com/magento/magento2/blob/7c6b6365a3c099509d6f6e6c306cb1821910aab0/lib/internal/Magento/Framework/DB/DataConverter/SerializedToJson.php
```
protected function isValidJsonValue($value)
{
i
Re: [PHP-DEV] RFC Idea - is_json - looking for feedback
El sáb., 30 de julio de 2022 16:48, David Gebler escribió: > > > On Sat, Jul 30, 2022 at 3:01 PM Nikita Popov wrote: > >> On Fri, Jul 29, 2022 at 4:27 PM juan carlos morales < >> dev.juan.mora...@gmail.com> wrote: >> >> > I am following the RFC guideline for the first time. ( >> > https://wiki.php.net/rfc/howto) >> > >> > As suggested there, I am here to get a feeling from you, regarding the >> > following RFC for PHP. >> > >> > # Change (draft): >> > >> > New function in php called like: >> > >> > is_json(string $string): bool >> > >> > ## Description >> > ### Parameters >> > string $string -> string to find out if is a valid JSON or not >> > >> > ### Return >> > Returns a bool. The function is capable to determine if the passed >> string >> > is a valid JSON (true) or not (false). >> > >> > # Why this function ? >> > >> > At the moment the only way to determine if a JSON-string is valid we >> have >> > to execute the json_decode() function. >> > >> > The drawback about this, is that json_decode() generates an in memory an >> > object/array (depending on parameters) while parsing the string; this >> leads >> > to a memory usage that is not needed (because we use memory for creating >> > the object/array) and also can cause an error for reaching the >> memory-limit >> > of the php process. >> > > In the last 15 years, the only time I've ever needed to know if a string > is valid JSON is if I'm about to decode or otherwise parse it as JSON. If > I'm decoding what I expect to be a large JSON blob, such that memory usage > might be a concern, personally I use > https://github.com/salsify/jsonstreamingparser but the point is userland > solutions are possible. > > What I'm asking is what's the practical use for this proposed function? > Where are you likely to need to know if a string is valid JSON but not have > to (try to, with error handling) parse it almost immediately afterwards > anyway? Unless there is some fairly commonplace use case for this I'm not > thinking of, you're going to be using that extra memory, or using a > streaming parser, at some point in your script regardless. If there is > genuine demand for it, I'd be in favour (I'm not a voting member so kind of > moot but...), otherwise I'm generally against introducing new core > functions which are either edge-case or can be perfectly well dealt with > via userland code. > > Cheers > > -Dave > > >> > >> > Sometimes we just need to know is the string is a valid json or not, and >> > nothing else. >> > >> > # Do we need something like this? If a check to an string is valid JSON >> > then for sure I will have to use it in my code either as an object or an >> > array. >> > >> > Well that is not true. There are plenty of cases where you just need to >> > check if a string is a valid json and that is it. Just looking into >> > stackoverflow will give you an idea about how many people is looking for >> > something like this in an efficient way. >> > >> >> Could you please give some specific examples where the proposed >> functionality would be useful? >> >> Regards, >> Nikita >> > Thanks everybody for the feedback. I think I got to a point that I need to write another e-mail with all my arguments and use cases for this. That is why I kindly ask you all for patience, and I Will write again soon. It will be a long e-mail for sure, and when It comes , I ask you all to take the time to read it. Thanks in advance. >
Re: [PHP-DEV] RFC Idea - is_json - looking for feedback
El sáb., 30 de julio de 2022 11:58, Oleksii Bulba
escribió:
> > $memoryAfter = memory_get_usage(true) / 1024 / 1024;
>
> I see that you used `memory_get_usage` that shows memory usage at the time
> of the function call.
> As far as I understand, your function does not return any value,
> so I suspect that it is obvious that the memory usage after the function
> call is the same as before.
> But what is the actual memory usage during the function call?
> Can you run the same benchmark but with `memory_get_peak_usage` function
> to see how your function uses memory?
>
> # $memoryAfter = memory_get_peak_usage(true) / 1024 / 1024;
>
> Also, I'm concerned if it would be better to name the function
> `is_json_valid`?
>
> On Sat, Jul 30, 2022 at 10:37 AM juan carlos morales <
> dev.juan.mora...@gmail.com> wrote:
>
>> Just want to clarify that when I mentioned the use of memory, I wrote down
>> the function "memory_get_usage()", which basically gives us the memory
>> handle by php that is related to the memory_limit INI setting.
>>
>> Now I will provide a benchmark I have done really quick:
>>
>> # Code used (I have the implementation of is_json() done already)
>>
>> > // make sure you set your memory limit to -1 before running this code
>> // Here we create a very very very big json string, really big
>> $limit = 100;
>> $jsonString = '{ "test": { "foo": "bar" },';
>>
>> for ($i=0; $i < $limit; $i++) {
>> $jsonString .= " \"test$i\": { \"foo\": { \"test\" : { \"foo\" : {
>> \"test\" : { \"foo\" : \"bar\" },";
>> }
>>
>> $jsonString .= ' "test": { "foo": "replaceme" } }';
>> //{ "test" : { "foo" : "bar" }}}
>>
>> $memoryBefore = memory_get_usage(true) / 1024 / 1024;
>> echo "Megas used before call: " . $memoryBefore . PHP_EOL;
>>
>> $start = microtime(true);
>>
>> json_decode($jsonString, null, $limit, 0);
>> //<-- un/comment to show/hide results for json_decode()
>> //is_json($jsonString);
>> //<-- un/comment to show/hide results for is_json()
>>
>> $memoryAfter = memory_get_usage(true) / 1024 / 1024;
>> echo "Megas used after call: " . $memoryAfter . PHP_EOL;
>>
>> echo "Difference: " . ($memoryAfter - $memoryBefore) . PHP_EOL;
>>
>> echo "Time: " . (microtime(true) - $start) . " seconds" . PHP_EOL;
>> return;
>>
>> # Results
>> ## json_decode()
>> Megas used before call: 79.23828125
>> Megas used after call: 3269.23828125
>> Difference: 3190
>> Time: 12.091101884842 seconds
>>
>> ## is_json()
>> Megas used before call: 79.23828125
>> Megas used after call: 79.23828125
>> Difference: 0
>> Time: 5.4537169933319 seconds
>>
>>
>> And yes, I am open to share the implementation, but after I write the RFC.
>>
>> Thanks for taking your time to give me a feedback.
>>
>> El sáb, 30 jul 2022 a las 3:50, Jordan LeDoux ()
>> escribió:
>>
>> >
>> >
>> > On Fri, Jul 29, 2022 at 7:27 AM juan carlos morales <
>> > dev.juan.mora...@gmail.com> wrote:
>> >
>> >> # Why this function ?
>> >>
>> >> At the moment the only way to determine if a JSON-string is valid we
>> have
>> >> to execute the json_decode() function.
>> >>
>> >> The drawback about this, is that json_decode() generates an in memory
>> an
>> >> object/array (depending on parameters) while parsing the string; this
>> >> leads
>> >> to a memory usage that is not needed (because we use memory for
>> creating
>> >> the object/array) and also can cause an error for reaching the
>> >> memory-limit
>> >> of the php process.
>> >>
>> >> Sometimes we just need to know is the string is a valid json or not,
>> and
>> >> nothing else.
>> >>
>> >
>> > You say that you have a first-pass at the implementation done. I'd be
>> > curious to see it. My initial thought was that in order to validate the
>> > string, you likely need to allocate extra memory as part of the
>> validation
>> > that depends on the string size. You'd definitely save the overhead of a
>> > ZVAL, but for such an object that overhead is likely negligible.
>> >
>> > So I guess my question would be: in the actual implementation that
>> lands,
>> > how much memory would this actually save compared to json_decode()? This
>> > seems like it would make the RFC tricky, as the purported benefit of the
>> > RFC depends very tightly on the implementation that lands.
>> >
>> > Jordan
>> >
>>
>
Hello Jordan, thanks for the feedback.
I think the benchmark talks by itself (also for the memory Save question).
Also by the fact that in order to run it for json_decode() rhe memory limit
needs to be super high or -1 (no limit, not a good idea in production
right?)
The advantage here is to be able to parse huge strings without reaching the
memory limit set in the INI settings.
I take It as a "IF THIS IS AS GOOD AS IT SEEMS THEN YES" :D
Once again... Thanks
Re: [PHP-DEV] RFC Idea - is_json - looking for feedback
El sáb, 30 jul 2022 a las 8:30, Dusk () escribió: > On Jul 29, 2022, at 23:05, Aleksander Machniak wrote: > > json_decode() has $depth argument, I think is_json() probably also > should. And I'm not sure about JSON_INVALID_UTF8_IGNORE flag. > > > > My point is that if you use these with json_decode() you might also need > to use these with is_json() > > How about if validation were a flag to json_decode() -- perhaps > JSON_VALIDATE_ONLY -- which made it return a placeholder value like true > instead of the decoded structure? (It can't return null on success; that's > used for errors if JSON_THROW_ON_ERROR isn't set.) > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: https://www.php.net/unsub.php > > I thought about that too, and yes can also be a valid approach that would not interfeer with my development. What I personally dont like about that approach is that a developer is able to specify flags that are related to exclusively with result generation, and when they get mixed with JSON_VALIDATE_ONLY would not make sense. for example: json_decode($jsonStringHere, false, 512, JSON_BIGINT_AS_STRING | JSON_INVALID_UTF8_SUBSTITUTE | JSON_OBJECT_AS_ARRAY | JSON_THROW_ON_ERROR | JSON_VALIDATE_ONLY); So the end result, is not clear by reading the line itself, I mean , the mechanism (of having another constant in json_decode()) is not enough to make the function self-expressive.
Re: [PHP-DEV] RFC Idea - is_json - looking for feedback
Just want to clarify that when I mentioned the use of memory, I wrote down the function "memory_get_usage()", which basically gives us the memory handle by php that is related to the memory_limit INI setting. Now I will provide a benchmark I have done really quick: # Code used (I have the implementation of is_json() done already) ) escribió: > > > On Fri, Jul 29, 2022 at 7:27 AM juan carlos morales < > dev.juan.mora...@gmail.com> wrote: > >> # Why this function ? >> >> At the moment the only way to determine if a JSON-string is valid we have >> to execute the json_decode() function. >> >> The drawback about this, is that json_decode() generates an in memory an >> object/array (depending on parameters) while parsing the string; this >> leads >> to a memory usage that is not needed (because we use memory for creating >> the object/array) and also can cause an error for reaching the >> memory-limit >> of the php process. >> >> Sometimes we just need to know is the string is a valid json or not, and >> nothing else. >> > > You say that you have a first-pass at the implementation done. I'd be > curious to see it. My initial thought was that in order to validate the > string, you likely need to allocate extra memory as part of the validation > that depends on the string size. You'd definitely save the overhead of a > ZVAL, but for such an object that overhead is likely negligible. > > So I guess my question would be: in the actual implementation that lands, > how much memory would this actually save compared to json_decode()? This > seems like it would make the RFC tricky, as the purported benefit of the > RFC depends very tightly on the implementation that lands. > > Jordan >
Re: [PHP-DEV] RFC Idea - is_json - looking for feedback
El vie, 29 jul 2022 a las 22:18, David Rodrigues () escribió: > I was about to say NO, but after being completely your argument, the idea > makes sense. > > Initially I thought about using json_decode() with error capture, but the > idea that it would overload memory makes perfect sense, compared to a > simple structure analysis, if that is indeed the user's intention. The > performance would also be absurdly better. > > What worries me above is the misuse of the function, like checking if > is_json() === true and using json_decode() right after. However, I believe > this can be easily optimized by the engine itself. > > My vote is YES. > > > Atenciosamente, > David Rodrigues > > > Em sex., 29 de jul. de 2022 às 16:32, Michał Marcin Brzuchalski < > michal.brzuchal...@gmail.com> escreveu: > > > Hi Juan, > > > > pt., 29 lip 2022, 16:26 użytkownik juan carlos morales < > > dev.juan.mora...@gmail.com> napisał: > > > > > I am following the RFC guideline for the first time. ( > > > https://wiki.php.net/rfc/howto) > > > > > > As suggested there, I am here to get a feeling from you, regarding the > > > following RFC for PHP. > > > > > > # Change (draft): > > > > > > New function in php called like: > > > > > > is_json(string $string): bool > > > > > > ## Description > > > ### Parameters > > > string $string -> string to find out if is a valid JSON or not > > > > > > ### Return > > > Returns a bool. The function is capable to determine if the passed > string > > > is a valid JSON (true) or not (false). > > > > > > # Why this function ? > > > > > > At the moment the only way to determine if a JSON-string is valid we > have > > > to execute the json_decode() function. > > > > > > The drawback about this, is that json_decode() generates an in memory > an > > > object/array (depending on parameters) while parsing the string; this > > leads > > > to a memory usage that is not needed (because we use memory for > creating > > > the object/array) and also can cause an error for reaching the > > memory-limit > > > of the php process. > > > > > > > Personally I'd vote NO. > > > > Cheers, > > Michał Marcin Brzuchalski > > > > > > > > Thanks for the feedback *** To: Michał Marcin Brzuchalski Thanks for the opinion, but it would be useful for me to know if there is a reason behind the "no" that could help me get better. *** To: David Rodrigues Gracias. Regarding "performance", I did some benchmarks. I tested is_json() vs json_decode() with a json-string with 1 million key-values; is_json was 100% faster than json_decode(), plus memory usage with is_json() was flat! , yes , flat. memory_get_usage() before and after is_json() returned the same value; with json_decode() I had to change the memory limit setting otherwise memory reaches max allowed before finishing parsing it. Regarding use cases, I can say that this idea came up , as in my current company , our product heavily uses json_decode() to know if an string is a valid json, and compromises the memory usage once in a while, plus, performance decreases; that is why this RFC-Idea. The good thing is that I already have something for this. Also in stack overflow, honestly, take a look, and check how much people is needing this. Unit tests could heavily benefit from such a functionality. A quick look in github for php code with a function called "is_json()" exposes tons of projects where developers are writting their own "is_json()" function relaying in json_decode() to make it work. Etc. :) Stay in contact David. and once again, thanks!
[PHP-DEV] RFC Idea - is_json - looking for feedback
I am following the RFC guideline for the first time. ( https://wiki.php.net/rfc/howto) As suggested there, I am here to get a feeling from you, regarding the following RFC for PHP. # Change (draft): New function in php called like: is_json(string $string): bool ## Description ### Parameters string $string -> string to find out if is a valid JSON or not ### Return Returns a bool. The function is capable to determine if the passed string is a valid JSON (true) or not (false). # Why this function ? At the moment the only way to determine if a JSON-string is valid we have to execute the json_decode() function. The drawback about this, is that json_decode() generates an in memory an object/array (depending on parameters) while parsing the string; this leads to a memory usage that is not needed (because we use memory for creating the object/array) and also can cause an error for reaching the memory-limit of the php process. Sometimes we just need to know is the string is a valid json or not, and nothing else. # Do we need something like this? If a check to an string is valid JSON then for sure I will have to use it in my code either as an object or an array. Well that is not true. There are plenty of cases where you just need to check if a string is a valid json and that is it. Just looking into stackoverflow will give you an idea about how many people is looking for something like this in an efficient way. # Who would develop the RFC ? I would develop the RFC with the help of the community if needed. I already have a first version of the is_json() function tested and ready for review. Thanks in advance, I am looking forward to hear your opinion on this. Kind Regards, Juan Carlos Morales.
