Hi, i have enhanced the set of test json files [1] (and related test cases) to make sure that the json parser works correctly and fails correctly if json is invalid.
The current JsonStreamParser.java class does not pass all tests. So i started to fix this but JsonStreamParser was hard to understand for me and i discovered a main problem (see below) which i could not fix and so i wrote a new implementation (JsonByteBufferStreamParser and JsonCharBufferStreamParser [2]). These implementations did pass all new (and old) tests (some older tests had issues because of IMHO wrong tests, see [3]). The main problem with JsonStreamParser was that it fails if the last character in char[] loadedChars, before a direct overflow, is a ending doublequote. Than the overflow happens and something goes wrong with preserving the previously readed value. So if the buffer size (=org.apache.fleece.default-char-buffer) and the json input have the right "bad" correlation then parsing would fail. There are now testcases for this. I did some JMH based benchmarking for the the new parser implementations [4] (based on the ideas from [5]). They show more or less equals figures [6] compared to the original JsonStreamParser. Please let me know what you think about that. Thanks Hendrik [1] https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/json https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core [2] https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonByteBufferStreamParser.java https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/src/main/java/org/apache/fleece/core/JsonCharBufferStreamParser.java [3] https://github.com/salyh/fleece_tmp/commit/8377bdf88f86e0ca07b344a87a48241788e4fd7b#commitcomment-6902656 [4] https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/resources/bench https://github.com/salyh/fleece_tmp/tree/benchmark-streamparser/fleece-core/src/test/java/org/apache/fleece/core/jmh/benchmark [5] https://github.com/RichardHightower/json-parsers-benchmark [6] https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/avg_benchmark_jmh_result_f2_t16_w3_i5.txt https://github.com/salyh/fleece_tmp/blob/benchmark-streamparser/fleece-core/thr_benchmark_jmh_result_f2_t16_w3_i5.txt -- Hendrik Saly (salyh, hendrikdev22) @hendrikdev22 PGP: 0x22D7F6EC
