> Thanks! Ideally there would be one commit to add the minimal portable > version, then separate commits for each optimized variant.
Would you like me to remove the Unsafe based impl from https://github.com/tukaani-project/xz-java/pull/13? > So far I have given it only a quick try. array_comp_incremental seems > faster than xz-java.git master. Compression time was reduced by about > 10 %. :-) This is with OpenJDK 21.0.2, only a quick test, and my > computer is old so I don't doubt your higher numbers. How are you testing? I am using jmh, so it has a warm up period before actually measuring, giving the jvm plenty of opportunity to perform optimizations. If you are doing single shot executions to compress a file, that could provide pretty different results. > With array_comparison_performance the improvement seems to be less, > maybe 5 %. I didn't test much yet but it still seems clear that > array_comp_incremental is faster on my computer. Going back to the previous question, this could be due to fact that I collapsed some class hierarchy in the _incremental pr. This could take the optimizer a bit longer to figure out. > However, your code produces different output compared to xz-java.git > master so the speed comparison isn't entirely fair. I assume there was > no intent to affect the encoder output with these changes so I wonder > what is going on. Both of your branches produce the same output so it's > something common between them that makes the difference. This was definitely not the intent, and I had not noticed this previously. With the 3 files I test with, none have any difference with preset of 3. The smallest file (ihe_ovly_pr.cm) also has no difference at preset 6. With the ~25MB image1.dcm (mostly a greyscale bmp), the PR versions produce more compressed content at preset 6. 1.9 = 4,041,476 PR = 4,004,156 There is a smaller difference with the ~50MB xml file, but strangely, the PR version is slightly bigger. 1.9 = 1,589,512 PR = 1,589,564 Given that I am only seeing differences with preset of 6, I am guessing the difference must be in BT4. The result still seems to be valid (at least the java XZInputStream reads it back correctly). There is clearly a subtle "defect" somewhere, but I cannot tell if it is in the current trunk or the PR. My best guess is that there is an off by 1 error in one or the other. Brett On Thu, Feb 29, 2024 at 11:35 AM Lasse Collin <lasse.col...@tukaani.org> wrote: > > On 2024-02-25 Brett Okken wrote: > > I created https://github.com/tukaani-project/xz-java/pull/13 with the > > bare bones changes to utilize a utility for array comparisons and an > > Unsafe implementation. > > When/if that is reviewed and approved, we can move on through the > > other implementation options. > > Thanks! Ideally there would be one commit to add the minimal portable > version, then separate commits for each optimized variant. > > So far I have given it only a quick try. array_comp_incremental seems > faster than xz-java.git master. Compression time was reduced by about > 10 %. :-) This is with OpenJDK 21.0.2, only a quick test, and my > computer is old so I don't doubt your higher numbers. > > With array_comparison_performance the improvement seems to be less, > maybe 5 %. I didn't test much yet but it still seems clear that > array_comp_incremental is faster on my computer. > > However, your code produces different output compared to xz-java.git > master so the speed comparison isn't entirely fair. I assume there was > no intent to affect the encoder output with these changes so I wonder > what is going on. Both of your branches produce the same output so it's > something common between them that makes the difference. > > I plan to get back to this next week. > > > > One thing I wonder is if JNI could help. > > > > It would most likely make things faster, but also more complicated. I > > like the java version for the simplicity. I am not necessarily looking > > to compete with native performance, but would like to get improvements > > where they are reasonably available. Here there is some complexity in > > supporting multiple implementations for different versions and/or > > architectures, but that complexity does not intrude into the core of > > the xz code. > > I think your thoughts are similar to mine here. Java version is clearly > slower but it's nicer code to read too. A separate class for buffer > comparisons indeed doesn't hurt the readability of the core code. > > On the other hand, if Java version happened to be used a lot then JNI > could save both time (up to 50 %) and even electricity. java.util.zip > uses native zlib for the performance-critical code. > > In the long run both faster Java code and JNI might be worth doing. > There's more than enough pure Java stuff to do for now so any JNI > thoughts have to wait. > > -- > Lasse Collin >
