Hi,
I was also wondering why parts of hppc were forked/copied to Lucene
Core, others not. IMHO it should be consistent.
I alaos agree that we should remove the classes completely from the util
package (public part of API) and move them to the non-exported packages
unter oal.internal. Of course this does not prevent classpath users form
using those classes (P.S.: for the SharedSecrets and Vectorization
theres stack inspection to prevent invalid callers from using them, but
that's not needed for packages here as they cannot bring any risk for
code when keeping public).
+1 to move the classes and fork everything of HPPC to oal.internal
package and only export it to specific modules in the module-info by a
specific export (like for test.framework).
Uwe
Am 26.05.2024 um 10:31 schrieb Dawid Weiss:
I will not have the time for this today but took a quick look and I
think these external dependencies on hppc can be removed after the
work Bruno has done to port some of these utility classes to the core.
I'd also move the entire Lucene hppc fork under internal and only
expose it to other Lucene modules that need it - would have to verify
that no class is part of the public API but I don't think it is (in
spatial3d and spatial-extras).
Dawid
On Sat, May 25, 2024 at 10:08 PM Dawid Weiss <dawid.we...@gmail.com>
wrote:
Hi Chris,
Since Elasticsearch is deployed as a module, then we need to
update to hppc 0.9.1 [2], but unfortunately this is not
straightforward. In fact, Ryan has a PR open [3] for the past
2 years without completion! The iteration order of some
collection types in hppc 0.9.x [*] is tickling some
inadvertent order dependencies in Elasticsearch. It may take
some time to track these down and fix them.
I understand it's a pain if the order changes from run to run but
I don't see a way this can be avoided ([1] is the issue you
mentioned on gh). Tests (and code) shouldn't rely on map/set
ordering, although I realize it may be difficult to weed out in
such a large codebase.
For what it's worth, the next version of HPPC will be a proper
module (with com.carrotsearch.hppc id). Would it change anything/
make it easier if I renamed it to just 'hppc'?
I wonder if others may run into either or both of these
issues, as we have in Elasticsearch, if we release 9.11 with
this change?
That's why I wasn't entirely sold on having HPPC as the dependency
from Lucene when Bruno mentioned it recently - the jar/module hell
will surface sooner than later... Maybe it'd be a better idea to
just copy what's needed to the core jar and expose those packages
to other Lucene modules (so that there is no explicit dependency
on HPPC at all)? Bruno copied a lot of those classes already
anyway - don't know how much of it is left to copy to drop the
dependency.
Dawid
[1] https://github.com/carrotsearch/hppc/issues/228
[2]
https://github.com/carrotsearch/hppc/commit/d569a8944091844c62349646f8eeaf35ebfb5ba6
-Chris.
[1] https://github.com/apache/lucene/pull/13392
[2] https://github.com/elastic/elasticsearch/pull/109006
[3] https://github.com/elastic/elasticsearch/pull/84168
[*] HPPC-186: A different strategy has been implemented for
collision avalanche avoidance. This results in removal of
Scatter* maps and sets and their unification with their Hash*
counterparts. This change should not affect any existing code
unless it relied on static, specific ordering of keys. A side
effect of this change is that key/value enumerators will
return a different ordering of their container's values on
each invocation. If your code relies on the order of values in
associative arrays, it must order them after they are
retrieved. (Bruno Roustant).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail:u...@thetaphi.de
