[GH] (incubator-fury): Workflow run "Fury CI" failed!
The GitHub Actions job "Fury CI" on incubator-fury.git has failed. Run started by GitHub user jasonmokk (triggered by jasonmokk). Head commit for run: 04f627e74ce90a77847d9582ad7fe6cea90d00d4 / Jason Mok fix: Add ASCII check before meta string encoding Signed-off-by: Jason Mok Report URL: https://github.com/apache/incubator-fury/actions/runs/9028244534 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(java): add set serializer for concurrent set [incubator-fury]
chaokunyang commented on code in PR #1616:
URL: https://github.com/apache/incubator-fury/pull/1616#discussion_r1596316514
##
java/fury-core/src/main/java/org/apache/fury/serializer/collection/CollectionSerializers.java:
##
@@ -411,6 +415,66 @@ public ConcurrentSkipListSet newCollection(MemoryBuffer
buffer) {
}
}
+ public static final class SetFromMapSerializer extends
CollectionSerializer> {
+
+private static final long MAP_FIELD_OFFSET;
+
+static {
+ try {
+Field mapField =
Class.forName("java.util.Collections$SetFromMap").getDeclaredField("m");
+MAP_FIELD_OFFSET = Platform.objectFieldOffset(mapField);
+ } catch (final Exception e) {
+throw new RuntimeException(e);
+ }
+}
+
+public SetFromMapSerializer(Fury fury, Class> type) {
+ super(fury, type, false);
+}
+
+@Override
+public Collection newCollection(MemoryBuffer buffer) {
+ int numElements = buffer.readVarUint32Small7();
+ setNumElements(numElements);
+ final ClassInfo mapClassInfo =
fury.getClassResolver().readClassInfo(buffer);
+ final MethodHandle methodHandle =
ReflectionUtils.getCtrHandle(mapClassInfo.getCls(), true);
Review Comment:
Could you add a test in graalvm_tests module?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
[PR] feat(go/java): Add ASCII check before meta string encoding [incubator-fury]
jasonmokk opened a new pull request, #1620: URL: https://github.com/apache/incubator-fury/pull/1620 ## What does this PR do? ## Related issues ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(java): add set serializer for concurrent set [incubator-fury]
MrChang0 commented on code in PR #1616:
URL: https://github.com/apache/incubator-fury/pull/1616#discussion_r1596305293
##
java/fury-core/src/main/java/org/apache/fury/serializer/collection/CollectionSerializers.java:
##
@@ -411,6 +415,66 @@ public ConcurrentSkipListSet newCollection(MemoryBuffer
buffer) {
}
}
+ public static final class SetFromMapSerializer extends
CollectionSerializer> {
+
+private static final long MAP_FIELD_OFFSET;
+
+static {
+ try {
+Field mapField =
Class.forName("java.util.Collections$SetFromMap").getDeclaredField("m");
+MAP_FIELD_OFFSET = Platform.objectFieldOffset(mapField);
+ } catch (final Exception e) {
+throw new RuntimeException(e);
+ }
+}
+
+public SetFromMapSerializer(Fury fury, Class> type) {
+ super(fury, type, false);
+}
+
+@Override
+public Collection newCollection(MemoryBuffer buffer) {
+ int numElements = buffer.readVarUint32Small7();
+ setNumElements(numElements);
+ final ClassInfo mapClassInfo =
fury.getClassResolver().readClassInfo(buffer);
+ final MethodHandle methodHandle =
ReflectionUtils.getCtrHandle(mapClassInfo.getCls(), true);
Review Comment:
ok, I change it. I test this version in graalvm and it works, I wandering
why.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [Java|Go] Check string is ascii before using meta string encoding [incubator-fury]
LiangliangSui commented on issue #1619: URL: https://github.com/apache/incubator-fury/issues/1619#issuecomment-2103704170 Great! thanks for the willingness to contribute to Fury. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [Java|Go] Check string is ascii before using meta string encoding [incubator-fury]
jasonmokk commented on issue #1619: URL: https://github.com/apache/incubator-fury/issues/1619#issuecomment-2103581626 Hi @chaokunyang I'd love to contribute to Apache Fury, can you please assign this issue to me? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[I] [Java|Go] Check string is ascii before using meta string encoding [incubator-fury]
chaokunyang opened a new issue, #1619: URL: https://github.com/apache/incubator-fury/issues/1619 ## Is your feature request related to a problem? Please describe. In #1514 and #1566 , we compress every char using 5/6 bytes. But we didn't check the encoding is ascii. In UTF-8, some byte may be in range of ascii char, but some not. We may take a utf-8 string as a meta string by accident? ## Describe the solution you'd like Check string is ascii before using meta string encoding ## Additional context -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] fix(java): fix fastutil Object2ObjectOpenHashMap serialization [incubator-fury]
chaokunyang merged PR #1618: URL: https://github.com/apache/incubator-fury/pull/1618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] In asyncCompilation to true, under the condition of it. The "it.unimi.dsi.fastutil.objects.Object2ObjectOpenHashMap" serialization failure [incubator-fury]
chaokunyang closed issue #1615: In asyncCompilation to true, under the condition of it. The "it.unimi.dsi.fastutil.objects.Object2ObjectOpenHashMap" serialization failure URL: https://github.com/apache/incubator-fury/issues/1615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: feat(java): add blocked stream utils (#1617)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 58bdf3e6 feat(java): add blocked stream utils (#1617)
58bdf3e6 is described below
commit 58bdf3e6c5a19c3204d0f3462db3e06126347dfc
Author: Shawn Yang
AuthorDate: Thu May 9 22:49:26 2024 +0800
feat(java): add blocked stream utils (#1617)
## What does this PR do?
Native stream is not feasible for every cases, this PR add blocked
stream utils to adapt to streaming API.
This is s serialization helper as the fallback of streaming
serialization/deserialization in FuryInputStream/FuryReadableChannel.
FuryInputStream/FuryReadableChannel will buffer and read more data,
which makes the original passed stream when constructing FuryInputStream
not usable. If this is not possible, use this BlockedStreamUtils instead
for streaming serialization and deserialization.
Note that this mode will disable streaming in essence. It's just a
helper for make the usage in streaming interface more easily. The
deserialization will read whole bytes before do the actual
deserialization, which don't have any streaming behaviour under the
hood.
## Related issues
#1451
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
---
.../src/main/java/org/apache/fury/Fury.java| 34 ++--
.../org/apache/fury/io/BlockedStreamUtils.java | 188 +
.../java/org/apache/fury/util/ExceptionUtils.java | 19 +++
.../src/test/java/org/apache/fury/CyclicTest.java | 31 ++--
.../org/apache/fury/io/BlockedStreamUtilsTest.java | 59 +++
5 files changed, 291 insertions(+), 40 deletions(-)
diff --git a/java/fury-core/src/main/java/org/apache/fury/Fury.java
b/java/fury-core/src/main/java/org/apache/fury/Fury.java
index b03527de..ee3ed4c8 100644
--- a/java/fury-core/src/main/java/org/apache/fury/Fury.java
+++ b/java/fury-core/src/main/java/org/apache/fury/Fury.java
@@ -24,26 +24,22 @@ import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteOrder;
import java.util.ArrayList;
-import java.util.Arrays;
import java.util.Iterator;
import java.util.List;
import java.util.function.Consumer;
import javax.annotation.concurrent.NotThreadSafe;
import org.apache.fury.builder.JITContext;
-import org.apache.fury.collection.ObjectArray;
import org.apache.fury.config.CompatibleMode;
import org.apache.fury.config.Config;
import org.apache.fury.config.FuryBuilder;
import org.apache.fury.config.Language;
import org.apache.fury.config.LongEncoding;
-import org.apache.fury.exception.DeserializationException;
import org.apache.fury.io.FuryInputStream;
import org.apache.fury.io.FuryReadableChannel;
import org.apache.fury.logging.Logger;
import org.apache.fury.logging.LoggerFactory;
import org.apache.fury.memory.MemoryBuffer;
import org.apache.fury.memory.MemoryUtils;
-import org.apache.fury.memory.Platform;
import org.apache.fury.resolver.ClassInfo;
import org.apache.fury.resolver.ClassInfoHolder;
import org.apache.fury.resolver.ClassResolver;
@@ -286,7 +282,7 @@ public final class Fury implements BaseFury {
throw e;
}
- private MemoryBuffer getBuffer() {
+ public MemoryBuffer getBuffer() {
MemoryBuffer buf = buffer;
if (buf == null) {
buf = buffer = MemoryBuffer.newHeapBuffer(64);
@@ -294,7 +290,7 @@ public final class Fury implements BaseFury {
return buf;
}
- private void resetBuffer() {
+ public void resetBuffer() {
MemoryBuffer buf = buffer;
if (buf != null && buf.size() > BUFFER_SIZE_LIMIT) {
buffer = MemoryBuffer.newHeapBuffer(BUFFER_SIZE_LIMIT);
@@ -759,7 +755,7 @@ public final class Fury implements BaseFury {
}
return obj;
} catch (Throwable t) {
- throw handleReadFailed(t);
+ throw ExceptionUtils.handleReadFailed(this, t);
} finally {
resetRead();
jitContext.unlock();
@@ -792,18 +788,6 @@ public final class Fury implements BaseFury {
return deserialize(buf, outOfBandBuffers);
}
- private RuntimeException handleReadFailed(Throwable t) {
-if (refResolver instanceof MapRefResolver) {
- ObjectArray readObjects = ((MapRefResolver)
refResolver).getReadObjects();
- // carry with read objects for better trouble shooting.
- List objects = Arrays.asList(readObjects.objects).subList(0,
readObjects.size);
- throw new DeserializationException(objects, t);
-} else {
- Platform.throwException(t);
- throw new IllegalStateException("unreachable");
-}
- }
-
private Object xdeserializeInternal(MemoryBuffer bu
(incubator-fury) branch main updated: fix(java): fix fastutil Object2ObjectOpenHashMap serialization (#1618)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 06a74a10 fix(java): fix fastutil Object2ObjectOpenHashMap
serialization (#1618)
06a74a10 is described below
commit 06a74a102c9191f40d4c78247b4e6f8c2e4e23cc
Author: Shawn Yang
AuthorDate: Thu May 9 22:49:17 2024 +0800
fix(java): fix fastutil Object2ObjectOpenHashMap serialization (#1618)
## What does this PR do?
fix fastutil Object2ObjectOpenHashMap serialization
## Related issues
Closes #1615
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
---
.../org/apache/fury/resolver/FieldResolver.java| 22
.../fury/serializer/CompatibleSerializer.java | 13 +++--
java/fury-testsuite/pom.xml| 6 ++
.../src/test/java/org/apache/fury/TestBase.java| 65 ++
.../fury/test/Object2ObjectOpenHashMapTest.java| 46 +++
5 files changed, 136 insertions(+), 16 deletions(-)
diff --git
a/java/fury-core/src/main/java/org/apache/fury/resolver/FieldResolver.java
b/java/fury-core/src/main/java/org/apache/fury/resolver/FieldResolver.java
index 047a2af1..17d6b2e8 100644
--- a/java/fury-core/src/main/java/org/apache/fury/resolver/FieldResolver.java
+++ b/java/fury-core/src/main/java/org/apache/fury/resolver/FieldResolver.java
@@ -50,8 +50,8 @@ import org.apache.fury.reflect.FieldAccessor;
import org.apache.fury.reflect.ReflectionUtils;
import org.apache.fury.reflect.TypeRef;
import org.apache.fury.serializer.PrimitiveSerializers;
-import org.apache.fury.serializer.collection.CollectionSerializer;
-import org.apache.fury.serializer.collection.MapSerializer;
+import org.apache.fury.serializer.collection.AbstractCollectionSerializer;
+import org.apache.fury.serializer.collection.AbstractMapSerializer;
import org.apache.fury.type.Descriptor;
import org.apache.fury.type.TypeUtils;
import org.apache.fury.util.MurmurHash3;
@@ -567,7 +567,8 @@ public class FieldResolver {
if (fieldType == FieldTypes.COLLECTION_ELEMENT_FINAL) {
ClassInfo elementClassInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
ClassInfo classInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
- CollectionSerializer collectionSerializer = (CollectionSerializer)
classInfo.getSerializer();
+ AbstractCollectionSerializer collectionSerializer =
+ (AbstractCollectionSerializer) classInfo.getSerializer();
try {
collectionSerializer.setElementSerializer(elementClassInfo.getSerializer());
o = collectionSerializer.read(buffer);
@@ -580,7 +581,7 @@ public class FieldResolver {
ClassInfo keyClassInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
ClassInfo valueClassInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
ClassInfo classInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
- MapSerializer mapSerializer = (MapSerializer) classInfo.getSerializer();
+ AbstractMapSerializer mapSerializer = (AbstractMapSerializer)
classInfo.getSerializer();
try {
mapSerializer.setKeySerializer(keyClassInfo.getSerializer());
mapSerializer.setValueSerializer(valueClassInfo.getSerializer());
@@ -594,7 +595,7 @@ public class FieldResolver {
} else if (fieldType == FieldTypes.MAP_KEY_FINAL) {
ClassInfo keyClassInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
ClassInfo classInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
- MapSerializer mapSerializer = (MapSerializer) classInfo.getSerializer();
+ AbstractMapSerializer mapSerializer = (AbstractMapSerializer)
classInfo.getSerializer();
try {
mapSerializer.setKeySerializer(keyClassInfo.getSerializer());
o = mapSerializer.read(buffer);
@@ -607,7 +608,7 @@ public class FieldResolver {
Preconditions.checkArgument(fieldType == FieldTypes.MAP_VALUE_FINAL);
ClassInfo valueClassInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
ClassInfo classInfo = classResolver.readClassInfo(buffer,
classInfoHolder);
- MapSerializer mapSerializer = (MapSerializer) classInfo.getSerializer();
+ AbstractMapSerializer mapSerializer = (AbstractMapSerializer)
classInfo.getSerializer();
try {
mapSerializer.setValueSerializer(valueClassInfo.getSerializer());
o = mapSerializer.read(buffer);
@@ -625,7 +626,8 @@ public class FieldResolver {
if (fieldType == FieldTypes.COLLECTION_ELEMENT_FINAL) {
ClassInfo elementClassInfo = classResolver.readClassInfo(buffer,
classInfoHolder)
Re: [PR] feat(java): add blocked stream utils [incubator-fury]
chaokunyang merged PR #1617: URL: https://github.com/apache/incubator-fury/pull/1617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" is working again!
The GitHub Actions job "Fury CI" on incubator-fury.git has succeeded. Run started by GitHub user chaokunyang (triggered by chaokunyang). Head commit for run: 364bb583501981c5899971687a885d98862bc17e / chaokunyang fix fastutil Object2ObjectOpenHashMap serialization Report URL: https://github.com/apache/incubator-fury/actions/runs/9018871779 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] fix(java): fix fastutil Object2ObjectOpenHashMap serialization [incubator-fury]
chaokunyang opened a new pull request, #1618: URL: https://github.com/apache/incubator-fury/pull/1618 ## What does this PR do? fix fastutil Object2ObjectOpenHashMap serialization ## Related issues Closes #1615 ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" failed!
The GitHub Actions job "Fury CI" on incubator-fury.git has failed. Run started by GitHub user chaokunyang (triggered by chaokunyang). Head commit for run: 1864db048c203a2daa8903e1818334d20dd4ac47 / chaokunyang fix checkstyle Report URL: https://github.com/apache/incubator-fury/actions/runs/9018671419 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(java): add set serializer for concurrent set [incubator-fury]
chaokunyang commented on code in PR #1616:
URL: https://github.com/apache/incubator-fury/pull/1616#discussion_r1595514648
##
java/fury-core/src/main/java/org/apache/fury/serializer/collection/CollectionSerializers.java:
##
@@ -411,6 +415,66 @@ public ConcurrentSkipListSet newCollection(MemoryBuffer
buffer) {
}
}
+ public static final class SetFromMapSerializer extends
CollectionSerializer> {
+
+private static final long MAP_FIELD_OFFSET;
+
+static {
+ try {
+Field mapField =
Class.forName("java.util.Collections$SetFromMap").getDeclaredField("m");
+MAP_FIELD_OFFSET = Platform.objectFieldOffset(mapField);
+ } catch (final Exception e) {
+throw new RuntimeException(e);
+ }
+}
+
+public SetFromMapSerializer(Fury fury, Class> type) {
+ super(fury, type, false);
+}
+
+@Override
+public Collection newCollection(MemoryBuffer buffer) {
+ int numElements = buffer.readVarUint32Small7();
+ setNumElements(numElements);
+ final ClassInfo mapClassInfo =
fury.getClassResolver().readClassInfo(buffer);
+ final MethodHandle methodHandle =
ReflectionUtils.getCtrHandle(mapClassInfo.getCls(), true);
Review Comment:
constructor will be invoked when creating Fury, which happens at graalvm
build time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" failed!
The GitHub Actions job "Fury CI" on incubator-fury.git has failed. Run started by GitHub user chaokunyang (triggered by chaokunyang). Head commit for run: f8be82ef4a36260cc1ec458a18676df5e21f3ebd / chaokunyang add blocked stream tests Report URL: https://github.com/apache/incubator-fury/actions/runs/9018580177 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(java): add set serializer for concurrent set [incubator-fury]
chaokunyang commented on code in PR #1616:
URL: https://github.com/apache/incubator-fury/pull/1616#discussion_r1595512601
##
java/fury-core/src/main/java/org/apache/fury/serializer/collection/CollectionSerializers.java:
##
@@ -411,6 +415,66 @@ public ConcurrentSkipListSet newCollection(MemoryBuffer
buffer) {
}
}
+ public static final class SetFromMapSerializer extends
CollectionSerializer> {
+
+private static final long MAP_FIELD_OFFSET;
+
+static {
+ try {
+Field mapField =
Class.forName("java.util.Collections$SetFromMap").getDeclaredField("m");
+MAP_FIELD_OFFSET = Platform.objectFieldOffset(mapField);
+ } catch (final Exception e) {
+throw new RuntimeException(e);
+ }
+}
+
+public SetFromMapSerializer(Fury fury, Class> type) {
+ super(fury, type, false);
+}
+
+@Override
+public Collection newCollection(MemoryBuffer buffer) {
+ int numElements = buffer.readVarUint32Small7();
+ setNumElements(numElements);
+ final ClassInfo mapClassInfo =
fury.getClassResolver().readClassInfo(buffer);
+ final MethodHandle methodHandle =
ReflectionUtils.getCtrHandle(mapClassInfo.getCls(), true);
Review Comment:
This won't work for graalvm. It's too late, we should move it to
constructor, so we can generate code at graalvm build time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
[PR] feat(java): add blocked stream utils [incubator-fury]
chaokunyang opened a new pull request, #1617: URL: https://github.com/apache/incubator-fury/pull/1617 ## What does this PR do? Native stream is not feasible for every cases, this PR add blocked stream utils to adapt to streaming API. ## Related issues #1451 ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] feat(java): add set serializer for concurrent set [incubator-fury]
MrChang0 opened a new pull request, #1616: URL: https://github.com/apache/incubator-fury/pull/1616 ## What does this PR do? we use `Sets.newConcurrentHashSet()` to create set, with lastest guava version it use `ConcurrentHashMapKeySetView` and old version it use `Collections.newSetFromMap(map)`. remove `CopyOnWriteArrayListSerializer` from native-image.properties. #1614 forget delete. ## Related issues ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] In asyncCompilation to true, under the condition of it. The "it.unimi.dsi.fastutil.objects.Object2ObjectOpenHashMap" serialization failure [incubator-fury]
foxmia commented on issue #1615: URL: https://github.com/apache/incubator-fury/issues/1615#issuecomment-2102385305 > Hi @foxmia could you provide code for Object2ObjectOpenHashMap? This is dependency ```xml it.unimi.dsi fastutil 8.5.6 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] In asyncCompilation to true, under the condition of it. The "it.unimi.dsi.fastutil.objects.Object2ObjectOpenHashMap" serialization failure [incubator-fury]
chaokunyang commented on issue #1615: URL: https://github.com/apache/incubator-fury/issues/1615#issuecomment-2102329470 Hi @foxmia could you provide code for Object2ObjectOpenHashMap? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[I] In asyncCompilation to true, under the condition of it. The "it.unimi.dsi.fastutil.objects.Object2ObjectOpenHashMap" serialization failure [incubator-fury]
foxmia opened a new issue, #1615:
URL: https://github.com/apache/incubator-fury/issues/1615
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-fury/issues) and found no similar
issues.
### Version
0.5.0、0.4.1
### Component(s)
Java
### Minimal reproduce step
Serialization fails only if `asyncCompilation` is `true`, there is no
problem if `asyncCompilation` is `false`
```java
@Data
public class Object2ObjectOpenHashMapTest {
Map ext = new Object2ObjectOpenHashMap<>();
}
```
```java
ThreadSafeFury fury = Fury.builder()
.requireClassRegistration(false)
.withCompatibleMode(CompatibleMode.COMPATIBLE)
.withAsyncCompilation(true)
.buildThreadSafeFury();
Object2ObjectOpenHashMapTest object2ObjectOpenHashMapTest = new
Object2ObjectOpenHashMapTest();
byte[] bytes = fury.serializeJavaObject(object2ObjectOpenHashMapTest);
```
The error message is as follows
```
Exception in thread "main" java.lang.ClassCastException: class
org.apache.fury.serializer.collection.MapSerializers$JDKCompatibleMapSerializer
cannot be cast to class org.apache.fury.serializer.collection.MapSerializer
(org.apache.fury.serializer.collection.MapSerializers$JDKCompatibleMapSerializer
and org.apache.fury.serializer.collection.MapSerializer are in unnamed module
of loader 'app')
at
org.apache.fury.serializer.CompatibleSerializer.writeMapKVFinal(CompatibleSerializer.java:258)
at
org.apache.fury.serializer.CompatibleSerializer.writeSeparateFieldValue(CompatibleSerializer.java:226)
at
org.apache.fury.serializer.CompatibleSerializer.readAndWriteFieldValue(CompatibleSerializer.java:155)
at
org.apache.fury.serializer.CompatibleSerializer.write(CompatibleSerializer.java:110)
at org.apache.fury.Fury.writeData(Fury.java:550)
at org.apache.fury.Fury.serializeJavaObject(Fury.java:1050)
at org.apache.fury.Fury.serializeJavaObject(Fury.java:1025)
at
org.apache.fury.ThreadLocalFury.serializeJavaObject(ThreadLocalFury.java:180)
```
### What did you expect to see?
expect it to work.
### What did you see instead?
class
org.apache.fury.serializer.collection.MapSerializers$JDKCompatibleMapSerializer
cannot be cast to class org.apache.fury.serializer.collection.MapSerializer
(org.apache.fury.serializer.collection.MapSerializers$JDKCompatibleMapSerializer
and org.apache.fury.serializer.collection.MapSerializer are in unnamed module
of loader 'app')
### Anything Else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
