Re: [I] [JavaScript] Implement meta string encoding algorithm for JavaScript [incubator-fury]
chaokunyang commented on issue #1542: URL: https://github.com/apache/incubator-fury/issues/1542#issuecomment-2146440945 > Thank you @theweipeng . @chaokunyang , Please assign this issue to me. I would love to work on this Great, assigned to you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [JavaScript] Implement meta string encoding algorithm for JavaScript [incubator-fury]
Forchapeatl commented on issue #1542: URL: https://github.com/apache/incubator-fury/issues/1542#issuecomment-2145829664 Thank you @theweipeng . @chaokunyang , Please assign this issue to me. I would love to work on this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [JavaScript] Implement meta string encoding algorithm for JavaScript [incubator-fury]
theweipeng commented on issue #1542: URL: https://github.com/apache/incubator-fury/issues/1542#issuecomment-2144763402 > Hello @chaokunyang , I am Forcha Pearl from Cameroon . I am very good bit manipulation, endianess, encoding DataStructures and Algorithm. I also experienced with Java, python and javascript. I am interested in participting in OSPP 2024 under Apache furry this year. Please has this issue ben reserved for the OSPP contribution period. @Forchapeatl Hi, I am the mentor for the OSPP Fury Javascript project. Your participation is welcome. However, I noticed that the OSPP application period is about to end on June 4th at 18:00. You should submit your application on OSPP as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Add ClassInfo ClassBytes generation conditions. [incubator-fury]
LiangliangSui merged PR #1667: URL: https://github.com/apache/incubator-fury/pull/1667 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: perf(java): Add ClassInfo ClassBytes generation conditions. (#1667)
This is an automated email from the ASF dual-hosted git repository.
suiliangliang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 31d37f9c perf(java): Add ClassInfo ClassBytes generation conditions.
(#1667)
31d37f9c is described below
commit 31d37f9cf2021899f3dc2f1b06d5b45e79099251
Author: LiangliangSui <116876207+liangliang...@users.noreply.github.com>
AuthorDate: Mon Jun 3 16:02:50 2024 +0800
perf(java): Add ClassInfo ClassBytes generation conditions. (#1667)
## What does this PR do?
`ClassInfo#classNameBytes` and `ClassInfo#packageNameBytes` are only
used when `classInfo.classId == NO_CLASS_ID && !metaContextShareEnabled`
```java
public void writeClass(MemoryBuffer buffer, ClassInfo classInfo) {
if (classInfo.classId == NO_CLASS_ID) { // no class id provided.
// use classname
if (metaContextShareEnabled) {
buffer.writeByte(USE_CLASS_VALUE_FLAG);
// FIXME(chaokunyang) Register class but not register serializer
can't be used with
// meta share mode, because no class def are sent to peer.
writeClassWithMetaShare(buffer, classInfo);
} else {
// if it's null, it's a bug.
assert classInfo.packageNameBytes != null;
metaStringResolver.writeMetaStringBytesWithFlag(buffer,
classInfo.packageNameBytes);
assert classInfo.classNameBytes != null;
metaStringResolver.writeMetaStringBytes(buffer,
classInfo.classNameBytes);
}
} else {
// use classId
buffer.writeVarUint32(classInfo.classId << 1);
}
}
```
## Related issues
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
-
Signed-off-by: LiangliangSui
---
.../main/java/org/apache/fury/meta/Encoders.java | 6 ++-
.../java/org/apache/fury/resolver/ClassInfo.java | 15 ---
.../org/apache/fury/resolver/ClassResolver.java| 2 +-
.../apache/fury/resolver/MetaStringResolver.java | 3 +-
.../org/apache/fury/resolver/ClassInfoTest.java| 46 ++
5 files changed, 63 insertions(+), 9 deletions(-)
diff --git a/java/fury-core/src/main/java/org/apache/fury/meta/Encoders.java
b/java/fury-core/src/main/java/org/apache/fury/meta/Encoders.java
index 11434221..a1b1f4c2 100644
--- a/java/fury-core/src/main/java/org/apache/fury/meta/Encoders.java
+++ b/java/fury-core/src/main/java/org/apache/fury/meta/Encoders.java
@@ -36,8 +36,10 @@ import org.apache.fury.util.StringUtils;
/** A class used to encode package/class/field name. */
public class Encoders {
- public static final MetaStringEncoder PACKAGE_ENCODER = new
MetaStringEncoder('.', '_');
- public static final MetaStringDecoder PACKAGE_DECODER = new
MetaStringDecoder('.', '_');
+ public static final MetaStringEncoder GENERIC_ENCODER = new
MetaStringEncoder('.', '_');
+ public static final MetaStringDecoder GENERIC_DECODER = new
MetaStringDecoder('.', '_');
+ public static final MetaStringEncoder PACKAGE_ENCODER = GENERIC_ENCODER;
+ public static final MetaStringDecoder PACKAGE_DECODER = GENERIC_DECODER;
public static final MetaStringEncoder TYPE_NAME_ENCODER = new
MetaStringEncoder('$', '_');
public static final MetaStringDecoder TYPE_NAME_DECODER = new
MetaStringDecoder('$', '_');
public static final String ARRAY_PREFIX = "1";
diff --git
a/java/fury-core/src/main/java/org/apache/fury/resolver/ClassInfo.java
b/java/fury-core/src/main/java/org/apache/fury/resolver/ClassInfo.java
index b5aac593..562343c7 100644
--- a/java/fury-core/src/main/java/org/apache/fury/resolver/ClassInfo.java
+++ b/java/fury-core/src/main/java/org/apache/fury/resolver/ClassInfo.java
@@ -19,8 +19,8 @@
package org.apache.fury.resolver;
+import static org.apache.fury.meta.Encoders.GENERIC_ENCODER;
import static org.apache.fury.meta.Encoders.PACKAGE_ENCODER;
-import static org.apache.fury.meta.Encoders.TYPE_NAME_ENCODER;
import org.apache.fury.collection.Tuple2;
import org.apache.fury.config.Language;
@@ -82,18 +82,23 @@ public class ClassInfo {
if (cls != null && classResolver.getFury().getLanguage() != Language.JAVA)
{
this.fullClassNameBytes =
metaStringResolver.getOrCreateMetaStringBytes(
- PACKAGE_ENCODER.encode(cls.getName(), Encoding.UTF_8));
+ GENERIC_ENCODER.encode(cls.getName(), Encoding.UTF_8));
} else {
this.fullClassNameBytes = null;
}
+// When `classId == ClassResolver.REPLACE_STUB_ID` was established,
+// means only classes are serialized, not the instance. If we
+// serialize such class only, we need to wri
Re: [PR] perf(java): Add ClassInfo ClassBytes generation conditions. [incubator-fury]
LiangliangSui commented on code in PR #1667:
URL: https://github.com/apache/incubator-fury/pull/1667#discussion_r1623947168
##
java/fury-core/src/main/java/org/apache/fury/meta/Encoders.java:
##
@@ -38,6 +38,8 @@
public class Encoders {
public static final MetaStringEncoder PACKAGE_ENCODER = new
MetaStringEncoder('.', '_');
public static final MetaStringDecoder PACKAGE_DECODER = new
MetaStringDecoder('.', '_');
+ public static final MetaStringEncoder GENERIC_ENCODER = PACKAGE_ENCODER;
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Add ClassInfo ClassBytes generation conditions. [incubator-fury]
chaokunyang commented on code in PR #1667:
URL: https://github.com/apache/incubator-fury/pull/1667#discussion_r1623920605
##
java/fury-core/src/main/java/org/apache/fury/meta/Encoders.java:
##
@@ -38,6 +38,8 @@
public class Encoders {
public static final MetaStringEncoder PACKAGE_ENCODER = new
MetaStringEncoder('.', '_');
public static final MetaStringDecoder PACKAGE_DECODER = new
MetaStringDecoder('.', '_');
+ public static final MetaStringEncoder GENERIC_ENCODER = PACKAGE_ENCODER;
Review Comment:
How about move GENERIC_ENCODER front and make `PACKAGE_ENCODER =
GENERIC_ENCODER`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Add ClassInfo ClassBytes generation conditions. [incubator-fury]
chaokunyang commented on code in PR #1667:
URL: https://github.com/apache/incubator-fury/pull/1667#discussion_r1623918682
##
java/fury-core/src/main/java/org/apache/fury/resolver/ClassInfo.java:
##
@@ -86,14 +85,19 @@ public class ClassInfo {
} else {
this.fullClassNameBytes = null;
}
+// When `classId == ClassResolver.REPLACE_STUB_ID` was established,
+// means only classes are serialized, not the instance. If we
+// serialize such class only, we need to write classname bytes.
if (cls != null
-&& (classId == ClassResolver.NO_CLASS_ID || classId ==
ClassResolver.REPLACE_STUB_ID)) {
+&& ((classId == ClassResolver.NO_CLASS_ID
+&& !classResolver.getFury().getConfig().isMetaShareEnabled())
+|| classId == ClassResolver.REPLACE_STUB_ID)) {
// REPLACE_STUB_ID for write replace class in `ClassSerializer`.
Tuple2 tuple2 = Encoders.encodePkgAndClass(cls);
this.packageNameBytes =
-
metaStringResolver.getOrCreateMetaStringBytes(PACKAGE_ENCODER.encode(tuple2.f0));
+
metaStringResolver.getOrCreateMetaStringBytes(Encoders.encodePackage(tuple2.f0));
this.classNameBytes =
-
metaStringResolver.getOrCreateMetaStringBytes(TYPE_NAME_ENCODER.encode(tuple2.f1));
+
metaStringResolver.getOrCreateMetaStringBytes(Encoders.encodeTypeName(tuple2.f1));
Review Comment:
I misunderstood it. `Encoders.encodePkgName` use `UTF_8,
ALL_TO_LOWER_SPECIAL, LOWER_UPPER_DIGIT_SPECIAL` encoding. The
`ClassDefEncoder` use index in `UTF_8, ALL_TO_LOWER_SPECIAL,
LOWER_UPPER_DIGIT_SPECIAL` to store encoding. But `getOrCreateMetaStringBytes`
uses the encoding in returned `MetaString` instead, so it will use
`metaString.getEncoding().getValue()` as the index, and there won't be
inconsistency when decoding the meta string
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Add ClassInfo ClassBytes generation conditions. [incubator-fury]
LiangliangSui commented on code in PR #1667:
URL: https://github.com/apache/incubator-fury/pull/1667#discussion_r1623885654
##
java/fury-core/src/main/java/org/apache/fury/resolver/ClassResolver.java:
##
@@ -1701,7 +1701,7 @@ public Class xreadClass(MemoryBuffer buffer) {
Class cls = classNameBytes2Class.get(byteString);
if (cls == null) {
Preconditions.checkNotNull(byteString);
- String className = byteString.decode('.', '_');
+ String className = byteString.decode(Encoders.PACKAGE_DECODER);
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" is working again!
The GitHub Actions job "Fury CI" on incubator-fury.git has succeeded. Run started by GitHub user LiangliangSui (triggered by LiangliangSui). Head commit for run: ab8445a5091f8427ebdf77507a625943f3318022 / LiangliangSui fix error Signed-off-by: LiangliangSui Report URL: https://github.com/apache/incubator-fury/actions/runs/9345672104 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
