[I] [Discuss][License] do wen need to move license content from NOTICE to LICENSE file [incubator-fury]
chaokunyang opened a new issue, #1498: URL: https://github.com/apache/incubator-fury/issues/1498 ## Is your feature request related to a problem? Please describe. Fury borrowed some code from spark/guava/v8/etc. Currently we add the license of those project in NOTICE file. Projects [flink](https://github.com/apache/opendal/blob/master/NOTICE)/[pytorch](https://github.com/pytorch/pytorch/blob/main/NOTICE) use this method. Other projects such as [spark](https://github.com/apache/spark/blob/master/LICENSE)/[arrow](https://github.com/apache/arrow/blob/main/LICENSE.txt)/etc all put license into `LICENSE` file. Both ways look good to me, @tisonkun @pjfanning do you have any suggestions? ## Additional context #1440 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: feat(java): optimize string serialization by concating coder and length (#1486)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 46db7e0a feat(java): optimize string serialization by concating coder
and length (#1486)
46db7e0a is described below
commit 46db7e0ae4948f2a2924fa7993d351a8fa205b6d
Author: Shawn Yang
AuthorDate: Fri Apr 12 14:21:58 2024 +0800
feat(java): optimize string serialization by concating coder and length
(#1486)
## What does this PR do?
This PR optimize string serialization by concating coder and length into
a long and serialize it together, it can save one byte for small string
which the length is less than 32
## Related issues
#1240
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
Before
```
Benchmark (bufferType) (objectType)
(references) Mode CntScore Error Units
UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT
false thrpt 50 2576580.395 ± 55698.851 ops/s
```
This PR:
```java
Benchmark (bufferType) (objectType)
(references) Mode CntScore Error Units
UserTypeDeserializeSuite.fury_deserialize array MEDIA_CONTENT
false thrpt 50 2591695.102 ± 48174.940 ops/s
```
---
.../integration_tests/JDKCompatibilityTest.java| 3 +-
.../java/org/apache/fury/memory/MemoryBuffer.java | 263 -
.../java/org/apache/fury/memory/MemoryUtils.java | 8 +-
.../org/apache/fury/serializer/Serializers.java| 11 +-
.../apache/fury/serializer/StringSerializer.java | 166 ++---
.../org/apache/fury/memory/MemoryBufferTest.java | 36 +++
.../fury/serializer/StringSerializerTest.java | 13 +-
7 files changed, 292 insertions(+), 208 deletions(-)
diff --git
a/integration_tests/jdk_compatibility_tests/src/test/java/org/apache/fury/integration_tests/JDKCompatibilityTest.java
b/integration_tests/jdk_compatibility_tests/src/test/java/org/apache/fury/integration_tests/JDKCompatibilityTest.java
index f191277f..53d2ee09 100644
---
a/integration_tests/jdk_compatibility_tests/src/test/java/org/apache/fury/integration_tests/JDKCompatibilityTest.java
+++
b/integration_tests/jdk_compatibility_tests/src/test/java/org/apache/fury/integration_tests/JDKCompatibilityTest.java
@@ -29,7 +29,6 @@ import org.apache.fury.Fury;
import org.apache.fury.config.CompatibleMode;
import org.apache.fury.config.FuryBuilder;
import org.apache.fury.config.Language;
-import org.apache.fury.test.bean.BeanA;
import org.apache.fury.util.Platform;
import org.testng.Assert;
import org.testng.annotations.Test;
@@ -42,7 +41,7 @@ public class JDKCompatibilityTest {
Object createObject() {
// test non latin1 string
-return Arrays.asList("Hello", "Hello，你好", BeanA.createBeanA(2));
+return Arrays.asList("Hello", "Hello，你好");
}
@Test
diff --git
a/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
b/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
index 68630df1..7f2be270 100644
--- a/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
+++ b/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
@@ -1252,7 +1252,7 @@ public final class MemoryBuffer {
* to avoid using two memory operations.
*/
public int unsafeWritePositiveVarInt(int v) {
-int varintBytes = unsafePutPositiveVarInt(writerIndex, v);
+int varintBytes = unsafePutVarUint36Small(writerIndex, v);
writerIndex += varintBytes;
return varintBytes;
}
@@ -1260,40 +1260,110 @@ public final class MemoryBuffer {
/**
* Caller must ensure there must be at least 8 bytes for writing, otherwise
the crash may occur.
*/
- public int unsafePutPositiveVarInt(int index, int value) {
-// The encoding algorithm are based on kryo UnsafeMemoryOutput.writeVarInt
-// varint are written using little endian byte order.
-// This version should have better performance since it remove an index
update.
-long varInt = (value & 0x7F);
+ public int unsafePutVarUint36Small(int index, long value) {
+long encoded = (value & 0x7F);
if (value >>> 7 == 0) {
- UNSAFE.putByte(heapMemory, address + index, (byte) varInt);
+ UNSAFE.putByte(heapMemory, address + index, (byte) value);
return 1;
}
// bit 8 `set` indicates have next data bytes.
// 0x3f80: 0b111 << 7
-varInt |= (((value & 0x3f80) << 1) | 0x80);
+encoded |= (((value & 0x3f80) << 1) | 0x80);
if (value >>> 14 == 0)
Re: [PR] feat(java): optimize string serialization by concating coder and length [incubator-fury]
chaokunyang merged PR #1486: URL: https://github.com/apache/incubator-fury/pull/1486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[I] [Java] Improve Fury Logger documentation. [incubator-fury]
LiangliangSui opened a new issue, #1497: URL: https://github.com/apache/incubator-fury/issues/1497 ## Is your feature request related to a problem? Please describe. Fury Logger is optional between `FuryLogger` and `Slf4jLogger`, or disable Fury log completely. ## Describe the solution you'd like We should improve the usage documentation in [java_object_graph_guide.md](https://github.com/apache/incubator-fury/blob/main/docs/guide/java_object_graph_guide.md). In addition, we should also improve the [LoggerFactory](https://github.com/apache/incubator-fury/blob/main/java/fury-core/src/main/java/org/apache/fury/logging/LoggerFactory.java) javadoc. ## Additional context Logger related PR - https://github.com/apache/incubator-fury/pull/1485 - https://github.com/apache/incubator-fury/pull/1492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: chore(java): FuryLogger add level control. (#1492)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 0b3e7d15 chore(java): FuryLogger add level control. (#1492)
0b3e7d15 is described below
commit 0b3e7d15773b6e503f3c84bc99dac998d2094842
Author: LiangliangSui <116876207+liangliang...@users.noreply.github.com>
AuthorDate: Fri Apr 12 00:20:27 2024 +0800
chore(java): FuryLogger add level control. (#1492)
## What does this PR do?
FuryLooger adds default log level control.
## Related issues
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
-
Signed-off-by: LiangliangSui
---
.../java/org/apache/fury/logging/FuryLogger.java | 70 +-
.../java/org/apache/fury/logging/LogLevel.java | 41 +
.../org/apache/fury/logging/LoggerFactory.java | 14 +++--
.../java/org/apache/fury/logging/Slf4jLogger.java | 13 ++--
4 files changed, 111 insertions(+), 27 deletions(-)
diff --git
a/java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java
b/java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java
index 78301805..d46f76e3 100644
--- a/java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java
+++ b/java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java
@@ -19,6 +19,10 @@
package org.apache.fury.logging;
+import static org.apache.fury.logging.LogLevel.ERROR_LEVEL;
+import static org.apache.fury.logging.LogLevel.INFO_LEVEL;
+import static org.apache.fury.logging.LogLevel.WARN_LEVEL;
+
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
@@ -26,85 +30,119 @@ public class FuryLogger implements Logger {
private static final DateTimeFormatter dateTimeFormatter =
DateTimeFormatter.ofPattern("-MM-dd hh:mm:ss");
private final String name;
+ private final int level;
public FuryLogger(Class targetClass) {
+this(targetClass, INFO_LEVEL);
+ }
+
+ public FuryLogger(Class targetClass, int level) {
this.name = targetClass.getSimpleName();
+this.level = level;
}
// The implementation should not forward to other method, otherwise the
fileNumber won't be right.
@Override
public void info(String msg) {
-log("INFO", msg, new Object[0], false);
+if (level >= INFO_LEVEL) {
+ log(INFO_LEVEL, msg, new Object[0], false);
+}
}
@Override
public void info(String msg, Object arg) {
-log("INFO", msg, new Object[] {arg}, false);
+if (level >= INFO_LEVEL) {
+ log(INFO_LEVEL, msg, new Object[] {arg}, false);
+}
}
@Override
public void info(String msg, Object arg1, Object arg2) {
-log("INFO", msg, new Object[] {arg1, arg2}, false);
+if (level >= INFO_LEVEL) {
+ log(INFO_LEVEL, msg, new Object[] {arg1, arg2}, false);
+}
}
@Override
public void info(String msg, Object... args) {
-log("INFO", msg, args, false);
+if (level >= INFO_LEVEL) {
+ log(INFO_LEVEL, msg, args, false);
+}
}
@Override
public void warn(String msg) {
-log("WARN", msg, new Object[0], false);
+if (level >= WARN_LEVEL) {
+ log(WARN_LEVEL, msg, new Object[0], false);
+}
}
@Override
public void warn(String msg, Object arg) {
-log("WARN", msg, new Object[] {arg}, true);
+if (level >= WARN_LEVEL) {
+ log(WARN_LEVEL, msg, new Object[] {arg}, true);
+}
}
@Override
public void warn(String msg, Object arg1, Object arg2) {
-log("WARN", msg, new Object[] {arg1, arg2}, true);
+if (level >= WARN_LEVEL) {
+ log(WARN_LEVEL, msg, new Object[] {arg1, arg2}, true);
+}
}
@Override
public void warn(String msg, Object... args) {
-log("WARN", msg, args, true);
+if (level >= WARN_LEVEL) {
+ log(WARN_LEVEL, msg, args, true);
+}
}
@Override
public void warn(String msg, Throwable t) {
-log("WARN", msg, new Object[] {t}, true);
+if (level >= WARN_LEVEL) {
+ log(WARN_LEVEL, msg, new Object[] {t}, true);
+}
}
@Override
public void error(String msg) {
-log("ERROR", msg, new Object[0], false);
+if (level >= ERROR_LEVEL) {
+ log(WARN_LEVEL, msg, new Object[0], false);
+}
}
@Override
public void error(String msg, Object arg) {
-log("ERROR", msg, new Object[] {arg}, true);
+if (level >= ERROR_LEVEL) {
+ log(ERROR_LEVEL, msg, new Object[] {arg}, true);
+}
}
@Override
public void error(String msg, Object arg1, Object arg2) {
-log("ERROR", msg, new Object[] {arg1, arg2}, true);
+if (level >= ERROR_LEVEL) {
+ log(
Re: [PR] chore(java): FuryLogger add level control. [incubator-fury]
chaokunyang merged PR #1492: URL: https://github.com/apache/incubator-fury/pull/1492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: perf(java): generate list fori loop instead of iterator loop for list serialization (#1493)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 10ee9470 perf(java): generate list fori loop instead of iterator loop
for list serialization (#1493)
10ee9470 is described below
commit 10ee94704b82e9e50b16a137d70101181ee15812
Author: Shawn Yang
AuthorDate: Fri Apr 12 00:19:21 2024 +0800
perf(java): generate list fori loop instead of iterator loop for list
serialization (#1493)
## What does this PR do?
generate list fori loop instead of iterator loop for list serialization
Before this PR:
```java
private void sameElementClassWrite(MemoryBuffer memoryBuffer2, int
value3, int value4, java.util.Collection collection, boolean value5) {
boolean isDeclType = (value4 & 4) != 4;
Serializer serializer1;
if (isDeclType) {
serializer1 = serializer0;
} else {
serializer1 = imageClassInfoHolder.getSerializer();
}
java.util.Iterator iter = collection.iterator();
int i = 0;
while (iter.hasNext()) {
org.apache.fury.benchmark.data.Image elemValue =
(org.apache.fury.benchmark.data.Image)iter.next();
if (value5) {
if ((elemValue == null)) {
memoryBuffer2.writeByte(((byte)-3));
} else {
memoryBuffer2.writeByte(((byte)0));
serializer1.write(memoryBuffer2, elemValue);
}
} else {
serializer1.write(memoryBuffer2, elemValue);
}
i++;
}
}
```
With this PR:
```java
private void sameElementClassWrite(MemoryBuffer memoryBuffer2,
java.util.List list2, int value3, int value4, boolean value5) {
boolean isDeclType = (value3 & 4) != 4;
Serializer serializer1;
if (isDeclType) {
serializer1 = serializer0;
} else {
serializer1 = imageClassInfoHolder.getSerializer();
}
for (int i = 0; i < value4; i+=1) {
Object object = list2.get(i);
org.apache.fury.benchmark.data.Image castedValue =
(org.apache.fury.benchmark.data.Image)object;
if (value5) {
if ((castedValue == null)) {
memoryBuffer2.writeByte(((byte)-3));
} else {
memoryBuffer2.writeByte(((byte)0));
serializer1.write(memoryBuffer2, castedValue);
}
} else {
serializer1.write(memoryBuffer2, castedValue);
}
}
}
```
## Related issues
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
```
Before:
Benchmark (bufferType) (objectType)
(references) Mode CntScoreError Units
UserTypeSerializeSuite.fury_serialize array MEDIA_CONTENT
false thrpt 30 3617413.349 ± 140849.598 ops/s
After:
Benchmark (bufferType) (objectType)
(references) Mode CntScore Error Units
UserTypeSerializeSuite.fury_serialize array MEDIA_CONTENT
false thrpt 50 3795239.909 ± 77404.887 ops/s
```
---
.../main/java/org/apache/fury/builder/BaseObjectCodecBuilder.java| 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git
a/java/fury-core/src/main/java/org/apache/fury/builder/BaseObjectCodecBuilder.java
b/java/fury-core/src/main/java/org/apache/fury/builder/BaseObjectCodecBuilder.java
index 0acb7ebd..0b91cf7b 100644
---
a/java/fury-core/src/main/java/org/apache/fury/builder/BaseObjectCodecBuilder.java
+++
b/java/fury-core/src/main/java/org/apache/fury/builder/BaseObjectCodecBuilder.java
@@ -34,6 +34,7 @@ import static
org.apache.fury.collection.Collections.ofHashSet;
import static
org.apache.fury.serializer.CodegenSerializer.LazyInitBeanSerializer;
import static org.apache.fury.type.TypeUtils.CLASS_TYPE;
import static org.apache.fury.type.TypeUtils.COLLECTION_TYPE;
+import static org.apache.fury.type.TypeUtils.LIST_TYPE;
import static org.apache.fury.type.TypeUtils.MAP_TYPE;
import static org.apache.fury.type.TypeUtils.OBJECT_TYPE;
import static org.apache.fury.type.TypeUtils.PRIMIT
Re: [PR] perf(java): generate list fori loop instead of iterator loop for list serialization [incubator-fury]
chaokunyang merged PR #1493: URL: https://github.com/apache/incubator-fury/pull/1493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: fix(java): fix benchmark register class and create object (#1495)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 1b0b93df fix(java): fix benchmark register class and create object
(#1495)
1b0b93df is described below
commit 1b0b93dfc4844cb7f476eee22871e98a3e84dba2
Author: Shawn Yang
AuthorDate: Fri Apr 12 00:19:08 2024 +0800
fix(java): fix benchmark register class and create object (#1495)
## What does this PR do?
fix benchmark register class and create object
## Related issues
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
---
.../org/apache/fury/benchmark/state/FuryState.java| 2 +-
.../org/apache/fury/benchmark/state/HessionState.java | 18 +-
.../org/apache/fury/benchmark/state/JDKState.java | 19 +--
.../org/apache/fury/benchmark/state/JsonbState.java | 18 +-
.../apache/fury/benchmark/state/ProtostuffState.java | 19 +--
5 files changed, 5 insertions(+), 71 deletions(-)
diff --git
a/java/benchmark/src/main/java/org/apache/fury/benchmark/state/FuryState.java
b/java/benchmark/src/main/java/org/apache/fury/benchmark/state/FuryState.java
index 7b0605d3..eaa01914 100644
---
a/java/benchmark/src/main/java/org/apache/fury/benchmark/state/FuryState.java
+++
b/java/benchmark/src/main/java/org/apache/fury/benchmark/state/FuryState.java
@@ -131,7 +131,7 @@ public class FuryState {
furyBuilder.withCompatibleMode(CompatibleMode.COMPATIBLE);
}
fury = furyBuilder.build();
- switch (ObjectType.MEDIA_CONTENT) {
+ switch (objectType) {
case SAMPLE:
case STRUCT:
case STRUCT2:
diff --git
a/java/benchmark/src/main/java/org/apache/fury/benchmark/state/HessionState.java
b/java/benchmark/src/main/java/org/apache/fury/benchmark/state/HessionState.java
index fa9322ed..81abe89a 100644
---
a/java/benchmark/src/main/java/org/apache/fury/benchmark/state/HessionState.java
+++
b/java/benchmark/src/main/java/org/apache/fury/benchmark/state/HessionState.java
@@ -29,8 +29,6 @@ import org.apache.fury.benchmark.LongStringSerializationSuite;
import org.apache.fury.benchmark.LongsSerializationSuite;
import org.apache.fury.benchmark.StringSerializationSuite;
import org.apache.fury.benchmark.data.Data;
-import org.apache.fury.benchmark.data.MediaContent;
-import org.apache.fury.benchmark.data.Sample;
import org.apache.fury.benchmark.data.Struct;
import org.apache.fury.logging.Logger;
import org.apache.fury.logging.LoggerFactory;
@@ -106,21 +104,7 @@ public class HessionState {
@Override
public void setup() {
super.setup();
- switch (objectType) {
-case SAMPLE:
- object = new Sample().populate(references);
- break;
-case MEDIA_CONTENT:
- object = new MediaContent().populate(references);
- break;
-case STRUCT:
- object = Struct.create(false);
- break;
-case STRUCT2:
- object = Struct.create(true);
- break;
- }
-
+ object = ObjectType.createObject(objectType, references);
bos.reset();
out.reset();
serialize(out, object);
diff --git
a/java/benchmark/src/main/java/org/apache/fury/benchmark/state/JDKState.java
b/java/benchmark/src/main/java/org/apache/fury/benchmark/state/JDKState.java
index 45c98f0d..553006da 100644
--- a/java/benchmark/src/main/java/org/apache/fury/benchmark/state/JDKState.java
+++ b/java/benchmark/src/main/java/org/apache/fury/benchmark/state/JDKState.java
@@ -29,9 +29,6 @@ import org.apache.fury.benchmark.LongStringSerializationSuite;
import org.apache.fury.benchmark.LongsSerializationSuite;
import org.apache.fury.benchmark.StringSerializationSuite;
import org.apache.fury.benchmark.data.Data;
-import org.apache.fury.benchmark.data.MediaContent;
-import org.apache.fury.benchmark.data.Sample;
-import org.apache.fury.benchmark.data.Struct;
import org.apache.fury.io.ClassLoaderObjectInputStream;
import org.apache.fury.util.Preconditions;
import org.openjdk.jmh.annotations.Level;
@@ -61,21 +58,7 @@ public class JDKState {
@Override
public void setup() {
super.setup();
- switch (objectType) {
-case SAMPLE:
- object = new Sample().populate(references);
- break;
-case MEDIA_CONTENT:
- object = new MediaContent().populate(references);
- break;
-case STRUCT:
- object = Struct.create(false);
- break;
-case STRUCT2:
- object = Struct.create(true);
- break;
- }
-
+ object = ObjectType.createObje
Re: [PR] fix(java): fix benchmark register class and create object [incubator-fury]
chaokunyang merged PR #1495: URL: https://github.com/apache/incubator-fury/pull/1495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] chore(java): FuryLogger add level control. [incubator-fury]
chaokunyang commented on code in PR #1492:
URL: https://github.com/apache/incubator-fury/pull/1492#discussion_r1561291038
##
java/fury-core/src/main/java/org/apache/fury/logging/Slf4jLogger.java:
##
@@ -25,10 +25,13 @@
public class Slf4jLogger implements Logger {
private static final String FQCN = Slf4jLogger.class.getName();
+ private final boolean isLocationAwareLogger;
+
private final org.slf4j.Logger logger;
public Slf4jLogger(Class cls) {
this.logger = org.slf4j.LoggerFactory.getLogger(cls);
+this.isLocationAwareLogger = logger instanceof LocationAwareLogger;
Review Comment:
Good optimization!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
[PR] chore(java): reduce fury caller stack [incubator-fury]
chaokunyang opened a new pull request, #1496: URL: https://github.com/apache/incubator-fury/pull/1496 ## What does this PR do? reduce fury caller stack ## Related issues ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] chore(java): FuryLogger add level control. [incubator-fury]
chaokunyang commented on code in PR #1492:
URL: https://github.com/apache/incubator-fury/pull/1492#discussion_r1561262892
##
java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java:
##
@@ -23,88 +23,139 @@
import java.time.format.DateTimeFormatter;
public class FuryLogger implements Logger {
+
+ private static final int ERROR_LEVEL = 0;
+
+ private static final int WARN_LEVEL = 1;
+
+ private static final int INFO_LEVEL = 2;
+
+ // Default is INFO level.
+ private static final int LEVEL = INFO_LEVEL;
Review Comment:
We can add a static method in LoggerFactory named `setLoggerLevel`, when
creating logger, we can pass that level to FuryLogger constructor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
[PR] fix(java): make slf4j optional [incubator-fury]
chaokunyang opened a new pull request, #1494: URL: https://github.com/apache/incubator-fury/pull/1494 ## What does this PR do? make slf4j optional ## Related issues #1485 ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] fix(java): fix benchmark register class and create object [incubator-fury]
chaokunyang opened a new pull request, #1495: URL: https://github.com/apache/incubator-fury/pull/1495 ## What does this PR do? fix benchmark register class and create object ## Related issues ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] chore(java): FuryLogger add level control. [incubator-fury]
LiangliangSui commented on code in PR #1492:
URL: https://github.com/apache/incubator-fury/pull/1492#discussion_r1561236822
##
java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java:
##
@@ -23,88 +23,139 @@
import java.time.format.DateTimeFormatter;
public class FuryLogger implements Logger {
+
+ private static final int ERROR_LEVEL = 0;
+
+ private static final int WARN_LEVEL = 1;
+
+ private static final int INFO_LEVEL = 2;
+
+ // Default is INFO level.
+ private static final int LEVEL = INFO_LEVEL;
Review Comment:
Pass in `Level` when calling `LoggerFactory#getLogger`, and then pass in
`Level` through the `FuryLogger` constructor?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): generate list fori loop instead of iterator loop for list serialization [incubator-fury]
chaokunyang commented on PR #1493: URL: https://github.com/apache/incubator-fury/pull/1493#issuecomment-2049964650 @theweipeng @LiangliangSui please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] perf(java): generate list fori loop instead of iterator loop for list serialization [incubator-fury]
chaokunyang opened a new pull request, #1493:
URL: https://github.com/apache/incubator-fury/pull/1493
## What does this PR do?
generate list fori loop instead of iterator loop for list serialization
Before this PR:
```java
private void sameElementClassWrite(MemoryBuffer memoryBuffer2, int value3,
int value4, java.util.Collection collection, boolean value5) {
boolean isDeclType = (value4 & 4) != 4;
Serializer serializer1;
if (isDeclType) {
serializer1 = serializer0;
} else {
serializer1 = imageClassInfoHolder.getSerializer();
}
java.util.Iterator iter = collection.iterator();
int i = 0;
while (iter.hasNext()) {
org.apache.fury.benchmark.data.Image elemValue =
(org.apache.fury.benchmark.data.Image)iter.next();
if (value5) {
if ((elemValue == null)) {
memoryBuffer2.writeByte(((byte)-3));
} else {
memoryBuffer2.writeByte(((byte)0));
serializer1.write(memoryBuffer2, elemValue);
}
} else {
serializer1.write(memoryBuffer2, elemValue);
}
i++;
}
}
```
With this PR:
```java
private void sameElementClassWrite(MemoryBuffer memoryBuffer2,
java.util.List list2, int value3, int value4, boolean value5) {
boolean isDeclType = (value3 & 4) != 4;
Serializer serializer1;
if (isDeclType) {
serializer1 = serializer0;
} else {
serializer1 = imageClassInfoHolder.getSerializer();
}
for (int i = 0; i < value4; i+=1) {
Object object = list2.get(i);
org.apache.fury.benchmark.data.Image castedValue =
(org.apache.fury.benchmark.data.Image)object;
if (value5) {
if ((castedValue == null)) {
memoryBuffer2.writeByte(((byte)-3));
} else {
memoryBuffer2.writeByte(((byte)0));
serializer1.write(memoryBuffer2, castedValue);
}
} else {
serializer1.write(memoryBuffer2, castedValue);
}
}
}
```
## Related issues
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [Java] Map serialization bug [incubator-fury]
Munoon closed issue #1481: [Java] Map serialization bug URL: https://github.com/apache/incubator-fury/issues/1481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [Java] Map serialization bug [incubator-fury]
Munoon commented on issue #1481: URL: https://github.com/apache/incubator-fury/issues/1481#issuecomment-2049856877 Looks like fixed in #1485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: feat(JavaScript): implement xlang protocol (#1487)
This is an automated email from the ASF dual-hosted git repository.
wangweipeng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 417f064b feat(JavaScript): implement xlang protocol (#1487)
417f064b is described below
commit 417f064b705a55fd9fe0270b464863d54199c327
Author: weipeng
AuthorDate: Thu Apr 11 22:32:03 2024 +0800
feat(JavaScript): implement xlang protocol (#1487)
## What does this PR do?
1. Remove unsigned types which are removed in xlang protocol.
2. Rename `long int double` to `int64 int32 float64` to ensure
consistency with other languages
3. Implements the float16 writer and float16 reader.
## Related issues
## Does this PR introduce any user-facing change?
The JavaScript implementation has not been released yet, so there are no
user-facing changes.
## Benchmark
There are no fluctuations in the benchmark.
| (index) | serialize | deserialize |
|--|---|-|
| fury | 116 | 118 |
| protobuf | 25| 33 |
| json | 20| 23 |
-
Co-authored-by: wangweipeng
---
javascript/packages/fury/lib/classResolver.ts | 120 +++
javascript/packages/fury/lib/description.ts| 132 ++---
javascript/packages/fury/lib/gen/any.ts| 3 +-
javascript/packages/fury/lib/gen/builder.ts| 28 +++--
javascript/packages/fury/lib/gen/collection.ts | 4 +-
javascript/packages/fury/lib/gen/datetime.ts | 4 +-
javascript/packages/fury/lib/gen/index.ts | 7 +-
javascript/packages/fury/lib/gen/number.ts | 57 +
javascript/packages/fury/lib/gen/object.ts | 9 +-
javascript/packages/fury/lib/gen/oneof.ts | 115 --
javascript/packages/fury/lib/gen/serializer.ts | 5 +-
javascript/packages/fury/lib/gen/set.ts| 2 +-
javascript/packages/fury/lib/gen/typedArray.ts | 17 +--
javascript/packages/fury/lib/meta.ts | 64 +-
javascript/packages/fury/lib/reader/index.ts | 41 ++-
javascript/packages/fury/lib/referenceResolver.ts | 3 +-
javascript/packages/fury/lib/type.ts | 66 ++-
.../fury/lib/{writer.ts => writer/index.ts}| 24 ++--
javascript/packages/fury/lib/writer/number.ts | 47
javascript/test/array.test.ts | 73 ++--
javascript/test/binary.test.ts | 2 +-
javascript/test/datetime.test.ts | 2 +-
javascript/test/fixtures/tuple.ts | 2 +-
javascript/test/fury.test.ts | 2 +-
javascript/test/io.test.ts | 18 +--
javascript/test/number.test.ts | 86 ++
javascript/test/object.test.ts | 24 ++--
javascript/test/oneof.test.ts | 90 --
javascript/test/protocol/struct.test.ts| 4 +-
javascript/test/referenceResolve.test.ts | 5 +-
30 files changed, 447 insertions(+), 609 deletions(-)
diff --git a/javascript/packages/fury/lib/classResolver.ts
b/javascript/packages/fury/lib/classResolver.ts
index 62cc1b0b..2156640b 100644
--- a/javascript/packages/fury/lib/classResolver.ts
+++ b/javascript/packages/fury/lib/classResolver.ts
@@ -84,7 +84,7 @@ export default class SerializerResolver {
private writeStringIndex: number[] = [];
private registerSerializer(fury: Fury, description: TypeDescription) {
-return fury.classResolver.registerSerializerById(description.type,
generateSerializer(fury, description));
+return
fury.classResolver.registerSerializerById(SerializerResolver.getTypeIdByInternalSerializerType(description.type),
generateSerializer(fury, description));
}
private initInternalSerializer(fury: Fury) {
@@ -92,27 +92,27 @@ export default class SerializerResolver {
this.registerSerializer(fury, Type.array(Type.any()));
this.registerSerializer(fury, Type.map(Type.any(), Type.any()));
this.registerSerializer(fury, Type.bool());
-this.registerSerializer(fury, Type.uint8());
this.registerSerializer(fury, Type.int8());
-this.registerSerializer(fury, Type.uint16());
this.registerSerializer(fury, Type.int16());
-this.registerSerializer(fury, Type.uint32());
this.registerSerializer(fury, Type.int32());
-this.registerSerializer(fury, Type.uint64());
+this.registerSerializer(fury, Type.varInt32());
this.registerSerializer(fury, Type.int64());
-this.registerSerializer(fury, Type.float());
-this.registerSerializer(fury, Type.double());
+this.registerSerializer(fury, Type.sliInt64());
+this.registerSerializer(fury, Type.float16())
Re: [PR] feat(JavaScript): implement xlang protocol [incubator-fury]
theweipeng merged PR #1487: URL: https://github.com/apache/incubator-fury/pull/1487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] chore(java): FuryLogger add level control. [incubator-fury]
chaokunyang commented on code in PR #1492:
URL: https://github.com/apache/incubator-fury/pull/1492#discussion_r156480
##
java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java:
##
@@ -23,88 +23,139 @@
import java.time.format.DateTimeFormatter;
public class FuryLogger implements Logger {
+
+ private static final int ERROR_LEVEL = 0;
+
+ private static final int WARN_LEVEL = 1;
+
+ private static final int INFO_LEVEL = 2;
+
+ // Default is INFO level.
+ private static final int LEVEL = INFO_LEVEL;
Review Comment:
how aboubt make it as a final field of `FuryLogger`, and pass it from
`LoggerFactory` to allow users to custimize level
##
java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java:
##
@@ -23,88 +23,139 @@
import java.time.format.DateTimeFormatter;
public class FuryLogger implements Logger {
+
+ private static final int ERROR_LEVEL = 0;
+
+ private static final int WARN_LEVEL = 1;
+
+ private static final int INFO_LEVEL = 2;
+
+ // Default is INFO level.
+ private static final int LEVEL = INFO_LEVEL;
Review Comment:
how about make it as a final field of `FuryLogger`, and pass it from
`LoggerFactory` to allow users to custimize level
##
java/fury-core/src/main/java/org/apache/fury/logging/FuryLogger.java:
##
@@ -23,88 +23,139 @@
import java.time.format.DateTimeFormatter;
public class FuryLogger implements Logger {
+
+ private static final int ERROR_LEVEL = 0;
+
+ private static final int WARN_LEVEL = 1;
+
+ private static final int INFO_LEVEL = 2;
+
+ // Default is INFO level.
+ private static final int LEVEL = INFO_LEVEL;
Review Comment:
how about making it as a final field of `FuryLogger`, and pass it from
`LoggerFactory` to allow users to custimize level
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
[PR] chore(java): FuryLogger adds level control. [incubator-fury]
LiangliangSui opened a new pull request, #1492: URL: https://github.com/apache/incubator-fury/pull/1492 ## What does this PR do? FuryLooger adds default log level control. ## Related issues ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: chore(java): add jmh benchmark params to doc (#1488)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 730c7131 chore(java): add jmh benchmark params to doc (#1488)
730c7131 is described below
commit 730c7131821ebbf99d23bcf1b6325ff966b1ef12
Author: Shawn Yang
AuthorDate: Thu Apr 11 22:07:42 2024 +0800
chore(java): add jmh benchmark params to doc (#1488)
## What does this PR do?
This PR add jmh params to doc, so one can select which benchmark to run
easily for performance optimization
## Related issues
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
---
java/benchmark/README.md | 2 +-
.../java/org/apache/fury/benchmark/UserTypeDeserializeSuite.java | 3 ++-
.../main/java/org/apache/fury/benchmark/UserTypeSerializeSuite.java | 5 +++--
3 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/java/benchmark/README.md b/java/benchmark/README.md
index 3c3b5c9a..4d152a31 100644
--- a/java/benchmark/README.md
+++ b/java/benchmark/README.md
@@ -29,7 +29,7 @@ cd ../java && mvn install -DskipTests -Dcheckstyle.skip
-Dlicense.skip -Dmaven.j
mvn package -Pjmh
# run benchmark
nohup java -jar target/benchmarks.jar -f 5 -wi 3 -i 5 -t 1 -w 3s -r 5s -rf csv
>bench.log 2>&1 &
-java -jar target/benchmarks.jar "org.apache.fury.*\.deserialize$" -f 1 -wi 1
-i 3 -t 1 -w 2s -r 2s -rf csv
+java -jar target/benchmarks.jar "org.apache.fury.*\.deserialize$" -f 1 -wi 1
-i 3 -t 1 -w 2s -r 2s -rf csv -p objectType=MEDIA_CONTENT -p bufferType=array
-p references=false
```
Generate Protobuf/Flatbuffers code manually:
diff --git
a/java/benchmark/src/main/java/org/apache/fury/benchmark/UserTypeDeserializeSuite.java
b/java/benchmark/src/main/java/org/apache/fury/benchmark/UserTypeDeserializeSuite.java
index 1fe199fd..e9f31411 100644
---
a/java/benchmark/src/main/java/org/apache/fury/benchmark/UserTypeDeserializeSuite.java
+++
b/java/benchmark/src/main/java/org/apache/fury/benchmark/UserTypeDeserializeSuite.java
@@ -136,7 +136,8 @@ public class UserTypeDeserializeSuite {
public static void main(String[] args) throws IOException {
if (args.length == 0) {
String commandLine =
- "org.apache.fury.*UserTypeDeserializeSuite.kryo* -f 0 -wi 3 -i 3 -t
1 -w 2s -r 2s -rf csv";
+ "org.apache.fury.*UserTypeDeserializeSuite.fury* -f 1 -wi 5 -i 10 -t
1 -w 2s -r 2s -rf csv "
+ + "-p objectType=MEDIA_CONTENT -p bufferType=array -p
references=false";
System.out.println(commandLine);
args = commandLine.split(" ");
}
diff --git
a/java/benchmark/src/main/java/org/apache/fury/benchmark/UserTypeSerializeSuite.java
b/java/benchmark/src/main/java/org/apache/fury/benchmark/UserTypeSerializeSuite.java
index 58788ac2..462403fe 100644
---
a/java/benchmark/src/main/java/org/apache/fury/benchmark/UserTypeSerializeSuite.java
+++
b/java/benchmark/src/main/java/org/apache/fury/benchmark/UserTypeSerializeSuite.java
@@ -117,7 +117,7 @@ public class UserTypeSerializeSuite {
return state.bos;
}
- @Benchmark
+ // @Benchmark
public byte[] jsonb_serialize(JsonbState.JsonbUserTypeState state, Blackhole
bh) {
return JsonbState.serialize(bh, state, state.object);
}
@@ -144,7 +144,8 @@ public class UserTypeSerializeSuite {
public static void main(String[] args) throws IOException {
if (args.length == 0) {
String commandLine =
- "org.apache.fury.*UserTypeSerializeSuite.jsonb_serialize -f 3 -wi 3
-i 3 -t 1 -w 2s -r 2s -rf csv";
+ "org.apache.fury.*UserTypeSerializeSuite.fury -f 3 -wi 5 -i 10 -t 1
-w 2s -r 2s -rf csv "
+ + "-p objectType=MEDIA_CONTENT -p bufferType=array -p
references=false";
System.out.println(commandLine);
args = commandLine.split(" ");
}
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] chore(java): add jmh benchmark params to doc [incubator-fury]
chaokunyang merged PR #1488: URL: https://github.com/apache/incubator-fury/pull/1488 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(java): optimize string serialization by concating coder and length [incubator-fury]
chaokunyang commented on PR #1486: URL: https://github.com/apache/incubator-fury/pull/1486#issuecomment-2049777488 @theweipeng @LiangliangSui Could you please help review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: perf(java): Reduce unsafeWritePositiveVarLong bytecode size. (#1491)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 20344b98 perf(java): Reduce unsafeWritePositiveVarLong bytecode size.
(#1491)
20344b98 is described below
commit 20344b981def63224c803c709fe00a27ff26d1a9
Author: PAN <46820719+pandale...@users.noreply.github.com>
AuthorDate: Thu Apr 11 21:23:47 2024 +0800
perf(java): Reduce unsafeWritePositiveVarLong bytecode size. (#1491)
## Reduce unsafeWritePositiveVarLong bytecode size
Reduce bytecode size from 431 to 413.
## Related issues
Through the calculation in advance to reduce memory
#1465
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
pre
https://github.com/apache/incubator-fury/assets/46820719/6627e8ef-b681-46cb-9edd-66839d875459";>
after
https://github.com/apache/incubator-fury/assets/46820719/ec97c0ed-3939-4149-981b-32957fd38f93";>
---
.../java/org/apache/fury/memory/MemoryBuffer.java | 40 +-
1 file changed, 16 insertions(+), 24 deletions(-)
diff --git
a/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
b/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
index 6dda1e78..68630df1 100644
--- a/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
+++ b/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
@@ -1697,64 +1697,56 @@ public final class MemoryBuffer {
final int writerIndex = this.writerIndex;
int varInt;
varInt = (int) (value & 0x7F);
-value >>>= 7;
-if (value == 0) {
+if (value >>> 7 == 0) {
UNSAFE.putByte(heapMemory, address + writerIndex, (byte) varInt);
this.writerIndex = writerIndex + 1;
return 1;
}
-varInt |= (int) ((value & 0x7F) << 8) | 0x80;
-value >>>= 7;
-if (value == 0) {
+varInt |= (((value & 0x3f80) << 1) | 0x80);
+if (value >>> 14 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 2;
return 2;
}
-varInt |= (int) ((value & 0x7F) << 16) | 0x8000;
-value >>>= 7;
-if (value == 0) {
+varInt |= (((value & 0x1fc000) << 2) | 0x8000);
+if (value >>> 21 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 3;
return 3;
}
-varInt |= (int) ((value & 0x7F) << 24) | 0x80;
-value >>>= 7;
-if (value == 0) {
+varInt |= ((value & 0xfe0) << 3) | 0x80;
+if (value >>> 28 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 4;
return 4;
}
long varLong = (varInt & 0xL);
-varLong |= ((value & 0x7F) << 32) | 0x8000L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x7f000L) << 4) | 0x8000L;
+if (value >>> 35 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 5;
return 5;
}
-varLong |= ((value & 0x7F) << 40) | 0x80L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x3f8L) << 5) | 0x80L;
+if (value >>> 42 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 6;
return 6;
}
-varLong |= ((value & 0x7F) << 48) | 0x8000L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x1fc00L) << 6) | 0x8000L;
+if (value >>> 49 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 7;
return 7;
}
-varLong |= ((value & 0x7F) << 56) | 0x80L;
-value >>>= 7;
+varLong |= ((value & 0xfeL) << 7) | 0x80L;
+value >>>= 56;
if (value == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 8;
return 8;
}
-varLong |= 0x8000L;
-unsafePutLong(writerIndex, varLong);
+unsafePutLong(writerIndex, varLong | 0x8000L);
UNSAFE.putByte(heapMemory, address + writerIndex + 8, (byte) (value &
0xFF));
this.writerIndex = writerIndex + 9;
return 9;
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Reduce unsafeWritePositiveVarLong bytecode size. [incubator-fury]
chaokunyang merged PR #1491: URL: https://github.com/apache/incubator-fury/pull/1491 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Reduce unsafeWritePositiveVarLong bytecode size. [incubator-fury]
pandalee99 commented on code in PR #1491:
URL: https://github.com/apache/incubator-fury/pull/1491#discussion_r1560963486
##
java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java:
##
@@ -1697,64 +1697,56 @@ public int unsafeWritePositiveVarLong(long value) {
final int writerIndex = this.writerIndex;
int varInt;
varInt = (int) (value & 0x7F);
-value >>>= 7;
-if (value == 0) {
+if (value >>> 7 == 0) {
UNSAFE.putByte(heapMemory, address + writerIndex, (byte) varInt);
this.writerIndex = writerIndex + 1;
return 1;
}
-varInt |= (int) ((value & 0x7F) << 8) | 0x80;
-value >>>= 7;
-if (value == 0) {
+varInt |= (((value & 0x3f80) << 1) | 0x80);
+if (value >>> 14 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 2;
return 2;
}
-varInt |= (int) ((value & 0x7F) << 16) | 0x8000;
-value >>>= 7;
-if (value == 0) {
+varInt |= (((value & 0x1fc000) << 2) | 0x8000);
+if (value >>> 21 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 3;
return 3;
}
-varInt |= (int) ((value & 0x7F) << 24) | 0x80;
-value >>>= 7;
-if (value == 0) {
+varInt |= ((value & 0xfe0) << 3) | 0x80;
+if (value >>> 28 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 4;
return 4;
}
long varLong = (varInt & 0xL);
-varLong |= ((value & 0x7F) << 32) | 0x8000L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x7f000L) << 4) | 0x8000L;
+if (value >>> 35 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 5;
return 5;
}
-varLong |= ((value & 0x7F) << 40) | 0x80L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x3f8L) << 5) | 0x80L;
+if (value >>> 42 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 6;
return 6;
}
-varLong |= ((value & 0x7F) << 48) | 0x8000L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x1fc00L) << 6) | 0x8000L;
+if (value >>> 49 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 7;
return 7;
}
-varLong |= ((value & 0x7F) << 56) | 0x80L;
-value >>>= 7;
+varLong |= ((value & 0xfeL) << 7) | 0x80L;
+value >>>= 56;
Review Comment:
sure，I already tested it.
I originally thought
```java
if (value >>> 56 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 8;
return 8;
}
unsafePutLong(writerIndex, varLong | 0x8000L);
UNSAFE.putByte(heapMemory, address + writerIndex + 8, (byte) (value >>>
56 & 0xFF));
this.writerIndex = writerIndex + 9;
```
but, this bytecode size will be from 413 to 414.
https://github.com/apache/incubator-fury/assets/46820719/d157d496-6511-4c1e-8d7c-c1b5ee6cf3e5";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Reduce unsafeWritePositiveVarLong bytecode size. [incubator-fury]
pandalee99 commented on code in PR #1491:
URL: https://github.com/apache/incubator-fury/pull/1491#discussion_r1560963486
##
java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java:
##
@@ -1697,64 +1697,56 @@ public int unsafeWritePositiveVarLong(long value) {
final int writerIndex = this.writerIndex;
int varInt;
varInt = (int) (value & 0x7F);
-value >>>= 7;
-if (value == 0) {
+if (value >>> 7 == 0) {
UNSAFE.putByte(heapMemory, address + writerIndex, (byte) varInt);
this.writerIndex = writerIndex + 1;
return 1;
}
-varInt |= (int) ((value & 0x7F) << 8) | 0x80;
-value >>>= 7;
-if (value == 0) {
+varInt |= (((value & 0x3f80) << 1) | 0x80);
+if (value >>> 14 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 2;
return 2;
}
-varInt |= (int) ((value & 0x7F) << 16) | 0x8000;
-value >>>= 7;
-if (value == 0) {
+varInt |= (((value & 0x1fc000) << 2) | 0x8000);
+if (value >>> 21 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 3;
return 3;
}
-varInt |= (int) ((value & 0x7F) << 24) | 0x80;
-value >>>= 7;
-if (value == 0) {
+varInt |= ((value & 0xfe0) << 3) | 0x80;
+if (value >>> 28 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 4;
return 4;
}
long varLong = (varInt & 0xL);
-varLong |= ((value & 0x7F) << 32) | 0x8000L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x7f000L) << 4) | 0x8000L;
+if (value >>> 35 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 5;
return 5;
}
-varLong |= ((value & 0x7F) << 40) | 0x80L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x3f8L) << 5) | 0x80L;
+if (value >>> 42 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 6;
return 6;
}
-varLong |= ((value & 0x7F) << 48) | 0x8000L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x1fc00L) << 6) | 0x8000L;
+if (value >>> 49 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 7;
return 7;
}
-varLong |= ((value & 0x7F) << 56) | 0x80L;
-value >>>= 7;
+varLong |= ((value & 0xfeL) << 7) | 0x80L;
+value >>>= 56;
Review Comment:
sure，I already tested it.
I originally thought
```java
if (value >>> 56 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 8;
return 8;
}
unsafePutLong(writerIndex, varLong | 0x8000L);
UNSAFE.putByte(heapMemory, address + writerIndex + 8, (byte) (value >>>
56 & 0xFF));
this.writerIndex = writerIndex + 9;
```
but, this bytecode size will be from 413 to 414.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Reduce unsafeWritePositiveVarLong bytecode size. [incubator-fury]
LiangliangSui commented on code in PR #1491:
URL: https://github.com/apache/incubator-fury/pull/1491#discussion_r1560930643
##
java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java:
##
@@ -1697,64 +1697,56 @@ public int unsafeWritePositiveVarLong(long value) {
final int writerIndex = this.writerIndex;
int varInt;
varInt = (int) (value & 0x7F);
-value >>>= 7;
-if (value == 0) {
+if (value >>> 7 == 0) {
UNSAFE.putByte(heapMemory, address + writerIndex, (byte) varInt);
this.writerIndex = writerIndex + 1;
return 1;
}
-varInt |= (int) ((value & 0x7F) << 8) | 0x80;
-value >>>= 7;
-if (value == 0) {
+varInt |= (((value & 0x3f80) << 1) | 0x80);
+if (value >>> 14 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 2;
return 2;
}
-varInt |= (int) ((value & 0x7F) << 16) | 0x8000;
-value >>>= 7;
-if (value == 0) {
+varInt |= (((value & 0x1fc000) << 2) | 0x8000);
+if (value >>> 21 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 3;
return 3;
}
-varInt |= (int) ((value & 0x7F) << 24) | 0x80;
-value >>>= 7;
-if (value == 0) {
+varInt |= ((value & 0xfe0) << 3) | 0x80;
+if (value >>> 28 == 0) {
unsafePutInt(writerIndex, varInt);
this.writerIndex = writerIndex + 4;
return 4;
}
long varLong = (varInt & 0xL);
-varLong |= ((value & 0x7F) << 32) | 0x8000L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x7f000L) << 4) | 0x8000L;
+if (value >>> 35 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 5;
return 5;
}
-varLong |= ((value & 0x7F) << 40) | 0x80L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x3f8L) << 5) | 0x80L;
+if (value >>> 42 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 6;
return 6;
}
-varLong |= ((value & 0x7F) << 48) | 0x8000L;
-value >>>= 7;
-if (value == 0) {
+varLong |= ((value & 0x1fc00L) << 6) | 0x8000L;
+if (value >>> 49 == 0) {
unsafePutLong(writerIndex, varLong);
this.writerIndex = writerIndex + 7;
return 7;
}
-varLong |= ((value & 0x7F) << 56) | 0x80L;
-value >>>= 7;
+varLong |= ((value & 0xfeL) << 7) | 0x80L;
+value >>>= 56;
Review Comment:
`value` does not need to be reassigned here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] fix(java): optimize unsafePutPositiveVarInt bytecode size [incubator-fury]
pandalee99 commented on PR #1489: URL: https://github.com/apache/incubator-fury/pull/1489#issuecomment-2049512845 is so cool -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] perf(java): Reduce unsafeWritePositiveVarLong bytecode size. [incubator-fury]
pandalee99 opened a new pull request, #1491: URL: https://github.com/apache/incubator-fury/pull/1491 ## Reduce unsafeWritePositiveVarLong bytecode size Reduce bytecode size from 431 to 413. ## Related issues Through the calculation in advance to reduce memory #1465 ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark pre https://github.com/apache/incubator-fury/assets/46820719/6627e8ef-b681-46cb-9edd-66839d875459";> after https://github.com/apache/incubator-fury/assets/46820719/ec97c0ed-3939-4149-981b-32957fd38f93";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] fix(java): optimize unsafePutPositiveVarInt bytecode size [incubator-fury]
LiangliangSui commented on PR #1489: URL: https://github.com/apache/incubator-fury/pull/1489#issuecomment-2049500891 Hi @pandalee99 , thanks for contributing to Fury. We can further simplify some intermediate steps to reduce the generated bytecode. I submitted a PR to do this. https://github.com/apache/incubator-fury/pull/1490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" is working again!
The GitHub Actions job "Fury CI" on incubator-fury.git has succeeded. Run started by GitHub user chaokunyang (triggered by chaokunyang). Head commit for run: 81b3db852bc80dcfa59d4ac666aad10f5772b2d2 / chaokunyang fix merge error Report URL: https://github.com/apache/incubator-fury/actions/runs/8645486823 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" failed!
The GitHub Actions job "Fury CI" on incubator-fury.git has failed. Run started by GitHub user chaokunyang (triggered by chaokunyang). Head commit for run: 8264eeb3885192710d215d7baed4b276f13178ae / chaokunyang Merge remote-tracking branch 'ant/main' into concate_string_coder_with_len Report URL: https://github.com/apache/incubator-fury/actions/runs/8645465376 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: perf(java): Reduce unsafePutPositiveVarInt bytecode size. (#1490)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 4ed1ff4a perf(java): Reduce unsafePutPositiveVarInt bytecode size.
(#1490)
4ed1ff4a is described below
commit 4ed1ff4aa9892f0e5aab18876acf624c8adeec5f
Author: LiangliangSui <116876207+liangliang...@users.noreply.github.com>
AuthorDate: Thu Apr 11 18:19:14 2024 +0800
perf(java): Reduce unsafePutPositiveVarInt bytecode size. (#1490)
## What does this PR do?
Reduce unsafePutPositiveVarInt bytecode size from 192 to 151.
## Related issues
https://github.com/apache/incubator-fury/issues/1466
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
before
with this pr
Signed-off-by: LiangliangSui
---
.../java/org/apache/fury/memory/MemoryBuffer.java | 28 ++
1 file changed, 13 insertions(+), 15 deletions(-)
diff --git
a/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
b/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
index b8e431cb..6dda1e78 100644
--- a/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
+++ b/java/fury-core/src/main/java/org/apache/fury/memory/MemoryBuffer.java
@@ -1260,38 +1260,36 @@ public final class MemoryBuffer {
/**
* Caller must ensure there must be at least 8 bytes for writing, otherwise
the crash may occur.
*/
- public int unsafePutPositiveVarInt(int index, int v) {
+ public int unsafePutPositiveVarInt(int index, int value) {
// The encoding algorithm are based on kryo UnsafeMemoryOutput.writeVarInt
// varint are written using little endian byte order.
// This version should have better performance since it remove an index
update.
-long value = v;
long varInt = (value & 0x7F);
-value >>>= 7;
-if (value == 0) {
+if (value >>> 7 == 0) {
UNSAFE.putByte(heapMemory, address + index, (byte) varInt);
return 1;
}
// bit 8 `set` indicates have next data bytes.
-varInt |= ((value & 0x7F) << 8) | 0x80;
-value >>>= 7;
-if (value == 0) {
+// 0x3f80: 0b111 << 7
+varInt |= (((value & 0x3f80) << 1) | 0x80);
+if (value >>> 14 == 0) {
unsafePutInt(index, (int) varInt);
return 2;
}
-varInt |= ((value & 0x7F) << 16) | 0x8000;
-value >>>= 7;
-if (value == 0) {
+// 0x1fc000: 0b111 << 14
+varInt |= (((value & 0x1fc000) << 2) | 0x8000);
+if (value >>> 21 == 0) {
unsafePutInt(index, (int) varInt);
return 3;
}
-varInt |= ((value & 0x7F) << 24) | 0x80;
-value >>>= 7;
-if (value == 0) {
+// 0xfe0: 0b111 << 21
+varInt |= ((value & 0xfe0) << 3) | 0x80;
+if (value >>> 28 == 0) {
unsafePutInt(index, (int) varInt);
return 4;
}
-varInt |= ((value & 0x7F) << 32) | 0x8000L;
-varInt &= 0xFL;
+// 0xfe0: 0b111 << 28
+varInt |= ((value & 0x7f000L) << 4) | 0x8000L;
unsafePutLong(index, varInt);
return 5;
}
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] perf(java): Reduce unsafePutPositiveVarInt bytecode size. [incubator-fury]
chaokunyang merged PR #1490: URL: https://github.com/apache/incubator-fury/pull/1490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [Java] optimize MemoryBuffer.unsafePutPositiveVarInt bytecode size [incubator-fury]
chaokunyang closed issue #1466: [Java] optimize MemoryBuffer.unsafePutPositiveVarInt bytecode size URL: https://github.com/apache/incubator-fury/issues/1466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] perf(java): Reduce unsafePutPositiveVarInt bytecode size. [incubator-fury]
LiangliangSui opened a new pull request, #1490: URL: https://github.com/apache/incubator-fury/pull/1490 Reduce unsafePutPositiveVarInt bytecode size from 192 to 151. ## What does this PR do? Reduce unsafePutPositiveVarInt bytecode size from 192 to 151. ## Related issues https://github.com/apache/incubator-fury/issues/1466 ## Does this PR introduce any user-facing change? - [ ] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? ## Benchmark before  with this pr  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] fix(java): optimize unsafePutPositiveVarInt bytecode size [incubator-fury]
chaokunyang merged PR #1489: URL: https://github.com/apache/incubator-fury/pull/1489 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated (71c5b76f -> da785489)
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/incubator-fury.git from 71c5b76f feat(java): channel stream reader (#1483) add da785489 fix(java): optimize unsafePutPositiveVarInt bytecode size (#1489) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/fury/memory/MemoryBuffer.java | 12 1 file changed, 4 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(JavaScript): implement xlang protocol [incubator-fury]
LiangliangSui commented on code in PR #1487:
URL: https://github.com/apache/incubator-fury/pull/1487#discussion_r1560517624
##
javascript/packages/fury/lib/classResolver.ts:
##
@@ -84,35 +84,34 @@ export default class SerializerResolver {
private writeStringIndex: number[] = [];
private registerSerializer(fury: Fury, description: TypeDescription) {
-return fury.classResolver.registerSerializerById(description.type,
generateSerializer(fury, description));
+return
fury.classResolver.registerSerializerById(SerializerResolver.getTypeIdByInternalSerializerType(description.type),
generateSerializer(fury, description));
}
private initInternalSerializer(fury: Fury) {
this.registerSerializer(fury, Type.string());
this.registerSerializer(fury, Type.array(Type.any()));
this.registerSerializer(fury, Type.map(Type.any(), Type.any()));
this.registerSerializer(fury, Type.bool());
-this.registerSerializer(fury, Type.uint8());
this.registerSerializer(fury, Type.int8());
-this.registerSerializer(fury, Type.uint16());
this.registerSerializer(fury, Type.int16());
-this.registerSerializer(fury, Type.uint32());
this.registerSerializer(fury, Type.int32());
-this.registerSerializer(fury, Type.uint64());
+this.registerSerializer(fury, Type.varInt32());
this.registerSerializer(fury, Type.int64());
-this.registerSerializer(fury, Type.float());
-this.registerSerializer(fury, Type.double());
+this.registerSerializer(fury, Type.sliInt64());
+this.registerSerializer(fury, Type.float16());
+this.registerSerializer(fury, Type.float32());
+this.registerSerializer(fury, Type.float64());
this.registerSerializer(fury, Type.timestamp());
-this.registerSerializer(fury, Type.date());
+this.registerSerializer(fury, Type.duration());
this.registerSerializer(fury, Type.set(Type.any()));
this.registerSerializer(fury, Type.binary());
-this.registerSerializer(fury, Type.stringTypedArray());
-this.registerSerializer(fury, Type.boolTypedArray());
-this.registerSerializer(fury, Type.shortTypedArray());
-this.registerSerializer(fury, Type.intTypedArray());
-this.registerSerializer(fury, Type.longTypedArray());
-this.registerSerializer(fury, Type.floatTypedArray());
-this.registerSerializer(fury, Type.doubleTypedArray());
+this.registerSerializer(fury, Type.boolArray());
Review Comment:
`int8Array` should also be added here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Lint PR" is working again!
The GitHub Actions job "Lint PR" on incubator-fury.git has succeeded. Run started by GitHub user pandalee99 (triggered by pandalee99). Head commit for run: ac19a529e92b61fa3722aadf8921aeae5d508aa0 / PAN <46820719+pandale...@users.noreply.github.com> Merge branch 'main' into fix_issue_1466 Report URL: https://github.com/apache/incubator-fury/actions/runs/8642797104 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Lint PR" failed!
The GitHub Actions job "Lint PR" on incubator-fury.git has failed. Run started by GitHub user pandalee99 (triggered by pandalee99). Head commit for run: ac19a529e92b61fa3722aadf8921aeae5d508aa0 / PAN <46820719+pandale...@users.noreply.github.com> Merge branch 'main' into fix_issue_1466 Report URL: https://github.com/apache/incubator-fury/actions/runs/8642761780 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Lint PR" failed!
The GitHub Actions job "Lint PR" on incubator-fury.git has failed. Run started by GitHub user pandalee99 (triggered by pandalee99). Head commit for run: f446344be5d64a8638dace97a163d7025c9545ce / lipan <1162953...@qq.com> reduce mem to 192 Report URL: https://github.com/apache/incubator-fury/actions/runs/8642759810 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
