(incubator-fury) branch main updated: docs: Remove extra symbols (#1310)
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-fury.git The following commit(s) were added to refs/heads/main by this push: new 2d04ac1a docs: Remove extra symbols (#1310) 2d04ac1a is described below commit 2d04ac1afe2f243fd5dc2f0efa20ca854d6fa4e9 Author: Caican Cai <77189278+caican...@users.noreply.github.com> AuthorDate: Sun Jan 7 01:14:41 2024 +0800 docs: Remove extra symbols (#1310) --- CONTRIBUTING.md | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 471335a5..1f8c33e3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -81,7 +81,7 @@ mvn checkstyle:check ### Python ```bash -cd python +cd python # install dependencies fro styling pip install black==22.1.0 flake8==3.9.1 flake8-quotes flake8-bugbear # format python code @@ -115,7 +115,7 @@ cargo fmt Fury supports dump jit generated code into local file for better debug by configuring environment variables: - `FURY_CODE_DIR`：The directory for fury to dump generated code. Set to empty by default to skip dump code. -- `ENABLE_FURY_GENERATED_CLASS_UNIQUE_ID`: Append an unique id for dynamically generated files by default to avoid serializer collision for different classes with same name. Set this to `false` to keep serializer name same for multiple execution or `AOT` codegen. +- `ENABLE_FURY_GENERATED_CLASS_UNIQUE_ID`: Append an unique id for dynamically generated files by default to avoid serializer collision for different classes with same name. Set this to `false` to keep serializer name same for multiple execution or `AOT` codegen. By using those environment variables, we can generate code to source directory and debug the generated code in next run. @@ -144,12 +144,12 @@ See the [Debugging C++](docs/cpp_debug.md) doc. Enable core dump on Macos Monterey 12.1: ```bash -/usr/libexec/PlistBuddy -c "Add :com.apple.security.get-task-allow bool true" tmp.entitlements +/usr/libexec/PlistBuddy -c "Add :com.apple.security.get-task-allow bool true" tmp.entitlements codesign -s - -f --entitlements tmp.entitlements /Users/chaokunyang/anaconda3/envs/py3.8/bin/python ulimit -c unlimited ``` -... then, run the code: +then run the code: ```bash python fury_serializer.py - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] docs: Remove extra symbols [incubator-fury]
chaokunyang merged PR #1310: URL: https://github.com/apache/incubator-fury/pull/1310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(javascript): Refactor & Compress Long [incubator-fury]
chaokunyang merged PR #1313: URL: https://github.com/apache/incubator-fury/pull/1313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(javascript): Refactor & Compress Long [incubator-fury]
theweipeng commented on code in PR #1313:
URL: https://github.com/apache/incubator-fury/pull/1313#discussion_r1443794087
##
javascript/packages/fury/lib/reader.ts:
##
@@ -144,21 +190,67 @@ export const BinaryReader = (config: Config) => {
return (v >> 1) ^ -(v & 1);
}
+ function zigZagBigInt(v: bigint) {
+return (v >> 1n) ^ -(v & 1n);
+ }
+
function varUInt32() {
-let byte_ = int8();
+let byte_ = uint8();
let result = byte_ & 0x7f;
if ((byte_ & 0x80) != 0) {
- byte_ = int8();
+ byte_ = uint8();
result |= (byte_ & 0x7f) << 7;
if ((byte_ & 0x80) != 0) {
-byte_ = int8();
+byte_ = uint8();
result |= (byte_ & 0x7f) << 14;
if ((byte_ & 0x80) != 0) {
- byte_ = int8();
+ byte_ = uint8();
result |= (byte_ & 0x7f) << 21;
if ((byte_ & 0x80) != 0) {
-byte_ = int8();
-result |= (byte_ & 0x7f) << 28;
+byte_ = uint8();
+result |= (byte_) << 28;
+ }
+}
+ }
+}
+return result;
+ }
+
+ function bigUInt8() {
+return BigInt(uint8() >>> 0);
+ }
+
+ function varUInt64() {
+let byte_ = bigUInt8();
Review Comment:
Great suggestion! Currently, the maximum bit length that the Number type can
safely represent in JavaScript is 53 bits, which is insufficient to decode our
varUInt64. BigInt serves as a workaround; however, its performance in
JavaScript is significantly suboptimal. I plan to address the performance
issues associated with handling 64-bit numbers comprehensively in subsequent
pull requests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(javascript): Refactor & Compress Long [incubator-fury]
chaokunyang commented on code in PR #1313:
URL: https://github.com/apache/incubator-fury/pull/1313#discussion_r1443785892
##
javascript/packages/fury/lib/gen/router.ts:
##
@@ -17,28 +17,22 @@
* under the License.
*/
-import { Fury } from "../type";
-import { InternalSerializerType, RefFlags } from "../type";
+import { TypeDescription } from "../description";
+import { SerializerGenerator } from "./serializer";
+import { InternalSerializerType } from "../type";
+import { Builder } from "./builder";
+import { Scope } from "./scope";
-export default (fury: Fury) => {
- const { binaryReader, binaryWriter, referenceResolver } = fury;
- const { uint8: readUInt8 } = binaryReader;
- const { int8: writeInt8, uint8: writeUInt8 } = binaryWriter;
- return {
-...referenceResolver.deref(() => {
- return readUInt8() === 0 ? false : true;
-}),
-write:
referenceResolver.withNotNullableWriter(InternalSerializerType.BOOL, false, (v:
boolean) => {
- writeUInt8(v ? 1 : 0);
-}),
-writeWithoutType: (v: boolean) => {
- writeInt8(RefFlags.NotNullValueFlag);
- writeUInt8(v ? 1 : 0);
-},
-config: () => {
- return {
-reserve: 4,
- };
-},
- };
-};
+type SerializerGeneratorConstructor = new (description: TypeDescription,
builder: Builder, scope: Scope) => SerializerGenerator;
+
+export class Register {
Review Comment:
In java we have a class named as `CodeGenerator`, and has a class named
`CodecBuilder`. Hope it can give some inspiration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(javascript): Refactor & Compress Long [incubator-fury]
chaokunyang commented on code in PR #1313:
URL: https://github.com/apache/incubator-fury/pull/1313#discussion_r1443785512
##
javascript/packages/fury/lib/gen/router.ts:
##
@@ -17,28 +17,22 @@
* under the License.
*/
-import { Fury } from "../type";
-import { InternalSerializerType, RefFlags } from "../type";
+import { TypeDescription } from "../description";
+import { SerializerGenerator } from "./serializer";
+import { InternalSerializerType } from "../type";
+import { Builder } from "./builder";
+import { Scope } from "./scope";
-export default (fury: Fury) => {
- const { binaryReader, binaryWriter, referenceResolver } = fury;
- const { uint8: readUInt8 } = binaryReader;
- const { int8: writeInt8, uint8: writeUInt8 } = binaryWriter;
- return {
-...referenceResolver.deref(() => {
- return readUInt8() === 0 ? false : true;
-}),
-write:
referenceResolver.withNotNullableWriter(InternalSerializerType.BOOL, false, (v:
boolean) => {
- writeUInt8(v ? 1 : 0);
-}),
-writeWithoutType: (v: boolean) => {
- writeInt8(RefFlags.NotNullValueFlag);
- writeUInt8(v ? 1 : 0);
-},
-config: () => {
- return {
-reserve: 4,
- };
-},
- };
-};
+type SerializerGeneratorConstructor = new (description: TypeDescription,
builder: Builder, scope: Scope) => SerializerGenerator;
+
+export class Register {
Review Comment:
Seems it's codegen related, maybe we can name it as `CodegenRegistry` or
something like
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(javascript): Refactor & Compress Long [incubator-fury]
chaokunyang commented on code in PR #1313:
URL: https://github.com/apache/incubator-fury/pull/1313#discussion_r1443784881
##
javascript/packages/fury/lib/classResolver.ts:
##
@@ -17,87 +17,102 @@
* under the License.
*/
-import { arraySerializer, stringArraySerializer, boolArraySerializer,
shortArraySerializer, intArraySerializer, longArraySerializer,
floatArraySerializer, doubleArraySerializer } from "./internalSerializer/array";
-import stringSerializer from "./internalSerializer/string";
-import binarySerializer from "./internalSerializer/binary";
-import { dateSerializer, timestampSerializer } from
"./internalSerializer/datetime";
-import mapSerializer from "./internalSerializer/map";
-import setSerializer from "./internalSerializer/set";
-import boolSerializer from "./internalSerializer/bool";
-import { uInt16Serializer, int16Serializer, int32Serializer, uInt32Serializer,
uInt64Serializer, floatSerializer, doubleSerializer, uInt8Serializer,
int64Serializer, int8Serializer } from "./internalSerializer/number";
import { InternalSerializerType, Serializer, Fury, BinaryReader, BinaryWriter
as TBinaryWriter } from "./type";
-import anySerializer from "./internalSerializer/any";
+import anySerializer from "./any";
import { fromString } from "./platformBuffer";
import { x64hash128 } from "./murmurHash3";
import { BinaryWriter } from "./writer";
+import { generateSerializer } from "./gen";
+import { Type, TypeDescription } from "./description";
const USESTRINGVALUE = 0;
const USESTRINGID = 1;
-class Lazystring {
+class LazyString {
private string: string | null = null;
private start: number | null = null;
private len: number | null = null;
static fromPair(start: number, len: number) {
-const result = new Lazystring();
+const result = new LazyString();
result.start = start;
result.len = len;
return result;
}
static fromString(str: string) {
-const result = new Lazystring();
+const result = new LazyString();
result.string = str;
return result;
}
toString(binaryReader: BinaryReader) {
if (this.string == null) {
- const str = binaryReader.stringUtf8At(this.start!, this.len!);
- return str;
+ this.string = binaryReader.stringUtf8At(this.start!, this.len!);
}
return this.string;
}
}
+const uninitSerialize = {
+ read: () => {
+throw new Error("uninitSerialize");
+ },
+ write: () => {
+throw new Error("uninitSerialize");
+ },
+ readInner: () => {
+throw new Error("uninitSerialize");
+ },
+ writeInner: () => {
+throw new Error("uninitSerialize");
+ },
+ meta: {
+fixedSize: 0,
+noneable: false,
+ },
+};
+
export default class SerializerResolver {
private internalSerializer: Serializer[] = new Array(300);
private customSerializer: { [key: string]: Serializer } = {
};
- private readStringPool: Lazystring[] = [];
+ private readStringPool: LazyString[] = [];
private writeStringCount = 0;
private writeStringIndex: number[] = [];
+ private regSerializer(fury: Fury, description: TypeDescription) {
+return fury.classResolver.registerSerializerById(description.type,
generateSerializer(fury, description));
+ }
+
private initInternalSerializer(fury: Fury) {
-const _anySerializer = anySerializer(fury);
-this.internalSerializer[InternalSerializerType.ANY] = _anySerializer;
-this.internalSerializer[InternalSerializerType.STRING] =
stringSerializer(fury);
-this.internalSerializer[InternalSerializerType.ARRAY] =
arraySerializer(fury, _anySerializer);
-this.internalSerializer[InternalSerializerType.MAP] = mapSerializer(fury,
_anySerializer, _anySerializer);
-this.internalSerializer[InternalSerializerType.BOOL] =
boolSerializer(fury);
-this.internalSerializer[InternalSerializerType.UINT8] =
uInt8Serializer(fury);
-this.internalSerializer[InternalSerializerType.INT8] =
int8Serializer(fury);
-this.internalSerializer[InternalSerializerType.UINT16] =
uInt16Serializer(fury);
-this.internalSerializer[InternalSerializerType.INT16] =
int16Serializer(fury);
-this.internalSerializer[InternalSerializerType.UINT32] =
uInt32Serializer(fury);
-this.internalSerializer[InternalSerializerType.INT32] =
int32Serializer(fury);
-this.internalSerializer[InternalSerializerType.UINT64] =
uInt64Serializer(fury);
-this.internalSerializer[InternalSerializerType.INT64] =
int64Serializer(fury);
-this.internalSerializer[InternalSerializerType.FLOAT] =
floatSerializer(fury);
-this.internalSerializer[InternalSerializerType.DOUBLE] =
doubleSerializer(fury);
-this.internalSerializer[InternalSerializerType.TIMESTAMP] =
timestampSerializer(fury);
-this.internalSerializer[InternalSerializerType.DATE] =
dateSerializer(fury);
-this.internalSerializer[InternalSerializerType.FURY_SET] =
setSerializer(fury, anySerializer(fury));
-this.internalSerializer[InternalSerializer
Re: [PR] feat(javascript): Refactor & Compress Long [incubator-fury]
chaokunyang commented on code in PR #1313:
URL: https://github.com/apache/incubator-fury/pull/1313#discussion_r1443784379
##
javascript/packages/fury/lib/reader.ts:
##
@@ -144,21 +190,67 @@ export const BinaryReader = (config: Config) => {
return (v >> 1) ^ -(v & 1);
}
+ function zigZagBigInt(v: bigint) {
+return (v >> 1n) ^ -(v & 1n);
+ }
+
function varUInt32() {
-let byte_ = int8();
+let byte_ = uint8();
let result = byte_ & 0x7f;
if ((byte_ & 0x80) != 0) {
- byte_ = int8();
+ byte_ = uint8();
result |= (byte_ & 0x7f) << 7;
if ((byte_ & 0x80) != 0) {
-byte_ = int8();
+byte_ = uint8();
result |= (byte_ & 0x7f) << 14;
if ((byte_ & 0x80) != 0) {
- byte_ = int8();
+ byte_ = uint8();
result |= (byte_ & 0x7f) << 21;
if ((byte_ & 0x80) != 0) {
-byte_ = int8();
-result |= (byte_ & 0x7f) << 28;
+byte_ = uint8();
+result |= (byte_) << 28;
+ }
+}
+ }
+}
+return result;
+ }
+
+ function bigUInt8() {
+return BigInt(uint8() >>> 0);
+ }
+
+ function varUInt64() {
+let byte_ = bigUInt8();
Review Comment:
If remaing bytes is greater than 8, we can get a long first, and check its
bits. This will have better performance than read byte by byte
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(javascript): Refactor & Compress Long [incubator-fury]
chaokunyang commented on code in PR #1313:
URL: https://github.com/apache/incubator-fury/pull/1313#discussion_r1443783805
##
javascript/packages/fury/lib/gen/router.ts:
##
@@ -17,28 +17,22 @@
* under the License.
*/
-import { Fury } from "../type";
-import { InternalSerializerType, RefFlags } from "../type";
+import { TypeDescription } from "../description";
+import { SerializerGenerator } from "./serializer";
+import { InternalSerializerType } from "../type";
+import { Builder } from "./builder";
+import { Scope } from "./scope";
-export default (fury: Fury) => {
- const { binaryReader, binaryWriter, referenceResolver } = fury;
- const { uint8: readUInt8 } = binaryReader;
- const { int8: writeInt8, uint8: writeUInt8 } = binaryWriter;
- return {
-...referenceResolver.deref(() => {
- return readUInt8() === 0 ? false : true;
-}),
-write:
referenceResolver.withNotNullableWriter(InternalSerializerType.BOOL, false, (v:
boolean) => {
- writeUInt8(v ? 1 : 0);
-}),
-writeWithoutType: (v: boolean) => {
- writeInt8(RefFlags.NotNullValueFlag);
- writeUInt8(v ? 1 : 0);
-},
-config: () => {
- return {
-reserve: 4,
- };
-},
- };
-};
+type SerializerGeneratorConstructor = new (description: TypeDescription,
builder: Builder, scope: Scope) => SerializerGenerator;
+
+export class Register {
Review Comment:
This can be named as `Registry`, so `reg` can be renamed to `register`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] feat(javascript): Refactor & Compress Long [incubator-fury]
chaokunyang commented on code in PR #1313:
URL: https://github.com/apache/incubator-fury/pull/1313#discussion_r1443783653
##
javascript/packages/fury/lib/gen/number.ts:
##
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { TypeDescription } from "../description";
+import { Builder } from "./builder";
+import { BaseSerializerGenerator } from "./serializer";
+import { Register } from "./router";
+import { InternalSerializerType } from "../type";
+import { Scope } from "./scope";
+
+function buildNumberSerializer(writeFun: (builder: Builder, accessor: string)
=> string, read: (builder: Builder) => string) {
+ return class NumberSerializerGenerator extends BaseSerializerGenerator {
+description: TypeDescription;
+
+constructor(description: TypeDescription, builder: Builder, scope: Scope) {
+ super(description, builder, scope);
+ this.description = description;
+}
+
+writeStmt(accessor: string): string {
+ return writeFun(this.builder, accessor);
+}
+
+readStmt(accessor: (expr: string) => string): string {
+ return accessor(read(this.builder));
+}
+ };
+}
+
+Register.reg(InternalSerializerType.UINT8,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.uint8(accessor),
+builder => builder.reader.uint8()
+ )
+);
+Register.reg(InternalSerializerType.INT8,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.int8(accessor),
+builder => builder.reader.int8()
+ )
+);
+Register.reg(InternalSerializerType.UINT16,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.uint16(accessor),
+builder => builder.reader.uint16()
+ )
+);
+Register.reg(InternalSerializerType.INT16,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.int16(accessor),
+builder => builder.reader.int16()
+ )
+);
+Register.reg(InternalSerializerType.UINT32,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.uint32(accessor),
+builder => builder.reader.uint32()
+ )
+);
+Register.reg(InternalSerializerType.INT32,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.int32(accessor),
+builder => builder.reader.int32()
+ )
+);
+Register.reg(InternalSerializerType.UINT64,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.varUInt64(accessor),
+builder => builder.reader.varUInt64()
+ )
+);
+Register.reg(InternalSerializerType.INT64,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.sliLong(accessor),
+builder => builder.reader.sliLong()
+ )
+);
+Register.reg(InternalSerializerType.FLOAT,
+ buildNumberSerializer(
+(builder, accessor) => builder.writer.float(accessor),
+builder => builder.reader.float()
+ )
+);
+Register.reg(InternalSerializerType.DOUBLE,
Review Comment:
```suggestion
Registry.register(InternalSerializerType.DOUBLE,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] docs: Remove extra symbols [incubator-fury]
caicancai commented on PR #1310: URL: https://github.com/apache/incubator-fury/pull/1310#issuecomment-1879674076 thank your review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: chore: Update CODEOWNERS (#1316)
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-fury.git The following commit(s) were added to refs/heads/main by this push: new f49e6620 chore: Update CODEOWNERS (#1316) f49e6620 is described below commit f49e6620c84d2f624f5ad68eb852730e3146229b Author: Shawn Yang AuthorDate: Sat Jan 6 20:52:41 2024 +0800 chore: Update CODEOWNERS (#1316) Update code reviewer for cpp module, see more in #1296 - Co-authored-by: Twice --- .github/CODEOWNERS | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 28b3eb48..5af1992e 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,11 +1,10 @@ -* @chaokunyang java @chaokunyang scala @chaokunyang go @chaokunyang integration_tests @chaokunyang python @chaokunyang -src @chaokunyang +src @chaokunyang @PragmaTwice rust @theweipeng @chaokunyang javascript @theweipeng -docs @chaokunyang @theweipeng +docs @chaokunyang @theweipeng @PragmaTwice bazel @chaokunyang - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch chaokunyang-patch-1 deleted (was 3e21d1e5)
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a change to branch chaokunyang-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git was 3e21d1e5 Merge branch 'main' into chaokunyang-patch-1 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] chore: Update CODEOWNERS [incubator-fury]
chaokunyang merged PR #1316: URL: https://github.com/apache/incubator-fury/pull/1316 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [Feature] Add PR and issue specification documents [incubator-fury]
caicancai closed issue #1308: [Feature] Add PR and issue specification documents URL: https://github.com/apache/incubator-fury/issues/1308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [I] [Feature] Add PR and issue specification documents [incubator-fury]
caicancai commented on issue #1308: URL: https://github.com/apache/incubator-fury/issues/1308#issuecomment-1879673253 thank everyone https://github.com/apache/incubator-fury/pull/1317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Lint PR" is working again!
The GitHub Actions job "Lint PR" on incubator-fury.git has succeeded. Run started by GitHub user PragmaTwice (triggered by PragmaTwice). Head commit for run: 3e21d1e513f97802ba901059f03b1984f7a90f99 / Twice Merge branch 'main' into chaokunyang-patch-1 Report URL: https://github.com/apache/incubator-fury/actions/runs/7431347983 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Lint PR" failed!
The GitHub Actions job "Lint PR" on incubator-fury.git has failed. Run started by GitHub user PragmaTwice (triggered by PragmaTwice). Head commit for run: 3e21d1e513f97802ba901059f03b1984f7a90f99 / Twice Merge branch 'main' into chaokunyang-patch-1 Report URL: https://github.com/apache/incubator-fury/actions/runs/7431345510 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch chaokunyang-patch-1 updated (e232662c -> 3e21d1e5)
This is an automated email from the ASF dual-hosted git repository. twice pushed a change to branch chaokunyang-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git from e232662c Remove * add 498a8e27 ci: add PR title lint following the conventional commits (#1317) add 3e21d1e5 Merge branch 'main' into chaokunyang-patch-1 No new revisions were added by this update. Summary of changes: .asf.yaml | 2 +- .github/workflows/ci.yml | 4 +- .github/workflows/pr-lint.yml | 91 ++ .github/workflows/release.yaml | 2 +- CONTRIBUTING.md| 12 ++ 5 files changed, 107 insertions(+), 4 deletions(-) create mode 100644 .github/workflows/pr-lint.yml - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: ci: add PR title lint following the conventional commits (#1317)
This is an automated email from the ASF dual-hosted git repository.
tison pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-fury.git
The following commit(s) were added to refs/heads/main by this push:
new 498a8e27 ci: add PR title lint following the conventional commits
(#1317)
498a8e27 is described below
commit 498a8e27519bbdae9cf6f547327792d32dd8758e
Author: Twice
AuthorDate: Sat Jan 6 19:36:32 2024 +0800
ci: add PR title lint following the conventional commits (#1317)
---
.asf.yaml | 2 +-
.github/workflows/ci.yml | 4 +-
.github/workflows/pr-lint.yml | 91 ++
.github/workflows/release.yaml | 2 +-
CONTRIBUTING.md| 12 ++
5 files changed, 107 insertions(+), 4 deletions(-)
diff --git a/.asf.yaml b/.asf.yaml
index e9940948..875b4cd9 100644
--- a/.asf.yaml
+++ b/.asf.yaml
@@ -45,7 +45,7 @@ github:
enabled_merge_buttons:
squash: true
merge: false
-rebase: true
+rebase: false
protected_branches:
main:
required_status_checks:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 1da0c44a..cec6b775 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -93,7 +93,7 @@ jobs:
cd scala && sbt +test && cd -
integration_tests:
-name: integration_tests
+name: Integration Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
@@ -110,7 +110,7 @@ jobs:
run: ./ci/run_ci.sh integration_tests
javascript:
-name: Javascript CI
+name: JavaScript CI
runs-on: ubuntu-latest
strategy:
matrix:
diff --git a/.github/workflows/pr-lint.yml b/.github/workflows/pr-lint.yml
new file mode 100644
index ..bbd4c57f
--- /dev/null
+++ b/.github/workflows/pr-lint.yml
@@ -0,0 +1,91 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: "Lint PR"
+
+on:
+ pull_request_target:
+types:
+ - opened
+ - edited
+ - synchronize
+
+permissions:
+ pull-requests: read
+
+jobs:
+ main:
+name: Validate PR title
+runs-on: ubuntu-latest
+steps:
+ - uses: amannn/action-semantic-pull-request@v5
+env:
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+with:
+ # Configure which types are allowed (newline-delimited).
+ # Default: https://github.com/commitizen/conventional-commit-types
+ types: |
+fix
+feat
+build
+chore
+ci
+docs
+perf
+refactor
+revert
+style
+test
+ # Configure which scopes are allowed (newline-delimited).
+ # These are regex patterns auto-wrapped in `^ $`.
+ scopes: |
+\S+
+ # Configure that a scope must always be provided.
+ requireScope: false
+ # Configure which scopes are disallowed in PR titles
(newline-delimited).
+ # For instance by setting the value below, `chore(release): ...`
(lowercase)
+ # and `ci(e2e,release): ...` (unknown scope) will be rejected.
+ # These are regex patterns auto-wrapped in `^ $`.
+ # disallowScopes: |
+ # release
+ # [A-Z]+
+ # Configure additional validation for the subject based on a regex.
+ # This example ensures the subject doesn't start with an uppercase
character.
+ # subjectPattern: ^(?![A-Z]).+$
+ # If `subjectPattern` is configured, you can use this property to
override
+ # the default error message that is shown when the pattern doesn't
match.
+ # The variables `subject` and `title` can be used within the message.
+ # subjectPatternError: |
+ # The subject "{subject}" found in the pull request title "{title}"
+ # didn't match the configured pattern. Please ensure that the
subject
+ # doesn't start with an uppercase character.
+ # The GitHub base URL will be automatically set to the correct value
from the GitHub context variable.
+ # If you want to override this, you can do so here (not rec
Re: [PR] ci: add PR title lint following the conventional commits [incubator-fury]
tisonkun merged PR #1317: URL: https://github.com/apache/incubator-fury/pull/1317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Refactor & Compress Long [incubator-fury]
theweipeng commented on PR #1313: URL: https://github.com/apache/incubator-fury/pull/1313#issuecomment-1879650870 > Looks good to me, but has some typo. > > And we can improve the style of generated code, this can be done in further I fixed it and added an afterCodeGenerated hook to allow users to format the code. Formatting is not a default action, as I believe we should keep things simple and avoid including too many dependencies. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] ci: add PR title lint following the conventional commits [incubator-fury]
PragmaTwice opened a new pull request, #1317: URL: https://github.com/apache/incubator-fury/pull/1317 We can add a CI workflow to check if the PR title follows [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch chaokunyang-patch-1 updated (e96007c4 -> e232662c)
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a change to branch chaokunyang-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git from e96007c4 Update CODEOWNERS for doc add e232662c Remove * No new revisions were added by this update. Summary of changes: .github/CODEOWNERS | 1 - 1 file changed, 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch chaokunyang-patch-1 updated (8bdd0d0a -> e96007c4)
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a change to branch chaokunyang-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git from 8bdd0d0a Update CODEOWNERS add e96007c4 Update CODEOWNERS for doc No new revisions were added by this update. Summary of changes: .github/CODEOWNERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] Update CODEOWNERS [incubator-fury]
chaokunyang commented on PR #1316: URL: https://github.com/apache/incubator-fury/pull/1316#issuecomment-1879636075 cc @PragmaTwice -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury-site) branch deploy updated: deploy: 3ec0f18c3846699f20e29c58e5a4d423e912237a
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch deploy
in repository https://gitbox.apache.org/repos/asf/incubator-fury-site.git
The following commit(s) were added to refs/heads/deploy by this push:
new 0851868 deploy: 3ec0f18c3846699f20e29c58e5a4d423e912237a
0851868 is described below
commit 08518684a9f7533e6b33bd5132c915b776e0f52f
Author: chaokunyang
AuthorDate: Sat Jan 6 10:51:06 2024 +
deploy: 3ec0f18c3846699f20e29c58e5a4d423e912237a
---
404.html | 4 +--
assets/js/7b95f1e0.18859c09.js | 1 +
assets/js/7b95f1e0.3d27b59c.js | 1 -
assets/js/935f2afb.2e1389b4.js | 1 -
assets/js/935f2afb.dcb40604.js | 1 +
assets/js/{main.71093cf5.js => main.8121992e.js} | 4 +--
...js.LICENSE.txt => main.8121992e.js.LICENSE.txt} | 0
...e~main.fd0c6d3d.js => runtime~main.4bcea430.js} | 2 +-
blog/archive/index.html| 4 +--
blog/fury_0_1_0_release/index.html | 4 +--
blog/fury_0_1_1_release/index.html | 4 +--
blog/fury_0_1_2_release/index.html | 4 +--
blog/fury_0_2_0_release/index.html | 4 +--
blog/fury_0_2_1_release/index.html | 4 +--
blog/fury_0_3_0_release/index.html | 4 +--
blog/fury_0_3_1_release/index.html | 4 +--
blog/fury_0_4_0_release/index.html | 4 +--
blog/fury_0_4_1_release/index.html | 4 +--
.../index.html | 4 +--
blog/index.html| 4 +--
blog/tags/fury/index.html | 4 +--
blog/tags/index.html | 4 +--
docs/guide/development/index.html | 34 ++
docs/guide/graalvm_guide/index.html| 4 +--
docs/guide/java_object_graph_guide/index.html | 4 +--
docs/guide/row_format_guide/index.html | 4 +--
docs/guide/scala_guide/index.html | 4 +--
docs/guide/xlang_object_graph_guide/index.html | 4 +--
docs/introduction/benchmark/index.html | 4 +--
docs/introduction/features/index.html | 4 +--
docs/introduction/index.html | 4 +--
docs/start/install/index.html | 4 +--
docs/start/usage/index.html| 4 +--
index.html | 4 +--
lunr-index-1704467152724.json | 1 -
lunr-index-1704538237555.json | 1 +
lunr-index.json| 2 +-
markdown-page/index.html | 4 +--
search-doc-1704467152724.json | 1 -
search-doc-1704538237555.json | 1 +
search-doc.json| 2 +-
41 files changed, 79 insertions(+), 81 deletions(-)
diff --git a/404.html b/404.html
index b14ef9b..c8a501f 100644
--- a/404.html
+++ b/404.html
@@ -5,8 +5,8 @@
Page Not Found | Apache Fury (incubating)https://fury.apache.org/404.html";>
-
-
+
+
!function(){function
t(t){document.documentElement.setAttribute("data-theme",t)}var
e=function(){try{return new
URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}}()||function(){try{return
localStorage.getItem("theme")}catch(t){}}();t(null!==e?e:"light")}(),function(){try{const
c=new URLSearchParams(window.location.search).entries();for(var[t,e]of
c)if(t.startsWith("docusaurus-data-")){var
a=t.replace("docusaurus-data-","data-");document.documentElement.se [...]
diff --git a/assets/js/7b95f1e0.18859c09.js b/assets/js/7b95f1e0.18859c09.js
new file mode 100644
index 000..b654a0f
--- /dev/null
+++ b/assets/js/7b95f1e0.18859c09.js
@@ -0,0 +1 @@
+"use
strict";(self.webpackChunkfury_site=self.webpackChunkfury_site||[]).push([[2580],{3637:(e,n,r)=>{r.r(n),r.d(n,{assets:()=>d,contentTitle:()=>s,default:()=>c,frontMatter:()=>l,metadata:()=>u,toc:()=>o});var
i=r(5893),t=r(1151);const
l={title:"Development",sidebar_position:6,id:"development"},s="How to build
Fury",u={id:"guide/development",title:"Development",description:"Please
checkout the source tree from
https://github.com/apache/incubator-fury.",source:"@site/docs/guide/DEVELOPME
[...]
\ No newline at end of file
diff --git a/assets/js/7b95f1e0.3d27b59c.js b/assets/js/7b95f1e0.3d27b59c.js
deleted file mode 100644
index bf75ebe..000
--- a/assets/js/7b95f1e0.3d27b59c.js
+++ /dev/null
@@ -1 +0,0 @@
-"use
strict";(self.webpackChunkfury_site=self.webpackChunkfury_site||[]).push([[2580],{3637:(e,n,i)=>{i.r(n),i.d(n,{assets:()=>d,contentTitle:()=>u,default:()=>c,frontMatter:()=>t,metadata:()=>s,toc:()=>
[PR] Update CODEOWNERS [incubator-fury]
chaokunyang opened a new pull request, #1316: URL: https://github.com/apache/incubator-fury/pull/1316 Update code reviewer for cpp module -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) 01/01: Update CODEOWNERS
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a commit to branch chaokunyang-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git commit 8bdd0d0a17b00a125ff156952aef8c06b127da8f Author: Shawn Yang AuthorDate: Sat Jan 6 18:50:49 2024 +0800 Update CODEOWNERS Update code reviewer for cpp module --- .github/CODEOWNERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 28b3eb48..d9b24c4e 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -4,7 +4,7 @@ scala @chaokunyang go @chaokunyang integration_tests @chaokunyang python @chaokunyang -src @chaokunyang +src @chaokunyang @PragmaTwice rust @theweipeng @chaokunyang javascript @theweipeng docs @chaokunyang @theweipeng - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch chaokunyang-patch-1 created (now 8bdd0d0a)
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a change to branch chaokunyang-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git at 8bdd0d0a Update CODEOWNERS This branch includes the following new commits: new 8bdd0d0a Update CODEOWNERS The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [DOC] Refine build steps and wording in DEVELOPMENT [incubator-fury]
PragmaTwice merged PR #1315: URL: https://github.com/apache/incubator-fury/pull/1315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury-site) branch main updated: 🔄 synced local 'docs/guide/' with remote 'docs/guide/'
This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-fury-site.git The following commit(s) were added to refs/heads/main by this push: new 3ec0f18 🔄 synced local 'docs/guide/' with remote 'docs/guide/' 3ec0f18 is described below commit 3ec0f18c3846699f20e29c58e5a4d423e912237a Author: chaokunyang AuthorDate: Sat Jan 6 10:50:06 2024 + 🔄 synced local 'docs/guide/' with remote 'docs/guide/' --- docs/guide/DEVELOPMENT.md | 33 +++-- 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/docs/guide/DEVELOPMENT.md b/docs/guide/DEVELOPMENT.md index cf94e23..01b660c 100644 --- a/docs/guide/DEVELOPMENT.md +++ b/docs/guide/DEVELOPMENT.md @@ -4,15 +4,11 @@ sidebar_position: 6 id: development --- -# How to build to Fury +# How to build Fury -## Get the source code +Please checkout the source tree from https://github.com/apache/incubator-fury. -Github repo: https://github.com/apache/incubator-fury - -## Building Fury 🏋🏿♀️ - -### Building Fury Java +### Build Fury Java ```bash cd java @@ -24,7 +20,7 @@ mvn clean compile -DskipTests - java 1.8+ - maven 3.6.3+ -### Building Fury Python +### Build Fury Python ```bash cd python @@ -34,29 +30,30 @@ pip install -v -e . Environment Requirements -- python3.6+ +- python 3.6+ -### Building Fury C++ +### Build Fury C++ -Build fury_util.so: +Build fury row format： ```bash -bazel build //src/fury/util:fury_util +pip install pyarrow==14.0.0 +bazel build //src/fury/row:fury_row_format ``` -Build fury row format： +Build fury row format encoder: ```bash pip install pyarrow==14.0.0 -bazel build //src/fury/row:fury_row_format +bazel build //src/fury/encoder:fury_encoder ``` Environment Requirements -- cpp 11+ +- compilers with C++17 support - bazel 6.3.2 -### Building Fury GoLang +### Build Fury GoLang ```bash cd go/fury @@ -68,9 +65,9 @@ go test -v fury_xlang_test.go Environment Requirements -- go1.3+ +- go 1.13+ -### Building Fury Rust +### Build Fury Rust ```bash cd rust - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: [DOC] Refine build steps and wording in DEVELOPMENT (#1315)
This is an automated email from the ASF dual-hosted git repository. twice pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-fury.git The following commit(s) were added to refs/heads/main by this push: new 30ba9cf0 [DOC] Refine build steps and wording in DEVELOPMENT (#1315) 30ba9cf0 is described below commit 30ba9cf0a74a0ddbe5953b094298564fe50edcdc Author: Twice AuthorDate: Sat Jan 6 18:49:51 2024 +0800 [DOC] Refine build steps and wording in DEVELOPMENT (#1315) See changes. --- docs/guide/DEVELOPMENT.md | 33 +++-- 1 file changed, 15 insertions(+), 18 deletions(-) diff --git a/docs/guide/DEVELOPMENT.md b/docs/guide/DEVELOPMENT.md index cf94e236..01b660c4 100644 --- a/docs/guide/DEVELOPMENT.md +++ b/docs/guide/DEVELOPMENT.md @@ -4,15 +4,11 @@ sidebar_position: 6 id: development --- -# How to build to Fury +# How to build Fury -## Get the source code +Please checkout the source tree from https://github.com/apache/incubator-fury. -Github repo: https://github.com/apache/incubator-fury - -## Building Fury 🏋🏿♀️ - -### Building Fury Java +### Build Fury Java ```bash cd java @@ -24,7 +20,7 @@ mvn clean compile -DskipTests - java 1.8+ - maven 3.6.3+ -### Building Fury Python +### Build Fury Python ```bash cd python @@ -34,29 +30,30 @@ pip install -v -e . Environment Requirements -- python3.6+ +- python 3.6+ -### Building Fury C++ +### Build Fury C++ -Build fury_util.so: +Build fury row format： ```bash -bazel build //src/fury/util:fury_util +pip install pyarrow==14.0.0 +bazel build //src/fury/row:fury_row_format ``` -Build fury row format： +Build fury row format encoder: ```bash pip install pyarrow==14.0.0 -bazel build //src/fury/row:fury_row_format +bazel build //src/fury/encoder:fury_encoder ``` Environment Requirements -- cpp 11+ +- compilers with C++17 support - bazel 6.3.2 -### Building Fury GoLang +### Build Fury GoLang ```bash cd go/fury @@ -68,9 +65,9 @@ go test -v fury_xlang_test.go Environment Requirements -- go1.3+ +- go 1.13+ -### Building Fury Rust +### Build Fury Rust ```bash cd rust - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [Hotfix] Remove extra symbols [incubator-fury]
chaokunyang commented on code in PR #1310: URL: https://github.com/apache/incubator-fury/pull/1310#discussion_r1443689506 ## CONTRIBUTING.md: ## @@ -132,12 +132,12 @@ See the [Debugging C++](docs/cpp_debug.md) doc. Enable core dump on Macos Monterey 12.1: ```bash -/usr/libexec/PlistBuddy -c "Add :com.apple.security.get-task-allow bool true" tmp.entitlements +/usr/libexec/PlistBuddy -c "Add :com.apple.security.get-task-allow bool true" tmp.entitlements codesign -s - -f --entitlements tmp.entitlements /Users/chaokunyang/anaconda3/envs/py3.8/bin/python ulimit -c unlimited ``` -... then, run the code: Review Comment: then run the code: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] [DOC] Refine build steps and wording in DEVELOPMENT [incubator-fury]
PragmaTwice opened a new pull request, #1315: URL: https://github.com/apache/incubator-fury/pull/1315 See changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] Refine English writing about benchmark in README [incubator-fury]
tisonkun merged PR #1314: URL: https://github.com/apache/incubator-fury/pull/1314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch PragmaTwice-patch-1 deleted (was 6dfe3730)
This is an automated email from the ASF dual-hosted git repository. tison pushed a change to branch PragmaTwice-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git was 6dfe3730 Refine English writing about benchmark in README The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch main updated: Refine English writing about benchmark in README (#1314)
This is an automated email from the ASF dual-hosted git repository. tison pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-fury.git The following commit(s) were added to refs/heads/main by this push: new 84cd52d0 Refine English writing about benchmark in README (#1314) 84cd52d0 is described below commit 84cd52d0361d742c6bf0cb25d6a35ac49e44ed5c Author: Twice AuthorDate: Sat Jan 6 18:08:38 2024 +0800 Refine English writing about benchmark in README (#1314) See changes. --- README.md | 14 ++ 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 7075dcb5..df7d9f66 100644 --- a/README.md +++ b/README.md @@ -27,9 +27,9 @@ In addition to cross-language serialization, Fury also features at: - Drop-in replace Java serialization frameworks such as JDK/Kryo/Hessian, but 100x faster at most, which can greatly improve the efficiency of high-performance RPC calls, data transfer, and object persistence. -- **100% compatible** with JDK serialization API with much faster implementation: supporting JDK `writeObject/readObject/writeReplace/readResolve/readObjectNoData/Externalizable` API. +- **100% compatible** with JDK serialization API with much faster implementation: supporting JDK `writeObject`/`readObject`/`writeReplace`/`readResolve`/`readObjectNoData`/`Externalizable` API. - Supports **Java 8~21**, Java 17+ `record` is supported too. -- Support [AOT compilation serialization](docs/guide/graalvm_guide.md) for **GraalVM native image**, and no reflection/serialization json config are needed. +- Supports [AOT compilation serialization](docs/guide/graalvm_guide.md) for **GraalVM native image**, and no reflection/serialization json config are needed. - Supports shared and circular reference object serialization for golang. - Supports [scala serialization](docs/guide/scala_guide.md) - Supports automatic object serialization for golang. @@ -55,14 +55,12 @@ Different serialization frameworks are suitable for different scenarios, and ben If you need to benchmark for your specific scenario, make sure all serialization frameworks are appropriately configured for that scenario. Dynamic serialization frameworks support polymorphism and references, but they often come with a higher cost compared to static serialization frameworks, unless they utilize JIT techniques like Fury does. -Because Fury generates code at runtime, it is recommended to **warm up** the system before collecting benchmark statistics. +To ensure accurate benchmark statistics, it is advisable to **warm up** the system before collecting data due to Fury's runtime code generation. ### Java Serialization -Title containing "compatible" represent schema compatible mode: support type forward/backward compatibility. +In these charts below, titles containing "compatible" represent schema compatible mode: type forward/backward compatibility is enabled; while titles without "compatible" represent schema consistent mode: class schema must be the same between serialization and deserialization. -Title without "compatible" represent schema consistent mode: class schema must be the same between serialization and deserialization. - -`Struct` is a class with [100 primitive fields](https://github.com/apache/incubator-fury/tree/main/docs/benchmarks#Struct), `MediaContent` is a class from [jvm-serializers](https://github.com/eishay/jvm-serializers/blob/master/tpc/src/data/media/MediaContent.java), `Sample` is a class from [kryo benchmark](https://github.com/EsotericSoftware/kryo/blob/master/benchmarks/src/main/java/com/esotericsoftware/kryo/benchmarks/data/Sample.java). +Where `Struct` is a class with [100 primitive fields](https://github.com/apache/incubator-fury/tree/main/docs/benchmarks#Struct), `MediaContent` is a class from [jvm-serializers](https://github.com/eishay/jvm-serializers/blob/master/tpc/src/data/media/MediaContent.java), and `Sample` is a class from [kryo benchmark](https://github.com/EsotericSoftware/kryo/blob/master/benchmarks/src/main/java/com/esotericsoftware/kryo/benchmarks/data/Sample.java). @@ -152,7 +150,7 @@ Here we give a quick start about how to use Fury, see [user guide](https://githu ### Fury java object graph serialization If you don't have cross-language requirements, using this mode will -have better performance. +result in better performance. ```java import org.apache.fury.*; import org.apache.fury.config.*; - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Refactor & Compress Long [incubator-fury]
bytemain commented on code in PR #1313:
URL: https://github.com/apache/incubator-fury/pull/1313#discussion_r1443663449
##
javascript/packages/fury/lib/classResolver.ts:
##
@@ -158,26 +184,15 @@ export default class SerializerResolver {
this.writeStringIndex[idx] = this.writeStringCount++;
binaryWriter.buffer(fullBuffer);
},
- bufferLen,
};
}
- detectTag(binaryReader: BinaryReader) {
-const flag = binaryReader.uint8();
-if (flag === USESTRINGVALUE) {
- binaryReader.skip(8); // The tag hash is not needed at the moment.
- return binaryReader.stringUtf8(binaryReader.int16());
-} else {
- return this.readStringPool[binaryReader.int16()].toString(binaryReader);
-}
- }
-
readTag(binaryReader: BinaryReader) {
const flag = binaryReader.uint8();
if (flag === USESTRINGVALUE) {
binaryReader.skip(8); // The tag hash is not needed at the moment.
- const start = binaryReader.getCursor();
const len = binaryReader.int16();
+ const start = binaryReader.getCursor();
binaryReader.skip(len);
this.readStringPool.push(Lazystring.fromPair(start, len));
Review Comment:
I think the class name should use camel case
```diff
- class Lazystring
+ class LazyString
```
##
javascript/packages/fury/lib/gen/index.ts:
##
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import { InternalSerializerType, Fury } from "../type";
+import { ArrayTypeDescription, MapTypeDescription, ObjectTypeDescription,
SetTypeDescription, TupleTypeDescription, TypeDescription } from
"../description";
+import { Register } from "./router";
+import { Builder } from "./builder";
+import { Scope } from "./scope";
+import "./array";
+import "./object";
+import "./string";
+import "./binary";
+import "./bool";
+import "./datetime";
+import "./map";
+import "./number";
+import "./set";
+import "./any";
+import "./tuple";
+import "./typedArray";
+
+export const generate = (fury: Fury, description: TypeDescription) => {
+ const InnerGeneratorClass = Register.get(description.type);
+ if (!InnerGeneratorClass) {
+throw new Error(`${description.type} generator not exists`);
+ }
+ const scope = new Scope();
+ const generator = new InnerGeneratorClass(description, new Builder(scope,
fury), scope);
+
+ const funcString = generator.toSerializer();
+ return new Function(funcString);
+};
+
+function regDependences(fury: Fury, description: TypeDescription) {
Review Comment:
```diff
- regDependences
+ regDependencies
```
##
javascript/packages/fury/lib/type.ts:
##
@@ -28,7 +29,7 @@ export type BinaryReader = ReturnType;
export enum InternalSerializerType {
STRING = 13,
ARRAY = 25,
- TUPLE = 25,
+ TUPLE = 25.1,
Review Comment:
after the enum value change, we can remove the redundant code now:
https://github.com/apache/incubator-fury/assets/13938334/54a7cfd9-3541-43bc-82b3-b33207cfab32";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Refactor & Compress Long [incubator-fury]
bytemain commented on PR #1313: URL: https://github.com/apache/incubator-fury/pull/1313#issuecomment-1879609292 Looks good to me, but has some typo. And we can improve the style of generated code, this can be done in further -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Refactor & Compress Long [incubator-fury]
bytemain commented on PR #1313: URL: https://github.com/apache/incubator-fury/pull/1313#issuecomment-1879608581 > @bytemain Hi, could you please take a review, It remove the limit which mentioned in #1304 I think you can add some test case in the #1304: - https://github.com/apache/incubator-fury/pull/1304/files#diff-9d33aa5fce03357373ad7c54b9636452600284f3e7153064438e277158a3e9cfR60 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Refactor & Compress Long [incubator-fury]
theweipeng commented on PR #1313: URL: https://github.com/apache/incubator-fury/pull/1313#issuecomment-1879597937 > it seem that this is not a problem, all the call will happen in this closure > > https://private-user-images.githubusercontent.com/13938334/294660531-5a43b6e1-523e-4145-82ed-5900f5ca1e6e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDQ1MzEzMDcsIm5iZiI6MTcwNDUzMTAwNywicGF0aCI6Ii8xMzkzODMzNC8yOTQ2NjA1MzEtNWE0M2I2ZTEtNTIzZS00MTQ1LTgyZWQtNTkwMGY1Y2ExZTZlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAxMDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMTA2VDA4NTAwN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWZiZjc2ZmJlNmJhYTQ5OWRmNTRkY2VhNzYxZTFhY2YxZTg4Yjk0OTE4MzdjZjczMGY5ZmIwZTgyYjFiZDI5NGUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.D3ZFZNlAogNWEPaogXbBpI_jPsSwUvjKg_LRS-YTEnk";> > I checked the snapshot and donot sure is this a problem? This is not a problem, but declare the readInner before the read is better. I will change it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Refactor & Compress Long [incubator-fury]
bytemain commented on PR #1313: URL: https://github.com/apache/incubator-fury/pull/1313#issuecomment-1879596076 fantasitic work, this is the thing I wanted of a serialization framework -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Refactor & Compress Long [incubator-fury]
bytemain commented on PR #1313: URL: https://github.com/apache/incubator-fury/pull/1313#issuecomment-1879595821 https://github.com/apache/incubator-fury/assets/13938334/5a43b6e1-523e-4145-82ed-5900f5ca1e6e";> I checked the snapshot and donot sure is this a problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Serialize any type [incubator-fury]
bytemain closed pull request #1304: [JavaScript] Serialize any type URL: https://github.com/apache/incubator-fury/pull/1304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Refactor & Compress Long [incubator-fury]
theweipeng commented on PR #1313: URL: https://github.com/apache/incubator-fury/pull/1313#issuecomment-1879593751 @bytemain Hi, could you please take a review, It remove the limit which mentioned in #1304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] Refine English writing about benchmark in README [incubator-fury]
PragmaTwice opened a new pull request, #1314: URL: https://github.com/apache/incubator-fury/pull/1314 See changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" is working again!
The GitHub Actions job "Fury CI" on incubator-fury.git has succeeded. Run started by GitHub user theweipeng (triggered by theweipeng). Head commit for run: f1758317f0f9e15ad8e82c25b8247bbf42d823f5 / theweipeng chore: lint fix feature: refactor to enhancement Report URL: https://github.com/apache/incubator-fury/actions/runs/7430484106 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) branch PragmaTwice-patch-1 created (now 6dfe3730)
This is an automated email from the ASF dual-hosted git repository. twice pushed a change to branch PragmaTwice-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git at 6dfe3730 Refine English writing about benchmark in README This branch includes the following new commits: new 6dfe3730 Refine English writing about benchmark in README The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
(incubator-fury) 01/01: Refine English writing about benchmark in README
This is an automated email from the ASF dual-hosted git repository. twice pushed a commit to branch PragmaTwice-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-fury.git commit 6dfe37301cb13c990c6e02784d17fd4edef3089b Author: Twice AuthorDate: Sat Jan 6 16:28:03 2024 +0800 Refine English writing about benchmark in README --- README.md | 14 ++ 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 7075dcb5..df7d9f66 100644 --- a/README.md +++ b/README.md @@ -27,9 +27,9 @@ In addition to cross-language serialization, Fury also features at: - Drop-in replace Java serialization frameworks such as JDK/Kryo/Hessian, but 100x faster at most, which can greatly improve the efficiency of high-performance RPC calls, data transfer, and object persistence. -- **100% compatible** with JDK serialization API with much faster implementation: supporting JDK `writeObject/readObject/writeReplace/readResolve/readObjectNoData/Externalizable` API. +- **100% compatible** with JDK serialization API with much faster implementation: supporting JDK `writeObject`/`readObject`/`writeReplace`/`readResolve`/`readObjectNoData`/`Externalizable` API. - Supports **Java 8~21**, Java 17+ `record` is supported too. -- Support [AOT compilation serialization](docs/guide/graalvm_guide.md) for **GraalVM native image**, and no reflection/serialization json config are needed. +- Supports [AOT compilation serialization](docs/guide/graalvm_guide.md) for **GraalVM native image**, and no reflection/serialization json config are needed. - Supports shared and circular reference object serialization for golang. - Supports [scala serialization](docs/guide/scala_guide.md) - Supports automatic object serialization for golang. @@ -55,14 +55,12 @@ Different serialization frameworks are suitable for different scenarios, and ben If you need to benchmark for your specific scenario, make sure all serialization frameworks are appropriately configured for that scenario. Dynamic serialization frameworks support polymorphism and references, but they often come with a higher cost compared to static serialization frameworks, unless they utilize JIT techniques like Fury does. -Because Fury generates code at runtime, it is recommended to **warm up** the system before collecting benchmark statistics. +To ensure accurate benchmark statistics, it is advisable to **warm up** the system before collecting data due to Fury's runtime code generation. ### Java Serialization -Title containing "compatible" represent schema compatible mode: support type forward/backward compatibility. +In these charts below, titles containing "compatible" represent schema compatible mode: type forward/backward compatibility is enabled; while titles without "compatible" represent schema consistent mode: class schema must be the same between serialization and deserialization. -Title without "compatible" represent schema consistent mode: class schema must be the same between serialization and deserialization. - -`Struct` is a class with [100 primitive fields](https://github.com/apache/incubator-fury/tree/main/docs/benchmarks#Struct), `MediaContent` is a class from [jvm-serializers](https://github.com/eishay/jvm-serializers/blob/master/tpc/src/data/media/MediaContent.java), `Sample` is a class from [kryo benchmark](https://github.com/EsotericSoftware/kryo/blob/master/benchmarks/src/main/java/com/esotericsoftware/kryo/benchmarks/data/Sample.java). +Where `Struct` is a class with [100 primitive fields](https://github.com/apache/incubator-fury/tree/main/docs/benchmarks#Struct), `MediaContent` is a class from [jvm-serializers](https://github.com/eishay/jvm-serializers/blob/master/tpc/src/data/media/MediaContent.java), and `Sample` is a class from [kryo benchmark](https://github.com/EsotericSoftware/kryo/blob/master/benchmarks/src/main/java/com/esotericsoftware/kryo/benchmarks/data/Sample.java). @@ -152,7 +150,7 @@ Here we give a quick start about how to use Fury, see [user guide](https://githu ### Fury java object graph serialization If you don't have cross-language requirements, using this mode will -have better performance. +result in better performance. ```java import org.apache.fury.*; import org.apache.fury.config.*; - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
Re: [PR] [JavaScript] Serialize any type [incubator-fury]
theweipeng commented on PR #1304: URL: https://github.com/apache/incubator-fury/pull/1304#issuecomment-1879593382 I have refactored the code generator in another pr #1313 , which had been remove the limit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" failed!
The GitHub Actions job "Fury CI" on incubator-fury.git has failed. Run started by GitHub user theweipeng (triggered by theweipeng). Head commit for run: fe13f5f5d4bdc071b67eb1fcabdeca5254056815 / weipeng Merge branch 'main' into compress_u64 Report URL: https://github.com/apache/incubator-fury/actions/runs/7430445806 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[GH] (incubator-fury): Workflow run "Fury CI" failed!
The GitHub Actions job "Fury CI" on incubator-fury.git has failed. Run started by GitHub user theweipeng (triggered by theweipeng). Head commit for run: e4695dc47667925aaf6e9dc58942a5715c70b5e1 / theweipeng feature: refactor to enhancement Report URL: https://github.com/apache/incubator-fury/actions/runs/7430445209 With regards, GitHub Actions via GitBox - To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org For additional commands, e-mail: commits-h...@fury.apache.org
[PR] [JavaScript] Refactor & Compress Long [incubator-fury]
theweipeng opened a new pull request, #1313:
URL: https://github.com/apache/incubator-fury/pull/1313
## Refactor the code generator
1. Currently, JavaScript can only register a Object by Type.object, because
the generator is too simple
2. Primitive generator of types like string、number can't be inlined in the
holder collection, which cause necessary polymorphic that affect the performance
## Enhancement performance
1. Currently, Buffer.latin1Slice would transfer from native code generated
by JIT to v8 runtime, This has a significant impact on performance, so we use
String.fromCharCode to create little string when the string length is less than
15. It is a magic number, but I have tested it and it works fine.
## Improve
When useSliceString is disabled, the performance of deserialization improves
by 100%. It is 3 times faster than native JSON and twice as fast as protobuf.
When useSliceString is enabled, the performance of deserialization improves
by 30%. It is 5 times faster than native JSON and 3 times faster than protobuf.
### Before:
Disable useSliceString
Enable useSliceString
### After:
Disable useSliceString
Enable useSliceString
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@fury.apache.org
For additional commands, e-mail: commits-h...@fury.apache.org
