[jira] [Created] (AVRO-3897) Disallow invalid namespace in fully qualified name for Rust SDK
Kousuke Saruta created AVRO-3897:
Summary: Disallow invalid namespace in fully qualified name for
Rust SDK
Key: AVRO-3897
URL: https://issues.apache.org/jira/browse/AVRO-3897
Project: Apache Avro
Issue Type: Bug
Components: rust
Reporter: Kousuke Saruta
Currently, the Rust SDK allows the following fully qualified names with
Name::new.
{code}
Name::new("ns.0.record1")
Name::new("ns..record1")
{code}
But they should be disallowed according to the specification.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3880) Upgrade maven-antrun-plugin to 3.1.0
Kousuke Saruta created AVRO-3880:
Summary: Upgrade maven-antrun-plugin to 3.1.0
Key: AVRO-3880
URL: https://issues.apache.org/jira/browse/AVRO-3880
Project: Apache Avro
Issue Type: Improvement
Components: build, java
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
In the current main branch, if we try to generate .class files from .proto
files for test, we will get an error like as follows.
{code}
$ mvn -P protobuf-generate generate-test-sources
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:3.1.0:run (default) on project
avro-protobuf: You are using 'tasks' which has been removed from the
maven-antrun-plugin. Please use 'target' and refer to the >>Major Version
Upgrade to version 3.0.0<< on the plugin site. -> [Help 1]
{code}
This is because maven-antrun-plugin 3.1.0 is automatically downloaded but
"tasks" tag is not supported by the version.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3870) Speed up CI for Rust
Kousuke Saruta created AVRO-3870: Summary: Speed up CI for Rust Key: AVRO-3870 URL: https://issues.apache.org/jira/browse/AVRO-3870 Project: Apache Avro Issue Type: Improvement Components: build, rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta In the current master, there are something wrong about actions/cache in `test-lang-rust-ci.yml`. First, a directory "target" is tend to be cached but the path is wrong. the correct path is "lang/rust/target", not "~/target". Second, as of Rust 1.70.0, Cargo changes the way to download dependencies. https://blog.rust-lang.org/2023/06/01/Rust-1.70.0.html#sparse-by-default-for-cratesio So, it's better not to share the cache for "~/.cargo" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3868) Check consistency between the doc comment in lib.rs and README.md
Kousuke Saruta created AVRO-3868: Summary: Check consistency between the doc comment in lib.rs and README.md Key: AVRO-3868 URL: https://issues.apache.org/jira/browse/AVRO-3868 Project: Apache Avro Issue Type: Improvement Components: build, rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta AVRO-3849 added cargo-rdme. So it's nice if we can check the consistency between avro/src/lib.rs and avro/README.md -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3867) Link to the Python API doc is broken
Kousuke Saruta created AVRO-3867: Summary: Link to the Python API doc is broken Key: AVRO-3867 URL: https://issues.apache.org/jira/browse/AVRO-3867 Project: Apache Avro Issue Type: Bug Reporter: Kousuke Saruta I built the website by ./build.sh dist and then tried to access to the Python API doc but the link seems broken. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3866) [Build][Python] Files are leftover after a build
Kousuke Saruta created AVRO-3866: Summary: [Build][Python] Files are leftover after a build Key: AVRO-3866 URL: https://issues.apache.org/jira/browse/AVRO-3866 Project: Apache Avro Issue Type: Bug Reporter: Kousuke Saruta Similar to AVRO-3865, lang/py/VERSION.txt is leftover after lang/py/build.sh doc -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3863) Delete temporary test data after tests finish
Kousuke Saruta created AVRO-3863:
Summary: Delete temporary test data after tests finish
Key: AVRO-3863
URL: https://issues.apache.org/jira/browse/AVRO-3863
Project: Apache Avro
Issue Type: Improvement
Components: java
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
Tests for Java SDK creates some test data, which are left even after tests
finish.
{code}
ls -1 /tmp/*.avro
/tmp/junit1533190586260098046testMappedByteBuffer.avro
/tmp/junit3099644739767498712testMappedByteBuffer.avro
/tmp/junit4466003251314064556testMappedByteBuffer.avro
/tmp/junit4974226498248565286testMappedByteBuffer.avro
/tmp/junit6473921034404349045testMappedByteBuffer.avro
/tmp/junit8662732084083941415testMappedByteBuffer.avro
/tmp/random.avro
/tmp/testIgnoreSchemaValidationOnRead275054571669736256.avro
/tmp/testIgnoreSchemaValidationOnRead4615547521362396523.avro
/tmp/testIgnoreSchemaValidationOnRead4955268403025511495.avro
/tmp/testIgnoreSchemaValidationOnRead5426593551205571746.avro
/tmp/testIgnoreSchemaValidationOnRead7554021276748093417.avro
/tmp/testIgnoreSchemaValidationOnRead8241302423385070851.avro
/tmp/testInputStreamEOF3549506421974960237.avro
/tmp/testInputStreamEOF4423343183305481378.avro
/tmp/testInputStreamEOF7397178073669402143.avro
/tmp/testInputStreamEOF8065492409408481522.avro
/tmp/testInputStreamEOF8087280538995909098.avro
/tmp/testInputStreamEOF8719004614093216771.avro
/tmp/testInvalidMagicBytes1940432228654910095.avro
/tmp/testInvalidMagicBytes2703760186774533143.avro
/tmp/testInvalidMagicBytes5088097518917799234.avro
/tmp/testInvalidMagicBytes863787801374013591.avro
/tmp/testInvalidMagicBytes887543761182735490.avro
/tmp/testInvalidMagicBytes980334707534164945.avro
/tmp/testInvalidMagicLength1346115615984572207.avro
/tmp/testInvalidMagicLength1511998921770126285.avro
/tmp/testInvalidMagicLength1824057536245960603.avro
/tmp/testInvalidMagicLength2005669502062311523.avro
/tmp/testInvalidMagicLength7068591900276715585.avro
/tmp/testInvalidMagicLength8356756206873381473.avro
/tmp/testThrottledInputStream2962195154373996754.avro
/tmp/testThrottledInputStream3610702487927451328.avro
/tmp/testThrottledInputStream4661398720877824185.avro
/tmp/testThrottledInputStream5592809458916764863.avro
/tmp/testThrottledInputStream648963793454476.avro
/tmp/testThrottledInputStream8013323018361761899.avro
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3862) Add aliases and doc methods to Schema in Rust SDK
Kousuke Saruta created AVRO-3862:
Summary: Add aliases and doc methods to Schema in Rust SDK
Key: AVRO-3862
URL: https://issues.apache.org/jira/browse/AVRO-3862
Project: Apache Avro
Issue Type: Improvement
Components: rust
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
Named types (Record, Enum and Fixed) have common attributes {*}name{*},
*aliases* and {*}doc{*}.
We have already have *fn name* in Schema so it's nice to have *fn aliases* and
*fn doc* too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3857) Automate dist build
Kousuke Saruta created AVRO-3857: Summary: Automate dist build Key: AVRO-3857 URL: https://issues.apache.org/jira/browse/AVRO-3857 Project: Apache Avro Issue Type: Improvement Components: build Affects Versions: 1.12.0 Reporter: Kousuke Saruta Although we can build dist by ./build.sh dist, currently it's not fully automated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3853) Suport Local timestamp logical types for the Rust SDK
Kousuke Saruta created AVRO-3853: Summary: Suport Local timestamp logical types for the Rust SDK Key: AVRO-3853 URL: https://issues.apache.org/jira/browse/AVRO-3853 Project: Apache Avro Issue Type: Improvement Components: rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta Local timestamp logical types are missing in the Rust SDK. It's nice to support them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3851) Validate default value for record fields and enums on parsing
Kousuke Saruta created AVRO-3851: Summary: Validate default value for record fields and enums on parsing Key: AVRO-3851 URL: https://issues.apache.org/jira/browse/AVRO-3851 Project: Apache Avro Issue Type: Improvement Components: rust Reporter: Kousuke Saruta Currently, default values for record fields are not validated on parsing except for union type fields. Similarly, default values for enum are not also validated on parsing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3850) Don't publish Cargo.lock
Kousuke Saruta created AVRO-3850: Summary: Don't publish Cargo.lock Key: AVRO-3850 URL: https://issues.apache.org/jira/browse/AVRO-3850 Project: Apache Avro Issue Type: Improvement Components: rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta Currently, Cargo.lock is published but it should not be because all the crates are libraries. https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3849) make readme doesn't work
Kousuke Saruta created AVRO-3849: Summary: make readme doesn't work Key: AVRO-3849 URL: https://issues.apache.org/jira/browse/AVRO-3849 Project: Apache Avro Issue Type: Bug Components: doc, rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta In the Rust binding, avro/README.md is intended to be created by make readme. But it doesn't work due to some reasons. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3847) Record field doesn't accept default value if field type is union and the type of default value is pre-defined name
Kousuke Saruta created AVRO-3847:
Summary: Record field doesn't accept default value if field type
is union and the type of default value is pre-defined name
Key: AVRO-3847
URL: https://issues.apache.org/jira/browse/AVRO-3847
Project: Apache Avro
Issue Type: Bug
Components: rust
Reporter: Kousuke Saruta
Given we have a schema like as follows.
{code}
{
"name": "record1",
"type": "record",
"fields": [
{
"name": "f1",
"type": {
"name": "record2",
"type": "record",
"fields": [
{
"name": "f1_1",
"type": "int"
}
]
}
}, {
"name": "f2",
"type": ["record2", "int"],
"default": {
"f1_1": 100
}
}
]
}
{code}
The type of the field f2 is union of record2 and int, and the default value is
of a value of record2, which is pre-defined.
Current Rust binding doesn't accept such schemas, raising a error message like
as follows.
{code}
Error: One union type Ref must match the `default`'s value type Map
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3846) Race condition can happen among serde tests
Kousuke Saruta created AVRO-3846: Summary: Race condition can happen among serde tests Key: AVRO-3846 URL: https://issues.apache.org/jira/browse/AVRO-3846 Project: Apache Avro Issue Type: Bug Components: rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta Sometimes one of tests named avro_3747 fails. These tests are run concurrently by Cargo test and those tests load/store the same atomic variable so This seems race condition -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3841) Align the specification of encoding NaN to the actual implementations
Kousuke Saruta created AVRO-3841:
Summary: Align the specification of encoding NaN to the actual
implementations
Key: AVRO-3841
URL: https://issues.apache.org/jira/browse/AVRO-3841
Project: Apache Avro
Issue Type: Bug
Components: spec
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
The specification says about the way to encode float/double like as follows.
{code}
a float is written as 4 bytes. The float is converted into a 32-bit integer
using a method equivalent to Java’s floatToIntBits and then encoded in
little-endian format.
a double is written as 8 bytes. The double is converted into a 64-bit integer
using a method equivalent to Java’s doubleToLongBits and then encoded in
little-endian format.
{code}
But the actual implementation in Java uses
floatToRawIntBits/doubleToRawLongBits rather than
floatToIntBits/doubleToLongBits.
The they are different in the way to encode NaN.
floatToIntBits/doubleToLongBits doesn't distinguish between NaN and -NaN but
floatToRawIntBits/doubleToRawLongBits does.
I confirmed all the implementation distinguish between NaN and -NaN.
So, I think it's better to modify the specification.
Java
{code}
public static int encodeFloat(float f, byte[] buf, int pos) {
final int bits = Float.floatToRawIntBits(f);
buf[pos + 3] = (byte) (bits >>> 24);
buf[pos + 2] = (byte) (bits >>> 16);
buf[pos + 1] = (byte) (bits >>> 8);
buf[pos] = (byte) (bits);
return 4;
}
public static int encodeDouble(double d, byte[] buf, int pos) {
final long bits = Double.doubleToRawLongBits(d);
int first = (int) (bits & 0x);
int second = (int) ((bits >>> 32) & 0x);
// the compiler seems to execute this order the best, likely due to
// register allocation -- the lifetime of constants is minimized.
buf[pos] = (byte) (first);
buf[pos + 4] = (byte) (second);
buf[pos + 5] = (byte) (second >>> 8);
buf[pos + 1] = (byte) (first >>> 8);
buf[pos + 2] = (byte) (first >>> 16);
buf[pos + 6] = (byte) (second >>> 16);
buf[pos + 7] = (byte) (second >>> 24);
buf[pos + 3] = (byte) (first >>> 24);
return 8;
}
{code}
Rust
{code}
Value::Float(x) => buffer.extend_from_slice(_le_bytes()),
Value::Double(x) => buffer.extend_from_slice(_le_bytes()),
{code}
Python
{code}
def write_float(self, datum: float) -> None:
"""
A float is written as 4 bytes.
The float is converted into a 32-bit integer using a method equivalent
to
Java's floatToIntBits and then encoded in little-endian format.
"""
self.write(STRUCT_FLOAT.pack(datum))
def write_double(self, datum: float) -> None:
"""
A double is written as 8 bytes.
The double is converted into a 64-bit integer using a method equivalent
to
Java's doubleToLongBits and then encoded in little-endian format.
"""
self.write(STRUCT_DOUBLE.pack(datum))
{code}
C
{code}
static int write_float(avro_writer_t writer, const float f)
{
#if AVRO_PLATFORM_IS_BIG_ENDIAN
uint8_t buf[4];
#endif
union {
float f;
int32_t i;
} v;
v.f = f;
#if AVRO_PLATFORM_IS_BIG_ENDIAN
buf[0] = (uint8_t) (v.i >> 0);
buf[1] = (uint8_t) (v.i >> 8);
buf[2] = (uint8_t) (v.i >> 16);
buf[3] = (uint8_t) (v.i >> 24);
AVRO_WRITE(writer, buf, 4);
#else
AVRO_WRITE(writer, (void *), 4);
#endif
return 0;
}
static int write_double(avro_writer_t writer, const double d)
{
#if AVRO_PLATFORM_IS_BIG_ENDIAN
uint8_t buf[8];
#endif
union {
[jira] [Created] (AVRO-3837) Disallow invalid namespace for the Rust binding
Kousuke Saruta created AVRO-3837:
Summary: Disallow invalid namespace for the Rust binding
Key: AVRO-3837
URL: https://issues.apache.org/jira/browse/AVRO-3837
Project: Apache Avro
Issue Type: Bug
Components: rust
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
The current Rust binding doesn't accept invalid namespaces if such namespaces
are in name field.
{code}
{
"name": "ns1.invalid-ns.record1",
"type": "record"
"fields": []
}
{code}
But if a invalid namespace in namespace field doesn't validate.
{code}
"name": "record1",
"namespace": "ns1.invalid-ns",
"type": "record",
"fields": []
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3832) Python test doesn't work with Docker
Kousuke Saruta created AVRO-3832: Summary: Python test doesn't work with Docker Key: AVRO-3832 URL: https://issues.apache.org/jira/browse/AVRO-3832 Project: Apache Avro Issue Type: Bug Components: python Affects Versions: 1.12.0 Reporter: Kousuke Saruta Python test doesn't work successfully with ./build.sh docker-test. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3830) Handle namespace property if a name starts with dot
Kousuke Saruta created AVRO-3830:
Summary: Handle namespace property if a name starts with dot
Key: AVRO-3830
URL: https://issues.apache.org/jira/browse/AVRO-3830
Project: Apache Avro
Issue Type: Bug
Reporter: Kousuke Saruta
The specification says about the name and namespace like as follows.
??The empty string may also be used as a namespace to indicate the null
namespace??
??If the name specified contains a dot, then it is assumed to be a fullname,
and any namespace also specified is ignored??
According to this specification, if a name in a name field starts with a dot,
it's considered that the namespace is null and the corresponding namespace
field should be ignored.
For example, given the following schema.
{code}
{
"name": ".record1",
"namespace": "ns1",
"type": "record",
"fields": []
}
{code}
The name and namespace should be "record1" and null respectively.
But the namespace is considered as "ns1" in the current Rust binding .
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3827) Disallow duplicate field names
Kousuke Saruta created AVRO-3827:
Summary: Disallow duplicate field names
Key: AVRO-3827
URL: https://issues.apache.org/jira/browse/AVRO-3827
Project: Apache Avro
Issue Type: Bug
Components: rust
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
If a schema contains a record and some of its fields have the same field name,
such schema should not be allowed.
{code}
{
"name": "my_schema",
"type": "record",
"fields": [
{
"name": "f1",
"type": {
"name": "a",
"type": "record",
"fields": []
}
} {
"name": "f1",
"type": {
"name": "b",
"type": "record",
"fields": []
}
}
]
}
{code}
But the current Rust binding accept.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3825) Disallow invalid namespace
Kousuke Saruta created AVRO-3825:
Summary: Disallow invalid namespace
Key: AVRO-3825
URL: https://issues.apache.org/jira/browse/AVRO-3825
Project: Apache Avro
Issue Type: Bug
Components: java
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
According to the specification, each portion of a namespace separated by dot
should be [a-z,A-Z,_][a-z,A-Z,0-9_].
[https://avro.apache.org/docs/1.11.1/specification/#names]
{code:java}
The name portion of the fullname of named types, record field names, and enum
symbols must:
start with [A-Za-z_]
subsequently contain only [A-Za-z0-9_]
A namespace is a dot-separated sequence of such names. The empty string may
also be used as a namespace to indicate the null namespace. Equality of names
(including field names and enum symbols) as well as fullnames is case-sensitive.
The null namespace may not be used in a dot-separated sequence of names. So the
grammar for a namespace is:
| [()*]
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3824) The instruction for building the website should be more precise
Kousuke Saruta created AVRO-3824: Summary: The instruction for building the website should be more precise Key: AVRO-3824 URL: https://issues.apache.org/jira/browse/AVRO-3824 Project: Apache Avro Issue Type: Bug Components: doc Affects Versions: 1.12.0 Reporter: Kousuke Saruta How to build the website is instructed in doc/README.md but we cannot build even though we follow it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3823) Show helpful error messages
Kousuke Saruta created AVRO-3823:
Summary: Show helpful error messages
Key: AVRO-3823
URL: https://issues.apache.org/jira/browse/AVRO-3823
Project: Apache Avro
Issue Type: Bug
Components: rust
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
The current Rust binding doesn't show helpful error messages.
Actually, error types are implemented with helpful error messages.
This is an example.
{code:java}
#[error("No `name` field")]
GetNameField,
{code}
But those error messages are not shown.
Given we try to a invalid schema which contains no name field, we expect to get
"No `name` field" but the actual is "GetNameFIeld", which makes it difficult
for users to resolve the problem.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3820) Don't allow invalid field names
Kousuke Saruta created AVRO-3820:
Summary: Don't allow invalid field names
Key: AVRO-3820
URL: https://issues.apache.org/jira/browse/AVRO-3820
Project: Apache Avro
Issue Type: Bug
Components: rust
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
Given we have a schema where a field name doesn't match [A-Za-z_][A-Za-z0-9_]*
like f1.x.
{code}
{
"name": "my_record",
"type": "record",
"fields": [
{
"name": "f1.x",
"type": {
"name": "my_enum",
"type": "enum",
"symbols": ["a"]
}
}
]
}
{code}
The current Rust binding accepts such a schema but it seems illegal.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3818) Enclosing namespace should be inherited to the inner named types if they have no their own namespaces
Kousuke Saruta created AVRO-3818:
Summary: Enclosing namespace should be inherited to the inner
named types if they have no their own namespaces
Key: AVRO-3818
URL: https://issues.apache.org/jira/browse/AVRO-3818
Project: Apache Avro
Issue Type: Bug
Components: rust
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
Given we have a schema like as follows.
{code}
{
"namespace": "my_ns",
"type": "record",
"aliases": ["my_alias"],
"name": "my_schema",
"fields": [
{
"name": "f1",
"type": {
"name": "enum1",
"type": "enum",
"symbols": ["a"]
}
}
]
}
{code}
The canonical form of this schema should be
{code}
{"name":"my_ns.my_schema","type":"record","fields":[{"name":"f1","type":{"name":"my_ns.enum1","type":"enum","symbols":["a"]}}]}
{code}
But with the current Rust binding, enum1 doesn't inherit the enclosing
namespace "my_ns".
{code}
{"name":"my_ns.my_schema","type":"record","fields":[{"name":"f1","type":{"name":"enum1","type":"enum","symbols":["a"]}}]}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3815) Broken indentation in the specification doc
Kousuke Saruta created AVRO-3815: Summary: Broken indentation in the specification doc Key: AVRO-3815 URL: https://issues.apache.org/jira/browse/AVRO-3815 Project: Apache Avro Issue Type: Improvement Components: doc Affects Versions: 1.12.0 Reporter: Kousuke Saruta Attachments: complex-types-layout.png See the attachment. The items _order_ and _aliases_ below the table should be one more deeper indentation level. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3812) Handle null namespace properly
Kousuke Saruta created AVRO-3812:
Summary: Handle null namespace properly
Key: AVRO-3812
URL: https://issues.apache.org/jira/browse/AVRO-3812
Project: Apache Avro
Issue Type: Improvement
Components: rust
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
Considering the following schema, which contains namespaces of "".
{code}
{
"namespace": "",
"type": "record",
"name": "my_schema",
"fields": [
{
"name": "a",
"type": {
"type": "enum",
"name": "my_enum",
"namespace": "",
"symbols": ["a", "b"]
}
}, {
"name": "b",
"type": {
"type": "fixed",
"name": "my_fixed",
"namespace": "",
"size": 10
}
}
]
}
{code}
If we try to canonicalize this schema with the following code
{code}
let schema = Schema::parse_str(schema_str).unwrap().canonical_form();
println!("{schema}");
{code}
We get the following result.
{code}
{"name":".my_schema","type":"record","fields":[{"name":"a","type":{"name":".my_enum","type":"enum","symbols":["a","b"]}},{"name":"b","type":{"name":".my_fixed","type":"fixed","size":10}}]}
{code}
But .my_schema, .my_enum and .my_fixed should not starts with a dot.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3811) Mention about xz and zstandard as default registerd codecs in JavaDoc
Kousuke Saruta created AVRO-3811:
Summary: Mention about xz and zstandard as default registerd
codecs in JavaDoc
Key: AVRO-3811
URL: https://issues.apache.org/jira/browse/AVRO-3811
Project: Apache Avro
Issue Type: Improvement
Components: doc, java
Affects Versions: 1.12.0
Reporter: Kousuke Saruta
The JavaDoc for CodecFactory says
{code}
* Currently there are three codecs registered by default:
*
* {@code null}
* {@code deflate}
* {@code snappy}
* {@code bzip2}
*
{code}
But xz and zstandard are also registered by default.
{code}
static {
addCodec(DataFileConstants.NULL_CODEC, nullCodec());
addCodec(DataFileConstants.DEFLATE_CODEC,
deflateCodec(DEFAULT_DEFLATE_LEVEL));
addCodec(DataFileConstants.BZIP2_CODEC, bzip2Codec());
addCodec(DataFileConstants.XZ_CODEC, xzCodec(DEFAULT_XZ_LEVEL));
addCodec(DataFileConstants.ZSTANDARD_CODEC,
zstandardCodec(DEFAULT_ZSTANDARD_LEVEL, DEFAULT_ZSTANDARD_BUFFERPOOL));
addCodec(DataFileConstants.SNAPPY_CODEC, snappyCodec());
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (AVRO-3800) profile section should be declared in the root package.
Kousuke Saruta created AVRO-3800: Summary: profile section should be declared in the root package. Key: AVRO-3800 URL: https://issues.apache.org/jira/browse/AVRO-3800 Project: Apache Avro Issue Type: Bug Components: build, rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta In wasm-demo/Cargo.toml, [profile.release] is declared but it's ignored. It should be declared in Cargo.toml of root package. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3799) Enable the schema parser to read and parse from input streams for Rust binding
Kousuke Saruta created AVRO-3799: Summary: Enable the schema parser to read and parse from input streams for Rust binding Key: AVRO-3799 URL: https://issues.apache.org/jira/browse/AVRO-3799 Project: Apache Avro Issue Type: Improvement Components: rust Affects Versions: 1.12.0 Reporter: Kousuke Saruta The Java binding supports reading and parsing schema from input streams. So, it's great if the feature is supported for the Rust binding too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (AVRO-3741) Note about the version requirement of Rust in BUILD.md
Kousuke Saruta created AVRO-3741: Summary: Note about the version requirement of Rust in BUILD.md Key: AVRO-3741 URL: https://issues.apache.org/jira/browse/AVRO-3741 Project: Apache Avro Issue Type: Improvement Components: doc Affects Versions: 1.12.0 Reporter: Kousuke Saruta The requirement version of Rust is not noted in BUILD.md. It's better to note like other languages. -- This message was sent by Atlassian Jira (v8.20.10#820010)
