[jira] [Updated] (AVRO-3972) [Build] pypy3.8 fails with 'Buffer' object is not iterable
[
https://issues.apache.org/jira/browse/AVRO-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3972:
--
Priority: Blocker (was: Major)
> [Build] pypy3.8 fails with 'Buffer' object is not iterable
> --
>
> Key: AVRO-3972
> URL: https://issues.apache.org/jira/browse/AVRO-3972
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.12.0
>Reporter: Ryan Skraba
>Priority: Blocker
>
> In the docker ubertool:
> {code}
> interop_data.py 46 296%
> avro/test/mock_tether_parent.py 48 3038%
> avro/test/sample_http_client.py 30 30 0%
> avro/test/sample_http_server.py 34 34 0%
> avro/test/test_bench.py 42 0 100%
> avro/test/test_compatibility.py 161 0 100%
> avro/test/test_datafile.py85 0 100%
> avro/test/test_datafile_interop.py28 389%
> avro/test/test_init.py 5 0 100%
> avro/test/test_io.py 212 0 100%
> avro/test/test_ipc.py 11 0 100%
> avro/test/test_name.py95 0 100%
> avro/test/test_protocol.py74 0 100%
> avro/test/test_schema.py 284 499%
> avro/test/test_script.py 139 299%
> avro/test/test_tether_task.py 45 198%
> avro/test/test_tether_task_runner.py 68 0 100%
> avro/test/test_tether_word_count.py 67 199%
> avro/test/word_count_task.py 21 0 100%
> avro/tether/__init__.py4 0 100%
> avro/tether/tether_task.py 159 3876%
> avro/tether/tether_task_runner.py118 3769%
> avro/tether/util.py7 0 100%
> avro/timezones.py 18 383%
> avro/tool.py 108108 0%
> avro/utils.py 10 0 100%
> --
> TOTAL 449368685%
> py311: OK ✔ in 20.39 seconds
> pypy3.7: skipped because could not find python interpreter with spec(s):
> pypy3.7
> pypy3.7: SKIP ⚠ in 6.58 seconds
> pypy3.8: install_deps> python -I -m pip install coverage python-snappy
> zstandard
> pypy3.8: install_package_deps> python -I -m pip install 'typing-extensions;
> python_version < "3.8"'
> pypy3.8: install_package> python -I -m pip install --force-reinstall
> --no-deps
> /home/ryan.skraba/avro/lang/py/.tox/.tmp/package/8/avro-1.12.0+snapshot.tar.gz
> pypy3.8: commands_pre[0]> mkdir -p avro/test/interop
> /home/ryan.skraba/avro/lang/py/../../build/interop/data
> pypy3.8: commands_pre[1]> cp -r
> /home/ryan.skraba/avro/lang/py/../../build/interop/data avro/test/interop
> pypy3.8: commands_pre[2]> coverage run -pm avro.test.gen_interop_data
> avro/interop.avsc avro/test/interop/data/py.avro
> Traceback (most recent call last):
> File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line
> 103, in
> raise SystemExit(main())
> File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line
> 98, in main
> generate(args.schema_path, op)
> File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line
> 71, in generate
> for codec, data in output:
> File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line
> 67, in
> output = ((codec, gen_data(codec, datum_writer, interop_schema)) for
> codec in CODECS_TO_VALIDATE)
> File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line
> 60, in gen_data
> dfw.flush()
> File "/home/ryan.skraba/avro/lang/py/avro/datafile.py", line 277, in flush
> self._write_block()
> File "/home/ryan.skraba/avro/lang/py/avro/datafile.py", line 241, in
> _write_block
> compressed_data, compressed_data_length =
> codec.compress(uncompressed_data)
> File "/home/ryan.skraba/avro/lang/py/avro/codecs.py", line 151, in compress
> compressed_data = snappy.compress(data)
> File
> "/home/ryan.skraba/avro/lang/py/.tox/pypy3.8/lib/pypy3.8/site-packages/snappy/snappy.py",
> line 78, in compress
> return bytes(_compress(data))
> TypeError: 'Buffer' object is not iterable
> pypy3.8: exit 1 (1.00 seconds) /home/ryan.skraba/avro/lang/py> coverage run
> -pm avro.test.gen_interop_data avro/interop.avsc
> avro/test/interop/data/py.avro pid=34269
> pypy3.8: commands_post[0]> coverage combine --append
> Combined data file .coverage.4fcb8e11055e.34269.XLQAYhgx
> pypy3.8: commands_post[1]> coverage report
> Name Stmts Miss Cover
> --
>
[jira] [Updated] (AVRO-3972) [Build] pypy3.8 fails with 'Buffer' object is not iterable
[
https://issues.apache.org/jira/browse/AVRO-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3972:
--
Description:
In the docker ubertool:
{code}
interop_data.py 46 296%
avro/test/mock_tether_parent.py 48 3038%
avro/test/sample_http_client.py 30 30 0%
avro/test/sample_http_server.py 34 34 0%
avro/test/test_bench.py 42 0 100%
avro/test/test_compatibility.py 161 0 100%
avro/test/test_datafile.py85 0 100%
avro/test/test_datafile_interop.py28 389%
avro/test/test_init.py 5 0 100%
avro/test/test_io.py 212 0 100%
avro/test/test_ipc.py 11 0 100%
avro/test/test_name.py95 0 100%
avro/test/test_protocol.py74 0 100%
avro/test/test_schema.py 284 499%
avro/test/test_script.py 139 299%
avro/test/test_tether_task.py 45 198%
avro/test/test_tether_task_runner.py 68 0 100%
avro/test/test_tether_word_count.py 67 199%
avro/test/word_count_task.py 21 0 100%
avro/tether/__init__.py4 0 100%
avro/tether/tether_task.py 159 3876%
avro/tether/tether_task_runner.py118 3769%
avro/tether/util.py7 0 100%
avro/timezones.py 18 383%
avro/tool.py 108108 0%
avro/utils.py 10 0 100%
--
TOTAL 449368685%
py311: OK ✔ in 20.39 seconds
pypy3.7: skipped because could not find python interpreter with spec(s): pypy3.7
pypy3.7: SKIP ⚠ in 6.58 seconds
pypy3.8: install_deps> python -I -m pip install coverage python-snappy zstandard
pypy3.8: install_package_deps> python -I -m pip install 'typing-extensions;
python_version < "3.8"'
pypy3.8: install_package> python -I -m pip install --force-reinstall --no-deps
/home/ryan.skraba/avro/lang/py/.tox/.tmp/package/8/avro-1.12.0+snapshot.tar.gz
pypy3.8: commands_pre[0]> mkdir -p avro/test/interop
/home/ryan.skraba/avro/lang/py/../../build/interop/data
pypy3.8: commands_pre[1]> cp -r
/home/ryan.skraba/avro/lang/py/../../build/interop/data avro/test/interop
pypy3.8: commands_pre[2]> coverage run -pm avro.test.gen_interop_data
avro/interop.avsc avro/test/interop/data/py.avro
Traceback (most recent call last):
File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line
103, in
raise SystemExit(main())
File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line 98,
in main
generate(args.schema_path, op)
File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line 71,
in generate
for codec, data in output:
File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line 67,
in
output = ((codec, gen_data(codec, datum_writer, interop_schema)) for codec
in CODECS_TO_VALIDATE)
File "/home/ryan.skraba/avro/lang/py/avro/test/gen_interop_data.py", line 60,
in gen_data
dfw.flush()
File "/home/ryan.skraba/avro/lang/py/avro/datafile.py", line 277, in flush
self._write_block()
File "/home/ryan.skraba/avro/lang/py/avro/datafile.py", line 241, in
_write_block
compressed_data, compressed_data_length = codec.compress(uncompressed_data)
File "/home/ryan.skraba/avro/lang/py/avro/codecs.py", line 151, in compress
compressed_data = snappy.compress(data)
File
"/home/ryan.skraba/avro/lang/py/.tox/pypy3.8/lib/pypy3.8/site-packages/snappy/snappy.py",
line 78, in compress
return bytes(_compress(data))
TypeError: 'Buffer' object is not iterable
pypy3.8: exit 1 (1.00 seconds) /home/ryan.skraba/avro/lang/py> coverage run -pm
avro.test.gen_interop_data avro/interop.avsc avro/test/interop/data/py.avro
pid=34269
pypy3.8: commands_post[0]> coverage combine --append
Combined data file .coverage.4fcb8e11055e.34269.XLQAYhgx
pypy3.8: commands_post[1]> coverage report
Name Stmts Miss Cover
--
avro/__init__.py 3 0 100%
avro/__main__.py 143143 0%
avro/codecs.py 104 595%
avro/compatibility.py208 896%
avro/constants.py 12 0 100%
avro/datafile.py 227 1096%
avro/errors.py43 491%
avro/io.py 65011283%
avro/ipc.py 309 5881%
avro/name.py 74
[jira] [Updated] (AVRO-3908) Update project logo everywhere
[ https://issues.apache.org/jira/browse/AVRO-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3908: -- Description: With AVRO-3554, we now have a new project logo. We should update the logo online in various locations (please add if needed): * Avro Website: [PR #2594|https://github.com/apache/avro/pull/2594] (linked to AVRO-3554) * (/) Apache Project Logos: AVRO-3907 * (/) Wikipedia: [https://en.wikipedia.org/wiki/Apache_Avro] * (/) Wikimedia: [https://commons.wikimedia.org/wiki/Category:Apache_Avro_logos] * (/) ApacheAvro twitter: [https://x.com/ApacheAvro] was: With AVRO-3554, we now have a new project logo. We should update the logo online in various locations (please add if needed): * Avro Website: [PR #2594|https://github.com/apache/avro/pull/2594] (linked to AVRO-3554) * Apache Project Logos: AVRO-3907 * (/) Wikipedia: [https://en.wikipedia.org/wiki/Apache_Avro] * (/) Wikimedia: [https://commons.wikimedia.org/wiki/Category:Apache_Avro_logos] * (/) ApacheAvro twitter: [https://x.com/ApacheAvro] > Update project logo everywhere > -- > > Key: AVRO-3908 > URL: https://issues.apache.org/jira/browse/AVRO-3908 > Project: Apache Avro > Issue Type: Improvement >Reporter: Oscar Westra van Holthe - Kind >Assignee: Oscar Westra van Holthe - Kind >Priority: Major > > With AVRO-3554, we now have a new project logo. We should update the logo > online in various locations (please add if needed): > * Avro Website: [PR #2594|https://github.com/apache/avro/pull/2594] (linked > to AVRO-3554) > * (/) Apache Project Logos: AVRO-3907 > * (/) Wikipedia: [https://en.wikipedia.org/wiki/Apache_Avro] > * (/) Wikimedia: > [https://commons.wikimedia.org/wiki/Category:Apache_Avro_logos] > * (/) ApacheAvro twitter: [https://x.com/ApacheAvro] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (AVRO-3907) Update project logo on https://www.apache.org/logos/
[ https://issues.apache.org/jira/browse/AVRO-3907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba resolved AVRO-3907. --- Resolution: Fixed Thanks so much – apparently any committer can upload to that "special" SVN location, they're all set to the new version! > Update project logo on https://www.apache.org/logos/ > > > Key: AVRO-3907 > URL: https://issues.apache.org/jira/browse/AVRO-3907 > Project: Apache Avro > Issue Type: Improvement >Reporter: Oscar Westra van Holthe - Kind >Assignee: Emma Kellam >Priority: Major > Attachments: avro-1.svg, avro-2.svg, avro-3.svg, avro-4.svg, > avro-5.svg, avro-pb-2.svg, avro-pb.svg > > > When completing the work for AVRO-3554 by merging [PR > #2594|https://github.com/apache/avro/pull/2594], we should also update the > logo on the Apache Project logos list: https://www.apache.org/logos/ > Attached are the new logos, replacing the old one. I've created an explicit > "powered by" logo, as the default won't do. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3908) Update project logo everywhere
[ https://issues.apache.org/jira/browse/AVRO-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3908: -- Description: With AVRO-3554, we now have a new project logo. We should update the logo online in various locations (please add if needed): * Avro Website: [PR #2594|https://github.com/apache/avro/pull/2594] (linked to AVRO-3554) * Apache Project Logos: AVRO-3907 * (/) Wikipedia: [https://en.wikipedia.org/wiki/Apache_Avro] * (/) Wikimedia: [https://commons.wikimedia.org/wiki/Category:Apache_Avro_logos] * (/) ApacheAvro twitter: [https://x.com/ApacheAvro] was: With AVRO-3554, we now have a new project logo. We should update the logo online in various locations (please add if needed): * Avro Website: [PR #2594|https://github.com/apache/avro/pull/2594] (linked to AVRO-3554) * Apache Project Logos: AVRO-3907 * (/) Wikipedia: [https://en.wikipedia.org/wiki/Apache_Avro] * (/) Wikimedia: [https://commons.wikimedia.org/wiki/Category:Apache_Avro_logos] > Update project logo everywhere > -- > > Key: AVRO-3908 > URL: https://issues.apache.org/jira/browse/AVRO-3908 > Project: Apache Avro > Issue Type: Improvement >Reporter: Oscar Westra van Holthe - Kind >Assignee: Oscar Westra van Holthe - Kind >Priority: Major > > With AVRO-3554, we now have a new project logo. We should update the logo > online in various locations (please add if needed): > * Avro Website: [PR #2594|https://github.com/apache/avro/pull/2594] (linked > to AVRO-3554) > * Apache Project Logos: AVRO-3907 > * (/) Wikipedia: [https://en.wikipedia.org/wiki/Apache_Avro] > * (/) Wikimedia: > [https://commons.wikimedia.org/wiki/Category:Apache_Avro_logos] > * (/) ApacheAvro twitter: [https://x.com/ApacheAvro] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (AVRO-3885) Update the maillist link
[ https://issues.apache.org/jira/browse/AVRO-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba resolved AVRO-3885. --- Resolution: Fixed > Update the maillist link > > > Key: AVRO-3885 > URL: https://issues.apache.org/jira/browse/AVRO-3885 > Project: Apache Avro > Issue Type: Improvement >Affects Versions: 1.11.3 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (AVRO-3885) Update the maillist link
[ https://issues.apache.org/jira/browse/AVRO-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba closed AVRO-3885. - > Update the maillist link > > > Key: AVRO-3885 > URL: https://issues.apache.org/jira/browse/AVRO-3885 > Project: Apache Avro > Issue Type: Improvement >Affects Versions: 1.11.3 >Reporter: Fokko Driesprong >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0 > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (AVRO-3554) Create original art for the Avro logo
[ https://issues.apache.org/jira/browse/AVRO-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788455#comment-17788455 ] Ryan Skraba commented on AVRO-3554: --- Hello [~opwvhk]! I saw the PR for the website! Can you create a separate "umbrella" JIRA for logo updates? We should start keeping track of the different places that the logo needs to be updated! Wikipedia comes to mind. > Create original art for the Avro logo > - > > Key: AVRO-3554 > URL: https://issues.apache.org/jira/browse/AVRO-3554 > Project: Apache Avro > Issue Type: Improvement >Reporter: Ryan Skraba >Assignee: Emma Kellam >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0 > > Attachments: Apache-Avro-Logo-Finalists.png, > Apache-Avro-Logo-Ideas.pdf, Apache-Avro-Logo-Ideas.png, > Apache-Avro-Logo-Revisions-V1.pdf, Apache-Avro-Logo-Revisions-V2.png, > Apache-Avro-Logos-1.png, Apache-Avro-Logos-2.png, OldAsmLogo.png, > adjusted-upwards-avro-logo.png, avro-logo-v9.2-adjusted-to-upward-angle.png, > cuttlefish oscar 2.svg, cuttlefish oscar.svg > > Time Spent: 10m > Remaining Estimate: 0h > > There was a quick discussion with Apache Trademarks along the lines "If it > ever came to a legal challenge, would we care enough to defend our usage? If > not, changing it discreetly and voluntarily is the best route." > The result: we _must_ change our logo this year to some original art. > > Criteria for a new logo, as discussed on the mailing list: > [https://lists.apache.org/thread/m5w7fmkjnrl8m4hlbw8xhzr4v69xg3ml]: > Must: > * be unique and not similar to anything “out there” > * be available in a vector format (SVG) so it can be scaled with full > accuracy > Should: > * show either something of the name (like Beam, Airflow, Ant, Drill) or what > it does (like Mahout, Bookkeeper, Ratis, Pulsar) > * have 2 variants: Square/Circle (social media, favicon, etc) and rectangle > (top banner) > Could: > * have the banner logo be the compact logo with the name attached, or more > elaborate > Would: > * have limited colors; at most 5-10 (so like Iceberg, not like Flex with the > gradients) > > _To allow sufficient artistic freedom, we should limit new requirements. > Instead, please add ideas in the comments._ > > See [https://www.apache.org/logos/] for many examples -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (AVRO-3888) CVE with common compress
[ https://issues.apache.org/jira/browse/AVRO-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778980#comment-17778980 ] Ryan Skraba commented on AVRO-3888: --- Cherry-picked to [branch-1.11|https://github.com/apache/avro/commit/9480d15f96b1da4fa936d2c52ffccb1fbc79ef23]. > CVE with common compress > > > Key: AVRO-3888 > URL: https://issues.apache.org/jira/browse/AVRO-3888 > Project: Apache Avro > Issue Type: Bug >Reporter: Christophe Le Saec >Assignee: Christophe Le Saec >Priority: Major > Fix For: 1.12.0, 1.11.4 > > > Avoid [CVE from common > compress|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-42503] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3888) CVE with common compress
[ https://issues.apache.org/jira/browse/AVRO-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3888: -- Fix Version/s: 1.12.0 1.11.4 > CVE with common compress > > > Key: AVRO-3888 > URL: https://issues.apache.org/jira/browse/AVRO-3888 > Project: Apache Avro > Issue Type: Bug >Reporter: Christophe Le Saec >Assignee: Christophe Le Saec >Priority: Major > Fix For: 1.12.0, 1.11.4 > > > Avoid [CVE from common > compress|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-42503] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (AVRO-3863) Delete temporary test data after tests finish
[
https://issues.apache.org/jira/browse/AVRO-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3863.
---
Fix Version/s: 1.12.0
Assignee: Kousuke Saruta
Resolution: Fixed
> Delete temporary test data after tests finish
> -
>
> Key: AVRO-3863
> URL: https://issues.apache.org/jira/browse/AVRO-3863
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.12.0
>Reporter: Kousuke Saruta
>Assignee: Kousuke Saruta
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Tests for Java SDK creates some test data, which are left even after tests
> finish.
> {code}
> ls -1 /tmp/*.avro
> /tmp/junit1533190586260098046testMappedByteBuffer.avro
> /tmp/junit3099644739767498712testMappedByteBuffer.avro
> /tmp/junit4466003251314064556testMappedByteBuffer.avro
> /tmp/junit4974226498248565286testMappedByteBuffer.avro
> /tmp/junit6473921034404349045testMappedByteBuffer.avro
> /tmp/junit8662732084083941415testMappedByteBuffer.avro
> /tmp/random.avro
> /tmp/testIgnoreSchemaValidationOnRead275054571669736256.avro
> /tmp/testIgnoreSchemaValidationOnRead4615547521362396523.avro
> /tmp/testIgnoreSchemaValidationOnRead4955268403025511495.avro
> /tmp/testIgnoreSchemaValidationOnRead5426593551205571746.avro
> /tmp/testIgnoreSchemaValidationOnRead7554021276748093417.avro
> /tmp/testIgnoreSchemaValidationOnRead8241302423385070851.avro
> /tmp/testInputStreamEOF3549506421974960237.avro
> /tmp/testInputStreamEOF4423343183305481378.avro
> /tmp/testInputStreamEOF7397178073669402143.avro
> /tmp/testInputStreamEOF8065492409408481522.avro
> /tmp/testInputStreamEOF8087280538995909098.avro
> /tmp/testInputStreamEOF8719004614093216771.avro
> /tmp/testInvalidMagicBytes1940432228654910095.avro
> /tmp/testInvalidMagicBytes2703760186774533143.avro
> /tmp/testInvalidMagicBytes5088097518917799234.avro
> /tmp/testInvalidMagicBytes863787801374013591.avro
> /tmp/testInvalidMagicBytes887543761182735490.avro
> /tmp/testInvalidMagicBytes980334707534164945.avro
> /tmp/testInvalidMagicLength1346115615984572207.avro
> /tmp/testInvalidMagicLength1511998921770126285.avro
> /tmp/testInvalidMagicLength1824057536245960603.avro
> /tmp/testInvalidMagicLength2005669502062311523.avro
> /tmp/testInvalidMagicLength7068591900276715585.avro
> /tmp/testInvalidMagicLength8356756206873381473.avro
> /tmp/testThrottledInputStream2962195154373996754.avro
> /tmp/testThrottledInputStream3610702487927451328.avro
> /tmp/testThrottledInputStream4661398720877824185.avro
> /tmp/testThrottledInputStream5592809458916764863.avro
> /tmp/testThrottledInputStream648963793454476.avro
> /tmp/testThrottledInputStream8013323018361761899.avro
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (AVRO-3872) [Build][C#] Warning on nuget upload about README
[
https://issues.apache.org/jira/browse/AVRO-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3872.
---
Fix Version/s: 1.12.0
1.11.4
Resolution: Fixed
> [Build][C#] Warning on nuget upload about README
>
>
> Key: AVRO-3872
> URL: https://issues.apache.org/jira/browse/AVRO-3872
> Project: Apache Avro
> Issue Type: New Feature
>Reporter: Ryan Skraba
>Assignee: Zoltan Csizmadia
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.4
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> There's already a C# specific description for the Avro package in nuget:
> https://www.nuget.org/packages/Apache.Avro
> For some reason, publishing the artifact has a warning about the missing
> readme:
> {code}
> --
> Pushing Apache.Avro.1.11.3.nupkg to 'https://www.nuget.org/api/v2/package'...
> PUT https://www.nuget.org/api/v2/package/
> warn : Readme missing. Go to https://aka.ms/nuget-include-readme learn How to
> include a readme file within the package.
> Created https://www.nuget.org/api/v2/package/ 1134ms
> Your package was pushed.
> {code}
> This doesn't block anything. Is there an action we should take?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (AVRO-3872) [Build][C#] Warning on nuget upload about README
[
https://issues.apache.org/jira/browse/AVRO-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba reassigned AVRO-3872:
-
Assignee: Zoltan Csizmadia
> [Build][C#] Warning on nuget upload about README
>
>
> Key: AVRO-3872
> URL: https://issues.apache.org/jira/browse/AVRO-3872
> Project: Apache Avro
> Issue Type: New Feature
>Reporter: Ryan Skraba
>Assignee: Zoltan Csizmadia
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> There's already a C# specific description for the Avro package in nuget:
> https://www.nuget.org/packages/Apache.Avro
> For some reason, publishing the artifact has a warning about the missing
> readme:
> {code}
> --
> Pushing Apache.Avro.1.11.3.nupkg to 'https://www.nuget.org/api/v2/package'...
> PUT https://www.nuget.org/api/v2/package/
> warn : Readme missing. Go to https://aka.ms/nuget-include-readme learn How to
> include a readme file within the package.
> Created https://www.nuget.org/api/v2/package/ 1134ms
> Your package was pushed.
> {code}
> This doesn't block anything. Is there an action we should take?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3873) [Build][Java] use an avro.version property to help releases
[
https://issues.apache.org/jira/browse/AVRO-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3873:
--
Priority: Minor (was: Major)
> [Build][Java] use an avro.version property to help releases
> ---
>
> Key: AVRO-3873
> URL: https://issues.apache.org/jira/browse/AVRO-3873
> Project: Apache Avro
> Issue Type: New Feature
>Reporter: Ryan Skraba
>Priority: Minor
>
> The current release process requires updating a number of files. While many
> of them are read from {{share/VERSION.txt}}, some of them need to be embedded
> directly as strings.
> It would be easier and more reliable to script the update process in the Java
> pom.xml files if they were defined as properties. This is already the case
> in our [avro
> archetypes|https://github.com/apache/avro/blob/d75abd591540ef34a334a3d73314f2748c8d868d/lang/java/archetypes/avro-service-archetype/src/main/pom/pom.xml#L36].
> The following pom files should be updated to define and use ${avro.version}:
> *
> lang/java/maven-plugin/src/test/resources/unit/idl/pom-injecting-velocity-tools.xml
> * lang/java/maven-plugin/src/test/resources/unit/idl/pom.xml
> *
> lang/java/maven-plugin/src/test/resources/unit/protocol/pom-injecting-velocity-tools.xml
> * lang/java/maven-plugin/src/test/resources/unit/protocol/pom.xml
> *
> lang/java/maven-plugin/src/test/resources/unit/schema/pom-nonexistent-file.xml
> *
> lang/java/maven-plugin/src/test/resources/unit/schema/pom-nonexistent-second-file.xml
> * lang/java/maven-plugin/src/test/resources/unit/schema/pom.xml
> * doc/examples/java-example/pom.xml
> * doc/examples/mr-example/pom.xml
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (AVRO-3859) [Build][C#] build.sh clean fails to remove some C# files
[ https://issues.apache.org/jira/browse/AVRO-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-3859: - Assignee: Zoltan Csizmadia > [Build][C#] build.sh clean fails to remove some C# files > > > Key: AVRO-3859 > URL: https://issues.apache.org/jira/browse/AVRO-3859 > Project: Apache Avro > Issue Type: Bug >Reporter: Ryan Skraba >Assignee: Zoltan Csizmadia >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.4 > > Time Spent: 10m > Remaining Estimate: 0h > > Some of the obj/ and bin/ directories should be removed with a *./build.sh > clean* in the C# SDK: > * lang/csharp/src/apache/benchmark/obj/ > * lang/csharp/src/apache/codec/Avro.File.BZip2.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.BZip2/bin/ > * lang/csharp/src/apache/codec/Avro.File.BZip2/obj/ > * lang/csharp/src/apache/codec/Avro.File.Snappy.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.Snappy/bin/ > * lang/csharp/src/apache/codec/Avro.File.Snappy/obj/ > * lang/csharp/src/apache/codec/Avro.File.XZ.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.XZ/bin/ > * lang/csharp/src/apache/codec/Avro.File.XZ/obj/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard/bin/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard/obj/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3859) [Build][C#] build.sh clean fails to remove some C# files
[ https://issues.apache.org/jira/browse/AVRO-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3859: -- Fix Version/s: 1.11.4 1.12.0 > [Build][C#] build.sh clean fails to remove some C# files > > > Key: AVRO-3859 > URL: https://issues.apache.org/jira/browse/AVRO-3859 > Project: Apache Avro > Issue Type: Bug >Reporter: Ryan Skraba >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.4 > > Time Spent: 10m > Remaining Estimate: 0h > > Some of the obj/ and bin/ directories should be removed with a *./build.sh > clean* in the C# SDK: > * lang/csharp/src/apache/benchmark/obj/ > * lang/csharp/src/apache/codec/Avro.File.BZip2.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.BZip2/bin/ > * lang/csharp/src/apache/codec/Avro.File.BZip2/obj/ > * lang/csharp/src/apache/codec/Avro.File.Snappy.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.Snappy/bin/ > * lang/csharp/src/apache/codec/Avro.File.Snappy/obj/ > * lang/csharp/src/apache/codec/Avro.File.XZ.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.XZ/bin/ > * lang/csharp/src/apache/codec/Avro.File.XZ/obj/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard/bin/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard/obj/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (AVRO-3859) [Build][C#] build.sh clean fails to remove some C# files
[ https://issues.apache.org/jira/browse/AVRO-3859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba resolved AVRO-3859. --- Resolution: Fixed > [Build][C#] build.sh clean fails to remove some C# files > > > Key: AVRO-3859 > URL: https://issues.apache.org/jira/browse/AVRO-3859 > Project: Apache Avro > Issue Type: Bug >Reporter: Ryan Skraba >Assignee: Zoltan Csizmadia >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.4 > > Time Spent: 10m > Remaining Estimate: 0h > > Some of the obj/ and bin/ directories should be removed with a *./build.sh > clean* in the C# SDK: > * lang/csharp/src/apache/benchmark/obj/ > * lang/csharp/src/apache/codec/Avro.File.BZip2.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.BZip2/bin/ > * lang/csharp/src/apache/codec/Avro.File.BZip2/obj/ > * lang/csharp/src/apache/codec/Avro.File.Snappy.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.Snappy/bin/ > * lang/csharp/src/apache/codec/Avro.File.Snappy/obj/ > * lang/csharp/src/apache/codec/Avro.File.XZ.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.XZ/bin/ > * lang/csharp/src/apache/codec/Avro.File.XZ/obj/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard.Test/obj/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard/bin/ > * lang/csharp/src/apache/codec/Avro.File.Zstandard/obj/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (AVRO-3554) Create original art for the Avro logo
[ https://issues.apache.org/jira/browse/AVRO-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766915#comment-17766915 ] Ryan Skraba commented on AVRO-3554: --- Hey -- this is all amazing work! I can see the A on the tail of the V10s -- it gets a bonus bump :D I prefer without the block outline on the plane, so v10.2 and v9.2 but I like the block outline on the pigeon v6.1 Haha, I really don't see the blob or splatters as data-loss. It *does* feel like the plane is disintegrating, but that's what Avro does! (Well, temporarily). I checked and the [PT Mono|https://www.paratype.com/fonts/pt/pt-mono/regular] font is distributed under the SIL Open Font License (OFL1.1) and doesn't require attribution [but it's a nice thing to do|https://scripts.sil.org/cms/scripts/page.php?item_id=OFL-FAQ_web#e71fabc0]. I think it looks great! > Create original art for the Avro logo > - > > Key: AVRO-3554 > URL: https://issues.apache.org/jira/browse/AVRO-3554 > Project: Apache Avro > Issue Type: Improvement >Reporter: Ryan Skraba >Assignee: Emma Kellam >Priority: Major > Fix For: 1.12.0 > > Attachments: Apache-Avro-Logo-Ideas.pdf, Apache-Avro-Logo-Ideas.png, > Apache-Avro-Logo-Revisions-V1.pdf, Apache-Avro-Logo-Revisions-V2.png, > Apache-Avro-Logos-1.png, Apache-Avro-Logos-2.png, OldAsmLogo.png, cuttlefish > oscar 2.svg, cuttlefish oscar.svg > > > There was a quick discussion with Apache Trademarks along the lines "If it > ever came to a legal challenge, would we care enough to defend our usage? If > not, changing it discreetly and voluntarily is the best route." > The result: we _must_ change our logo this year to some original art. > > Criteria for a new logo, as discussed on the mailing list: > [https://lists.apache.org/thread/m5w7fmkjnrl8m4hlbw8xhzr4v69xg3ml]: > Must: > * be unique and not similar to anything “out there” > * be available in a vector format (SVG) so it can be scaled with full > accuracy > Should: > * show either something of the name (like Beam, Airflow, Ant, Drill) or what > it does (like Mahout, Bookkeeper, Ratis, Pulsar) > * have 2 variants: Square/Circle (social media, favicon, etc) and rectangle > (top banner) > Could: > * have the banner logo be the compact logo with the name attached, or more > elaborate > Would: > * have limited colors; at most 5-10 (so like Iceberg, not like Flex with the > gradients) > > _To allow sufficient artistic freedom, we should limit new requirements. > Instead, please add ideas in the comments._ > > See [https://www.apache.org/logos/] for many examples -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (AVRO-3858) [Build] Add some config to ./build.sh sign
[
https://issues.apache.org/jira/browse/AVRO-3858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3858.
---
Resolution: Fixed
> [Build] Add some config to ./build.sh sign
> ---
>
> Key: AVRO-3858
> URL: https://issues.apache.org/jira/browse/AVRO-3858
> Project: Apache Avro
> Issue Type: Bug
> Components: build
>Reporter: Ryan Skraba
>Assignee: Ryan Skraba
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When signing artifacts for a release in docker, the user will see:
> {code}
> + set +x
> Enter password: gpg: using "MY_DEFAULT_KEY" as default secret key for signing
> gpg: signing failed: No pinentry
> gpg: signing failed: No pinentry
> {code}
> 1) This might be because it wants to pop up a query for the user's key
> password? In any case, we give the password on the command line, a pop-up
> isn't necessary.
> 2) MY_DEFAULT_KEY isn't the key that I use to sign apache releases (which is
> very temporarily only mounted during release signing).
> We should be able to configure the script with --local-user and
> --pinentry-mode for signing releases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (AVRO-3858) [Build] Add some config to ./build.sh sign
[
https://issues.apache.org/jira/browse/AVRO-3858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba reassigned AVRO-3858:
-
Assignee: Ryan Skraba
> [Build] Add some config to ./build.sh sign
> ---
>
> Key: AVRO-3858
> URL: https://issues.apache.org/jira/browse/AVRO-3858
> Project: Apache Avro
> Issue Type: Bug
> Components: build
>Reporter: Ryan Skraba
>Assignee: Ryan Skraba
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When signing artifacts for a release in docker, the user will see:
> {code}
> + set +x
> Enter password: gpg: using "MY_DEFAULT_KEY" as default secret key for signing
> gpg: signing failed: No pinentry
> gpg: signing failed: No pinentry
> {code}
> 1) This might be because it wants to pop up a query for the user's key
> password? In any case, we give the password on the command line, a pop-up
> isn't necessary.
> 2) MY_DEFAULT_KEY isn't the key that I use to sign apache releases (which is
> very temporarily only mounted during release signing).
> We should be able to configure the script with --local-user and
> --pinentry-mode for signing releases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3855) [rust] lint/clippy fails in ubertool
[
https://issues.apache.org/jira/browse/AVRO-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3855:
--
Description:
The rust SDK is currently failing when running in the ubertool:
{code}
ryan.skraba@0c537f861c03:~/avro/lang/rust$ ./build.sh lint
warning: /home/ryan.skraba/avro/lang/rust/avro/Cargo.toml: version requirement
`0.12.4+zstd.1.5.2` for dependency `zstd` includes semver metadata which will
be ignored, removing the metadata is recommended to avoid confusion
Checking apache-avro v0.16.0 (/home/ryan.skraba/avro/lang/rust/avro)
Building [=> ] 236/259: apache-avro, apache-avro,...
error: redundant clone
--> avro/src/schema.rs:726:25
|
726 | .to_owned()
| ^^^ help: remove this
|
= note: `-D clippy::redundant-clone` implied by `-D clippy::all`
note: this value is dropped without further use
--> avro/src/schema.rs:725:36
|
725 | let resolved = avro_value
| ^
726 | | .to_owned()
| |^
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: redundant clone
--> avro/src/schema.rs:1638:17
|
1638 | .to_owned()
| ^^^ help: remove this
|
note: this value is dropped without further use
--> avro/src/schema.rs:1637:28
|
1637 | let resolved = types::Value::from(value.clone())
| ^
1638 | | .to_owned()
| |^
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: could not compile `apache-avro` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error: could not compile `apache-avro` due to 2 previous errors
error: could not compile `apache-avro` due to 2 previous errors
{code}
Taking a very naive look, like there's a couple of things that might be
different from the working GitHub action:
* The clippy command line includes {{-Dunused_imports}} in the GitHub action,
and
* *It appears that 1.72.0 is being used for the lint, regardless of the GitHub
actions strategy matrix.* (versus 1.65.0 in the build.sh)
was:
The rust SDK is currently failing when running in the ubertool:
{code}
ryan.skraba@0c537f861c03:~/avro/lang/rust$ ./build.sh lint
warning: /home/ryan.skraba/avro/lang/rust/avro/Cargo.toml: version requirement
`0.12.4+zstd.1.5.2` for dependency `zstd` includes semver metadata which will
be ignored, removing the metadata is recommended to avoid confusion
Checking apache-avro v0.16.0 (/home/ryan.skraba/avro/lang/rust/avro)
Building [=> ] 236/259: apache-avro, apache-avro,...
error: redundant clone
--> avro/src/schema.rs:726:25
|
726 | .to_owned()
| ^^^ help: remove this
|
= note: `-D clippy::redundant-clone` implied by `-D clippy::all`
note: this value is dropped without further use
--> avro/src/schema.rs:725:36
|
725 | let resolved = avro_value
| ^
726 | | .to_owned()
| |^
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: redundant clone
--> avro/src/schema.rs:1638:17
|
1638 | .to_owned()
| ^^^ help: remove this
|
note: this value is dropped without further use
--> avro/src/schema.rs:1637:28
|
1637 | let resolved = types::Value::from(value.clone())
| ^
1638 | | .to_owned()
| |^
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: could not compile `apache-avro` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error: could not compile `apache-avro` due to 2 previous errors
error: could not compile `apache-avro` due to 2 previous errors
{code}
Taking a very naive look, like there's a couple of things that might be
different from the working GitHub action:
* The clippy command line includes {{-Dunused_imports}} in the GitHub action,
and
* *It appears that 1.72.0 is being used for the lint, regardless of the GitHub
actions strategy matrix.*
> [rust] lint/clippy fails in ubertool
>
>
> Key: AVRO-3855
> URL: https://issues.apache.org/jira/browse/AVRO-3855
> Project: Apache Avro
> Issue Type: Bug
>Reporter: Ryan
[jira] [Updated] (AVRO-3855) [rust] lint/clippy fails in ubertool
[
https://issues.apache.org/jira/browse/AVRO-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3855:
--
Description:
The rust SDK is currently failing when running in the ubertool:
{code}
ryan.skraba@0c537f861c03:~/avro/lang/rust$ ./build.sh lint
warning: /home/ryan.skraba/avro/lang/rust/avro/Cargo.toml: version requirement
`0.12.4+zstd.1.5.2` for dependency `zstd` includes semver metadata which will
be ignored, removing the metadata is recommended to avoid confusion
Checking apache-avro v0.16.0 (/home/ryan.skraba/avro/lang/rust/avro)
Building [=> ] 236/259: apache-avro, apache-avro,...
error: redundant clone
--> avro/src/schema.rs:726:25
|
726 | .to_owned()
| ^^^ help: remove this
|
= note: `-D clippy::redundant-clone` implied by `-D clippy::all`
note: this value is dropped without further use
--> avro/src/schema.rs:725:36
|
725 | let resolved = avro_value
| ^
726 | | .to_owned()
| |^
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: redundant clone
--> avro/src/schema.rs:1638:17
|
1638 | .to_owned()
| ^^^ help: remove this
|
note: this value is dropped without further use
--> avro/src/schema.rs:1637:28
|
1637 | let resolved = types::Value::from(value.clone())
| ^
1638 | | .to_owned()
| |^
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: could not compile `apache-avro` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error: could not compile `apache-avro` due to 2 previous errors
error: could not compile `apache-avro` due to 2 previous errors
{code}
Taking a very naive look, like there's a couple of things that might be
different from the working GitHub action:
* The clippy command line includes {{-Dunused_imports}} in the GitHub action,
and
* *It appears that 1.72.0 is being used for the lint, regardless of the GitHub
actions strategy matrix.*
was:
The rust SDK is currently failing when running in the ubertool:
{code}
ryan.skraba@0c537f861c03:~/avro/lang/rust$ ./build.sh lint
warning: /home/ryan.skraba/avro/lang/rust/avro/Cargo.toml: version requirement
`0.12.4+zstd.1.5.2` for dependency `zstd` includes semver metadata which will
be ignored, removing the metadata is recommended to avoid confusion
Checking apache-avro v0.16.0 (/home/ryan.skraba/avro/lang/rust/avro)
Building [=> ] 236/259: apache-avro, apache-avro,...
error: redundant clone
--> avro/src/schema.rs:726:25
|
726 | .to_owned()
| ^^^ help: remove this
|
= note: `-D clippy::redundant-clone` implied by `-D clippy::all`
note: this value is dropped without further use
--> avro/src/schema.rs:725:36
|
725 | let resolved = avro_value
| ^
726 | | .to_owned()
| |^
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: redundant clone
--> avro/src/schema.rs:1638:17
|
1638 | .to_owned()
| ^^^ help: remove this
|
note: this value is dropped without further use
--> avro/src/schema.rs:1637:28
|
1637 | let resolved = types::Value::from(value.clone())
| ^
1638 | | .to_owned()
| |^
= help: for further information visit
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: could not compile `apache-avro` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error: could not compile `apache-avro` due to 2 previous errors
error: could not compile `apache-avro` due to 2 previous errors
{code}
Taking a very naive look, like there's a couple of things that might be
different from the working GitHub action:
* The clippy command line includes {{-Dunused_imports}} in the GitHub action,
and
* It appears that 1.72.0 is being used for the lint, regardless of the strategy
matrix.
> [rust] lint/clippy fails in ubertool
>
>
> Key: AVRO-3855
> URL: https://issues.apache.org/jira/browse/AVRO-3855
> Project: Apache Avro
> Issue Type: Bug
>Reporter: Ryan Skraba
>Priority: Major
> Fix
[jira] [Commented] (AVRO-3554) Create original art for the Avro logo
[ https://issues.apache.org/jira/browse/AVRO-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17763678#comment-17763678 ] Ryan Skraba commented on AVRO-3554: --- Thanks so much for bringing this topic back to life! I have to say, *all* three of the logos are so well thought out and appropriate. As a parisian and a gardener the pigeon resonates the most with me :D Also, the origami-style also suggests tangram-style, as if the "whole picture" could be made of basic building blocks. However, also in my opinion, the airplane logo with the blobby tail or pixelated tail is such a good representation for the project, I'd probably end up voting for one of those! The style goes so well with many of the other ASF project icons and the simple elegant style would probably be a bit more flexible in product UIs that refer to Avro files or data. I really have little opinion on the colour, but I think the bright blue is just a nice vibrant choice! Thanks again, I took the liberty of assigning you this JIRA :D > Create original art for the Avro logo > - > > Key: AVRO-3554 > URL: https://issues.apache.org/jira/browse/AVRO-3554 > Project: Apache Avro > Issue Type: Improvement >Reporter: Ryan Skraba >Assignee: Emma Kellam >Priority: Major > Fix For: 1.12.0 > > Attachments: Apache-Avro-Logo-Ideas.pdf, > Apache-Avro-Logo-Revisions-V1.pdf, OldAsmLogo.png, cuttlefish oscar 2.svg, > cuttlefish oscar.svg > > > There was a quick discussion with Apache Trademarks along the lines "If it > ever came to a legal challenge, would we care enough to defend our usage? If > not, changing it discreetly and voluntarily is the best route." > The result: we _must_ change our logo this year to some original art. > > Criteria for a new logo, as discussed on the mailing list: > [https://lists.apache.org/thread/m5w7fmkjnrl8m4hlbw8xhzr4v69xg3ml]: > Must: > * be unique and not similar to anything “out there” > * be available in a vector format (SVG) so it can be scaled with full > accuracy > Should: > * show either something of the name (like Beam, Airflow, Ant, Drill) or what > it does (like Mahout, Bookkeeper, Ratis, Pulsar) > * have 2 variants: Square/Circle (social media, favicon, etc) and rectangle > (top banner) > Could: > * have the banner logo be the compact logo with the name attached, or more > elaborate > Would: > * have limited colors; at most 5-10 (so like Iceberg, not like Flex with the > gradients) > > _To allow sufficient artistic freedom, we should limit new requirements. > Instead, please add ideas in the comments._ > > See [https://www.apache.org/logos/] for many examples -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (AVRO-3554) Create original art for the Avro logo
[ https://issues.apache.org/jira/browse/AVRO-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-3554: - Assignee: Emma Kellam > Create original art for the Avro logo > - > > Key: AVRO-3554 > URL: https://issues.apache.org/jira/browse/AVRO-3554 > Project: Apache Avro > Issue Type: Improvement >Reporter: Ryan Skraba >Assignee: Emma Kellam >Priority: Major > Fix For: 1.12.0 > > Attachments: Apache-Avro-Logo-Ideas.pdf, > Apache-Avro-Logo-Revisions-V1.pdf, OldAsmLogo.png, cuttlefish oscar 2.svg, > cuttlefish oscar.svg > > > There was a quick discussion with Apache Trademarks along the lines "If it > ever came to a legal challenge, would we care enough to defend our usage? If > not, changing it discreetly and voluntarily is the best route." > The result: we _must_ change our logo this year to some original art. > > Criteria for a new logo, as discussed on the mailing list: > [https://lists.apache.org/thread/m5w7fmkjnrl8m4hlbw8xhzr4v69xg3ml]: > Must: > * be unique and not similar to anything “out there” > * be available in a vector format (SVG) so it can be scaled with full > accuracy > Should: > * show either something of the name (like Beam, Airflow, Ant, Drill) or what > it does (like Mahout, Bookkeeper, Ratis, Pulsar) > * have 2 variants: Square/Circle (social media, favicon, etc) and rectangle > (top banner) > Could: > * have the banner logo be the compact logo with the name attached, or more > elaborate > Would: > * have limited colors; at most 5-10 (so like Iceberg, not like Flex with the > gradients) > > _To allow sufficient artistic freedom, we should limit new requirements. > Instead, please add ideas in the comments._ > > See [https://www.apache.org/logos/] for many examples -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (AVRO-3819) [Java] Rationalize the system properties that limit allocation
[ https://issues.apache.org/jira/browse/AVRO-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba resolved AVRO-3819. --- Resolution: Fixed > [Java] Rationalize the system properties that limit allocation > -- > > Key: AVRO-3819 > URL: https://issues.apache.org/jira/browse/AVRO-3819 > Project: Apache Avro > Issue Type: Bug > Components: java >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Labels: pull-request-available > Fix For: 1.11.3 > > Time Spent: 50m > Remaining Estimate: 0h > > There are currently some system properties that limit datum allocation size: > * org.apache.avro.limits.byte.maxLength > * org.apache.avro.limits.string.maxLength > These are hidden in two different classes (Utf8 and BinaryDecoder). It would > make sense to centralize them in one place to make it clearer how to limit > the damage untrusted data could do while deserializing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (AVRO-3819) [Java] Rationalize the system properties that limit allocation
[ https://issues.apache.org/jira/browse/AVRO-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-3819: - > [Java] Rationalize the system properties that limit allocation > -- > > Key: AVRO-3819 > URL: https://issues.apache.org/jira/browse/AVRO-3819 > Project: Apache Avro > Issue Type: Bug > Components: java >Reporter: Ryan Skraba >Assignee: Ryan Skraba >Priority: Major > Fix For: 1.11.3 > > > There are currently some system properties that limit datum allocation size: > * org.apache.avro.limits.byte.maxLength > * org.apache.avro.limits.string.maxLength > These are hidden in two different classes (Utf8 and BinaryDecoder). It would > make sense to centralize them in one place to make it clearer how to limit > the damage untrusted data could do while deserializing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (AVRO-3732) Code drop existing sources, updating headers, RAT, NOTICES, etc.
[ https://issues.apache.org/jira/browse/AVRO-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746523#comment-17746523 ] Ryan Skraba commented on AVRO-3732: --- Hey -- it looks like there's going to be a version 1.11.3 that needs to be released first! I'm willing to pick up the IP Clearance checklist after that, but it might take me a while to get there. if anybody wants to jump in on this, that would be fine too. In the meantime, thanks for your patience [~singingbush]... I'm not yet entirely sure what the "code drop" tag needs to look like. Ideally the next steps (as an ASF contributor!) would be in separate PRs that can be applied directly on top of this one. I am wildly enthusiastic about someone else doing this work :D Thanks so much!!! > Code drop existing sources, updating headers, RAT, NOTICES, etc. > > > Key: AVRO-3732 > URL: https://issues.apache.org/jira/browse/AVRO-3732 > Project: Apache Avro > Issue Type: Sub-task >Reporter: Ryan Skraba >Assignee: Samael Bate >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Move the source to https://github.com/apache/avro/tree/master/lang/java in a > directory called gradle-plugin (parallel to maven-plugin). > To decide -- should we do a code dump to the destination or attempt to > maintain the history from the original repo? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-2284) Incorrect EnumSymbol initialization in TestReadingWritingDataInEvolvedSchemas.java
[ https://issues.apache.org/jira/browse/AVRO-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-2284: -- Fix Version/s: 1.12.0 1.11.3 > Incorrect EnumSymbol initialization in > TestReadingWritingDataInEvolvedSchemas.java > -- > > Key: AVRO-2284 > URL: https://issues.apache.org/jira/browse/AVRO-2284 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.2 >Reporter: Zoltan Farkas >Assignee: Christophe Le Saec >Priority: Minor > Labels: pull-request-available > Fix For: 1.12.0, 1.11.3 > > Time Spent: 20m > Remaining Estimate: 0h > > EnumSymbol is initialized with Record schema instead of Enum schema at: > https://github.com/apache/avro/blob/master/lang/java/avro/src/test/java/org/apache/avro/TestReadingWritingDataInEvolvedSchemas.java#L310 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-2284) Incorrect EnumSymbol initialization in TestReadingWritingDataInEvolvedSchemas.java
[ https://issues.apache.org/jira/browse/AVRO-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-2284: -- Release Note: Thanks, I'll cherry-pick this. It's weird that this worked in the first place! Resolution: Fixed Status: Resolved (was: Patch Available) > Incorrect EnumSymbol initialization in > TestReadingWritingDataInEvolvedSchemas.java > -- > > Key: AVRO-2284 > URL: https://issues.apache.org/jira/browse/AVRO-2284 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.2 >Reporter: Zoltan Farkas >Assignee: Christophe Le Saec >Priority: Minor > Labels: pull-request-available > Fix For: 1.12.0, 1.11.3 > > Time Spent: 20m > Remaining Estimate: 0h > > EnumSymbol is initialized with Record schema instead of Enum schema at: > https://github.com/apache/avro/blob/master/lang/java/avro/src/test/java/org/apache/avro/TestReadingWritingDataInEvolvedSchemas.java#L310 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (AVRO-1938) Python support for generating canonical forms of schema
[ https://issues.apache.org/jira/browse/AVRO-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744190#comment-17744190 ] Ryan Skraba commented on AVRO-1938: --- Awesome work -- thanks for the contribution and collaboration (and patience!) in updating fingerprinting in the Python SDK! Is this finished and targetted for 1.12.0? Should we update the title of this JIRA to reflect the full work done in the PR? > Python support for generating canonical forms of schema > --- > > Key: AVRO-1938 > URL: https://issues.apache.org/jira/browse/AVRO-1938 > Project: Apache Avro > Issue Type: Improvement > Components: python >Reporter: Erik Forsberg >Assignee: Michael A. Smith >Priority: Major > Labels: pull-request-available > Time Spent: 4h > Remaining Estimate: 0h > > The python implementation(s) lack support for generating canonical forms of > schema. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (AVRO-3795) Correct handling of nonexistent files in maven-plugin
[ https://issues.apache.org/jira/browse/AVRO-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743121#comment-17743121 ] Ryan Skraba commented on AVRO-3795: --- Thanks for the contribution! > Correct handling of nonexistent files in maven-plugin > - > > Key: AVRO-3795 > URL: https://issues.apache.org/jira/browse/AVRO-3795 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.11.2 >Reporter: Michal Jagielski >Assignee: Michal Jagielski >Priority: Minor > Labels: pull-request-available > Fix For: 1.11.3 > > Original Estimate: 1h > Time Spent: 1.5h > Remaining Estimate: 0h > > Nonexistent files included in "import" declarations of _maven-plugin_ are > silently ignored. I think it's erroneous behavior that may be fixed easily. > It already caused bugs in our company and I see that we're not the only ones. > I've created a PR from branch with testcase and working fix. I will be > grateful if you can take a look on that and either merge or suggest other > solution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3795) Correct handling of nonexistent files in maven-plugin
[ https://issues.apache.org/jira/browse/AVRO-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3795: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Correct handling of nonexistent files in maven-plugin > - > > Key: AVRO-3795 > URL: https://issues.apache.org/jira/browse/AVRO-3795 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.11.2 >Reporter: Michal Jagielski >Assignee: Michal Jagielski >Priority: Minor > Labels: pull-request-available > Fix For: 1.11.3 > > Original Estimate: 1h > Time Spent: 1.5h > Remaining Estimate: 0h > > Nonexistent files included in "import" declarations of _maven-plugin_ are > silently ignored. I think it's erroneous behavior that may be fixed easily. > It already caused bugs in our company and I see that we're not the only ones. > I've created a PR from branch with testcase and working fix. I will be > grateful if you can take a look on that and either merge or suggest other > solution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3713) Regression AVRO-1760 : Thread scalability problem with the use of SynchronizedMap
[ https://issues.apache.org/jira/browse/AVRO-3713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3713: -- Fix Version/s: 1.11.3 > Regression AVRO-1760 : Thread scalability problem with the use of > SynchronizedMap > - > > Key: AVRO-3713 > URL: https://issues.apache.org/jira/browse/AVRO-3713 > Project: Apache Avro > Issue Type: Bug >Reporter: Niels Basjes >Assignee: Niels Basjes >Priority: Major > Labels: pull-request-available > Fix For: 1.11.3 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > This is a regression of AVRO-1760 which is causing production performance > problems. > In AVRO-1760 the issue was fixed that there is a scalability problem with > AVRO being used in multithreaded situations. > The solution that was implemented for this ticket uses a different Map > implementation (part of Guava) that does not have these locking issues: > [https://github.com/apache/avro/commit/363360229e7ea5f816d87f9d5747dd4494d3b862#diff-422f44890f5e0ab1f508a7c75d0d18fc7831a1c3ca13e0c97780617570c50d5cL970-L973] > Investigating the performance of our internal application has made us > conclude the original problem was reintroduced with this commit: > [https://github.com/apache/avro/commit/c78a3d5ea4e7e511ba9f6182157956db3e65e1a0#diff-422f44890f5e0ab1f508a7c75d0d18fc7831a1c3ca13e0c97780617570c50d5cL986-L988] > which seems to be related to this jira AVRO-2265 "Remove Guava as a > dependency". -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (AVRO-3795) Correct handling of nonexistent files in maven-plugin
[ https://issues.apache.org/jira/browse/AVRO-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-3795: - Assignee: Michal Jagielski > Correct handling of nonexistent files in maven-plugin > - > > Key: AVRO-3795 > URL: https://issues.apache.org/jira/browse/AVRO-3795 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.11.2 >Reporter: Michal Jagielski >Assignee: Michal Jagielski >Priority: Minor > Labels: pull-request-available > Fix For: 1.11.3 > > Original Estimate: 1h > Time Spent: 20m > Remaining Estimate: 40m > > Nonexistent files included in "import" declarations of _maven-plugin_ are > silently ignored. I think it's erroneous behavior that may be fixed easily. > It already caused bugs in our company and I see that we're not the only ones. > I've created a PR from branch with testcase and working fix. I will be > grateful if you can take a look on that and either merge or suggest other > solution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3795) Correct handling of nonexistent files in maven-plugin
[ https://issues.apache.org/jira/browse/AVRO-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3795: -- Status: Patch Available (was: Open) > Correct handling of nonexistent files in maven-plugin > - > > Key: AVRO-3795 > URL: https://issues.apache.org/jira/browse/AVRO-3795 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.11.2 >Reporter: Michal Jagielski >Assignee: Michal Jagielski >Priority: Minor > Labels: pull-request-available > Fix For: 1.11.3 > > Original Estimate: 1h > Time Spent: 20m > Remaining Estimate: 40m > > Nonexistent files included in "import" declarations of _maven-plugin_ are > silently ignored. I think it's erroneous behavior that may be fixed easily. > It already caused bugs in our company and I see that we're not the only ones. > I've created a PR from branch with testcase and working fix. I will be > grateful if you can take a look on that and either merge or suggest other > solution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3795) Correct handling of nonexistent files in maven-plugin
[ https://issues.apache.org/jira/browse/AVRO-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3795: -- Fix Version/s: 1.11.3 > Correct handling of nonexistent files in maven-plugin > - > > Key: AVRO-3795 > URL: https://issues.apache.org/jira/browse/AVRO-3795 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.11.2 >Reporter: Michal Jagielski >Priority: Minor > Labels: pull-request-available > Fix For: 1.11.3 > > Original Estimate: 1h > Time Spent: 20m > Remaining Estimate: 40m > > Nonexistent files included in "import" declarations of _maven-plugin_ are > silently ignored. I think it's erroneous behavior that may be fixed easily. > It already caused bugs in our company and I see that we're not the only ones. > I've created a PR from branch with testcase and working fix. I will be > grateful if you can take a look on that and either merge or suggest other > solution. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (AVRO-3789) Comparing maps in GenericData is wrong for certain combinations and fails for empty maps
[
https://issues.apache.org/jira/browse/AVRO-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba reassigned AVRO-3789:
-
Assignee: Felix Krull
> Comparing maps in GenericData is wrong for certain combinations and fails for
> empty maps
>
>
> Key: AVRO-3789
> URL: https://issues.apache.org/jira/browse/AVRO-3789
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.11.2
>Reporter: Felix Krull
>Assignee: Felix Krull
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.3
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The rewritten map comparison implementation in GenericData (AVRO-2943
> according to the commit message) throws a NoSuchElementException when used to
> compare empty maps. Partial stacktrace:
> {code}
> Caused by: java.util.NoSuchElementException
> at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1513)
> at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1534)
> at
> org.apache.avro.generic.GenericData.compareMaps(GenericData.java:1163)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1250)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1229)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at
> org.apache.avro.specific.SpecificRecordBase.equals(SpecificRecordBase.java:88)
> at scala.runtime.BoxesRunTime.equals2(BoxesRunTime.java:133)
> at scala.runtime.BoxesRunTime.equals(BoxesRunTime.java:119)
> at
> org.scalactic.DefaultEquality$.areEqualComparingArraysStructurally(DefaultEquality.scala:70)
> at org.scalactic.DefaultEquality.areEqual(DefaultEquality.scala:37)
> at org.mockito.package$$anon$2.areEqual(mockito.scala:614)
> at
> org.scalactic.TripleEqualsSupport$Equalizer.$eq$eq$eq(TripleEqualsSupport.scala:117)
> at org.mockito.matchers.EqTo.matches(EqTo.scala:11)
> ...
> {code}
>
>
> Also, the check in line 1170 that's intended to shortcircuit for maps of
> different sizes is incorrect. Because of this, maps will incorrectly compare
> equal when:
> * their sizes are different
> * one is a superset of the other map
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (AVRO-3789) Comparing maps in GenericData is wrong for certain combinations and fails for empty maps
[
https://issues.apache.org/jira/browse/AVRO-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba closed AVRO-3789.
-
> Comparing maps in GenericData is wrong for certain combinations and fails for
> empty maps
>
>
> Key: AVRO-3789
> URL: https://issues.apache.org/jira/browse/AVRO-3789
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.11.2
>Reporter: Felix Krull
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.3
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The rewritten map comparison implementation in GenericData (AVRO-2943
> according to the commit message) throws a NoSuchElementException when used to
> compare empty maps. Partial stacktrace:
> {code}
> Caused by: java.util.NoSuchElementException
> at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1513)
> at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1534)
> at
> org.apache.avro.generic.GenericData.compareMaps(GenericData.java:1163)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1250)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1229)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at
> org.apache.avro.specific.SpecificRecordBase.equals(SpecificRecordBase.java:88)
> at scala.runtime.BoxesRunTime.equals2(BoxesRunTime.java:133)
> at scala.runtime.BoxesRunTime.equals(BoxesRunTime.java:119)
> at
> org.scalactic.DefaultEquality$.areEqualComparingArraysStructurally(DefaultEquality.scala:70)
> at org.scalactic.DefaultEquality.areEqual(DefaultEquality.scala:37)
> at org.mockito.package$$anon$2.areEqual(mockito.scala:614)
> at
> org.scalactic.TripleEqualsSupport$Equalizer.$eq$eq$eq(TripleEqualsSupport.scala:117)
> at org.mockito.matchers.EqTo.matches(EqTo.scala:11)
> ...
> {code}
>
>
> Also, the check in line 1170 that's intended to shortcircuit for maps of
> different sizes is incorrect. Because of this, maps will incorrectly compare
> equal when:
> * their sizes are different
> * one is a superset of the other map
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (AVRO-3789) Comparing maps in GenericData is wrong for certain combinations and fails for empty maps
[
https://issues.apache.org/jira/browse/AVRO-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3789.
---
Resolution: Fixed
> Comparing maps in GenericData is wrong for certain combinations and fails for
> empty maps
>
>
> Key: AVRO-3789
> URL: https://issues.apache.org/jira/browse/AVRO-3789
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.11.2
>Reporter: Felix Krull
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.3
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The rewritten map comparison implementation in GenericData (AVRO-2943
> according to the commit message) throws a NoSuchElementException when used to
> compare empty maps. Partial stacktrace:
> {code}
> Caused by: java.util.NoSuchElementException
> at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1513)
> at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1534)
> at
> org.apache.avro.generic.GenericData.compareMaps(GenericData.java:1163)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1250)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1229)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at
> org.apache.avro.specific.SpecificRecordBase.equals(SpecificRecordBase.java:88)
> at scala.runtime.BoxesRunTime.equals2(BoxesRunTime.java:133)
> at scala.runtime.BoxesRunTime.equals(BoxesRunTime.java:119)
> at
> org.scalactic.DefaultEquality$.areEqualComparingArraysStructurally(DefaultEquality.scala:70)
> at org.scalactic.DefaultEquality.areEqual(DefaultEquality.scala:37)
> at org.mockito.package$$anon$2.areEqual(mockito.scala:614)
> at
> org.scalactic.TripleEqualsSupport$Equalizer.$eq$eq$eq(TripleEqualsSupport.scala:117)
> at org.mockito.matchers.EqTo.matches(EqTo.scala:11)
> ...
> {code}
>
>
> Also, the check in line 1170 that's intended to shortcircuit for maps of
> different sizes is incorrect. Because of this, maps will incorrectly compare
> equal when:
> * their sizes are different
> * one is a superset of the other map
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3789) Comparing maps in GenericData is wrong for certain combinations and fails for empty maps
[
https://issues.apache.org/jira/browse/AVRO-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741677#comment-17741677
]
Ryan Skraba commented on AVRO-3789:
---
Hello! I'm adding this to the release notes for 1.11.2, sorry :/ It's not
possible to "unpublish" a maven release, but we can do a 1.11.3 really quickly!
> Comparing maps in GenericData is wrong for certain combinations and fails for
> empty maps
>
>
> Key: AVRO-3789
> URL: https://issues.apache.org/jira/browse/AVRO-3789
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.11.2
>Reporter: Felix Krull
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.3
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The rewritten map comparison implementation in GenericData (AVRO-2943
> according to the commit message) throws a NoSuchElementException when used to
> compare empty maps. Partial stacktrace:
> {code}
> Caused by: java.util.NoSuchElementException
> at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1513)
> at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1534)
> at
> org.apache.avro.generic.GenericData.compareMaps(GenericData.java:1163)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1250)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1229)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at
> org.apache.avro.specific.SpecificRecordBase.equals(SpecificRecordBase.java:88)
> at scala.runtime.BoxesRunTime.equals2(BoxesRunTime.java:133)
> at scala.runtime.BoxesRunTime.equals(BoxesRunTime.java:119)
> at
> org.scalactic.DefaultEquality$.areEqualComparingArraysStructurally(DefaultEquality.scala:70)
> at org.scalactic.DefaultEquality.areEqual(DefaultEquality.scala:37)
> at org.mockito.package$$anon$2.areEqual(mockito.scala:614)
> at
> org.scalactic.TripleEqualsSupport$Equalizer.$eq$eq$eq(TripleEqualsSupport.scala:117)
> at org.mockito.matchers.EqTo.matches(EqTo.scala:11)
> ...
> {code}
>
>
> Also, the check in line 1170 that's intended to shortcircuit for maps of
> different sizes is incorrect. Because of this, maps will incorrectly compare
> equal when:
> * their sizes are different
> * one is a superset of the other map
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3789) Comparing maps in GenericData is wrong for certain combinations and fails for empty maps
[
https://issues.apache.org/jira/browse/AVRO-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3789:
--
Fix Version/s: 1.11.3
> Comparing maps in GenericData is wrong for certain combinations and fails for
> empty maps
>
>
> Key: AVRO-3789
> URL: https://issues.apache.org/jira/browse/AVRO-3789
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.11.2
>Reporter: Felix Krull
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.3
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The rewritten map comparison implementation in GenericData (AVRO-2943
> according to the commit message) throws a NoSuchElementException when used to
> compare empty maps. Partial stacktrace:
> {code}
> Caused by: java.util.NoSuchElementException
> at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1513)
> at java.base/java.util.HashMap$KeyIterator.next(HashMap.java:1534)
> at
> org.apache.avro.generic.GenericData.compareMaps(GenericData.java:1163)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1250)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at org.apache.avro.generic.GenericData.compare(GenericData.java:1229)
> at org.apache.avro.specific.SpecificData.compare(SpecificData.java:476)
> at
> org.apache.avro.specific.SpecificRecordBase.equals(SpecificRecordBase.java:88)
> at scala.runtime.BoxesRunTime.equals2(BoxesRunTime.java:133)
> at scala.runtime.BoxesRunTime.equals(BoxesRunTime.java:119)
> at
> org.scalactic.DefaultEquality$.areEqualComparingArraysStructurally(DefaultEquality.scala:70)
> at org.scalactic.DefaultEquality.areEqual(DefaultEquality.scala:37)
> at org.mockito.package$$anon$2.areEqual(mockito.scala:614)
> at
> org.scalactic.TripleEqualsSupport$Equalizer.$eq$eq$eq(TripleEqualsSupport.scala:117)
> at org.mockito.matchers.EqTo.matches(EqTo.scala:11)
> ...
> {code}
>
>
> Also, the check in line 1170 that's intended to shortcircuit for maps of
> different sizes is incorrect. Because of this, maps will incorrectly compare
> equal when:
> * their sizes are different
> * one is a superset of the other map
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3377) Deserialization of record of mangled Java class throws ClassCastException
[
https://issues.apache.org/jira/browse/AVRO-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3377:
--
Fix Version/s: 1.12.0
(was: 1.11.2)
> Deserialization of record of mangled Java class throws ClassCastException
> -
>
> Key: AVRO-3377
> URL: https://issues.apache.org/jira/browse/AVRO-3377
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Affects Versions: 1.11.0
>Reporter: Kyle Carter
>Assignee: Kyle Carter
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When a Java class based on a Avro schema is created that includes a mangled
> part of it's fully qualified name and a reusable class is not provided a
> {{ClassCastException}} is thrown.
> This is an edge case and there is a work around in that the reusable class
> can be provided to get around this issue but being able to avoid this edge
> case would be preferred.
> POC repo: https://github.com/kylec32/avromangledeserializationpoc
> Issue was discovered when testing: AVRO-3305
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-2872) 'conversions' array is not populated for Avro Union Logicaltype fields
[
https://issues.apache.org/jira/browse/AVRO-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-2872:
--
Fix Version/s: 1.12.0
(was: 1.11.2)
> 'conversions' array is not populated for Avro Union Logicaltype fields
> ---
>
> Key: AVRO-2872
> URL: https://issues.apache.org/jira/browse/AVRO-2872
> Project: Apache Avro
> Issue Type: Bug
> Components: logical types
>Affects Versions: 1.9.2
> Environment: * Apache Avro Version [1.9.2]
> * Java Version [11]
>Reporter: Pádraig de Buitléar
>Assignee: Josh Cooke
>Priority: Minor
> Fix For: 1.12.0
>
>
> Steps to reproduce :
> # Using the maven/gradle plugin generate code with the following avsc:
>
> {code:java}
> {
> "type": "record",
> "name": "Messages",
> "namespace": "com.somedomain",
> "fields": [
> {
> "name": "start",
> "type": {
> "type": "long",
> "logicalType": "timestamp-millis"
> }
> },
> {
> "name": "optional_date",
> "type": [
> "null",
> {
> "type": "long",
> "logicalType": "timestamp-millis"
> }
> ],
> "default": null
> }
> ]
> }{code}
>
> *Actual behavior*
> In the generated code, the return types of the getter methods for both
> fields are correct, however the conversions array only has an element for the
> field which isn't a union.
>
> {code:java}
> private static final org.apache.avro.Conversion[] conversions =
> new org.apache.avro.Conversion[] {
> new org.apache.avro.data.TimeConversions.TimestampMillisConversion(),
> null,
> null
> };
> {code}
>
> *Expected output:*
> Based on the above avsc the following is expected.
>
> {code:java}
> private static final org.apache.avro.Conversion[] conversions =
> new org.apache.avro.Conversion[] {
> new org.apache.avro.data.TimeConversions.TimestampMillisConversion(),
> new org.apache.avro.data.TimeConversions.TimestampMillisConversion(),
> null
> };
> {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3760) Using enum with default symbol, cannot parse future value
[
https://issues.apache.org/jira/browse/AVRO-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3760:
--
Fix Version/s: 1.12.0
> Using enum with default symbol, cannot parse future value
> -
>
> Key: AVRO-3760
> URL: https://issues.apache.org/jira/browse/AVRO-3760
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code}
> $ pip freeze | grep -i avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems like support for default symbols is broken. In the example below,
> since I'm using default symbols, I expected to be able to add new values to
> the enum and see the default value when parsing using the old schema.
> {code:python}
> import io
> from avro.io import DatumReader, DatumWriter, BinaryDecoder, BinaryEncoder
> import avro.schema
> current_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> # Future schema adds the "crc32_be" symbol.
> future_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be",
> "crc32_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(future_schema)
> encoder = BinaryEncoder(buffer)
> writer.write({"checksum_algorithm": "crc32_be"}, encoder)
> buffer.seek(0)
> reader = DatumReader(current_schema)
> decoder = BinaryDecoder(buffer)
> decoded = reader.read(decoder)
> print(decoded)
> {code}
> Instead, this results in an exception:
> {code}
> Traceback (most recent call last):
> File "reproduce-avro.py", line 58, in
> decoded = reader.read(decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 649, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 727, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 922, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 720, in read_data
> return self.read_enum(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 779, in read_enum
> raise avro.errors.SchemaResolutionException(
> avro.errors.SchemaResolutionException: Can't access enum index 2 for enum
> with 2 symbols
> Writer's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> Reader's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3760) Using enum with default symbol, cannot parse future value
[
https://issues.apache.org/jira/browse/AVRO-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3760:
--
Fix Version/s: (was: 1.11.2)
> Using enum with default symbol, cannot parse future value
> -
>
> Key: AVRO-3760
> URL: https://issues.apache.org/jira/browse/AVRO-3760
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code}
> $ pip freeze | grep -i avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems like support for default symbols is broken. In the example below,
> since I'm using default symbols, I expected to be able to add new values to
> the enum and see the default value when parsing using the old schema.
> {code:python}
> import io
> from avro.io import DatumReader, DatumWriter, BinaryDecoder, BinaryEncoder
> import avro.schema
> current_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> # Future schema adds the "crc32_be" symbol.
> future_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be",
> "crc32_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(future_schema)
> encoder = BinaryEncoder(buffer)
> writer.write({"checksum_algorithm": "crc32_be"}, encoder)
> buffer.seek(0)
> reader = DatumReader(current_schema)
> decoder = BinaryDecoder(buffer)
> decoded = reader.read(decoder)
> print(decoded)
> {code}
> Instead, this results in an exception:
> {code}
> Traceback (most recent call last):
> File "reproduce-avro.py", line 58, in
> decoded = reader.read(decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 649, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 727, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 922, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 720, in read_data
> return self.read_enum(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 779, in read_enum
> raise avro.errors.SchemaResolutionException(
> avro.errors.SchemaResolutionException: Can't access enum index 2 for enum
> with 2 symbols
> Writer's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> Reader's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3732) Code drop existing sources, updating headers, RAT, NOTICES, etc.
[ https://issues.apache.org/jira/browse/AVRO-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17738973#comment-17738973 ] Ryan Skraba commented on AVRO-3732: --- Thanks for getting this started! I like that we kept the history. This is one of the few cases where I think it's important to do a merge, as opposed to a merge-squash. I don't have any objection to having the CI fail for a short period while we're cleaning up the donation (workflows, build, etc.) As soon as 1.11.2 gets published, this is going to be a priority for 1.12.0 ! > Code drop existing sources, updating headers, RAT, NOTICES, etc. > > > Key: AVRO-3732 > URL: https://issues.apache.org/jira/browse/AVRO-3732 > Project: Apache Avro > Issue Type: Sub-task >Reporter: Ryan Skraba >Assignee: Samael Bate >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Move the source to https://github.com/apache/avro/tree/master/lang/java in a > directory called gradle-plugin (parallel to maven-plugin). > To decide -- should we do a code dump to the destination or attempt to > maintain the history from the original repo? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3690) Detected a flaky test ‘testMultipleFieldAliases’
[ https://issues.apache.org/jira/browse/AVRO-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3690: -- Fix Version/s: 1.11.2 > Detected a flaky test ‘testMultipleFieldAliases’ > > > Key: AVRO-3690 > URL: https://issues.apache.org/jira/browse/AVRO-3690 > Project: Apache Avro > Issue Type: Test >Reporter: Zhengxu Jin >Assignee: Niels Basjes >Priority: Trivial > Labels: pull-request-available > Fix For: 1.11.2 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Test `testMultipleFieldAliases' under 'TestReflect.java' is a flaky test. To > check its flakiness, you can run maven with nondex using the following > command: > ``` > mvn -pl lang/java/avro edu.illinois:nondex-maven-plugin:2.1.1:nondex > -Dtest=org.apache.avro.reflect.TestReflect#testMultipleFieldAliases > ``` > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (AVRO-3690) Detected a flaky test ‘testMultipleFieldAliases’
[ https://issues.apache.org/jira/browse/AVRO-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-3690: - Assignee: Niels Basjes > Detected a flaky test ‘testMultipleFieldAliases’ > > > Key: AVRO-3690 > URL: https://issues.apache.org/jira/browse/AVRO-3690 > Project: Apache Avro > Issue Type: Test >Reporter: Zhengxu Jin >Assignee: Niels Basjes >Priority: Trivial > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Test `testMultipleFieldAliases' under 'TestReflect.java' is a flaky test. To > check its flakiness, you can run maven with nondex using the following > command: > ``` > mvn -pl lang/java/avro edu.illinois:nondex-maven-plugin:2.1.1:nondex > -Dtest=org.apache.avro.reflect.TestReflect#testMultipleFieldAliases > ``` > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3731) Integrate software donation of gradle-avro-plugin
[
https://issues.apache.org/jira/browse/AVRO-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3731:
--
Description:
The Apache project has
[voted|https://lists.apache.org/thread/hz8fomzcwt2yhyz6l4ntzp3h0vpqd8xg] to
accept the software donation of the
[gradle-avro-plugin|https://github.com/davidmc24/gradle-avro-plugin/discussions/208]
graciously donated by the maintainer (David M. Carr). Thanks!
This umbrella Jira is tracking the steps we need to take to integrate the code
into the project.
We already have the Software Grant Agreement to cover the incoming code, which
is already licensed under the Apache 2.0 license with source code headers
including {{Copyright © 2013-2019 Commerce Technologies, LLC.}}
Some useful resources for the process are provided by the incubator:
* https://incubator.apache.org/guides/transitioning_asf.html
was:
The Apache project has
[voted|https://lists.apache.org/thread/hz8fomzcwt2yhyz6l4ntzp3h0vpqd8xg] to
accept the software donation of the
[gradle-avro-plugin|https://github.com/davidmc24/gradle-avro-plugin/discussions/208]
graciously donated by the maintainer (David M. Carr). Thanks!
This umbrella Jira is tracking the steps we need to take to integrate the code
into the project.
We already have the Software Grant Agreement to cover the incoming code, which
is already licensed under the Apache 2.0 license with source code headers
including {{Copyright © 2013-2019 Commerce Technologies, LLC.}}
> Integrate software donation of gradle-avro-plugin
> -
>
> Key: AVRO-3731
> URL: https://issues.apache.org/jira/browse/AVRO-3731
> Project: Apache Avro
> Issue Type: New Feature
>Affects Versions: 1.12.0
>Reporter: Ryan Skraba
>Priority: Major
> Fix For: 1.12.0
>
>
> The Apache project has
> [voted|https://lists.apache.org/thread/hz8fomzcwt2yhyz6l4ntzp3h0vpqd8xg] to
> accept the software donation of the
> [gradle-avro-plugin|https://github.com/davidmc24/gradle-avro-plugin/discussions/208]
> graciously donated by the maintainer (David M. Carr). Thanks!
> This umbrella Jira is tracking the steps we need to take to integrate the
> code into the project.
> We already have the Software Grant Agreement to cover the incoming code,
> which is already licensed under the Apache 2.0 license with source code
> headers including {{Copyright © 2013-2019 Commerce Technologies, LLC.}}
> Some useful resources for the process are provided by the incubator:
> * https://incubator.apache.org/guides/transitioning_asf.html
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3732) Code drop existing sources, updating headers, RAT, NOTICES, etc.
[ https://issues.apache.org/jira/browse/AVRO-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3732: -- Description: Move the source to https://github.com/apache/avro/tree/master/lang/java in a directory called gradle-plugin (parallel to maven-plugin). To decide -- should we do a code dump to the destination or attempt to maintain the history from the original repo? was:Move the source to https://github.com/apache/avro/tree/master/lang/java in a directory called gradle-plugin (parallel to maven-plugin). > Code drop existing sources, updating headers, RAT, NOTICES, etc. > > > Key: AVRO-3732 > URL: https://issues.apache.org/jira/browse/AVRO-3732 > Project: Apache Avro > Issue Type: Sub-task >Reporter: Ryan Skraba >Priority: Major > > Move the source to https://github.com/apache/avro/tree/master/lang/java in a > directory called gradle-plugin (parallel to maven-plugin). > To decide -- should we do a code dump to the destination or attempt to > maintain the history from the original repo? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3732) Code drop existing sources, updating headers, RAT, NOTICES, etc.
[ https://issues.apache.org/jira/browse/AVRO-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3732: -- Description: Move the source to https://github.com/apache/avro/tree/master/lang/java in a directory called gradle-plugin (parallel to maven-plugin). > Code drop existing sources, updating headers, RAT, NOTICES, etc. > > > Key: AVRO-3732 > URL: https://issues.apache.org/jira/browse/AVRO-3732 > Project: Apache Avro > Issue Type: Sub-task >Reporter: Ryan Skraba >Priority: Major > > Move the source to https://github.com/apache/avro/tree/master/lang/java in a > directory called gradle-plugin (parallel to maven-plugin). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3731) Integrate software donation of gradle-avro-plugin
[
https://issues.apache.org/jira/browse/AVRO-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3731:
--
Fix Version/s: 1.12.0
> Integrate software donation of gradle-avro-plugin
> -
>
> Key: AVRO-3731
> URL: https://issues.apache.org/jira/browse/AVRO-3731
> Project: Apache Avro
> Issue Type: New Feature
>Affects Versions: 1.12.0
>Reporter: Ryan Skraba
>Priority: Major
> Fix For: 1.12.0
>
>
> The Apache project has
> [voted|https://lists.apache.org/thread/hz8fomzcwt2yhyz6l4ntzp3h0vpqd8xg] to
> accept the software donation of the
> [gradle-avro-plugin|https://github.com/davidmc24/gradle-avro-plugin/discussions/208]
> graciously donated by the maintainer (David M. Carr). Thanks!
> This umbrella Jira is tracking the steps we need to take to integrate the
> code into the project.
> We already have the Software Grant Agreement to cover the incoming code,
> which is already licensed under the Apache 2.0 license with source code
> headers including {{Copyright © 2013-2019 Commerce Technologies, LLC.}}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (AVRO-3783) [Java] Deserialize byte lengths as LONG but limit to INT
[
https://issues.apache.org/jira/browse/AVRO-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba closed AVRO-3783.
-
> [Java] Deserialize byte lengths as LONG but limit to INT
>
>
> Key: AVRO-3783
> URL: https://issues.apache.org/jira/browse/AVRO-3783
> Project: Apache Avro
> Issue Type: Bug
>Reporter: Jack Klamer
>Assignee: Jack Klamer
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the spec, the encoding for the bytes type is bytes are encoded as a long
> followed by that many bytes of data.. In the Java binary decoders they are
> read as ints this is not a correctness issue because the VLE of the long is
> the same as that of an equivalently valued int. The int is used (I assume) to
> enable easier interop with the ByteBuffer java class. But in the rare cases
> where validly encoded data of more than MAX_ARRAY_SIZE bytes is found, this
> change will cause an error of
> {code}
> throw new UnsupportedOperationException(
> "Cannot read arrays longer than " + MAX_ARRAY_SIZE + " bytes in
> Java library");
> {code}
> instead of
> {code}
> throw new InvalidNumberEncodingException("Invalid int encoding");
> {code}
> This type of checking is consistent with what happens in readString which
> reads its size as a long.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3783) [Java] Deserialize byte lengths as LONG but limit to INT
[
https://issues.apache.org/jira/browse/AVRO-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733634#comment-17733634
]
Ryan Skraba commented on AVRO-3783:
---
Cherry-picked to
[branch-1.11|https://github.com/apache/avro/commit/841d85e1fa11d2ac83b096253cb7980a4125f7cd].
> [Java] Deserialize byte lengths as LONG but limit to INT
>
>
> Key: AVRO-3783
> URL: https://issues.apache.org/jira/browse/AVRO-3783
> Project: Apache Avro
> Issue Type: Bug
>Reporter: Jack Klamer
>Assignee: Jack Klamer
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the spec, the encoding for the bytes type is bytes are encoded as a long
> followed by that many bytes of data.. In the Java binary decoders they are
> read as ints this is not a correctness issue because the VLE of the long is
> the same as that of an equivalently valued int. The int is used (I assume) to
> enable easier interop with the ByteBuffer java class. But in the rare cases
> where validly encoded data of more than MAX_ARRAY_SIZE bytes is found, this
> change will cause an error of
> {code}
> throw new UnsupportedOperationException(
> "Cannot read arrays longer than " + MAX_ARRAY_SIZE + " bytes in
> Java library");
> {code}
> instead of
> {code}
> throw new InvalidNumberEncodingException("Invalid int encoding");
> {code}
> This type of checking is consistent with what happens in readString which
> reads its size as a long.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (AVRO-3783) [Java] Deserialize byte lengths as LONG but limit to INT
[
https://issues.apache.org/jira/browse/AVRO-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3783.
---
Resolution: Fixed
> [Java] Deserialize byte lengths as LONG but limit to INT
>
>
> Key: AVRO-3783
> URL: https://issues.apache.org/jira/browse/AVRO-3783
> Project: Apache Avro
> Issue Type: Bug
>Reporter: Jack Klamer
>Assignee: Jack Klamer
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In the spec, the encoding for the bytes type is bytes are encoded as a long
> followed by that many bytes of data.. In the Java binary decoders they are
> read as ints this is not a correctness issue because the VLE of the long is
> the same as that of an equivalently valued int. The int is used (I assume) to
> enable easier interop with the ByteBuffer java class. But in the rare cases
> where validly encoded data of more than MAX_ARRAY_SIZE bytes is found, this
> change will cause an error of
> {code}
> throw new UnsupportedOperationException(
> "Cannot read arrays longer than " + MAX_ARRAY_SIZE + " bytes in
> Java library");
> {code}
> instead of
> {code}
> throw new InvalidNumberEncodingException("Invalid int encoding");
> {code}
> This type of checking is consistent with what happens in readString which
> reads its size as a long.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (AVRO-3783) [Java] Deserialize byte lengths as LONG but limit to INT
[
https://issues.apache.org/jira/browse/AVRO-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba reassigned AVRO-3783:
-
> [Java] Deserialize byte lengths as LONG but limit to INT
>
>
> Key: AVRO-3783
> URL: https://issues.apache.org/jira/browse/AVRO-3783
> Project: Apache Avro
> Issue Type: Bug
>Reporter: Jack Klamer
>Assignee: Jack Klamer
>Priority: Major
> Fix For: 1.11.2
>
>
> In the spec, the encoding for the bytes type is bytes are encoded as a long
> followed by that many bytes of data.. In the Java binary decoders they are
> read as ints this is not a correctness issue because the VLE of the long is
> the same as that of an equivalently valued int. The int is used (I assume) to
> enable easier interop with the ByteBuffer java class. But in the rare cases
> where validly encoded data of more than MAX_ARRAY_SIZE bytes is found, this
> change will cause an error of
> {code}
> throw new UnsupportedOperationException(
> "Cannot read arrays longer than " + MAX_ARRAY_SIZE + " bytes in
> Java library");
> {code}
> instead of
> {code}
> throw new InvalidNumberEncodingException("Invalid int encoding");
> {code}
> This type of checking is consistent with what happens in readString which
> reads its size as a long.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (AVRO-3775) [Ruby] decimal default is not converted to BigDecimal
[
https://issues.apache.org/jira/browse/AVRO-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba reassigned AVRO-3775:
-
Assignee: Rich
> [Ruby] decimal default is not converted to BigDecimal
> -
>
> Key: AVRO-3775
> URL: https://issues.apache.org/jira/browse/AVRO-3775
> Project: Apache Avro
> Issue Type: Bug
> Components: ruby
>Affects Versions: 1.11.1
>Reporter: Rich
>Assignee: Rich
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> *Background*
> https://issues.apache.org/jira/browse/AVRO-3773 is to fix the validation of
> decimal default
> After it is fixed (or we skip it), we are going to decode it.
> the default value is used when resolving schema resolution.
> *Expect*
> the decoded record should have default value in BigDecimal type, e.g.
> {code:java}
> {"sales" => BigDecimal("12.34"), "tax" => BigDecimal("0.000")} {code}
> *Actual*
> the decoded record have default value in string/bytes type
> {code:java}
> {"sales" => BigDecimal("12.34"), "tax" => "\u"} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (AVRO-3750) Python Optional UUIDs are not allowed
[
https://issues.apache.org/jira/browse/AVRO-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3750.
---
Resolution: Duplicate
> Python Optional UUIDs are not allowed
> -
>
> Key: AVRO-3750
> URL: https://issues.apache.org/jira/browse/AVRO-3750
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
>Reporter: Daniel Young
>Priority: Major
>
> If a union type of null and logical type of uuid is used, Avro will try to
> decode the object as a UUID and create a TypeError exception, which is not
> caught.
> Example Schema:
> {code:java}
> {
> "name": "userId",
> "type": [
> "null",
> {
> "type": "string",
> "logicalType": "uuid"
> }
> ]
> }{code}
> if the example schema is given a userId of None then the application will
> stop with an uncaught exception.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3750) Python Optional UUIDs are not allowed
[
https://issues.apache.org/jira/browse/AVRO-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733627#comment-17733627
]
Ryan Skraba commented on AVRO-3750:
---
Oh hey! Thanks Daniel -- I didn't see this JIRA, but I saw AVRO-3761 go by
with an identical fix to yours! Thanks for pointing it out, this should be
released in the next release.
> Python Optional UUIDs are not allowed
> -
>
> Key: AVRO-3750
> URL: https://issues.apache.org/jira/browse/AVRO-3750
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
>Reporter: Daniel Young
>Priority: Major
>
> If a union type of null and logical type of uuid is used, Avro will try to
> decode the object as a UUID and create a TypeError exception, which is not
> caught.
> Example Schema:
> {code:java}
> {
> "name": "userId",
> "type": [
> "null",
> {
> "type": "string",
> "logicalType": "uuid"
> }
> ]
> }{code}
> if the example schema is given a userId of None then the application will
> stop with an uncaught exception.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3772) [Rust] Deserialize Errors for an Unknown Enum Symbol instead of Returning Default
[
https://issues.apache.org/jira/browse/AVRO-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733623#comment-17733623
]
Ryan Skraba commented on AVRO-3772:
---
Hello! After a long (LONG) delay, I'm really interested in getting the 1.11.2
release candidate out! Is someone interested in continuing [~mgrigorov]'s work
in the meantime?
There's a couple of alternatives, all of them are fine!
1. Some can fix this forward quickly enough to make it into 1.11.2
2. We leave this as-is as a known issue with Schema resolution in the Rust SDK
(which was also broken before, and it appears to me that the current state is
not worse).
3. We revert the original fix and keep the 1.11.1 behaviour with this known
issue.
4. We block the release until this is fixed.
I'm leaning towards (2) or (4) -- I don't think that I'll be learning Rust
quickly enough to be useful for (1) ! :D
> [Rust] Deserialize Errors for an Unknown Enum Symbol instead of Returning
> Default
> -
>
> Key: AVRO-3772
> URL: https://issues.apache.org/jira/browse/AVRO-3772
> Project: Apache Avro
> Issue Type: Bug
> Components: rust
>Affects Versions: 1.9.0, 1.10.0, 1.9.1, 1.9.2, 1.11.0, 1.10.1, 1.10.2,
> 1.11.1
>Reporter: Evan Blackwell
>Assignee: Martin Tzvetanov Grigorov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The rust implementation of avro appears to behave according to an old
> specification when deserializing messages with a symbol in the writer's enum
> that is not in the reader's enum. The deserializer is returning an error even
> if the reader schema specifies a default. Starting in version 1.9.0, the
> default should be used in this situation according to this section of the
> documentation about schema resolution:
> [https://avro.apache.org/docs/current/spec.html#Schema+Resolution]
> {code:java}
> if both are enums:
> if the writer's symbol is not present in the reader's enum and the reader has
> a default value, then that value is used, otherwise an error is signalled.
> {code}
>
> This test currently in master expects the deserializer to error because it
> was written 5 years ago before spec 1.9.0 was released. (summary of the test:
> writer has symbol "clubs," the reader does not have the "clubs" symbol but
> does have a default, record gets made with symbol "clubs," the result is an
> error)
> [https://github.com/apache/avro/blob/6f4162e3d71e602bc398563b102d569846d5f39f/lang/rust/avro/src/lib.rs#L871]
>
> {code:java}
> #[test]
> fn test_enum_resolution() {
> let writer_raw_schema = r#"
> {
> "type": "record",
> "name": "test",
> "fields": [
> {"name": "a", "type": "long", "default": 42},
> {"name": "b", "type": "string"},
> {
> "name": "c",
> "type": {
> "type": "enum",
> "name": "suit",
> "symbols": ["diamonds", "spades", "clubs", "hearts"]
> },
> "default": "spades"
> }
> ]
> }
> "#;
> let reader_raw_schema = r#"
> {
> "type": "record",
> "name": "test",
> "fields": [
> {"name": "a", "type": "long", "default": 42},
> {"name": "b", "type": "string"},
> {
> "name": "c",
> "type": {
> "type": "enum",
> "name": "suit",
> "symbols": ["diamonds", "spades", "ninja",
> "hearts"]
> },
> "default": "spades"
> }
> ]
> }
> "#;
> let writer_schema = Schema::parse_str(writer_raw_schema).unwrap();
> let reader_schema = Schema::parse_str(reader_raw_schema).unwrap();
> let mut writer = Writer::with_codec(_schema, Vec::new(),
> Codec::Null);
> let mut record = Record::new(writer.schema()).unwrap();
> record.put("a", 27i64);
> record.put("b", "foo");
> record.put("c", "clubs");
> writer.append(record).unwrap();
> let input = writer.into_inner().unwrap();
> let mut reader = Reader::with_schema(_schema, [..]).unwrap();
> assert!(reader.next().unwrap().is_err());
> assert!(reader.next().is_none());
> } {code}
>
>
> If the deserializer was correctly using the default, I would expect the last
> two lines to instead assert the first two values were as expected with c
> defaulting to "spades"
>
> {code:java}
> assert_eq!(
>
[jira] [Closed] (AVRO-3761) UUID validation breaks nullable field
[
https://issues.apache.org/jira/browse/AVRO-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba closed AVRO-3761.
-
> UUID validation breaks nullable field
> -
>
> Key: AVRO-3761
> URL: https://issues.apache.org/jira/browse/AVRO-3761
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code:perl}
> $ pip freeze | grep avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Using logical type UUID for a nullable field breaks when writing `None`.
> {code:python}
> import io
> import uuid
> from avro.io import DatumWriter, BinaryEncoder
> import avro.schema
> schema = avro.schema.parse("""
> {
> "name": "Metadata",
> "type": "record",
> "fields": [
> {
> "name": "topic_id",
> "type": [
> {
> "logicalType": "uuid",
> "type": "string"
> },
> "null"
> ]
> }
> ]
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(schema)
> encoder = BinaryEncoder(buffer)
> # This works.
> writer.write({"topic_id": uuid.uuid4().hex}, encoder)
> # This breaks.
> writer.write({"topic_id": None}, encoder)
> {code}
> Results in:
> {code:perl}
> Traceback (most recent call last):
> File "reproduce-avro-uuid.py", line 32, in
> writer.write({"topic_id": None}, encoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 1013, in write
> validate(self.writers_schema, datum, raise_on_error=True)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 142, in validate
> validated_schema = current_node.schema.validate(current_node.datum)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/schema.py",
> line 801, in validate
> if branch.validate(datum) is not None:
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/schema.py",
> line 1048, in validate
> val = uuid.UUID(datum)
> File "/Users/anton/.pyenv/versions/3.8.16/lib/python3.8/uuid.py", line 165,
> in __init__
> raise TypeError('one of the hex, bytes, bytes_le, fields, '
> TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be
> given
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3761) UUID validation breaks nullable field
[
https://issues.apache.org/jira/browse/AVRO-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3761:
--
Fix Version/s: 1.11.2
> UUID validation breaks nullable field
> -
>
> Key: AVRO-3761
> URL: https://issues.apache.org/jira/browse/AVRO-3761
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code:perl}
> $ pip freeze | grep avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Using logical type UUID for a nullable field breaks when writing `None`.
> {code:python}
> import io
> import uuid
> from avro.io import DatumWriter, BinaryEncoder
> import avro.schema
> schema = avro.schema.parse("""
> {
> "name": "Metadata",
> "type": "record",
> "fields": [
> {
> "name": "topic_id",
> "type": [
> {
> "logicalType": "uuid",
> "type": "string"
> },
> "null"
> ]
> }
> ]
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(schema)
> encoder = BinaryEncoder(buffer)
> # This works.
> writer.write({"topic_id": uuid.uuid4().hex}, encoder)
> # This breaks.
> writer.write({"topic_id": None}, encoder)
> {code}
> Results in:
> {code:perl}
> Traceback (most recent call last):
> File "reproduce-avro-uuid.py", line 32, in
> writer.write({"topic_id": None}, encoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 1013, in write
> validate(self.writers_schema, datum, raise_on_error=True)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 142, in validate
> validated_schema = current_node.schema.validate(current_node.datum)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/schema.py",
> line 801, in validate
> if branch.validate(datum) is not None:
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/schema.py",
> line 1048, in validate
> val = uuid.UUID(datum)
> File "/Users/anton/.pyenv/versions/3.8.16/lib/python3.8/uuid.py", line 165,
> in __init__
> raise TypeError('one of the hex, bytes, bytes_le, fields, '
> TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be
> given
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (AVRO-3761) UUID validation breaks nullable field
[
https://issues.apache.org/jira/browse/AVRO-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba reassigned AVRO-3761:
-
Assignee: Anton Agestam
> UUID validation breaks nullable field
> -
>
> Key: AVRO-3761
> URL: https://issues.apache.org/jira/browse/AVRO-3761
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code:perl}
> $ pip freeze | grep avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Using logical type UUID for a nullable field breaks when writing `None`.
> {code:python}
> import io
> import uuid
> from avro.io import DatumWriter, BinaryEncoder
> import avro.schema
> schema = avro.schema.parse("""
> {
> "name": "Metadata",
> "type": "record",
> "fields": [
> {
> "name": "topic_id",
> "type": [
> {
> "logicalType": "uuid",
> "type": "string"
> },
> "null"
> ]
> }
> ]
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(schema)
> encoder = BinaryEncoder(buffer)
> # This works.
> writer.write({"topic_id": uuid.uuid4().hex}, encoder)
> # This breaks.
> writer.write({"topic_id": None}, encoder)
> {code}
> Results in:
> {code:perl}
> Traceback (most recent call last):
> File "reproduce-avro-uuid.py", line 32, in
> writer.write({"topic_id": None}, encoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 1013, in write
> validate(self.writers_schema, datum, raise_on_error=True)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 142, in validate
> validated_schema = current_node.schema.validate(current_node.datum)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/schema.py",
> line 801, in validate
> if branch.validate(datum) is not None:
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/schema.py",
> line 1048, in validate
> val = uuid.UUID(datum)
> File "/Users/anton/.pyenv/versions/3.8.16/lib/python3.8/uuid.py", line 165,
> in __init__
> raise TypeError('one of the hex, bytes, bytes_le, fields, '
> TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be
> given
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (AVRO-3761) UUID validation breaks nullable field
[
https://issues.apache.org/jira/browse/AVRO-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3761.
---
Resolution: Fixed
> UUID validation breaks nullable field
> -
>
> Key: AVRO-3761
> URL: https://issues.apache.org/jira/browse/AVRO-3761
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code:perl}
> $ pip freeze | grep avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Using logical type UUID for a nullable field breaks when writing `None`.
> {code:python}
> import io
> import uuid
> from avro.io import DatumWriter, BinaryEncoder
> import avro.schema
> schema = avro.schema.parse("""
> {
> "name": "Metadata",
> "type": "record",
> "fields": [
> {
> "name": "topic_id",
> "type": [
> {
> "logicalType": "uuid",
> "type": "string"
> },
> "null"
> ]
> }
> ]
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(schema)
> encoder = BinaryEncoder(buffer)
> # This works.
> writer.write({"topic_id": uuid.uuid4().hex}, encoder)
> # This breaks.
> writer.write({"topic_id": None}, encoder)
> {code}
> Results in:
> {code:perl}
> Traceback (most recent call last):
> File "reproduce-avro-uuid.py", line 32, in
> writer.write({"topic_id": None}, encoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 1013, in write
> validate(self.writers_schema, datum, raise_on_error=True)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 142, in validate
> validated_schema = current_node.schema.validate(current_node.datum)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/schema.py",
> line 801, in validate
> if branch.validate(datum) is not None:
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/schema.py",
> line 1048, in validate
> val = uuid.UUID(datum)
> File "/Users/anton/.pyenv/versions/3.8.16/lib/python3.8/uuid.py", line 165,
> in __init__
> raise TypeError('one of the hex, bytes, bytes_le, fields, '
> TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be
> given
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3760) Using enum with default symbol, cannot parse future value
[
https://issues.apache.org/jira/browse/AVRO-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733178#comment-17733178
]
Ryan Skraba commented on AVRO-3760:
---
Hello! I think that your example code might be wrong, but the bug exists
regardless.
If I understand the intention of AVRO-1340 correctly, the test code must have
both the reader and writer schema present in order to use the default:
{code}
reader = DatumReader(future_schema, current_schema)
{code}
In this case you still get an exception:
**avro.errors.SchemaResolutionException: Symbol crc32_be not present in
Reader's Schema**
It's a bit vague to me, but my understanding is that the default in an enum is
meant to serve as the "fail safely" value when a symbol is removed during
schema evolution, but not when on corrupted or unexpected data (like an enum
index out of bounds). Would it suit your needs if the default symbol was only
used when both schemas are known?
To be clear, the bug is still valid, just the implementation would change.
> Using enum with default symbol, cannot parse future value
> -
>
> Key: AVRO-3760
> URL: https://issues.apache.org/jira/browse/AVRO-3760
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code}
> $ pip freeze | grep -i avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
> Fix For: 1.11.2
>
>
> It seems like support for default symbols is broken. In the example below,
> since I'm using default symbols, I expected to be able to add new values to
> the enum and see the default value when parsing using the old schema.
> {code:python}
> import io
> from avro.io import DatumReader, DatumWriter, BinaryDecoder, BinaryEncoder
> import avro.schema
> current_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> # Future schema adds the "crc32_be" symbol.
> future_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be",
> "crc32_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(future_schema)
> encoder = BinaryEncoder(buffer)
> writer.write({"checksum_algorithm": "crc32_be"}, encoder)
> buffer.seek(0)
> reader = DatumReader(current_schema)
> decoder = BinaryDecoder(buffer)
> decoded = reader.read(decoder)
> print(decoded)
> {code}
> Instead, this results in an exception:
> {code}
> Traceback (most recent call last):
> File "reproduce-avro.py", line 58, in
> decoded = reader.read(decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 649, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 727, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 922, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 720, in read_data
> return self.read_enum(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 779, in read_enum
> raise avro.errors.SchemaResolutionException(
> avro.errors.SchemaResolutionException: Can't access enum index 2 for enum
> with 2 symbols
> Writer's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> Reader's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
>
[jira] [Assigned] (AVRO-3760) Using enum with default symbol, cannot parse future value
[
https://issues.apache.org/jira/browse/AVRO-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba reassigned AVRO-3760:
-
Assignee: Anton Agestam
> Using enum with default symbol, cannot parse future value
> -
>
> Key: AVRO-3760
> URL: https://issues.apache.org/jira/browse/AVRO-3760
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code}
> $ pip freeze | grep -i avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
>
> It seems like support for default symbols is broken. In the example below,
> since I'm using default symbols, I expected to be able to add new values to
> the enum and see the default value when parsing using the old schema.
> {code:python}
> import io
> from avro.io import DatumReader, DatumWriter, BinaryDecoder, BinaryEncoder
> import avro.schema
> current_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> # Future schema adds the "crc32_be" symbol.
> future_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be",
> "crc32_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(future_schema)
> encoder = BinaryEncoder(buffer)
> writer.write({"checksum_algorithm": "crc32_be"}, encoder)
> buffer.seek(0)
> reader = DatumReader(current_schema)
> decoder = BinaryDecoder(buffer)
> decoded = reader.read(decoder)
> print(decoded)
> {code}
> Instead, this results in an exception:
> {code}
> Traceback (most recent call last):
> File "reproduce-avro.py", line 58, in
> decoded = reader.read(decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 649, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 727, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 922, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 720, in read_data
> return self.read_enum(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 779, in read_enum
> raise avro.errors.SchemaResolutionException(
> avro.errors.SchemaResolutionException: Can't access enum index 2 for enum
> with 2 symbols
> Writer's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> Reader's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3760) Using enum with default symbol, cannot parse future value
[
https://issues.apache.org/jira/browse/AVRO-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3760:
--
Fix Version/s: 1.11.2
> Using enum with default symbol, cannot parse future value
> -
>
> Key: AVRO-3760
> URL: https://issues.apache.org/jira/browse/AVRO-3760
> Project: Apache Avro
> Issue Type: Bug
> Components: python
>Affects Versions: 1.11.1
> Environment: {code}
> $ pip freeze | grep -i avro
> avro==1.11.1
> $ python --version
> Python 3.8.16
> {code}
>Reporter: Anton Agestam
>Assignee: Anton Agestam
>Priority: Major
> Fix For: 1.11.2
>
>
> It seems like support for default symbols is broken. In the example below,
> since I'm using default symbols, I expected to be able to add new values to
> the enum and see the default value when parsing using the old schema.
> {code:python}
> import io
> from avro.io import DatumReader, DatumWriter, BinaryDecoder, BinaryEncoder
> import avro.schema
> current_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> # Future schema adds the "crc32_be" symbol.
> future_schema = avro.schema.parse("""
> {
> "fields": [
> {
> "default": "unknown",
> "name": "checksum_algorithm",
> "type": {
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be",
> "crc32_be"
> ],
> "type": "enum",
> "default": "unknown"
> }
> }
> ],
> "name": "Metadata",
> "type": "record"
> }
> """)
> with io.BytesIO() as buffer:
> writer = DatumWriter(future_schema)
> encoder = BinaryEncoder(buffer)
> writer.write({"checksum_algorithm": "crc32_be"}, encoder)
> buffer.seek(0)
> reader = DatumReader(current_schema)
> decoder = BinaryDecoder(buffer)
> decoded = reader.read(decoder)
> print(decoded)
> {code}
> Instead, this results in an exception:
> {code}
> Traceback (most recent call last):
> File "reproduce-avro.py", line 58, in
> decoded = reader.read(decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 649, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 727, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 922, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 720, in read_data
> return self.read_enum(writers_schema, readers_schema, decoder)
> File
> "/Users/anton/.pyenv/versions/karapace/lib/python3.8/site-packages/avro/io.py",
> line 779, in read_enum
> raise avro.errors.SchemaResolutionException(
> avro.errors.SchemaResolutionException: Can't access enum index 2 for enum
> with 2 symbols
> Writer's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> Reader's Schema: {
> "type": "enum",
> "default": "unknown",
> "name": "ChecksumAlgorithm",
> "symbols": [
> "unknown",
> "xxhash3_64_be"
> ]
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (AVRO-3473) Automatically register Conversion classes
[
https://issues.apache.org/jira/browse/AVRO-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba closed AVRO-3473.
-
> Automatically register Conversion classes
>
>
> Key: AVRO-3473
> URL: https://issues.apache.org/jira/browse/AVRO-3473
> Project: Apache Avro
> Issue Type: Improvement
> Components: java, logical types
>Reporter: Oscar Westra van Holthe - Kind
>Assignee: Oscar Westra van Holthe - Kind
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.11.2
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Manually registering a {{Conversion}} is cumbersome, but necessary if
> creating the factory requires constructor parameters.
> For most cases though, a {{Conversion}} gets all the information it needs
> from the schema. This makes it a good candidate for a SPI (using the Java 6
> {{ServiceLoader}} class).
> This is the addendum to AVRO-3441, which does the same thing for
> {{{}LogicalTypeFactory{}}}.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3473) Automatically register Conversion classes
[
https://issues.apache.org/jira/browse/AVRO-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3473:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Automatically register Conversion classes
>
>
> Key: AVRO-3473
> URL: https://issues.apache.org/jira/browse/AVRO-3473
> Project: Apache Avro
> Issue Type: Improvement
> Components: java, logical types
>Reporter: Oscar Westra van Holthe - Kind
>Assignee: Oscar Westra van Holthe - Kind
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.11.2
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Manually registering a {{Conversion}} is cumbersome, but necessary if
> creating the factory requires constructor parameters.
> For most cases though, a {{Conversion}} gets all the information it needs
> from the schema. This makes it a good candidate for a SPI (using the Java 6
> {{ServiceLoader}} class).
> This is the addendum to AVRO-3441, which does the same thing for
> {{{}LogicalTypeFactory{}}}.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (AVRO-3560) avro ignores input after end of avsc json
[
https://issues.apache.org/jira/browse/AVRO-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3560.
---
Resolution: Fixed
Cherry-picked to
[branch-1.11|https://github.com/apache/avro/commit/41b248100774e145262cbee2122384f69f93b8c2].
It would be amazing if you could watch the [mailing
list|https://lists.apache.org/list?d...@avro.apache.org] for the release
candidate and validate it!
> avro ignores input after end of avsc json
> -
>
> Key: AVRO-3560
> URL: https://issues.apache.org/jira/browse/AVRO-3560
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.11.0
>Reporter: Radai Rosenblatt
>Assignee: Radai Rosenblatt
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> try the following unit test:
> {code}
> @Test
> public void littleBobbySchemas() throws Exception {
> Schema.Parser parser = new Schema.Parser();
> parser.setValidate(true);
> parser.setValidateDefaults(true);
> Schema schema = parser.parse("{\"type\": \"string\"}; DROP TABLE
> STUDENTS");
> Assert.assertNotNull(schema);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (AVRO-2943) Map comparison between Utf8 and String keys fails
[
https://issues.apache.org/jira/browse/AVRO-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732626#comment-17732626
]
Ryan Skraba edited comment on AVRO-2943 at 6/14/23 4:42 PM:
I've cherry-picked this to
[branch-1.11|https://github.com/apache/avro/commit/204dcd1f970458b83cc54672fcddafc6056de3bc]
(and
[here|https://github.com/apache/avro/commit/b0f40b218cf8879fc2f62c6f01ab420b8a8215b5]).
It would be amazing if you could watch the [mailing
list|https://lists.apache.org/list?d...@avro.apache.org] for the release
candidate and validate it! Thanks again for your patience and the
contribution.
was (Author: ryanskraba):
I've cherry-picked this to
[branch-1.11|https://github.com/apache/avro/commit/204dcd1f970458b83cc54672fcddafc6056de3bc]
(and
[here|https://github.com/apache/avro/commit/b0f40b218cf8879fc2f62c6f01ab420b8a8215b5]).
It would be amazing if you could watch the [mailing
list|https://lists.apache.org/list?d...@avro.apache.org] for the release
candidate and validate it! Thanks again for your patience and the
contribution.
> Map comparison between Utf8 and String keys fails
> -
>
> Key: AVRO-2943
> URL: https://issues.apache.org/jira/browse/AVRO-2943
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.10.0
> Environment: Mac OS Catalina 10.15.6
>
> openjdk version "1.8.0_265"
> OpenJDK Runtime Environment Corretto-8.265.01.1 (build 1.8.0_265-b01)
> OpenJDK 64-Bit Server VM Corretto-8.265.01.1 (build 25.265-b01, mixed mode)
>Reporter: Frank Grimes
>Assignee: Frank Grimes
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Attachments: AVRO-2943-frankgrimes97.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The following test I locally added to org.apache.avro.generic.TestGenericData
> on master demonstrates the problem:
> {code:java}
> @Test
> public void testMapKeyEquals() {
> Schema mapSchema = new Schema.Parser().parse("{\"type\": \"map\",
> \"values\": \"string\"}");
> Field myMapField = new Field("my_map", Schema.createMap(mapSchema), null,
> null);
> Schema schema = Schema.createRecord("my_record", "doc", "mytest", false);
> schema.setFields(Arrays.asList(myMapField));
> GenericRecord r0 = new GenericData.Record(schema);
> GenericRecord r1 = new GenericData.Record(schema);
> HashMap pair1 = new HashMap<>();
> pair1.put("keyOne", "valueOne");
> r0.put("my_map", pair1);
> HashMap pair2 = new HashMap<>();
> pair2.put(new Utf8("keyOne"), "valueOne");
> r1.put("my_map", pair2);
> assertEquals(r0, r1);
> assertEquals(r1, r0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3756) Support writing types back to the user in memory without writing files to disk
[ https://issues.apache.org/jira/browse/AVRO-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732631#comment-17732631 ] Ryan Skraba commented on AVRO-3756: --- Thanks for your contribution! Cherry-picked to [branch-1.11|https://github.com/apache/avro/commit/aa450f35caf4ed209018bf3a70fb9352989b936c]. It would be amazing if you could watch the [mailing list|https://lists.apache.org/list?d...@avro.apache.org] for the release candidate and validate it! Thanks again for your patience and the contribution. > Support writing types back to the user in memory without writing files to disk > -- > > Key: AVRO-3756 > URL: https://issues.apache.org/jira/browse/AVRO-3756 > Project: Apache Avro > Issue Type: Bug > Components: csharp >Reporter: Alexander Rosenfeld >Assignee: Alexander Rosenfeld >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.2 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > As per the title, Avro C# should support writing types directly back to the > user, instead of [forcing the user to write to > disk|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/CodeGen/CodeGen.cs#L1144]. > This is critical for use in scenarios where the code generation is done in a > CI job, where disk permissions might not be flexible. > > I already have code for this and intend to submit a patch along with this > ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (AVRO-2943) Map comparison between Utf8 and String keys fails
[
https://issues.apache.org/jira/browse/AVRO-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732626#comment-17732626
]
Ryan Skraba edited comment on AVRO-2943 at 6/14/23 4:40 PM:
I've cherry-picked this to
[branch-1.11|https://github.com/apache/avro/commit/204dcd1f970458b83cc54672fcddafc6056de3bc]
(and
[here|https://github.com/apache/avro/commit/b0f40b218cf8879fc2f62c6f01ab420b8a8215b5]).
It would be amazing if you could watch the [mailing
list|https://lists.apache.org/list?d...@avro.apache.org] for the release
candidate and validate it! Thanks again for your patience and the
contribution.
was (Author: ryanskraba):
I've cherry-picked this to
[branch-1.11|https://github.com/apache/avro/commit/204dcd1f970458b83cc54672fcddafc6056de3bc]
(and
[here|https://github.com/apache/avro/commit/b0f40b218cf8879fc2f62c6f01ab420b8a8215b5]).
It would be amazing if you could watch the [mailing
list|https://lists.apache.org/list?d...@avro.apache.org] for the release
candidate and validate it! Thanks again for your patience and the
contribution.
> Map comparison between Utf8 and String keys fails
> -
>
> Key: AVRO-2943
> URL: https://issues.apache.org/jira/browse/AVRO-2943
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.10.0
> Environment: Mac OS Catalina 10.15.6
>
> openjdk version "1.8.0_265"
> OpenJDK Runtime Environment Corretto-8.265.01.1 (build 1.8.0_265-b01)
> OpenJDK 64-Bit Server VM Corretto-8.265.01.1 (build 25.265-b01, mixed mode)
>Reporter: Frank Grimes
>Assignee: Frank Grimes
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Attachments: AVRO-2943-frankgrimes97.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The following test I locally added to org.apache.avro.generic.TestGenericData
> on master demonstrates the problem:
> {code:java}
> @Test
> public void testMapKeyEquals() {
> Schema mapSchema = new Schema.Parser().parse("{\"type\": \"map\",
> \"values\": \"string\"}");
> Field myMapField = new Field("my_map", Schema.createMap(mapSchema), null,
> null);
> Schema schema = Schema.createRecord("my_record", "doc", "mytest", false);
> schema.setFields(Arrays.asList(myMapField));
> GenericRecord r0 = new GenericData.Record(schema);
> GenericRecord r1 = new GenericData.Record(schema);
> HashMap pair1 = new HashMap<>();
> pair1.put("keyOne", "valueOne");
> r0.put("my_map", pair1);
> HashMap pair2 = new HashMap<>();
> pair2.put(new Utf8("keyOne"), "valueOne");
> r1.put("my_map", pair2);
> assertEquals(r0, r1);
> assertEquals(r1, r0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3756) Support writing types back to the user in memory without writing files to disk
[ https://issues.apache.org/jira/browse/AVRO-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3756: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Support writing types back to the user in memory without writing files to disk > -- > > Key: AVRO-3756 > URL: https://issues.apache.org/jira/browse/AVRO-3756 > Project: Apache Avro > Issue Type: Bug > Components: csharp >Reporter: Alexander Rosenfeld >Assignee: Alexander Rosenfeld >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.2 > > Time Spent: 3.5h > Remaining Estimate: 0h > > As per the title, Avro C# should support writing types directly back to the > user, instead of [forcing the user to write to > disk|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/CodeGen/CodeGen.cs#L1144]. > This is critical for use in scenarios where the code generation is done in a > CI job, where disk permissions might not be flexible. > > I already have code for this and intend to submit a patch along with this > ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (AVRO-3756) Support writing types back to the user in memory without writing files to disk
[ https://issues.apache.org/jira/browse/AVRO-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba closed AVRO-3756. - > Support writing types back to the user in memory without writing files to disk > -- > > Key: AVRO-3756 > URL: https://issues.apache.org/jira/browse/AVRO-3756 > Project: Apache Avro > Issue Type: Bug > Components: csharp >Reporter: Alexander Rosenfeld >Assignee: Alexander Rosenfeld >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.2 > > Time Spent: 3.5h > Remaining Estimate: 0h > > As per the title, Avro C# should support writing types directly back to the > user, instead of [forcing the user to write to > disk|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/CodeGen/CodeGen.cs#L1144]. > This is critical for use in scenarios where the code generation is done in a > CI job, where disk permissions might not be flexible. > > I already have code for this and intend to submit a patch along with this > ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (AVRO-3737) [C] memcheck_test_avro_commons_schema is failing
[
https://issues.apache.org/jira/browse/AVRO-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba closed AVRO-3737.
-
> [C] memcheck_test_avro_commons_schema is failing
>
>
> Key: AVRO-3737
> URL: https://issues.apache.org/jira/browse/AVRO-3737
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.11.1, 1.12.0
>Reporter: Ryan Skraba
>Assignee: Christophe Le Saec
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Even with the fix in AVRO-3454, one of the memcheck test is failing in the
> release docker:
> *{{cd lang/c && ./build.sh test}}*
> {code}
> ... (snip) ...
> 98% tests passed, 1 tests failed out of 48
> Total Test time (real) = 20.64 sec
> The following tests FAILED:
> 7 - memcheck_test_avro_commons_schema (Failed)
> Errors while running CTest
> make: *** [Makefile:85: test] Error 8
> {code}
> WIth a bit more detail in the logs:
> *{{tail -n 25 ../../build/c/tests/memcheck_test_avro_commons_schema.log}}*
> {code}
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== 172,133 (131,232 direct, 40,901 indirect) bytes in 2 blocks are
> definitely lost in loss record 80 of 80
> ==1209==at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:211)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:196)
> ==1209==by 0x1107A5: avro_file_writer_create (datafile.c:179)
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== LEAK SUMMARY:
> ==1209==definitely lost: 131,392 bytes in 4 blocks
> ==1209==indirectly lost: 57,315 bytes in 146 blocks
> ==1209== possibly lost: 0 bytes in 0 blocks
> ==1209==still reachable: 1,888 bytes in 4 blocks
> ==1209== suppressed: 0 bytes in 0 blocks
> ==1209==
> ==1209== For lists of detected and suppressed errors, rerun with: -s
> ==1209== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (AVRO-2943) Map comparison between Utf8 and String keys fails
[
https://issues.apache.org/jira/browse/AVRO-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba closed AVRO-2943.
-
> Map comparison between Utf8 and String keys fails
> -
>
> Key: AVRO-2943
> URL: https://issues.apache.org/jira/browse/AVRO-2943
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.10.0
> Environment: Mac OS Catalina 10.15.6
>
> openjdk version "1.8.0_265"
> OpenJDK Runtime Environment Corretto-8.265.01.1 (build 1.8.0_265-b01)
> OpenJDK 64-Bit Server VM Corretto-8.265.01.1 (build 25.265-b01, mixed mode)
>Reporter: Frank Grimes
>Assignee: Frank Grimes
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Attachments: AVRO-2943-frankgrimes97.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The following test I locally added to org.apache.avro.generic.TestGenericData
> on master demonstrates the problem:
> {code:java}
> @Test
> public void testMapKeyEquals() {
> Schema mapSchema = new Schema.Parser().parse("{\"type\": \"map\",
> \"values\": \"string\"}");
> Field myMapField = new Field("my_map", Schema.createMap(mapSchema), null,
> null);
> Schema schema = Schema.createRecord("my_record", "doc", "mytest", false);
> schema.setFields(Arrays.asList(myMapField));
> GenericRecord r0 = new GenericData.Record(schema);
> GenericRecord r1 = new GenericData.Record(schema);
> HashMap pair1 = new HashMap<>();
> pair1.put("keyOne", "valueOne");
> r0.put("my_map", pair1);
> HashMap pair2 = new HashMap<>();
> pair2.put(new Utf8("keyOne"), "valueOne");
> r1.put("my_map", pair2);
> assertEquals(r0, r1);
> assertEquals(r1, r0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (AVRO-2943) Map comparison between Utf8 and String keys fails
[
https://issues.apache.org/jira/browse/AVRO-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-2943.
---
Resolution: Fixed
> Map comparison between Utf8 and String keys fails
> -
>
> Key: AVRO-2943
> URL: https://issues.apache.org/jira/browse/AVRO-2943
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.10.0
> Environment: Mac OS Catalina 10.15.6
>
> openjdk version "1.8.0_265"
> OpenJDK Runtime Environment Corretto-8.265.01.1 (build 1.8.0_265-b01)
> OpenJDK 64-Bit Server VM Corretto-8.265.01.1 (build 25.265-b01, mixed mode)
>Reporter: Frank Grimes
>Assignee: Frank Grimes
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Attachments: AVRO-2943-frankgrimes97.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The following test I locally added to org.apache.avro.generic.TestGenericData
> on master demonstrates the problem:
> {code:java}
> @Test
> public void testMapKeyEquals() {
> Schema mapSchema = new Schema.Parser().parse("{\"type\": \"map\",
> \"values\": \"string\"}");
> Field myMapField = new Field("my_map", Schema.createMap(mapSchema), null,
> null);
> Schema schema = Schema.createRecord("my_record", "doc", "mytest", false);
> schema.setFields(Arrays.asList(myMapField));
> GenericRecord r0 = new GenericData.Record(schema);
> GenericRecord r1 = new GenericData.Record(schema);
> HashMap pair1 = new HashMap<>();
> pair1.put("keyOne", "valueOne");
> r0.put("my_map", pair1);
> HashMap pair2 = new HashMap<>();
> pair2.put(new Utf8("keyOne"), "valueOne");
> r1.put("my_map", pair2);
> assertEquals(r0, r1);
> assertEquals(r1, r0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-2943) Map comparison between Utf8 and String keys fails
[
https://issues.apache.org/jira/browse/AVRO-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732626#comment-17732626
]
Ryan Skraba commented on AVRO-2943:
---
I've cherry-picked this to
[branch-1.11|https://github.com/apache/avro/commit/204dcd1f970458b83cc54672fcddafc6056de3bc]
(and
[here|https://github.com/apache/avro/commit/b0f40b218cf8879fc2f62c6f01ab420b8a8215b5]).
It would be amazing if you could watch the [mailing
list|https://lists.apache.org/list?d...@avro.apache.org] for the release
candidate and validate it! Thanks again for your patience and the
contribution.
> Map comparison between Utf8 and String keys fails
> -
>
> Key: AVRO-2943
> URL: https://issues.apache.org/jira/browse/AVRO-2943
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.10.0
> Environment: Mac OS Catalina 10.15.6
>
> openjdk version "1.8.0_265"
> OpenJDK Runtime Environment Corretto-8.265.01.1 (build 1.8.0_265-b01)
> OpenJDK 64-Bit Server VM Corretto-8.265.01.1 (build 25.265-b01, mixed mode)
>Reporter: Frank Grimes
>Assignee: Frank Grimes
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Attachments: AVRO-2943-frankgrimes97.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The following test I locally added to org.apache.avro.generic.TestGenericData
> on master demonstrates the problem:
> {code:java}
> @Test
> public void testMapKeyEquals() {
> Schema mapSchema = new Schema.Parser().parse("{\"type\": \"map\",
> \"values\": \"string\"}");
> Field myMapField = new Field("my_map", Schema.createMap(mapSchema), null,
> null);
> Schema schema = Schema.createRecord("my_record", "doc", "mytest", false);
> schema.setFields(Arrays.asList(myMapField));
> GenericRecord r0 = new GenericData.Record(schema);
> GenericRecord r1 = new GenericData.Record(schema);
> HashMap pair1 = new HashMap<>();
> pair1.put("keyOne", "valueOne");
> r0.put("my_map", pair1);
> HashMap pair2 = new HashMap<>();
> pair2.put(new Utf8("keyOne"), "valueOne");
> r1.put("my_map", pair2);
> assertEquals(r0, r1);
> assertEquals(r1, r0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3737) [C] memcheck_test_avro_commons_schema is failing
[
https://issues.apache.org/jira/browse/AVRO-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732621#comment-17732621
]
Ryan Skraba commented on AVRO-3737:
---
As discussed, this has been cherry-picked to
[branch-1.11|http://example.com://github.com/apache/avro/commit/db70e1fa9706051fcaa26301a5d06b885138ac90].
Thanks for the contribution!
> [C] memcheck_test_avro_commons_schema is failing
>
>
> Key: AVRO-3737
> URL: https://issues.apache.org/jira/browse/AVRO-3737
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.11.1, 1.12.0
>Reporter: Ryan Skraba
>Assignee: Christophe Le Saec
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Even with the fix in AVRO-3454, one of the memcheck test is failing in the
> release docker:
> *{{cd lang/c && ./build.sh test}}*
> {code}
> ... (snip) ...
> 98% tests passed, 1 tests failed out of 48
> Total Test time (real) = 20.64 sec
> The following tests FAILED:
> 7 - memcheck_test_avro_commons_schema (Failed)
> Errors while running CTest
> make: *** [Makefile:85: test] Error 8
> {code}
> WIth a bit more detail in the logs:
> *{{tail -n 25 ../../build/c/tests/memcheck_test_avro_commons_schema.log}}*
> {code}
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== 172,133 (131,232 direct, 40,901 indirect) bytes in 2 blocks are
> definitely lost in loss record 80 of 80
> ==1209==at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:211)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:196)
> ==1209==by 0x1107A5: avro_file_writer_create (datafile.c:179)
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== LEAK SUMMARY:
> ==1209==definitely lost: 131,392 bytes in 4 blocks
> ==1209==indirectly lost: 57,315 bytes in 146 blocks
> ==1209== possibly lost: 0 bytes in 0 blocks
> ==1209==still reachable: 1,888 bytes in 4 blocks
> ==1209== suppressed: 0 bytes in 0 blocks
> ==1209==
> ==1209== For lists of detected and suppressed errors, rerun with: -s
> ==1209== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3737) [C] memcheck_test_avro_commons_schema is failing
[
https://issues.apache.org/jira/browse/AVRO-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3737:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> [C] memcheck_test_avro_commons_schema is failing
>
>
> Key: AVRO-3737
> URL: https://issues.apache.org/jira/browse/AVRO-3737
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.11.1, 1.12.0
>Reporter: Ryan Skraba
>Assignee: Christophe Le Saec
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Even with the fix in AVRO-3454, one of the memcheck test is failing in the
> release docker:
> *{{cd lang/c && ./build.sh test}}*
> {code}
> ... (snip) ...
> 98% tests passed, 1 tests failed out of 48
> Total Test time (real) = 20.64 sec
> The following tests FAILED:
> 7 - memcheck_test_avro_commons_schema (Failed)
> Errors while running CTest
> make: *** [Makefile:85: test] Error 8
> {code}
> WIth a bit more detail in the logs:
> *{{tail -n 25 ../../build/c/tests/memcheck_test_avro_commons_schema.log}}*
> {code}
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== 172,133 (131,232 direct, 40,901 indirect) bytes in 2 blocks are
> definitely lost in loss record 80 of 80
> ==1209==at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:211)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:196)
> ==1209==by 0x1107A5: avro_file_writer_create (datafile.c:179)
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== LEAK SUMMARY:
> ==1209==definitely lost: 131,392 bytes in 4 blocks
> ==1209==indirectly lost: 57,315 bytes in 146 blocks
> ==1209== possibly lost: 0 bytes in 0 blocks
> ==1209==still reachable: 1,888 bytes in 4 blocks
> ==1209== suppressed: 0 bytes in 0 blocks
> ==1209==
> ==1209== For lists of detected and suppressed errors, rerun with: -s
> ==1209== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3756) Support writing types back to the user in memory without writing files to disk
[ https://issues.apache.org/jira/browse/AVRO-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3756: -- Fix Version/s: 1.12.0 1.11.2 > Support writing types back to the user in memory without writing files to disk > -- > > Key: AVRO-3756 > URL: https://issues.apache.org/jira/browse/AVRO-3756 > Project: Apache Avro > Issue Type: Bug > Components: csharp >Reporter: Alexander Rosenfeld >Assignee: Alexander Rosenfeld >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.2 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > As per the title, Avro C# should support writing types directly back to the > user, instead of [forcing the user to write to > disk|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/CodeGen/CodeGen.cs#L1144]. > This is critical for use in scenarios where the code generation is done in a > CI job, where disk permissions might not be flexible. > > I already have code for this and intend to submit a patch along with this > ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (AVRO-3756) Support writing types back to the user in memory without writing files to disk
[ https://issues.apache.org/jira/browse/AVRO-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-3756: - Assignee: Alexander Rosenfeld > Support writing types back to the user in memory without writing files to disk > -- > > Key: AVRO-3756 > URL: https://issues.apache.org/jira/browse/AVRO-3756 > Project: Apache Avro > Issue Type: Bug > Components: csharp >Reporter: Alexander Rosenfeld >Assignee: Alexander Rosenfeld >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > As per the title, Avro C# should support writing types directly back to the > user, instead of [forcing the user to write to > disk|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/CodeGen/CodeGen.cs#L1144]. > This is critical for use in scenarios where the code generation is done in a > CI job, where disk permissions might not be flexible. > > I already have code for this and intend to submit a patch along with this > ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (AVRO-3774) next release? want python 3.11 support
[ https://issues.apache.org/jira/browse/AVRO-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba closed AVRO-3774. - > next release? want python 3.11 support > -- > > Key: AVRO-3774 > URL: https://issues.apache.org/jira/browse/AVRO-3774 > Project: Apache Avro > Issue Type: Wish >Reporter: t oo >Priority: Trivial > > when is next release? This is last lib holding me back from python 3.11 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (AVRO-3774) next release? want python 3.11 support
[ https://issues.apache.org/jira/browse/AVRO-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba resolved AVRO-3774. --- Resolution: Information Provided Hello! I finally got the 1.11.2 release into a better state, and heading towards a release candidate this week. The best place to follow the discussion for the release candidate is [https://lists.apache.org/thread/b47spn5h7fyhp8dso6zlsrls058fcby8] . Once a release candidate is out, there's a [VOTE] period for people to test it – it would be exceptionally useful if you were to follow the mailing list and check the avro-python artifacts for Python 3.11 ! > next release? want python 3.11 support > -- > > Key: AVRO-3774 > URL: https://issues.apache.org/jira/browse/AVRO-3774 > Project: Apache Avro > Issue Type: Wish >Reporter: t oo >Priority: Trivial > > when is next release? This is last lib holding me back from python 3.11 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3573) Duplicate symbols (EnumSchema)
[ https://issues.apache.org/jira/browse/AVRO-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3573: -- Fix Version/s: (was: 1.11.2) > Duplicate symbols (EnumSchema) > -- > > Key: AVRO-3573 > URL: https://issues.apache.org/jira/browse/AVRO-3573 > Project: Apache Avro > Issue Type: Improvement >Affects Versions: 1.11.0 >Reporter: Igor Izvekov >Priority: Minor > Labels: pull-request-available > Fix For: 1.12.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > If EnumSchema has duplicate symbols, an error will raise. Instead of a list > of duplicate symbols or a value of duplicate symbol, error shows all list of > symbols. Improvement removes this defect and shows a message "Duplicate > symbol" with the symbol, if it is one, or "Duplicates symbols" with the list > of duplicate symbols, if there are more than one symbol. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3637) [python] Add validate_enum_defaults to schema parsing.
[
https://issues.apache.org/jira/browse/AVRO-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3637:
--
Fix Version/s: 1.12.0
(was: 1.11.2)
> [python] Add validate_enum_defaults to schema parsing.
> --
>
> Key: AVRO-3637
> URL: https://issues.apache.org/jira/browse/AVRO-3637
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.11.1
>Reporter: Ryan Skraba
>Priority: Major
> Fix For: 1.12.0
>
>
> AVRO-2174 fixed a bug where invalid enum symbols were permitted. For
> backwards compatibility we added {{validate_enum_symbols=True}} [in this
> PR|https://github.com/apache/avro/pull/830/files#diff-206b4316aff1144af133de14f814399f776f1818ace77c035b60b98eb556bf5bR550].
> AVRO-3299 fixed a bug where invaliid [enum defaults were
> permitted|https://github.com/kojiromike/avro/commit/28f54cb35af3fab56985b710892cf9469aa7022d]
> (badly named or an unknown symbol). The same type of escape route should
> exist for this case (perhaps reusing validate_enum_symbols?)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-2943) Map comparison between Utf8 and String keys fails
[
https://issues.apache.org/jira/browse/AVRO-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730139#comment-17730139
]
Ryan Skraba commented on AVRO-2943:
---
Hello! Thanks for reaching out – this almost certainly would not have been
backported if you hadn't. I'm going to set the Fix Version and leave this open
until it's been cherry-picked. It's not too late, I'm wrapping up the list of
cherry-picks to propose for an RC!
> Map comparison between Utf8 and String keys fails
> -
>
> Key: AVRO-2943
> URL: https://issues.apache.org/jira/browse/AVRO-2943
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.10.0
> Environment: Mac OS Catalina 10.15.6
>
> openjdk version "1.8.0_265"
> OpenJDK Runtime Environment Corretto-8.265.01.1 (build 1.8.0_265-b01)
> OpenJDK 64-Bit Server VM Corretto-8.265.01.1 (build 25.265-b01, mixed mode)
>Reporter: Frank Grimes
>Assignee: Frank Grimes
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Attachments: AVRO-2943-frankgrimes97.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The following test I locally added to org.apache.avro.generic.TestGenericData
> on master demonstrates the problem:
> {code:java}
> @Test
> public void testMapKeyEquals() {
> Schema mapSchema = new Schema.Parser().parse("{\"type\": \"map\",
> \"values\": \"string\"}");
> Field myMapField = new Field("my_map", Schema.createMap(mapSchema), null,
> null);
> Schema schema = Schema.createRecord("my_record", "doc", "mytest", false);
> schema.setFields(Arrays.asList(myMapField));
> GenericRecord r0 = new GenericData.Record(schema);
> GenericRecord r1 = new GenericData.Record(schema);
> HashMap pair1 = new HashMap<>();
> pair1.put("keyOne", "valueOne");
> r0.put("my_map", pair1);
> HashMap pair2 = new HashMap<>();
> pair2.put(new Utf8("keyOne"), "valueOne");
> r1.put("my_map", pair2);
> assertEquals(r0, r1);
> assertEquals(r1, r0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-2943) Map comparison between Utf8 and String keys fails
[
https://issues.apache.org/jira/browse/AVRO-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-2943:
--
Fix Version/s: 1.12.0
1.11.2
> Map comparison between Utf8 and String keys fails
> -
>
> Key: AVRO-2943
> URL: https://issues.apache.org/jira/browse/AVRO-2943
> Project: Apache Avro
> Issue Type: Bug
> Components: java
>Affects Versions: 1.10.0
> Environment: Mac OS Catalina 10.15.6
>
> openjdk version "1.8.0_265"
> OpenJDK Runtime Environment Corretto-8.265.01.1 (build 1.8.0_265-b01)
> OpenJDK 64-Bit Server VM Corretto-8.265.01.1 (build 25.265-b01, mixed mode)
>Reporter: Frank Grimes
>Assignee: Frank Grimes
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Attachments: AVRO-2943-frankgrimes97.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> The following test I locally added to org.apache.avro.generic.TestGenericData
> on master demonstrates the problem:
> {code:java}
> @Test
> public void testMapKeyEquals() {
> Schema mapSchema = new Schema.Parser().parse("{\"type\": \"map\",
> \"values\": \"string\"}");
> Field myMapField = new Field("my_map", Schema.createMap(mapSchema), null,
> null);
> Schema schema = Schema.createRecord("my_record", "doc", "mytest", false);
> schema.setFields(Arrays.asList(myMapField));
> GenericRecord r0 = new GenericData.Record(schema);
> GenericRecord r1 = new GenericData.Record(schema);
> HashMap pair1 = new HashMap<>();
> pair1.put("keyOne", "valueOne");
> r0.put("my_map", pair1);
> HashMap pair2 = new HashMap<>();
> pair2.put(new Utf8("keyOne"), "valueOne");
> r1.put("my_map", pair2);
> assertEquals(r0, r1);
> assertEquals(r1, r0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3700) Publish Java SBOM artifacts with CycloneDX
[ https://issues.apache.org/jira/browse/AVRO-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3700: -- Fix Version/s: 1.11.2 > Publish Java SBOM artifacts with CycloneDX > -- > > Key: AVRO-3700 > URL: https://issues.apache.org/jira/browse/AVRO-3700 > Project: Apache Avro > Issue Type: Sub-task > Components: build >Affects Versions: 1.12.0 >Reporter: Dongjoon Hyun >Assignee: Dongjoon Hyun >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.2 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > h3. Why are the changes needed? > Here is an article to give some context. > - > [https://www.activestate.com/blog/why-the-us-government-is-mandating-software-bill-of-materials-sbom/] > Software Bill of Materials (SBOM) are additional artifacts containing the > aggregate of all direct and transitive dependencies of a project. The US > Government (based on NIST recommendations) currently accepts only the three > most popular SBOM standards as valid, namely: > [CycloneDX]([https://cyclonedx.org/]), [Software Identification (SWID) > tag]([https://csrc.nist.gov/projects/Software-Identification-SWID]), > [Software Package Data Exchange® (SPDX)]([https://spdx.dev/]). > This PR uses [CycloneDX maven > plugin]([https://github.com/CycloneDX/cyclonedx-maven-plugin]), a lightweight > software bill of materials (SBOM) standard designed for use in application > security contexts and supply chain component analysis. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (AVRO-3670) Add .NET 7 support
[ https://issues.apache.org/jira/browse/AVRO-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba reassigned AVRO-3670: - Assignee: Zoltan Csizmadia > Add .NET 7 support > -- > > Key: AVRO-3670 > URL: https://issues.apache.org/jira/browse/AVRO-3670 > Project: Apache Avro > Issue Type: Improvement > Components: csharp >Reporter: Zoltan Csizmadia >Assignee: Zoltan Csizmadia >Priority: Minor > Labels: pull-request-available > Fix For: 1.12.0 > > Original Estimate: 24h > Time Spent: 3h 50m > Remaining Estimate: 20h 10m > > .NET 7.0 SDK is released > [https://dotnet.microsoft.com/en-us/download/dotnet/7.0] > Add support for building and testing using NET 7 SDK as well. Published > class libraries will be still target netstandard2.0 and netstandard2.1, > however published tools and executables (e.g. AvroGen) will traget . NET 7 as > well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (AVRO-3670) Add .NET 7 support
[ https://issues.apache.org/jira/browse/AVRO-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726353#comment-17726353 ] Ryan Skraba commented on AVRO-3670: --- Hello! I noticed this was merged to master – is the Jira finished or is there remaining work to be done? Should this be back-ported to branch-1.11 for a 1.11.2 release? > Add .NET 7 support > -- > > Key: AVRO-3670 > URL: https://issues.apache.org/jira/browse/AVRO-3670 > Project: Apache Avro > Issue Type: Improvement > Components: csharp >Reporter: Zoltan Csizmadia >Assignee: Zoltan Csizmadia >Priority: Minor > Labels: pull-request-available > Fix For: 1.12.0 > > Original Estimate: 24h > Time Spent: 3h 50m > Remaining Estimate: 20h 10m > > .NET 7.0 SDK is released > [https://dotnet.microsoft.com/en-us/download/dotnet/7.0] > Add support for building and testing using NET 7 SDK as well. Published > class libraries will be still target netstandard2.0 and netstandard2.1, > however published tools and executables (e.g. AvroGen) will traget . NET 7 as > well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (AVRO-3670) Add .NET 7 support
[ https://issues.apache.org/jira/browse/AVRO-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Skraba updated AVRO-3670: -- Fix Version/s: 1.12.0 > Add .NET 7 support > -- > > Key: AVRO-3670 > URL: https://issues.apache.org/jira/browse/AVRO-3670 > Project: Apache Avro > Issue Type: Improvement > Components: csharp >Reporter: Zoltan Csizmadia >Priority: Minor > Labels: pull-request-available > Fix For: 1.12.0 > > Original Estimate: 24h > Time Spent: 3h 50m > Remaining Estimate: 20h 10m > > .NET 7.0 SDK is released > [https://dotnet.microsoft.com/en-us/download/dotnet/7.0] > Add support for building and testing using NET 7 SDK as well. Published > class libraries will be still target netstandard2.0 and netstandard2.1, > however published tools and executables (e.g. AvroGen) will traget . NET 7 as > well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (AVRO-3644) [JAVA] Support java.util.Optional in reflect package
[
https://issues.apache.org/jira/browse/AVRO-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba resolved AVRO-3644.
---
Resolution: Fixed
> [JAVA] Support java.util.Optional in reflect package
>
>
> Key: AVRO-3644
> URL: https://issues.apache.org/jira/browse/AVRO-3644
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: Daniel Heinrich
>Assignee: Daniel Heinrich
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> java.util.Optional can currently not be used at all when using reflection
> based schemas.
> _org.apache.avro.reflect.ReflectData#getSchema_ will currently produce the
> following type for fields using Optional:
> {"type":"record","name":"Optional","namespace":"java.util","fields":[]}
> The generated schema is completly unusable (it does not contain any data). I
> would like to propose to map Optional fields as if they were not using
> Optional and would be annotated with @Nullable. So resulting in the following
> type:
> ["null","something"]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (AVRO-3737) [C] memcheck_test_avro_commons_schema is failing
[
https://issues.apache.org/jira/browse/AVRO-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720344#comment-17720344
]
Ryan Skraba commented on AVRO-3737:
---
Thanks for the fix! This needs to be in 1.11.2, but I'm currently working on a
backlog of cherry-picks that I'd like to do in the same order as they occurred
on master... I'll close this when I arrive at the end :D
> [C] memcheck_test_avro_commons_schema is failing
>
>
> Key: AVRO-3737
> URL: https://issues.apache.org/jira/browse/AVRO-3737
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.11.1, 1.12.0
>Reporter: Ryan Skraba
>Assignee: Christophe Le Saec
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Even with the fix in AVRO-3454, one of the memcheck test is failing in the
> release docker:
> *{{cd lang/c && ./build.sh test}}*
> {code}
> ... (snip) ...
> 98% tests passed, 1 tests failed out of 48
> Total Test time (real) = 20.64 sec
> The following tests FAILED:
> 7 - memcheck_test_avro_commons_schema (Failed)
> Errors while running CTest
> make: *** [Makefile:85: test] Error 8
> {code}
> WIth a bit more detail in the logs:
> *{{tail -n 25 ../../build/c/tests/memcheck_test_avro_commons_schema.log}}*
> {code}
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== 172,133 (131,232 direct, 40,901 indirect) bytes in 2 blocks are
> definitely lost in loss record 80 of 80
> ==1209==at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:211)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:196)
> ==1209==by 0x1107A5: avro_file_writer_create (datafile.c:179)
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== LEAK SUMMARY:
> ==1209==definitely lost: 131,392 bytes in 4 blocks
> ==1209==indirectly lost: 57,315 bytes in 146 blocks
> ==1209== possibly lost: 0 bytes in 0 blocks
> ==1209==still reachable: 1,888 bytes in 4 blocks
> ==1209== suppressed: 0 bytes in 0 blocks
> ==1209==
> ==1209== For lists of detected and suppressed errors, rerun with: -s
> ==1209== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3737) [C] memcheck_test_avro_commons_schema is failing
[
https://issues.apache.org/jira/browse/AVRO-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3737:
--
Fix Version/s: 1.11.1
> [C] memcheck_test_avro_commons_schema is failing
>
>
> Key: AVRO-3737
> URL: https://issues.apache.org/jira/browse/AVRO-3737
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.11.1, 1.12.0
>Reporter: Ryan Skraba
>Assignee: Christophe Le Saec
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.11.1, 1.12.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Even with the fix in AVRO-3454, one of the memcheck test is failing in the
> release docker:
> *{{cd lang/c && ./build.sh test}}*
> {code}
> ... (snip) ...
> 98% tests passed, 1 tests failed out of 48
> Total Test time (real) = 20.64 sec
> The following tests FAILED:
> 7 - memcheck_test_avro_commons_schema (Failed)
> Errors while running CTest
> make: *** [Makefile:85: test] Error 8
> {code}
> WIth a bit more detail in the logs:
> *{{tail -n 25 ../../build/c/tests/memcheck_test_avro_commons_schema.log}}*
> {code}
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== 172,133 (131,232 direct, 40,901 indirect) bytes in 2 blocks are
> definitely lost in loss record 80 of 80
> ==1209==at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:211)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:196)
> ==1209==by 0x1107A5: avro_file_writer_create (datafile.c:179)
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== LEAK SUMMARY:
> ==1209==definitely lost: 131,392 bytes in 4 blocks
> ==1209==indirectly lost: 57,315 bytes in 146 blocks
> ==1209== possibly lost: 0 bytes in 0 blocks
> ==1209==still reachable: 1,888 bytes in 4 blocks
> ==1209== suppressed: 0 bytes in 0 blocks
> ==1209==
> ==1209== For lists of detected and suppressed errors, rerun with: -s
> ==1209== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3737) [C] memcheck_test_avro_commons_schema is failing
[
https://issues.apache.org/jira/browse/AVRO-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3737:
--
Fix Version/s: 1.11.2
(was: 1.11.1)
> [C] memcheck_test_avro_commons_schema is failing
>
>
> Key: AVRO-3737
> URL: https://issues.apache.org/jira/browse/AVRO-3737
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.11.1, 1.12.0
>Reporter: Ryan Skraba
>Assignee: Christophe Le Saec
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Even with the fix in AVRO-3454, one of the memcheck test is failing in the
> release docker:
> *{{cd lang/c && ./build.sh test}}*
> {code}
> ... (snip) ...
> 98% tests passed, 1 tests failed out of 48
> Total Test time (real) = 20.64 sec
> The following tests FAILED:
> 7 - memcheck_test_avro_commons_schema (Failed)
> Errors while running CTest
> make: *** [Makefile:85: test] Error 8
> {code}
> WIth a bit more detail in the logs:
> *{{tail -n 25 ../../build/c/tests/memcheck_test_avro_commons_schema.log}}*
> {code}
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== 172,133 (131,232 direct, 40,901 indirect) bytes in 2 blocks are
> definitely lost in loss record 80 of 80
> ==1209==at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:211)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:196)
> ==1209==by 0x1107A5: avro_file_writer_create (datafile.c:179)
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== LEAK SUMMARY:
> ==1209==definitely lost: 131,392 bytes in 4 blocks
> ==1209==indirectly lost: 57,315 bytes in 146 blocks
> ==1209== possibly lost: 0 bytes in 0 blocks
> ==1209==still reachable: 1,888 bytes in 4 blocks
> ==1209== suppressed: 0 bytes in 0 blocks
> ==1209==
> ==1209== For lists of detected and suppressed errors, rerun with: -s
> ==1209== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (AVRO-3737) [C] memcheck_test_avro_commons_schema is failing
[
https://issues.apache.org/jira/browse/AVRO-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3737:
--
Fix Version/s: 1.12.0
> [C] memcheck_test_avro_commons_schema is failing
>
>
> Key: AVRO-3737
> URL: https://issues.apache.org/jira/browse/AVRO-3737
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.11.1, 1.12.0
>Reporter: Ryan Skraba
>Assignee: Christophe Le Saec
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Even with the fix in AVRO-3454, one of the memcheck test is failing in the
> release docker:
> *{{cd lang/c && ./build.sh test}}*
> {code}
> ... (snip) ...
> 98% tests passed, 1 tests failed out of 48
> Total Test time (real) = 20.64 sec
> The following tests FAILED:
> 7 - memcheck_test_avro_commons_schema (Failed)
> Errors while running CTest
> make: *** [Makefile:85: test] Error 8
> {code}
> WIth a bit more detail in the logs:
> *{{tail -n 25 ../../build/c/tests/memcheck_test_avro_commons_schema.log}}*
> {code}
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== 172,133 (131,232 direct, 40,901 indirect) bytes in 2 blocks are
> definitely lost in loss record 80 of 80
> ==1209==at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:211)
> ==1209==by 0x11032F: avro_file_writer_create_with_codec_fp
> (datafile.c:196)
> ==1209==by 0x1107A5: avro_file_writer_create (datafile.c:179)
> ==1209==by 0x10F97C: create_writer (test_avro_commons_schema.c:70)
> ==1209==by 0x10F97C: read_data (test_avro_commons_schema.c:92)
> ==1209==by 0x10F97C: run_tests (test_avro_commons_schema.c:113)
> ==1209==by 0x10F97C: main (test_avro_commons_schema.c:139)
> ==1209==
> ==1209== LEAK SUMMARY:
> ==1209==definitely lost: 131,392 bytes in 4 blocks
> ==1209==indirectly lost: 57,315 bytes in 146 blocks
> ==1209== possibly lost: 0 bytes in 0 blocks
> ==1209==still reachable: 1,888 bytes in 4 blocks
> ==1209== suppressed: 0 bytes in 0 blocks
> ==1209==
> ==1209== For lists of detected and suppressed errors, rerun with: -s
> ==1209== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (AVRO-3738) [Build][C#] The release build fails with .NET 7.0 target
[
https://issues.apache.org/jira/browse/AVRO-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba reassigned AVRO-3738:
-
Assignee: Zoltan Csizmadia
> [Build][C#] The release build fails with .NET 7.0 target
>
>
> Key: AVRO-3738
> URL: https://issues.apache.org/jira/browse/AVRO-3738
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.12.0, 1.11.2
>Reporter: Ryan Skraba
>Assignee: Zoltan Csizmadia
>Priority: Critical
>
> {code:java}
> *** No errors detected
> + cd lang/csharp
> + ./build.sh test
> ++ dirname ./build.sh
> + cd .
> + ROOT=../..
> ++ cat ../../share/VERSION.txt
> + VERSION=1.12.0-SNAPSHOT
> + for target in "$@"
> + case "$target" in
> + dotnet build --configuration Release Avro.slnWelcome to .NET 6.0!
> -
> SDK Version: 6.0.408Telemetry
> -
> The .NET tools collect usage data in order to help us improve your
> experience. It is collected by Microsoft and shared with the community. You
> can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT
> environment variable to '1' or 'true' using your favorite shell.Read more
> about .NET CLI Tools telemetry:
> https://aka.ms/dotnet-cli-telemetry
> Installed an ASP.NET Core HTTPS development certificate.
> To trust the certificate run 'dotnet dev-certs https --trust' (Windows and
> macOS only).
> Learn about HTTPS: https://aka.ms/dotnet-https
>
> Write your first app: https://aka.ms/dotnet-hello-world
> Find out what's new: https://aka.ms/dotnet-whats-new
> Explore documentation: https://aka.ms/dotnet-docs
> Report issues and find source on GitHub: https://github.com/dotnet/core
> Use 'dotnet --help' to see available commands or visit:
> https://aka.ms/dotnet-cli
> --
> MSBuild version 17.3.2+561848881 for .NET
> Determining projects to restore...
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/test/Avro.test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/perf/Avro.perf.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codec/Avro.File.BZip2.Test/Avro.File.BZip2.Test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codec/Avro.File.Snappy.Test/Avro.File.Snappy.Test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codec/Avro.File.Zstandard.Test/Avro.File.Zstandard.Test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codegen/Avro.codegen.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codec/Avro.File.XZ.Test/Avro.File.XZ.Test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
[jira] [Updated] (AVRO-3738) [Build][C#] The release build fails with .NET 7.0 target
[
https://issues.apache.org/jira/browse/AVRO-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-3738:
--
Fix Version/s: 1.12.0
1.11.2
> [Build][C#] The release build fails with .NET 7.0 target
>
>
> Key: AVRO-3738
> URL: https://issues.apache.org/jira/browse/AVRO-3738
> Project: Apache Avro
> Issue Type: Bug
>Affects Versions: 1.12.0, 1.11.2
>Reporter: Ryan Skraba
>Assignee: Zoltan Csizmadia
>Priority: Critical
> Fix For: 1.12.0, 1.11.2
>
>
> {code:java}
> *** No errors detected
> + cd lang/csharp
> + ./build.sh test
> ++ dirname ./build.sh
> + cd .
> + ROOT=../..
> ++ cat ../../share/VERSION.txt
> + VERSION=1.12.0-SNAPSHOT
> + for target in "$@"
> + case "$target" in
> + dotnet build --configuration Release Avro.slnWelcome to .NET 6.0!
> -
> SDK Version: 6.0.408Telemetry
> -
> The .NET tools collect usage data in order to help us improve your
> experience. It is collected by Microsoft and shared with the community. You
> can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT
> environment variable to '1' or 'true' using your favorite shell.Read more
> about .NET CLI Tools telemetry:
> https://aka.ms/dotnet-cli-telemetry
> Installed an ASP.NET Core HTTPS development certificate.
> To trust the certificate run 'dotnet dev-certs https --trust' (Windows and
> macOS only).
> Learn about HTTPS: https://aka.ms/dotnet-https
>
> Write your first app: https://aka.ms/dotnet-hello-world
> Find out what's new: https://aka.ms/dotnet-whats-new
> Explore documentation: https://aka.ms/dotnet-docs
> Report issues and find source on GitHub: https://github.com/dotnet/core
> Use 'dotnet --help' to see available commands or visit:
> https://aka.ms/dotnet-cli
> --
> MSBuild version 17.3.2+561848881 for .NET
> Determining projects to restore...
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/test/Avro.test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/perf/Avro.perf.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codec/Avro.File.BZip2.Test/Avro.File.BZip2.Test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codec/Avro.File.Snappy.Test/Avro.File.Snappy.Test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codec/Avro.File.Zstandard.Test/Avro.File.Zstandard.Test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codegen/Avro.codegen.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The current .NET SDK does not support targeting .NET 7.0.
> Either target .NET 6.0 or lower, or use a version of the .NET SDK that
> supports .NET 7.0.
> [/home/ryan.skraba/avro/lang/csharp/src/apache/codec/Avro.File.XZ.Test/Avro.File.XZ.Test.csproj]
> /opt/dotnet/sdk/6.0.408/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(144,5):
> error NETSDK1045: The
