Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-16 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1604353293 ## cpp/src/arrow/ipc/CMakeLists.txt: ## @@ -6,7 +6,7 @@ # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-16 Thread via GitHub
kou commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1604344494 ## cpp/src/arrow/ipc/CMakeLists.txt: ## @@ -6,7 +6,7 @@ # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the Lic

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-16 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2116532006 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 74f7578f77adca6b0fd79f7d37e28941330221eb. There were no

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-16 Thread via GitHub
amoeba merged PR #40392: URL: https://github.com/apache/arrow/pull/40392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-16 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2116360578 Thanks all. The submodule change to help CI pass has been reverted in this PR. I'm merging this now. -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2114193299 Revision: 4ebe53bd21fd1f28ce6babebf6802488283c692c Submitted crossbow builds: [ursacomputing/crossbow @ actions-d1985bc136](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
pitrou commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602710047 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,81 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
pitrou commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2114185667 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2113930475 Parquet tests were failing all over on CI and due to needing to bump submodules so I pushed 4ebe53bd21fd1f28ce6babebf6802488283c692c so we could see CI. -- This is an automated message

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2113695113 Thanks @pitrou for the review. I rebased from apache/main, accepted your three changes as-is, and added one extra change I caught while dealing with conflicts on rebase, 4bd0339e596d410bd5

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602388341 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,80 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602387512 ## cpp/src/arrow/ipc/CMakeLists.txt: ## @@ -41,6 +41,7 @@ add_arrow_test(feather_test) add_arrow_ipc_test(json_simple_test) add_arrow_ipc_test(read_write_test EXTRA_LI

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602388341 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,80 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602387673 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,80 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602387512 ## cpp/src/arrow/ipc/CMakeLists.txt: ## @@ -41,6 +41,7 @@ add_arrow_test(feather_test) add_arrow_ipc_test(json_simple_test) add_arrow_ipc_test(read_write_test EXTRA_LI

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602388341 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,80 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602387673 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,80 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-15 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1602387512 ## cpp/src/arrow/ipc/CMakeLists.txt: ## @@ -41,6 +41,7 @@ add_arrow_test(feather_test) add_arrow_ipc_test(json_simple_test) add_arrow_ipc_test(read_write_test EXTRA_LI

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-02 Thread via GitHub
pitrou commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1587695689 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,80 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2089179333 Revision: 4a81743474b50fdfbb9df50724f8a921059f0252 Submitted crossbow builds: [ursacomputing/crossbow @ actions-20acdf73a0](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2089177523 I'll let CI and the crossbow jobs run then merge if all looks good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2089176534 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1586855179 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,82 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
kou commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1586850006 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,82 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2089162900 Thanks for taking a look @kou, I accepted all your changes. I feel pretty good about the state of this PR at this point and am not sure we need another review. Let me know what you think.

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1586844048 ## cpp/src/arrow/ipc/message_internal_test.cc: ## @@ -0,0 +1,89 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1586756287 ## cpp/src/arrow/ipc/metadata_internal.cc: ## @@ -477,7 +477,9 @@ static Status GetDictionaryEncoding(FBB& fbb, const std::shared_ptr& fiel static KeyValueOffset App

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-05-01 Thread via GitHub
amoeba commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1586754681 ## cpp/src/arrow/ipc/metadata_internal.cc: ## @@ -477,7 +477,9 @@ static Status GetDictionaryEncoding(FBB& fbb, const std::shared_ptr& fiel static KeyValueOffset App

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-30 Thread via GitHub
kou commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1585922763 ## cpp/src/arrow/ipc/metadata_internal.cc: ## @@ -477,7 +477,9 @@ static Status GetDictionaryEncoding(FBB& fbb, const std::shared_ptr& fiel static KeyValueOffset Append

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2087841375 I've pushed up a fix to address the includes issue and the crossbow jobs looked clear (failures are unrelated and the test we expect to pass does indeed pass). I've marked this as r

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2087796050 Revision: c3af79dbaabf59cd61fee65cdd9e47a47a9bdd62 Submitted crossbow builds: [ursacomputing/crossbow @ actions-08f54365f0](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2087794194 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-12 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2052967916 Things look good locally (testing on macOS-aarch64-clang and Linux-amd64-gcc) but I see an issue with my includes in a few of the above jobs. I'll fix those soon. -- This is an automate

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-12 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2052637242 Revision: 911794535bb22b6012a2de8d262c37384324017e Submitted crossbow builds: [ursacomputing/crossbow @ actions-e9bd83c5da](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-12 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2052635378 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-12 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2052605240 ``` Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in. The Archery job run can be found a

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-12 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2052604718 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-04 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2038434367 I put up a draft PR with a test to exercise this and hopefully watch it fail on CI at https://github.com/apache/arrow/pull/41018. It fails on my amd64 linux machine so I think it's close t

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-03 Thread via GitHub
noamross commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2035637386 Thanks! I expect the files attached to https://github.com/apache/arrow/issues/40202#issuecomment-1977273164 could serve for a test of whether a generated file is identical to a reference

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-03 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2035621794 No problem at all @noamross, it's awesome you filed a PR for the issue and it's very much appreciated. I'll work on adding and testing the test. -- This is an automated message from the

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-03 Thread via GitHub
noamross commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2034520818 @amoeba Sorry to open this and abandon it, it just became a very busy month and my C++ is pretty limited. I'd be happy if you would! -- This is an automated message from the Apache Gi

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-02 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2033265513 Hi @noamross, would you be interested in adding the test described above to this PR? If not, I could take a shot at it. -- This is an automated message from the Apache Git Service. To re

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-08 Thread via GitHub
pitrou commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-1985768837 As discussed on the GH issue, it would be useful to add a test using output from your machine as a reference. (also, perhaps open a separate PR with just the test to validate that it wo

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-1982345583 :warning: GitHub issue #40361 **has no components**, please add labels for components. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-06 Thread via GitHub
kou commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-1982345337 Approved! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] GH-40361 [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-06 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-1982303549 Thanks @noamross! @kou can you please approve all workflows to run? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] GH-40361 [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-06 Thread via GitHub
kou commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1515360635 ## cpp/src/arrow/ipc/metadata_internal.cc: ## @@ -477,7 +477,9 @@ static Status GetDictionaryEncoding(FBB& fbb, const std::shared_ptr& fiel static KeyValueOffset Append

Re: [PR] GH-40361 [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-1982075431 :warning: GitHub issue #40361 **has no components**, please add labels for components. -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] GH-40361 [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-1982074757 :warning: GitHub issue #40361 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] GH-40361 [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-1982074765 :warning: GitHub issue #40361 **has no components**, please add labels for components. -- This is an automated message from the Apache Git Service. To respond to the message

[PR] GH-40361 [C++] Make flatbuffers serialization more deterministic [arrow]

2024-03-06 Thread via GitHub
noamross opened a new pull request, #40392: URL: https://github.com/apache/arrow/pull/40392 ### Rationale for this change This is the start of a PR to address #40361, and in turn #40202, to make parquet files written by arrow to be identical irrespective of the platform configuration