I just learned that arrayIngestMode is not actually new, just https://github.com/apache/druid/pull/15588 is. However this will still make it more likely that people accidentally break their tables, so I am still -0. Just, slightly less so. I still think it would be a good idea, for Druid 29, to add string-to-array type validation to Druid 29's INSERT / REPLACE handling to compensate for the new web console support for arrayIngestMode, and the UI changes to push people towards setting it to "array".
I could be convinced that it's ok to do that in a 29.0.1. I don't think it should wait for 30, given the impact that can happen if people end up with mixed types without planning for it. On Fri, Feb 16, 2024 at 2:16 PM Gian Merlino <g...@apache.org> wrote: > Thanks for managing this release! > > My vote is -0, let me explain why. I am concerned about usability issues > with the new arrayIngestMode feature. There are various issues when mixing > MVD strings and string arrays in the same column: as soon as arrays show up > in a column, various "classic MVD-style" queries will fail at validation or > at runtime. I believe that the new feature, and especially the changes to > the web console in https://github.com/apache/druid/pull/15588, will make > it more likely that people will do this by accident and experience > brokenness. > > When this occurs, there is not an easy way to fix it; data needs to be > reingested or queries need to be adjusted. I believe that in some cases, > queries can't be adjusted without suffering from performance loss. > > This is something that could happen even before Druid 29, if you did a > Kafka ingest with auto types or useSchemaDiscovery, followed by a SQL > REPLACE from that table into itself. In that case, arrays written by Kafka > ingest would get rewritten as MVDs. But with this Druid 29 RC, there are > additional pathways created that enable people to get into this problematic > scenario with increased likelihood. I suggest we make some adjustments that > prevent it, such as: > > - Validating that SQL INSERT and SQL REPLACE do not insert an array type > into a column that previously contained strings, or vice versa. > - Making it obvious in the web console whether a tab is in > "arrayIngestMode: array" or "arrayIngestMode: mvd" or "server default". > > Thank you for considering this viewpoint. > > Gian > > On Tue, Feb 13, 2024 at 4:34 AM Laksh Singla <lakshsin...@apache.org> > wrote: > >> Hi all, >> >> I have created a build for Apache Druid 29.0.0, release >> candidate 1. >> >> Thanks to everyone who has helped contribute to the release! You can read >> the proposed release notes here: >> https://github.com/apache/druid/issues/15896 >> >> The release candidate has been tagged in GitHub as >> druid-29.0.0-rc1 (869bd3978f0c835ef8eb7c1f25c468e23472a81b), >> available here: >> https://github.com/apache/druid/tree/druid-29.0.0-rc1 >> >> The artifacts to be voted on are located here: >> https://dist.apache.org/repos/dist/dev/druid/29.0.0-rc1/ >> >> A staged Maven repository is available for review at: >> https://repository.apache.org/content/repositories/orgapachedruid-1061/ >> >> Staged druid.apache.org website documentation is available here: >> https://druid.staged.apache.org/docs/29.0.0/design/ >> >> A Docker image containing the binary of the release candidate can be >> retrieved via: >> docker pull apache/druid:29.0.0-rc1 >> >> artifact checksums >> src: >> >> 1948fab4500f3571591f887a638631b6a05f040b88b35004406ca852e16884e5d2269a74e8d07c79c477d8b02a2489efec85776f8ef46f4e8defedce4efb9931 >> bin: >> >> f2a11ddc71b59a648d01d7c220a6fae527c34d702d4f1e7aed954803a3576543c4cd7149f34e8948eccf89240435f09a8512db83b46991ac6e354eeba8cbada4 >> docker: 2bddcf692f2137dc4094908b63d2043423fca895ba0f479f0db34ec8016c4472 >> >> Release artifacts are signed with the following key: >> https://people.apache.org/keys/committer/lakshsingla.asc >> >> This key and the key of other committers can also be found in the >> project's >> KEYS file here: >> https://dist.apache.org/repos/dist/release/druid/KEYS >> >> (If you are a committer, please feel free to add your own key to that file >> by following the instructions in the file's header.) >> >> >> Verify checksums: >> diff <(shasum -a512 apache-druid-29.0.0-src.tar.gz | \ >> cut -d ' ' -f1) \ >> <(cat apache-druid-29.0.0-src.tar.gz.sha512 ; echo) >> >> diff <(shasum -a512 apache-druid-29.0.0-bin.tar.gz | \ >> cut -d ' ' -f1) \ >> <(cat apache-druid-29.0.0-bin.tar.gz.sha512 ; echo) >> >> Verify signatures: >> gpg --verify apache-druid-29.0.0-src.tar.gz.asc \ >> apache-druid-29.0.0-src.tar.gz >> >> gpg --verify apache-druid-29.0.0-bin.tar.gz.asc \ >> apache-druid-29.0.0-bin.tar.gz >> >> Please review the proposed artifacts and vote. Note that Apache has >> specific requirements that must be met before +1 binding votes can be cast >> by PMC members. Please refer to the policy at >> http://www.apache.org/legal/release-policy.html#policy for more details. >> >> As part of the validation process, the release artifacts can be generated >> from source by running: >> mvn clean install -Papache-release,dist -Dgpg.skip >> >> The RAT license check can be run from source by: >> mvn apache-rat:check -Prat >> >> This vote will be open for at least 72 hours. The vote will pass if a >> majority of at least three +1 PMC votes are cast. >> >> [ ] +1 Release this package as Apache Druid 29.0.0 >> [ ] 0 I don't feel strongly about it, but I'm okay with the release >> [ ] -1 Do not release this package because... >> >> Thanks! >> >
