-1 for me Do not change spark.sql.legacy.createHiveTableByDefault because:
1. We have not had enough time to "DISCUSS" this matter. The discussion thread was opened almost 24 hours ago. 2. Compatibility: Changing the default behavior could potentially break existing workflows or pipelines that rely on the current behavior. Many users may have scripts or applications that expect Hive tables to be created by default, and altering this behavior without careful consideration could lead to unexpected issues. 3. User Experience: For users who are familiar with the current behavior, having Hive tables created by default may be more intuitive and convenient. Changing the default behavior could require users to modify their scripts or workflows, leading to confusion and productivity loss. 4. Flexibility: Retaining the option to create Hive tables by default allows users to leverage the features and optimizations provided by the Hive metastore. While Spark native tables may offer certain advantages, there are use cases where Hive tables are preferred, such as integration with the existing Hive ecosystems or compatibility with other tools as I brought up in the "DISCUSS" thread. 5. Many users have built workflows, scripts, or applications based on this behaviour, and any changes to it could impact their ability to effectively use Spark SQL in their data processing pipelines. IMO, these reasons warrant the importance of carefully evaluating the impact of changing the default behaviour. Mich TalebzadehTechnologist | Architect | Data Engineer | Generative AI | FinCrime London United Kingdom view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* The information provided is correct to the best of my knowledge but of course cannot be guaranteed . It is essential to note that, as with any advice, quote "one test result is worth one-thousand expert opinions (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". On Fri, 26 Apr 2024 at 20:06, L. C. Hsieh <vii...@gmail.com> wrote: > +1 > > On Fri, Apr 26, 2024 at 10:01 AM Dongjoon Hyun <dongj...@apache.org> > wrote: > > > > I'll start with my +1. > > > > Dongjoon. > > > > On 2024/04/26 16:45:51 Dongjoon Hyun wrote: > > > Please vote on SPARK-46122 to set > spark.sql.legacy.createHiveTableByDefault > > > to `false` by default. The technical scope is defined in the following > PR. > > > > > > - DISCUSSION: > > > https://lists.apache.org/thread/ylk96fg4lvn6klxhj6t6yh42lyqb8wmd > > > - JIRA: https://issues.apache.org/jira/browse/SPARK-46122 > > > - PR: https://github.com/apache/spark/pull/46207 > > > > > > The vote is open until April 30th 1AM (PST) and passes > > > if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. > > > > > > [ ] +1 Set spark.sql.legacy.createHiveTableByDefault to false by > default > > > [ ] -1 Do not change spark.sql.legacy.createHiveTableByDefault because > ... > > > > > > Thank you in advance. > > > > > > Dongjoon > > > > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
