Re: [DISCUSS] Diagnostic reporter

2022-10-14 Thread Forward Xu
+1, Thanks Shiyan Xu and Zhang Yue, This is a very useful function.

Best,
Forward

sagar sumit  于2022年9月12日周一 18:39写道:

> Thanks Zhang Yue for drafting the RFC.
> It's an interesting read! I have left some comments.
>
> While exposing certain info such as "sample_hoodie_key",
> we have to consider masking/obfuscation.
>
> Looking forward to the implementation.
>
> Regards,
> Sagar
>
> On Wed, Sep 7, 2022 at 1:49 PM Yue Zhang  wrote:
>
> > Hi Hudi,
> > Just raise a RFC about this diagnostic reporter
> > https://github.com/apache/hudi/pull/6600. PLEASE feel free to leave any
> > comments or concerns if you are interested!
> >
> >
> > | |
> > Yue Zhang
> > |
> > |
> > zhangyue921...@163.com
> > |
> >
> >
> > On 08/4/2022 19:38,Yue Zhang wrote:
> > Hi Shiyan and everyone,
> > This is a great idea! As one of Hudi user, I also struggle to Hudi
> > troubleshooting sometimes. With this feature, it will definitely be able
> to
> > reduce the burden.
> > So I volunteer to draft a discuss and maybe raise a RFC about if you
> > don't mind. Thanks :)
> >
> >
> > | |
> > Yue Zhang
> > |
> > |
> > zhangyue921...@163.com
> > |
> >
> >
> > On 08/3/2022 00:44,冯健 wrote:
> > Maybe we can start this with an audit feature? Since we need some sort of
> > "images" to represent “facts”, can create an identity of a writer to link
> > them. and in this audit file, we can label each operation with IP,
> > environment, platform, version, write config and etc.
> >
> > On Sun, 31 Jul 2022 at 12:18, Shiyan Xu 
> > wrote:
> >
> > To bubble this up
> >
> > On Wed, Jun 15, 2022 at 11:47 PM Vinoth Chandar 
> wrote:
> >
> > +1 from me.
> >
> > It will be very useful if we can have something that can gather
> > troubleshooting info easily.
> > This part takes a while currently.
> >
> > On Mon, May 30, 2022 at 9:52 AM Shiyan Xu 
> > wrote:
> >
> > Hi all,
> >
> > When troubleshooting Hudi jobs in users' environments, we always ask
> > users
> > to share configs, environment info, check spark UI, etc. Here is an RFC
> > idea: can we extend the Hudi metrics system and make a diagnostic
> > reporter?
> > It can be turned on like a normal metrics reporter. it should collect
> > common troubleshooting info and save to json or other human-readable
> > text
> > format. Users should be able to run with it and share the diagnosis
> > file.
> > The RFC should discuss what info should / can be collected.
> >
> > Does this make sense? Anyone interested in driving the RFC design and
> > implementation work?
> >
> > --
> > Best,
> > Shiyan
> >
> >
> > --
> > Best,
> > Shiyan
> >
> >
>


[RESULT] [VOTE] Release 0.12.1, release candidate #2

2022-10-14 Thread zhaojing yu
Hi everyone,

I'm happy to announce that we have unanimously approved this release.

There are 8 approving votes, 5 of which are binding. Here is the breakdown:

+1 (binding) : 5

* Bhavani Sudha Saktheeswaran
* Sivabalan Narayanan
* Danny Chan
* Raymond Xu
* Udit Mehrotra

-1 (binding) : 0

+1 (non-binding) : 3

* Ethan Guo
* Rahil C
* Sagar Sumit

-1 (non-binding) : 0

Thanks, everyone!


Re: [VOTE] Release 0.12.1, release candidate #2

2022-10-14 Thread Udit Mehrotra
+1 (binding)

- Builds with spark 2 and spark 3
- Release validation script succeeds
- EMRs internal tests succeed with the spark3 bundle on latest release 6.8.0

Thanks,
Udit

On Thu, Oct 13, 2022 at 10:22 PM Bhavani Sudha  wrote:
>
> +1 (binding)
>
>
> [OK] Build successfully multiple supported spark versions
>
> [OK] Ran validation script
>
> [OK] Ran some IDE tests
>
>
> sudha[21:59:36] scripts % ./release/validate_staged_release.sh
> --release=0.12.1 --rc_num=2
> /tmp/validation_scratch_dir_001 ~/hudi/scripts
> Downloading from svn co https://dist.apache.org/repos/dist/dev/hudi
> Validating hudi-0.12.1-rc2 with release type "dev"
> Checking Checksum of Source Release
> Checksum Check of Source Release - [OK]
>
>   % Total% Received % Xferd  Average Speed   TimeTime Time
>  Current
>  Dload  Upload   Total   SpentLeft
>  Speed
> 100 65803  100 658030 0   135k  0 --:--:-- --:--:-- --:--:--
>  138k
> Checking Signature
> Signature Check - [OK]
>
> Checking for binary files in source release
> No Binary Files in Source Release? - [OK]
>
> Checking for DISCLAIMER
> DISCLAIMER file exists ? [OK]
>
> Checking for LICENSE and NOTICE
> License file exists ? [OK]
> Notice file exists ? [OK]
>
> Performing custom Licensing Check
> Licensing Check Passed [OK]
>
> Running RAT Check
> RAT Check Passed [OK]
>
> ~/hudi/scripts
>
>
> Thanks,
>
> Sudha
>
> On Thu, Oct 13, 2022 at 9:22 PM Danny Chan  wrote:
>
> > +1 (binding)
> >
> > Flink quickstart OK
> > Long-running Flink SQL Job OK
> > Flink Hive Sync OK
> > Flink compaction and cleaning OK
> > Compile the source code OK
> >
> > Regards,
> > Danny
> >
> > Rahil C  于2022年10月14日周五 02:46写道:
> > >
> > > +1 (non-binding)
> > >
> > > Ran hudi-spark bundle against EMR integration tests
> > >
> > >
> > >
> > > On Thu, Oct 13, 2022 at 11:09 AM Shiyan Xu 
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Primary key fingerprint: B430 5519 F36D D7E8 B7E6  A684 58B8 5B81 4778
> > 3CE2
> > > > Signature Check - [OK]
> > > >
> > > > Checking for binary files in source release
> > > > No Binary Files in Source Release? - [OK]
> > > >
> > > > Checking for DISCLAIMER
> > > > DISCLAIMER file exists ? [OK]
> > > >
> > > > Checking for LICENSE and NOTICE
> > > > License file exists ? [OK]
> > > > Notice file exists ? [OK]
> > > >
> > > > Performing custom Licensing Check
> > > > Licensing Check Passed [OK]
> > > >
> > > >   RAT Check Passed [OK]
> > > >
> > > > On Fri, Oct 14, 2022 at 12:43 AM Sivabalan  wrote:
> > > >
> > > > > +1 binding.
> > > > >
> > > > > Ran a suite of integration tests spanning diff spark versions,
> > > > > deltastreamer, spark datasource writer for both table types and w/
> > and
> > > > w/o
> > > > > metadata table.
> > > > >
> > > > > On Thu, 13 Oct 2022 at 05:16, sagar sumit  wrote:
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Spark quickstart OK
> > > > > > Long-running delta streamer OK
> > > > > > Hive Sync OK
> > > > > > Async table services OK
> > > > > >
> > > > > > Regards,
> > > > > > Sagar
> > > > > >
> > > > > > On Tue, Oct 11, 2022 at 8:20 PM zhaojing yu 
> > > > wrote:
> > > > > >
> > > > > > > Hi everyone,
> > > > > > >
> > > > > > > Please review and vote on the release candidate #2 for the
> > version
> > > > > > 0.12.1,
> > > > > > > as follows:
> > > > > > >
> > > > > > > [ ] +1, Approve the release
> > > > > > > [ ] -1, Do not approve the release (please provide specific
> > comments)
> > > > > > >
> > > > > > > The complete staging area is available for your review, which
> > > > includes:
> > > > > > >
> > > > > > > * JIRA release notes [1],
> > > > > > > * the official Apache source release and binary convenience
> > releases
> > > > to
> > > > > > be
> > > > > > > deployed to dist.apache.org [2], which are signed with the key
> > with
> > > > > > > fingerprint B4305519F36DD7E8B7E6A68458B85B8147783CE2 [3],
> > > > > > > * all artifacts to be deployed to the Maven Central Repository
> > [4],
> > > > > > > * source code tag "release-0.12.1-rc2" [5],
> > > > > > >
> > > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > > majority
> > > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Release Manager
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12322822=12352182
> > > > > > > [2] https://dist.apache.org/repos/dist/dev/hudi/hudi-0.12.1-rc2/
> > > > > > > [3] https://dist.apache.org/repos/dist/dev/hudi/KEYS
> > > > > > > [4]
> > > > > >
> > https://repository.apache.org/content/repositories/orgapachehudi-1101/
> > > > > > > [5]
> > https://github.com/apache/hudi/releases/tag/release-0.12.1-rc2
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > > -Sivabalan
> > > > >
> > > >
> > > >
> > > > --
> > > > Best,
> > > > Shiyan
> > > >