[GitHub] [flink] flinkbot edited a comment on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot edited a comment on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * 09879982e64d6fb8feaa6402d1df6c5c73b05cf0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11557) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-20826) Fix bug in streaming SQL examples
[ https://issues.apache.org/jira/browse/FLINK-20826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu closed FLINK-20826. --- Assignee: Shuo Cheng Resolution: Fixed Fixed in master: 854562112885c075a0d5f46aeadc37c5cb117009 > Fix bug in streaming SQL examples > - > > Key: FLINK-20826 > URL: https://issues.apache.org/jira/browse/FLINK-20826 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API >Affects Versions: 1.12.0 >Reporter: Shuo Cheng >Assignee: Shuo Cheng >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > There is some minor bug in `UpdatingTopCityExample`. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] wuchong merged pull request #14537: [FLINK-20826][table] Fix minor bug in `UpdatingTopCityExample`
wuchong merged pull request #14537: URL: https://github.com/apache/flink/pull/14537 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot edited a comment on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * Unknown: [CANCELED](TBD) * 09879982e64d6fb8feaa6402d1df6c5c73b05cf0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11557) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot edited a comment on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * Unknown: [CANCELED](TBD) * 09879982e64d6fb8feaa6402d1df6c5c73b05cf0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] xuewang commented on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
xuewang commented on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753259534 @flinkbot run azure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot edited a comment on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * 72dc489d2eb615514a72df76dc6a4ea9acb4c578 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11556) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot edited a comment on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * Unknown: [CANCELED](TBD) * 72dc489d2eb615514a72df76dc6a4ea9acb4c578 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11556) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot edited a comment on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * Unknown: [CANCELED](TBD) * 72dc489d2eb615514a72df76dc6a4ea9acb4c578 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] xuewang commented on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
xuewang commented on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753234656 @flinkbot run azure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14084: [FLINK-15867][table-planner-blink] Support time-related types for FIRST_VALUE and LAST_VALUE aggregate functions
flinkbot edited a comment on pull request #14084: URL: https://github.com/apache/flink/pull/14084#issuecomment-728296283 ## CI report: * a74b9bc950d5a3e42953e2a46535914e8299a208 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11550) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot edited a comment on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * a9cbd5110401104135a3ecf96190cb8032cafed1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11552) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14526: [FLINK-14814][WebUI] Highlight back pressured and busy nodes in the WebUI
flinkbot edited a comment on pull request #14526: URL: https://github.com/apache/flink/pull/14526#issuecomment-752592411 ## CI report: * 6cff7c3ed0b6694c5494bf4be9a1393cf7bec631 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11549) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot edited a comment on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * a9cbd5110401104135a3ecf96190cb8032cafed1 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11552) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot commented on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-753000655 ## CI report: * a9cbd5110401104135a3ecf96190cb8032cafed1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14538: [FLINK-20681][yarn] Support remote path for shipping archives and files
flinkbot edited a comment on pull request #14538: URL: https://github.com/apache/flink/pull/14538#issuecomment-752991523 ## CI report: * af57879d33f5d7e1717518a475fd601916c5c7c1 UNKNOWN * b1c20c731702fc0ee41ca744acb3287737170dc7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11551) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14538: [FLINK-20681][yarn] Support remote path for shipping archives and files
flinkbot edited a comment on pull request #14538: URL: https://github.com/apache/flink/pull/14538#issuecomment-752991523 ## CI report: * af57879d33f5d7e1717518a475fd601916c5c7c1 UNKNOWN * b1c20c731702fc0ee41ca744acb3287737170dc7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
flinkbot commented on pull request #14539: URL: https://github.com/apache/flink/pull/14539#issuecomment-752995836 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a9cbd5110401104135a3ecf96190cb8032cafed1 (Thu Dec 31 16:08:05 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20321) Get NPE when using AvroDeserializationSchema to deserialize null input
[
https://issues.apache.org/jira/browse/FLINK-20321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257043#comment-17257043
]
Xue Wang commented on FLINK-20321:
--
Thank you, [~jark]. I just submitted the
[PR|https://github.com/apache/flink/pull/14539].
> Get NPE when using AvroDeserializationSchema to deserialize null input
> --
>
> Key: FLINK-20321
> URL: https://issues.apache.org/jira/browse/FLINK-20321
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.12.0
>Reporter: Shengkai Fang
>Assignee: Xue Wang
>Priority: Major
> Labels: pull-request-available, sprint, starter
> Fix For: 1.13.0
>
>
> You can reproduce the bug by adding the code into the
> {{AvroDeserializationSchemaTest}}.
> The code follows
> {code:java}
> @Test
> public void testSpecificRecord2() throws Exception {
> DeserializationSchema deserializer =
> AvroDeserializationSchema.forSpecific(Address.class);
> Address deserializedAddress = deserializer.deserialize(null);
> assertEquals(null, deserializedAddress);
> }
> {code}
> Exception stack:
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.flink.formats.avro.utils.MutableByteArrayInputStream.setBuffer(MutableByteArrayInputStream.java:43)
> at
> org.apache.flink.formats.avro.AvroDeserializationSchema.deserialize(AvroDeserializationSchema.java:131)
> at
> org.apache.flink.formats.avro.AvroDeserializationSchemaTest.testSpecificRecord2(AvroDeserializationSchemaTest.java:69)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-20321) Get NPE when using AvroDeserializationSchema to deserialize null input
[
https://issues.apache.org/jira/browse/FLINK-20321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-20321:
---
Labels: pull-request-available sprint starter (was: sprint starter)
> Get NPE when using AvroDeserializationSchema to deserialize null input
> --
>
> Key: FLINK-20321
> URL: https://issues.apache.org/jira/browse/FLINK-20321
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.12.0
>Reporter: Shengkai Fang
>Assignee: Xue Wang
>Priority: Major
> Labels: pull-request-available, sprint, starter
> Fix For: 1.13.0
>
>
> You can reproduce the bug by adding the code into the
> {{AvroDeserializationSchemaTest}}.
> The code follows
> {code:java}
> @Test
> public void testSpecificRecord2() throws Exception {
> DeserializationSchema deserializer =
> AvroDeserializationSchema.forSpecific(Address.class);
> Address deserializedAddress = deserializer.deserialize(null);
> assertEquals(null, deserializedAddress);
> }
> {code}
> Exception stack:
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.flink.formats.avro.utils.MutableByteArrayInputStream.setBuffer(MutableByteArrayInputStream.java:43)
> at
> org.apache.flink.formats.avro.AvroDeserializationSchema.deserialize(AvroDeserializationSchema.java:131)
> at
> org.apache.flink.formats.avro.AvroDeserializationSchemaTest.testSpecificRecord2(AvroDeserializationSchemaTest.java:69)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] xuewang opened a new pull request #14539: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input
xuewang opened a new pull request #14539: URL: https://github.com/apache/flink/pull/14539 ## What is the purpose of the change This pull request aims to fix JIRA issue [FLINK-20321](https://issues.apache.org/jira/browse/FLINK-20321). ## Brief change log Make `deserialize(byte[])` method of following classes return null when argument is null, instead of throwing NPE. Test cases added. - `AvroDeserializationSchema` - `AvroRowDataDeserializationSchema` - `RegistryAvroDeserializationSchema` - `CsvRowDataDeserializationSchema` - `JsonRowDataDeserializationSchema` ## Verifying this change This change added tests and can be verified via test cases. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no - The S3 file system connector: don't know ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14538: [FLINK-20681][yarn] Support remote path for shipping archives and files
flinkbot commented on pull request #14538: URL: https://github.com/apache/flink/pull/14538#issuecomment-752991523 ## CI report: * af57879d33f5d7e1717518a475fd601916c5c7c1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14538: [FLINK-20681][yarn] Support remote path for shipping archives and files
flinkbot commented on pull request #14538: URL: https://github.com/apache/flink/pull/14538#issuecomment-752988877 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit af57879d33f5d7e1717518a475fd601916c5c7c1 (Thu Dec 31 15:32:19 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-20681) Support specifying the hdfs path when ship archives or files
[ https://issues.apache.org/jira/browse/FLINK-20681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-20681: --- Labels: pull-request-available pull-requests-available (was: pull-requests-available) > Support specifying the hdfs path when ship archives or files > - > > Key: FLINK-20681 > URL: https://issues.apache.org/jira/browse/FLINK-20681 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Affects Versions: 1.12.0 >Reporter: Ruguo Yu >Assignee: Ruguo Yu >Priority: Major > Labels: pull-request-available, pull-requests-available > Fix For: 1.13.0 > > Attachments: image-2020-12-23-20-58-41-234.png, > image-2020-12-24-01-01-10-021.png > > > Currently, our team try to submit flink job that depends extra resource with > yarn-application target, and use two options: "yarn.ship-archives" and > "yarn.ship-files". > But above options only support specifying local resource and shiping them to > hdfs, besides if it can support remote resource on distributed filesystem > (such as hdfs), then get the following benefits： > * client will exclude the local resource uploading to accelerate the job > submission process > * yarn will cache them on the nodes so that they doesn't need to be > downloaded for application -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] yuruguo opened a new pull request #14538: [FLINK-20681][yarn] Support remote path for shipping archives and files
yuruguo opened a new pull request #14538: URL: https://github.com/apache/flink/pull/14538 ## What is the purpose of the change This PR can support config options 'yarn.ship-archives' and 'yarn.ship-files' specify remote resources on distributed files ystem (such as hdfs) to be shipped to the YARN cluster, which could get the following benefits: - client will exclude the local resource uploading to accelerate the job submission process - yarn will cache them on the nodes so that they doesn't need to be downloaded for application **NOTE**: before that, above two options only supported local resources on client. ## Brief change log - The ship Archives and Files changed from File to Path in YarnClusterDescriptor - Archives and Files belongs to remote resource are copied to Hadoop filesystem in YarnApplicationFileUploader ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14537: [FLINK-20826][table] Fix minor bug in `UpdatingTopCityExample`
flinkbot edited a comment on pull request #14537: URL: https://github.com/apache/flink/pull/14537#issuecomment-752927633 ## CI report: * 0305b23e0bb621cc571b3ff08a1c00fe64bc0ad8 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11546) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14534: [FLINK-20822][hive] Don't check whether a function is generic in hive…
flinkbot edited a comment on pull request #14534: URL: https://github.com/apache/flink/pull/14534#issuecomment-752886442 ## CI report: * 5b018f902fac3974828e119081396fea2b999c60 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11544) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
flinkbot edited a comment on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-751755915 ## CI report: * 932157724cf6bd963643aa9970c82ce0e0b2c8fe Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11542) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14084: [FLINK-15867][table-planner-blink] Support time-related types for FIRST_VALUE and LAST_VALUE aggregate functions
flinkbot edited a comment on pull request #14084: URL: https://github.com/apache/flink/pull/14084#issuecomment-728296283 ## CI report: * 54c17b2c00b65af7eec5fcb13592e0893b2d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9657) * a74b9bc950d5a3e42953e2a46535914e8299a208 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11550) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14084: [FLINK-15867][table-planner-blink] Support time-related types for FIRST_VALUE and LAST_VALUE aggregate functions
flinkbot edited a comment on pull request #14084: URL: https://github.com/apache/flink/pull/14084#issuecomment-728296283 ## CI report: * 54c17b2c00b65af7eec5fcb13592e0893b2d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9657) * a74b9bc950d5a3e42953e2a46535914e8299a208 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14536: [FLINK-20812][Connector][Hbase] hbase in sql mode,can use 'properties.*' add Configuration parameter.
flinkbot edited a comment on pull request #14536: URL: https://github.com/apache/flink/pull/14536#issuecomment-752909965 ## CI report: * f66faaa7fd4901814134ab3d8993c52838fd77bf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11540) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14460: [docs/javadoc][hotfix] Explicitly Document task cancellation timeout …
flinkbot edited a comment on pull request #14460: URL: https://github.com/apache/flink/pull/14460#issuecomment-749464826 ## CI report: * 9f7dd68a258dd3ccc7655a5e4c8e819822fc1d8c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11537) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14526: [FLINK-14814][WebUI] Highlight back pressured and busy nodes in the WebUI
flinkbot edited a comment on pull request #14526: URL: https://github.com/apache/flink/pull/14526#issuecomment-752592411 ## CI report: * eb574e5180fff6ceaf7b76035fc8c45c27a9fa09 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11548) * 6cff7c3ed0b6694c5494bf4be9a1393cf7bec631 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11549) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14526: [FLINK-14814][WebUI] Highlight back pressured and busy nodes in the WebUI
flinkbot edited a comment on pull request #14526: URL: https://github.com/apache/flink/pull/14526#issuecomment-752592411 ## CI report: * eb574e5180fff6ceaf7b76035fc8c45c27a9fa09 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11548) * 6cff7c3ed0b6694c5494bf4be9a1393cf7bec631 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14534: [FLINK-20822][hive] Don't check whether a function is generic in hive…
flinkbot edited a comment on pull request #14534: URL: https://github.com/apache/flink/pull/14534#issuecomment-752886442 ## CI report: * 7c096f9a7dcf044d1d60afc33e3782379286093b Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11536) * 5b018f902fac3974828e119081396fea2b999c60 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11544) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #8952: [FLINK-10868][flink-runtime] Add failure rater for resource manager
flinkbot edited a comment on pull request #8952: URL: https://github.com/apache/flink/pull/8952#issuecomment-513724324 ## CI report: * d083b630115604e34b0a74498890aedbff61b2a7 UNKNOWN * 50dd8154daca82c54bc3d0decc8e94b6b7cba8d6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11535) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14526: [FLINK-14814][webui] Highlighe back pressured and busy nodes in the WebUI
flinkbot edited a comment on pull request #14526: URL: https://github.com/apache/flink/pull/14526#issuecomment-752592411 ## CI report: * eb574e5180fff6ceaf7b76035fc8c45c27a9fa09 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11548) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14399: [FLINK-17827] [scala-shell] scala-shell.sh should fail early if no mo…
flinkbot edited a comment on pull request #14399: URL: https://github.com/apache/flink/pull/14399#issuecomment-745854885 ## CI report: * 5d12faf957a2e129df5a32031e9a88e09f7b6ec7 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11534) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14537: [FLINK-20826][table] Fix minor bug in `UpdatingTopCityExample`
flinkbot edited a comment on pull request #14537: URL: https://github.com/apache/flink/pull/14537#issuecomment-752927633 ## CI report: * 0305b23e0bb621cc571b3ff08a1c00fe64bc0ad8 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11546) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14526: [FLINK-14814][webui] Highlighe back pressured and busy nodes in the WebUI
flinkbot edited a comment on pull request #14526: URL: https://github.com/apache/flink/pull/14526#issuecomment-752592411 ## CI report: * af89d36ef1ec641dd9acb240bd211132a52f3798 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11543) * eb574e5180fff6ceaf7b76035fc8c45c27a9fa09 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14537: [FLINK-20826][table] Fix minor bug in `UpdatingTopCityExample`
flinkbot commented on pull request #14537: URL: https://github.com/apache/flink/pull/14537#issuecomment-752927633 ## CI report: * 0305b23e0bb621cc571b3ff08a1c00fe64bc0ad8 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14508: [FLINK-20773][format] Support allow-unescaped-control-chars option for JSON format.
flinkbot edited a comment on pull request #14508: URL: https://github.com/apache/flink/pull/14508#issuecomment-751734163 ## CI report: * f1332b021d33a6e4681b0a08ad1c5b58f153c417 UNKNOWN * f4d02e921d2641fc5692617a4dd50ba2fda1128c UNKNOWN * fd8cbf90a807292b0db7b85bda26f1e717b87767 UNKNOWN * 3c03189754755222ce29f4d17485c91532da4a8b UNKNOWN * 5663475ec56efe4b84e6ae2e6cabd6d58db34bf2 UNKNOWN * d7c11e1d1a290f6c3745039cb6af6c2327b1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11507) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14534: [FLINK-20822][hive] Don't check whether a function is generic in hive…
flinkbot edited a comment on pull request #14534: URL: https://github.com/apache/flink/pull/14534#issuecomment-752886442 ## CI report: * 7c096f9a7dcf044d1d60afc33e3782379286093b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11536) * 5b018f902fac3974828e119081396fea2b999c60 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11544) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14508: [FLINK-20773][format] Support allow-unescaped-control-chars option for JSON format.
flinkbot edited a comment on pull request #14508: URL: https://github.com/apache/flink/pull/14508#issuecomment-751734163 ## CI report: * f1332b021d33a6e4681b0a08ad1c5b58f153c417 UNKNOWN * f4d02e921d2641fc5692617a4dd50ba2fda1128c UNKNOWN * fd8cbf90a807292b0db7b85bda26f1e717b87767 UNKNOWN * 3c03189754755222ce29f4d17485c91532da4a8b UNKNOWN * 5663475ec56efe4b84e6ae2e6cabd6d58db34bf2 UNKNOWN * f5f641fcde2b6d33b89c640994fcb69cafaa00a8 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11528) * d7c11e1d1a290f6c3745039cb6af6c2327b1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14537: [FLINK-20826][table] Fix minor bug in `UpdatingTopCityExample`
flinkbot commented on pull request #14537: URL: https://github.com/apache/flink/pull/14537#issuecomment-752922898 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 0305b23e0bb621cc571b3ff08a1c00fe64bc0ad8 (Thu Dec 31 10:51:08 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-20826).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] cshuo commented on pull request #14537: [FLINK-20826][table] Fix minor bug in `UpdatingTopCityExample`
cshuo commented on pull request #14537: URL: https://github.com/apache/flink/pull/14537#issuecomment-752922865 @godfreyhe Could you help review this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-20826) Fix bug in streaming SQL examples
[ https://issues.apache.org/jira/browse/FLINK-20826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-20826: --- Labels: pull-request-available (was: ) > Fix bug in streaming SQL examples > - > > Key: FLINK-20826 > URL: https://issues.apache.org/jira/browse/FLINK-20826 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API >Affects Versions: 1.12.0 >Reporter: Shuo Cheng >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > There is some minor bug in `UpdatingTopCityExample`. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] cshuo opened a new pull request #14537: [FLINK-20826][table] Fix minor bug in `UpdatingTopCityExample`
cshuo opened a new pull request #14537: URL: https://github.com/apache/flink/pull/14537 ## What is the purpose of the change There is a minor bug in streaming sql example `UpdatingTopCityExample` ## Brief change log - `UpdatingTopCityExample` ## Verifying this change - Covered by the example itself. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? no This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wuchong merged pull request #14533: [hotfix][docs]fix typo in Create statements page
wuchong merged pull request #14533: URL: https://github.com/apache/flink/pull/14533 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-20809) Limit push down with Hive table doesn't work when using with filter
[
https://issues.apache.org/jira/browse/FLINK-20809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-20809.
---
Fix Version/s: (was: 1.13.0)
Resolution: Not A Problem
> Limit push down with Hive table doesn't work when using with filter
> ---
>
> Key: FLINK-20809
> URL: https://issues.apache.org/jira/browse/FLINK-20809
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
>Affects Versions: 1.12.0
>Reporter: Jun Zhang
>Priority: Major
>
> when I use flink sql to query hive table , like this
> {code:java}
> // select * from hive_table where id = 1 limit 1
> {code}
>
> when the sql contain query conditions in where clause, I found that the limit
> push down is invalid.
> I look up the comment on source code , I think it is should be push down , is
> it a bug ?
> [the comment
> |https://github.com/apache/flink/blob/master/flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/rules/logical/PushLimitIntoTableSourceScanRule.java#L64]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-20809) Limit push down with Hive table doesn't work when using with filter
[
https://issues.apache.org/jira/browse/FLINK-20809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256939#comment-17256939
]
Jark Wu commented on FLINK-20809:
-
I agree with you [~fsk119]. We can't push down limit if there is a filter,
otherwise, the result is not correct.
I will close this issue.
> Limit push down with Hive table doesn't work when using with filter
> ---
>
> Key: FLINK-20809
> URL: https://issues.apache.org/jira/browse/FLINK-20809
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
>Affects Versions: 1.12.0
>Reporter: Jun Zhang
>Priority: Major
> Fix For: 1.13.0
>
>
> when I use flink sql to query hive table , like this
> {code:java}
> // select * from hive_table where id = 1 limit 1
> {code}
>
> when the sql contain query conditions in where clause, I found that the limit
> push down is invalid.
> I look up the comment on source code , I think it is should be push down , is
> it a bug ?
> [the comment
> |https://github.com/apache/flink/blob/master/flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/rules/logical/PushLimitIntoTableSourceScanRule.java#L64]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-20826) Fix bug in streaming SQL examples
[ https://issues.apache.org/jira/browse/FLINK-20826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256934#comment-17256934 ] Jark Wu commented on FLINK-20826: - Could you add more descriptions about which code has the bug? > Fix bug in streaming SQL examples > - > > Key: FLINK-20826 > URL: https://issues.apache.org/jira/browse/FLINK-20826 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API >Affects Versions: 1.12.0 >Reporter: Shuo Cheng >Priority: Major > Fix For: 1.13.0 > > > There is some minor bug in `UpdatingTopCityExample`. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] wuchong commented on pull request #14536: [FLINK-20812][Connector][Hbase] hbase in sql mode,can use 'properties.*' add Configuration parameter.
wuchong commented on pull request #14536: URL: https://github.com/apache/flink/pull/14536#issuecomment-752920523 cc @leonardBang for reviewing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14534: [FLINK-20822][hive] Don't check whether a function is generic in hive…
flinkbot edited a comment on pull request #14534: URL: https://github.com/apache/flink/pull/14534#issuecomment-752886442 ## CI report: * 7c096f9a7dcf044d1d60afc33e3782379286093b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11536) * 5b018f902fac3974828e119081396fea2b999c60 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14526: [FLINK-14814][webui] Display back pressure and causing back pressure ratios
flinkbot edited a comment on pull request #14526: URL: https://github.com/apache/flink/pull/14526#issuecomment-752592411 ## CI report: * af89d36ef1ec641dd9acb240bd211132a52f3798 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11543) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14530: [FLINK-20348][kafka] Make "schema-registry.subject" optional for Kafka sink with avro-confluent format
flinkbot edited a comment on pull request #14530: URL: https://github.com/apache/flink/pull/14530#issuecomment-752828495 ## CI report: * 2f237865183b4f66838366b4d7114515a612ed81 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11529) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
flinkbot edited a comment on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-751755915 ## CI report: * Unknown: [CANCELED](TBD) * 932157724cf6bd963643aa9970c82ce0e0b2c8fe Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11542) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #10847: [FLINK-15578][connectors/jdbc] implement exactly once JDBC sink
wuchong commented on a change in pull request #10847:
URL: https://github.com/apache/flink/pull/10847#discussion_r550422198
##
File path:
flink-connectors/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/xa/XidImpl.java
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.connector.jdbc.xa;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.util.Preconditions;
+
+import javax.annotation.Nonnull;
+import javax.transaction.xa.Xid;
+
+import java.io.Serializable;
+import java.util.Arrays;
+import java.util.Objects;
+
+/**
+ * A simple {@link Xid} implementation that stores branch and global
transaction identifiers as byte
+ * arrays.
+ */
+@Internal
+final class XidImpl implements Xid, Serializable {
+
+private final int formatId;
+@Nonnull private final byte[] globalTransactionId;
+@Nonnull private final byte[] branchQualifier;
+
+XidImpl(int formatId, byte[] globalTransactionId, byte[] branchQualifier) {
+Preconditions.checkArgument(globalTransactionId.length <=
Xid.MAXGTRIDSIZE);
+Preconditions.checkArgument(branchQualifier.length <=
Xid.MAXBQUALSIZE);
+this.formatId = formatId;
+this.globalTransactionId = Arrays.copyOf(globalTransactionId,
globalTransactionId.length);
+this.branchQualifier = Arrays.copyOf(branchQualifier,
branchQualifier.length);
+}
+
+@Override
+public int getFormatId() {
+return formatId;
+}
+
+@Override
+public byte[] getGlobalTransactionId() {
+return globalTransactionId;
+}
+
+@Override
+public byte[] getBranchQualifier() {
+return branchQualifier;
+}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) {
+return true;
+}
+if (!(o instanceof XidImpl)) {
+return false;
+}
+XidImpl xid = (XidImpl) o;
+return formatId == xid.formatId
+&& Arrays.equals(globalTransactionId, xid.globalTransactionId)
+&& Arrays.equals(branchQualifier, xid.branchQualifier);
+}
+
+@Override
+public int hashCode() {
+int result = Objects.hash(formatId);
+result = 31 * result + Arrays.hashCode(globalTransactionId);
+result = 31 * result + Arrays.hashCode(branchQualifier);
+return result;
+}
+
+@Override
+public String toString() {
+return formatId + ":" + bytesToHex(globalTransactionId) + ":" +
bytesToHex(branchQualifier);
Review comment:
I think we can use `StringUtils.byteToHexString(bytes)` here to avoid
duplicate implementation.
##
File path:
flink-connectors/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/xa/XidImpl.java
##
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.connector.jdbc.xa;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.util.Preconditions;
+
+import javax.annotation.Nonnull;
+import javax.transaction.xa.Xid;
+
+import java.io.Serializable;
+import java.util.Arrays;
+import java.util.Objects;
+
+/**
+ * A simple {@link Xid} implementation that stores branch and global
transaction identifiers as byte
+ * arrays.
+ */
+@Internal
+final class XidImpl implements Xid, Serializable {
Review comment:
Add a `serialVersionUID` to the class?
##
[jira] [Created] (FLINK-20826) Fix bug in streaming SQL examples
Shuo Cheng created FLINK-20826: -- Summary: Fix bug in streaming SQL examples Key: FLINK-20826 URL: https://issues.apache.org/jira/browse/FLINK-20826 Project: Flink Issue Type: New Feature Components: Table SQL / API Affects Versions: 1.12.0 Reporter: Shuo Cheng Fix For: 1.13.0 There is some minor bug in `UpdatingTopCityExample`. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20632) Missing docker images for 1.12 release
[ https://issues.apache.org/jira/browse/FLINK-20632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256929#comment-17256929 ] Xintong Song commented on FLINK-20632: -- The docker guys are still not responding to our 1.12.0 image PR, neither the [previous one|https://github.com/docker-library/official-images/pull/9249] nor the [new one|https://github.com/docker-library/official-images/pull/9345]. I'm not sure is this is because of the Christmas vacation or not. I do see there are new PRs got merged recently. Shall we consider hosting the Docker image ourselves starting 1.12.1? > Missing docker images for 1.12 release > -- > > Key: FLINK-20632 > URL: https://issues.apache.org/jira/browse/FLINK-20632 > Project: Flink > Issue Type: Bug > Components: Deployment / Kubernetes >Affects Versions: 1.12.0 >Reporter: Piotr Gwiazda >Priority: Critical > > Images for Flink 1.12 are missing in Docker hub > https://hub.docker.com/_/flink. As a result Kubernetes deployment as in the > documentation example is not working. > https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/native_kubernetes.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-20632) Missing docker images for 1.12 release
[ https://issues.apache.org/jira/browse/FLINK-20632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256929#comment-17256929 ] Xintong Song edited comment on FLINK-20632 at 12/31/20, 10:26 AM: -- The docker guys are still not responding to our 1.12.0 image PR, neither the [previous one|https://github.com/docker-library/official-images/pull/9249] nor the [new one|https://github.com/docker-library/official-images/pull/9345]. I'm not sure if this is because of the Christmas vacation or not. I do see there are new PRs got merged recently. Shall we consider hosting the Docker image ourselves starting 1.12.1? was (Author: xintongsong): The docker guys are still not responding to our 1.12.0 image PR, neither the [previous one|https://github.com/docker-library/official-images/pull/9249] nor the [new one|https://github.com/docker-library/official-images/pull/9345]. I'm not sure is this is because of the Christmas vacation or not. I do see there are new PRs got merged recently. Shall we consider hosting the Docker image ourselves starting 1.12.1? > Missing docker images for 1.12 release > -- > > Key: FLINK-20632 > URL: https://issues.apache.org/jira/browse/FLINK-20632 > Project: Flink > Issue Type: Bug > Components: Deployment / Kubernetes >Affects Versions: 1.12.0 >Reporter: Piotr Gwiazda >Priority: Critical > > Images for Flink 1.12 are missing in Docker hub > https://hub.docker.com/_/flink. As a result Kubernetes deployment as in the > documentation example is not working. > https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/native_kubernetes.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-20718) Create busyTimeMsPerSecond metrics
[
https://issues.apache.org/jira/browse/FLINK-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-20718:
---
Description:
{{busyTimeMsPerSecond}} is defined as {{1000 - idleTimeMsPerSecond}}, and
that's also how it will be calculated. This indirect approach is used because
the direct approach would require {{System.currentTimeMillis()}} calls for
example once per record.
was:
{{busyTimeMsPerSecond}} is defined as {{1000 - idleTimeMsPerSecond}}, and
that's also how it will be calculated. This indirect approach is used because
the direct approach would require {{System.currentTimeMillis()}} calls for
example once per record.
> Create busyTimeMsPerSecond metrics
> --
>
> Key: FLINK-20718
> URL: https://issues.apache.org/jira/browse/FLINK-20718
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Task
>Reporter: Piotr Nowojski
>Assignee: Piotr Nowojski
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> {{busyTimeMsPerSecond}} is defined as {{1000 - idleTimeMsPerSecond}}, and
> that's also how it will be calculated. This indirect approach is used because
> the direct approach would require {{System.currentTimeMillis()}} calls for
> example once per record.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-20718) Create busyTimeMsPerSecond metrics
[
https://issues.apache.org/jira/browse/FLINK-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-20718:
---
Description:
{{busyTimeMsPerSecond}} is defined as {{1000 - idleTimeMsPerSecond}}, and
that's also how it will be calculated. This indirect approach is used because
the direct approach would require {{System.currentTimeMillis()}} calls for
example once per record.
was:create {{isCausingBackPressureRatio}} and {{isCausingBackPressure}}
metrics. Based on either {{!isBackPressured && inPoolUsage > 0.5}} or
{{!isBackPressured && idleTimeMsPerSecond < 1ms}} . Probably the latter version
is better, as it would neatly ignore the problem that {{inPoolUsage}} metric
doesn't work with {{LocalInputChannel}}
> Create busyTimeMsPerSecond metrics
> --
>
> Key: FLINK-20718
> URL: https://issues.apache.org/jira/browse/FLINK-20718
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Task
>Reporter: Piotr Nowojski
>Assignee: Piotr Nowojski
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> {{busyTimeMsPerSecond}} is defined as {{1000 - idleTimeMsPerSecond}}, and
> that's also how it will be calculated. This indirect approach is used because
> the direct approach would require {{System.currentTimeMillis()}} calls for
> example once per record.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14814) Show the vertex that produces the backpressure source in the job
[
https://issues.apache.org/jira/browse/FLINK-14814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256927#comment-17256927
]
Piotr Nowojski commented on FLINK-14814:
Previous approach using {{isBackPressuredRatio}} and
{{isCausingBackPressureRatio}} had a major problem with accuracy of
measurements, if load spikes were happening quicker/faster then the sampling
rate (it's impossible to accurately sample a wave, with sampling rate smaller
then half of the wave's frequency).
Because of that I switched to another approach: using
{{backPressuredTimeMsPerSecond}} and {{busyTimeMsPerSecond}} which we can
calculate much more accurately.
> Show the vertex that produces the backpressure source in the job
>
>
> Key: FLINK-14814
> URL: https://issues.apache.org/jira/browse/FLINK-14814
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Network, Runtime / REST,
> Runtime / Web Frontend
>Reporter: lining
>Assignee: Piotr Nowojski
>Priority: Major
> Labels: pull-request-available
> Attachments: 2B0E910D-6D95-401F-B450-1F6B1AFB9BEA.png, Screenshot
> 2020-12-30 at 14.09.19.png, Screenshot 2020-12-31 at 10.27.52.png
>
>
> By checking the status of output and input buffer pools exposed via
> FLINK-14815 (output buffer empty, input buffer full) it is possible to
> display which node is a source of the back pressure. This information could
> be displayed/accessible in the Web Frontend.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #14536: [FLINK-20812][Connector][Hbase] hbase in sql mode,can use 'properties.*' add Configuration parameter.
flinkbot edited a comment on pull request #14536: URL: https://github.com/apache/flink/pull/14536#issuecomment-752909965 ## CI report: * f66faaa7fd4901814134ab3d8993c52838fd77bf Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11540) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14535: [FLINK-20026][Connector][Jdbc] Jdbc connector support regular expression
flinkbot edited a comment on pull request #14535: URL: https://github.com/apache/flink/pull/14535#issuecomment-752901366 ## CI report: * 14d72b3b9718fa2f404534d3569332825a7b2680 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11539) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14531: [FLINK-20777][Connector][Kafka] Property "partition.discovery.interval.ms" shoule be enabled by default for unbounded mode, and disab
flinkbot edited a comment on pull request #14531: URL: https://github.com/apache/flink/pull/14531#issuecomment-752828536 ## CI report: * 665de48bdcb5dd56e9fc415183f324e5d7a15e38 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11538) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14526: [FLINK-14814][webui] Display back pressure and causing back pressure ratios
flinkbot edited a comment on pull request #14526: URL: https://github.com/apache/flink/pull/14526#issuecomment-752592411 ## CI report: * 581fee7ea3a656a997664368665cee270e8d8dc7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11503) * af89d36ef1ec641dd9acb240bd211132a52f3798 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
flinkbot edited a comment on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-751755915 ## CI report: * Unknown: [CANCELED](TBD) * 932157724cf6bd963643aa9970c82ce0e0b2c8fe UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14508: [FLINK-20773][format] Support allow-unescaped-control-chars option for JSON format.
flinkbot edited a comment on pull request #14508: URL: https://github.com/apache/flink/pull/14508#issuecomment-751734163 ## CI report: * f1332b021d33a6e4681b0a08ad1c5b58f153c417 UNKNOWN * f4d02e921d2641fc5692617a4dd50ba2fda1128c UNKNOWN * fd8cbf90a807292b0db7b85bda26f1e717b87767 UNKNOWN * 3c03189754755222ce29f4d17485c91532da4a8b UNKNOWN * 5663475ec56efe4b84e6ae2e6cabd6d58db34bf2 UNKNOWN * f5f641fcde2b6d33b89c640994fcb69cafaa00a8 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11528) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-20718) Create busyTimeMsPerSecond metrics
[
https://issues.apache.org/jira/browse/FLINK-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-20718:
---
Summary: Create busyTimeMsPerSecond metrics (was: Create
isCausingBackPressureRatio and isCausingBackPressure metrics)
> Create busyTimeMsPerSecond metrics
> --
>
> Key: FLINK-20718
> URL: https://issues.apache.org/jira/browse/FLINK-20718
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Task
>Reporter: Piotr Nowojski
>Assignee: Piotr Nowojski
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> create {{isCausingBackPressureRatio}} and {{isCausingBackPressure}} metrics.
> Based on either {{!isBackPressured && inPoolUsage > 0.5}} or
> {{!isBackPressured && idleTimeMsPerSecond < 1ms}} . Probably the latter
> version is better, as it would neatly ignore the problem that {{inPoolUsage}}
> metric doesn't work with {{LocalInputChannel}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-20717) Create backPressuredTimeMsPerSecond metric
[
https://issues.apache.org/jira/browse/FLINK-20717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-20717:
---
Description: Create {{backPressuredTimeMsPerSecond}} metric, measured
similarly to the existing {{idleTimeMsPerSecond}}. (was: Create
{{isBackPressuredRatio}} metric, based on a new {{GaugeView}} (equivalent of
{{MeterView}} for {{Gauge}} that {{implements Meter, View}}))
> Create backPressuredTimeMsPerSecond metric
> --
>
> Key: FLINK-20717
> URL: https://issues.apache.org/jira/browse/FLINK-20717
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Task
>Reporter: Piotr Nowojski
>Assignee: Piotr Nowojski
>Priority: Major
> Fix For: 1.13.0
>
>
> Create {{backPressuredTimeMsPerSecond}} metric, measured similarly to the
> existing {{idleTimeMsPerSecond}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-20717) Create backPressuredTimeMsPerSecond metric
[
https://issues.apache.org/jira/browse/FLINK-20717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-20717:
---
Summary: Create backPressuredTimeMsPerSecond metric (was: Create
isBackPressuredRatio metric)
> Create backPressuredTimeMsPerSecond metric
> --
>
> Key: FLINK-20717
> URL: https://issues.apache.org/jira/browse/FLINK-20717
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Metrics, Runtime / Task
>Reporter: Piotr Nowojski
>Assignee: Piotr Nowojski
>Priority: Major
> Fix For: 1.13.0
>
>
> Create {{isBackPressuredRatio}} metric, based on a new {{GaugeView}}
> (equivalent of {{MeterView}} for {{Gauge}} that {{implements Meter, View}})
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-14814) Show the vertex that produces the backpressure source in the job
[ https://issues.apache.org/jira/browse/FLINK-14814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256926#comment-17256926 ] Piotr Nowojski edited comment on FLINK-14814 at 12/31/20, 10:14 AM: New visualisation based on the updated approach: !Screenshot 2020-12-31 at 10.27.52.png|width=1000! was (Author: pnowojski): New visualisation based on the updated approach: !Screenshot 2020-12-31 at 10.27.52.png! > Show the vertex that produces the backpressure source in the job > > > Key: FLINK-14814 > URL: https://issues.apache.org/jira/browse/FLINK-14814 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Metrics, Runtime / Network, Runtime / REST, > Runtime / Web Frontend >Reporter: lining >Assignee: Piotr Nowojski >Priority: Major > Labels: pull-request-available > Attachments: 2B0E910D-6D95-401F-B450-1F6B1AFB9BEA.png, Screenshot > 2020-12-30 at 14.09.19.png, Screenshot 2020-12-31 at 10.27.52.png > > > By checking the status of output and input buffer pools exposed via > FLINK-14815 (output buffer empty, input buffer full) it is possible to > display which node is a source of the back pressure. This information could > be displayed/accessible in the Web Frontend. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14814) Show the vertex that produces the backpressure source in the job
[ https://issues.apache.org/jira/browse/FLINK-14814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Nowojski updated FLINK-14814: --- Attachment: Screenshot 2020-12-31 at 10.27.52.png > Show the vertex that produces the backpressure source in the job > > > Key: FLINK-14814 > URL: https://issues.apache.org/jira/browse/FLINK-14814 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Metrics, Runtime / Network, Runtime / REST, > Runtime / Web Frontend >Reporter: lining >Assignee: Piotr Nowojski >Priority: Major > Labels: pull-request-available > Attachments: 2B0E910D-6D95-401F-B450-1F6B1AFB9BEA.png, Screenshot > 2020-12-30 at 14.09.19.png, Screenshot 2020-12-31 at 10.27.52.png > > > By checking the status of output and input buffer pools exposed via > FLINK-14815 (output buffer empty, input buffer full) it is possible to > display which node is a source of the back pressure. This information could > be displayed/accessible in the Web Frontend. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-14814) Show the vertex that produces the backpressure source in the job
[ https://issues.apache.org/jira/browse/FLINK-14814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256926#comment-17256926 ] Piotr Nowojski edited comment on FLINK-14814 at 12/31/20, 10:14 AM: New visualisation based on the updated approach: !Screenshot 2020-12-31 at 10.27.52.png|width=1500! was (Author: pnowojski): New visualisation based on the updated approach: !Screenshot 2020-12-31 at 10.27.52.png|width=1000! > Show the vertex that produces the backpressure source in the job > > > Key: FLINK-14814 > URL: https://issues.apache.org/jira/browse/FLINK-14814 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Metrics, Runtime / Network, Runtime / REST, > Runtime / Web Frontend >Reporter: lining >Assignee: Piotr Nowojski >Priority: Major > Labels: pull-request-available > Attachments: 2B0E910D-6D95-401F-B450-1F6B1AFB9BEA.png, Screenshot > 2020-12-30 at 14.09.19.png, Screenshot 2020-12-31 at 10.27.52.png > > > By checking the status of output and input buffer pools exposed via > FLINK-14815 (output buffer empty, input buffer full) it is possible to > display which node is a source of the back pressure. This information could > be displayed/accessible in the Web Frontend. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-14814) Show the vertex that produces the backpressure source in the job
[ https://issues.apache.org/jira/browse/FLINK-14814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256926#comment-17256926 ] Piotr Nowojski commented on FLINK-14814: New visualisation based on the updated approach: !Screenshot 2020-12-31 at 10.27.52.png! > Show the vertex that produces the backpressure source in the job > > > Key: FLINK-14814 > URL: https://issues.apache.org/jira/browse/FLINK-14814 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Metrics, Runtime / Network, Runtime / REST, > Runtime / Web Frontend >Reporter: lining >Assignee: Piotr Nowojski >Priority: Major > Labels: pull-request-available > Attachments: 2B0E910D-6D95-401F-B450-1F6B1AFB9BEA.png, Screenshot > 2020-12-30 at 14.09.19.png, Screenshot 2020-12-31 at 10.27.52.png > > > By checking the status of output and input buffer pools exposed via > FLINK-14815 (output buffer empty, input buffer full) it is possible to > display which node is a source of the back pressure. This information could > be displayed/accessible in the Web Frontend. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-20769) Support minibatch to optimize Python UDAF
[ https://issues.apache.org/jira/browse/FLINK-20769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu closed FLINK-20769. --- Assignee: Huang Xingbo Resolution: Fixed Merged to master via f131acd4982aab6c83c3d192326953cf174e808e > Support minibatch to optimize Python UDAF > - > > Key: FLINK-20769 > URL: https://issues.apache.org/jira/browse/FLINK-20769 > Project: Flink > Issue Type: Improvement > Components: API / Python >Reporter: Huang Xingbo >Assignee: Huang Xingbo >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > Support minibatch to optimize Python UDAF -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] dianfu commented on pull request #14524: [FLINK-20769][python] Support minibatch to optimize Python UDAF
dianfu commented on pull request #14524: URL: https://github.com/apache/flink/pull/14524#issuecomment-752913645 Thanks @HuangXingBo for this great work and thanks @WeiZhong94 for the review. Have merged the PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] dianfu closed pull request #14524: [FLINK-20769][python] Support minibatch to optimize Python UDAF
dianfu closed pull request #14524: URL: https://github.com/apache/flink/pull/14524 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] pnowojski closed pull request #14507: [FLINK-20718][FLINK-20717][metrics] Add isCausingBackPressureRatio and isBackPressuredRatio metrics
pnowojski closed pull request #14507: URL: https://github.com/apache/flink/pull/14507 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] pnowojski commented on pull request #14507: [FLINK-20718][FLINK-20717][metrics] Add isCausingBackPressureRatio and isBackPressuredRatio metrics
pnowojski commented on pull request #14507: URL: https://github.com/apache/flink/pull/14507#issuecomment-752912932 Due to problems with accuracy (caused by sampling intervals) I've abandoned this approach. This PR was superseded by https://github.com/apache/flink/pull/14526 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #14528: [FLINK-20615] Clean PartitionRequestClientFactory up if createPartitionRequestClient fails
rkhachatryan commented on a change in pull request #14528:
URL: https://github.com/apache/flink/pull/14528#discussion_r550447001
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##
@@ -69,36 +68,34 @@
NettyPartitionRequestClient createPartitionRequestClient(ConnectionID
connectionId)
throws IOException, InterruptedException {
while (true) {
-AtomicBoolean isTheFirstOne = new AtomicBoolean(false);
-CompletableFuture clientFuture =
-clients.computeIfAbsent(
-connectionId,
-unused -> {
-isTheFirstOne.set(true);
-return new CompletableFuture<>();
-});
-if (isTheFirstOne.get()) {
+final CompletableFuture
newClientFuture =
+new CompletableFuture<>();
+
+final CompletableFuture clientFuture =
+clients.putIfAbsent(connectionId, newClientFuture);
+
+final NettyPartitionRequestClient client;
+
+if (clientFuture == null) {
try {
-clientFuture.complete(connectWithRetries(connectionId));
-} catch (InterruptedException e) {
-clientFuture.complete(null); // let others waiting know
that they should retry
+client = connectWithRetries(connectionId);
+} catch (RemoteTransportException | InterruptedException e) {
Review comment:
I think it will fail the current job but not necessarily subsequent jobs
using the same `ConnectionId`.
In either case, I would not bake in this assumption, as it can change and
makes more difficult to reason about.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14536: [FLINK-20812][Connector][Hbase] hbase in sql mode,can use 'properties.*' add Configuration parameter.
flinkbot commented on pull request #14536: URL: https://github.com/apache/flink/pull/14536#issuecomment-752909965 ## CI report: * f66faaa7fd4901814134ab3d8993c52838fd77bf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
echauchot commented on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-752909374 CI does not seem to be triggered. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
echauchot commented on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-752908798 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
echauchot commented on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-752908614 @flinkbot run azure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] echauchot removed a comment on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
echauchot removed a comment on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-752908017 @flinkbot run azure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] echauchot commented on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
echauchot commented on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-752908017 @flinkbot run azure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20809) Limit push down with Hive table doesn't work when using with filter
[
https://issues.apache.org/jira/browse/FLINK-20809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17256920#comment-17256920
]
Shengkai Fang commented on FLINK-20809:
---
Hi all. I don't think it's a bug.
Currently, the rule {{PushLimitIntoTableSourceScanRule}} captures the structure
that the {{FlinkLogicalSort}} node is the parent of the
{{FlinkLogicalTableSourceScan}}.
In this case, we have a Calc(filter) node between the Sort node and the Scan
node, which makes the rule fails to apply.
In semantic, we can only push down the filter first and then push down the
limit. If we only push the limit into the source, the output row may not
satisify the condition.
Currently, we have the rule to push down the filter, project and limit. The
main problem is to support filter push down for hive connector.
> Limit push down with Hive table doesn't work when using with filter
> ---
>
> Key: FLINK-20809
> URL: https://issues.apache.org/jira/browse/FLINK-20809
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
>Affects Versions: 1.12.0
>Reporter: Jun Zhang
>Priority: Major
> Fix For: 1.13.0
>
>
> when I use flink sql to query hive table , like this
> {code:java}
> // select * from hive_table where id = 1 limit 1
> {code}
>
> when the sql contain query conditions in where clause, I found that the limit
> push down is invalid.
> I look up the comment on source code , I think it is should be push down , is
> it a bug ?
> [the comment
> |https://github.com/apache/flink/blob/master/flink-table/flink-table-planner-blink/src/main/java/org/apache/flink/table/planner/plan/rules/logical/PushLimitIntoTableSourceScanRule.java#L64]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] echauchot commented on pull request #14510: [FLINK-19656] [metrics] filter delimiter in metrics components
echauchot commented on pull request #14510: URL: https://github.com/apache/flink/pull/14510#issuecomment-752907676 @zentol I remade the work from scratch using _FrontMetricGroup_ for filter/delimiter injection: - I made sure that the configured delimiter is used in filtering when creating the metrics identifier - I added a _MetricsGroup#getAllVariables()_ version that takes a _CharacterFilter_. And made sure that the configured delimiter is used in filtering variable names and values. - I added the related tests in _AbstractMetricGroupTest_. PTAL. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #14528: [FLINK-20615] Clean PartitionRequestClientFactory up if createPartitionRequestClient fails
tillrohrmann commented on a change in pull request #14528:
URL: https://github.com/apache/flink/pull/14528#discussion_r550443289
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestClientFactory.java
##
@@ -69,36 +68,34 @@
NettyPartitionRequestClient createPartitionRequestClient(ConnectionID
connectionId)
throws IOException, InterruptedException {
while (true) {
-AtomicBoolean isTheFirstOne = new AtomicBoolean(false);
-CompletableFuture clientFuture =
-clients.computeIfAbsent(
-connectionId,
-unused -> {
-isTheFirstOne.set(true);
-return new CompletableFuture<>();
-});
-if (isTheFirstOne.get()) {
+final CompletableFuture
newClientFuture =
+new CompletableFuture<>();
+
+final CompletableFuture clientFuture =
+clients.putIfAbsent(connectionId, newClientFuture);
+
+final NettyPartitionRequestClient client;
+
+if (clientFuture == null) {
try {
-clientFuture.complete(connectWithRetries(connectionId));
-} catch (InterruptedException e) {
-clientFuture.complete(null); // let others waiting know
that they should retry
+client = connectWithRetries(connectionId);
+} catch (RemoteTransportException | InterruptedException e) {
Review comment:
My assumption was that uncaught exceptions will trigger a fatal failure.
If this isn't the case, then we should indeed catch all exceptions here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14535: [FLINK-20026][Connector][Jdbc] Jdbc connector support regular expression
flinkbot edited a comment on pull request #14535: URL: https://github.com/apache/flink/pull/14535#issuecomment-752901366 ## CI report: * 14d72b3b9718fa2f404534d3569332825a7b2680 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11539) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14531: [FLINK-20777][Connector][Kafka] Property "partition.discovery.interval.ms" shoule be enabled by default for unbounded mode, and disab
flinkbot edited a comment on pull request #14531: URL: https://github.com/apache/flink/pull/14531#issuecomment-752828536 ## CI report: * 9dfbf0f8575de20c3c3d672ce1bf64f3c5e2ed93 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11525) * 665de48bdcb5dd56e9fc415183f324e5d7a15e38 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11538) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14460: [docs/javadoc][hotfix] Explicitly Document task cancellation timeout …
flinkbot edited a comment on pull request #14460: URL: https://github.com/apache/flink/pull/14460#issuecomment-749464826 ## CI report: * 5f59e7821f3e7649b5449307a7a4817a4d0036f5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11447) * 9f7dd68a258dd3ccc7655a5e4c8e819822fc1d8c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=11537) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-20825) Upsert-kafka should throw a better exception when the key of the record is NULL
Shengkai Fang created FLINK-20825: - Summary: Upsert-kafka should throw a better exception when the key of the record is NULL Key: FLINK-20825 URL: https://issues.apache.org/jira/browse/FLINK-20825 Project: Flink Issue Type: Improvement Components: Connectors / Kafka, Table SQL / API Affects Versions: 1.12.0 Reporter: Shengkai Fang Currently, it will get NPE when reading from the upsert-kafka with record whose key is NULL. The records in the compacted topic always have keys. Therefore, it's mainly wrong usage of the upsert-kafka that causing NPE. We can throw a better exception to notify the users. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #14536: [FLINK-20812][Connector][Hbase] hbase in sql mode,can use 'properties.*' add Configuration parameter.
flinkbot commented on pull request #14536: URL: https://github.com/apache/flink/pull/14536#issuecomment-752903382 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit f66faaa7fd4901814134ab3d8993c52838fd77bf (Thu Dec 31 09:34:26 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-20812) flink connector hbase(1.4,2.2) too few control parameters provided
[ https://issues.apache.org/jira/browse/FLINK-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-20812: --- Labels: pull-request-available (was: ) > flink connector hbase(1.4,2.2) too few control parameters provided > -- > > Key: FLINK-20812 > URL: https://issues.apache.org/jira/browse/FLINK-20812 > Project: Flink > Issue Type: Improvement > Components: Connectors / HBase, Table SQL / Ecosystem >Reporter: WeiNan Zhao >Assignee: WeiNan Zhao >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > When I use the cdh cluster, I need to use kerberos authentication, and I need > to add some kerberos authentication parameters of hbase, but the current > hbase connector structure does not provide this entry, I wonder if it can be > modified, if possible, I can submit for hbase connector a pr. > e.g hbase parameter > hbase.security.authentication='kerberos', > hbase.master.kerberos.principal='...', > hbase.kerberos.regionserver.principal='...', > hbase.security.auth.enable = 'true', > hbase.sasl.clientconfig = 'Client' -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] chaozwn opened a new pull request #14536: [FLINK-20812][Connector][Hbase] hbase in sql mode,can use 'properties.*' add Configuration parameter.
chaozwn opened a new pull request #14536: URL: https://github.com/apache/flink/pull/14536 ## What is the purpose of the change This pull request makes hbase in sql mode,can use 'properties.*' add Configuration parameter e.g 'properties.hbase.security.authentication'='kerberos', 'properties.hbase.master.kerberos.principal'='...', ## Brief change log Without affecting the original function, perfect access, custom Configuration parameters ## Verifying this change This change is already covered by existing tests, such as *(please describe tests)*. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? not documented This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14535: [FLINK-20026][Connector][Jdbc] Jdbc connector support regular expression
flinkbot commented on pull request #14535: URL: https://github.com/apache/flink/pull/14535#issuecomment-752901366 ## CI report: * 14d72b3b9718fa2f404534d3569332825a7b2680 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
