[jira] [Created] (FLINK-22286) Fix the state doesn't support custom python class

[Huang Xingbo (Jira)]

Huang Xingbo created FLINK-22286:


 Summary: Fix the state doesn't support custom python class
 Key: FLINK-22286
 URL: https://issues.apache.org/jira/browse/FLINK-22286
 Project: Flink
  Issue Type: Sub-task
  Components: API / Python
Affects Versions: 1.12.2, 1.13.0
Reporter: Huang Xingbo
Assignee: Huang Xingbo
 Fix For: 1.13.0, 1.12.3


Currently Pyflink state backend use pickle to serialize data, which won't work 
when the data is Python custom class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22287) Java documentation of mini cluster should mention usage of before() and after()

[Svend Vanderveken (Jira)]

Svend Vanderveken created FLINK-22287:
-

 Summary: Java documentation of mini cluster should mention usage 
of before() and after()
 Key: FLINK-22287
 URL: https://issues.apache.org/jira/browse/FLINK-22287
 Project: Flink
  Issue Type: Improvement
  Components: Documentation
Reporter: Svend Vanderveken



The java code examples [for Mini cluster usage in unit tests in the current 
documentation|https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/stream/testing.html#junit-rule-miniclusterwithclientresource]
  do not mention that the user should call {{before()}} and {{after()}} methods 
on the mini cluster instance. 

I believe this is a mistake since omitting those calls would make, as far as a 
I understand, the creation of the mini cluster ignored.

The scala code samples seem correct.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22288) Remove unnecesary argument from JdbcSink.exactlyOnceSink

[Roman Khachatryan (Jira)]

Roman Khachatryan created FLINK-22288:
-

 Summary: Remove unnecesary argument from JdbcSink.exactlyOnceSink
 Key: FLINK-22288
 URL: https://issues.apache.org/jira/browse/FLINK-22288
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / JDBC
Reporter: Roman Khachatryan
 Fix For: 1.13.0


As mentioned in 
https://issues.apache.org/jira/browse/FLINK-22141?focusedCommentId=17321934&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17321934

 
{quote}3.2 `JDBC.exactlyOnceSink` has a input arg `JdbcConnectionOptions 
connectionOptions` never used, and the connection info is provided by 
XADataSource. So do we consider remove `JdbcConnectionOptions 
connectionOptions`, it is not used and my confuse users?
{quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22289) Update JDBC XA sink docs

[Roman Khachatryan (Jira)]

Roman Khachatryan created FLINK-22289:
-

 Summary: Update JDBC XA sink docs
 Key: FLINK-22289
 URL: https://issues.apache.org/jira/browse/FLINK-22289
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / JDBC, Documentation
Reporter: Roman Khachatryan
 Fix For: 1.13.0


According to 
https://issues.apache.org/jira/browse/FLINK-22141?focusedCommentId=17321934&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17321934

(except code changes)

 

cc:[~ym]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Welcome to create Pulsar + Flink solutions at Pulsar Hackathon!

[Dianjin Wang]

Hi Flink community members,

I would like to share that the first-ever Apache Pulsar Hackathon (Virtual)
is open for registration[1], until April 28, 2021. The goal of this event
is to engage the open source community, drive contributions, and generate
ideas to enhance Pulsar and its Big Data ecosystems including *Apache
Flink*.

*Apache Pulsar + Flink* Solution is one of the most important categories at
this event - *developing end-to-end general data processing solutions based
on Pulsar and Apache Flink*.

We will announce the winner at the Pulsar Summit North America 2021 on June
16-17, 2021. First place will win $5,000! If you're interested in creative
ideas on Pulsar + Flink, whether you’re a newbie or expert, welcome to join
us! This blog[2] maybe can help you learn more about our event.

I'm not sure this is the right mailing list to inform the Flink community.
If you have any questions, please let me know. I am glad to help.

[1]:
https://www.eventbrite.com/e/apache-pulsar-hackathon-2021-tickets-143906003731
[2]:
https://streamnative.io/en/blog/community/2021-03-11-introducing-the-apache-pulsar-hackathon-2021

Best,
Dianjin Wang

[jira] [Created] (FLINK-22290) Add new unaligned checkpiont options to Python API.

[Arvid Heise (Jira)]

Arvid Heise created FLINK-22290:
---

 Summary: Add new unaligned checkpiont options to Python API.
 Key: FLINK-22290
 URL: https://issues.apache.org/jira/browse/FLINK-22290
 Project: Flink
  Issue Type: Improvement
  Components: API / Python, Runtime / Checkpointing
Reporter: Arvid Heise
Assignee: Arvid Heise
 Fix For: 1.13.0


There is currently no python equivalent of

{noformat}
CheckpointConfig#setAlignmentTimeout
CheckpointConfig#setForceUnalignedCheckpoints
{noformat}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22291) Fix split switching in HiveInputFormatPartitionReader

[Miklos Gergely (Jira)]

Miklos Gergely created FLINK-22291:
--

 Summary: Fix split switching in HiveInputFormatPartitionReader
 Key: FLINK-22291
 URL: https://issues.apache.org/jira/browse/FLINK-22291
 Project: Flink
  Issue Type: Bug
Reporter: Miklos Gergely


Due to a bug if HiveInputFormatPartitionReader has read all data from an input 
split, and switches to a next one in hasNext(), no data will be read from that 
new one, thus it will return an empty row. The problem is that the reachedEnd() 
function call "loads" the next row, which is not invoked in case of split 
switching, thus the row will be empty.
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22292) Add documentation about state access in Python DataStream API

[Dian Fu (Jira)]

Dian Fu created FLINK-22292:
---

 Summary: Add documentation about state access in Python DataStream 
API
 Key: FLINK-22292
 URL: https://issues.apache.org/jira/browse/FLINK-22292
 Project: Flink
  Issue Type: Improvement
  Components: API / Python, Documentation
Reporter: Dian Fu
Assignee: Dian Fu
 Fix For: 1.13.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Please submit talk proposals for the ApacheCons as soon as possilbe

[Christofer Dutz]

Hi Flink folks,

As the chair of the IoT Track for the NA/EU ApacheCon and Co-Chair of the Asia, 
I regularly review the submissions for the IoT tracks.

I have noticed that there aren't any submissions from your project.

If you plan on submitting something, I strongly ask to you do it soon ... by 
submitting on one of the last days you eliminate the chance for coordination 
and are more likely to get a "Rejected". If you submit early, we can all work 
on getting the submissions into shape that the chance of getting accepted is 
higher.

AAND you help getting my life a lot easier, because I don't have to do all 
the planning stuff in 2-3 days which I normally have to do.

So ... if you plan on submitting: Do it ASAP ... keep in mind, you can always 
edit proposals, add/remove speakers and even withdraw a talk without any effort.

Thanks,

Chris

[jira] [Created] (FLINK-22293) Throw IllegalArgumentException when getNumFiles with partitions on different hdfs nameservices

[Junfan Zhang (Jira)]

Junfan Zhang created FLINK-22293:


 Summary: Throw IllegalArgumentException when getNumFiles with 
partitions on different hdfs nameservices
 Key: FLINK-22293
 URL: https://issues.apache.org/jira/browse/FLINK-22293
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Hive
Affects Versions: 1.12.2
Reporter: Junfan Zhang






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22294) Throw IllegalArgumentException when getNumFiles with partitions on different hdfs nameservices

[Junfan Zhang (Jira)]

Junfan Zhang created FLINK-22294:


 Summary: Throw IllegalArgumentException when getNumFiles with 
partitions on different hdfs nameservices
 Key: FLINK-22294
 URL: https://issues.apache.org/jira/browse/FLINK-22294
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Hive
Affects Versions: 1.12.2
Reporter: Junfan Zhang






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22295) Document sort-merge blocking shuffle

[Till Rohrmann (Jira)]

Till Rohrmann created FLINK-22295:
-

 Summary: Document sort-merge blocking shuffle
 Key: FLINK-22295
 URL: https://issues.apache.org/jira/browse/FLINK-22295
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / Network
Affects Versions: 1.12.2, 1.13.0
Reporter: Till Rohrmann
 Fix For: 1.13.0


We should document how to use the  sort-merge blocking shuffle. At the moment 
only the configuration options are documented. I suggest to also explain when 
to use this shuffle and what the benefits are.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[ANNOUNCE] Apache Flink Stateful Functions 3.0.0 released

[Tzu-Li (Gordon) Tai]

The Apache Flink community is very happy to announce the release of Apache
Flink Stateful Functions (StateFun) 3.0.0.

StateFun is a cross-platform stack for building Stateful Serverless
applications, making it radically simpler to develop scalable, consistent,
and elastic distributed applications.

Please check out the release blog post for an overview of the release:
https://flink.apache.org/news/2021/04/15/release-statefun-3.0.0.html

The release is available for download at:
https://flink.apache.org/downloads.html

Maven artifacts for StateFun can be found at:
https://search.maven.org/search?q=g:org.apache.flink%20statefun

Python SDK for StateFun published to the PyPI index can be found at:
https://pypi.org/project/apache-flink-statefun/

Official Docker images for StateFun are published to Docker Hub:
https://hub.docker.com/r/apache/flink-statefun

The full release notes are available in Jira:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12348822

We would like to thank all contributors of the Apache Flink community who
made this release possible!

Cheers,
Gordon

Re: [VOTE] Deprecating Mesos support

[Wanqiang Ji]

+1 (non-binding)

BR,
Wanqiang Ji

On Thu, Apr 15, 2021 at 9:57 AM Yang Wang  wrote:

> +1
>
> Best,
> Yang
>
> Robert Metzger  于2021年4月15日周四 上午2:49写道：
>
> > +1
> >
> > On Wed, Apr 14, 2021 at 11:32 AM Arvid Heise  wrote:
> >
> > > +1
> > >
> > > On Wed, Apr 14, 2021 at 11:21 AM Xintong Song 
> > > wrote:
> > >
> > > > +1
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song
> > > >
> > > >
> > > >
> > > > On Wed, Apr 14, 2021 at 4:58 PM Yangze Guo 
> wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Best,
> > > > > Yangze Guo
> > > > >
> > > > > On Wed, Apr 14, 2021 at 4:56 PM Chesnay Schepler <
> ches...@apache.org
> > >
> > > > > wrote:
> > > > > >
> > > > > > +1
> > > > > >
> > > > > > On 4/14/2021 10:11 AM, Till Rohrmann wrote:
> > > > > > > +1
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Till
> > > > > > >
> > > > > > > On Wed, Apr 14, 2021 at 10:10 AM Konstantin Knauf <
> > > kna...@apache.org
> > > > >
> > > > > wrote:
> > > > > > >
> > > > > > >> +1
> > > > > > >>
> > > > > > >> On Wed, Apr 14, 2021 at 9:46 AM Matthias Pohl <
> > > > matth...@ververica.com
> > > > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Hi everyone,
> > > > > > >>> After the discussion of deprecating the Mesos support went on
> > > for a
> > > > > while
> > > > > > >>> now in [1] and considering that we already plan to retire it
> > > > mid-term
> > > > > > >> [2],
> > > > > > >>> I want to start a vote on deprecating the Mesos support in
> the
> > > > Flink
> > > > > > >>> documentation as a next step. Ideally, we could even add this
> > > > change
> > > > > to
> > > > > > >>> 1.13.
> > > > > > >>>
> > > > > > >>> Please vote +1 to approve, or -1 with a comment. The vote
> will
> > be
> > > > > open at
> > > > > > >>> least until Friday, Mar 16, 2021.
> > > > > > >>>
> > > > > > >>> Best,
> > > > > > >>> Matthias
> > > > > > >>>
> > > > > > >>> [1]
> > > > > > >>>
> > > > > > >>>
> > > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/SURVEY-Remove-Mesos-support-td45974.html
> > > > > > >>> [2] https://flink.apache.org/roadmap.html#feature-radar
> > > > > > >>>
> > > > > > >>
> > > > > > >> --
> > > > > > >>
> > > > > > >> Konstantin Knauf
> > > > > > >>
> > > > > > >> https://twitter.com/snntrable
> > > > > > >>
> > > > > > >> https://github.com/knaufk
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS]FLIP-150: Introduce Hybrid Source

[Thomas Weise]

Hi Stephan,

Thanks for the feedback!

I agree with the approach of starting with a simple implementation
that can address a well understood, significant portion of use cases.

I'm planning to continue work on the prototype that I had shared.
There is production level usage waiting for it fairly soon. I expect
to open a PR in the coming weeks.

Thomas





On Tue, Apr 13, 2021 at 12:15 PM Stephan Ewen  wrote:
>
> Thanks all for this discussion. Looks like there are lots of ideas and
> folks that are eager to do things, so let's see how we can get this moving.
>
> My take on this is the following:
>
> There will probably not be one Hybrid source, but possibly multiple ones,
> because of different strategies/requirements.
> - One may be very simple, with switching points known up-front. Would
> be good to have this in a very simple implementation.
> - There may be one where the switch is dynamic and the readers need to
> report back where they left off.
> - There may be one that switches back and forth multiple times during a
> job, for example Kakfa going to DFS whenever it falls behind retention, in
> order to catch up again.
>
> This also seems hard to "design on paper"; I expect there are nuances in a
> production setup that affect some details of the design. So I'd feel most
> comfortable in adding a variant of the hybrid source to Flink that has been
> used already in a real use case (not necessarily in production, but maybe
> in a testing/staging environment, so it seems to meet all requirements).
>
>
> What do you think about the following approach?
>   - If there is a tested PoC, let's try to get it contributed to Flink
> without trying to make it much more general.
>   - When we see similar but a bit different requirements for another hybrid
> source, then let's try to evolve the contributed one.
>   - If we see new requirements that are so different that they don't fit
> well with the existing hybrid source, then let us look at building a second
> hybrid source for those requirements.
>
> We need to make connector contributions in general more easy, and I think
> it is not a bad thing to end up with different approaches and see how these
> play out against each other when being used by users. For example switching
> with known boundaries, dynamic switching, back-and-forth-switching, etc.
> (I know some committers are planning to do some work on making
> connector contributions easier, with standardized testing frameworks,
> decoupled CI, etc.)
>
> Best,
> Stephan
>
>
> On Thu, Mar 25, 2021 at 4:41 AM Thomas Weise  wrote:
>
> > Hi,
> >
> > As mentioned in my previous email, I had been working on a prototype for
> > the hybrid source.
> >
> > You can find it at https://github.com/tweise/flink/pull/1
> >
> > It contains:
> > * Switching with configurable chain of sources
> > * Fixed or dynamic start positions
> > * Test with MockSource and FileSource
> >
> > The purpose of the above PR is to gather feedback and help drive consensus
> > on the FLIP.
> >
> > * How to support a dynamic start position within the source chain?
> >
> > Relevant in those (few?) cases where start positions are not known upfront.
> > You can find an example of what that might look like in the tests:
> >
> >
> > https://github.com/tweise/flink/pull/1/files#diff-8eda4e21a8a53b70c46d30ceecfbfd8ffdb11c14580ca048fa4210564f63ada3R62
> >
> > https://github.com/tweise/flink/pull/1/files#diff-3a5658515bb213f9a66db88d45a85971d8c68f64cdc52807622acf27fa703255R132
> >
> > When switching, the enumerator of the previous source needs to
> > supply information about consumed splits that allows to set the start
> > position for the next source. That could be something like the last
> > processed file, timestamp, etc. (Currently StaticFileSplitEnumerator
> > doesn't track finished splits.)
> >
> > See previous discussion regarding start/end position. The prototype shows
> > the use of checkpoint state with converter function.
> >
> > * Should readers be deployed dynamically?
> >
> > The prototype assumes a static source chain that is fixed at job submission
> > time. Conceivably there could be use cases that require more flexibility.
> > Such as switching one KafkaSource for another. A step in that direction
> > would be to deploy the actual readers dynamically, at the time of switching
> > source.
> >
> > Looking forward to feedback and suggestions for next steps!
> >
> > Thomas
> >
> > On Sun, Mar 14, 2021 at 11:17 AM Thomas Weise  wrote:
> >
> > > Hi Nicholas,
> > >
> > > Thanks for the reply. I had implemented a small PoC. It switches a
> > > configurable sequence of sources with predefined bounds. I'm using the
> > > unmodified MockSource for illustration. It does not require a
> > "Switchable"
> > > interface. I looked at the code you shared and the delegation and
> > signaling
> > > works quite similar. That's a good validation.
> > >
> > > Hi Kezhu,
> > >
> > > Thanks for bringing the more detailed discussion regarding the s

[jira] [Created] (FLINK-22296) Introduce Preconditions-util into Python API

[Roc Marshal (Jira)]

Roc Marshal created FLINK-22296:
---

 Summary: Introduce Preconditions-util into Python API
 Key: FLINK-22296
 URL: https://issues.apache.org/jira/browse/FLINK-22296
 Project: Flink
  Issue Type: Improvement
  Components: API / Python
Reporter: Roc Marshal


Similar to the Preconditions c[lass in flink 
java|https://github.com/apache/flink/blob/87efae4d3180a52e16240a0b4bbb197f85acd22c/flink-core/src/main/java/org/apache/flink/util/Preconditions.java#L43]
 API



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: handle SUSPENDED in ZooKeeperLeaderRetrievalService

[Chen Qin]

Hi there,

Thanks for providing points to related changes and jira. Some updates from
our side, we applied a path by merging FLINK-10052
 with master as well as
only handling lost state leveraging SessionConnectionStateErrorPolicy
FLINK-10052  introduced.

Preliminary results were good, the same workload (240 TM) on the same
environment runs stable without frequent restarts due to suspended state
(seems false positive). We are working on more stringent load testing as
well as chaos testing (blocking zk). Will keep folks posted.

Thanks,
Chen


On Tue, Apr 13, 2021 at 1:34 AM Till Rohrmann  wrote:

> Hi Chenqin,
>
> The current rationale behind assuming a leadership loss when seeing a
> SUSPENDED connection is to assume the worst and to be on the safe side.
>
> Yang Wang is correct. FLINK-10052 [1] has the goal to make the behaviour
> configurable. Unfortunately, the community did not have enough time to
> complete this feature.
>
> [1] https://issues.apache.org/jira/browse/FLINK-10052
>
> Cheers,
> Till
>
> On Tue, Apr 13, 2021 at 8:25 AM Yang Wang  wrote:
>
> > This might be related with FLINK-10052[1].
> > Unfortunately, we do not have any progress on this ticket.
> >
> > cc @Till Rohrmann 
> >
> > Best,
> > Yang
> >
> > chenqin  于2021年4月13日周二 上午7:31写道：
> >
> >> Hi there,
> >>
> >> We observed several 1.11 job running in 1.11 restart due to job leader
> >> lost.
> >> Dig deeper, the issue seems related to SUSPENDED state handler in
> >> ZooKeeperLeaderRetrievalService.
> >>
> >> ASFAIK, suspended state is expected when zk is not certain if leader is
> >> still alive. It can follow up with RECONNECT or LOST. In current
> >> implementation [1] , we treat suspended state same as lost state and
> >> actively shutdown job. This pose stability issue on large HA setting.
> >>
> >> My question is can we get some insight behind this decision and could we
> >> add
> >> some tunable configuration for user to decide how long they can endure
> >> such
> >> uncertain suspended state in their jobs.
> >>
> >> Thanks,
> >> Chen
> >>
> >> [1]
> >>
> >>
> https://github.com/apache/flink/blob/release-1.11/flink-runtime/src/main/java/org/apache/flink/runtime/leaderretrieval/ZooKeeperLeaderRetrievalService.java#L201
> >>
> >>
> >>
> >>
> >> --
> >> Sent from:
> >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
> >>
> >
>

Re: handle SUSPENDED in ZooKeeperLeaderRetrievalService

[Yang Wang]

Thanks for trying the unfinished PR and sharing the testing results. Glad
to here that it could work
and really hope the result of more stringent load testing.

After then I think we could revive this ticket.


Best,
Yang

Chen Qin  于2021年4月16日周五 上午2:01写道：

> Hi there,
>
> Thanks for providing points to related changes and jira. Some updates from
> our side, we applied a path by merging FLINK-10052
>  with master as well
> as only handling lost state leveraging SessionConnectionStateErrorPolicy
> FLINK-10052 
>  introduced.
>
> Preliminary results were good, the same workload (240 TM) on the same
> environment runs stable without frequent restarts due to suspended state
> (seems false positive). We are working on more stringent load testing as
> well as chaos testing (blocking zk). Will keep folks posted.
>
> Thanks,
> Chen
>
>
> On Tue, Apr 13, 2021 at 1:34 AM Till Rohrmann 
> wrote:
>
>> Hi Chenqin,
>>
>> The current rationale behind assuming a leadership loss when seeing a
>> SUSPENDED connection is to assume the worst and to be on the safe side.
>>
>> Yang Wang is correct. FLINK-10052 [1] has the goal to make the behaviour
>> configurable. Unfortunately, the community did not have enough time to
>> complete this feature.
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-10052
>>
>> Cheers,
>> Till
>>
>> On Tue, Apr 13, 2021 at 8:25 AM Yang Wang  wrote:
>>
>> > This might be related with FLINK-10052[1].
>> > Unfortunately, we do not have any progress on this ticket.
>> >
>> > cc @Till Rohrmann 
>> >
>> > Best,
>> > Yang
>> >
>> > chenqin  于2021年4月13日周二 上午7:31写道：
>> >
>> >> Hi there,
>> >>
>> >> We observed several 1.11 job running in 1.11 restart due to job leader
>> >> lost.
>> >> Dig deeper, the issue seems related to SUSPENDED state handler in
>> >> ZooKeeperLeaderRetrievalService.
>> >>
>> >> ASFAIK, suspended state is expected when zk is not certain if leader is
>> >> still alive. It can follow up with RECONNECT or LOST. In current
>> >> implementation [1] , we treat suspended state same as lost state and
>> >> actively shutdown job. This pose stability issue on large HA setting.
>> >>
>> >> My question is can we get some insight behind this decision and could
>> we
>> >> add
>> >> some tunable configuration for user to decide how long they can endure
>> >> such
>> >> uncertain suspended state in their jobs.
>> >>
>> >> Thanks,
>> >> Chen
>> >>
>> >> [1]
>> >>
>> >>
>> https://github.com/apache/flink/blob/release-1.11/flink-runtime/src/main/java/org/apache/flink/runtime/leaderretrieval/ZooKeeperLeaderRetrievalService.java#L201
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Sent from:
>> >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>> >>
>> >
>>
>

[jira] [Created] (FLINK-22297) Perform early check to ensure that the length of the result is the same as the input for Pandas UDF

[Dian Fu (Jira)]

Dian Fu created FLINK-22297:
---

 Summary: Perform early check to ensure that the length of the 
result is the same as the input for Pandas UDF
 Key: FLINK-22297
 URL: https://issues.apache.org/jira/browse/FLINK-22297
 Project: Flink
  Issue Type: Improvement
  Components: API / Python
Reporter: Dian Fu
Assignee: Dian Fu


For Pandas UDF, the input type for each input argument is Pandas.Series and the 
result type is also of type Pandas.Series. Besides, the length of the result 
should be the same as the inputs. If this is not the case, currently the 
behavior is unclear. We should perform early check for this and provide a clear 
error message.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22298) The ExecNode's id should always start from 1 in json plan test

[godfrey he (Jira)]

godfrey he created FLINK-22298:
--

 Summary: The ExecNode's id should always start from 1 in json plan 
test
 Key: FLINK-22298
 URL: https://issues.apache.org/jira/browse/FLINK-22298
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: godfrey he
Assignee: godfrey he
 Fix For: 1.13.0


The ExecNode's id should always start from 1 in json plan test, which could 
make the test more stable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22299) should remind users to ignoreParseErrors option when parser failed

[Jingsong Lee (Jira)]

Jingsong Lee created FLINK-22299:


 Summary: should remind users to ignoreParseErrors option when 
parser failed
 Key: FLINK-22299
 URL: https://issues.apache.org/jira/browse/FLINK-22299
 Project: Flink
  Issue Type: Improvement
  Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table 
SQL / Ecosystem
Reporter: Jingsong Lee
 Fix For: 1.14.0


For JSON and CSV, we provide ignoreParseErrors option, but user may not aware 
of it.

We can remind users when parser failed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22300) Why TimeEvictor of Keyed Windows evictor do not support ProcessingTime of TimeCharacteristic

[Bo Huang (Jira)]

Bo Huang created FLINK-22300:


 Summary: Why TimeEvictor of Keyed Windows evictor do not support 
ProcessingTime of TimeCharacteristic
 Key: FLINK-22300
 URL: https://issues.apache.org/jira/browse/FLINK-22300
 Project: Flink
  Issue Type: Improvement
  Components: API / DataStream
Affects Versions: 1.11.3
Reporter: Bo Huang


StreamExecutionEnvironment.setStreamTimeCharacteristic(TimeCharacteristic.ProcessingTime).

The data after Windows process can not be evicted by TimeEvictor of Keyed 
Windows Beause TimestampedValue have no timestamp value.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (FLINK-22301) Statebackend and CheckpointStorage type is not shown in the Web UI

[Yun Gao (Jira)]

Yun Gao created FLINK-22301:
---

 Summary: Statebackend and CheckpointStorage type is not shown in 
the Web UI
 Key: FLINK-22301
 URL: https://issues.apache.org/jira/browse/FLINK-22301
 Project: Flink
  Issue Type: Improvement
  Components: Runtime / Checkpointing, Runtime / Web Frontend
Affects Versions: 1.13.0
Reporter: Yun Gao






--
This message was sent by Atlassian Jira
(v8.3.4#803005)