[jira] [Updated] (FLINK-11454) Support MergedStream operation
[
https://issues.apache.org/jira/browse/FLINK-11454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-11454:
---
Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
(was: auto-deprioritized-major auto-unassigned stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Support MergedStream operation
> --
>
> Key: FLINK-11454
> URL: https://issues.apache.org/jira/browse/FLINK-11454
> Project: Flink
> Issue Type: New Feature
> Components: API / DataStream
>Reporter: Rong Rong
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned
>
> {{Following SlicedStream, the mergedStream operator merges results from
> sliced stream and produces windowing results.
> {code:java}
> val slicedStream: SlicedStream = inputStream
> .keyBy("key")
> .sliceWindow(Time.seconds(5L)) // new “slice window” concept: to
> combine
>// tumble results based on discrete
>// non-overlapping windows.
> .aggregate(aggFunc)
> val mergedStream1: MergedStream = slicedStream
> .slideOver(Time.second(10L)) // combine slice results with same
>
>// windowing function, equivalent to
>// WindowOperator with an aggregate
> state
>// and derived aggregate function.
> val mergedStream2: MergedStream = slicedStream
> .slideOver(Count.of(5))
> .apply(windowFunction) // apply a different window function
> over
>// the sliced results.{code}
> MergedStream are produced by MergeOperator:
> {{slideOver}} and {{apply}} can be combined into a {{OVER AGGREGATE}}
> implementation similar to the one in TableAPI.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-11868) [filesystems] Introduce listStatusIterator API to file system
[
https://issues.apache.org/jira/browse/FLINK-11868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-11868:
---
Labels: auto-deprioritized-major auto-unassigned stale-assigned (was:
auto-deprioritized-major auto-unassigned)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issue is assigned but has not
received an update in 30 days, so it has been labeled "stale-assigned".
If you are still working on the issue, please remove the label and add a
comment updating the community on your progress. If this issue is waiting on
feedback, please consider this a reminder to the committer/reviewer. Flink is a
very active project, and so we appreciate your patience.
If you are no longer working on the issue, please unassign yourself so someone
else may work on it.
> [filesystems] Introduce listStatusIterator API to file system
> -
>
> Key: FLINK-11868
> URL: https://issues.apache.org/jira/browse/FLINK-11868
> Project: Flink
> Issue Type: New Feature
> Components: FileSystems
>Reporter: Yun Tang
>Assignee: Yun Tang
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-assigned
> Fix For: 1.15.0
>
>
> From existed experience, we know {{listStatus}} is expensive for many
> distributed file systems especially when the folder contains too many files.
> This method would not only block the thread until result is return but also
> could cause OOM due to the returned array of {{FileStatus}} is really large.
> I think we should already learn it from FLINK-7266 and FLINK-8540.
> However, list file status under a path is really helpful in many situations.
> Thankfully, many distributed file system noticed that and provide API such as
> {{[listStatusIterator|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#listStatusIterator(org.apache.hadoop.fs.Path)]}}
> to call the file system on demand.
>
> We should also introduce this API and replace current implementation which
> used previous {{listStatus}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-21117) KafkaProducerExactlyOnceITCase fails with "Exceeded checkpoint tolerable failure threshold."
[
https://issues.apache.org/jira/browse/FLINK-21117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-21117:
---
Labels: auto-deprioritized-major pull-request-available stale-assigned
test-stability (was: auto-deprioritized-major pull-request-available
test-stability)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issue is assigned but has not
received an update in 30 days, so it has been labeled "stale-assigned".
If you are still working on the issue, please remove the label and add a
comment updating the community on your progress. If this issue is waiting on
feedback, please consider this a reminder to the committer/reviewer. Flink is a
very active project, and so we appreciate your patience.
If you are no longer working on the issue, please unassign yourself so someone
else may work on it.
> KafkaProducerExactlyOnceITCase fails with "Exceeded checkpoint tolerable
> failure threshold."
>
>
> Key: FLINK-21117
> URL: https://issues.apache.org/jira/browse/FLINK-21117
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Tests
>Affects Versions: 1.12.2, 1.13.0
>Reporter: Dawid Wysakowicz
>Assignee: Qingsheng Ren
>Priority: Minor
> Labels: auto-deprioritized-major, pull-request-available,
> stale-assigned, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12398&view=logs&j=4be4ed2b-549a-533d-aa33-09e28e360cc8&t=0db94045-2aa0-53fa-f444-0130d6933518
> {code}
> 2021-01-22T23:26:30.2357543Z
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-01-22T23:26:30.2359069Z at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-01-22T23:26:30.2360668Z at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$2(MiniClusterJobClient.java:117)
> 2021-01-22T23:26:30.2361340Z at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-01-22T23:26:30.2361840Z at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-01-22T23:26:30.2362915Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T23:26:30.2363367Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T23:26:30.2364353Z at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:238)
> 2021-01-22T23:26:30.2364961Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-01-22T23:26:30.2365453Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-01-22T23:26:30.2365945Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T23:26:30.2366487Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T23:26:30.2367049Z at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046)
> 2021-01-22T23:26:30.2367478Z at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-01-22T23:26:30.2367854Z at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-01-22T23:26:30.2368504Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-01-22T23:26:30.2368897Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-01-22T23:26:30.2369329Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
> 2021-01-22T23:26:30.2369870Z at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-01-22T23:26:30.2370616Z at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68)
> 2021-01-22T23:26:30.2371383Z at
> scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:284)
> 2021-01-22T23:26:30.2372228Z at
> scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:284)
> 2021-01-22T23:26:30.2373105Z at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:284)
> 2021-01-22T23:26:30.2373665Z at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:573)
> 2021-01-22T23:26:30.2374466Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-01-22T23:26:30.2375351Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-01-22T23:26:30.2376150Z at
> scala.concurrent.Future.$anonfun$andThen$1(Future.scala:532)
> 2021-
[jira] [Updated] (FLINK-21716) Support higher precision for Data Type TIME(p)
[ https://issues.apache.org/jira/browse/FLINK-21716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-21716: --- Labels: auto-deprioritized-major auto-unassigned stale-minor (was: auto-deprioritized-major auto-unassigned) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Support higher precision for Data Type TIME(p) > --- > > Key: FLINK-21716 > URL: https://issues.apache.org/jira/browse/FLINK-21716 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Reporter: Leonard Xu >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > Due to the historical reason, we only support TIME(3) yet, we can support > higher precision eg. TIME(9). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-21877) Add E2E test for upsert-kafka connector
[ https://issues.apache.org/jira/browse/FLINK-21877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-21877: --- Labels: auto-deprioritized-major auto-unassigned stale-minor (was: auto-deprioritized-major auto-unassigned) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Add E2E test for upsert-kafka connector > --- > > Key: FLINK-21877 > URL: https://issues.apache.org/jira/browse/FLINK-21877 > Project: Flink > Issue Type: Technical Debt > Components: Connectors / Kafka, Table SQL / Ecosystem >Reporter: Shengkai Fang >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11372) Incorrect delegation of compatibility checks to new snapshots in CollectionSerializerConfigSnapshot
[
https://issues.apache.org/jira/browse/FLINK-11372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-11372:
---
Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
pull-request-available (was: auto-deprioritized-major auto-unassigned
pull-request-available stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Incorrect delegation of compatibility checks to new snapshots in
> CollectionSerializerConfigSnapshot
> ---
>
> Key: FLINK-11372
> URL: https://issues.apache.org/jira/browse/FLINK-11372
> Project: Flink
> Issue Type: Bug
> Components: API / Type Serialization System
>Reporter: Tzu-Li (Gordon) Tai
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned, pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In {{CollectionSerializerConfigSnapshot}}:
> {code}
> @Override
> public TypeSerializerSchemaCompatibility
> resolveSchemaCompatibility(TypeSerializer newSerializer) {
> if (newSerializer instanceof ListSerializer) {
> ListSerializer newListSerializer = (ListSerializer)
> newSerializer;
> ListSerializerSnapshot listSerializerSnapshot = new
> ListSerializerSnapshot<>(newListSerializer);
> @SuppressWarnings("unchecked")
> TypeSerializerSchemaCompatibility result =
> (TypeSerializerSchemaCompatibility)
>
> listSerializerSnapshot.resolveSchemaCompatibility(newListSerializer);
> return result;
> } else {
> return super.resolveSchemaCompatibility(newSerializer);
> }
> }
> {code}
> Compatibility check of {{ListSerializer}} is delegated to the new list
> serializer snapshot class, {{ListSerializerSnapshot}}.
> However, it is incorrect to let the delegate wrap the new serializer (and
> therefore the new nested element serializer). By doing that, we're
> essentially checking compatibility of the new serializer with itself, whereas
> it should be checking compatibility with the restored serializer.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-22190) no guarantee on Flink exactly_once sink to Kafka
[
https://issues.apache.org/jira/browse/FLINK-22190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-22190:
---
Labels: auto-deprioritized-major stale-minor (was:
auto-deprioritized-major)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> no guarantee on Flink exactly_once sink to Kafka
> -
>
> Key: FLINK-22190
> URL: https://issues.apache.org/jira/browse/FLINK-22190
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.12.2
> Environment: *flink: 1.12.2*
> *kafka: 2.7.0*
>Reporter: Spongebob
>Priority: Minor
> Labels: auto-deprioritized-major, stale-minor
>
> When I tried to test the function of flink exactly_once sink to kafka, I
> found it can not run as expectation. here's the pipline of the flink
> applications: raw data(flink app0)-> kafka topic1 -> flink app1 -> kafka
> topic2 -> flink app2, flink tasks may met / byZeroException in random. Below
> shows the codes:
> {code:java}
> //代码占位符
> raw data, flink app0:
> class SimpleSource1 extends SourceFunction[String] {
> var switch = true
> val students: Array[String] = Array("Tom", "Jerry", "Gory")
> override def run(sourceContext: SourceFunction.SourceContext[String]): Unit
> = {
> var i = 0
> while (switch) {
> sourceContext.collect(s"${students(Random.nextInt(students.length))},$i")
> i += 1
> Thread.sleep(5000)
> }
> }
> override def cancel(): Unit = switch = false
> }
> val streamEnv = StreamExecutionEnvironment.getExecutionEnvironment
> val dataStream = streamEnv.addSource(new SimpleSource1)
> dataStream.addSink(new FlinkKafkaProducer[String]("xfy:9092",
> "single-partition-topic-2", new SimpleStringSchema()))
> streamEnv.execute("sink kafka")
>
> flink-app1:
> val streamEnv = StreamExecutionEnvironment.getExecutionEnvironment
> streamEnv.enableCheckpointing(1000, CheckpointingMode.EXACTLY_ONCE)
> val prop = new Properties()
> prop.setProperty("bootstrap.servers", "xfy:9092")
> prop.setProperty("group.id", "test")
> val dataStream = streamEnv.addSource(new FlinkKafkaConsumer[String](
> "single-partition-topic-2",
> new SimpleStringSchema,
> prop
> ))
> val resultStream = dataStream.map(x => {
> val data = x.split(",")
> (data(0), data(1), data(1).toInt / Random.nextInt(5)).toString()
> }
> )
> resultStream.print().setParallelism(1)
> val propProducer = new Properties()
> propProducer.setProperty("bootstrap.servers", "xfy:9092")
> propProducer.setProperty("transaction.timeout.ms", s"${1000 * 60 * 5}")
> resultStream.addSink(new FlinkKafkaProducer[String](
> "single-partition-topic",
> new MyKafkaSerializationSchema("single-partition-topic"),
> propProducer,
> Semantic.EXACTLY_ONCE))
> streamEnv.execute("sink kafka")
>
> flink-app2:
> val streamEnv = StreamExecutionEnvironment.getExecutionEnvironment
> val prop = new Properties()
> prop.setProperty("bootstrap.servers", "xfy:9092")
> prop.setProperty("group.id", "test")
> prop.setProperty("isolation_level", "read_committed")
> val dataStream = streamEnv.addSource(new FlinkKafkaConsumer[String](
> "single-partition-topic",
> new SimpleStringSchema,
> prop
> ))
> dataStream.print().setParallelism(1)
> streamEnv.execute("consumer kafka"){code}
>
> flink app1 will print some duplicate numbers, and to my expectation flink
> app2 will deduplicate them but the fact shows not.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-22441) In Flink v1.11.3 contains netty(version:3.10.6) netty(version:4.1.60) . There are many vulnerabilities, like CVE-2021-21409 etc. please confirm these version and fix. th
[ https://issues.apache.org/jira/browse/FLINK-22441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-22441: --- Labels: auto-deprioritized-major auto-deprioritized-minor (was: auto-deprioritized-major stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > In Flink v1.11.3 contains netty(version:3.10.6) netty(version:4.1.60) . There > are many vulnerabilities, like CVE-2021-21409 etc. please confirm these > version and fix. thx > -- > > Key: FLINK-22441 > URL: https://issues.apache.org/jira/browse/FLINK-22441 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination >Affects Versions: 1.11.3, 1.12.2, 1.13.0 >Reporter: 张健 >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > > In Flink v1.11.3 contains netty(version:3.10.6) netty(version:4.1.60) . There > are many vulnerabilities, like CVE-2021-21409 CVE-2021-21295 etc. please > confirm these version and fix. thx -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-19380) Add support for a gRPC transport for the RequestReply protocol.
[ https://issues.apache.org/jira/browse/FLINK-19380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-19380: --- Labels: auto-deprioritized-major auto-deprioritized-minor (was: auto-deprioritized-major stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Add support for a gRPC transport for the RequestReply protocol. > --- > > Key: FLINK-19380 > URL: https://issues.apache.org/jira/browse/FLINK-19380 > Project: Flink > Issue Type: New Feature > Components: Stateful Functions >Reporter: Igal Shilman >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-21153) yarn-per-job deployment target ignores yarn options
[
https://issues.apache.org/jira/browse/FLINK-21153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-21153:
---
Labels: auto-deprioritized-major auto-unassigned stale-minor usability
(was: auto-deprioritized-major auto-unassigned usability)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> yarn-per-job deployment target ignores yarn options
> ---
>
> Key: FLINK-21153
> URL: https://issues.apache.org/jira/browse/FLINK-21153
> Project: Flink
> Issue Type: Bug
> Components: Command Line Client, Deployment / YARN
>Affects Versions: 1.12.1, 1.13.0
>Reporter: Till Rohrmann
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-minor,
> usability
>
> While looking into the problem reported in FLINK-6949, I stumbled across an
> odd behaviour of Flink. I tried to deploy a Flink cluster on Yarn and ship
> some files to the cluster. Only the first command successfully shipped the
> additional files to the cluster:
> 1) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -m yarn-cluster
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> 2) {{bin/flink run -p 1 --yarnship ../flink-test-job/cluster -t yarn-per-job
> ../flink-test-job/target/flink-test-job-1.0-SNAPSHOT.jar}}
> The problem seems to be that the second command does not activate the
> {{FlinkYarnSessionCli}} but uses the {{GenericCLI}}.
> [~kkl0u], [~aljoscha], [~tison] what is the intended behaviour in this case.
> I always thought that {{-m yarn-cluster}} and {{-t yarn-per-job}} would be
> equivalent.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-11425) Support of “Hash Teams” in Hybrid Hash Join
[ https://issues.apache.org/jira/browse/FLINK-11425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11425: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Support of “Hash Teams” in Hybrid Hash Join > --- > > Key: FLINK-11425 > URL: https://issues.apache.org/jira/browse/FLINK-11425 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Runtime >Reporter: Ji Liu >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > > Hybrid Hash Join is already supported in current version. The join starts > operating in memory and gradually starts spilling contents to disk, when the > memory is not sufficient. > > Current hash join only support two inputs, so when a job contains multiple > hash joins which have the same join keys, it will consume some unnecessary > resources (I/O, memory, etc) because some upstream output data may useless > for downstream hash join. > > According to the above observations, we want to provide a HashTeamManager to > implement multiway inputs hash join by combining several two way hash join > which have same join keys. HashTeamManager manage the relations of multiple > HashTables and improve efficiency in memory use and lower I/O operations by > joining multiple relations at one time. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-22004) Translate Flink Roadmap to Chinese.
[ https://issues.apache.org/jira/browse/FLINK-22004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-22004: --- Labels: auto-deprioritized-major auto-unassigned pull-request-available stale-minor (was: auto-deprioritized-major auto-unassigned pull-request-available) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Translate Flink Roadmap to Chinese. > --- > > Key: FLINK-22004 > URL: https://issues.apache.org/jira/browse/FLINK-22004 > Project: Flink > Issue Type: New Feature > Components: Documentation >Reporter: Yuan Mei >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, > pull-request-available, stale-minor > Attachments: Screen Shot 2021-04-11 at 10.24.02 PM.png > > > https://flink.apache.org/roadmap.html -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11407) Allow providing reason messages for TypeSerializerSchemaCompatibility.incompatible()
[ https://issues.apache.org/jira/browse/FLINK-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11407: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Allow providing reason messages for > TypeSerializerSchemaCompatibility.incompatible() > > > Key: FLINK-11407 > URL: https://issues.apache.org/jira/browse/FLINK-11407 > Project: Flink > Issue Type: Improvement > Components: API / Type Serialization System >Reporter: Tzu-Li (Gordon) Tai >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > > There are a few different scenarios where a new serializer can be determined > incompatible in a compatibility check. > Allowing the incompatible result to be accompanied by a message indicating > why the new serializer is incompatible would be beneficial, and allows the > state backends to throw more meaningful exceptions when they do encounter an > incompatible new serializer. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11401) Allow compression on ParquetBulkWriter
[ https://issues.apache.org/jira/browse/FLINK-11401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11401: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available (was: auto-deprioritized-major auto-unassigned pull-request-available stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Allow compression on ParquetBulkWriter > -- > > Key: FLINK-11401 > URL: https://issues.apache.org/jira/browse/FLINK-11401 > Project: Flink > Issue Type: Improvement > Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile) >Affects Versions: 1.7.1 >Reporter: Fokko Driesprong >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11421) Add compilation options to allow compiling generated code with JDK compiler
[ https://issues.apache.org/jira/browse/FLINK-11421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11421: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available (was: auto-deprioritized-major auto-unassigned pull-request-available stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Add compilation options to allow compiling generated code with JDK compiler > > > Key: FLINK-11421 > URL: https://issues.apache.org/jira/browse/FLINK-11421 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Runtime >Reporter: Liya Fan >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned, pull-request-available > Original Estimate: 240h > Time Spent: 40m > Remaining Estimate: 239h 20m > > Flink supports some operators (like Calc, Hash Agg, Hash Join, etc.) by code > generation. That is, Flink generates their source code dynamically, and then > compile it into Java Byte Code, which is load and executed at runtime. > > By default, Flink compiles the generated source code by Janino. This is fast, > as the compilation often finishes in hundreds of milliseconds. The generated > Java Byte Code, however, is of poor quality. To illustrate, we use Java > Compiler API (JCA) to compile the generated code. Experiments on TPC-H (1 TB) > queries show that the E2E time can be more than 10% shorter, when operators > are compiled by JCA, despite that it takes more time (a few seconds) to > compile with JCA. > > Therefore, we believe it is beneficial to compile generated code by JCA in > the following scenarios: 1) For batch jobs, the E2E time is relatively long, > so it is worth of spending more time compiling and generating high quality > Java Byte Code. 2) For repeated stream jobs, the generated code will be > compiled once and run many times. Therefore, it pays to spend more time > compiling for the first time, and enjoy the high byte code qualities for > later runs. > > According to the above observations, we want to provide a compilation option > (Janino, JCA, or dynamic) for Flink, so that the user can choose the one > suitable for their specific scenario and obtain better performance whenever > possible. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-6166) Broken quickstart for snapshots
[
https://issues.apache.org/jira/browse/FLINK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-6166:
--
Labels: auto-deprioritized-critical auto-deprioritized-major stale-minor
(was: auto-deprioritized-critical auto-deprioritized-major)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Broken quickstart for snapshots
> ---
>
> Key: FLINK-6166
> URL: https://issues.apache.org/jira/browse/FLINK-6166
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.3.0
>Reporter: Greg Hogan
>Priority: Minor
> Labels: auto-deprioritized-critical, auto-deprioritized-major,
> stale-minor
>
> I am unable to [run the
> quickstart|https://ci.apache.org/projects/flink/flink-docs-release-1.3/quickstart/java_api_quickstart.html]
> for 1.3-SNAPSHOT. This looks to be due to the
> [3.0.0|http://maven.apache.org/archetype/maven-archetype-plugin/generate-mojo.html]
> release of the Archetype plugin, which now uses {{archetype-catalog.xml}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-6116) Watermarks don't work when unioning with same DataStream
[
https://issues.apache.org/jira/browse/FLINK-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-6116:
--
Labels: auto-deprioritized-critical auto-deprioritized-major
pull-request-available stale-minor (was: auto-deprioritized-critical
auto-deprioritized-major pull-request-available)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Watermarks don't work when unioning with same DataStream
>
>
> Key: FLINK-6116
> URL: https://issues.apache.org/jira/browse/FLINK-6116
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Aljoscha Krettek
>Priority: Minor
> Labels: auto-deprioritized-critical, auto-deprioritized-major,
> pull-request-available, stale-minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In this example job we don't get any watermarks in the {{WatermarkObserver}}:
> {code}
> public class WatermarkTest {
> public static void main(String[] args) throws Exception {
> final StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
>
> env.setStreamTimeCharacteristic(TimeCharacteristic.IngestionTime);
> env.getConfig().setAutoWatermarkInterval(1000);
> env.setParallelism(1);
> DataStreamSource input = env.addSource(new
> SourceFunction() {
> @Override
> public void run(SourceContext ctx) throws
> Exception {
> while (true) {
> ctx.collect("hello!");
> Thread.sleep(800);
> }
> }
> @Override
> public void cancel() {
> }
> });
> input.union(input)
> .flatMap(new IdentityFlatMap())
> .transform("WatermarkOp",
> BasicTypeInfo.STRING_TYPE_INFO, new WatermarkObserver());
> env.execute();
> }
> public static class WatermarkObserver
> extends AbstractStreamOperator
> implements OneInputStreamOperator {
> @Override
> public void processElement(StreamRecord element) throws
> Exception {
> System.out.println("GOT ELEMENT: " + element);
> }
> @Override
> public void processWatermark(Watermark mark) throws Exception {
> super.processWatermark(mark);
> System.out.println("GOT WATERMARK: " + mark);
> }
> }
> private static class IdentityFlatMap
> extends RichFlatMapFunction {
> @Override
> public void flatMap(String value, Collector out) throws
> Exception {
> out.collect(value);
> }
> }
> }
> {code}
> When commenting out the `union` it works.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-6131) Add side inputs for DataStream API
[ https://issues.apache.org/jira/browse/FLINK-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-6131: -- Labels: auto-deprioritized-major auto-unassigned stale-minor (was: auto-deprioritized-major auto-unassigned) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Add side inputs for DataStream API > -- > > Key: FLINK-6131 > URL: https://issues.apache.org/jira/browse/FLINK-6131 > Project: Flink > Issue Type: Improvement > Components: API / DataStream >Reporter: Aljoscha Krettek >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > This is an umbrella issue for tracking the implementation of FLIP-17: > https://cwiki.apache.org/confluence/display/FLINK/FLIP-17+Side+Inputs+for+DataStream+API. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-24349) Support customized Calalogs via JDBC
[ https://issues.apache.org/jira/browse/FLINK-24349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-24349: --- Labels: pull-request-available stale-major (was: pull-request-available) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Major but is unassigned and neither itself nor its Sub-Tasks have been updated for 60 days. I have gone ahead and added a "stale-major" to the issue". If this ticket is a Major, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Support customized Calalogs via JDBC > > > Key: FLINK-24349 > URL: https://issues.apache.org/jira/browse/FLINK-24349 > Project: Flink > Issue Type: Improvement > Components: Connectors / JDBC >Affects Versions: 1.15.0 >Reporter: Bo Cui >Priority: Major > Labels: pull-request-available, stale-major > > Support customized catalogs in flink-connector-jdbc -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-6109) Add "consumer lag" report metric to FlinkKafkaConsumer
[
https://issues.apache.org/jira/browse/FLINK-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-6109:
--
Labels: auto-deprioritized-major auto-unassigned stale-minor (was:
auto-deprioritized-major auto-unassigned)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Add "consumer lag" report metric to FlinkKafkaConsumer
> --
>
> Key: FLINK-6109
> URL: https://issues.apache.org/jira/browse/FLINK-6109
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Common, Connectors / Kafka
>Reporter: Tzu-Li (Gordon) Tai
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-minor
>
> This is a feature discussed in this ML:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Telling-if-a-job-has-caught-up-with-Kafka-td12261.html.
> As discussed, we can expose two kinds of "consumer lag" metrics for this:
> - *current consumer lag per partition:* the current difference between the
> latest offset and the last collected record. This metric is calculated and
> updated at a configurable interval. This metric basically serves as an
> indicator of how the consumer is keeping up with the head of partitions. I
> propose to name this {{currentOffsetLag}}.
> - *Consumer lag of last checkpoint per partition:* the difference between
> the latest offset and the offset stored in the checkpoint. This metric is
> only updated when checkpoints are completed. It serves as an indicator of how
> much data may need to be replayed in case of a failure. I propose to name
> this {{lastCheckpointedOffsetLag}}.
> I don't think it is reasonable to define a metric of whether or not a
> consumer has "caught up" with the HEAD. That would imply a threshold for the
> offset difference. We should probably leave this "caught up" logic for the
> user to determine themselves when they query this metric.
> The granularity of the metric is per-FlinkKafkaConsumer, and independent of
> the consumer group.id used (the offset used to calculate consumer lag is the
> internal offset state of the FlinkKafkaConsumer, not the consumer group's
> committed offsets in Kafka).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-6081) Offset/Fetch support for SQL Streaming
[
https://issues.apache.org/jira/browse/FLINK-6081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-6081:
--
Labels: auto-deprioritized-major auto-unassigned stale-minor (was:
auto-deprioritized-major auto-unassigned)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Offset/Fetch support for SQL Streaming
> --
>
> Key: FLINK-6081
> URL: https://issues.apache.org/jira/browse/FLINK-6081
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
>Reporter: radu
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-minor
> Attachments: offset.png
>
>
> Time target: Proc Time
> The main scope of Offset/Fetch is for pagination support. In the context
> of streaming Offset and Fetch would make sense within the scope of
> certain window constructs as they refer to buffered data from the stream
> (with a main usage to restrict the output that is shown at a certain
> moment). Therefore they should be applied to the output of the types of
> windows supported by the ORDER BY clauses. Moreover, in accordance to
> the SQL best practices, they can only be used with an ORDER BY clause.
> SQL targeted query examples:
>
> Window defined based on group by clause
> Q1:
> {code}
> SELECT a ORDER BY b OFFSET n ROWS FROM stream1 GROUP BY HOP(proctime,
> INTERVAL '1' HOUR, INTERVAL '3' HOUR)
> {code}
> Window defined based on where clause time boundaries
> Q2:
> {code}
> SELECT a ORDER BY b OFFSET n WHERE procTime() BETWEEN current_timestamp -
> INTERVAL '1' HOUR AND current_timestamp FROM stream1
> {code}
> ~~Window defined as sliding windows (aggregates) ~~
> Q3:
> {code}
> SELECT SUM(a) OVER (ORDER BY proctime RANGE INTERVAL '1' HOUR PRECEDING b
> OFFSET n ROWS) FROM stream1
> {code}
> Comment: Supporting offset over sliding windows (within the window) does
> not make sense because the main scope of OFFSET/FETCH is for pagination
> support. Therefore this functionality example should only be supported in
> relation to the
> output of a query. Hence, Q3 will not be supported
> The general grammar (Calcite version) for OFFSET/FECTH with available
> parameters is shown below:
> {code}
> Select […]
> [ ORDER BY orderItem [, orderItem ]* ]
> [ OFFSET start { ROW | ROWS } ]
> [ FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } ]
> {code}
> Description
> ---
> Offset and Fetch are primary used for pagination support (i.e., restrict
> the output that is shown at some point). They were mainly designed to
> support web page display of the contents. Building on this scenario we
> can imagine a similar role for OFFSET and FETCH for streams that would
> display contents via a web page. In such a scenario the number of
> outputs to be displayed would be limited using such operators (probably
> for pagination and aesthetic reasons). However, as for any stream
> application there is a natural evolution in time, the operators output
> should evolve with the update rate of the application. The fact that
> there is an update rate and a collection of events related to a stream
> points to window constructs. Therefore the OFFSET/FETCH functionality
> would be related to the window mechanisms/boundaries defined by the
> query. Hence when the window construct would be re-triggered the output
> would be filtered again from the cardinality point of view based on the
> logic of the OFFSET/FETCH.
> Because of the primary reasons of supporting pagination (and controlling
> the number of outputs) we limit the usage of OFFSET/Fetch for window
> constructs that would be related to the output. Because of this
> supporting those on sliding window with query aggregates (e.g., Q3 query
> example) would not make sense. Additionally there is an implicit need
> for some ordering clause due to the fact that OFFSET and FETCH point to
> ordering positions. That is why these functions would be supported only
> if an ORDER BY clause is present.
> Functionality example
> -
> We exemplify the usage of OFFSET below using the following query. Event
> schema is in the form (a,b).
> {code}
> SELECT a ORDER BY b OFFSET 2 ROWS FROM stream1 GROUP BY GROUP BY
> CEIL(proctime TO HOUR)
> {code}
> ||Proctime|| IngestionTime(Event)|| Stream1|| Output||
> | |10:00:01| (a1, 7)| |
> | |10:05:00| (c1, 2)| |
> | |10:12:00| (
[jira] [Updated] (FLINK-11374) See more failover and can filter by time range
[ https://issues.apache.org/jira/browse/FLINK-11374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11374: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > See more failover and can filter by time range > -- > > Key: FLINK-11374 > URL: https://issues.apache.org/jira/browse/FLINK-11374 > Project: Flink > Issue Type: Improvement > Components: Runtime / REST, Runtime / Web Frontend >Reporter: lining >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > Attachments: image-2019-01-22-11-40-53-135.png, > image-2019-01-22-11-42-33-808.png > > > Now failover just show limit size task failover latest time(ps:According to > the current flink > [code|https://github.com/apache/flink/blob/34b5399f4effb679baabd8bca312cbf92ec34165/flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/job/JobExceptionsHandler.java#L54] > ). > If exceptions are more then 20, user could not see more. > So we could update the rest api for supporting user could query with size. > As it may many, webUi need to filter them by time. > Now just show current attempt exception, we need add prior attempt exception. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-22705) SQL Client end-to-end test (Old planner) Elasticsearch (v7.5.1) failed due to fail to download the tar
[
https://issues.apache.org/jira/browse/FLINK-22705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-22705:
---
Labels: auto-deprioritized-major stale-minor test-stability (was:
auto-deprioritized-major test-stability)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> SQL Client end-to-end test (Old planner) Elasticsearch (v7.5.1) failed due to
> fail to download the tar
> --
>
> Key: FLINK-22705
> URL: https://issues.apache.org/jira/browse/FLINK-22705
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ElasticSearch
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Priority: Minor
> Labels: auto-deprioritized-major, stale-minor, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=18100&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=ff888d9b-cd34-53cc-d90f-3e446d355529&l=18408
> {code:java}
> May 18 17:24:23 Preparing Elasticsearch (version=7)...
> May 18 17:24:23 Downloading Elasticsearch from
> https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-7.5.1-no-jdk-linux-x86_64.tar.gz
> ...
> --2021-05-18 17:24:23--
> https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-7.5.1-no-jdk-linux-x86_64.tar.gz
> Resolving artifacts.elastic.co (artifacts.elastic.co)... 34.120.127.130,
> 2600:1901:0:1d7::
> Connecting to artifacts.elastic.co
> (artifacts.elastic.co)|34.120.127.130|:443... failed: Connection timed out.
> Connecting to artifacts.elastic.co
> (artifacts.elastic.co)|2600:1901:0:1d7::|:443... failed: Network is
> unreachable.
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
> 0curl: (7) Failed to connect to localhost port 9200: Connection refused
> May 18 17:26:34 [FAIL] Test script contains errors.
> May 18 17:26:34 Checking for errors...
> May 18 17:26:34 No errors in log files.
> May 18 17:26:34 Checking for exceptions...
> May 18 17:26:34 No exceptions in log files.
> May 18 17:26:34 Checking for non-empty .out files...
> grep: /home/vsts/work/_temp/debug_files/flink-logs/*.out: No such file or
> directory
> May 18 17:26:34 No non-empty .out files.
> May 18 17:26:34
> May 18 17:26:34 [FAIL] 'SQL Client end-to-end test (Old planner)
> Elasticsearch (v7.5.1)' failed after 2 minutes and 36 seconds! Test exited
> with exit code 1
> May 18 17:26:34
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-11442) Upgrade OSS SDK Version
[ https://issues.apache.org/jira/browse/FLINK-11442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11442: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available (was: auto-deprioritized-major auto-unassigned pull-request-available stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Upgrade OSS SDK Version > --- > > Key: FLINK-11442 > URL: https://issues.apache.org/jira/browse/FLINK-11442 > Project: Flink > Issue Type: Improvement > Components: FileSystems >Affects Versions: 1.8.0 >Reporter: wujinhu >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned, pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Upgrade oss sdk version to exclude org.json dependency. > [INFO] +- com.aliyun.oss:aliyun-sdk-oss:jar:3.1.0:compile > [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.3:compile > [INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.6:compile > [INFO] | +- org.jdom:jdom:jar:1.1:compile > [INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile > [INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:compile > [INFO] | | | \- stax:stax-api:jar:1.0.1:compile > [INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile > [INFO] | | | \- javax.xml.bind:jaxb-api:jar:2.2.2:compile > [INFO] | | | +- javax.xml.stream:stax-api:jar:1.0-2:compile > [INFO] | | | \- javax.activation:activation:jar:1.1:compile > [INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.3:compile > [INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.8.3:compile > [INFO] | +- com.aliyun:aliyun-java-sdk-core:jar:3.4.0:compile > [INFO] | | \- org.json:json:jar:20170516:compile > [INFO] | +- com.aliyun:aliyun-java-sdk-ram:jar:3.0.0:compile > [INFO] | +- com.aliyun:aliyun-java-sdk-sts:jar:3.0.0:compile > [INFO] | \- com.aliyun:aliyun-java-sdk-ecs:jar:4.2.0:compile > > The license of org.json:json:jar:20170516:compile is JSON License, which > cannot be included. > [https://www.apache.org/legal/resolved.html#json] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25443) Embedded statefulfunction jakson /java8 date time issue
Ashok created FLINK-25443: - Summary: Embedded statefulfunction jakson /java8 date time issue Key: FLINK-25443 URL: https://issues.apache.org/jira/browse/FLINK-25443 Project: Flink Issue Type: Bug Components: Stateful Functions Affects Versions: shaded-14.0 Reporter: Ashok Hi I have the jackson dependency in the pom.xml.But getting following error . com.fasterxml.jackson.datatype jackson-datatype-jsr310 2.13.1 provided Caused by: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Java 8 date/time type `java.time.Duration` not supported by default: add Module "org.apache.flink.shaded.jackson2.com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling (through reference chain: org.apache.flink.statefun.flink.core.httpfn.DefaultHttpRequestReplyClientSpec["timeouts"]->org.apache.flink.statefun.flink.core.httpfn.DefaultHttpRequestReplyClientSpec$Timeouts["call"]) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1276) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.impl.UnsupportedTypeSerializer.serialize(UnsupportedTypeSerializer.java:35) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:770) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:770) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3126) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.valueToTree(ObjectMapper.java:3307) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11276) Sliding Window Optimization
[
https://issues.apache.org/jira/browse/FLINK-11276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-11276:
---
Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
(was: auto-deprioritized-major auto-unassigned stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Sliding Window Optimization
> ---
>
> Key: FLINK-11276
> URL: https://issues.apache.org/jira/browse/FLINK-11276
> Project: Flink
> Issue Type: New Feature
> Components: API / DataStream
>Affects Versions: 1.7.0
>Reporter: Rong Rong
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned
>
> This umbrella JIRA focus on the improvement of the existing window operator
> *WITHOUT* changing the public facing API. Please find the initial design plan
> in:
> [https://docs.google.com/document/d/1CvjPJl1Fm1PCpsuuZ4Qc-p_iUzUosBePX_rWNUt8lRw/edit?usp=sharing]
>
> and the execution plan discussion in:
>
> [http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Improvement-to-Flink-Window-Operator-with-Slicing-tt25750.html]
> Per the discussion in the dev mailing list. We would like to only focus on
> improvement in the following perspective of the {{WindowOperator:(}}
> 1. State optimization
> 2. Internal Window Function
> 3. Backward-compatibility
>
> Reference: Initial improvement discussion can be found in:
>
> [https://docs.google.com/document/d/1ziVsuW_HQnvJr_4a9yKwx_LEnhVkdlde2Z5l6sx5HlY/edit?usp=sharing]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-11258) Add badge to the README
[ https://issues.apache.org/jira/browse/FLINK-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11258: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available (was: auto-deprioritized-major auto-unassigned pull-request-available stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Add badge to the README > --- > > Key: FLINK-11258 > URL: https://issues.apache.org/jira/browse/FLINK-11258 > Project: Flink > Issue Type: Improvement > Components: Documentation >Reporter: Fokko Driesprong >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned, pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > I think we should add the badge to the docs to check if master is still happy. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11172) Remove the max retention time in StreamQueryConfig
[ https://issues.apache.org/jira/browse/FLINK-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11172: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Remove the max retention time in StreamQueryConfig > -- > > Key: FLINK-11172 > URL: https://issues.apache.org/jira/browse/FLINK-11172 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Affects Versions: 1.8.0 >Reporter: Yangze Guo >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > > [Stream Query > Config|https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/query_configuration.html] > is an important and useful feature to make a tradeoff between accuracy and > resource consumption when some query executed in unbounded streaming data. > This feature first proposed in > [FLINK-6491|https://issues.apache.org/jira/browse/FLINK-6491]. > At the first, *QueryConfig* take two parameters, i.e. > minIdleStateRetentionTime and maxIdleStateRetentionTime, to avoid to register > many timers if we have more freedom when to discard state. However, this > approach may cause new data expired earlier than old data and thus greater > accuracy loss appeared in some case. For example, we have an unbounded keyed > streaming data. We process key *_a_* in _*t0*_ and _*b*_ in _*t1,*_ *_t0 < > t1_*. *_a_* will expired in _*a+maxIdleStateRetentionTime*_ while _*b*_ > expired in *_b+maxIdleStateRetentionTime_*. Now, another data with key *_a_* > arrived in _*t2 (t1 < t2)*_. But _*t2+minIdleStateRetentionTime*_ < > _*a+maxIdleStateRetentionTime*_. The state of key *_a_* will still be expired > in _*a+maxIdleStateRetentionTime*_ which is early than the state of key > _*b*_. According to the guideline of > [LRU|https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU)] > that the element has been most heavily used in the past few instructions are > most likely to be used heavily in the next few instructions too. The state > with key _*a*_ should live longer than the state with key _*b*_. Current > approach against this idea. > I think we now have a good chance to remove the maxIdleStateRetentionTime > argument in *StreamQueryConfig.* Below are my reasons. > * [FLINK-9423|https://issues.apache.org/jira/browse/FLINK-9423] implement > efficient deletes for heap-based timer service. We can leverage the deletion > op to mitigate the abuse of timer registration. > * Current approach can cause new data expired earlier than old data and thus > greater accuracy loss appeared in some case. Users need to fine-tune these > two parameter to avoid this scenario. Directly following the idea of LRU > looks like a better solution. > So, I plan to remove maxIdleStateRetentionTime, update the expire time only > depends on _*minIdleStateRetentionTime.*_ > cc to [~sunjincheng121], [~fhueske] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11313) [checkpoint] Introduce LZ4 compression for keyed state in full checkpoints and savepoints
[ https://issues.apache.org/jira/browse/FLINK-11313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11313: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available (was: auto-deprioritized-major auto-unassigned pull-request-available stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > [checkpoint] Introduce LZ4 compression for keyed state in full checkpoints > and savepoints > - > > Key: FLINK-11313 > URL: https://issues.apache.org/jira/browse/FLINK-11313 > Project: Flink > Issue Type: Improvement > Components: Runtime / Checkpointing >Reporter: Yun Tang >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > In our production environment, we prefer to use LZ4 instead of Snappy as > compression for better performance, which is also [recommended by RocksDB > |https://github.com/facebook/rocksdb/wiki/Compression#configuration]. > I plan to introduce LZ4 except for now existing snappy compression for keyed > state in full checkpoint and savepoints. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11198) Access to MetricGroup in an AggregateFunction(Non Rich)
[ https://issues.apache.org/jira/browse/FLINK-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11198: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Access to MetricGroup in an AggregateFunction(Non Rich) > --- > > Key: FLINK-11198 > URL: https://issues.apache.org/jira/browse/FLINK-11198 > Project: Flink > Issue Type: New Feature > Components: API / DataStream >Affects Versions: 1.6.2 >Reporter: Chirag Dewan >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > > The only way to add custom metrics from UDF is through RuntimeContext. And, > RuntimeContext is wired in every RichFunction implementation. > However, for aggregate() in Windowed Stream, we cannot use the Rich version > of AggregateFunction. As I remotely understand, is done to avoid exposing the > state in the Aggregate UDF. > But can we have some minimal context which does not expose state but provide > metrics, classloader etc.in the UDF? -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-6027) Ignore the exception thrown by the subsuming of old completed checkpoints
[
https://issues.apache.org/jira/browse/FLINK-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-6027:
--
Labels: auto-deprioritized-major auto-unassigned stale-minor (was:
auto-deprioritized-major auto-unassigned)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Ignore the exception thrown by the subsuming of old completed checkpoints
> -
>
> Key: FLINK-6027
> URL: https://issues.apache.org/jira/browse/FLINK-6027
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Reporter: Xiaogang Shi
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-minor
>
> When a checkpoint is added into the {{CompletedCheckpointStore}} via the
> method {{addCheckpoint()}}, the oldest checkpoints will be removed from the
> store if the number of stored checkpoints exceeds the given limit. The
> subsuming of old checkpoints may fail and make {{addCheckpoint()}} throw
> exceptions which are caught by {{CheckpointCoordinator}}. Finally, the states
> in the new checkpoint will be deleted by {{CheckpointCoordinator}}. Because
> the new checkpoint is still in the store, we may recover the job from the new
> checkpoint. But the recovery will fail as the states of the checkpoint are
> all deleted.
> We should ignore the exceptions thrown by the subsuming of old checkpoints
> because we can always recover from the new checkpoint when successfully
> adding it into the store. The ignorance may produce some dirty data, but it's
> acceptable because they can be cleaned with the cleanup hook introduced in
> the near future.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5735) Non-overlapping sliding window is not deterministic
[
https://issues.apache.org/jira/browse/FLINK-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-5735:
--
Labels: auto-deprioritized-major auto-unassigned stale-minor (was:
auto-deprioritized-major auto-unassigned)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Non-overlapping sliding window is not deterministic
> ---
>
> Key: FLINK-5735
> URL: https://issues.apache.org/jira/browse/FLINK-5735
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Reporter: Timo Walther
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-minor
>
> I don't know if this is a problem of the Table API or the underlying API. We
> have to investigate this as part of the issue.
> The following code leads to different results from time to time. Sometimes
> the count of "Hello" is 1 sometimes 2.
> {code}
> val data = List(
> (1L, 1, "Hi"),
> (2L, 2, "Hallo"),
> (3L, 2, "Hello"),
> (6L, 3, "Hello"),
> (4L, 5, "Hello"),
> (16L, 4, "Hello world"),
> (8L, 3, "Hello world"))
> @Test
> def testEventTimeSlidingWindowNonOverlapping(): Unit = {
> val env = StreamExecutionEnvironment.getExecutionEnvironment
> env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
> val tEnv = TableEnvironment.getTableEnvironment(env)
> StreamITCase.testResults = mutable.MutableList()
> val stream = env
> .fromCollection(data)
> .assignTimestampsAndWatermarks(new TimestampWithEqualWatermark())
> val table = stream.toTable(tEnv, 'long, 'int, 'string)
> val windowedTable = table
> .window(Slide over 5.milli every 10.milli on 'rowtime as 'w)
> .groupBy('w, 'string)
> .select('string, 'int.count, 'w.start, 'w.end)
> val results = windowedTable.toDataStream[Row]
> results.addSink(new StreamITCase.StringSink)
> env.execute()
> val expected = Seq(
> "Hallo,1,1970-01-01 00:00:00.0,1970-01-01 00:00:00.005",
> "Hello,2,1970-01-01 00:00:00.0,1970-01-01 00:00:00.005",
> "Hi,1,1970-01-01 00:00:00.0,1970-01-01 00:00:00.005")
> assertEquals(expected.sorted, StreamITCase.testResults.sorted)
> }
> class TimestampWithEqualWatermark extends
> AssignerWithPunctuatedWatermarks[(Long, Int, String)] {
> override def checkAndGetNextWatermark(
> lastElement: (Long, Int, String),
> extractedTimestamp: Long)
> : Watermark = {
> new Watermark(extractedTimestamp)
> }
> override def extractTimestamp(
> element: (Long, Int, String),
> previousElementTimestamp: Long): Long = {
> element._1
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-11221) Support delimited identifiers in TableAPI
[
https://issues.apache.org/jira/browse/FLINK-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-11221:
---
Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
(was: auto-deprioritized-major auto-unassigned stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Support delimited identifiers in TableAPI
> -
>
> Key: FLINK-11221
> URL: https://issues.apache.org/jira/browse/FLINK-11221
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Reporter: Hequn Cheng
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned
>
> There are two kinds of identifiers in SQL: regular identifiers and delimited
> identifiers. Different from regular identifiers, delimited identifiers are
> identifiers enclosed in double backticks(``), double quotation marks (") or
> brackets ([ ]).
> Currently, delimited identifiers in double backticks have been supported in
> Flink SQL(thanks to Calcite). In Calcite, delimited identifiers may contain
> virtually any character, including spaces and other punctuation. With
> delimited identifiers, we can name some special identifiers such as `a-a`
> which may be an essential requirement.
> However, delimited identifiers are not supported in TableApi. It would be
> nice if we support it in TableApi.
> Any suggestions are welcomed! Thank you.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5178) allow BlobCache to use a distributed file system irrespective of the HA mode
[ https://issues.apache.org/jira/browse/FLINK-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5178: -- Labels: auto-deprioritized-major auto-unassigned stale-minor (was: auto-deprioritized-major auto-unassigned) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > allow BlobCache to use a distributed file system irrespective of the HA mode > > > Key: FLINK-5178 > URL: https://issues.apache.org/jira/browse/FLINK-5178 > Project: Flink > Issue Type: Improvement > Components: Runtime / Network >Reporter: Nico Kruber >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > After FLINK-5129, high availability (HA) mode adds the ability for the > BlobCache instances at the task managers to download blobs directly from the > distributed file system. It would be nice if this also worked in non-HA mode. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5284) Make output of bucketing sink compatible with other processing framework like mapreduce
[ https://issues.apache.org/jira/browse/FLINK-5284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5284: -- Labels: auto-deprioritized-major auto-unassigned stale-minor (was: auto-deprioritized-major auto-unassigned) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Make output of bucketing sink compatible with other processing framework like > mapreduce > --- > > Key: FLINK-5284 > URL: https://issues.apache.org/jira/browse/FLINK-5284 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Reporter: Wenlong Lyu >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > Currently bucketing sink cannot move the in-progress and pending files to > final output when the stream finished, and when recovering, the current > output file will contain some invalid content, which can only be identified > by the file-length meta file. These make the final output of the job > incompatible to other processing framework like mapreduce. There are two > things to do to solve the problem: > 1. add direct output option to bucketing sink, which writes output to the > final file, and delete/truncate the some file when fail over. direct output > will be quite useful specially for finite stream job, which can enable user > to migrate there batch job to streaming, taking advantage of features such as > checkpointing. > 2. add truncate by copy option to enable bucketing sink to resize output file > by copying content valid in current file instead of creating a length meta > file. truncate by copy will make some more extra IO operation, but can make > the output more clean. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11157) Web UI should show timestamps relative to server time zone
[ https://issues.apache.org/jira/browse/FLINK-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11157: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Web UI should show timestamps relative to server time zone > -- > > Key: FLINK-11157 > URL: https://issues.apache.org/jira/browse/FLINK-11157 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Affects Versions: 1.7.0 >Reporter: Nico Kruber >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > > Currently, it seems the web UI is fetching timestamps, e.g. for checkpoint, > as milliseconds from Epoch and simply converts them using the Browser's clock > and time zone. It should be based on the server's time zone instead. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-11286) Support to send StreamStatus.IDLE for non-source operators
[
https://issues.apache.org/jira/browse/FLINK-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-11286:
---
Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
(was: auto-deprioritized-major auto-unassigned stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Support to send StreamStatus.IDLE for non-source operators
> ---
>
> Key: FLINK-11286
> URL: https://issues.apache.org/jira/browse/FLINK-11286
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
>Reporter: vinoyang
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned
>
> Currently, only stream source tasks can be marked as temporary idle. But many
> times, this approach has limitations.
> Considering such a scenario, there is a DAG as follows:
> {{source->map->filter->flatmap->keyBy->window}}, with a degree of parallelism
> of 10. Among them, the watermark is not sent by the source operator, but is
> downstream, such as flatmap. Every source subtask will not be idle. However,
> after the filter, some pipelines generate "idle". For example, there are 3
> pipelines that will no longer have data sent downstream. At this time, we
> can't call the {{markAsTemporarilyIdle}} method to mark the current pipeline
> in the idle state. This will affect the downstream window.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-11184) Rework TableSource and TableSink interfaces
[
https://issues.apache.org/jira/browse/FLINK-11184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-11184:
---
Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
(was: auto-deprioritized-major auto-unassigned stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Rework TableSource and TableSink interfaces
> ---
>
> Key: FLINK-11184
> URL: https://issues.apache.org/jira/browse/FLINK-11184
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
>Reporter: Timo Walther
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> auto-unassigned
>
> There are a couple of shortcomings with the current {{TableSource}} and
> {{TableSink}} interface design. Some of the issues are covered in a [basic
> design
> document|https://docs.google.com/document/d/1Yaxp1UJUFW-peGLt8EIidwKIZEWrrA-pznWLuvaH39Y/edit#heading=h.41fd6rs7b3cf]
> that was published a while ago.
> The design document has not been updated for some time and partially overlaps
> with the [current SQL DDL
> discussion|http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-SQL-DDL-Design-td25006.html]
> for the {{CREATE TABLE}} statement on the ML.
> What needs to be solved:
> - How to unify sources and sinks in regards of schema and time attributes?
> - How to define watermarks, timestamp extractors or timestamp ingestion?
> - How to define primary keys and partitioning keys?
> - How to differentiate between update modes for tables (i.e. how to read from
> a append, retraction, or update table)?
> - How to express all of the above without pulling in to many dependencies on
> other Flink modules if source and sink interfaces are located in
> {{flink-table-spi}} package?
> As of the current state of the discussion, it seems that we might extend
> {{TableSchema}} to allow for returning the information above and remove
> current interfaces such as {{DefinedRowtimeAttribute}} or
> {{DefinedFieldMapping}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5319) ClassCastException when reusing an inherited method reference as KeySelector for different classes
[
https://issues.apache.org/jira/browse/FLINK-5319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-5319:
--
Labels: auto-deprioritized-major auto-unassigned stale-minor (was:
auto-deprioritized-major auto-unassigned)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> ClassCastException when reusing an inherited method reference as KeySelector
> for different classes
> --
>
> Key: FLINK-5319
> URL: https://issues.apache.org/jira/browse/FLINK-5319
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.2.0
>Reporter: Alexander Chermenin
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-minor
>
> Code sample:
> {code}static abstract class A {
> int id;
> A(int id) {this.id = id; }
> int getId() { return id; }
> }
> static class B extends A { B(int id) { super(id % 3); } }
> static class C extends A { C(int id) { super(id % 2); } }
> private static B b(int id) { return new B(id); }
> private static C c(int id) { return new C(id); }
> /**
> * Main method.
> */
> public static void main(String[] args) throws Exception {
> StreamExecutionEnvironment environment =
> StreamExecutionEnvironment.getExecutionEnvironment();
> B[] bs = IntStream.range(0, 10).mapToObj(Test::b).toArray(B[]::new);
> C[] cs = IntStream.range(0, 10).mapToObj(Test::c).toArray(C[]::new);
> DataStreamSource bStream = environment.fromElements(bs);
> DataStreamSource cStream = environment.fromElements(cs);
> bStream.keyBy((KeySelector) A::getId).print();
> cStream.keyBy((KeySelector) A::getId).print();
> environment.execute();
> }
> {code}
> This code throws next exception:
> {code}Exception in thread "main"
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> at
> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$7.apply$mcV$sp(JobManager.scala:901)
> at
> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$7.apply(JobManager.scala:844)
> at
> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$7.apply(JobManager.scala:844)
> at
> scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
> at
> scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
> at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
> at
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:401)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:1253)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1346)
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> Caused by: java.lang.RuntimeException: Could not extract key from
> org.sample.flink.examples.Test$C@5e1a8111
> at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:75)
> at
> org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:39)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:746)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:724)
> at
> org.apache.flink.streaming.api.operators.StreamSourceContexts$NonTimestampContext.collect(StreamSourceContexts.java:84)
> at
> org.apache.flink.streaming.api.functions.source.FromElementsFunction.run(FromElementsFunction.java:127)
> at
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:75)
> at
> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56)
> at
> org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:269)
> at org.apach
[jira] [Updated] (FLINK-11209) Provide more complete guidance for the log usage documentation
[ https://issues.apache.org/jira/browse/FLINK-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-11209: --- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Provide more complete guidance for the log usage documentation > -- > > Key: FLINK-11209 > URL: https://issues.apache.org/jira/browse/FLINK-11209 > Project: Flink > Issue Type: Improvement > Components: Documentation >Reporter: vinoyang >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > > The current documentation does not provide detailed guidance when users > attempt to use logback as the underlying logging framework. Since Flink only > contains log4j dependency jars. The documentation says that if you want to > switch to logback, you only need to exclude the log4j dependency, but it does > not remind the user to add a dependency on the logback. I hope to add the > following information: > * Remove the dependencies of log4j and slf4j-log4j in the lib folder; > * Where to download the logback-* jar > * Introduce logback-core/logback-classic/logback-access dependencies -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5325) Introduce interface for CloseableRegistry to separate user from system-facing functionality
[
https://issues.apache.org/jira/browse/FLINK-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-5325:
--
Labels: auto-deprioritized-major auto-unassigned stale-minor (was:
auto-deprioritized-major auto-unassigned)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Introduce interface for CloseableRegistry to separate user from system-facing
> functionality
> ---
>
> Key: FLINK-5325
> URL: https://issues.apache.org/jira/browse/FLINK-5325
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Reporter: Stefan Richter
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-minor
>
> Currently, the API of {{CloseableRegistry}} exposes the {{close}} method to
> all client code. We should separate the API into a user-facing interface
> (allowing only for un/registration of {{Closeable}} and a system-facing part
> that also exposes the {{close}} method. This prevents users from accidentally
> calling {{close}}, thus closing resources that other callers registered.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5740) Make WrappingFunction an interface and move to flink-core
[
https://issues.apache.org/jira/browse/FLINK-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-5740:
--
Labels: auto-deprioritized-major auto-unassigned stale-minor (was:
auto-deprioritized-major auto-unassigned)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Make WrappingFunction an interface and move to flink-core
> -
>
> Key: FLINK-5740
> URL: https://issues.apache.org/jira/browse/FLINK-5740
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream
>Reporter: Aljoscha Krettek
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, stale-minor
>
> Making it an interface and having an {{AbstractWrappingFunction}} will allow
> implementations to have different classes as base classes. Also, we should
> change {{FunctionUtils}} to work like {{StreamingFunctionUtils}} so that
> wrapping functions don't have to implement the methods of {{RichFunction}}.
> These tests are disabled for now because we cannot properly implement the
> required features:
> - WindowFoldITCase.testFoldWithProcessWindowFunction() (Scala)
> - WindowFunctionITCase.testRichProcessWindowFunction() (Scala)
> - WindowReduceITCase.testReduceWithProcessWindowFunction (Scala)
> Because of the ignored tests I'm updating to "blocker".
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5944) Flink should support reading Snappy Files
[ https://issues.apache.org/jira/browse/FLINK-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5944: -- Labels: auto-deprioritized-major auto-unassigned features stale-minor (was: auto-deprioritized-major auto-unassigned features) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > Flink should support reading Snappy Files > - > > Key: FLINK-5944 > URL: https://issues.apache.org/jira/browse/FLINK-5944 > Project: Flink > Issue Type: New Feature > Components: Connectors / Hadoop Compatibility >Reporter: Ilya Ganelin >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, features, > stale-minor > > Snappy is an extremely performant compression format that's widely used > offering fast decompression/compression. > This can be easily implemented by creating a SnappyInflaterInputStreamFactory > and updating the initDefaultInflateInputStreamFactories in FileInputFormat. > Flink already includes the Snappy dependency in the project. > There is a minor gotcha in this. If we wish to use this with Hadoop, then we > must provide two separate implementations since Hadoop uses a different > version of the snappy format than Snappy Java (which is the xerial/snappy > included in Flink). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5230) Safety nets against leaving dysfunctional JobManagers
[
https://issues.apache.org/jira/browse/FLINK-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-5230:
--
Labels: auto-deprioritized-major stale-minor (was:
auto-deprioritized-major)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Safety nets against leaving dysfunctional JobManagers
> -
>
> Key: FLINK-5230
> URL: https://issues.apache.org/jira/browse/FLINK-5230
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Reporter: Stephan Ewen
>Priority: Minor
> Labels: auto-deprioritized-major, stale-minor
>
> There are certain ways that a {{JobManager}} can become dysfunctional.
> If the JobManager process continues to exist (not restarted by YARN / Mesos)
> etc, but is not doing its work properly and more, it makes the Streaming Job
> unavailable.
> There some safety nets to bring into place for that, see sub issues.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-5914) remove aggregateResultType from streaming.api.datastream.aggregate
[ https://issues.apache.org/jira/browse/FLINK-5914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-5914: -- Labels: auto-deprioritized-major auto-unassigned stale-minor (was: auto-deprioritized-major auto-unassigned) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issues has been marked as Minor but is unassigned and neither itself nor its Sub-Tasks have been updated for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is still Minor, please either assign yourself or give an update. Afterwards, please remove the label or in 7 days the issue will be deprioritized. > remove aggregateResultType from streaming.api.datastream.aggregate > -- > > Key: FLINK-5914 > URL: https://issues.apache.org/jira/browse/FLINK-5914 > Project: Flink > Issue Type: Improvement > Components: API / DataStream >Reporter: Shaoxuan Wang >Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned, stale-minor > > aggregateResultType does not seem necessary for > streaming.api.datastream.aggregate. We will anyway not serialize the > aggregateResult between aggregate and window function. Aggregate function > itself provides a function to getResult(), window function here should just > emit the same results as aggregate output. So aggregateResultType should be > same as resultType. I think we can safely remove aggregateResultType, thereby > user will not have to provide two same types for the > streaming.api.datastream.aggregate. > [~StephanEwen], what do you think? -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-5865) Throw original exception in states
[
https://issues.apache.org/jira/browse/FLINK-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-5865:
--
Labels: auto-deprioritized-major auto-unassigned pull-request-available
stale-minor (was: auto-deprioritized-major auto-unassigned
pull-request-available)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Throw original exception in states
> --
>
> Key: FLINK-5865
> URL: https://issues.apache.org/jira/browse/FLINK-5865
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Affects Versions: 1.3.0
>Reporter: Xiaogang Shi
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned,
> pull-request-available, stale-minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Now all exception thrown in RocksDB states are converted to
> {{RuntimeException}}. It's unnecessary and will print useless stacks in the
> log.
> I think it's better to throw the original exception, without any wrapping.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25423) Enable loading state backend via configuration in state processor api
[
https://issues.apache.org/jira/browse/FLINK-25423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465294#comment-17465294
]
Yuan Mei commented on FLINK-25423:
--
Thanks Seth!
> Enable loading state backend via configuration in state processor api
> -
>
> Key: FLINK-25423
> URL: https://issues.apache.org/jira/browse/FLINK-25423
> Project: Flink
> Issue Type: Improvement
> Components: API / State Processor, Runtime / State Backends
>Reporter: Yun Tang
>Assignee: Seth Wiesman
>Priority: Major
> Fix For: 1.15.0
>
>
> Currently, state processor API would load savepoint via explictly
> initalizated state backend on client side, which is like
> {{StreamExecutionEnvironment#setStateBackend(stateBackend)}}:
> {code:java}
> Savepoint.load(bEnv, "hdfs://path/", new HashMapStateBackend());
> {code}
> As we all konw, stream env also support to load state backend via
> configuration to provide flexibility to load state backends especially some
> customized state backend. This could also benefit state processor API with
> similiar ability.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] knaufk merged pull request #18168: [FLINK-25375] [BP-1.13] Update log4j2 dependency to 2.17.0 to address
knaufk merged pull request #18168: URL: https://github.com/apache/flink/pull/18168 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] knaufk merged pull request #18166: [FLINK-25375] [BP-1.12] Update log4j2 dependency to 2.17.0 to address
knaufk merged pull request #18166: URL: https://github.com/apache/flink/pull/18166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] knaufk merged pull request #18167: [FLINK-25375] [BP-1.14] Update log4j2 dependency to 2.17.0 to address
knaufk merged pull request #18167: URL: https://github.com/apache/flink/pull/18167 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (FLINK-25375) Update Log4j to 2.17.0
[ https://issues.apache.org/jira/browse/FLINK-25375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Knauf resolved FLINK-25375. -- Resolution: Fixed > Update Log4j to 2.17.0 > -- > > Key: FLINK-25375 > URL: https://issues.apache.org/jira/browse/FLINK-25375 > Project: Flink > Issue Type: Bug >Affects Versions: 1.11.6, 1.12.7, 1.13.5, 1.14.2 >Reporter: Sergey Nuyanzin >Assignee: Konstantin Knauf >Priority: Critical > Labels: pull-request-available > Fix For: 1.15.0, 1.12.8, 1.13.6, 1.14.3 > > > Log4j 2.17.0 has been released [1] > This release contains the changes noted below: > Address CVE-2021-45105. > Require components that use JNDI to be enabled individually via system > properties. > Remove LDAP and LDAPS as supported protocols from JNDI. > [1] > https://github.com/apache/logging-log4j2/blob/6b1581901ba7a107cdc4a2208ecec03655722b44/RELEASE-NOTES.md#apache-log4j-2170-release-notes -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25375) Update Log4j to 2.17.0
[ https://issues.apache.org/jira/browse/FLINK-25375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465307#comment-17465307 ] Konstantin Knauf commented on FLINK-25375: -- release-1.14: 49971b8a527b81b5e00169d5b187e21a274184e2 release-1.13: d903739d6cf95bcac4be46eedafe016d6dd02446 release-1.12: 1561f0f908a96b8efa83e79d44cfe579cb7d29ef > Update Log4j to 2.17.0 > -- > > Key: FLINK-25375 > URL: https://issues.apache.org/jira/browse/FLINK-25375 > Project: Flink > Issue Type: Bug >Affects Versions: 1.11.6, 1.12.7, 1.13.5, 1.14.2 >Reporter: Sergey Nuyanzin >Assignee: Konstantin Knauf >Priority: Critical > Labels: pull-request-available > Fix For: 1.15.0, 1.12.8, 1.13.6, 1.14.3 > > > Log4j 2.17.0 has been released [1] > This release contains the changes noted below: > Address CVE-2021-45105. > Require components that use JNDI to be enabled individually via system > properties. > Remove LDAP and LDAPS as supported protocols from JNDI. > [1] > https://github.com/apache/logging-log4j2/blob/6b1581901ba7a107cdc4a2208ecec03655722b44/RELEASE-NOTES.md#apache-log4j-2170-release-notes -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (FLINK-25375) Update Log4j to 2.17.0
[ https://issues.apache.org/jira/browse/FLINK-25375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Knauf closed FLINK-25375. > Update Log4j to 2.17.0 > -- > > Key: FLINK-25375 > URL: https://issues.apache.org/jira/browse/FLINK-25375 > Project: Flink > Issue Type: Bug >Affects Versions: 1.11.6, 1.12.7, 1.13.5, 1.14.2 >Reporter: Sergey Nuyanzin >Assignee: Konstantin Knauf >Priority: Critical > Labels: pull-request-available > Fix For: 1.15.0, 1.12.8, 1.13.6, 1.14.3 > > > Log4j 2.17.0 has been released [1] > This release contains the changes noted below: > Address CVE-2021-45105. > Require components that use JNDI to be enabled individually via system > properties. > Remove LDAP and LDAPS as supported protocols from JNDI. > [1] > https://github.com/apache/logging-log4j2/blob/6b1581901ba7a107cdc4a2208ecec03655722b44/RELEASE-NOTES.md#apache-log4j-2170-release-notes -- This message was sent by Atlassian Jira (v8.20.1#820001)
