Andrey Zagrebin created FLINK-20469:
---
Summary: Enable TaskManager start and terminate in MiniCluster
Key: FLINK-20469
URL: https://issues.apache.org/jira/browse/FLINK-20469
Project: Flink
Andrey Zagrebin created FLINK-20468:
---
Summary: Enable leadership control in MiniCluster to test JM
failover
Key: FLINK-20468
URL: https://issues.apache.org/jira/browse/FLINK-20468
Project: Flink
Andrey Zagrebin created FLINK-20290:
---
Summary: Duplicated output in FileSource continuous ITCase with TM
failover
Key: FLINK-20290
URL: https://issues.apache.org/jira/browse/FLINK-20290
Project
Andrey Zagrebin created FLINK-20261:
---
Summary: Uncaught exception in ExecutorNotifier due to split
assignment broken by failed task
Key: FLINK-20261
URL: https://issues.apache.org/jira/browse/FLINK-20261
Andrey Zagrebin created FLINK-20171:
---
Summary: Improve error message for Flink process memory
configuration
Key: FLINK-20171
URL: https://issues.apache.org/jira/browse/FLINK-20171
Project: Flink
Andrey Zagrebin created FLINK-20078:
---
Summary: Factor out an ExecutionGraph factory method for
DefaultExecutionTopology
Key: FLINK-20078
URL: https://issues.apache.org/jira/browse/FLINK-20078
Andrey Zagrebin created FLINK-19954:
---
Summary: Move execution deployment tracking logic from legacy EG
code to SchedulerNG
Key: FLINK-19954
URL: https://issues.apache.org/jira/browse/FLINK-19954
Andrey Zagrebin created FLINK-19923:
---
Summary: Remove BulkSlotProvider, its implementation and tests
Key: FLINK-19923
URL: https://issues.apache.org/jira/browse/FLINK-19923
Project: Flink
Andrey Zagrebin created FLINK-19918:
---
Summary: RocksIncrementalCheckpointRescalingTest.testScalingDown
fails on Windows
Key: FLINK-19918
URL: https://issues.apache.org/jira/browse/FLINK-19918
Andrey Zagrebin created FLINK-19917:
---
Summary: RocksDBInitTest.testTempLibFolderDeletedOnFail fails on
Windows
Key: FLINK-19917
URL: https://issues.apache.org/jira/browse/FLINK-19917
Project
Andrey Zagrebin created FLINK-19860:
---
Summary: Consider skipping restart and traverse regions which are
already being restarted in RestartPipelinedRegionFailoverStrategy
Key: FLINK-19860
URL: https
Andrey Zagrebin created FLINK-19832:
---
Summary: Improve handling of immediately failed physical slot in
SlotSharingExecutionSlotAllocator
Key: FLINK-19832
URL: https://issues.apache.org/jira/browse/FLINK-19832
Thanks a lot for this nice UI guys!
+1 and for closed issues that is just because many steps have been
already done.
Best,
Andrey
On Fri, Oct 23, 2020 at 11:12 AM Till Rohrmann wrote:
> Thanks for reviving this Flip Yadong! The changes look good to me and the
> new memory UI looks awesome :-)
For the option name, maybe:
*flink.main*
or
*flink.managed* (this may be a bit confusing for existing users as we said
that the overall managed memory is managed by Flink)
On Wed, Sep 9, 2020 at 9:56 AM Andrey Zagrebin wrote:
> +1
>
> Best,
> Andrey
>
> On Tue, Sep 8, 202
+1
Best,
Andrey
On Tue, Sep 8, 2020 at 2:16 PM Yu Li wrote:
> +1
>
> Best Regards,
> Yu
>
>
> On Tue, 8 Sep 2020 at 17:03, Aljoscha Krettek wrote:
>
> > +1
> >
> > We just need to make sure to find a good name before the release but
> > shouldn't block any work on this.
> >
> > Aljoscha
> >
>
Andrey Zagrebin created FLINK-19142:
---
Summary: Investigate slot hijacking from preceding pipelined
regions after failover
Key: FLINK-19142
URL: https://issues.apache.org/jira/browse/FLINK-19142
Hi Sidhant,
(1) If we are not using Flink's HA services then how we can dynamically
> configure task manager nodes to connect to job manager? Any suggestions or
> best practices?
Not sure what you mean by 'dynamically'.
I think you have to restart the task manager with the new configuration
to
Hi All,
Thanks for reviving the discussion, Matthias!
This would mean that we could adapt the current proposal to replace the
> Nonheap usage pane by a pane displaying the Metaspace usage.
>
I do not know the value of having the Nonheap usage in metrics. I can see
that the metaspace metric can
Andrey Zagrebin created FLINK-18957:
---
Summary: Implement bulk fulfil-ability timeout tracking for shared
slots
Key: FLINK-18957
URL: https://issues.apache.org/jira/browse/FLINK-18957
Project: Flink
+1 for the consolidation
Best,
Andrey
On Fri, Aug 7, 2020 at 3:38 PM Till Rohrmann wrote:
> +1 for unifying Deployment / Docker, Dockerfiles and Release System /
> Docker into Docker.
>
> Cheers,
> Till
>
> On Fri, Aug 7, 2020 at 12:18 PM Robert Metzger
> wrote:
>
> > Hi all,
> >
> > we now
Andrey Zagrebin created FLINK-18751:
---
Summary: Implement SlotSharingExecutionSlotAllocator
Key: FLINK-18751
URL: https://issues.apache.org/jira/browse/FLINK-18751
Project: Flink
Issue Type
Andrey Zagrebin created FLINK-18739:
---
Summary: Implement MergingSharedSlotProfileRetriever
Key: FLINK-18739
URL: https://issues.apache.org/jira/browse/FLINK-18739
Project: Flink
Issue Type
Andrey Zagrebin created FLINK-18709:
---
Summary: Implement PhysicalSlotProvider
Key: FLINK-18709
URL: https://issues.apache.org/jira/browse/FLINK-18709
Project: Flink
Issue Type: Sub-task
Andrey Zagrebin created FLINK-18689:
---
Summary: Deterministic Slot Sharing
Key: FLINK-18689
URL: https://issues.apache.org/jira/browse/FLINK-18689
Project: Flink
Issue Type: New Feature
Andrey Zagrebin created FLINK-18690:
---
Summary: Implement LocalInputPreferredSlotSharingStrategy
Key: FLINK-18690
URL: https://issues.apache.org/jira/browse/FLINK-18690
Project: Flink
Issue
Hi Flink Devs,
I would like to remind you that we have a 'starter' label [1] to annotate
Jira issues which need a contribution and which are not very
complicated for the new contributors. The starter issues can be a good
opportunity for the new contributors who want to learn about Flink but do
Andrey Zagrebin created FLINK-18646:
---
Summary: Managed memory released check can block RPC thread
Key: FLINK-18646
URL: https://issues.apache.org/jira/browse/FLINK-18646
Project: Flink
Andrey Zagrebin created FLINK-18467:
---
Summary: Document what can be reconfigured for state with TTL
between job restarts
Key: FLINK-18467
URL: https://issues.apache.org/jira/browse/FLINK-18467
Andrey Zagrebin created FLINK-18454:
---
Summary: Add a code contribution section about how to look for
what to contribute
Key: FLINK-18454
URL: https://issues.apache.org/jira/browse/FLINK-18454
Andrey Zagrebin created FLINK-18309:
---
Summary: Recommend avoiding uppercase to emphasise statements in
doc style
Key: FLINK-18309
URL: https://issues.apache.org/jira/browse/FLINK-18309
Project
Andrey Zagrebin created FLINK-18308:
---
Summary: KafkaProducerTestBase->Kafka011ProducerExactlyOnceITCase.
testExactlyOnceCustomOperator hangs in Azure
Key: FLINK-18308
URL: https://issues.apache.org/jira/bro
Andrey Zagrebin created FLINK-18250:
---
Summary: Enrich OOM error messages with more details in
ClusterEntrypoint
Key: FLINK-18250
URL: https://issues.apache.org/jira/browse/FLINK-18250
Project
Welcome to committers and congrats, Xintong!
Cheers,
Andrey
On Fri, Jun 5, 2020 at 4:22 PM Till Rohrmann wrote:
> Congratulations!
>
> Cheers,
> Till
>
> On Fri, Jun 5, 2020 at 10:00 AM Dawid Wysakowicz
> wrote:
>
> > Congratulations!
> >
> > Best,
> >
> > Dawid
> >
> > On 05/06/2020 09:10,
Andrey Zagrebin created FLINK-17811:
---
Summary: Update docker hub Flink page
Key: FLINK-17811
URL: https://issues.apache.org/jira/browse/FLINK-17811
Project: Flink
Issue Type: Task
Andrey Zagrebin created FLINK-17740:
---
Summary: Remove flink-container/kubernetes
Key: FLINK-17740
URL: https://issues.apache.org/jira/browse/FLINK-17740
Project: Flink
Issue Type: Sub-task
Andrey Zagrebin created FLINK-17652:
---
Summary: Legacy JM heap options should fallback to new
JVM_HEAP_MEMORY in standalone
Key: FLINK-17652
URL: https://issues.apache.org/jira/browse/FLINK-17652
Andrey Zagrebin created FLINK-17546:
---
Summary: Consider setting the number of TM CPU cores to the actual
number of cores
Key: FLINK-17546
URL: https://issues.apache.org/jira/browse/FLINK-17546
Hi lsyldliu,
You can try to tune the StateTtlConfig. As the documentation suggests [1]
the TTL incremental cleanup can decrease the per record performance. This
is the price of the automatic cleanup.
If the only thing, which happens mostly in your operator, is working with
state then even
Andrey Zagrebin created FLINK-17465:
---
Summary: Update Chinese user documentation for job manager memory
model
Key: FLINK-17465
URL: https://issues.apache.org/jira/browse/FLINK-17465
Project: Flink
Andrey Zagrebin created FLINK-17344:
---
Summary: RecordWriterTest.testIdleTime possibly deadlocks on Travis
Key: FLINK-17344
URL: https://issues.apache.org/jira/browse/FLINK-17344
Project: Flink
Andrey Zagrebin created FLINK-17167:
---
Summary: Extend entry point script and docs with history server
mode
Key: FLINK-17167
URL: https://issues.apache.org/jira/browse/FLINK-17167
Project: Flink
Andrey Zagrebin created FLINK-17166:
---
Summary: Modify the log4j-console.properties to also output logs
into the files for WebUI
Key: FLINK-17166
URL: https://issues.apache.org/jira/browse/FLINK-17166
Andrey Zagrebin created FLINK-17165:
---
Summary: Remove flink-container/docker
Key: FLINK-17165
URL: https://issues.apache.org/jira/browse/FLINK-17165
Project: Flink
Issue Type: Sub-task
Andrey Zagrebin created FLINK-17164:
---
Summary: Extend entry point script and docs with job cluster mode
and user job artefacts
Key: FLINK-17164
URL: https://issues.apache.org/jira/browse/FLINK-17164
Andrey Zagrebin created FLINK-17163:
---
Summary: Remove flink-contrib/docker-flink
Key: FLINK-17163
URL: https://issues.apache.org/jira/browse/FLINK-17163
Project: Flink
Issue Type: Sub-task
Andrey Zagrebin created FLINK-17162:
---
Summary: Document examples of how to extend the official docker
hub image
Key: FLINK-17162
URL: https://issues.apache.org/jira/browse/FLINK-17162
Project
Andrey Zagrebin created FLINK-17161:
---
Summary: Document the official docker hub image and examples of
how to run
Key: FLINK-17161
URL: https://issues.apache.org/jira/browse/FLINK-17161
Project
Andrey Zagrebin created FLINK-17160:
---
Summary: FLIP-111: Docker image unification
Key: FLINK-17160
URL: https://issues.apache.org/jira/browse/FLINK-17160
Project: Flink
Issue Type
周二 下午11:04写道：
> > > >
> > > > > Thanks for driving this effort Andrey.
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > Cheers,
> > > > > Till
> > > > >
> > > > > On Tue,
Andrey Zagrebin created FLINK-17048:
---
Summary: Add memory related JVM args to Mesos JM startup scripts
Key: FLINK-17048
URL: https://issues.apache.org/jira/browse/FLINK-17048
Project: Flink
Hi All,
As discussed in these threads [1] and [2],
we suggest to unify the docker topic in Flink for users [3].
This mainly means refactoring of the existing code and introducing more
docs as a first step.
The effort should enable further improvements and follow-ups for the docker
integration
> > Additional note:
>> >
>> >1. Job/Task attempt logging instead of task manager logging.
>> >*I realize this has nothing to do with the docker images*
>> > I found something "hard to work with" while running some tests last
>> > week.
>
Hi guys,
Thanks for more details Zhijiang.
It also looks to me that mapped memory size is mostly driven by OS limits
and bit-ness of JVM (32/64).
Thinking more about the 'Metrics' tab layout, couple of more things have
come into my mind.
# 'Metrics' tab -> 'Memory': 'Metrics' and
Hi everyone,
Patrick and Ufuk, thanks a lot for more ideas and suggestions!
I have updated the FLIP according to the current state of discussion.
Now it also contains the implementation steps and future follow-ups.
Please, review if there are any concerns.
The order of the steps aims for keeping
Andrey Zagrebin created FLINK-16946:
---
Summary: Update user documentation for job manager memory model
Key: FLINK-16946
URL: https://issues.apache.org/jira/browse/FLINK-16946
Project: Flink
:16 PM Gary Yao wrote:
> +1 (binding)
>
> Best,
> Gary
>
> On Wed, Mar 18, 2020 at 3:16 PM Andrey Zagrebin
> wrote:
>
> > Hi All,
> >
> > The discussion for FLIP-116 looks to be resolved [1].
> > Therefore, I start the vote for it.
> > The vote
Andrey Zagrebin created FLINK-16754:
---
Summary: Consider refactoring of ProcessMemoryUtilsTestBase to
avoid inheritance
Key: FLINK-16754
URL: https://issues.apache.org/jira/browse/FLINK-16754
Andrey Zagrebin created FLINK-16742:
---
Summary: Extend and use BashJavaUtils to start JM JVM process and
pass JVM memory args
Key: FLINK-16742
URL: https://issues.apache.org/jira/browse/FLINK-16742
Andrey Zagrebin created FLINK-16746:
---
Summary: Deprecate/remove legacy memory options for JM and expose
the new ones
Key: FLINK-16746
URL: https://issues.apache.org/jira/browse/FLINK-16746
Project
Andrey Zagrebin created FLINK-16745:
---
Summary: Use JobManagerProcessUtils to start JM container and pass
JVM memory args
Key: FLINK-16745
URL: https://issues.apache.org/jira/browse/FLINK-16745
eb_ui_port 8081). It
> should be fine enough to do it via flink_docker-utils conifgure rest.port
> 8081 if we cannot solve it via the general configuration mechanism.
>
> Cheers,
> Till
>
> On Wed, Mar 18, 2020 at 6:38 AM Yangze Guo wrote:
>
>> I second
Andrey Zagrebin created FLINK-16686:
---
Summary: [State TTL] Make user class loader available in native
RocksDB compaction thread
Key: FLINK-16686
URL: https://issues.apache.org/jira/browse/FLINK-16686
is not fulfilled. As already
>> mentioned, it would be hard to determine the min heap size. And if we make
>> the min heap configurable, then in any case that users need to configure
>> the min heap, they can configure the heap size directly.
>>
>> Thank you~
>>
&g
if it
is not fulfilled.
We could experiment with what is the working minimum but It is hard to come
up with this limit because it again can depend on the job and environment.
Best,
Andrey
On Wed, Mar 18, 2020 at 5:03 PM Andrey Zagrebin
wrote:
> Hi all,
>
> Thanks for the feedback, Xintong
Hi All,
The discussion for FLIP-116 looks to be resolved [1].
Therefore, I start the vote for it.
The vote will end at 6pm CET on Monday, 23 March.
Best,
Andrey
[1]
trigger GC in most cases, and the max
> direct
> > memory limit can act as a safe net. Moreover, I think the cases should be
> > very rare that we need native memory for user codes. Therefore, we
> probably
> > should not break the JM/TM consistency for potential risks in suc
Hi Lake,
When the Flink doc mentions a state entry in RocksDB, we mean one key/value
pair stored by user code over any keyed state API
(keyed context in keyed operators obtained e.g. from keyBy()
transformation).
In case of map or list, the doc means map key/value and list element.
-
Andrey Zagrebin created FLINK-16615:
---
Summary: Introduce data structures and utilities to calculate Job
Manager memory components
Key: FLINK-16615
URL: https://issues.apache.org/jira/browse/FLINK-16615
Andrey Zagrebin created FLINK-16614:
---
Summary: FLIP-116 Unified Memory Configuration for Job Manager
Key: FLINK-16614
URL: https://issues.apache.org/jira/browse/FLINK-16614
Project: Flink
uld second that it is desirable to support Java 11 and in general
> use a base image that allows the (straightforward) use of more recent
> versions of other software (Python etc.)
> >
> >
> https://github.com/apache/flink-docker/blob/d3416e720377e9b4c07a2d0f4591965264ac74c5/Dock
> >
> > > > Xintong Song
> > > >
> > > >
> > > >
> > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen
> > wrote:
> > > >
> > > > > No need to revert it now - I am not saying it should not go into
&g
Ewen wrote:
>
> @Andrey about the increase in metaspace size
> - I have no concerns for 1.11.0.
> - For 1.10.1 I am not completely sure, because users expect to upgrade
> that without config adjustments. That might not be possible with that
> change.
>
> On Thu, Mar
perience when using SQL CLI
>>>
>>> Best,
>>> Jark
>>>
>>>
>>> On Wed, 11 Mar 2020 at 20:37, Yu Li wrote:
>>>
>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225
>> fix
>>>> versions and p
Hi All,
As you may have noticed, 1.10 release included an extensive improvements to
memory management and configuration of Task Managers, FLIP-49: [1]. The
memory configuration of Job Managers has not been touched in 1.10.
Although, Job Manager's memory model does not look so sophisticated as
Hi Yu,
Thanks for kicking off the 1.10.1 release discussion!
Apart from
- FLINK-16406 Increase default value for JVM Metaspace to minimise its
OutOfMemoryError
which should be merged soon
I think we should also try to get in the following issues:
- [FLINK-16225] Metaspace Out Of Memory should
convenient for users without any learning curve.
>> `docker run flink session_jobmanager -D rest.bind-port=8081`
>>
>>
>> > About the logging
>>
>> Updating the `log4j-console.properties` to support multiple appender is a
>> better option.
>> Currently, the
Hi All,
If you have ever touched the docker topic in Flink, you
probably noticed that we have multiple places in docs and repos which
address its various concerns.
We have prepared a FLIP [1] to simplify the perception of docker topic in
Flink by users. It mostly advocates for an approach of
Andrey Zagrebin created FLINK-16406:
---
Summary: Increase default value for JVM Metaspace to minimise its
OutOfMemoryError
Key: FLINK-16406
URL: https://issues.apache.org/jira/browse/FLINK-16406
Andrey Zagrebin created FLINK-16198:
---
Summary: FileUtilsTest fails on Mac OS
Key: FLINK-16198
URL: https://issues.apache.org/jira/browse/FLINK-16198
Project: Flink
Issue Type: Bug
Andrey Zagrebin created FLINK-15991:
---
Summary: Create Chinese documentation for FLIP-49 TM memory model
Key: FLINK-15991
URL: https://issues.apache.org/jira/browse/FLINK-15991
Project: Flink
Hi,
@Jingsong Lee
Regarding "OutOfMemoryError: Direct buffer memory" in
FileChannelBoundedData$FileBufferReader
I saw you created a specific issue issue:
https://issues.apache.org/jira/browse/FLINK-15981
In general, I think we could rewrap this error
in
Andrey Zagrebin created FLINK-15989:
---
Summary: Rewrap OutOfMemoryError in allocateUnpooledOffHeap with
better message
Key: FLINK-15989
URL: https://issues.apache.org/jira/browse/FLINK-15989
Project
Andrey Zagrebin created FLINK-15946:
---
Summary: Task manager Kubernetes pods take long time to terminate
Key: FLINK-15946
URL: https://issues.apache.org/jira/browse/FLINK-15946
Project: Flink
Andrey Zagrebin created FLINK-15942:
---
Summary: Improve logging of infinite resource profile
Key: FLINK-15942
URL: https://issues.apache.org/jira/browse/FLINK-15942
Project: Flink
Issue
.x. We can move to the
> Jira ticket to discuss further more.
>
> Andrey Zagrebin 于2020年2月6日周四 下午11:30写道：
>
> > Hi Benchao,
> >
> > Do you observe this issue FLINK-15938 with 1.9 or 1.10?
> > If with 1.9, I suggest to check with 1.10.
> >
> > Thanks,
>
Hi Benchao,
Do you observe this issue FLINK-15938 with 1.9 or 1.10?
If with 1.9, I suggest to check with 1.10.
Thanks,
Andrey
On Thu, Feb 6, 2020 at 4:07 PM Benchao Li wrote:
> Hi all,
>
> I found another issue[1], I don't know if it should be a blocker. But it
> does affects joins without
Andrey Zagrebin created FLINK-15774:
---
Summary: Consider adding an explicit network memory size config
option
Key: FLINK-15774
URL: https://issues.apache.org/jira/browse/FLINK-15774
Project: Flink
Andrey Zagrebin created FLINK-15763:
---
Summary: Set necessary resource configuration options to defaults
for local execution ignoring FLIP-49
Key: FLINK-15763
URL: https://issues.apache.org/jira/browse/FLINK
Andrey Zagrebin created FLINK-15758:
---
Summary: Investigate potential out-of-memory problems due to
managed unsafe memory allocation
Key: FLINK-15758
URL: https://issues.apache.org/jira/browse/FLINK-15758
Andrey Zagrebin created FLINK-15741:
---
Summary: Fix TTL docs after enabling RocksDB compaction filter by
default
Key: FLINK-15741
URL: https://issues.apache.org/jira/browse/FLINK-15741
Project
Hi Stephan,
Thanks for starting this discussion. I am +1 for this change.
In general, number of timer state keys can have the same order as number of
main state keys.
So if RocksDB is used for main state for scalability, it makes sense to
have timers there as well
unless timers are used for only
Andrey Zagrebin created FLINK-15621:
---
Summary: State TTL: Remove deprecated option and method to disable
TTL compaction filter
Key: FLINK-15621
URL: https://issues.apache.org/jira/browse/FLINK-15621
Andrey Zagrebin created FLINK-15620:
---
Summary: State TTL: Remove deprecated enable default background
cleanup
Key: FLINK-15620
URL: https://issues.apache.org/jira/browse/FLINK-15620
Project: Flink
Andrey Zagrebin created FLINK-15606:
---
Summary: Deprecate enable default background cleanup of state with
TTL
Key: FLINK-15606
URL: https://issues.apache.org/jira/browse/FLINK-15606
Project: Flink
Andrey Zagrebin created FLINK-15605:
---
Summary: Remove deprecated in 1.9 StateTtlConfig.TimeCharacteristic
Key: FLINK-15605
URL: https://issues.apache.org/jira/browse/FLINK-15605
Project: Flink
HI HuWeihua,
I think your issue should resolve with 1.9.2 and 1.10 (not released but in
progress).
You can check the related Jira ticket [1].
Best,
Andrey
[1] https://jira.apache.org/jira/browse/FLINK-12122
On Wed, Jan 15, 2020 at 10:08 AM HuWeihua wrote:
> Hi, All
> We encountered some
Andrey Zagrebin created FLINK-15597:
---
Summary: Relax sanity check of JVM memory overhead to be within
its min/max
Key: FLINK-15597
URL: https://issues.apache.org/jira/browse/FLINK-15597
Project
the config.yaml.
> Maybe we don't need to double it, but something like 1280m would be okay?
>
> On Tue, Jan 14, 2020 at 3:47 PM Andrey Zagrebin
> wrote:
>
> > Hi all!
> >
> > Great that we have already tried out new FLIP-49 with the bigger jobs.
> >
>
; wrote:
> > > >
> > > > > I think we need an interpretation of "-tm" regardless of what is in
> > the
> > > > > default configuration, because we can always have a modified
> > > > configuration
> > > > > and then
Hi all!
Great that we have already tried out new FLIP-49 with the bigger jobs.
I am also +1 for the JVM metaspace and overhead changes.
Regarding 0.3 vs 0.4 for managed memory, +1 for having more managed memory
for Rocksdb limiting case.
In general, this looks mostly to be about memory
1 - 100 of 250 matches
Mail list logo
