unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
发自我的 iPhone > 在 2015年12月3日,上午1:41,Maximilian Michels 写道: > > Great. Here is the commit to try out: > https://github.com/mxm/flink/commit/f49b9635bec703541f19cb8c615f302a07ea88b3 > > If you already have the Flink repository, check it out using > > git fetch https://github.com/mxm/flink/ > f49b9635bec703541f19cb8c615f302a07ea88b3 && git checkout FETCH_HEAD > > Alternatively, here's a direct download link to the sources with the > fix included: > https://github.com/mxm/flink/archive/f49b9635bec703541f19cb8c615f302a07ea88b3.zip > > Thanks a lot, > Max > >> On Wed, Dec 2, 2015 at 5:44 PM, Niels Basjes wrote: >> Sure, just give me the git repo url to build and I'll give it a try. >> >> Niels >> >>> On Wed, Dec 2, 2015 at 4:28 PM, Maximilian Michels wrote: >>> >>> I mentioned that the exception gets thrown when requesting container >>> status information. We need this to send a heartbeat to YARN but it is >>> not very crucial if this fails once for the running job. Possibly, we >>> could work around this problem by retrying N times in case of an >>> exception. >>> >>> Would it be possible for you to deploy a custom Flink 0.10.1 version >>> we provide and test again? >>> On Wed, Dec 2, 2015 at 4:03 PM, Niels Basjes wrote: No, I was just asking. No upgrade is possible for the next month or two. This week is our busiest day of the year ... Our shop is doing about 10 orders per second these days ... So they won't upgrade until next January/February Niels On Wed, Dec 2, 2015 at 3:59 PM, Maximilian Michels wrote: > > Hi Niels, > > You mentioned you have the option to update Hadoop and redeploy the > job. Would be great if you could do that and let us know how it turns > out. > > Cheers, > Max > >> On Wed, Dec 2, 2015 at 3:45 PM, Niels Basjes wrote: >> Hi, >> >> I posted the entire log from the first log line at the moment of >> failure >> to >> the very end of the logfile. >> This is all I have. >> >> As far as I understand the Kerberos and Keytab mechanism in Hadoop >> Yarn >> is >> that it catches the "Invalid Token" and then (if keytab) gets a new >> Kerberos >> ticket (or tgt?). >> When the new ticket has been obtained it retries the call that >> previously >> failed. >> To me it seemed that this call can fail over the invalid Token yet it >> cannot >> be retried. >> >> At this moment I'm thinking a bug in Hadoop. >> >> Niels >> >> On Wed, Dec 2, 2015 at 2:51 PM, Maximilian Michels >> wrote: >>> >>> Hi Niels, >>> >>> Sorry for hear you experienced this exception. From a first glance, >>> it >>> looks like a bug in Hadoop to me. >>> "Not retrying because the invoked method is not idempotent, and unable to determine whether it was invoked" >>> >>> That is nothing to worry about. This is Hadoop's internal retry >>> mechanism that re-attempts to do actions which previously failed if >>> that's possible. Since the action is not idempotent (it cannot be >>> executed again without risking to change the state of the execution) >>> and it also doesn't track its execution states, it won't be retried >>> again. >>> >>> The main issue is this exception: >>> org.apache.hadoop.security.token.SecretManager$InvalidToken: Invalid AMRMToken from >appattempt_1443166961758_163901_01 >>> >>> From the stack trace it is clear that this exception occurs upon >>> requesting container status information from the Resource Manager: >>> at org.apache.flink.yarn.YarnJobManager$$anonfun$handleYarnMessage$1.applyOrElse(YarnJobManager.scala:259) >>> >>> Are there any more exceptions in the log? Do you have the complete >>> logs available and could you share them? >>> >>> >>> Best regards, >>> Max >>> >>> On Wed, Dec 2, 2015 at 11:47 AM, Niels Basjes >>> wrote: Hi, We have a Kerberos secured Yarn cluster here and I'm experimenting with Apache Flink on top of that. A few days ago I started a very simple Flink application (just stream the time as a String into HBase 10 times per second). I (deliberately) asked our IT-ops guys to make my account have a max ticket time of 5 minutes and a max renew time of 10 minutes (yes, ridiculously low timeout values because I needed to validate this https://issues.apache.org/jira/browse/FLINK-2977). This job is started with a keytab file and after running for 31 hours it suddenly failed with the exception you see b
Unsubscribe
Unsubscribe
Unsubscribe
-- Regards, Harshvardhan
unsubscribe
unsubscribe
-- With thanks, Katherin Pudikova
unsubscribe
unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe This electronic correspondence, including any attachments, is intended solely for the use of the intended recipient(s) and may contain legally privileged, proprietary and/or confidential information. If you are not the intended recipient, please immediately notify the sender by reply e-mail and permanently delete all copies of this electronic correspondence and associated attachments. Any use, disclosure, dissemination, distribution or copying of this electronic correspondence and any attachments for any purposes that have not been specifically authorized by the sender is strictly prohibited.
UNSUBSCRIBE
Greetings Please unsubscribe me from your mailing list JOhn M
Unsubscribe
Unsubscribe
unsubscribe
Unsubscribe
Unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe Thanks & Regards, Kailash Kota Product Development | JDA Software Pvt Ltd. Ph: +91 80 6101 8649 Think BIG > Start small > Run fast! Download JDA's AI/ML buyers guide<http://bit.ly/2W68ef5> and get started on your autonomous SCM journey! To the extent permitted by law, we may monitor electronic communications for the purposes of ensuring compliance with our legal and regulatory obligations and internal policies. We may also collect email traffic headers for analyzing patterns of network traffic and managing client relationships. For additional information see https://jda.com/privacy-policy.
Unsubscribe
Hi, I would like to unsubscribe from the mailing list. Best, Paolo
Unsubscribe
-- Narendra Joshi
unsubscribe
-- Andrew Whitaker | andrew.whita...@braintreepayments.com -- Note: this information is confidential. It is prohibited to share, post online or otherwise publicize without Braintree's prior written consent.
unsubscribe
unsubscribe
Unsubscribe
How can I unsubscribe from the Apache Flink user mailing list? I have tried emailing user-unsubscr...@flink.apache.org, but am still receiving messages. Thank you.
unsubscribe
unsubscribe << {CH} {TS} Anything that can possibly go wrong, it does. >>
unsubscribe
unsubscribe
unsubscribe
unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscibe pon., 5 lip 2021, 08:19 użytkownik Dan Pettersson < dan.pettersso...@gmail.com> napisał: > Unsubscribe >
Unsubscribe
Hello i want to unsubscribe
unsubscribe
unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe Regards, Amit Bhatia
Unsubscribe
Unsubscribe Regards, Bhawana
Unsubscribe
Unsubscribe
unsubscribe
unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
unsubscribe
unsubscribe 退订 chenshu...@foxmail.com
unsubscribe
want to unsubscribe .
unsubscribe
Please unsubscribe me
unsubscribe
unsubscribe
unsubscribe jester_jim jester_...@sina.com 签名由 网易邮箱大师 定制
Unsubscribe
Unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
unsubscribe
Unsubscribe
-- Regards Soumen Choudhury Cell : +91865316168 mail to : sou@gmail.com
Unsubscribe
unsubscribe
unsubscribe -- Thanks & Regards Sandeep Sharat Kumar
unsubscribe
unsubscribe
unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe wt., 28 lut 2023, 17:45 użytkownik Reme Ajayi napisał: > Unsubscribe
unsubscribe
Unsubscribe
Unsubscribe -- Hi , Thank you.
Unsubscribe
Unsubscribe
Unsubscribe- 原始邮件 -发件人:Lu Niu发送时间:2023-06-15 07:26:34收件人:Teoh, Hong抄送:d...@flink.apache.org;user主 题:Re: AsyncFunction vs Async SinkThanks, Hong!I understand that if the user case is to simply write sth to an external service, Async Sink is a good option that provides features like batching, state management and rate limiting. I have some follow up questions:1. Is there any problem if we use Async Function for such a user case? We can simply drop the output and use Unordered mode.2. For AsyncFunction and Async Sink. does it make sense that both could share the same underlying implementation and the features like batching and rate limiting can benefit both?BestLu On Wed, Jun 14, 2023 at 2:20 PM Teoh, Hong <lian...@amazon.co.uk> wrote:Hi Lu, Thanks for your question. See below for my understanding. I would recommend using the Async Sink if you are writing to the external service as the final output of your job graph, and if you don’t have the ordered requirement that updates to the external system must be done before updates to some other external system within the same job graph. (More explained later). The abstraction of the Async Sink is a sink, meaning it is a terminal operator in the job graph. The abstraction is intended to simplify the writing of a sink - meaning the base implementation will handle batching, state management and rate limiting. You only need to provide the client and request structure to be used to interact with the external service. This makes writing and maintaining the sink easier (if you simply want to write to a destination with at least once processing). The AsyncFunction, as I understand it is more used for data enrichment, and is not a terminal operator in the job graph. This means the return value from the external service will continue to be passed on down the Flink Job graph. This is useful for data enrichment using the external service, or if we want to ensure the system being called in the AsyncFunction is updated BEFORE any data is written to the sinks further down the job graph. For example: Kinesis Source -> Map -> AsyncFunction (Updates DynamoDB) -> Kinesis Sink We can be sure that the updates to DynamoDB for a particular record happens before the record is written to the Kinesis Sink. Hope the above clarifies your question! Regards,Hong On 14 Jun 2023, at 19:27, Lu Niu <qqib...@gmail.com> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi, Flink dev and usersIf I want to async write to an external service, which API shall I use, AsyncFunction or Async Sink?My understanding after checking the code are:Both APIs guarantee at least once write to external service. As both API internally stores in-flight requests in the checkpoint.Async Sink provides a batching request feature. This can be implemented with Map + AsyncFunction. Map function groups requests in batches and pass it to AsyncFunction.The batching implementation can refer to AbstractMapBundleOperator if don’t want to use state.Async Sink supports retry on failed requests. AsyncFunction also supports retry in latest flink version.Async Sink supports rate limiting, AsyncFunction doesn’t.AsyncFunction can be used to implement read-update-write. Async Sink cannot.BestLu
Unsubscribe
Unsubscribe- 原始邮件 -发件人:Robin Cassan via user发送时间:2023-06-14 23:13:09收件人:Gyula Fóra抄送:user主 题:Re: Kubernetes operator: config for taskmanager.memory.process.size ignoredThanks again, maybe the jvm overhead param will act as the margin I want, I'll try that :)RobinLe mer. 14 juin 2023 à 15:28, Gyula Fóra <gyula.f...@gmail.com> a écrit :Again, this has absolutely nothing to do with the Kubernetes Operator, but simply how Flink Kubernetes Memory configs work:https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/memory/mem_tuning/#configure-memory-for-containers You can probably play around with: jobmanager.memory.jvm-overhead.fraction You can set a larger memory size in the TM spec and increase the jvm overhead fraction. Gyula On Wed, Jun 14, 2023 at 2:46 PM Robin Cassan <robin.cas...@contentsquare.com> wrote: Thanks Gyula for your answer! I'm wondering about your claim: > In Flink kubernetes the process is the pod so pod memory is always equal to process memoryWhy should the flink TM process use the whole container (and so, the whole pod) memory? Before migrating to the k8s operator, we still used Flink on kubernetes (without the operator) and left a little bit of margin between the process memory and the pod memory, which helped stability. It looks like it cannot be done with the k8s operator though and I wonder why the choice of removing this granularity in the settings Robin Le mer. 14 juin 2023 à 12:20, Gyula Fóra <gyula.f...@gmail.com> a écrit : Basically what happens is that whatever you set to the spec.taskManager.resource.memory will be set in the config as process memory.In Flink kubernetes the process is the pod so pod memory is always equal to process memory. So basically the spec is a config shorthand, there is no reason to override it as you won't get a different behaviour at the end of the day. Gyula On Wed, Jun 14, 2023 at 11:55 AM Robin Cassan via user <user@flink.apache.org> wrote: Hello all! I am using the flink kubernetes operator and I would like to set the value for `taskmanager.memory.process.size`. I set the desired value in the flinkdeployment resource specs (here, I want 55gb), however it looks like the value that is effectively passed to the taskmanager is the same as the pod memory setting (which is set to 59gb). For example, this flinkdeployment configuration: ```Spec: Flink Configuration: taskmanager.memory.process.size: 55gb Task Manager: Resource: Cpu: 6 Memory: 59Gb ```will create a pod with 59Gb total memory (as expected) but will also give 59Gb to the memory.process.size instead of 55Gb, as seen in this TM log: `Loading configuration property: taskmanager.memory.process.size, 59Gb` Maybe this part of the flink k8s operator code is responsible: https://github.com/apache/flink-kubernetes-operator/blob/d43e1ca9050e83b492b2e16b0220afdba4ffa646/flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/config/FlinkConfigBuilder.java#L393 If so, I wonder what is the rationale for forcing the flink process memory to be the same as the pod memory? Is there a way to bypass that, for example by setting the desired process.memory configuration differently? Thanks!
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe 获取 Outlook for iOS<https://aka.ms/o0ukef>
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe -- Thanks & Regards, Ajinkya Pathrudkar
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe --
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
unsubscribe
Unsubscribe
unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe
Unsubscribe - 原始邮件 - 发件人:Ganesh Walse 收件人:user@flink.apache.org 主题:Hikari data source and jdbc connection issue 日期:2024年09月04日 09点05分 Hi All, I am using Hikari data source with ojdbc 8 to connect with database in flink application. I have series of jobs which other java application submit to my jobmanager.And on each job submission I will connect to database after that my job manager and task manager metaspace increases by 10mb and it will never get released. Due to above scenario my application gets oom error after certain number of jobs. Please help me here to get out of this situation. Thanks and regards,Ganesh walse
Re: unsubscribe
To unsubscribe from this mailing list, please send a mail to /user-unsubscr...@flink.apache.org/ . On 12/14/2020 10:25 AM, yuguangyuan wrote: unsubscribe
Re: unsubscribe
Hi Abhishek, unsubscribing works by sending an email to user-unsubscr...@flink.apache.org as stated in [1]. Best, Matthias [1] https://flink.apache.org/community.html#mailing-lists On Sun, Jan 24, 2021 at 3:06 PM Abhishek Jain wrote: > unsubscribe >
Re: Unsubscribe
Please unsubscribe by sending a mail to user-unsubscr...@flink.apache.org On Thu, Apr 16, 2020 at 5:06 PM Jose Cisneros wrote: > Unsubscribe > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
Re: Unsubscribe
please send an empty email to user-unsubscr...@flink.apache.org to unsubscribe yourself. Thanks. highfei2011 wrote: Unsubscribe <mailto:user-unsubscr...@flink.apache.org>
Re: Unsubscribe
Hi Harshvardhan, You need to send an email to user-unsubscr...@flink.apache.org to unsubscribe. Best, Yangze Guo On Tue, Jul 21, 2020 at 7:12 PM Harshvardhan Agrawal wrote: > > -- > Regards, > Harshvardhan