Transform table

2022-09-11 Thread Lior Liviev
Hello, I want to make the input record be the content of a field in the output record. Can this be achieved with the Flink-SQL DSL? Let's say I'm getting an input that looks like this: {   ...   "fields": [{ "name": "intField", "type": "int"   }, {

Job id in logs

2022-07-21 Thread Lior Liviev
Hello, is there a way to add job Id to logs to distinguish between different jobs?

Re: Avoiding Dynamic Classloading for User Code

2021-12-27 Thread Lior Liviev
And everything happens at launch From: Lior Liviev Sent: Sunday, December 26, 2021 7:38 PM To: David Morávek Cc: user Subject: Re: Avoiding Dynamic Classloading for User Code CAUTION: external source Hey David, I placed the jar in flink folder but now I see

Re: Avoiding Dynamic Classloading for User Code

2021-12-26 Thread Lior Liviev
, December 23, 2021 2:06 PM To: Lior Liviev Cc: user Subject: Re: Avoiding Dynamic Classloading for User Code CAUTION: external source Then I don't really know what else to suggest in this direction. This approach should work in general, if you have control over the class path and you make sure all

Re: Avoiding Dynamic Classloading for User Code

2021-12-23 Thread Lior Liviev
We use hadoop in EMR 6.4 (if I'm not mistaken, emr has it's own version of hadoop so we don't define it) and we use flink 13.1 From: David Morávek Sent: Thursday, December 23, 2021 1:44 PM To: Lior Liviev Cc: user Subject: Re: Avoiding Dynamic Classloading

Re: Avoiding Dynamic Classloading for User Code

2021-12-23 Thread Lior Liviev
I get this: Caused by: java.lang.NoSuchMethodError: org.apache.avro.Schema.getLogicalType()Lorg/apache/avro/LogicalType; And I'm using avro 1.10 From: David Morávek Sent: Thursday, December 23, 2021 12:37 PM To: Lior Liviev ; user Subject: Re: Avoiding Dynamic

Re: Avoiding Dynamic Classloading for User Code

2021-12-22 Thread Lior Liviev
) From: David Morávek Sent: Tuesday, December 21, 2021 6:53 PM To: Lior Liviev Cc: user Subject: Re: Avoiding Dynamic Classloading for User Code CAUTION: external source "+ setting class-loading strategy to parent-first could" ... otherwise th

Re: Avoiding Dynamic Classloading for User Code

2021-12-21 Thread Lior Liviev
So again, after putting the jar in that folder I don’t need to configure anything else? Get Outlook for iOS<https://aka.ms/o0ukef> From: David Morávek Sent: Tuesday, December 21, 2021 6:39:10 PM To: Lior Liviev Cc: user Subject: Re: Avoiding D

Re: Avoiding Dynamic Classloading for User Code

2021-12-21 Thread Lior Liviev
Yes, I’m using "/jars/:jarid/run" Get Outlook for iOS<https://aka.ms/o0ukef> From: David Morávek Sent: Tuesday, December 21, 2021 6:08:51 PM To: Lior Liviev ; user Subject: Re: Avoiding Dynamic Classloading for User Code CAUTION: external sourc

Class loader

2021-12-21 Thread Lior Liviev
Hello, I wanted to know if I have my user code Jar in Flink, and I'm running it 3 times, will the class loader take the same classes at every execution?

Avoiding Dynamic Classloading for User Code

2021-12-21 Thread Lior Liviev
Hello, I have existing fixed cluster (not a new one with every job execution) and a single Jar +multiple executions with different params. Currently my procedure is: 1. Download Jar 2. Load Jar with API 3. Execute with API. I plan to avoid dynamic class loading by applying method described