Hi Paul,
> In my deployment, the hive connector (and its orc dependencies), which
contains `DefaultBucketFactoryImpl`, was packaged into the job uber jar.
On the other hand, the flink runtime, which contains `
HadoopPathBasedBulkFormatBuilder ` is located in the lib folder. Since the
two jars are
@Roman - yes, I have the error if I do that.
@Xingbo Huang - okay, I didn't know DDL was the more
recommended way.
Please let me know if you confirm that this is a bug.
Thanks!
On Mon, Jul 13, 2020 at 5:07 PM Xingbo Huang wrote:
> Hi Manas,
> Maybe it is the bug of Java Descriptor. You can try
Hi Jingsong,
Thanks to your pointer, I checked the dependencies and found out that it’s
caused by the classloaders.
In my deployment, the hive connector (and its orc dependencies), which contains
`DefaultBucketFactoryImpl`, was packaged into the job uber jar. On the other
hand, the flink runt
Maybe you could try out RAW[1]
[1]
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/types.html#raw
元始(Bob Hu) <657390...@qq.com> 于2020年7月14日周二 上午9:41写道：
> dear,
> i have two problems now:
> 1. when i stop a flink job using command "yarn application -kill "
> or "echo 'stop' | ./b
dear,
i have two problems now:
1. when i stop a flink job using command "yarn application -kill
Hi Murali,
A proper fix of this problem could take some time. It may or may not catch
the next bug-fix release (1.11.1).
At the meantime, you can try to workaround this by upgrading your JDK8 to a
recent release. E.g., the latest JDK8u252 [1].
Thank you~
Xintong Song
[1]
https://adoptopenjdk.
Hi,
I have a docker image of the Beam WordCount example that reads a
status file and produces a output one time with word counts etc.
This runs fine as a separate job-manager and task-manager when run from
docker-compose locally.
Now, I am trying to deploy and run this on my Kubernetes cluster a
Hello Guys,
Can someone please, explain to me how can I deploy a flink app on a server,
the steps I need to flow in order to achieve that ?
Sorry for disturbing you guys.
Aissa
Hi,
I would like to ask Flink Pojo Serialziation described in [1]
I have a case where my custom event source produces Events described by
Pojo:
public class DataPoint
{
public long timestamp;
public double value;
public BadPojo badPojo = new BadPojo();
public Data
Hi Antonio,
Please take a look at broadcast variables:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/batch/#broadcast-variables
Regards,
Roman
On Mon, Jul 13, 2020 at 3:49 PM Antonio Martínez Carratalá <
amarti...@alto-analytics.com> wrote:
> Hello
>
> I'm trying to implement the
I've read that part but ParameterTool has no utilities to validate and type
parameters. Indeed I was asking if there are suggestion about possible
alternatives (e.g. Apache Commons CLI is the first that comes into my mind).
Best,
Flavio
On Mon, Jul 13, 2020 at 5:28 PM Chesnay Schepler wrote:
>
/** * ... * * @deprecated These classes will be dropped in the next
version. Use {@link ParameterTool} or a third-party * command line
parsing library instead. */
On 13/07/2020 17:24, Flavio Pompermaier wrote:
In Flink 1.11 RequiredParameters and Option have been deprecated.
Is there any reco
In Flink 1.11 RequiredParameters and Option have been deprecated.
Is there any recommendation in this sense (about how to get rid of
deprecation warnings)?
Best,
Flavio
Thanks, that was definitely helpful!
On Mon, Jul 13, 2020 at 4:39 PM Jark Wu wrote:
> You can set string-based configuration on
> `tEnv.getConfig.getConfiguration.setString(..)` to replace them.
> Maybe you can try pipeline.default-kryo-serializers [1].
>
> Best,
> Jark
>
> [1]:
> https://ci.apa
You can set string-based configuration on
`tEnv.getConfig.getConfiguration.setString(..)` to replace them.
Maybe you can try pipeline.default-kryo-serializers [1].
Best,
Jark
[1]:
https://ci.apache.org/projects/flink/flink-docs-master/ops/config.html#pipeline-default-kryo-serializers
On Mon, 13
And what about the env.registerTypeWithKryoSerializer?
Now to create the table environment I don't use the ExecutionEnvironment
anymore..how can I register those serializers?
For example I used to run
env.registerTypeWithKryoSerializer(DateTime.class,
JodaDateTimeSerializer.class);
Best,
Flavio
O
Hello
I'm trying to implement the ForceAtlas2 (graph layout) algorithm in Flink
using datasets, it is an iterative algorithm and I have most of it ready,
but there is something I don't know how to do. Apart from the dataset with
the coordinates (x,y) of each node I need an additional variable to
r
Hi Flavio,
tableEnv.registerTableSource is deprecated in order to migrate to use DDL
and the new connector interface (i.e. FLIP-95 [1]).
You may need to implement a `ScanTableSource` that uses
`InputFormatProvider` as the `ScanTableSource#getScanRuntimeProvider`.
Best,
Jark
[1]:
https://ci.apach
Thanks for quick reply!
> On Jul 13, 2020, at 4:14 AM, Xintong Song wrote:
>
>
> FYI, I've opened FLINK-18581[1] for tracking this.
>
> Thank you~
> Xintong Song
>
> [1] https://issues.apache.org/jira/browse/FLINK-18581
>
>> On Mon, Jul 13, 2020 at 4:54 PM Xintong Song wrote:
>> I think
Ok..just one last thing: to use my TableSource I use the deprecated
API registerTableSource:
tableEnv.registerTableSource("MySourceDataset", tableSource);
The javadoc says to use executeSql but this requires some extra steps (that
are not mentioned in the documentation).
Do I have to create a Tab
I agree with you @Flavio Pompermaier , the exception
message definitely should be improved.
We created a similar issue a long time before
https://issues.apache.org/jira/browse/CALCITE-3038, but the fixing might be
complicated.
Best,
Jark
On Mon, 13 Jul 2020 at 19:59, Flavio Pompermaier
wrote:
You're right Jark..sorry I didn't see the typo. The backticks are also
mandatory.
Maybe the exception message could be more meaningful and specify the token
that caused the error instead of a general "SQL parse failed. Non-query
expression encountered in illegal context".
Thanks a lot for the supp
A typo of "INSERTO"? Try this?
tableEnv.executeSql("INSERT INTO `out` SELECT * FROM MySourceDataset");
Best,
Jark
On Mon, 13 Jul 2020 at 18:25, Flavio Pompermaier
wrote:
> Now I'm able to run my code but there's something I don't understand: what
> is the difference between the following two?
Hi Manas,
Maybe it is the bug of Java Descriptor. You can try the DDL[1] way which is
the more recommended way
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/
Best,
Xingbo
Khachatryan Roman 于2020年7月13日周一 下午7:23写道：
> Hi Manas,
>
> Do you have the same erro
Hi Manas,
Do you have the same error if you replace
.group_by("five_sec_window, monitorId") \
with
.group_by("five_sec_window") \
?
Regards,
Roman
On Mon, Jul 13, 2020 at 11:16 AM Manas Kale wrote:
> Hi,
> I have the following piece of code (for pyFlink v1.11) :
>
> t_env.from_pat
Dear community,
happy to share this week's community update with a discussion on
releasing Flink 1.11.1, FLIPs on a Python DataStream API and improvements
to the "Connect API" of the Table API and a bit more.
Flink Development
==
* [releases] Flink 1.11.0 was released. Check out the
Now I'm able to run my code but there's something I don't understand: what
is the difference between the following two?
//common code
final CsvTableSink outSink = new CsvTableSink("file:/tmp/test.tsv",
"\t", 1, WriteMode.OVERWRITE);
tableEnv.registerTableSink("out", dsFields,
myInpu
Hi, It looks really weird.
Is there any possibility of class conflict?
How do you manage your dependencies? Do you download bundle-jar to lib? [1]
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar
Best,
Jingsong
On Mon, Jul 13, 2020 at 5:48
Hi,
I’m trying out Flink 1.11 and trying to write data to Hive orc tables, but get
stuck with a weird exception. Wonder if anyone had met this before? The Java
version is 1.8.0_151.
java.lang.IllegalAccessError: tried to access class
org.apache.flink.streaming.api.functions.sink.filesystem.De
Hi,
I have the following piece of code (for pyFlink v1.11) :
t_env.from_path(INPUT_TABLE) \
.select("monitorId, data, rowtime") \
.window(Tumble.over("5.minutes").on("rowtime").alias("five_sec_window")) \
.group_by("five_sec_window, monitorId") \
.select("monitorId, data.avg, data.
FYI, I've opened FLINK-18581[1] for tracking this.
Thank you~
Xintong Song
[1] https://issues.apache.org/jira/browse/FLINK-18581
On Mon, Jul 13, 2020 at 4:54 PM Xintong Song wrote:
> I think the problem is that the package-private method
> `Reference.tryHandlePending` does not exist in 1.8.
I think the problem is that the package-private method
`Reference.tryHandlePending` does not exist in 1.8.0_40.
The method does not exist in OpenJDK 8u40[1], but can be found in the
latest AdoptOpenJDK [2].
It seems the method was first introduced in 8u202[3].
Thank you~
Xintong Song
[1]
https:
Hi,
Eventually flatMapWithState solved the problem. I started by looking into
KeyedProcessFunction which lead me to flatMapWithState. It's working very
well.
.keyBy(…)
.flatMapWithState[Event, Int] { (event, countOpt) =>
val count = countOpt.getOrElse(0)
if (count < config.limit) (List(event)
Hi Murali Pusala,
Thanks for reporting this issue.
Looks like JavaGcCleanerWrapper should use getDeclaredMethod instead of
getMethod to find Reference.tryHandlePending.
@Andrey, can you confirm this?
Regards,
Roman
On Mon, Jul 13, 2020 at 4:42 AM Murali Krishna Pusala <
muralipus...@icloud.co
34 matches
Mail list logo
