The Apache Flink community is very happy to announce the release of Apache
Flink 1.13.2, which is the second bugfix release for the Apache Flink 1.13
series.
Apache Flink® is an open-source stream processing framework for distributed,
high-performing, always-available, and accurate data
The Apache Flink community is very happy to announce the release of Apache
Flink 1.13.2, which is the second bugfix release for the Apache Flink 1.13
series.
Apache Flink® is an open-source stream processing framework for distributed,
high-performing, always-available, and accurate data
这个参数的意思是合并后的文件最大的size，你每个文件1k多，两个文件就大于2k了，所以没有触发合并
On Fri, Aug 6, 2021 at 11:59 AM Rui Li wrote:
> 可以把这个参数调大点试试呢，调成远大于单个文件的size
>
> On Thu, Aug 5, 2021 at 1:43 PM lixin58...@163.com
> wrote:
>
> > 你好，
> > 生成的三个文件挺小的，不到2kb，1k多一点，配这个是为了合并后比2k大
> >
> > --
> >
退订
On Wed, 4 Aug 2021 at 18:28, 洗你的头 <1264386...@qq.com.invalid> wrote:
> 退订
可以把这个参数调大点试试呢，调成远大于单个文件的size
On Thu, Aug 5, 2021 at 1:43 PM lixin58...@163.com
wrote:
> 你好，
> 生成的三个文件挺小的，不到2kb，1k多一点，配这个是为了合并后比2k大
>
> --
> lixin58...@163.com
>
>
> *发件人：* Rui Li
> *发送时间：* 2021-08-05 11:42
> *收件人：* user-zh
> *主题：* Re: filesystem
退订
本邮件载有秘密信息，请您恪守保密义务。未经许可不得披露、使用或允许他人使用。谢谢合作。
This email contains confidential information. Recipient is obliged to keep the
information confidential. Any unauthorized disclosure, use, or distribution of
the information in this email is strictly prohibited. Thank you.
Adding Sonam
Hi Colman,
I work for the Streaming SQL team at LinkedIn with Sonam, We have this in
our backlog. We are interested in solving this as well. If you are equally
interested, We can try to collaborate and solve this problem together.
Thanks,
Srini.
On Wed, Aug 4, 2021 at 8:05 PM
现在yarn上都是按需申请的
Best,
Yangze Guo
On Fri, Aug 6, 2021 at 10:31 AM 上官 wrote:
>
> 1.13版本中yarn模式提交时 -yn好像不能用了，请问现在要如何指定容器（taskmanager）数量？
退订
本邮件载有秘密信息，请您恪守保密义务。未经许可不得披露、使用或允许他人使用。谢谢合作。
This email contains confidential information. Recipient is obliged to keep the
information confidential. Any unauthorized disclosure, use, or distribution of
the information in this email is strictly prohibited. Thank you.
退订
Hi Joe,
Window TVF is supported since Flink 1.13, while 1.12 does not support yet.
Please upgrade to 1.13 version, or use the old Group Window Aggregate [1]
syntax in 1.12.
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/sql/queries.html#group-windows
Best,
JING ZHANG
Hi Jerome,
The type of value, list and map means that the structure of value to the
primary key. I am not sure what the set structure you mean here, if you want to
let the value as a set, and you can just leverage map state. As you might know,
java actually use HashMap to implement the
1.13版本中yarn模式提交时 -yn好像不能用了，请问现在要如何指定容器（taskmanager）数量？
Hi Sandeep,
If you set the flink-statebackend-rocksdb as provided scope, it should not
include the org.rocksdb classes, have you ever checked your application jar
package directly just as what I described?
Best
Yun Tang
From: Sandeep khanzode
Sent: Friday,
Hi all,
I am on flink 1.12.3. I am trying to get a tumbling window work with the table API as documented here:
https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/table/sql/queries/window-tvf/#tumble
I have a kafka topic as a flink source. I convert the stream
Hi,
I am new to Flink and state backend. I find Flink does provide ValueState,
ListState, and MapState. But it does not provide State object for HashSet. What
is the best practice of storing HashSet State in Flink? Should we use
ValueState and set the value to be HashSet? Or should we use
Hello Stephan, Yun,
Thanks for your insights.
All I have added is this:
org.apache.flink
flink-statebackend-rocksdb_2.12
${flink.version}
provided
No other library explicitly added. I am assuming, as mentioned, is that the
flink-dist.jar already contains the relevant classes
Hi mates !
I'm trying to find the best way to persist data into columnar format
(parquet) using Flink.
Each event contains a fixed list of properties and a variable list of
properties, defined by the user.
And I would like to save user defined properties into separate columns on
the fly.
Here
Hi everyone,
I had been using the DataSet API until now, but since that's been deprecated, I
started looking into the Table API. In my DataSet job I have a lot of POJOs,
all of which are even annotated with @TypeInfo and provide the corresponding
factories. The Table API documentation talks
@Yun Tang
Our FRocksDB has the same java package names (org.rocksdb.). Adding
'org.rocksdb' to parent-first patterns ensures it will be loaded only once,
and not accidentally multiple times (as Child-first classloading does).
The RocksDB code here is a bit like Flink internal components, which
退订
| |
jimandlice
|
|
邮箱：jimandl...@163.com
|
Hi suman,
FlinkKafkaConsumer.setStartFromLatest() means you always consume messages
from the latest whenever you restart the flink job，the consumer ignore any
committed group offsets.
auto.offset.reset=latest means the consumer fetch messages from the latest
if you never committed
FLINK-19358[1] might be related and we already have some discussion there.
[1]. https://issues.apache.org/jira/browse/FLINK-19358
Best,
Yang
Yun Tang 于2021年8月4日周三 上午11:50写道：
> Hi Harsh,
>
> The job id would be fixed as if using HA
> mode with native k8s, which
java.util.concurrent.CompletionException:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
~[?:1.8.0_141]
at
Hi Kirill, as far as I know SpecificRecordBase should work in Flink, I
don't know if there's any limitation in StateFun.
It seems that the typeClass passed to the generateFieldsFromAvroSchema from
the PravegaDeserializationSchema..
Maybe the pravega.LoadsSource does not bind correctly the Avro
Hi
Flink-1.13.2 的jar包正在同步到给个maven仓库，顺利的话，明天就可以正式announce了。
祝好
唐云
From: Jingsong Li
Sent: Wednesday, August 4, 2021 16:56
To: user-zh
Subject: Re: 1.14啥时候出呀
1.14还有1-2个月
1.13.2马上就出了，估计明天或后天或周一
On Wed, Aug 4, 2021 at 4:48 PM yidan zhao wrote:
>
Hi Stephan,
Since we use our own FRocksDB instead of the original RocksDB as dependency, I
am not sure whether this problem has relationship with this. From my knowledge,
more customers would include Flink classes within the application jar package,
and it might cause problems if the client
Hi Piotr,
1. Can we push for better benchmark coverage in the RocksDB project in the
future?
2. Sure, I think we could contribute what we did in flink-benchmarks to
improve their JMH benchmark [1]. And I will ask them how often will they run
the benchmark.
1. Can we try to catch
28 matches
Mail list logo
