I put this class together when trying to create my own upsertable table sink in
Java:
public class MyTableSink implements UpsertStreamTableSink {
@Override
public TableSink> configure(String[] fieldNames,
TypeInformation[] fieldTypes) {
return null;
}
@Override
I was hoping to join a StreamTableSource to a BatchTableSource, but I find it's
not simple. A couple of questions:
1) Other than just pushing the DataSet to a Kafka topic (either internally
or externally to the application) and reading it into a DataStream are there
any means of doing
hat I’d be
creating a version for each type I want to use it with (albeit using Generic s)
and registering the functions separately for use with the correct type of table
field?
Thanks,
James.
From: Timo Walther mailto:twal...@apache.org>>
Sent: 18 July 2018 12:21
To: Porritt, James mailto:jam
with the correct type of table
field?
Thanks,
James.
From: Timo Walther
Sent: 18 July 2018 12:21
To: Porritt, James
Cc: user@flink.apache.org
Subject: Re: Keeping only latest row by key?
Hi James,
the easiest solution for this bahavior is to use a user-defined LAST_VALUE
aggregate function
In Spark if I want to be able to get a set of unique rows by id, using the
criteria of keeping the row with the latest timestamp, I would do the following:
.withColumn("rn",
F.row_number().over(
"name" : "b",
"type" : [ "null", "string" ]
}]
}
But actually, I was putting the generated MyAvroSchema file into ‘my_stats’
namespace (along with my other application code) by adding a ‘package
my_stats;’ line at the top. When I added “
.apache.avro.io.DatumReader)MODEL$.createDatumReader(SCHEMA$);
@Override public void readExternal(java.io.ObjectInput in)
throws java.io.IOException {
READER$.read(this, SpecificData.getDecoder(in));
}
}
I will check out the other suggestions you make. One concern I have is that
fr
I've been trying to use the following code:
ExecutionEnvironment env =
ExecutionEnvironment.getExecutionEnvironment();
Path path = new Path("file:///home/myuser/test.avro");
AvroInputFormat my_format = new AvroInputFormat<>(path,
MyAvroSchema.class);
DataSet
