Thanks Arvid and Kurt. That's very helpful discussion.
Currently we will continue this with Lambda, but I'll definitely do a A-A
test between Lambda and Flink for this case.
Regards,
Jiawei
On Wed, Mar 11, 2020 at 5:40 PM Kurt Young wrote:
> > The second reason is this query need to scan the
> The second reason is this query need to scan the whole table. I think we
can do better :-)
Not necessarily, you said all the changes will trigger a DDB stream, you
can use Flink to consume such
stream incrementally.
For the 1st problem, I think you can use DataStream API and register a
timer
>
> About the problem, we have 2 choices. The first one is using Flink as
> described in this email thread. The second one is using AWS Lambda
> triggered by CDC stream and compute the latest 15 days record, which is a
> walk-around solution and looks not as elegant as Flink to me.
>
>
Currently
Hi Kurt,
What you said is the 1st reason.
The second reason is this query need to scan the whole table. I think we
can do better :-)
Best,
Jiawei
On Wed, Mar 11, 2020 at 10:52 AM Kurt Young wrote:
> Hi Jiawai,
>
> Sorry I still didn't fully get your question. What's wrong with your
> proposed
Hi Jiawai,
Sorry I still didn't fully get your question. What's wrong with your
proposed SQL?
> select vendorId, sum(inventory units)
> from dynamodb
> where today's time - inbound time > 15
> group by vendorId
My guess is that such query would only trigger calculations by new event.
So if a
Hi Robert,
Your answer really helps.
About the problem, we have 2 choices. The first one is using Flink as
described in this email thread. The second one is using AWS Lambda
triggered by CDC stream and compute the latest 15 days record, which is a
walk-around solution and looks not as elegant as
Hey Jiawei,
I'm sorry that you haven't received an answer yet.
So you basically have a stream of dynamodb table updates (let's call id CDC
stream), and you would like to maintain the inventory of the last 15 days
for each vendor.
Whenever there's an update in the inventory data (a new event
Hi flink users,
We have a problem and think flink may be a good solution for that. But I'm
new to flink and hope can get some insights from flink community :)
Here is the problem. Suppose we have a DynamoDB table which store the
inventory data, the schema is like:
* vendorId (primary key)
*
