Hi Julian,
I would really like to give you input about the internals and best practices of
Edgent, however I can't :-(
I started working on Edgent after hearing from it in Miami and had been
concentrating on fixing the build system (Did the Ant->Maven conversion). So I
really understand 100% of the build system, but shortly after releasing 1.2.0
the others seem to have shifted their priorities. So right now deep knowledge
of Edgent is scarce on this list. I am trying to dig in a little deeper every
day, but as you know ... most of my time currently goes into PLC4X. But for me
Edgent is THE tool for writing PLC4X Applications and I'm not going to abandon
it.
Interesting that you mention a drop-in replacement of Kafka, as I was planning
on suggesting this on this list. As for me there is a direct correlation
between Kafka Streams and Edgent Streams, I was planning on suggesting to
streamline the naming of things as well as the syntax. So someone able to write
applications for one of the two will be able to achieve the same in the other.
Chris
Am 07.08.18, 08:39 schrieb "Julian Feinauer" <j.feina...@pragmaticminds.de>:
Hey Chris,
hello to you on this list and thanks for your response.
I get your point and I agree that Akka streams is very complex which is
especially bad for simple use cases.
But, as you also say, Edgent and Akka streams seem to be the first options
for streaming in situations where one has limited resources (which is also
often times the situation for us).
What I liked about Edgent when I first looked into it (must be 2 years ago,
I think shortly after it was renamed from Quartz) I liked the command & control
approach but up till now I'm not able to understand the concept (it instruments
something wich MBeans and JMX it seems but I can't figure it out). Do you have
any hints for "good" documentation on that or at least some code samples which
help to get the principle?
Another point I am currently missing in the IoT Landscape is a (logical)
drop in replacement for something like Apache Kafka. We use it usually for two
reasons. First, the obvious decoupling from data generation and processing (and
the possibility to backpressure) and second as a message store (in the concept
of Event Sourcing) to allow for later reprocessing.
Stripping it down, without all the distributed stuff Kafka is "just" an
ordered queue (as Jay Kreps likes to emphasize also). So in the context of
stream processing it would love to have a Sink / Source with semantics similar
to kafkas (producer, consumer, topics, offset management) which is backed by a
simple append only file. One interesting Implementation of such a queue
(without the exact semantics, I think, is e.g., [1]).
Is there something similar in Edgent currently, I did not observe anything
like that.
Would you think that this is something which would fit in Edgents scope as
IoT Streaming project?
Best
Julian
[1] https://github.com/OpenHFT/Chronicle-Queue
Am 06.08.18, 23:32 schrieb "Christofer Dutz" <christofer.d...@c-ware.de>:
Hi Julian,
welcome here on the Edgent list :-)
Well I can't speak for the rest of the team, but for me Edgent is
currently sort of the following:
- For me Edgent is the ideal entry point for any sort of IoT system
- I like to compare it with Kakfa-Connect, but without the cloud (ok we
can read and write to clouds, but Edgent is not a cloud based system)
Especially if you want to operate in constrained environments, such as:
- Limited processing power
- Low energy consumption
- Low memory consumption
- Narrow/Unreliable/High-Latency network connectivity
Then is where you can cross all the cloud based streaming frameworks
off your list.
Regarding Akka Streams ... well the presentations I did some time ago
with my little Rapiro Robot, were based on Akka Streams ... that was why I fell
in love with Edgent when I heard about it at ApacheCon NA in Miami. For me,
Akka Streams totally fries my neurons. It's far too complicated. And as soon as
you want to work with Fan-In/Fan-Out and Backpressure ... well ... I thought it
was nuts ... but I have to admit that I'm no Akka or Scala pro, so please don't
treat this assumption as a fact. It's just my opinion.
Does that answer your question? If not ... feel free to ask :)
Chris
Am 06.08.18, 13:50 schrieb "Julian Feinauer"
<j.feina...@pragmaticminds.de>:
Hi all,
this is my first post on the Apache Edgent mailing list and first,
let me say, that I’m quite impressed by all that has been done with regards to
IoT and Edge Processing in this project.
As I observe the project for a long time now and we start to see
many use cases with our customers we decided to get deeper in touch with the
community.
When reasoning about the architecture of a new system for our new
costumers including a lot of “IoT” communication we obviously came by apache
edgent.
Can you say, what in your opinion the unique selling point of
edgent is?
Is the focus on the interaction with Cloud Based IoT Services or do
you see the low footprint stream processing also as one advantage of edgent?
I tried to get a really good overview over the remote control
capabilities and the MXBean integration, but I think I’m not at it, yet.
We looked at different stream processing platforms like Flink,
Spark (which are everything except low footprint) but also Akka streams which
seems to fit very well with “low footprint”.
So do you see Akka streams as an alternative to Edgent when one
only uses a “Simple” Topology with a DirectProvider and no integration to
Remote Services or Control and Command Servers or anything?
I would love to hear your opinion on that!
Best
Julian
pragmatic minds GmbH
