Re: [DISCUSS] Refactor the MQTT subsystem to work with external MQTT Brokers?

[Lukas Ott]

+1
as it is most likely other MQTT Brokers are used in an enterprise grade
context.

Additionally, if for some reason some use cases like to shift towards
other protocols such as AMQP or Zenoh it is helpful to have it based on the
architecture to have a "normal" MQTT or other protocol library.

I personally talked about that topic at ASF Community over Code in New
Orleans in 2022. ->
https://www.apachecon.com/acna2022/slides/01_Ott_From_Leadingedge_Iotprotocols.pdf
where I listed additional protocols that could be interesting to be
supported. Especially on the edge.

Just my five cents.

Lukas


Am Mo., 29. Apr. 2024 um 14:01 Uhr schrieb Christofer Dutz <
christofer.d...@c-ware.de>:

> Hi all,
>
> currently a lot of the plans we have for 2024 (at least my plans) involve
> MQTT as one form of transport for data into IoTDB.
>
> Unfortunately, currently all of IoTDBs MQTT support is built around the
> built-in Moquette MQTT broker.
>
> However, I think it’s totally out of the question, that any larger company
> would agree on using this broker as their communication back-bone.
>
> Therefore, I would propose to refactor IoTDB to be able to use any MQTT
> broker as client but to keep Moquette in there, if a user doesn’t have an
> MQTT broker available.
>
> The main change that we would need, would be to refactor the code that
> currently integrates into Moquette for processing messages and to rewrite
> this to use a normal MQTT client library.
>
>
> What do you others think about this?
>
> Chris
>
>

Re: Zenoh supporting IoTDB as Backend

[Lukas Ott]

Hi Xiangdong,

It combines the advantages of MQTT and DDS and currently has Influxdb or
InMemory backend. So technically it is a perfect fit for IoTDB to support
Zenoh as it aims with Hight throughput and low footprint to support all
kind of IoT usecases wether for ROS or standard IoT applications that need
an high throughput.

https://zenoh.io/docs/getting-started/key-concepts/

So probably I can just use REST interface.

https://zenoh.io/docs/manual/backends-list/

Right?

Best,

Lukas

Xiangdong Huang  schrieb am Mo., 23. Mai 2022, 02:36:

> Hi Lukas,
>
> we can not simply modify
> https://github.com/eclipse-zenoh/zenoh-backend-influxdb  using
> https://iotdb.apache.org/UserGuide/Master/API/InfluxDB-Protocol.html
> because it is Java only now.
>
> We may have to use iotdb-rust-client to implement the program.
> (we have no an official rust client now, but there are some contributors
> working on it, see [1]
>
> BTW, I quickly read the Zenoh project, it is a pub/sub system.
> I'd like to know why we are interested in the project (or why an IoT
> application should use Zenoh..)
> Because of its high throughput (I read an article said the throughput can
> reach to 45Gb/s for a 1MB payload)? Or its lightweight resource cost? (only
> takes 300 Bytes on an Arduino Uno) ?
>
> [1] https://www.google.com/search?q=iotdb-rust
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Lukas Ott  于2022年5月23日周一 01:06写道：
>
> > Hi IoTDB developers,
> >
> > https://zenoh.io/docs/manual/backends-list/ In the backend there is
> > currently listed InfluxDB and RocksDB. I was wondering what it would take
> > to support https://iotdb.apache.org/ or if it can just leverage the
> > existing InfluxDB Backend
> > https://github.com/eclipse-zenoh/zenoh-backend-influxdb with a few
> > changes:
> > https://iotdb.apache.org/UserGuide/Master/API/InfluxDB-Protocol.html so
> > that IoTdb is also supported. Can someone go into more details? (of
> course
> > supporting TsFile output would also be really interesting to see.)
> >
> > Thanks for giving your perspective and some hints on that feature idea.
> >
> > BR / otluk
> >
>

Zenoh supporting IoTDB as Backend

[Lukas Ott]

Hi IoTDB developers,

https://zenoh.io/docs/manual/backends-list/ In the backend there is
currently listed InfluxDB and RocksDB. I was wondering what it would take
to support https://iotdb.apache.org/ or if it can just leverage the
existing InfluxDB Backend
https://github.com/eclipse-zenoh/zenoh-backend-influxdb with a few changes:
https://iotdb.apache.org/UserGuide/Master/API/InfluxDB-Protocol.html so
that IoTdb is also supported. Can someone go into more details? (of course
supporting TsFile output would also be really interesting to see.)

Thanks for giving your perspective and some hints on that feature idea.

BR / otluk

[FEATURE] Does Apache IoTDB support Apache Arrow?

[Lukas Ott]

In the documenation I was looking for a linkage to Apache Arrow for example
to do similar things like this:  A gentle introduction to Apache Arrow with
Apache Spark and Pandas | by Antonio Cachuan | Towards Data Science


So would indeed be a neat feature to support Apache Arrow
specification directly:
Apache Arrow | Apache Arrow 

Specs: Arrow Columnar Format — Apache Arrow v7.0.0


So for instance that dataframes can be accessed directly via Python, R or
other programming languages.

What do you think?

Regards,
otluk

Interesting approach on combining blockchain technology and DBMS

[Lukas Ott]

Hi iotdb-developers,

Here is an interesting paper: https://arxiv.org/abs/1912.04820
on "Chainify" DBMS. May be a feature that could be interesting for iotdb as
well.

Have a look! :)

Regards,
otluk

Re: [Discussion] Unify metrics system with rest api.

[Lukas Ott]

+1 for that. Sounds great!

Boris Zhu  schrieb am Mi., 5. Feb. 2020, 02:30:

> I will implement the rest APIs for IoTDB by using jetty. I want to move all
> web resources into an independent package and the rest APIs make web
> development easier.
>

Re: Historical data from Ditto

[Lukas Ott]

Hi Julian,

Great to hear from you. Just for information I was active on
https://gitter.im/eclipse/ditto and raised same issues on stackoverflow
(ended up in a mini PR for Ditto concerning Kubernetes API break)

So lets see how we can bring these two systems closer together. :)

One important question for me would be the possibility to deploy Apache
IoTDB on Kubernetes (including Keycloak Authentication -> That is already
included in Eclipse Ditto)

Lukas

Am Sa., 11. Jan. 2020 um 13:41 Uhr schrieb Julian Feinauer <
j.feina...@pragmaticminds.de>:

> Hi Lukas,
>
> What a coincidence!
> Yesterday I met some of the ditto devs from Bosch and am currently
> thinking about exactly that!
>
> So let me familiarize a bit more with ditto then everything its possible!
>
> Julian
>
> Von meinem Mobiltelefon gesendet
>
>
> -------- Ursprüngliche Nachricht 
> Von: Lukas Ott 
> Datum: Sa., 11. Jan. 2020, 13:36
> An: dev@iotdb.apache.org
> Betreff: Historical data from Ditto
> Hi Iotdb developers,
>
> First short introduction: I met Julian Feinhauer and saw the talk about
> Apache IoTDB and then the following question came up: How can Apache IoTDB
> support the Histical data from Eclipse Ditto?
>
> This issue has been raised on Eclipse Ditto code repository:
> https://github.com/eclipse/ditto/issues/545 and i was wondering if there
> are plans to support this. As one of the developers already mentioned IoTDB
> as potential solution it would be awesome if IoTDB would provide a way to
> safe Historical Data of Ditto.
>
> Additionally
>
> https://stackoverflow.com/questions/56993884/how-to-get-historical-sensor-data-of-a-registered-hono-device
> Bosch IoT Suite seems to have an own way to safe historical data.
>
> Thanks in advance for your answers!
>
> Kind regards
> Lukas
>