Hi Philipp, I also like Java API.
Besides, I invite Steven Yurong ( @[email protected] <[email protected]> ) to join the discussion, who wrote the proposal. Best, ----------------------------------- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Philipp Zehnder <[email protected]> 于2020年4月1日周三 上午5:02写道： > Hi Xiangdong, > > yes I think this feature is exactly what we need. > > I also had a look at the proposal in the comments and it looks very > interesting. > > I have one question regarding the sinks. Do you also plan to integrate a > programming API (e.g. in Java) or do you plan to rely on broker > technologies? > For us it would be good to have a Java API, which registers the query and > a callback receiving the events. > Alternatively, we could use a message broker to forward the events into > the StreamPipes adapter, but then we would depend on another service. > What do you think about that? > > Philipp > > > > On 23. Mar 2020, at 15:17, Xiangdong Huang <[email protected]> wrote: > > > > Hi Philipp, > > > > That is really really big good news! > > > >> I think data set integration should be straightforward, but what about > > the data stream adapter? Is it possible to subscribe to changes in IoTDB? > > > > Yes the data set adapter is straightforward. > > > > As for the data stream adapter.... > > > >> Does this trigger have to be set in the database or can a client > > subscribe to changes? > > > > We are doing that! Have a look about issue [1]. It is called trigger in > > IoTDB. > > In our design, if a user registers a trigger for a time series and claims > > to send new data points to somewhere (e.g., Streampipes), > > then, you can get the changes of data on that timeseries in stream way. > > It is just a design and we have not implement it now. > > How do you think about this? > > > > [1] > > > https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-516?filter=allopenissues > > > > Best, > > ----------------------------------- > > Xiangdong Huang > > School of Software, Tsinghua University > > > > 黄向东 > > 清华大学 软件学院 > > > > > > Philipp Zehnder <[email protected]> 于2020年3月23日周一 下午8:17写道： > > > >> Hi Xiangdong, > >> > >> I come from the StreamPipes community and I also think it's a great idea > >> to work together. > >> We have already integrated IoTDB as a data sink in StreamPipes. Users > can > >> model their analytics pipelines and write the results directly into > IoTDB. > >> The next step would be to also get data from IoTDB, then we can read > data, > >> do stream calculations and write the results back. > >> > >> We have already integrated several adapters for databases (e.g. > InfluxDB, > >> mysql, ...) [1]. I would like to add such an adapter for IoTDB. > >> Usually we distinguish between dataset and datastream adapters. With a > >> data set adapter we poll the data once from the database and stream it > >> through the pipeline. > >> With data stream adapters we start the adapter and continuously read and > >> process events. > >> I think data set integration should be straightforward, but what about > the > >> data stream adapter? Is it possible to subscribe to changes in IoTDB? > >> > >> In [2] it is described that a user must add a trigger. Does this trigger > >> have to be set in the database or can a client subscribe to changes? > >> > >> Philipp > >> > >> [1] > >> > https://github.com/apache/incubator-streampipes-extensions/tree/dev/streampipes-connect-adapters/streampipes-connect-adapter/src/main/java/org/apache/streampipes/connect/adapters > >> < > >> > https://github.com/apache/incubator-streampipes-extensions/tree/dev/streampipes-connect-adapters/streampipes-connect-adapter/src/main/java/org/apache/streampipes/connect/adapters > >>> > >> [2] > >> > https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-516?filter=allopenissues > >> < > >> > https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-516?filter=allopenissues > >>> > >> > >>> On 20. Mar 2020, at 10:36, Julian Feinauer < > [email protected]> > >> wrote: > >>> > >>> Hi Xiangdong, > >>> > >>> very nice to share it here! > >>> Looking forward to Prometheus and Streampipes! > >>> Andi f I can support with DBCP (or commons pool2 or whatever, I'm happy > >> to help!). > >>> > >>> Julian > >>> > >>> Am 20.03.20, 02:55 schrieb "Xiangdong Huang" <[email protected]>: > >>> > >>> Hi all, > >>> > >>> I come from IoTDB community. > >>> Following Christofer's suggestion, I'd like to share the process of > >> the > >>> integration work between IoTDB and some other Apache projects to let > >> users > >>> manage time-series data easier. > >>> > >>> 1. PLC4X (done) > >>> Under the help of Julian and other guys, PLC4X has provided an > >> example to > >>> write data directly to IoTDB [1]. > >>> > >>> 2. Prometheus (just beginning) > >>> Prometheus is a popular data collection and event alert system for > >> many > >>> applications (but maybe not so popular for IoT). > >>> We'd like to integrate Prometheus with IoTDB, by replacing > >> Prometheus's > >>> data store with IoTDB [2]. > >>> > >>> 3. Flink and RocketMQ (in-progress) > >>> Both Flink and rocketMQ are message queues and in many applications, > >> they > >>> are the entrances of data and IoTDB is behind them [3] [4] [5]. > >>> > >>> 4. MiniFi (just beginning) > >>> MiniFi is a dataflow management system. We'd like to integrate IoTDB > >> with > >>> it to allow: write data from a processor to IoTDB, and consume data > >> from > >>> IoTDB to other processors [6]. > >>> > >>> 5. Streampipes (just beginning) > >>> We'd like to add trigger function in IoTDB to allow publish alerts, > >> or do > >>> some stream calculation. A current possible solution is integrating > >> with > >>> StreamPipes [8]. > >>> > >>> IoTDB also begin to integrate with some other projects (which are > >> irrelated > >>> to IoT) but make it friendly to use IoTDB. like: > >>> > >>> 6. Calcite (almost done) > >>> Calcite provides a Standard SQL language to IoTDB, which make it easy > >> to > >>> use. A pr is open and ask for code review [7]. > >>> > >>> 7. Zeppelin (in-progress) > >>> Zeppelin gives us a web-based GUI to let users operate IoTDB > >> interactively > >>> [9]. > >>> > >>> 8. DataBase Connection Pool (just beginning) > >>> with a DBCP, developers do not need to write too many codes in their > >>> business logic codes. > >>> > >>> Those are what we can consider. > >>> We also would like to hear more ideas to make it easier to use IoTDB > >> in IoT > >>> applications. > >>> Welcome to join us if you are interested in some integration ideas. > >>> > >>> [1] > >>> > >> > https://github.com/apache/plc4x/tree/develop/plc4j/examples/hello-integration-iotdb > >>> [2] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-519 > >>> [3] https://issues.apache.org/jira/browse/COMDEV-350 > >>> [4] https://issues.apache.org/jira/browse/IOTDB-560 > >>> [5] https://github.com/apache/incubator-iotdb/pull/817 > >>> [6] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-518 > >>> [7] https://github.com/apache/incubator-iotdb/pull/902 > >>> [8] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-516 > >>> [9] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-515 > >>> > >>> Best, > >>> ----------------------------------- > >>> Xiangdong Huang > >>> School of Software, Tsinghua University > >>> > >>> 黄向东 > >>> 清华大学 软件学院 > >>> > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [email protected] > >>> For additional commands, e-mail: [email protected] > >>> > >> > >> > >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
