Re: Stickers and Swag

[Julian Feinauer]

Hi,

no, as I understand it, they have IoTDB Stickers on their "todo" list.
They will come back to the PMC (via private@ list) for confirmation or if they 
have questions.

And yes, then, stickers will be available at ACNA.

So you do not need to do anything just wait until they approach you : )

Julian

Am 14.06.19, 10:05 schrieb "Xiangdong Huang" :

Nice! Thanks Julian!

I did not know about the sticker before you told us...

I am very interested to learn about how to do that... Just sending email to
Sally (what the mail address is?) and the priv...@iotdb.org?

And now,  we will get the stricter at ACNA, is that right?

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院
    

Julian Feinauer  于2019年6月14日周五 下午2:41写道：

> Hi all,
>
> I just got response from Sally.
> She added IoTDB to the list for Sticker preparation for Apache Con.
> I think the process is that the PMC for each project is contacted so this
> is something someone of you should take care of (cdutz, xiangdong?) 
because
> this is usually done via private@ list where I'm not subscribed.
>
> Julian
>
> Am 13.06.19, 13:42 schrieb "Julian Feinauer" <
> j.feina...@pragmaticminds.de>:
>
> Uh, nice one. Didnt know that page.
> Thank you Justin.
>
> I was just unsure if we have a full logo or "just" the Icon.
> But all good : )
>
> Julian
>
> Am 13.06.19, 13:32 schrieb "Justin Mclean" :
>
> Hi,
>
> They were likely to  just use the logo here
> http://www.apache.org/logos/ so all good.
>
> Thanks,
> Justin
>
>
>
>

Re: Stickers and Swag

[Julian Feinauer]

Uh, nice one. Didnt know that page.
Thank you Justin.

I was just unsure if we have a full logo or "just" the Icon.
But all good : )

Julian

Am 13.06.19, 13:32 schrieb "Justin Mclean" :

Hi,

They were likely to  just use the logo here  http://www.apache.org/logos/ 
so all good.

Thanks,
Justin

Re: Stickers and Swag

[Julian Feinauer]

Hi,

Yeah the logo is absolutely perfekt.
I'll contact Sally with that!


I uploaded a picture of both stickers on my one drive here: 
https://pragmaticminds-my.sharepoint.com/:i:/g/personal/j_feinauer_pragmaticminds_de/Ec9-SCaJGKNBhWZe_I6fz6QBq90CbtSGRS4NQeAeGOrLDw?e=HBg2dE

As sending images works not that good on the list : )

Julian

Am 13.06.19, 12:30 schrieb "Xiangdong Huang" :

Thanks Julian,
Absolutely IoTDB need that!

We have the logo of IoTDB[1].
But if possible, can you share the sticker of Edgent or PLC4X? I want to
know the difference between the content on the sticker with the IoTDB
project Logo.

Best,
[1]
https://svn.apache.org/repos/asf/comdev/project-logos/originals/iotdb-1.svg

---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年6月13日周四 下午5:55写道：

> Hi Xiangdong,
>
> exactly.
> We already have stickers for Edgent and PLC4X.
> The process is pretty easy with the lovely Sally and I'll talk to her.
> The only thing we need is a draft of how the sticker should look like and
> the Logo of course.
>
> If someone of you provides that (A svg or pdf of how the sticker should
> look like), I'll take care of the rest and contact Sally.
>
> Julian
>
> Am 13.06.19, 11:53 schrieb "Xiangdong Huang" :
>
> It is a good idea!
> So, stickers for Apache, IoTDB, Edgent, etc..?
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年6月13日周四 下午4:44写道：
>
> > Hi all,
> >
> > as ACNA is approaching and where are represented there with at least
> a
> > Talk (and I also try to be at the ASF booth) I think we NEED to have
> some
> > kind of swag and STICKERS (also need one for my laptop!).
> > Did anyone do anything about that yet?
> > Otherwise I would approach Sally and talk to her.
> >
> > Julian
> >
>
>
>

Re: Synchronization between TsFile and Server

[Julian Feinauer]

Hi Tianan,

this would be great.
Thank you very much!

Julian

Am 13.06.19, 12:17 schrieb "李天安" :

Hi，
After consideration, I intend to write a manual for the sync tool in 
these two days. According to the manual, you can use the tool and test it. I 
think this is a solution and I hope this will help you.

Best Regards,
—
Tianan Li
School of Software, Tsinghua University

> 在 2019年6月13日，下午5:38，李天安  写道：
> 
> Hi，
>   You’re right. The essence of sync tool is to deal with loading external 
tsfile in a running IoTDB service, which takes different strategies according 
to different tsfile types. We will consider your needs and look forward to 
cooperating with you.
> 
> Best Regards,
> —
> Tianan Li
> School of Software, Tsinghua University
    > 
    >> 在 2019年6月13日，下午5:28，Julian Feinauer  写道：
>> 
>> okay, this is also the impression I have __
>> But the general idea still is true, that there will be a way to simply 
ship a "local file" over to the server and he does some kind of batch import 
for me, right?
>> If we start to intensivy our tests I can of course help with the 
development, if you point me to the right directions.
> 
> 

Re: Stickers and Swag

[Julian Feinauer]

Hi Xiangdong,

exactly.
We already have stickers for Edgent and PLC4X.
The process is pretty easy with the lovely Sally and I'll talk to her.
The only thing we need is a draft of how the sticker should look like and the 
Logo of course.

If someone of you provides that (A svg or pdf of how the sticker should look 
like), I'll take care of the rest and contact Sally.

Julian

Am 13.06.19, 11:53 schrieb "Xiangdong Huang" :

It is a good idea!
So, stickers for Apache, IoTDB, Edgent, etc..?
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年6月13日周四 下午4:44写道：

> Hi all,
>
> as ACNA is approaching and where are represented there with at least a
> Talk (and I also try to be at the ASF booth) I think we NEED to have some
> kind of swag and STICKERS (also need one for my laptop!).
> Did anyone do anything about that yet?
> Otherwise I would approach Sally and talk to her.
>
> Julian
>

Re: Synchronization between TsFile and Server

[Julian Feinauer]

Hi,

okay, this is also the impression I have __
But the general idea still is true, that there will be a way to simply ship a 
"local file" over to the server and he does some kind of batch import for me, 
right?
If we start to intensivy our tests I can of course help with the development, 
if you point me to the right directions.

Julian

Am 13.06.19, 11:09 schrieb "李天安" :

Hi，
Thanks for your attention to IoTDB again. Well,  currently sync tools 
have not yet been fully tested and are temporarily instable,  so there is no 
open documentation. This tool plays an important role in distributed version of 
IoTDB, so it will be perfected with distributed development and will be 
published with distributed manual.

Best Regards,
—
Tianan Li
School of Software, Tsinghua University

> 在 2019年6月13日，下午4:39，Julian Feinauer  写道：
> 
> Hi lovely IoTDB-community,
> 
> I will soon have some time to play a bit more again with IoTDB.
> And one thing which is crucial for my use cases is the file based storage.
> But, as indicated in some docs there is the possibility (or there was, as 
least) to sync local TsFiles (e.g. on an edge device) with a running DB Server.
> Can someone help me point at the right place or comment on the current 
state of that?
> 
> Thanks!
> Julian

Stickers and Swag

[Julian Feinauer]

Hi all,

as ACNA is approaching and where are represented there with at least a Talk 
(and I also try to be at the ASF booth) I think we NEED to have some kind of 
swag and STICKERS (also need one for my laptop!).
Did anyone do anything about that yet?
Otherwise I would approach Sally and talk to her.

Julian

Synchronization between TsFile and Server

[Julian Feinauer]

Hi lovely IoTDB-community,

I will soon have some time to play a bit more again with IoTDB.
And one thing which is crucial for my use cases is the file based storage.
But, as indicated in some docs there is the possibility (or there was, as 
least) to sync local TsFiles (e.g. on an edge device) with a running DB Server.
Can someone help me point at the right place or comment on the current state of 
that?

Thanks!
Julian

AW: Proposal Accepted!

[Julian Feinauer]

Hi thanks for the hint... We should look there and also contribute that back, I 
agree...

J

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Re: Proposal Accepted!
Von: Christofer Dutz
An: dev@iotdb.apache.org
Cc:

Hi all,

And if you are planning on working on your slides together. The Apache Training 
podling might have just the right presentation framework for you ;-)

Chris

Outlook für Android<https://aka.ms/ghei36> herunterladen


From: Willem Jiang 
Sent: Wednesday, June 12, 2019 3:10:58 AM
To: dev@iotdb.apache.org
Subject: Re: Proposal Accepted!

Yeah, it's a good news.  My proposal was accepted too and hope to see
you in this year Apache Con NA.

Willem Jiang

Twitter: willemjiang
Weibo: 姜宁willem

On Tue, Jun 11, 2019 at 6:50 PM Julian Feinauer
 wrote:
>
> Hey,
>
> just wanted to share the happy news with the whole list, that the IoTDB 
> Proposal for Apache Con NA which was authored by Xiangdong and myself was 
> accepted.
> So IoTDB will be represented at the ACNA with a Talk!
>
> Really looking forward to that.
>
> Julian

Re: Proposal Accepted!

[Julian Feinauer]

Uh, nice, some pressure :D

Am 11.06.19, 13:18 schrieb "Christofer Dutz" :

Congrats :-)

looking forward to that ... even had Sharan reschedule my community talk so 
I could attend your talk :-)

Chris

Am 11.06.19, 12:50 schrieb "Julian Feinauer" :

Hey,

just wanted to share the happy news with the whole list, that the IoTDB 
Proposal for Apache Con NA which was authored by Xiangdong and myself was 
accepted.
So IoTDB will be represented at the ACNA with a Talk!

Really looking forward to that.

Julian

AW: Why Time Series Data Matters

[Julian Feinauer]

Hi Kevin,

thanks for forwarding this.
Indeed it is closely related to this project.
This whitepaper is from the influxDB guys.

I took the time to check it out and quickly skip over it, and my short summary 
is not worth the time :D
It states some things which can already be seen on their homepage.
As influxdb is pretty heavy on monitoring and that stuff it mostly focuses on 
that and of course, tells us that influx is superior to all other timeseries 
databases 

But nonetheless, its good to keep an eye on others and as a whole, bring this 
timeseries thing forward, as I really see it as a bottleneck for many iot use 
cases.
This is the reason why i love iotdb that much.. it solves things in an edge 
compliant way (which influx does not that much).

Julian

Von: Kevin A. McGrail 
Gesendet: Donnerstag, 6. Juni 2019 18:00
An: dev@iotdb.apache.org
Betreff: Fwd: Why Time Series Data Matters

Thought this might be of interest to the project



 Forwarded Message 
Subject: Why Time Series Data Matters
Date:Thu, 06 Jun 2019 15:54:56 + (UTC)
From:SD Times 
Reply-To:




Why Time Series Data Matters
Expand your knowledge with these whitepapers from SDTimes learning center!
Not rendering correctly? View this email as a web page here
.


SDTimes-large-logo.jpg



Screen Shot 2019-05-14 at 10.06.54 AM
Time
series data has historically been associated with applications in
finance. However, as developers and businesses move to instrument more
of their servers, applications, network infrastructure and the
physical world, timeseries is becoming the de facto standard for
how to think about storing, retrieving,  and mining this data for
real-time and historical insight. Read more

>

AW: FW: Travel Assistance for ApacheCon NA Las Vegas 2019 now open.

[Julian Feinauer]

Hi,

Yes of course. And I agree that we should do that!

Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Re: FW: Travel Assistance for ApacheCon NA Las Vegas 2019 now open.
Von: Xiangdong Huang
An: dev@iotdb.apache.org
Cc:

Hi, it is a good news, many Thanks!

And, is that possible to submit proposals (not TAC) on both Apache Con NA
and Europe?

Let more people know about IoTDB is very helpful for the project.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Christofer Dutz  于2019年5月17日周五 下午3:29写道：

> Hi all,
>
> I just wanted to let you know that we finally managed to open the
> applications for Apache TAC for ApacheCon NA 2019 in Las Vegas.
>
> For those of you that don’t know what TAC is, it’s Apaches Travel
> Assistance Committee. It’s the main goal of ApacheCons to get the
> communities together.
> Sometimes not everyone can afford to go. This is where TAC comes in. If
> the reason is, that you can’t afford coming, you can apply for travel
> assistance.
> If you are selected, Apache will pay for flight, hotel, conference-fees
> and even food and drinks while at the conference.
>
> I’ll never forget my first trip to ApacheCon in Denver about 5 years ago.
> It was an amazing experience.
>
> I should however mention, that we usually ask people to apply for tac to
> the closest ApacheCon in a year.
> So if you live in Europe applying for ApacheCon Europe in Berlin might be
> more apropreate (Applications for that will open soon)
> I’ll definitely be at both and looking forward to meet you.
>
> And if you do apply … please, please, please read the question and the
> comment under the answer field fully.
> This makes the lives of TAC judges so much easier and could even improve
> your chances of being selected.
>
> Chris
>
> Von: Gavin McDonald 
> Antworten an: "travel-assista...@apache.org"  >
> Datum: Freitag, 17. Mai 2019 um 09:12
> An: "committ...@apache.org" 
> Betreff: Travel Assistance for ApacheCon NA Las Vegas 2019 now open.
>
> Hi Committers!
>
> The Travel Assistance Committee (TAC) are pleased to announce that travel
> assistance applications for ApacheCon NA 2019 are now open!
>
> We will be supporting ApacheCon NA Las Vegas, Nevada on 9th - 12th
> September 2019
>
>  TAC exists to help those that would like to attend ApacheCon events, but
> are unable to do so for financial reasons.
> For more info on this years applications and qualifying criteria, please
> visit the TAC website at < http://www.apache.org/travel/ <
> http://www.apache.org/travel/> >. Applications opened today and will
> close around the 21st June.
>
> Important: Applications close on June 21st, 2019. Applicants have until
> the closing date above to submit their applications (which should contain
> as much supporting material as required to efficiently and accurately
> process their request), this will enable TAC to announce successful awards
> shortly afterwards.
>
> As usual, TAC expects to deal with a range of applications from a diverse
> range of backgrounds. We therefore encourage (as always) anyone thinking
> about sending in an application to do so ASAP.
> We look forward to greeting many of you in Las Vegas!
>
> Kind Regards,
> Gavin - (On behalf of the Travel Assistance Committee)
>

Re: [jira] [Created] (IOTDB-84) Out-of-Memory bug

[Julian Feinauer]

Hi Xiangdong,

first, thanks for bringing it back tot he list and the excellent design 
document.
I just went shortly over it, I need some more time to read it in detail and 
give, if I can, some sensible feedback.

Julian

Am 28.04.19, 15:41 schrieb "Xiangdong Huang" :

Hi,

Tian Jiang and I discussed about this issues and proposed a new design to
control memory usage for Overflow.

I leave the design document at:

https://cwiki.apache.org/confluence/display/IOTDB/New+Design+of+Overflow+and+the+Mergence+Process


Please leave your comment at
https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-84 or this
mailing list.

Best,

---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Xiangdong Huang  于2019年4月22日周一 下午12:50写道：

> Hi,
>
> I think we can split the task 1~3 as sub-tasks in JIRA.
>
> And, I recommend to learn how Cassandra manages memory (in package:
> org.apache.cassandra.utils.memory) and then design our strategy.
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> kangrong (JIRA)  于2019年4月22日周一 下午12:42写道：
>
>> kangrong created IOTDB-84:
>> -
>>
>>  Summary: Out-of-Memory bug
>>  Key: IOTDB-84
>>  URL: https://issues.apache.org/jira/browse/IOTDB-84
>>  Project: Apache IoTDB
>>   Issue Type: Bug
>> Reporter: kangrong
>>  Attachments: image-2019-04-22-12-38-04-903.png
>>
>> It occurs out-of-memory problem in the last long-term test of branch
>> "add_disabled_mem_control":
>>
>> !image-2019-04-22-12-38-04-903.png!
>>
>> We analyze the reason and try to solve it as follows:
>>  # 1. *Flushing to disk may double the memory cost*: A storage group
>> maintains a list of ChunkGroups in memory and will be flushed to disk 
when
>> its occupied memory exceeding the threshold (128MB by default).
>>  ## In the current implementation, when starting to flush data, a
>> ChunkGroup is encoded in memory and thereby a new byte array is kept in
>> memory. Until all ChunkGroups have been encoded in memory, their
>> corresponding byte arrays can be released together. Since the byte array
>> has a comparable size with original data (0.5× to 1×), the above strategy
>> may double the memory in the worst case.
>>  ## Solution: It is needed to redesign the flush strategy. In TsFile, a
>> Page is the minimal flush unit, where a ChunkGroup contains several 
Chunks
>> and a Chunk contains several pages. Once a page is encoded into a byte
>> array, we can flush the byte array to disk and then release it. In this
>> case, the extra memory is a page size (64KB by default) at most. This
>> modification involves a list of cascading change, including metadata 
format
>> and writing process.
>>  # *Memory Control Strategy*: It is needed to redesign the memory Control
>> Strategy. For example, assigning 60% memory to the writing process and 
30%
>> memory to the querying process. The writing memory includes the memory
>> table and the flush process. As an Insert coming, if its required memory
>> exceeds TotalMem * 0.6 - MemTableUsage - FlushUsage, the Insert will be
>> rejected.
>>  # *Is the memory statistics accuracy?* In current codes, the memory
>> usage of a TSRecord Java Object, corresponding to an Insert SQL, is
>> calculated by summating its DataPoints. e.g., "insert into
>> root.a.b.c(timestamp,v1, v2) values(1L, true, 1.2f)", its usage is 8 + 1 
+
>> 4=13, which ignores the size of object head and others. It is needed to
>> redesign the memory statistics accuracy carefully.
>>  # *Is there still the memory leak?* As shown in the log of the last
>> crash due to the out of memory exception, we find out the actual JVM 
memory
>> is 18G, whereas our memory statistic module only counts 8G. Besides the
>> inaccuracy mentioned in Q3, we doubt there are still memory leak or other
>> potential problems. We will continue to debug it.
>>
>>
>>
>>
>>
>> --
>> This message was sent by Atlassian JIRA
>> (v7.6.3#76005)
>>
>

AW: CFP opened for ApacheCon NA 2019 in Las Vegas

[Julian Feinauer]

Thanks xiangdong!

Von meinem Mobiltelefon gesendet

 Ursprüngliche Nachricht 
Betreff: Re: CFP opened for ApacheCon NA 2019 in Las Vegas
Von: Christofer Dutz
An: dev@iotdb.apache.org
Cc:

Thumbsup :-)

Chris

Am 16.04.19, 15:32 schrieb "Xiangdong Huang" :

Hi all,

I have submitted the proposal.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年4月15日周一 下午9:04写道：

> Hi all,
>
> Sorry for the delay.
> Will do this this afternoon.
>
> Julian
>
> Von meinem Mobiltelefon gesendet
>
>
>  Ursprüngliche Nachricht 
> Betreff: Re: CFP opened for ApacheCon NA 2019 in Las Vegas
> Von: Christofer Dutz
> An: dev@iotdb.apache.org
> Cc:
>
> Hi Xiangdong,
>
> as far as I know, you can always edit your submission after submitting ...
> I know you can't edit the individual details of a speaker (You have to
> delete and re-create in order to edit)
> But you can modify things.
>
> So I would suggest that you submit it and add Julian as soon as he
> provides information.
>
> Chris
>
>
>
> Am 15.04.19, 14:29 schrieb "Xiangdong Huang" :
>
> Hi,
>
> Thanks for the reminding.
> As discussed in another mail thread, we'd like to add Julian as the
> second
> speaker. So, I mailed to Julian for his BIO (the submission website
> requires the information).
> Once I get the information, I can finish the submission. :D
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Christofer Dutz  于2019年4月15日周一 下午8:13写道：
>
> > Hi Guys,
> >
> > co-chairing the IoT track, I just went through the CFP submissions
> to find
> > out which communities I need to approach in order to have all
> projects
> > represented.
> > I noticed even if you did discuss things, I couldn’t find a
> submission …
> >
> > Would be great if you are planning on doing so (I strongly suggest
> you do)
> > to submit this soon.
> > It helps make the lives of the track chairs a lot easier (my life).
> >
> > Chris
> >
>
>
>

AW: CFP opened for ApacheCon NA 2019 in Las Vegas

[Julian Feinauer]

Hi all,

Sorry for the delay.
Will do this this afternoon.

Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Re: CFP opened for ApacheCon NA 2019 in Las Vegas
Von: Christofer Dutz
An: dev@iotdb.apache.org
Cc:

Hi Xiangdong,

as far as I know, you can always edit your submission after submitting ...
I know you can't edit the individual details of a speaker (You have to delete 
and re-create in order to edit)
But you can modify things.

So I would suggest that you submit it and add Julian as soon as he provides 
information.

Chris



Am 15.04.19, 14:29 schrieb "Xiangdong Huang" :

Hi,

Thanks for the reminding.
As discussed in another mail thread, we'd like to add Julian as the second
speaker. So, I mailed to Julian for his BIO (the submission website
requires the information).
Once I get the information, I can finish the submission. :D

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Christofer Dutz  于2019年4月15日周一 下午8:13写道：

> Hi Guys,
>
> co-chairing the IoT track, I just went through the CFP submissions to find
> out which communities I need to approach in order to have all projects
> represented.
> I noticed even if you did discuss things, I couldn’t find a submission …
>
> Would be great if you are planning on doing so (I strongly suggest you do)
> to submit this soon.
> It helps make the lives of the track chairs a lot easier (my life).
>
> Chris
>

Re: Abstract for ApacheCon NA 2019

[Julian Feinauer]

Hi, yes, thats a very good idea Chris : )

Am 10.04.19, 09:21 schrieb "Christofer Dutz" :

You can always do a two-people talk ...

However Julian ... perhaps add yourself as secondary speaker ... I know you 
submitted another talk and only the primary speaker gets a free conference pass.
Just a cost-optimization suggestion ;-)

Chris



Am 10.04.19, 08:59 schrieb "Julian Feinauer" :

Hi Xiangdong,

thanks fort he compliment : )
As nobody else exposed his interest in this proposal I suggest to 
submit this proposal with both our names as Authors.
When ApacheCon is nearing we can sort out how to do it and who, in the 
end, makes the final polishing and gives the talk.

Do you agree?

Julian

Am 05.04.19, 04:00 schrieb "Xiangdong Huang" :

Hi,

After reading several times, I think @Julian's proposal is really 
good
enough for me... (And I notice that some contents come from the 
iotdb
website. Thanks the author :D).
I just added one sentence for introducing that IoT ask for a time 
series
database. And, added "and some use cases" in the third objective.

And, I notice that the proposal can be modified again and again 
until the
submitting channel is closed (due: May 13, 2019 9:00 PM 
(Asia/Shanghai
time)), so maybe we can submit a version these days.  Welcome more 
guys to
modify this proposal.

Title:
Hello World! Introducing Apache IoTDB - a Database for the Internet 
of
Things

Abstract:
Time series data management system is becoming increasingly 
important with
the rise of the Internet of Things, especially in the industrial 
area.
Apache IoTDB (incubating), one of the youngest projects in the 
Apache
Incubator,  is born for that!
Apache IoTDB is an integrated data management engine designed for 
time
series data, based on a file format which is able to store time 
series data
very efficiently. It provides users specific services for data 
collection,
storage and analysis. Due to its lightweight structure, high 
performance
and usable features together with its good integration with Apache 
Hadoop
and Apache Spark, it can meet the requirements of massive dataset 
storage,
high-speed data ingestion and complex data analysis for industrial 
IoT
applications.
As the project is very new to the ASF, the talk has three 
objectives.
First, introduce Apache IoTDB to the world. Second, to give an 
overview
about the architecture of the project and the technologies used 
inside and
third, to give an idea and some use cases about how IoTDB 
integrates with
other Apache Projects and where it fits in the (IoT) data processing
ecosystem.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Xiangdong Huang  于2019年3月27日周三 下午4:32写道：

> Hi Julian,
>
> Thanks for your reminding, and you really have given a great 
abstract, I
> think :D
>
> I think at least two or three of our initial committers (maybe me 
or not,
> and Prof. Wang, the dean of School of Software, Tsinghua 
University, who
> started up this project) will attend and we are glad to do 
presentation if
> there is a position.
>
> Now I am involved with reviewing the PRs (so many codes) and
> refactoring the overflow module
>
> The 3 objectives looks fine, and we can share some use cases that 
who are
> using (or trying to use) IoTDB.
>
> I will try to finish a proposal based on your abstract this 
weekend. And
> then, let's us discuss and modify it here :D
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
    >
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年3月27日周三 
下午4:10写道：
>
>> Hi all,
>>
>> as we already discussed here [1], I think that it would be 
really good
>

Re: For the first release version

[Julian Feinauer]

Also, +1 from my side for dev / master.
And indeed, a separate VM for this kind of tests would be a good idea.
And we could either manage that via separate Jenkins jobs or manual before 
releases or so.

How long do you mean with long-term? Hours, Days, Weeks, ... ?

Julian

Am 10.04.19, 09:19 schrieb "Christofer Dutz" :

Hi Xiangdong,

you can ask for a VM from Infra, if this is needed. We did that for the 
PLC4X project as we needed to do network stuff infra didn't want to have on 
it's official nodes.

And I agree having master and develop branches is a very good thing. 

Chris



Am 10.04.19, 05:12 schrieb "Xiangdong Huang" :

Hi,

Sadly, a long term test failed, see the JIRA for more details [1]. The
problem is caused by the modification file management, which is 
introduced
in PR [2].

Firstly, we need to fix this bug ASAP, otherwise the master branch does 
not
work.. ( @Tian Jiang,  the author of PR [2]).   And then we can 
keep go
on for the release version.

Secondly, it shows that for a DB, just a UT/IT test with JUnit is not
enough. Instead, a long term test is always needed once the main branch 
of
code is updated...
It seems that there is no such a public platform can support long term
test. I think we can open our lib's test environment for IoTDB if 
needed.

Thirdly, it is time to enable the dev branch...  (I remember that Julian
shared a link to introduce the functionality of the mater branch and dev
branch). We need to keep the master branch is stable all the time, and
develop in the dev branch.
If we enable that, we need to quickly resolve current bug to make the
master branch stable, and  moving all PRs to the dev branch, rather than
the master branch..

Best,

[1] https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-79
[2] https://github.com/apache/incubator-iotdb/pull/17
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Xiangdong Huang  于2019年4月3日周三 上午10:31写道：

> Hi,
>
> Now:
> - The Polding name search has finished [1]
> - The license self-check is finished [2]
>
> And we have some tasks left:
> - (1) prepare the final code version.
> - (2) modify the changes file (we have an initial version)
> - (3) begin to vote...
>
> For task (1) and (2):
>
> The following PRs have opened for several days,  if there is no other
> feedback, I will merge them:
>
> #79 [IOTDB-6]Value filter query optimization
> #97 [IOTDB-47][IOTDB-54][IOTDB-59][IOTDB-60]Aggregate+GroupBy+Fill
> #111 try to release memory asap in ReadOnlyMemChunk
> #123 provide unified query resource control interface
>
> Then I will start a long-term test (at least 5 days) with a heavy 
write
> workload for checking whether the current version is stable
>
> If everything is fine, we can begin task (2) and add a tag on the
> repository and finish task (1) and (2). And then, begin (3)...
>
> [1]
> 
https://lists.apache.org/thread.html/9d26c2eab119d345a542edd5bc5a657d451fbd1a92b33bad1fd74af5@%3Cdev.iotdb.apache.org%3E
> [2]
> 
https://lists.apache.org/thread.html/2bb4abd5a1cf9961dfbe50182bcf7cb39fcd408baa5b90fe4ddb0bd3@%3Cdev.iotdb.apache.org%3E
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>

Re: Abstract for ApacheCon NA 2019

[Julian Feinauer]

Hi Xiangdong,

thanks fort he compliment : )
As nobody else exposed his interest in this proposal I suggest to submit this 
proposal with both our names as Authors.
When ApacheCon is nearing we can sort out how to do it and who, in the end, 
makes the final polishing and gives the talk.

Do you agree?

Julian

Am 05.04.19, 04:00 schrieb "Xiangdong Huang" :

Hi,

After reading several times, I think @Julian's proposal is really good
enough for me... (And I notice that some contents come from the iotdb
website. Thanks the author :D).
I just added one sentence for introducing that IoT ask for a time series
database. And, added "and some use cases" in the third objective.

And, I notice that the proposal can be modified again and again until the
submitting channel is closed (due: May 13, 2019 9:00 PM (Asia/Shanghai
time)), so maybe we can submit a version these days.  Welcome more guys to
modify this proposal.

Title:
Hello World! Introducing Apache IoTDB - a Database for the Internet of
Things

Abstract:
Time series data management system is becoming increasingly important with
the rise of the Internet of Things, especially in the industrial area.
Apache IoTDB (incubating), one of the youngest projects in the Apache
Incubator,  is born for that!
Apache IoTDB is an integrated data management engine designed for time
series data, based on a file format which is able to store time series data
very efficiently. It provides users specific services for data collection,
storage and analysis. Due to its lightweight structure, high performance
and usable features together with its good integration with Apache Hadoop
and Apache Spark, it can meet the requirements of massive dataset storage,
high-speed data ingestion and complex data analysis for industrial IoT
applications.
As the project is very new to the ASF, the talk has three objectives.
First, introduce Apache IoTDB to the world. Second, to give an overview
about the architecture of the project and the technologies used inside and
third, to give an idea and some use cases about how IoTDB integrates with
other Apache Projects and where it fits in the (IoT) data processing
ecosystem.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Xiangdong Huang  于2019年3月27日周三 下午4:32写道：

> Hi Julian,
>
> Thanks for your reminding, and you really have given a great abstract, I
> think :D
>
> I think at least two or three of our initial committers (maybe me or not,
> and Prof. Wang, the dean of School of Software, Tsinghua University, who
> started up this project) will attend and we are glad to do presentation if
> there is a position.
>
> Now I am involved with reviewing the PRs (so many codes) and
> refactoring the overflow module
>
> The 3 objectives looks fine, and we can share some use cases that who are
> using (or trying to use) IoTDB.
>
> I will try to finish a proposal based on your abstract this weekend. And
> then, let's us discuss and modify it here :D
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年3月27日周三 下午4:10写道：
>
>> Hi all,
>>
>> as we already discussed here [1], I think that it would be really good
>> for the project to be represented with a  talk at ApacheCon NA.
>> I took the liberty to prepare an abstract for the project and suggest
>> that we submit this abstract.
>> If some of the initial committers like Xiangdong manage to come to the
>> Conference, it would be great if they do the presentation.
>> Otherwise, I can also do it (and we can assemble the slights together or
>> so).
>>
>> So please let me know what you think of this suggestion and how you like
>> this abstract:
>>
>> Hello World! Introducing Apache IoTDB - a Database for the Internet of
>> Things
>>
>> Apache IoTDB (incubating) is one of the youngest projects in the Apache
>> Incubator. It is an integrated data management engine designed for
>> timeseries data, based on a file format which is able to store timeseries
>> data very efficiently. It provides users specific services for data
>> collection, storage and analysis. Due to its lightweight structure, high
>> performance and usable features together with its good integration with
>> Apache Hadoop and Apach

Re: Reforma the output of TimeSeries in JSON

[Julian Feinauer]

Hi all,

I send this message especially as we have this discussion with the Calcite 
integration.
If we want to use Calcite as JDBC Frontend (which I highly suggest) then we 
have to handle things according to the (SQL) standard.
This means, that the result of this (Custom) Command would need a result type.
This could be a STRING with arbitrary format or a JSON which can be parsed by 
other programs or used in SQL directly (there is some JSON Support in the 
standard and also some in Calcite builtin).

So I highly recommend to consider this.

Julian


Am 04.04.19, 14:39 schrieb "Jiaye Wu" :

Or just show the series names like other RDBMS did for show tables and show
databases.
Detail information of series can be shown when `describe `.
Just a suggestion.

Thanks,
Jiaye

On Thu, Apr 4, 2019 at 8:35 PM Jiaye Wu  wrote:

> I think we should display human readable message for `show timeseries`. As
> it is a SQL command, maybe it is better to response a result *table* like
> other queries. I'am afraid JSON is too long for the CLI.
> JSON format is friendly for program to parse, so we can provide standard
> JSON message through RESTful API. Do we have plan for that?
>
> Thanks,
> Jiaye
>
> On Thu, Apr 4, 2019 at 5:14 PM Xiangdong Huang  wrote:
>
>> so, is the proposal of "time series schema with JSON format"  rejected?
>> ---
>> Xiangdong Huang
>> School of Software, Tsinghua University
>>
>>  黄向东
>> 清华大学 软件学院
>>
>>
>> 毛东方  于2019年4月4日周四 下午4:23写道：
>>
>> > Hi,
>> >
>> > Thanks for XuYi’s suggestion, now I remove all the JSONArray, so the
>> > output will be:
>> >
>> > {
>> > "a":{
>> > "b":{
>> > "d0":{
>> > "s0":{
>> > "args":{},
>> > "StorageGroup":"root.a.b.d0",
>> > "DataType":"INT32",
>> > "Compressor":"UNCOMPRESSED",
>> > "Encoding":"RLE"
>> > }
>> > }
>> > },
>> > "d0":{
>> > "s0":{
>> > "args":{},
>> > "StorageGroup":"root.a.d0",
>> > "DataType":"INT32",
>> > "Compressor":"UNCOMPRESSED",
>> > "Encoding":"RLE"
>> > },
>> > "s1":{
>> > "args":{},
>> > "StorageGroup":"root.a.d0",
>> > "DataType":"INT32",
>> > "Compressor":"UNCOMPRESSED",
>> > "Encoding":"RLE"
>> > }
>> > },
>> > "d1":{
>> > "s0":{
>> > "args":{},
>> > "StorageGroup":"root.a.d1",
>> > "DataType":"INT32",
>> > "Compressor":"UNCOMPRESSED",
>> > "Encoding":"RLE"
>> > },
>> >     "s1":{
>> > "args":{},
>> > "StorageGroup":"root.a.d1",
>> > "DataType":"INT32",
>> > "Compressor":"UNCOMPRESSED&q

AW: Reforma the output of TimeSeries in JSON

[Julian Feinauer]

+1

Von meinem Mobiltelefon gesendet

 Ursprüngliche Nachricht 
Betreff: Reforma the output of TimeSeries in JSON
Von: 毛东方
An: dev@iotdb.apache.org
Cc:

Hi,

Recently I found out that the output of TimeSeries metadata is not in standard 
JSON format, which creates difficulties for development of distributed version. 
As a result, I’m going to reformat the output of TImeSeries metadata inJSON, to 
make it in standard JSON format. The details of changes are as bellow:

BEFORE:

root:{
a:{
d0:{
s0:{
 DataType: INT32,
 Encoding: RLE,
 Compressor: UNCOMPRESSED,
 args: {},
 StorageGroup: root.a.d0
},
s1:{
 DataType: INT32,
 Encoding: RLE,
 Compressor: UNCOMPRESSED,
 args: {},
 StorageGroup: root.a.d0
}
},
d1:{
s0:{
 DataType: INT32,
 Encoding: RLE,
 Compressor: UNCOMPRESSED,
 args: {},
 StorageGroup: root.a.d1
},
s1:{
 DataType: INT32,
 Encoding: RLE,
 Compressor: UNCOMPRESSED,
 args: {},
 StorageGroup: root.a.d1
}
},
b:{
d0:{
s0:{
 DataType: INT32,
 Encoding: RLE,
 Compressor: UNCOMPRESSED,
 args: {},
 StorageGroup: root.a.b.d0
}
}
}
}
}



AFTER:

{
"root":[
{
"a":[
{
"d0":[
{
"s0":{
"args":{},

"StorageGroup":"root.a.d0",

"DataType":"INT32",

"Compressor":"UNCOMPRESSED",
"Encoding":"RLE"
}
},
{
"s1":{

"args":{"$ref":"$.root[0].a[0].d0[0].s0.args"},

"StorageGroup":"root.a.d0",

"DataType":"INT32",

"Compressor":"UNCOMPRESSED",
"Encoding":"RLE"
}
}
]
},
{
"d1":[
{
"s0":{

"args":{"$ref":"$.root[0].a[0].d0[0].s0.args"},

"StorageGroup":"root.a.d1",

"DataType":"INT32",

"Compressor":"UNCOMPRESSED",
"Encoding":"RLE"
}
},
{
"s1":{

"args":{"$ref":"$.root[0].a[0].d0[0].s0.args"},

"StorageGroup":"root.a.d1",

"DataType":"INT32",

"Compressor":"UNCOMPRESSED",
"Encoding":"RLE"
}
}
]
},
{
"b":[
{
"d0":[
   

Re: 答复: [External] Re: Commit tsfile-go

[Julian Feinauer]

Hi,

first, I think this is very cool!
To avoid this kind of problems (and make further implementations more easy) 
could we use something like protobuf or Avro or so as Serialization layer for 
the metadata objects?
This would help us to keep all drivers in sync...

What do you think?

Julian

Am 02.04.19, 11:11 schrieb "Lyndon Dong5 Li" :

Hi,

After investigation, I found the tsfile format of go version is almost the 
same with the latest TsFile format of java version, but unfortunately there are 
some differences between them: They have almost the same meta objects 
structures, but there are several attributes are not exactly the same.

Maybe we can eliminate these differences together if necessary.


-邮件原件-
发件人: Xiangdong Huang  
发送时间: 2019年4月1日 23:03
收件人: dev@iotdb.apache.org
主题: [External] Re: Commit tsfile-go

Hi,

Glad to see that Lenovo makes this module open sourced.

I want to confirm that Is this go version follow the latest TsFile format?
see https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Lyndon Dong5 Li  于2019年4月1日周一 下午7:33写道：

> Hi,
>
> I'm Lidong from Lenovo, I've just created a pull request to commit 
> project 'tsfile-go' to branch 'master'.
> Tsfile-go is a Golang version of tsfile(based on branch 'thanos'), 
> which was developed by Lenovo & TsingHua. we implemented all of 
> features in 'tsfile', including:
> 1. Writing ts data to a tsfile
> 2. Reading/querying ts data from an existing file 3. Encoding/decoding 
> with RLE/TS_2DIFF/GORILLA/PLAIN 4. Compression/decompression with 
> snappy
>
> Now we have applied tsfile-go to Lenovo IOT platform, for purpose of 
> caching & compressing ts data on edge devices.
>
> Best regards
> ---
> Lidong
> LCIG Big Data BU, Lenovo
>
>

Re: IoTDB supports distributed version

[Julian Feinauer]

Hi Felix,

could you elaborate a bit on your use cases?
I am a bit unsure about the consistency, so it would be interesting to hear 
where you see the important points.

Thanks!
Julian

Am 31.03.19, 20:25 schrieb "Felix Cheung" :

I, on the other hand, would be very interested in the strong consistency 
option.

(Very cool discussion!)



From: Julian Feinauer 
Sent: Thursday, March 28, 2019 1:10 AM
To: dev@iotdb.apache.org
Subject: Re: IoTDB supports distributed version

Hi,

this is a very interesting (and important) question.
I think we should really consider what we can skip (from an application 
perspective) and what to keep.
Perhaps a Token Ring architecture like Cassandra uses could also be a good 
fit, if we hash on the device id or something.
At least in the situations and use cases I know (strong) consistency is not 
soo important.

From a CAP perspective, for me, Availability is the only undiscussable 
necessary thing... for the others... we can discuss : )

Julian

PS.: Perhaps it would be beneficial to create a design doc in confluence?

Am 28.03.19, 08:57 schrieb "Xiangdong Huang" :

yep, I think the cluster is in P2P mode when they startup. Then a leader
election algorithm will change the cluster into the M/S mode (RAFT
algorithm is qualified). If the master is down, a new master can be elected
and lead the cluster.

By the way, we need to consider the cost of keeping strong consistency of
data. As time series data in IoT scenario is hard to conflict with each
other ( I mean, user1 sends data point (t1, v1) that represents device 1
and sensor 1, meanwhile user2 sends a data point (t2, v2) that
represents the same device and sensor and t2=t1). So, supporting multiple
consistency level is better for keeping high write performance.

Best,

---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院
    

Julian Feinauer  于2019年3月28日周四 下午3:21写道：

> Hi XuYi,
>
> I like the idea but I'm unsure if I like the master / slave approach.
> We often deal with "Shopfloor" Scenarios where the setup for the Database
> is basically "MultiMaster", because we need to sync data one the one hand,
> but if a system goes down, everything else should keep working.
> Would this be possible with your approach?
> Something like leader re-election with Zookeper (or better Curator?).
> What exactly are the use cases you have in mind?
>
> Thanks!
> Julian
>
> Am 28.03.19, 05:32 schrieb "徐毅" :
>
>
>
>
> Hi,
>
>
>
>
> IoTDB only supports stand-alone version now. We plan to develop
> distributed version in next two months.
>
> We initially decided to use the master-slave architecture. The master
> node is responsible for processing read and write requests, and the slave
> node, which is a copy of master node is responsible for processing
> read-only requests.
>
> In terms of implementation, we currently intend to use the raft
> protocol to ensure the data consistency of each replica node.
>
> I have created an issue on jira at [1]. If you have any suggestion,
> please comment on jira or reply to this email.
>
> [1]. https://issues.apache.org/jira/browse/IOTDB-68
>
>
>
>
> Thanks
>
> XuYi
>
>

Re: IoTDB supports distributed version

[Julian Feinauer]

Hi,

this is a very interesting (and important) question.
I think we should really consider what we can skip (from an application 
perspective) and what to keep.
Perhaps a Token Ring architecture like Cassandra uses could also be a good fit, 
if we hash on the device id or something.
At least in the situations and use cases I know (strong) consistency is not soo 
important.

From a CAP perspective, for me, Availability is the only undiscussable 
necessary thing... for the others... we can discuss : )

Julian

PS.: Perhaps it would be beneficial to create a design doc in confluence?

Am 28.03.19, 08:57 schrieb "Xiangdong Huang" :

yep, I think the cluster is in P2P mode when they startup. Then a leader
election algorithm will change the cluster into the M/S mode (RAFT
algorithm is qualified). If the master is down, a new master can be elected
and lead the cluster.

By the way, we need to consider the cost of keeping strong consistency of
data.  As time series data in IoT scenario is hard to conflict with each
other ( I mean, user1 sends data point  (t1, v1) that represents device 1
and sensor 1, meanwhile  user2 sends a data point (t2, v2) that
represents the same device and sensor  and t2=t1). So, supporting multiple
consistency level is better for keeping high write performance.

Best,

---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年3月28日周四 下午3:21写道：

> Hi XuYi,
>
> I like the idea but I'm unsure if I like the master / slave approach.
> We often deal with "Shopfloor" Scenarios where the setup for the Database
> is basically "MultiMaster", because we need to sync data one the one hand,
> but if a system goes down, everything else should keep working.
> Would this be possible with your approach?
> Something like leader re-election with Zookeper (or better Curator?).
> What exactly are the use cases you have in mind?
>
> Thanks!
> Julian
>
> Am 28.03.19, 05:32 schrieb "徐毅" :
>
>
>
>
> Hi,
>
>
>
>
> IoTDB only supports stand-alone version now. We plan to develop
> distributed version in next two months.
>
> We initially decided to use the master-slave architecture. The master
> node is responsible for processing read and write requests, and the slave
> node,  which is a copy of master node is responsible for processing
> read-only requests.
>
> In terms of implementation, we currently intend to use the raft
> protocol to ensure the data consistency of each replica node.
>
> I have created an issue on jira at [1]. If you have any suggestion,
> please comment on jira or reply to this email.
>
> [1]. https://issues.apache.org/jira/browse/IOTDB-68
>
>
>
>
> Thanks
>
> XuYi
>
>

Abstract for ApacheCon NA 2019

[Julian Feinauer]

Hi all,

as we already discussed here [1], I think that it would be really good for the 
project to be represented with a  talk at ApacheCon NA.
I took the liberty to prepare an abstract for the project and suggest that we 
submit this abstract.
If some of the initial committers like Xiangdong manage to come to the 
Conference, it would be great if they do the presentation.
Otherwise, I can also do it (and we can assemble the slights together or so).

So please let me know what you think of this suggestion and how you like this 
abstract:

Hello World! Introducing Apache IoTDB - a Database for the Internet of Things

Apache IoTDB (incubating) is one of the youngest projects in the Apache 
Incubator. It is an integrated data management engine designed for timeseries 
data, based on a file format which is able to store timeseries data very 
efficiently. It provides users specific services for data collection, storage 
and analysis. Due to its lightweight structure, high performance and usable 
features together with its good integration with Apache Hadoop and Apache 
Spark, it can meet the requirements of massive dataset storage, high-speed data 
ingestion and complex data analysis for industrial IoT applications.
As the project is very new to the ASF, the talk has three objectives. First, 
introduce Apache IoTDB to the world. Second, to give an overview about the 
architecture of the project and the technologies used inside and third, to give 
an idea how IoTDB integrates with other Apache Projects and where it fits in 
the (IoT) data processing ecosystem.

Best
Julian

[1] 
https://lists.apache.org/thread.html/fc6a3c311364bd0f704c1e6575da26f69ad7617497a32cef33cc687f@%3Cdev.iotdb.apache.org%3E

Re: IoTDB on Twitter?

[Julian Feinauer]

+1, I like that!

Am 26.03.19, 10:44 schrieb "Christofer Dutz" :

Hi all,

while sending out a „happy birthday” tweet for Apache’s 20th birthday I 
wanted to reference IoTDB and noticed there doesn’t seem to be an @ApacheIoTDB 
account … Now that the Podling Name  Search is approved, I think it would be a 
good idea to create such an account and to start tweeting about stuff you are 
doing.

I’m doing that for the PLC4X on Twitter and LinkedIn and I did manage to 
get quite a few new folks to the project … after all … coding is only part of 
what it means to be an Apache project … It’s all about community and 
community-building.

Chris

Re: Re: [VOTE] vote for IoTDB logo

[Julian Feinauer]

+1 (non-binding)

Julian

From: 乔嘉林 Jialin Qiao 
Sent: Saturday, March 23, 2019 7:57:02 AM
To: dev@iotdb.apache.org
Subject: Re: Re: [VOTE] vote for IoTDB logo

Hi,

+1 for this logo.

Thanks.
--
Jialin Qiao
School of Software, Tsinghua University

乔嘉林
清华大学 软件学院

> -原始邮件-
> 发件人: "Kevin A. McGrail" 
> 发送时间: 2019-03-23 10:34:36 (星期六)
> 收件人: dev@iotdb.apache.org, "Xiangdong Huang" 
> 抄送:
> 主题: Re: [VOTE] vote for IoTDB logo
>
> With the PNS done, I'm +1.  I also encourage you to change your logo and
> do contests to do it.  It's been a fun exercise for Apache SpamAssassin
> over the years.
>
> On 3/22/2019 10:20 PM, Xiangdong Huang wrote:
> > Hi all,
> >
> > I‘d like to call a vote for IoTDB logo.
> >
> > The logo design is discussed at [1], and the related JIRA is [2]. And, you
> > can find the logo figure at [2].
> >
> > The vote will be open for at least 72 hours.
> >
> > Please vote accordingly:
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove with the reason
> >
> > Ref:
> > [1]
> > https://lists.apache.org/thread.html/2c7e76e33a11d85359bbfa1705306946803cf5257bf5a76c243adb1c@%3Cdev.iotdb.apache.org%3E
> >
> > [2] https://issues.apache.org/jira/browse/IOTDB-42
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
>
> --
> Kevin A. McGrail
> Member, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
> https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Integrate Calcite in IoTDB

[Julian Feinauer]

Hey Gaofei,

a short update.
I pushed a change to go to Calcite 1.18.0 Release version.
Now everything should work out of the box.

Sorry again!
Julian

Am 20.03.19, 15:19 schrieb "Julian Feinauer" :

Hi Gaofei,

well it seems from the first message indeed as if you simply should build 
the local Calicte snapshot.
Calcite is currently about to do the 1.19 release so their build should 
work.

Let me see if we can simply downgrade everything on 1.18 release of 
calcite. If so, everything should be fine : )

Sorry for this snapshot dependency, I forgot that others could also use the 
code.

Julian

Am 20.03.19, 15:04 schrieb "Gaofei Cao" :

Hi Julian,

I cloned your repo 
https://github.com/JulianFeinauer/incubator-iotdb/tree/
calcite-integration, but an error appeared as below.

Could not resolve dependencies for project
>> org.apache.iotdb:iotdb-calcite:jar:0.8.0-SNAPSHOT: Failure to find
>> org.apache.calcite:calcite-core:jar:1.19.0-SNAPSHOT in
>> https://repository.apache.org/snapshots was cached in the local
>> repository, resolution will not be reattempted until the update 
interval of
>> apache.snapshots has elapsed or updates are forced -> [Help 1]
>
>
>
And I cloned Apache-Calcite, checkout to the branch branch-1.19, 
executed
`mvn clean install -Dmaven.test.skip=true`, but an error appeared as 
below.

[ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-dependency-plugin:3.1.1:analyze-only
>> (analyze) on project calcite-linq4j: Dependency problems found -> 
[Help 1]
>
> [ERROR]
>
> [ERROR] To see the full stack trace of the errors, re-run Maven with 
the
>> -e switch.
>
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>
> [ERROR]
>
> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>
> [ERROR] [Help 1]
>> 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>
> [ERROR]
>
> [ERROR] After correcting the problems, you can resume the build with 
the
>> command
>
> [ERROR]   mvn  -rf :calcite-linq4j
>
>
> So how to solve this problem. And maybe i think it's better to ask 
this
problem in Calcite community haha. 

Thanks & Best Regards
Gaofei

Julian Feinauer  于2019年2月28日周四 上午12:23写道：

> Hi Tianan,
>
> personally I found it difficould to learn Calcite from the 
Documentation,
> but the csv Examples are quite okay to follow.
> I also filed a Jira for this issue [1] and prepared a minimal working
> example for Calcite on top of a tsfile in my fork [2].
>
> The Example in the Repo is fully functional and maps each device from 
a
> tsfile to one Table and each measurement to a column (+ timestamp).
> It supports basically all SQL Queries.
>
> Of course there is no efficiency yet like predicate pushdown or path
> selection pushdown or so... this is the fun work for the future : )
>
> Best
> Julian
>
> [1] https://issues.apache.org/jira/browse/IOTDB-28
> [2]
> 
https://github.com/JulianFeinauer/incubator-iotdb/tree/calcite-integration
>
> Am 27.02.19, 16:12 schrieb "李天安" :
>
> Hi Julian,
> Nice to hear from you.
> I’m very glad to hear that you’d like to help us with the
> integration.
> We want to proceed the integration  as follows:
> Firstly, we will learn how you integrate calcite with other
> products from official website.
> After that, we discuss the structure of Calcite Integration 
with
> you.
>
> Tianan Li
>
> > 在 2019年2月26日，上午4:39，Julian Feinauer 

> 写道：
> >
> > Hi Tianan,
> >
> > nice to hear from you!
> > I just recently recognized how far you have come and how less 
time I
> commited to IoTDB.
> > I would really love to participate in this effort and help you 
with
> the Calcite Adapter.
> > What do you think is the best way to proceed?

Re: Integrate Calcite in IoTDB

[Julian Feinauer]

Hi Gaofei,

well it seems from the first message indeed as if you simply should build the 
local Calicte snapshot.
Calcite is currently about to do the 1.19 release so their build should work.

Let me see if we can simply downgrade everything on 1.18 release of calcite. If 
so, everything should be fine : )

Sorry for this snapshot dependency, I forgot that others could also use the 
code.

Julian

Am 20.03.19, 15:04 schrieb "Gaofei Cao" :

Hi Julian,

I cloned your repo https://github.com/JulianFeinauer/incubator-iotdb/tree/
calcite-integration, but an error appeared as below.

Could not resolve dependencies for project
>> org.apache.iotdb:iotdb-calcite:jar:0.8.0-SNAPSHOT: Failure to find
>> org.apache.calcite:calcite-core:jar:1.19.0-SNAPSHOT in
>> https://repository.apache.org/snapshots was cached in the local
>> repository, resolution will not be reattempted until the update interval 
of
>> apache.snapshots has elapsed or updates are forced -> [Help 1]
>
>
>
And I cloned Apache-Calcite, checkout to the branch branch-1.19, executed
`mvn clean install -Dmaven.test.skip=true`, but an error appeared as below.

[ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-dependency-plugin:3.1.1:analyze-only
>> (analyze) on project calcite-linq4j: Dependency problems found -> [Help 
1]
>
> [ERROR]
>
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>> -e switch.
>
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>
> [ERROR]
>
> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>
> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
>
> [ERROR]
>
> [ERROR] After correcting the problems, you can resume the build with the
>> command
>
> [ERROR]   mvn  -rf :calcite-linq4j
>
>
> So how to solve this problem. And maybe i think it's better to ask this
problem in Calcite community haha. 

Thanks & Best Regards
Gaofei

Julian Feinauer  于2019年2月28日周四 上午12:23写道：

> Hi Tianan,
>
> personally I found it difficould to learn Calcite from the Documentation,
> but the csv Examples are quite okay to follow.
> I also filed a Jira for this issue [1] and prepared a minimal working
> example for Calcite on top of a tsfile in my fork [2].
>
> The Example in the Repo is fully functional and maps each device from a
> tsfile to one Table and each measurement to a column (+ timestamp).
> It supports basically all SQL Queries.
>
> Of course there is no efficiency yet like predicate pushdown or path
> selection pushdown or so... this is the fun work for the future : )
>
> Best
> Julian
>
> [1] https://issues.apache.org/jira/browse/IOTDB-28
> [2]
> https://github.com/JulianFeinauer/incubator-iotdb/tree/calcite-integration
>
> Am 27.02.19, 16:12 schrieb "李天安" :
>
> Hi Julian,
> Nice to hear from you.
> I’m very glad to hear that you’d like to help us with the
> integration.
> We want to proceed the integration  as follows:
>     Firstly, we will learn how you integrate calcite with other
> products from official website.
> After that, we discuss the structure of Calcite Integration with
> you.
>
> Tianan Li
>
> > 在 2019年2月26日，上午4:39，Julian Feinauer 
> 写道：
> >
> > Hi Tianan,
> >
> > nice to hear from you!
> > I just recently recognized how far you have come and how less time I
> commited to IoTDB.
> > I would really love to participate in this effort and help you with
> the Calcite Adapter.
> > What do you think is the best way to proceed?
> > I can tell you something about the structure of Calcite
> Integratioons (or make a walkthrough through the Plc4x integration I’ve
> done).
> >
> > What do the others think?
> >
> > Julian
> >
> > Von: 李天安 
> > Datum: Montag, 25. Februar 2019 um 17:01
> > An: "jfeina...@apache.org" 
> > Betreff: Fwd: Integrate Calcite in IoTDB
> >
> >
> >
> >
> > 下面是被转发的邮件：
> >
> > 发件人: 李天安  lt...@mails.tsinghua.edu.cn>>

Re: Apache Con NA 2019

[Julian Feinauer]

Hi Xiangdong,

That sounds really good.
I can also only emphasize again how good I think it would be to be there with a 
talk.
As I wll definetly attend I can offer to hold a presentation there, if you have 
conflicts with your schedule but I think it would definetly better if you could 
manage to come over and do it : )

I think at least we should agree to submit a proposal for the Project.

Julian

Am 19.03.19, 14:21 schrieb "Xiangdong Huang" :

Hi,

Well, it is really a great chance to let more people know about IoTDB.

We  will try to coordinate the time as we are in Tsinghua University and
September is the start date of a new semester... If everything goes well, I
think we (some colleagues and me) can attend the conference and we'd like
to introduce IoTDB in the conference :).

By the way, is ddl of the poster 2 month later?

Thanks Kevin, Julian, and Chris to let us know about the Apache Conferences.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Christofer Dutz  于2019年3月19日周二 下午4:33写道：

> Hi Guys,
>
> 
> I am helping with planning the IoT track and this time we decided to try
> to sketch an overall plot for the track.
> So far we discussed that we are planning on describing a journey of data
> from the sources (iot devices (mynewt) or industrial controllers PLC4X 
over
> edge computing Camel, Edgent, MiNifi, ...) to cloud based processing
> (Spark, NiFi, Kafka, ...) to storage (Hadoop, Cassandra, IoTDB ...) so you
> guys not being there would be a real pity.
> 
>
> And please keep in mind that in a while we will be opening TAC
> applications (Apache Travel Assistance).
> For those that don't know what that is: Apache can act as a sponsor for
> those people financially not able to attend. TAC is the committee that's 
in
> charge of this process. Chosen applicants can get their flight, hotel,
> conference-fees paid by Apache.
> It's not intended to be "free-holidays" but should help get the community
> and especially new communities to join the rest of the Apache family.
> This time we are concentrating on getting "first-timers" to the
> conferences, so I think there should be good chances for at least one of
> you.
>
> So it would be great if you guys could whip up a talk that points out the
> benefits of IoTDB that people could follow even if they have never used
> IoTDB before.
>
> Chris
>
>
>
> Am 18.03.19, 12:04 schrieb "Julian Feinauer" <
> j.feina...@pragmaticminds.de>:
>
> Hi,
>
> agree with the Logo (was just somthing coming to my mind).
> And with the talk.. I think we should just decide if we (as the
> project) like to submit an abstract to the con.
> I would really like to have one there to show the project to others...
>
> Julian
>
> Am 18.03.19, 10:17 schrieb "Xiangdong Huang" :
>
> Hi,
>
> Thanks. But we need some time to confirm that we can join the
> conference.
>
> As for the Logo vote, I think we'd better to finish the polding
> name search
> first (I have created a ticket on JIRA). If we have to change the
> name,
> then the logo has to be modified
>
    > Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年3月18日周一
> 下午4:41写道：
>
> > Hi Xiangdong,
> >
> > thats a pitty, but I'm really looking forward on meeting you in
> Berlin (I
> > am from Germany, so I'll definetly be there!).
> > As Chris and I go there, I could offer to represent the project
> there, if
> > we manage to assemble some slides together?
> >
> > Oh and we should also start caring about stickers as soon as the
> Logo is
> > "official".
> >
> > Julian
> >
> > Am 18.03.19, 09:20 schrieb "Xiangdong Huang"  >:
> >
> > Hi,
> >
> > Thanks Julian.
> >
> > Apache Con in September is clashed w

Re: Apache Con NA 2019

[Julian Feinauer]

Hi,

agree with the Logo (was just somthing coming to my mind).
And with the talk.. I think we should just decide if we (as the project) like 
to submit an abstract to the con.
I would really like to have one there to show the project to others...

Julian

Am 18.03.19, 10:17 schrieb "Xiangdong Huang" :

Hi,

Thanks. But we need some time to confirm that we can join the conference.

As for the Logo vote, I think we'd better to finish the polding name search
first (I have created a ticket on JIRA). If we have to change the name,
then the logo has to be modified

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年3月18日周一 下午4:41写道：

> Hi Xiangdong,
>
> thats a pitty, but I'm really looking forward on meeting you in Berlin (I
> am from Germany, so I'll definetly be there!).
> As Chris and I go there, I could offer to represent the project there, if
> we manage to assemble some slides together?
>
> Oh and we should also start caring about stickers as soon as the Logo is
> "official".
>
> Julian
>
> Am 18.03.19, 09:20 schrieb "Xiangdong Huang" :
>
> Hi,
>
> Thanks Julian.
>
> Apache Con in September is clashed with us...
>
> We are scheduling to attend the Apache Con in Oct in Berlin, if
> possible. :D
>
> Best,
> ---
> Xiangdong Huang
>     School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年3月18日周一 下午3:22写道：
>
> > Hi all,
> >
> > as some of you may have noticed, the abstract submission is open for
> the
> > Apache Con NA 2019 in Las Vegas (https://asf.jamhosted.net/cfp.html
> ).
> > I think it would be really cool to submit a talk about IoTDB,
> especially
> > as there is a special subtrack “IoT” where we will try to represent
> PLC4X
> > and Edgent (both also incubating).
> > Has anybody here plans to go to Las Vegas?
> >
> > Julian
> >
> >
>
>
>

Re: Apache Con NA 2019

[Julian Feinauer]

Hi Xiangdong,

thats a pitty, but I'm really looking forward on meeting you in Berlin (I am 
from Germany, so I'll definetly be there!).
As Chris and I go there, I could offer to represent the project there, if we 
manage to assemble some slides together?

Oh and we should also start caring about stickers as soon as the Logo is 
"official".

Julian

Am 18.03.19, 09:20 schrieb "Xiangdong Huang" :

Hi,

Thanks Julian.

Apache Con in September is clashed with us...

We are scheduling to attend the Apache Con in Oct in Berlin, if possible. :D

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院
    

Julian Feinauer  于2019年3月18日周一 下午3:22写道：

> Hi all,
>
> as some of you may have noticed, the abstract submission is open for the
> Apache Con NA 2019 in Las Vegas (https://asf.jamhosted.net/cfp.html).
> I think it would be really cool to submit a talk about IoTDB, especially
> as there is a special subtrack “IoT” where we will try to represent PLC4X
> and Edgent (both also incubating).
> Has anybody here plans to go to Las Vegas?
>
> Julian
>
>

Apache Con NA 2019

[Julian Feinauer]

Hi all,

as some of you may have noticed, the abstract submission is open for the Apache 
Con NA 2019 in Las Vegas (https://asf.jamhosted.net/cfp.html).
I think it would be really cool to submit a talk about IoTDB, especially as 
there is a special subtrack “IoT” where we will try to represent PLC4X and 
Edgent (both also incubating).
Has anybody here plans to go to Las Vegas?

Julian

AW: IoTDB Logo Design

[Julian Feinauer]

Hi xiangdong,

I think we should do a formal vote for accepting the logo.

Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Re: IoTDB Logo Design
Von: Xiangdong Huang
An: dev@iotdb.apache.org
Cc:

Hi,

Now 7 persons showed their opinions and 6 of them voted +1. One person
proposed different opinion.

> The podling will at some point need to do a name search

I would like to know what is the next step?

Thanks,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Justin Mclean  于2019年3月12日周二 上午8:11写道：

> Hi,
>
> The podling will at some point need to do a name search now IoTDB is a
> very generic name and it may be that some other company is using it so it
> may be a good idea to do this sooner rather than later as we may find out
> we need to change the name.
>
> Thanks,
> Justin

AW: IoTDB Logo Design

[Julian Feinauer]

Cool!

Von meinem Mobiltelefon gesendet

 Ursprüngliche Nachricht 
Betreff: Re: IoTDB Logo Design
Von: 徐毅
An: dev@iotdb.apache.org
Cc:

Hi,


It looks good, I hope we can add this logo to our website as soon as possible.


Thanks
XuYi
On 3/11/2019 13:35，Xiangdong Huang wrote：
+1! I have been looking forward to it for a long time. :P

Best,

---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院


Stefanie Zhao  于2019年3月11日周一 上午11:21写道：


Sorry, the direct link is dropped by the email engine. The jira link is:
https://issues.apache.org/jira/browse/IOTDB-42.


--

赵馨逸
清华大学软件学院
信息系统与工程研究所
Xinyi Zhao(Stefanie)
School of Software, Tsinghua University
E-mail:stefanie_...@163.com



At 2019-03-11 11:07:33, "Stefanie Zhao"  wrote:
Dear all,


We are glad to see that IoTDB is getting more and more popular in the
community. To make it a better product, it is time to design a logo for it.
A Jira is opened and a draft design is attached in it. Any comments and
suggestions are welcome.


Best,
―
Xinyi Zhao(Stefanie)
School of Software, Tsinghua University
E-mail:stefanie_...@163.com

AW: Do we have a plan to release the first normal version of IoTDB this month?

[Julian Feinauer]

Hi Chris,

I think there is slight confusion. Currently iotdb has no develop and only 
master branch where development happens.
Thus I suggest to create the file there (as done in my PR #91).

Julian

Von meinem Mobiltelefon gesendet


 Urspr??ngliche Nachricht 
Betreff: Re: Do we have a plan to release the first normal version of IoTDB 
this month?
Von: Christofer Dutz
An: dev@iotdb.apache.org
Cc:

In other projects, we usually have release notes also in develop. Here we have 
a section for the next planned version right at the top (but with an 
"unreleased" appended to the version) and add stuff as it comes in. For a 
release the "unreleased" is removed and merged to master, then a new entry for 
the next version is added.

Chris

Outlook f??r Android<https://aka.ms/ghei36> herunterladen


From:  Sheng Wu 
Sent: Sunday, March 10, 2019 4:53:00 AM
To: dev; dev
Subject: Re: Do we have a plan to release the first normal version of IoTDB 
this month?

Hi.


Release note should be kept in master or further codes, I think. Users may be 
interesting in the changes of old versions, in order to choose right version.
And, make sense to create a new branch to do release, and versiob update. You 
could make the version to 0.9.0-snapshot, after the release branch created.



Sheng Wu
Apache SkyWalking, ShardingSphere, Zipkin

>From Wu Sheng 's phone.


-- Original --
From: Xiangdong Huang 
Date: Sun,Mar 10,2019 11:48 AM
To: dev 
Subject: Re: Do we have a plan to release the first normal version of IoTDB 
this month?



Hi,

Do we put this release note into the master branch, or create a new branch
called branch-0.8.0 and put this note into that branch?
(In the pom file of the master branch, the  version is still 0.8.0-SNAPSHOT)

Best,

---
Xiangdong Huang
School of Software, Tsinghua University

 ??
???? 


Julian Feinauer  ??2019??3??10?? 
4:07??

> Hi all,
>
> I just prepared the initial version of the RELEASE_NOTES as PR in [1].
> I just integrated all closed Jira issues.
> Now we should try to add all further (mentionable) improvements.
> Personally, I think this is not to crucial as this is the first apache
> release, so the question "what changed" or what changed incompatible is not
> that crucial as for further releases.
>
> Best
> Julian
>
> https://github.com/apache/incubator-iotdb/pull/91
>
> ?1?3Am 09.03.19, 11:48 schrieb "Justin Mclean" :
>
> Hi,
>
> > In brief, there are four steps: (1) setting Maven configuration; (2)
> > creating the release branch and prepare the codebase; (3) mvn
> release and
> > deploy and so on; (4) vote.
>
> You also need to check the LICENSE, NOTICE and DISCLAIMER files in the
> release and we can help you with that.
>
> Thanks,
> Justin
>
>

AW: Do we have a plan to release the first normal version of IoTDB this month?

[Julian Feinauer]

Hi,

I think a (first apache) release is a good idea. Especially to allow people to 
play around with the artifacts.

After what I have seen the code quality is good and the main functionality 
works well.

Julian


 Urspr??ngliche Nachricht 
Betreff: Re: Do we have a plan to release the first normal version of IoTDB 
this month?
Von:  Sheng Wu
An: dev ,dev
Cc:

I think we should try to do a release. At least a preview version. 3 months are 
not short time for a new project.
Of source, no rush, just when you think it is ready and make sense.



Sheng Wu
Apache SkyWalking, ShardingSphere, Zipkin

>From Wu Sheng 's phone.


-- Original --
From: yi xu 
Date: Fri,Mar 8,2019 11:09 AM
To: dev@iotdb.apache.org 
Subject: Re: Do we have a plan to release the first normal version of IoTDB 
this month?



Hi,

Over the last three months, we have improved IoTDB in several ways, such as 
code quality, document and read/write performance. So should we release a 
normal version to our users since we don??t have a normal version right now.

Thanks
XuYi

Re: Operation and robustness of iotDB

[Julian Feinauer]

Hey Xu Yi,

thanks fort he information.
I checked the code and indeed I was wrong.
Every Chunk also stores its timestamp.

So when I read values through a Query all timestamps are "interpolated" or 
merged together from all sensors, or?

Julian

Am 07.03.19, 18:48 schrieb "Xu yi" :

Hi,

In my opinion, different measurements use their own timestamp even though 
they are grouped into one chunk group.they don’t share from each other.

What do you think of this @xiangdong 

Thanks 
XuYi 

iPhoneから送信

2019/03/08 1:41、Julian Feinauer のメール:

> Hi,
> 
> Yes this is what I meant.
> 
> Julian
> 
> Von meinem Mobiltelefon gesendet
> 
> 
>  Ursprüngliche Nachricht 
> Betreff: Re: Operation and robustness of iotDB
> Von: 徐毅
> An: dev@iotdb.apache.org
> Cc:
> 
> Hi,
> In the definition of ChunkGroup, what is the meaning of 'share one time 
signal'? Do these measurements share same timestamps?
> 
> 
> Thanks
> XuYi
> On 3/8/2019 01:11，Julian Feinauer wrote：
> Hey Xiangdong,
> hey all,
> 
> I like the documentation much.
> The only thing I'm a bit unsure is about the names (as there is no 
clarification).
> So, before I update it with any wrong information I would like to ensure 
that I have the correct understanding.
> 
> I assume that most naming is similar to Parquet.
> 
> Page - Contains one Measurement, smallest source of compression
> Chunk - Collection of multiple Pages, still one measurement
> ChunkGroup - Collection of chunks of which share one time signal (one 
Chunk for each measurement)
> 
> Is this correct so?
> 
> Julian
> 
> Am 05.03.19, 12:26 schrieb "Xiangdong Huang" :
> 
> Hi,
> 
> 1. We have a document to introduce that:
> https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format
> 
> 2. The new API for recovering data is almost done. I am writing the UTs
> now. Maybe I can submit a PR tonight (if everything is fine...)
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
> 黄向东
> 清华大学 软件学院
> 
> 
> Julian Feinauer  于2019年3月5日周二 下午6:00写道：
> 
> Hi Xiangdong,
> 
> that sounds excellent.
> Do you have a short overview of how the file format is designed on disk?
> I know that its somewhat similar to parquet but I did not find more
> details.
> Basically what would suffice for us would be something like skipping an
> invalid column group (or how you name it) and go on with the next, or so.
> 
> Julian
> 
> Am 04.03.19, 13:21 schrieb "Xiangdong Huang" :
> 
> Hi,
> 
> If so, I think I need to add a new API to allow you continue to write
> data
> in an existing  but not closed correctly TsFile. Then everything is
> fine
> for you :D
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
> 黄向东
> 清华大学 软件学院
> 
> 
> Julian Feinauer  于2019年3月4日周一 下午8:08写道：
> 
> Hey Xiangdong,
> 
> thanks for the great explanation.
> And in fact, I agree with you that it would be best if we start to
> play
> around with it and reply all our findings or wishes back to this
> list (in
> fact that proved to be beneficial in plc4x as well).
> 
> You confirm my thoughts about the two "levels" of APIs (DB and file)
> and
> the file api is exactly what we looked for for our use case.
> As we do not care much about data loss (when an edge device fails
> its...
> gone).
> The crucial point for us is that no corrupt files can be generated.
> This means I'm fine when the last data submitted is lost but I'm not
> fine
> if we can get to a situation where the last datafile is completely
> lost
> (well, perhaps this could be acceptable).
> 
> @tim: Perhaps its best when you give some more information to
> Xiangdong
> about our idea, and we can also point to our current code in github
> 
> Julian
> 
> Am 04.03.19, 13:03 schrieb "Xiangdong Huang" :
> 
> Hi,
> 
> TsFile API is not deprecated. In fact, it is designed for this
> scenario and
> MapReduce/Spark computing.
> 
> If you just use Reader and Writer API, th

AW: Operation and robustness of iotDB

[Julian Feinauer]

Hi,

Yes this is what I meant.

Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Re: Operation and robustness of iotDB
Von: 徐毅
An: dev@iotdb.apache.org
Cc:

Hi,
In the definition of ChunkGroup, what is the meaning of 'share one time 
signal'? Do these measurements share same timestamps?


Thanks
XuYi
On 3/8/2019 01:11，Julian Feinauer wrote：
Hey Xiangdong,
hey all,

I like the documentation much.
The only thing I'm a bit unsure is about the names (as there is no 
clarification).
So, before I update it with any wrong information I would like to ensure that I 
have the correct understanding.

I assume that most naming is similar to Parquet.

Page - Contains one Measurement, smallest source of compression
Chunk - Collection of multiple Pages, still one measurement
ChunkGroup - Collection of chunks of which share one time signal (one Chunk for 
each measurement)

Is this correct so?

Julian

Am 05.03.19, 12:26 schrieb "Xiangdong Huang" :

Hi,

1. We have a document to introduce that:
https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format

2. The new API for recovering data is almost done. I am writing the UTs
now. Maybe I can submit a PR tonight (if everything is fine...)

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院


Julian Feinauer  于2019年3月5日周二 下午6:00写道：

Hi Xiangdong,

that sounds excellent.
Do you have a short overview of how the file format is designed on disk?
I know that its somewhat similar to parquet but I did not find more
details.
Basically what would suffice for us would be something like skipping an
invalid column group (or how you name it) and go on with the next, or so.

Julian

Am 04.03.19, 13:21 schrieb "Xiangdong Huang" :

Hi,

If so, I think I need to add a new API to allow you continue to write
data
in an existing  but not closed correctly TsFile. Then everything is
fine
for you :D

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院


Julian Feinauer  于2019年3月4日周一 下午8:08写道：

Hey Xiangdong,

thanks for the great explanation.
And in fact, I agree with you that it would be best if we start to
play
around with it and reply all our findings or wishes back to this
list (in
fact that proved to be beneficial in plc4x as well).

You confirm my thoughts about the two "levels" of APIs (DB and file)
and
the file api is exactly what we looked for for our use case.
As we do not care much about data loss (when an edge device fails
its...
gone).
The crucial point for us is that no corrupt files can be generated.
This means I'm fine when the last data submitted is lost but I'm not
fine
if we can get to a situation where the last datafile is completely
lost
(well, perhaps this could be acceptable).

@tim: Perhaps its best when you give some more information to
Xiangdong
about our idea, and we can also point to our current code in github

Julian

Am 04.03.19, 13:03 schrieb "Xiangdong Huang" :

Hi,

TsFile API is not deprecated. In fact, it is designed for this
scenario and
MapReduce/Spark computing.

If you just use Reader and Writer API, there is something you
need to
know:

Let's suppose your block size is x Bytes,
(tsfile-format.properties:
group_size_in_byte).

1. If you write data and a shutdown occurs, then all data that is
flushed
on disk is ok, and you can read the data ( class
org.apache.iotdb.tsfile.TsFileSequenceRead is an example, but
you need
to
change it a little. I think I can write an example.)

2. Actually, TsFile has the ability to allow you continue to
write
data at
the end of the incomplete file. However, We do not provide this
API
now...
If needed, I can add the API.

3. In this scenario, you will lose at most x Bytes data. If you
do not
accept that, something like WAL is needed. (It is not very
complex,
but I
am not sure that whether it should be an embedded function for
TsFile).

Up to now, we can consider that TsFile API is suitable for your
scenario
(even though we need to add a little more API if you desire).
And you
can
get the ability to compress data, and query data from the TsFile
rather
than scan the data from the head to the tail.

However, TsFile has one constraint: You can not write
out-of-order data
into a TsFile, otherwise the query API may return incomplete
result.
But I think it is ok for real applications, because I do not
think
that a
device can generate out-of-order data

For example, If you write two devices' data into one TsFile, it
is ok
if
you write data like:
- d1.t1, d1.t2, d2.t1, d2.t2, d2.t3, d1.t4, d1.t5 
or:
- d1.m1.t1, d1.m1.t2, d1.m2.t1, d1.m2.t2, d2.m1.t1 ...

But you can not write data like:
- d1.m1.t2, d1.m1.t1 ...

I think it is a good chance to improve TsFile to make it more
suitable
for
real applications, so please do not hesitate to tell me more
about
what you
think TsFile should want to have?

Best,
--

Re: Operation and robustness of iotDB

[Julian Feinauer]

Hey Xiangdong,
hey all,

I like the documentation much.
The only thing I'm a bit unsure is about the names (as there is no 
clarification).
So, before I update it with any wrong information I would like to ensure that I 
have the correct understanding.

I assume that most naming is similar to Parquet.

Page - Contains one Measurement, smallest source of compression
Chunk - Collection of multiple Pages, still one measurement
ChunkGroup - Collection of chunks of which share one time signal (one Chunk for 
each measurement)

Is this correct so?

Julian

Am 05.03.19, 12:26 schrieb "Xiangdong Huang" :

Hi,

1. We have a document to introduce that:
https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format

2. The new API for recovering data is almost done. I am writing the UTs
now. Maybe I can submit a PR tonight (if everything is fine...)

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年3月5日周二 下午6:00写道：

> Hi Xiangdong,
>
> that sounds excellent.
> Do you have a short overview of how the file format is designed on disk?
> I know that its somewhat similar to parquet but I did not find more
> details.
> Basically what would suffice for us would be something like skipping an
> invalid column group (or how you name it) and go on with the next, or so.
>
> Julian
>
> Am 04.03.19, 13:21 schrieb "Xiangdong Huang" :
>
> Hi,
>
> If so, I think I need to add a new API to allow you continue to write
> data
> in an existing  but not closed correctly TsFile. Then everything is
> fine
> for you :D
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年3月4日周一 下午8:08写道：
>
> > Hey Xiangdong,
> >
> > thanks for the great explanation.
> > And in fact, I agree with you that it would be best if we start to
> play
> > around with it and reply all our findings or wishes back to this
> list (in
> > fact that proved to be beneficial in plc4x as well).
> >
> > You confirm my thoughts about the two "levels" of APIs (DB and file)
> and
> > the file api is exactly what we looked for for our use case.
> > As we do not care much about data loss (when an edge device fails
> its...
> > gone).
> > The crucial point for us is that no corrupt files can be generated.
> > This means I'm fine when the last data submitted is lost but I'm not
> fine
> > if we can get to a situation where the last datafile is completely
> lost
> > (well, perhaps this could be acceptable).
> >
> > @tim: Perhaps its best when you give some more information to
> Xiangdong
> > about our idea, and we can also point to our current code in github
> >
> > Julian
> >
> > Am 04.03.19, 13:03 schrieb "Xiangdong Huang" :
> >
> > Hi,
> >
> > TsFile API is not deprecated. In fact, it is designed for this
> > scenario and
> > MapReduce/Spark computing.
> >
> > If you just use Reader and Writer API, there is something you
> need to
> > know:
> >
> > Let's suppose your block size is x Bytes,
> (tsfile-format.properties:
> > group_size_in_byte).
> >
> > 1. If you write data and a shutdown occurs, then all data that 
is
> > flushed
> > on disk is ok, and you can read the data ( class
> > org.apache.iotdb.tsfile.TsFileSequenceRead is an example, but
> you need
> > to
> > change it a little. I think I can write an example.)
> >
> > 2. Actually, TsFile has the ability to allow you continue to
> write
> > data at
> > the end of the incomplete file. However, We do not provide this
> API
> > now...
> > If needed, I can add the API.
> >
> > 3. In this scenario, you will lose at most x Bytes data. If you
> do not
> > accept that, something like WAL is needed. (It is not very
&g

Re: [tsfile] [query] path for device.measurement that contain dots

[Julian Feinauer]

Hi Xiangdong,

thanks for the clarification, indeed, then I was wrong.
In the light of that, I'm unsure whether I like the idea to allow dots for the 
names or if these should keep forbidden.
This makes it (visibily) clear what part is the measurement and what is the 
"path".

Julian

Am 07.03.19, 10:04 schrieb "Xiangdong Huang" :

Hi,

@Julian I think we support what you want now..

for example, you can create 3 timeseries by:
```
set storage group to root.sg1;
create timeseries root.sg1.location1.deviceType1.deviceID.m1 with
datatype=FLOAT, encoding=RLE;
create timeseries root.sg1.location1.deviceType2.deviceID.m1 with
datatype=FLOAT, encoding=RLE;
create timeseries root.sg1.location2.deviceType1.deviceID.m1 with
datatype=FLOAT, encoding=RLE;
```
Then, if you run:
`select m1 from root.sg1.*.deviceType1.deviceID`
or
`select deviceType1.deviceID.m1 from root.*`

you will get the data of  `root.sg1.location1.deviceType1.deviceID.m1`  and
`root.sg1.location2.deviceType1.deviceID.m1`

Actually we manage the metadata of measurements (rather than each
timeseries) on each storage group, for reduce memory cost and convert the
schema to a table.
( That is, you can not create a timeseries in sg1 storage group like:
```
create timeseries root.sg1.location1.deviceType3.deviceID.m1 with
datatype=INT32, encoding=RLE;
```
In this example, `m1` is defined as INT32, but it has been claimed as FLOAT
in timeseries  `root.sg1.location1.deviceType1.deviceID.m1`
But, if you create a new storage group `sg2`, you can define `m1` in `sg2`
as INT32 while `m1` in `sg1` is FLOAT)

So, it is important for IoTDB to recognize which is the measurement name
from a path. @Marko, that is why I said add a dot in the measurement will
cause many modifications.
(Maybe not to many... we just need to let users claim which part of a path
is a measurement name, and record it on disk, rather than just split the
path with a dot symbol and use the last substring).

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院
    

Julian Feinauer  于2019年3月7日周四 下午4:03写道：

> Hi Xiangdong,
>
> what do you think of a subtile generalization of the current "device"
> approach.
> If we would allow dots in the name, we could the possibility to create
> "trees" like its done e.g. for akka actors [1] or MQTT Topics [2].
> What I mean is that one could add more information in the "path" of a
> device to allow for filtering.
> Think of the three devices
>
> "location1.deviceType1.deviceId"
> "location1.deviceType2.deviceId"
> "location2.deviceType1.deviceId"
>
> Then I gould ask for:
>
> "all devices from location 1" with "location.*
> "all devices of type 2" with "%.deviceType2.%"
> "all devices" with "*"
>
> (where * means match multiple "hierarchies" and "%" menas match one
> hierarchy, a bit how MQTT does it with # and *).
>
> I think this would be really cool, to also allow these "device paths" for
> queries.
> What do you think?
>
> Julian
>
> [1] https://doc.akka.io/docs/akka/current/general/addressing.html
> [2]
> 
https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/
>
> Am 07.03.19, 05:40 schrieb "Xiangdong Huang" :
>
> Hi,
>
> It seems this will cause a big modification... (Change the Path class
> is
> easy, but it will lead to a confusion for SQL layer..) A solution is
> needed
> to let SQL recognize that.
>
> I have created a issue on JIRA:
> https://issues.apache.org/jira/browse/IOTDB-38
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Marko Friedemann  于2019年3月7日周四 上午4:06写道：
>
> > Hey,
> >
> > I want to report an issue with tsfile, specifically its query
> capabilities.
> >
> > The two-argument constructor for
> org.apache.iotdb.tsfile.read.common.Path
> > that is now available in the apache incubator project still does not
> allow
> > proper construction of a path where the measurement name contains
> dots.
> >
> 

Re: [tsfile] [query] path for device.measurement that contain dots

[Julian Feinauer]

Hi Xiangdong,

what do you think of a subtile generalization of the current "device" approach.
If we would allow dots in the name, we could the possibility to create "trees" 
like its done e.g. for akka actors [1] or MQTT Topics [2].
What I mean is that one could add more information in the "path" of a device to 
allow for filtering.
Think of the three devices

"location1.deviceType1.deviceId"
"location1.deviceType2.deviceId"
"location2.deviceType1.deviceId"

Then I gould ask for:

"all devices from location 1" with "location.*
"all devices of type 2" with "%.deviceType2.%"
"all devices" with "*"

(where * means match multiple "hierarchies" and "%" menas match one hierarchy, 
a bit how MQTT does it with # and *).

I think this would be really cool, to also allow these "device paths" for 
queries.
What do you think?

Julian

[1] https://doc.akka.io/docs/akka/current/general/addressing.html
[2] 
https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/

Am 07.03.19, 05:40 schrieb "Xiangdong Huang" :

Hi,

It seems this will cause a big modification... (Change the Path class is
easy, but it will lead to a confusion for SQL layer..) A solution is needed
to let SQL recognize that.

I have created a issue on JIRA:
https://issues.apache.org/jira/browse/IOTDB-38

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Marko Friedemann  于2019年3月7日周四 上午4:06写道：

> Hey,
>
> I want to report an issue with tsfile, specifically its query 
capabilities.
>
> The two-argument constructor for org.apache.iotdb.tsfile.read.common.Path
> that is now available in the apache incubator project still does not allow
> proper construction of a path where the measurement name contains dots.
>
> Specifically, the issue is that the two-argument constructor concatenates
> the two arguments with a path-separator character (the dot) and then 
splits
> the result again, using the seperator (the dot), instead of just using the
> supplied arguments as they are.
> This results in the path components being incorrect (device basically runs
> to lastIndexOf('.') for the full path) and the query failing.
>
> The rest of tsfile's write/read/query functionality doesn't seem to mind
> measurement names that contain dots (think RDF and IRIs) and the intended
> query can be run with a minor fix by sub-classing the Path class.
> (Unfortunately, Path::init() is also private, so the work-around is not
> readily possible.)
>
> Regards,
> Marko Friedemann
>

[DISCUSS] Send Jira Emails to the dev@ list

[Julian Feinauer]

Hi all,

I just noticed, that currently there is no notification or so for Jira Issues 
to the dev@ list.
Just recently we added notifications of new Jira Tickets to the dev@ list for 
the PLC4X (incubating) Project and I like it quite a lot.
It helps the devs to have “an eye” on what’s happening in Jira in a very easy 
way.

What do you think of that?

Julian

AW: Operation and robustness of iotDB

[Julian Feinauer]

Hi Xiangdong,

Great work!
I'll try to go through your code to understand the internals better and perhaps 
do a simulation test with hard jvm exits.
If everything goes fine we will incorporate that asap in test code and give you 
feedback!

Thank you!
Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Re: Operation and robustness of iotDB
Von: Xiangdong Huang
An: dev@iotdb.apache.org
Cc:

Hi,

I have added a new TsFileIOWriter, which supports to recover data from a
broken TsFile (I mean, a TsFile that is not closed correctly). The PR is
https://github.com/apache/incubator-iotdb/pull/87.

You can use the new feature by:
```
NativeRestorableIOWriter rWriter = new NativeRestorableIOWriter(file);
TsFileWriter writer = new TsFileWriter(rWriter);
```
1. If the file is a complete TsFile (e.g., the FileMetadata has been
persistent into the file and the tail magic string is correct), then you
can not write data into this TsFile anymore (actually I can remove all
FileMetadata and then let you continue to write, but it needs more coding
work...), `TsFileWriter writer = new TsFileWriter(rWriter);` will throw a
IoException. You can check it before you create TsFileWriter by using
`rWriter.canWrite()`.

2. ChunkGroup is the basic unit that can be recover. That is, if a
ChunkGroup is complete, i.e., its chunkGroupFooter is complete, I will
remain it, otherwise I will truncate the chunk group. However, I can not
know whether a chunkGroupFooter is complete (if only one byte lose, the
deserialize method does not throw exception, because  something like
`deserializeToInt(inputstream)` does not throw exception (it will use 3
bytes to convert them to a integer).  So, when I say "if a ChunkGroup is
complete", I mean that not only its chunkGroupFooter  is serialized
successfully, but also there is at least one more byte is serialized into
the file followed the footer.

3. I have tested many cases, such only a magic head string is persisted,
only a ChunkHeader is persisted,  the first ChunkHeader is not persisted
completely, only some Chunks are persisted, a ChunkGroupFooter is
persisted, Two ChunkGroupFooter are persisted, and a complete TsFile is
persisted, etc..  (`NativeRestorableIOWriterTest.java`, hope I have cover
all cases)

Hope TsFile can be used in edge device management applications ASAP!

Now please enjoy the new feature.

---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Julian Feinauer  于2019年3月5日周二 下午8:10写道：

> Hey,
>
> thank you fort he link... I did not know of this.. this is exactly what I
> was looking for!
>
> Julian
>
> PS.: Looking forward to your PR : )
>
> Am 05.03.19, 12:26 schrieb "Xiangdong Huang" :
>
> Hi,
>
> 1. We have a document to introduce that:
> https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format
>
> 2. The new API for recovering data is almost done. I am writing the UTs
> now. Maybe I can submit a PR tonight (if everything is fine...)
>
> Best,
> ---
> Xiangdong Huang
>     School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年3月5日周二 下午6:00写道：
>
> > Hi Xiangdong,
> >
> > that sounds excellent.
> > Do you have a short overview of how the file format is designed on
> disk?
> > I know that its somewhat similar to parquet but I did not find more
> > details.
> > Basically what would suffice for us would be something like skipping
> an
> > invalid column group (or how you name it) and go on with the next,
> or so.
> >
> > Julian
> >
> > Am 04.03.19, 13:21 schrieb "Xiangdong Huang" :
> >
> > Hi,
> >
> > If so, I think I need to add a new API to allow you continue to
> write
> > data
> > in an existing  but not closed correctly TsFile. Then everything
> is
> > fine
> > for you :D
> >
> > Best,
> > ---
> > Xiangdong Huang
> > School of Software, Tsinghua University
> >
> >  黄向东
> > 清华大学 软件学院
> >
> >
> > Julian Feinauer  于2019年3月4日周一
> 下午8:08写道：
> >
> > > Hey Xiangdong,
> > >
> > > thanks for the great explanation.
> > > And in fact, I agree with you that it would be best if we
> start to
> > play
> > > around with it and reply all our findings or wishes back to
> this
> > list (in
> > > fact that proved to be

Re: Operation and robustness of iotDB

[Julian Feinauer]

Hey,

thank you fort he link... I did not know of this.. this is exactly what I was 
looking for!

Julian

PS.: Looking forward to your PR : )

Am 05.03.19, 12:26 schrieb "Xiangdong Huang" :

Hi,

1. We have a document to introduce that:
https://cwiki.apache.org/confluence/display/IOTDB/TsFile+Format

2. The new API for recovering data is almost done. I am writing the UTs
now. Maybe I can submit a PR tonight (if everything is fine...)

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年3月5日周二 下午6:00写道：

> Hi Xiangdong,
>
> that sounds excellent.
> Do you have a short overview of how the file format is designed on disk?
> I know that its somewhat similar to parquet but I did not find more
> details.
> Basically what would suffice for us would be something like skipping an
> invalid column group (or how you name it) and go on with the next, or so.
>
> Julian
>
> Am 04.03.19, 13:21 schrieb "Xiangdong Huang" :
>
> Hi,
>
> If so, I think I need to add a new API to allow you continue to write
> data
> in an existing  but not closed correctly TsFile. Then everything is
> fine
> for you :D
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年3月4日周一 下午8:08写道：
>
> > Hey Xiangdong,
> >
> > thanks for the great explanation.
> > And in fact, I agree with you that it would be best if we start to
> play
> > around with it and reply all our findings or wishes back to this
> list (in
> > fact that proved to be beneficial in plc4x as well).
> >
> > You confirm my thoughts about the two "levels" of APIs (DB and file)
> and
> > the file api is exactly what we looked for for our use case.
> > As we do not care much about data loss (when an edge device fails
> its...
> > gone).
> > The crucial point for us is that no corrupt files can be generated.
> > This means I'm fine when the last data submitted is lost but I'm not
> fine
> > if we can get to a situation where the last datafile is completely
> lost
> > (well, perhaps this could be acceptable).
> >
> > @tim: Perhaps its best when you give some more information to
> Xiangdong
> > about our idea, and we can also point to our current code in github
> >
> > Julian
> >
> > Am 04.03.19, 13:03 schrieb "Xiangdong Huang" :
> >
> > Hi,
> >
> > TsFile API is not deprecated. In fact, it is designed for this
> > scenario and
> > MapReduce/Spark computing.
> >
> > If you just use Reader and Writer API, there is something you
> need to
> > know:
> >
> > Let's suppose your block size is x Bytes,
> (tsfile-format.properties:
> > group_size_in_byte).
> >
> > 1. If you write data and a shutdown occurs, then all data that 
is
> > flushed
> > on disk is ok, and you can read the data ( class
> > org.apache.iotdb.tsfile.TsFileSequenceRead is an example, but
> you need
> > to
> > change it a little. I think I can write an example.)
> >
> > 2. Actually, TsFile has the ability to allow you continue to
> write
> > data at
> > the end of the incomplete file. However, We do not provide this
> API
> > now...
> > If needed, I can add the API.
> >
> > 3. In this scenario, you will lose at most x Bytes data. If you
> do not
> > accept that, something like WAL is needed. (It is not very
> complex,
> > but I
> > am not sure that whether it should be an embedded function for
> TsFile).
> >
> > Up to now, we can consider that TsFile API is suitable for your
> > scenario
> > (even though we need to add a little more API if you desire).
> And you
>  

Re: Operation and robustness of iotDB

[Julian Feinauer]

Hi Xiangdong,

that sounds excellent.
Do you have a short overview of how the file format is designed on disk?
I know that its somewhat similar to parquet but I did not find more details.
Basically what would suffice for us would be something like skipping an invalid 
column group (or how you name it) and go on with the next, or so.

Julian

Am 04.03.19, 13:21 schrieb "Xiangdong Huang" :

Hi,

If so, I think I need to add a new API to allow you continue to write data
in an existing  but not closed correctly TsFile. Then everything is fine
for you :D

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年3月4日周一 下午8:08写道：

> Hey Xiangdong,
>
> thanks for the great explanation.
> And in fact, I agree with you that it would be best if we start to play
> around with it and reply all our findings or wishes back to this list (in
> fact that proved to be beneficial in plc4x as well).
>
> You confirm my thoughts about the two "levels" of APIs (DB and file) and
> the file api is exactly what we looked for for our use case.
> As we do not care much about data loss (when an edge device fails its...
> gone).
> The crucial point for us is that no corrupt files can be generated.
> This means I'm fine when the last data submitted is lost but I'm not fine
> if we can get to a situation where the last datafile is completely lost
> (well, perhaps this could be acceptable).
>
> @tim: Perhaps its best when you give some more information to Xiangdong
> about our idea, and we can also point to our current code in github
>
> Julian
>
> Am 04.03.19, 13:03 schrieb "Xiangdong Huang" :
>
> Hi,
>
> TsFile API is not deprecated. In fact, it is designed for this
> scenario and
> MapReduce/Spark computing.
>
> If you just use Reader and Writer API, there is something you need to
> know:
>
> Let's suppose your block size is x Bytes, (tsfile-format.properties:
> group_size_in_byte).
>
> 1. If you write data and a shutdown occurs, then all data that is
> flushed
> on disk is ok, and you can read the data ( class
> org.apache.iotdb.tsfile.TsFileSequenceRead is an example, but you need
> to
> change it a little. I think I can write an example.)
>
> 2. Actually, TsFile has the ability to allow you continue to write
> data at
> the end of the incomplete file. However, We do not provide this API
> now...
> If needed, I can add the API.
>
> 3. In this scenario, you will lose at most x Bytes data. If you do not
> accept that, something like WAL is needed. (It is not very complex,
> but I
> am not sure that whether it should be an embedded function for 
TsFile).
>
> Up to now, we can consider that TsFile API is suitable for your
> scenario
> (even though we need to add a little more API if you desire). And you
> can
> get the ability to compress data, and query data from the TsFile 
rather
> than scan the data from the head to the tail.
>
> However, TsFile has one constraint: You can not write out-of-order 
data
> into a TsFile, otherwise the query API may return incomplete result.
> But I think it is ok for real applications, because I do not think
> that a
> device can generate out-of-order data
>
> For example, If you write two devices' data into one TsFile, it is ok
> if
> you write data like:
> - d1.t1, d1.t2, d2.t1, d2.t2, d2.t3, d1.t4, d1.t5 
> or:
> - d1.m1.t1, d1.m1.t2, d1.m2.t1, d1.m2.t2, d2.m1.t1 ...
>
> But you can not write data like:
> - d1.m1.t2, d1.m1.t1 ...
>
> I think it is a good chance to improve TsFile to make it more suitable
> for
> real applications, so please do not hesitate to tell me more about
> what you
> think TsFile should want to have?
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
>  黄向东
> 清华大学 软件学院
>
>
> Julian Feinauer  于2019年3月4日周一 下午7:17写道：
>
> > Hi Xiangdong,
> >
> > thanks for the info.
> > How is it in the case when you use the Reader / Writer API for the
> tsfiles
> > dir

Re: Operation and robustness of iotDB

[Julian Feinauer]

Hey Xiangdong,

thanks for the great explanation.
And in fact, I agree with you that it would be best if we start to play around 
with it and reply all our findings or wishes back to this list (in fact that 
proved to be beneficial in plc4x as well).
 
You confirm my thoughts about the two "levels" of APIs (DB and file) and the 
file api is exactly what we looked for for our use case.
As we do not care much about data loss (when an edge device fails its... gone).
The crucial point for us is that no corrupt files can be generated. 
This means I'm fine when the last data submitted is lost but I'm not fine if we 
can get to a situation where the last datafile is completely lost (well, 
perhaps this could be acceptable).

@tim: Perhaps its best when you give some more information to Xiangdong about 
our idea, and we can also point to our current code in github

Julian

Am 04.03.19, 13:03 schrieb "Xiangdong Huang" :

Hi,

TsFile API is not deprecated. In fact, it is designed for this scenario and
MapReduce/Spark computing.

If you just use Reader and Writer API, there is something you need to know:

Let's suppose your block size is x Bytes, (tsfile-format.properties:
group_size_in_byte).

1. If you write data and a shutdown occurs, then all data that is flushed
on disk is ok, and you can read the data ( class
org.apache.iotdb.tsfile.TsFileSequenceRead is an example, but you need to
change it a little. I think I can write an example.)

2. Actually, TsFile has the ability to allow you continue to write data at
the end of the incomplete file. However, We do not provide this API now...
If needed, I can add the API.

3. In this scenario, you will lose at most x Bytes data. If you do not
accept that, something like WAL is needed. (It is not very complex, but I
am not sure that whether it should be an embedded function for TsFile).

Up to now, we can consider that TsFile API is suitable for your scenario
(even though we need to add a little more API if you desire). And you can
get the ability to compress data, and query data from the TsFile rather
than scan the data from the head to the tail.

However, TsFile has one constraint: You can not write out-of-order data
into a TsFile, otherwise the query API may return incomplete result.
But I think it is ok for real applications, because I do not think that a
device can generate out-of-order data

For example, If you write two devices' data into one TsFile, it is ok if
you write data like:
- d1.t1, d1.t2, d2.t1, d2.t2, d2.t3, d1.t4, d1.t5 
or:
- d1.m1.t1, d1.m1.t2, d1.m2.t1, d1.m2.t2, d2.m1.t1 ...

But you can not write data like:
- d1.m1.t2, d1.m1.t1 ...

I think it is a good chance to improve TsFile to make it more suitable for
real applications, so please do not hesitate to tell me more about what you
think TsFile should want to have?

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院
    

Julian Feinauer  于2019年3月4日周一 下午7:17写道：

> Hi Xiangdong,
>
> thanks for the info.
> How is it in the case when you use the Reader / Writer API for the tsfiles
> directly (or should this be considered "deprecated")?
> Can these files come to corrupted state?
>
> One Situation where we have to deal with these situations is "at the edge"
> when we have devices inside large machines.
> Usually at the end of the shift these machines (and therefore our device)
> is powered off hard, so no shutdown or de-initialization is possible.
>
> Best
> Julian
>
> Am 04.03.19, 12:14 schrieb "Xiangdong Huang" :
>
> Hi,
>
> IoTDB can support either on a server with 7*24 or a RaspberryPi. We
> have
> tested both the two scenario.
>
> When you shutdown an IoTDB instance in force (e.g., power off) and
> restart
> it again, no data loses ( if you enable the WAL).
>
> However, currently we do not optimize the time cost of the restart
> process.
> It is an important feature that we need to do, because we hope IoTDB
> can
> support data management either on the edge devices or the data center.
>
> And, the default configuration is not so suitable for running on the
> edge
> device. (e.g., block size is 128MB, which is too large for a
> RaspberryPi,
> and will slow down the restart process because there are too much WAL
> data
> on disk).
>
> Best,
> ---
> Xiangdong Huang
 

Re: Operation and robustness of iotDB

[Julian Feinauer]

Hi Xiangdong,

thanks for the info.
How is it in the case when you use the Reader / Writer API for the tsfiles 
directly (or should this be considered "deprecated")?
Can these files come to corrupted state?

One Situation where we have to deal with these situations is "at the edge" when 
we have devices inside large machines.
Usually at the end of the shift these machines (and therefore our device) is 
powered off hard, so no shutdown or de-initialization is possible.

Best
Julian

Am 04.03.19, 12:14 schrieb "Xiangdong Huang" :

Hi,

IoTDB can support either on a server with 7*24 or a RaspberryPi. We have
tested both the two scenario.

When you shutdown an IoTDB instance in force (e.g., power off) and restart
it again, no data loses ( if you enable the WAL).

However, currently we do not optimize the time cost of the restart process.
It is an important feature that we need to do, because we hope IoTDB can
support data management either on the edge devices or the data center.

And, the default configuration is not so suitable for running on the edge
device. (e.g., block size is 128MB, which is too large for a RaspberryPi,
and will slow down the restart process because there are too much WAL data
on disk).

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


Tim Mitsch  于2019年3月4日周一 下午6:53写道：

> Hello development-team
>
> First of all thanks for developing this kind of interesting project and
> bringing it into apache incubator.
>
> I have a question regarding the place of operation and robustness:
>
>   *   Is iotDB concepted as application on a server which is running 24/7
> or
>   *   Is it also possible to run it on a device like RaspberryPi or IPC,
> where operation can interrupt.
> I’m asking because i’m searching for solution for a temporary storage that
> is robust against spontaneous interrupt, e.g. switch off electricity
> without regular shutdown of OS – have u tested something like this yet?
>
> Best regards
> Tim
>
>
>

Re: Confluence Access Rights

[Julian Feinauer]

Hi Xiangdong,

thanks for the hint.
We should definetly try to keep it similar, yes.
I already pushed a minimal working example based on a tsfile [1].

And I would be happy to cooperate with Tianan and Wangminhao (we already had 
some mail exchanges).

Julian

[1] https://github.com/JulianFeinauer/incubator-iotdb/tree/calcite-integration

Am 04.03.19, 11:14 schrieb "Xiangdong Huang" :

Hi Julian,

Great to hear that you are working on Calcite integration.

I think, we can consider how the SparkSQL-IoTDB adapter is designed, which
is mainly developed by Jialin Qiao and Lei Rui.

In summary, a Storage Group is a table, and there are two types of column
mappings:

1.  the columns are: (timestamp, device_id, measurement1, measurement2...)

2. the columns are: (timestamp, device_1_measurement_1,
device_1_measurement_2, ..., device_2_measurement_1,
device_2_measurement_2, ...)

Tianan Li and Wangminhao Gou are also learning Calcite, maybe they can work
with you.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院


    Julian Feinauer  于2019年3月4日周一 下午4:40写道：

> It's "Julian.Feinauer"
>
> Thank you Justin!
>
> Am 04.03.19, 09:38 schrieb "Justin Mclean" :
>
> Hi,
>
> > So I wanted to create a confluence page for the design documentation
> on [1]. But it seems that I’m missing the permissions.
>
> What’s your user name?
>
> Thanks,
> Justin
>
>

Revive Edgent with some new (and cool) features

[Julian Feinauer]

Hi community,

as I finished some private projects, I feel like I have some time and new 
energy to spend more efforts to edgent.
I have several ideas on how we can kick off the project again and would like to 
share / discuss them with you guys.


  *   First, I suggest that we add adapters (similar to the IotpDevice for IBM 
Watson) for other relevant Cloud plattforms, like Azure, Google, AWS, … .
  *   Second, we open sourced our Framework CRUNCH for industrial analytics 
late last year [1]. I would like to integrate it in edgent to allow more 
refined edge analytics.
  *   Third, I think a good place for something like the PLC Scraper (from 
plc4x [2]) (perhaps with some configuration stuff as standalone) is Edgent. So 
we could build complete pipelines there from connecting to the plc over 
analytics with crunch to some kind of storage (local, cloud)
  *   Finally, the iotdb project [3] which recently joined the incubator is 
doing really great stuff and it would be good to also integrate it in edgent as 
a sink (and source). I see main purposes here, one as a buffer (think of kafka) 
and second as archive store.

I’m cross posting this mail to the lists for edgent, plc4x and iotdb to let all 
communities know but please reply to this mail only on the EDGENT list, as I 
think its best to keep the discussion here.

What do you think of these ideas?
And perhaps we can also win some contributors of the other projects (crunch, 
plc4x, iotdb) to support these efforts.

Thanks!
Julian

[1] https://github.com/pragmaticminds/crunch
[2] https://plc4x.apache.org/
[3] https://iotdb.apache.org/

Revive Edgent with some new (and cool) features

[Julian Feinauer]

Hi community,

as I finished some private projects, I feel like I have some time and new 
energy to spend more efforts to edgent.
I have several ideas on how we can kick off the project again and would like to 
share / discuss them with you guys.


  *   First, I suggest that we add adapters (similar to the IotpDevice for IBM 
Watson) for other relevant Cloud plattforms, like Azure, Google, AWS, … .
  *   Second, we open sourced our Framework CRUNCH for industrial analytics 
late last year [1]. I would like to integrate it in edgent to allow more 
refined edge analytics.
  *   Third, I think a good place for something like the PLC Scraper (from 
plc4x [2]) (perhaps with some configuration stuff as standalone) is Edgent. So 
we could build complete pipelines there from connecting to the plc over 
analytics with crunch to some kind of storage (local, cloud)
  *   Finally, the iotdb project [3] which recently joined the incubator is 
doing really great stuff and it would be good to also integrate it in edgent as 
a sink (and source). I see main purposes here, one as a buffer (think of kafka) 
and second as archive store.

I’m cross posting this mail to the lists for edgent, plc4x and iotdb to let all 
communities know but please reply to this mail only on the EDGENT list, as I 
think its best to keep the discussion here.

What do you think of these ideas?
And perhaps we can also win some contributors of the other projects (crunch, 
plc4x, iotdb) to support these efforts.

Thanks!
Julian

[1] https://github.com/pragmaticminds/crunch
[2] https://plc4x.apache.org/
[3] https://iotdb.apache.org/

Confluence Access Rights

[Julian Feinauer]

Hi all,

I wanted to start with a design page for the Calcite integration on the 
confluence.
Parallel to the technical things we should have a discussion how we map the 
iotdb notions and elements to standard sql (e.g. what is a table? A device? A 
node in the tree? …).
So I wanted to create a confluence page for the design documentation on [1].

But it seems that I’m missing the permissions.
Can anyone here give them?
@Chris, @Justin: can you?

Otherwise I will ask Infra to do so.

Thanks
Julian

[1] https://cwiki.apache.org/confluence/display/IOTDB/Project+Design

AW: [PR summary] #83 Enable float precision control

[Julian Feinauer]

Hi xiangdong,

I cannot comment much on the content as I'm still lacking detailed knowledge.

But a big +1 for the excellent write up on the list!

Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: [PR summary] #83 Enable float precision control
Von: Xiangdong Huang
An: dev@iotdb.apache.org
Cc:

Hi,

As what we discuss in other mailing thread, I will post the summary of my
new PRs to this mailing list.

This email is for summary the info of PR #83 from Github to this mailing
list.

PR #83 and #81 is for fixing the issue of IoTDB-31 on JIRA.  Yi Xu has
reviewed the codes and approved it.

Now the program logic is:
* When writing data into the memory, we do not control the precision, so
that we can get the best ingestion performance.
* When querying data from disk, we do not need additional operations,
because when data is flushing on disk, it will be processed to only remain
the required precision.
* When querying data from memory which is not flushed on disk, we process
this part of data to control its precision.

I use `Math.round()` method to implement the precision control, which only
takes  less than 1/10 additional time cost. (performance test codes: see
comments in JIRA 31)

As the limitation of Java's `Math.round(float)` and `Math.round(double)`,
the value range of float/double is constrained:

* For Float data value, The data range is (-Integer.MAX_VALUE,
Integer.MAX_VALUE), rather than Float.MAX_VALUE, and the max_point_number
is 19.
* For Double data value, The data range is (-Long.MAX_VALUE,
Long.MAX_VALUE), rather than Double.MAX_VALUE, and the max_point_number is
19.

As for the performance, I think the loss is less than 10% (if the queried
data is in memtable), or zero (if there is no suitable data in memtable),
JIRA [IoTDB-31] has a snippet of code for performance.

In this PR, I replace many parameters' declaration `TimeValueSortor` with
`ReadOnlyMemChunk`, because the owners of these parameters never receive a
`TimeValueSortor` instance which is not `ReadOnlyMemChunk`.
I think at this time, using a parent interface is not quite good, because
you will lose many features that  a child class has.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院

AW: AW: Problems about rat plugin and apache header check

[Julian Feinauer]

Von meinem Mobiltelefon gesendet
Hi Xiangdong,

Good observation.could it be due to slashes or something?

Julian

 Ursprüngliche Nachricht 
Betreff: Re: AW: Problems about rat plugin and apache header check
Von: 徐毅
An: dev@iotdb.apache.org
Cc:

It seems that `exclude` configuration does not work on windows, I run "mvn 
rat:check" command on win10 and jdk8.


Tha rat.txt outputs as follow:


*
Printing headers for files without AL header...




===
==.checkstyle
===




  

  
  

  



===
==.git/COMMIT_EDITMSG
===
add debug log info for jdbc service


===
==.git/config
===
[core]
repositoryformatversion = 0
filemode = false
bare = false
logallrefupdates = true
symlinks = false
ignorecase = true
[remote "origin"]
url = g...@github.com:apache/incubator-iotdb.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
remote = origin
merge = refs/heads/master
[branch "doc"]
remote = origin
merge = refs/heads/doc
[branch "add_iotdb_env_script_test"]


On 3/3/2019 22:02，Xiangdong Huang wrote：
Hi,

I notice that win-oraclejdk8, win-oraclejdk11 and linux-openjdk11 fails on
https://travis-ci.org/apache/incubator-iotdb/builds/500987110

It is easy that just skip apache-rat check on the above 3 jobs. But, I
wonder why they failed before I skip that...

Does anyone know how to print the file on Travis? Or, if someone has Win OS
with oracle jdk8/11, please run `mvn apache-rat:check` to verify the result.

[ERROR] Failed to execute goal
org.apache.rat:apache-rat-plugin:0.12:check (default-cli) on project root:
Too many files with unapproved license: 3057 See RAT report in:
C:\Users\travis\build\apache\incubator-iotdb\target\rat.txt

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院


Xiangdong Huang  于2019年3月3日周日 下午9:50写道：

Hi,

I am not sure whether travis can provide a separated pipeline to run
user-specified command.

What do you mean about `user-specified command`?

Best,

---
Xiangdong Huang
School of Software, Tsinghua University

黄向东
清华大学 软件学院


徐毅  于2019年3月3日周日 下午8:13写道：

Hi


When someone creates a pull request from forked repo, jenkins won't
trigger to run tests. So we use travis-ci to trigger tests on different os.
I am not sure whether travis can provide a separated pipeline to run
user-specified command.


Thanks
XuYi


On 3/3/2019 19:46，Julian Feinauer wrote：
Hi,

this behavior sounds strange.
But I think this pipeline should be build into the Jenkins build, or?
It should be pretty easy to force one build step to a specific os.

Perhaps we should open a Jira for that issue to finally track down the
real source of the problem.

Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Problems about rat plugin and apache header check
Von: 徐毅
An: dev@iotdb.apache.org
Cc:

Hi,


Now we use rat plugin to check apache header, it seems that our project
can pass rat check on linux but failed on windows.


If someone creates a pull request from his forked repo, we use travis-ci
to run tests on different os without checking apache header. If we add
apache header check in travis file(see [1]), it fails on some os.


Can we add a new pipeline which is only used to check apache header on
linux? I found that there many free continuous integration service at [2].
Shall we choose one of them?


[1]  https://github.com/apache/incubator-iotdb/pull/84
[2] https://github.com/marketplace/category/continuous-integration


Thanks
XuYi

AW: Problems about rat plugin and apache header check

[Julian Feinauer]

Hi,

this behavior sounds strange.
But I think this pipeline should be build into the Jenkins build, or?
It should be pretty easy to force one build step to a specific os.

Perhaps we should open a Jira for that issue to finally track down the real 
source of the problem.

Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Problems about rat plugin and apache header check
Von: 徐毅
An: dev@iotdb.apache.org
Cc:

Hi,


Now we use rat plugin to check apache header, it seems that our project can 
pass rat check on linux but failed on windows.


If someone creates a pull request from his forked repo, we use travis-ci to run 
tests on different os without checking apache header. If we add apache header 
check in travis file(see [1]), it fails on some os.


Can we add a new pipeline which is only used to check apache header on linux? I 
found that there many free continuous integration service at [2]. Shall we 
choose one of them?


[1]  https://github.com/apache/incubator-iotdb/pull/84
[2] https://github.com/marketplace/category/continuous-integration


Thanks
XuYi

TsFilySync

[Julian Feinauer]

Hi all,

While learning the fundamentals of the project I went through the code and the 
docs.
In the docs, especially in the images there is multiple times a mention of 
tsfilysync. But I cannot find code for this.
Is the docu out of sync or can you point me to where i find the classes and 
give a short explanation on how the db engine relates to the tsfiles?

Thank you!
Julian

Von meinem Mobiltelefon gesendet

AW: Podling Report Reminder - March 2019

[Julian Feinauer]

Hi,
I totally agree with what's been said by Chris and Justin.
This is one of the mature learnings towards becoming a tlp from my perspective. 
Especially when the community grows and more developer or groups join the 
project.

So with regards to calcite integration i can of course only integrate with you 
or give my advises and feedback through discussions on list.

Best
Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Re: Podling Report Reminder - March 2019
Von: Justin Mclean
An: dev@iotdb.apache.org
Cc:

Hi,

> I will remove the sentence ` We'd like to try to release the first binary
> version under the guide of mentors in the next month`.

I’m not sure that removing it answers Chris’s question, either this is planned 
and has been discussed off list or not. Please try and have all discussion on 
the list that way you attract more people and they will know what is going on 
and your mentors also know what is going on and can track your progress towards 
graduation.

Thanks,
Justin

AW: Welcome to the IoTDB project

[Julian Feinauer]

Hi,

I have to check the calcite code to see whether remarks are passed via the Meta 
or not. Generally your implementation of the table interface should provide its 
meta data via the present abstract functions.

Can you give a mwe or a link to a branch, then I could check.

Julian

Von meinem Mobiltelefon gesendet


 Ursprüngliche Nachricht 
Betreff: Re: Welcome to the IoTDB project
Von: 勾王敏浩
An: dev@iotdb.apache.org
Cc: dev@iotdb.apache.org


Hi,
Currently we face a problem,when we use calcite connection to get the 
comment of the table,the "REMARK" of table is returned with "null".

I wonder whether we can get table comment or column comment using calcite.

[cid:77efa623$1$169415a20e9$Coremail$gwmh18$mails.tsinghua.edu.cn]
Thank you
<https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=3=gwmh18=gwmh18%40mails.tsinghua.edu.cn=https%3A%2F%2Fmail-online.nosdn.127.net%2Fwzpmmc%2F8f50f1e94050373bafd9ccdb4491b047.jpg=%E5%A4%A7%E6%95%B0%E6%8D%AE%E5%B7%A5%E7%A8%8B%E5%9B%BD%E5%AE%B6%E5%AE%9E%E9%AA%8C%E5%AE%A4=%5B%22gwmh18%40mails.tsinghua.edu.cn%22%5D=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyeicon%2F56df7a374dbc375497f66a92d65a584d.png>
[https://mail-online.nosdn.127.net/wzpmmc/8f50f1e94050373bafd9ccdb4491b047.jpg]
gwmh18
大数据工程国家实验室
gwm...@mails.tsinghua.edu.cn

签名由 网易邮箱大师<https://mail.163.com/dashi/dlpro.html?from=mail81> 定制

On 12/3/2018 05:32，Julian 
Feinauer<mailto:jfeina...@apache.org> wrote：
Hi,

my name is Julian Feinauer and I come from Apache Calcite, Edgent and PLC4X. My 
interest is mostly in time series processing at the edge. In my daytime job I 
work for two companies called pragmatic minds and pragmatic industries which we 
founded nearly 3 years ago in South Germany.
I love the idea behind iotDB as it is realy the materialization of many of my 
thoughts about what we really need for efficient storage of ts data.
I would love to get my hands dirty on the query and filter api and perhaps even 
write a small Calcite Adapter, when I find some time in the next weeks.

We have many "IoT" projects with practical use cases in the industry so I'll 
try to bring my perspective in as best as possible.

Best
Julian


On 2018/12/01 12:51:47, Willem Jiang  wrote:
Hi,

My name is Willem, my Chinese name is 姜宁. I join the ASF since 2006 as
initial committer (just like most of you) of CXF. Then I joined the
development of Apache ActiveMQ, Apache Camel, Apache ServiceMix. Now
I'm working for Huawei and leading the project Apache SerivceComb.  As
IPMC member I had chance to mentor several  Apache Podling projects
which were started from China. Now I'm happy to be the mentor of
iotdb, and I will work closely with you (as I'm based in Beijing).
Please don't hesitate to ask if you have any questions about
incubating.

I know most of developer may not get use to use mailing list for
discussion.  So I wrote some blogs[1][2] to talk about it when
mentoring the RocketMQ. Please take a look let me know what you think.

[1]https://willemjiang.github.io/blog/2016/why-we-use-mailing-list/
[2]https://willemjiang.github.io/blog/2016/opensource-code-of-conduct/

Willem Jiang

Twitter: willemjiang
Weibo: 姜宁willem
On Fri, Nov 30, 2018 at 10:34 PM 李天安  wrote:

Hello, all,


I'm Tianan Li, it’s my first year of being a master student in School of 
Software at Tsinghua University. I used to take part in the development of 
postback model, which is used to transmit data from end to cloud.

It's my great honor to participate in such an awesome project and I hope to 
make more contributions for IoTDB.


Best Regards,
―
Tianan Li
School of Software, Tsinghua University

在 2018年11月30日，上午5:43，Justin Mclean  写道：

Hi,

I’m Justin and I volunteered to be a mentor for this project and help out where 
needed. So if you need any help or have any questions, in particular about how 
the ASF works, please ask on this list.

I’ve been involved in the ASF for a while now and mentor and help out on a few 
projects, several of them in the IoT space. I’m currently the V.P of the Apache 
incubator and of Apache Mynewt. I’ve been involved in number of commercial IoT 
projects (and scada systems) and I’m a electronics hobbyist. I run the IoT 
meetup in Sydney Australia, speak at conferences and run the occasional 
workshop. I’m a freelance developer and teacher and currently split my time 
between the two, teaching at an online college / being a casual academic at 
university and working as a developer or consultant on several projects.

Thanks,
Justin

Re: How to mark SonarLint issues as False Positive or Won't Fix

[Julian Feinauer]

Hi Tian,

I would prefer to use the annotation:

@java.lang.SuppressWarnings("squid:S00112")

And add a comment in code.
This is more robust then doing it in the ui (when code is resubmitted or 
something, sometimes the warnings reoccur).

Julian

Am 01.03.19, 09:32 schrieb "江天" :

Hello,

I have seen some sonar issues that are not really issues. Leaving them as 
they are makes it harder to find the true positives, so I want to mark them as 
False Positive or Won't Fix. However, I can’t find the option. Is this because 
I do not have the privilege?

Tian Jiang

Re: Move Integration Tests (*IT) to verify phase

[Julian Feinauer]

Hi Xiangdong,

thanks for your hint... I also found the "skipITs" by now so my last mail is 
basically useless.
Locally the mvn clean install fails on my machine, both tests fail in 
IoTDBDeletionIT.
In some document (I'm not able to find it right now) I read that local missing 
libs in some folder can lead to fail the test.

Is this the case here?

Julian



Am 27.02.19, 01:40 schrieb "Xiangdong Huang" :

Hi,

Now all *ITs are in the integration-test phase. `mvn test` does not trigger
the *ITs.  How do you think that?

We can skip all ITs by adding `-DskipITs`, e.g., `mvn install -DskipITs`.
There are some parameters to control which part of Tests are enable, there
is a doc at:

https://cwiki.apache.org/confluence/display/IOTDB/How+To+Compile+and+Test+IoTDB+Using+Maven

By the way, does `mvn clean install` fail when you compile current Master
branch of IoTDB?

Best,

--
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院
    

Julian Feinauer  于2019年2月26日周二 下午10:31写道：

> Hi all,
>
> I just noticed that the regular `mvn clean install` fails during the test
> phase due to IT’s (as stated in the README).
> We usually move all ITs to the verify phase (also mvn clean test is a bit
> faster then).
> And we usually have a separate switch to skip ITs.
>
> What do you think?
>
> Best
> Julian
>

Move Integration Tests (*IT) to verify phase

[Julian Feinauer]

Hi all,

I just noticed that the regular `mvn clean install` fails during the test phase 
due to IT’s (as stated in the README).
We usually move all ITs to the verify phase (also mvn clean test is a bit 
faster then).
And we usually have a separate switch to skip ITs.

What do you think?

Best
Julian

Re: Cannot login Apache Jenkins

[Julian Feinauer]

Hi XuYi,

I did a first check some hours ago and it took a long time (and didn't respond).
But now I can login without problems.
Do you use your apache id / pw for the login?

Julian

Am 26.02.19, 10:39 schrieb "yi xu" :

Hi

My classmates and I found we can not login Apache Jenkins at 
https://builds.apache.org/ . 
Does anyone face the same problem?

Thanks
XuYi

Re: iotdb website is online

[Julian Feinauer]

Looks good!
Congrats!

Am 21.12.18, 10:58 schrieb "徐毅" :

Hi


Our website is now available at https://iotdb.apache.org/. Thanks to 
Christofer and others' help.
Although only home page contains meaningful content, we'll finish the 
development of rest pages successively.
If you have any advice on our web page, please reply to this email.


Thanks,
Yi Xu