Re: [tsfile] [query] path for device.measurement that contain dots
Hi Xiangdong, thanks for the clarification, indeed, then I was wrong. In the light of that, I'm unsure whether I like the idea to allow dots for the names or if these should keep forbidden. This makes it (visibily) clear what part is the measurement and what is the "path". Julian Am 07.03.19, 10:04 schrieb "Xiangdong Huang" : Hi, @Julian I think we support what you want now.. for example, you can create 3 timeseries by: ``` set storage group to root.sg1; create timeseries root.sg1.location1.deviceType1.deviceID.m1 with datatype=FLOAT, encoding=RLE; create timeseries root.sg1.location1.deviceType2.deviceID.m1 with datatype=FLOAT, encoding=RLE; create timeseries root.sg1.location2.deviceType1.deviceID.m1 with datatype=FLOAT, encoding=RLE; ``` Then, if you run: `select m1 from root.sg1.*.deviceType1.deviceID` or `select deviceType1.deviceID.m1 from root.*` you will get the data of `root.sg1.location1.deviceType1.deviceID.m1` and `root.sg1.location2.deviceType1.deviceID.m1` Actually we manage the metadata of measurements (rather than each timeseries) on each storage group, for reduce memory cost and convert the schema to a table. ( That is, you can not create a timeseries in sg1 storage group like: ``` create timeseries root.sg1.location1.deviceType3.deviceID.m1 with datatype=INT32, encoding=RLE; ``` In this example, `m1` is defined as INT32, but it has been claimed as FLOAT in timeseries `root.sg1.location1.deviceType1.deviceID.m1` But, if you create a new storage group `sg2`, you can define `m1` in `sg2` as INT32 while `m1` in `sg1` is FLOAT) So, it is important for IoTDB to recognize which is the measurement name from a path. @Marko, that is why I said add a dot in the measurement will cause many modifications. (Maybe not to many... we just need to let users claim which part of a path is a measurement name, and record it on disk, rather than just split the path with a dot symbol and use the last substring). Best, --- Xiangdong Huang School of Software, Tsinghua University 黄向东 清华大学 软件学院 Julian Feinauer 于2019年3月7日周四 下午4:03写道： > Hi Xiangdong, > > what do you think of a subtile generalization of the current "device" > approach. > If we would allow dots in the name, we could the possibility to create > "trees" like its done e.g. for akka actors [1] or MQTT Topics [2]. > What I mean is that one could add more information in the "path" of a > device to allow for filtering. > Think of the three devices > > "location1.deviceType1.deviceId" > "location1.deviceType2.deviceId" > "location2.deviceType1.deviceId" > > Then I gould ask for: > > "all devices from location 1" with "location.* > "all devices of type 2" with "%.deviceType2.%" > "all devices" with "*" > > (where * means match multiple "hierarchies" and "%" menas match one > hierarchy, a bit how MQTT does it with # and *). > > I think this would be really cool, to also allow these "device paths" for > queries. > What do you think? > > Julian > > [1] https://doc.akka.io/docs/akka/current/general/addressing.html > [2] > https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/ > > Am 07.03.19, 05:40 schrieb "Xiangdong Huang" : > > Hi, > > It seems this will cause a big modification... (Change the Path class > is > easy, but it will lead to a confusion for SQL layer..) A solution is > needed > to let SQL recognize that. > > I have created a issue on JIRA: > https://issues.apache.org/jira/browse/IOTDB-38 > > Best, > --- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > Marko Friedemann 于2019年3月7日周四 上午4:06写道： > > > Hey, > > > > I want to report an issue with tsfile, specifically its query > capabilities. > > > > The two-argument constructor for > org.apache.iotdb.tsfile.read.common.Path > > that is now available in the apache incubator project still does not > allow > > proper construction of a path where the measurement name contains > dots. > > > > Specifically, the issue is that the two-argument constructor > concatenates > > the two arguments with a path-separator character (the dot) and then > splits > > the result again, using the seperator (the dot), instead of just > using the > > supplied arguments as they are. > > This results in the path components being incorrect (device > basically runs >
Re: [tsfile] [query] path for device.measurement that contain dots
Hi,
@Julian I think we support what you want now..
for example, you can create 3 timeseries by:
```
set storage group to root.sg1;
create timeseries root.sg1.location1.deviceType1.deviceID.m1 with
datatype=FLOAT, encoding=RLE;
create timeseries root.sg1.location1.deviceType2.deviceID.m1 with
datatype=FLOAT, encoding=RLE;
create timeseries root.sg1.location2.deviceType1.deviceID.m1 with
datatype=FLOAT, encoding=RLE;
```
Then, if you run:
`select m1 from root.sg1.*.deviceType1.deviceID`
or
`select deviceType1.deviceID.m1 from root.*`
you will get the data of `root.sg1.location1.deviceType1.deviceID.m1` and
`root.sg1.location2.deviceType1.deviceID.m1`
Actually we manage the metadata of measurements (rather than each
timeseries) on each storage group, for reduce memory cost and convert the
schema to a table.
( That is, you can not create a timeseries in sg1 storage group like:
```
create timeseries root.sg1.location1.deviceType3.deviceID.m1 with
datatype=INT32, encoding=RLE;
```
In this example, `m1` is defined as INT32, but it has been claimed as FLOAT
in timeseries `root.sg1.location1.deviceType1.deviceID.m1`
But, if you create a new storage group `sg2`, you can define `m1` in `sg2`
as INT32 while `m1` in `sg1` is FLOAT)
So, it is important for IoTDB to recognize which is the measurement name
from a path. @Marko, that is why I said add a dot in the measurement will
cause many modifications.
(Maybe not to many... we just need to let users claim which part of a path
is a measurement name, and record it on disk, rather than just split the
path with a dot symbol and use the last substring).
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Julian Feinauer 于2019年3月7日周四 下午4:03写道：
> Hi Xiangdong,
>
> what do you think of a subtile generalization of the current "device"
> approach.
> If we would allow dots in the name, we could the possibility to create
> "trees" like its done e.g. for akka actors [1] or MQTT Topics [2].
> What I mean is that one could add more information in the "path" of a
> device to allow for filtering.
> Think of the three devices
>
> "location1.deviceType1.deviceId"
> "location1.deviceType2.deviceId"
> "location2.deviceType1.deviceId"
>
> Then I gould ask for:
>
> "all devices from location 1" with "location.*
> "all devices of type 2" with "%.deviceType2.%"
> "all devices" with "*"
>
> (where * means match multiple "hierarchies" and "%" menas match one
> hierarchy, a bit how MQTT does it with # and *).
>
> I think this would be really cool, to also allow these "device paths" for
> queries.
> What do you think?
>
> Julian
>
> [1] https://doc.akka.io/docs/akka/current/general/addressing.html
> [2]
> https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/
>
> Am 07.03.19, 05:40 schrieb "Xiangdong Huang" :
>
> Hi,
>
> It seems this will cause a big modification... (Change the Path class
> is
> easy, but it will lead to a confusion for SQL layer..) A solution is
> needed
> to let SQL recognize that.
>
> I have created a issue on JIRA:
> https://issues.apache.org/jira/browse/IOTDB-38
>
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Marko Friedemann 于2019年3月7日周四 上午4:06写道：
>
> > Hey,
> >
> > I want to report an issue with tsfile, specifically its query
> capabilities.
> >
> > The two-argument constructor for
> org.apache.iotdb.tsfile.read.common.Path
> > that is now available in the apache incubator project still does not
> allow
> > proper construction of a path where the measurement name contains
> dots.
> >
> > Specifically, the issue is that the two-argument constructor
> concatenates
> > the two arguments with a path-separator character (the dot) and then
> splits
> > the result again, using the seperator (the dot), instead of just
> using the
> > supplied arguments as they are.
> > This results in the path components being incorrect (device
> basically runs
> > to lastIndexOf('.') for the full path) and the query failing.
> >
> > The rest of tsfile's write/read/query functionality doesn't seem to
> mind
> > measurement names that contain dots (think RDF and IRIs) and the
> intended
> > query can be run with a minor fix by sub-classing the Path class.
> > (Unfortunately, Path::init() is also private, so the work-around is
> not
> > readily possible.)
> >
> > Regards,
> > Marko Friedemann
> >
>
>
>
Re: [tsfile] [query] path for device.measurement that contain dots
Hi Xiangdong,
what do you think of a subtile generalization of the current "device" approach.
If we would allow dots in the name, we could the possibility to create "trees"
like its done e.g. for akka actors [1] or MQTT Topics [2].
What I mean is that one could add more information in the "path" of a device to
allow for filtering.
Think of the three devices
"location1.deviceType1.deviceId"
"location1.deviceType2.deviceId"
"location2.deviceType1.deviceId"
Then I gould ask for:
"all devices from location 1" with "location.*
"all devices of type 2" with "%.deviceType2.%"
"all devices" with "*"
(where * means match multiple "hierarchies" and "%" menas match one hierarchy,
a bit how MQTT does it with # and *).
I think this would be really cool, to also allow these "device paths" for
queries.
What do you think?
Julian
[1] https://doc.akka.io/docs/akka/current/general/addressing.html
[2]
https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/
Am 07.03.19, 05:40 schrieb "Xiangdong Huang" :
Hi,
It seems this will cause a big modification... (Change the Path class is
easy, but it will lead to a confusion for SQL layer..) A solution is needed
to let SQL recognize that.
I have created a issue on JIRA:
https://issues.apache.org/jira/browse/IOTDB-38
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Marko Friedemann 于2019年3月7日周四 上午4:06写道：
> Hey,
>
> I want to report an issue with tsfile, specifically its query
capabilities.
>
> The two-argument constructor for org.apache.iotdb.tsfile.read.common.Path
> that is now available in the apache incubator project still does not allow
> proper construction of a path where the measurement name contains dots.
>
> Specifically, the issue is that the two-argument constructor concatenates
> the two arguments with a path-separator character (the dot) and then
splits
> the result again, using the seperator (the dot), instead of just using the
> supplied arguments as they are.
> This results in the path components being incorrect (device basically runs
> to lastIndexOf('.') for the full path) and the query failing.
>
> The rest of tsfile's write/read/query functionality doesn't seem to mind
> measurement names that contain dots (think RDF and IRIs) and the intended
> query can be run with a minor fix by sub-classing the Path class.
> (Unfortunately, Path::init() is also private, so the work-around is not
> readily possible.)
>
> Regards,
> Marko Friedemann
>
Re: [tsfile] [query] path for device.measurement that contain dots
Hi,
It seems this will cause a big modification... (Change the Path class is
easy, but it will lead to a confusion for SQL layer..) A solution is needed
to let SQL recognize that.
I have created a issue on JIRA:
https://issues.apache.org/jira/browse/IOTDB-38
Best,
---
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Marko Friedemann 于2019年3月7日周四 上午4:06写道：
> Hey,
>
> I want to report an issue with tsfile, specifically its query capabilities.
>
> The two-argument constructor for org.apache.iotdb.tsfile.read.common.Path
> that is now available in the apache incubator project still does not allow
> proper construction of a path where the measurement name contains dots.
>
> Specifically, the issue is that the two-argument constructor concatenates
> the two arguments with a path-separator character (the dot) and then splits
> the result again, using the seperator (the dot), instead of just using the
> supplied arguments as they are.
> This results in the path components being incorrect (device basically runs
> to lastIndexOf('.') for the full path) and the query failing.
>
> The rest of tsfile's write/read/query functionality doesn't seem to mind
> measurement names that contain dots (think RDF and IRIs) and the intended
> query can be run with a minor fix by sub-classing the Path class.
> (Unfortunately, Path::init() is also private, so the work-around is not
> readily possible.)
>
> Regards,
> Marko Friedemann
>
[tsfile] [query] path for device.measurement that contain dots
Hey,
I want to report an issue with tsfile, specifically its query capabilities.
The two-argument constructor for org.apache.iotdb.tsfile.read.common.Path
that is now available in the apache incubator project still does not allow
proper construction of a path where the measurement name contains dots.
Specifically, the issue is that the two-argument constructor concatenates
the two arguments with a path-separator character (the dot) and then splits
the result again, using the seperator (the dot), instead of just using the
supplied arguments as they are.
This results in the path components being incorrect (device basically runs
to lastIndexOf('.') for the full path) and the query failing.
The rest of tsfile's write/read/query functionality doesn't seem to mind
measurement names that contain dots (think RDF and IRIs) and the intended
query can be run with a minor fix by sub-classing the Path class.
(Unfortunately, Path::init() is also private, so the work-around is not
readily possible.)
Regards,
Marko Friedemann
