Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
Thanks to Timo for bringing up an interesting topic. Personally, "OPAQUE" doesn't seem very intuitive with respect to types. (It suits pretty well to glasses, thought. :)) Anyway, could we just use "UNKNOWN", which is more explicit and true reflects its nature? Thanks, Xuefu On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: > Hi everyone, > > Stephan pointed out that our naming of a generic/blackbox/opaque type in > SQL might be not intuitive for users. As the term ANY rather describes a > "super-class of all types" which is not the case in our type system. Our > current ANY type stands for a type that is just a blackbox within SQL, > serialized by some custom serializer, that can only be modified within > UDFs. > > I also gathered feedback from a training instructor and native English > speaker (David in CC) where I received the following: > > "The way I’m thinking about this is this: there’s a concept here that > people have to become aware of, which is that Flink SQL is able to > operate generically on opaquely typed things — and folks need to be able > to connect what they see in code examples, etc. with this concept (which > they may be unaware of initially). > I feel like ANY misses the mark a little bit, but isn’t particularly > bad. I do worry that it may cause some confusion about its purpose and > power. I think OPAQUE would more clearly express what’s going on." > > Also resources like Wikipedia [1] show that this terminology is common: > > "a data type whose concrete data structure is not defined [...] its > values can only be manipulated by calling subroutines that have access > to the missing information" > > I would therefore vote for refactoring the type name because it is not > used much yet. > > Implications are: > > - a new parser keyword "OPAQUE" and changed SQL parser > > - changes for logical type root, logical type visitors, and their usages > > What do you think? > > Thanks, > > Timo > > [1] https://en.wikipedia.org/wiki/Opaque_data_type > > > -- Xuefu Zhang "In Honey We Trust!"
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
I prefer OPAQUE compared to ANY because any is often the root object in an object hierarchy and would indicate to users the wrong thing. Aljoscha > On 18. Oct 2019, at 18:41, Xuefu Z wrote: > > Thanks to Timo for bringing up an interesting topic. > > Personally, "OPAQUE" doesn't seem very intuitive with respect to types. (It > suits pretty well to glasses, thought. :)) Anyway, could we just use > "UNKNOWN", which is more explicit and true reflects its nature? > > Thanks, > Xuefu > > > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: > >> Hi everyone, >> >> Stephan pointed out that our naming of a generic/blackbox/opaque type in >> SQL might be not intuitive for users. As the term ANY rather describes a >> "super-class of all types" which is not the case in our type system. Our >> current ANY type stands for a type that is just a blackbox within SQL, >> serialized by some custom serializer, that can only be modified within >> UDFs. >> >> I also gathered feedback from a training instructor and native English >> speaker (David in CC) where I received the following: >> >> "The way I’m thinking about this is this: there’s a concept here that >> people have to become aware of, which is that Flink SQL is able to >> operate generically on opaquely typed things — and folks need to be able >> to connect what they see in code examples, etc. with this concept (which >> they may be unaware of initially). >> I feel like ANY misses the mark a little bit, but isn’t particularly >> bad. I do worry that it may cause some confusion about its purpose and >> power. I think OPAQUE would more clearly express what’s going on." >> >> Also resources like Wikipedia [1] show that this terminology is common: >> >> "a data type whose concrete data structure is not defined [...] its >> values can only be manipulated by calling subroutines that have access >> to the missing information" >> >> I would therefore vote for refactoring the type name because it is not >> used much yet. >> >> Implications are: >> >> - a new parser keyword "OPAQUE" and changed SQL parser >> >> - changes for logical type root, logical type visitors, and their usages >> >> What do you think? >> >> Thanks, >> >> Timo >> >> [1] https://en.wikipedia.org/wiki/Opaque_data_type >> >> >> > > -- > Xuefu Zhang > > "In Honey We Trust!"
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
+1 to rename ANY. I don't have strong opinion on the new name. I think "OPAQUE" is fine, because it is introduced in IBM Informix and Oracle. In Informix, it says [1]: "An opaque data type is fully encapsulated; the database server does not know about the internal format of an opaque data type. Therefore, the database server cannot make assumptions about how to access a column having an opaque data type. The database developer defines a data structure that holds the opaque-type information and support functions that tell the database server how to access this data structure." So, I think "opaque" is fine here. Another option is "RAW" which is introduced by Oracle [2] represents data that is not to be interpreted by system . Best, Jark [1]: https://www.ibm.com/support/knowledgecenter/en/SSGU8G_11.70.0/com.ibm.esqlc.doc/ids_esqlc_0357.htm [2]: https://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#CNCPT613 On Mon, 21 Oct 2019 at 15:49, Aljoscha Krettek wrote: > I prefer OPAQUE compared to ANY because any is often the root object in an > object hierarchy and would indicate to users the wrong thing. > > Aljoscha > > > On 18. Oct 2019, at 18:41, Xuefu Z wrote: > > > > Thanks to Timo for bringing up an interesting topic. > > > > Personally, "OPAQUE" doesn't seem very intuitive with respect to types. > (It > > suits pretty well to glasses, thought. :)) Anyway, could we just use > > "UNKNOWN", which is more explicit and true reflects its nature? > > > > Thanks, > > Xuefu > > > > > > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: > > > >> Hi everyone, > >> > >> Stephan pointed out that our naming of a generic/blackbox/opaque type in > >> SQL might be not intuitive for users. As the term ANY rather describes a > >> "super-class of all types" which is not the case in our type system. Our > >> current ANY type stands for a type that is just a blackbox within SQL, > >> serialized by some custom serializer, that can only be modified within > >> UDFs. > >> > >> I also gathered feedback from a training instructor and native English > >> speaker (David in CC) where I received the following: > >> > >> "The way I’m thinking about this is this: there’s a concept here that > >> people have to become aware of, which is that Flink SQL is able to > >> operate generically on opaquely typed things — and folks need to be able > >> to connect what they see in code examples, etc. with this concept (which > >> they may be unaware of initially). > >> I feel like ANY misses the mark a little bit, but isn’t particularly > >> bad. I do worry that it may cause some confusion about its purpose and > >> power. I think OPAQUE would more clearly express what’s going on." > >> > >> Also resources like Wikipedia [1] show that this terminology is common: > >> > >> "a data type whose concrete data structure is not defined [...] its > >> values can only be manipulated by calling subroutines that have access > >> to the missing information" > >> > >> I would therefore vote for refactoring the type name because it is not > >> used much yet. > >> > >> Implications are: > >> > >> - a new parser keyword "OPAQUE" and changed SQL parser > >> > >> - changes for logical type root, logical type visitors, and their usages > >> > >> What do you think? > >> > >> Thanks, > >> > >> Timo > >> > >> [1] https://en.wikipedia.org/wiki/Opaque_data_type > >> > >> > >> > > > > -- > > Xuefu Zhang > > > > "In Honey We Trust!" > >
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
OPAQUE seems to be a little bit advanced to a lot non-english speakers (including me). I think Xuefu raised a good alternative: UNKNOWN. What do you think about it? Best, Kurt On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek wrote: > I prefer OPAQUE compared to ANY because any is often the root object in an > object hierarchy and would indicate to users the wrong thing. > > Aljoscha > > > On 18. Oct 2019, at 18:41, Xuefu Z wrote: > > > > Thanks to Timo for bringing up an interesting topic. > > > > Personally, "OPAQUE" doesn't seem very intuitive with respect to types. > (It > > suits pretty well to glasses, thought. :)) Anyway, could we just use > > "UNKNOWN", which is more explicit and true reflects its nature? > > > > Thanks, > > Xuefu > > > > > > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: > > > >> Hi everyone, > >> > >> Stephan pointed out that our naming of a generic/blackbox/opaque type in > >> SQL might be not intuitive for users. As the term ANY rather describes a > >> "super-class of all types" which is not the case in our type system. Our > >> current ANY type stands for a type that is just a blackbox within SQL, > >> serialized by some custom serializer, that can only be modified within > >> UDFs. > >> > >> I also gathered feedback from a training instructor and native English > >> speaker (David in CC) where I received the following: > >> > >> "The way I’m thinking about this is this: there’s a concept here that > >> people have to become aware of, which is that Flink SQL is able to > >> operate generically on opaquely typed things — and folks need to be able > >> to connect what they see in code examples, etc. with this concept (which > >> they may be unaware of initially). > >> I feel like ANY misses the mark a little bit, but isn’t particularly > >> bad. I do worry that it may cause some confusion about its purpose and > >> power. I think OPAQUE would more clearly express what’s going on." > >> > >> Also resources like Wikipedia [1] show that this terminology is common: > >> > >> "a data type whose concrete data structure is not defined [...] its > >> values can only be manipulated by calling subroutines that have access > >> to the missing information" > >> > >> I would therefore vote for refactoring the type name because it is not > >> used much yet. > >> > >> Implications are: > >> > >> - a new parser keyword "OPAQUE" and changed SQL parser > >> > >> - changes for logical type root, logical type visitors, and their usages > >> > >> What do you think? > >> > >> Thanks, > >> > >> Timo > >> > >> [1] https://en.wikipedia.org/wiki/Opaque_data_type > >> > >> > >> > > > > -- > > Xuefu Zhang > > > > "In Honey We Trust!" > >
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
Hi, IMHO, `UNKNOWN` does not fully reflects the situation here, because the types are actually “known” to users, and users just want to leave them out of Flink type system. +1 for `RAW`, for it's more intuitive than `OPAQUE`. Best, Paul Lam > 在 2019年10月21日，16:43，Kurt Young 写道： > > OPAQUE seems to be a little bit advanced to a lot non-english > speakers (including me). I think Xuefu raised a good alternative: > UNKNOWN. What do you think about it? > > Best, > Kurt > > > On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek > wrote: > >> I prefer OPAQUE compared to ANY because any is often the root object in an >> object hierarchy and would indicate to users the wrong thing. >> >> Aljoscha >> >>> On 18. Oct 2019, at 18:41, Xuefu Z wrote: >>> >>> Thanks to Timo for bringing up an interesting topic. >>> >>> Personally, "OPAQUE" doesn't seem very intuitive with respect to types. >> (It >>> suits pretty well to glasses, thought. :)) Anyway, could we just use >>> "UNKNOWN", which is more explicit and true reflects its nature? >>> >>> Thanks, >>> Xuefu >>> >>> >>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: >>> Hi everyone, Stephan pointed out that our naming of a generic/blackbox/opaque type in SQL might be not intuitive for users. As the term ANY rather describes a "super-class of all types" which is not the case in our type system. Our current ANY type stands for a type that is just a blackbox within SQL, serialized by some custom serializer, that can only be modified within UDFs. I also gathered feedback from a training instructor and native English speaker (David in CC) where I received the following: "The way I’m thinking about this is this: there’s a concept here that people have to become aware of, which is that Flink SQL is able to operate generically on opaquely typed things — and folks need to be able to connect what they see in code examples, etc. with this concept (which they may be unaware of initially). I feel like ANY misses the mark a little bit, but isn’t particularly bad. I do worry that it may cause some confusion about its purpose and power. I think OPAQUE would more clearly express what’s going on." Also resources like Wikipedia [1] show that this terminology is common: "a data type whose concrete data structure is not defined [...] its values can only be manipulated by calling subroutines that have access to the missing information" I would therefore vote for refactoring the type name because it is not used much yet. Implications are: - a new parser keyword "OPAQUE" and changed SQL parser - changes for logical type root, logical type visitors, and their usages What do you think? Thanks, Timo [1] https://en.wikipedia.org/wiki/Opaque_data_type >>> >>> -- >>> Xuefu Zhang >>> >>> "In Honey We Trust!" >> >>
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
I also think `UNKNOWN` is not suitable here. Because we already have `UNKNOWN` value in SQL, i.e. the three-valued logic (TRUE, FALSE, UNKNOWN) of BOOLEAN type. It will confuse users here, what's the relationship between them. Best, Jark On Mon, 21 Oct 2019 at 17:53, Paul Lam wrote: > Hi, > > IMHO, `UNKNOWN` does not fully reflects the situation here, because the > types are > actually “known” to users, and users just want to leave them out of Flink > type system. > > +1 for `RAW`, for it's more intuitive than `OPAQUE`. > > Best, > Paul Lam > > > 在 2019年10月21日，16:43，Kurt Young 写道： > > > > OPAQUE seems to be a little bit advanced to a lot non-english > > speakers (including me). I think Xuefu raised a good alternative: > > UNKNOWN. What do you think about it? > > > > Best, > > Kurt > > > > > > On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek > > wrote: > > > >> I prefer OPAQUE compared to ANY because any is often the root object in > an > >> object hierarchy and would indicate to users the wrong thing. > >> > >> Aljoscha > >> > >>> On 18. Oct 2019, at 18:41, Xuefu Z wrote: > >>> > >>> Thanks to Timo for bringing up an interesting topic. > >>> > >>> Personally, "OPAQUE" doesn't seem very intuitive with respect to types. > >> (It > >>> suits pretty well to glasses, thought. :)) Anyway, could we just use > >>> "UNKNOWN", which is more explicit and true reflects its nature? > >>> > >>> Thanks, > >>> Xuefu > >>> > >>> > >>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther > wrote: > >>> > Hi everyone, > > Stephan pointed out that our naming of a generic/blackbox/opaque type > in > SQL might be not intuitive for users. As the term ANY rather > describes a > "super-class of all types" which is not the case in our type system. > Our > current ANY type stands for a type that is just a blackbox within SQL, > serialized by some custom serializer, that can only be modified within > UDFs. > > I also gathered feedback from a training instructor and native English > speaker (David in CC) where I received the following: > > "The way I’m thinking about this is this: there’s a concept here that > people have to become aware of, which is that Flink SQL is able to > operate generically on opaquely typed things — and folks need to be > able > to connect what they see in code examples, etc. with this concept > (which > they may be unaware of initially). > I feel like ANY misses the mark a little bit, but isn’t particularly > bad. I do worry that it may cause some confusion about its purpose and > power. I think OPAQUE would more clearly express what’s going on." > > Also resources like Wikipedia [1] show that this terminology is > common: > > "a data type whose concrete data structure is not defined [...] its > values can only be manipulated by calling subroutines that have access > to the missing information" > > I would therefore vote for refactoring the type name because it is not > used much yet. > > Implications are: > > - a new parser keyword "OPAQUE" and changed SQL parser > > - changes for logical type root, logical type visitors, and their > usages > > What do you think? > > Thanks, > > Timo > > [1] https://en.wikipedia.org/wiki/Opaque_data_type > > > > >>> > >>> -- > >>> Xuefu Zhang > >>> > >>> "In Honey We Trust!" > >> > >> > >
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
I would also avoid `UNKNOWN` because of the mentioned reasons. I'm fine with `RAW`. I will wait another day or two until I conclude the discussion. Thanks, Timo On 21.10.19 12:23, Jark Wu wrote: I also think `UNKNOWN` is not suitable here. Because we already have `UNKNOWN` value in SQL, i.e. the three-valued logic (TRUE, FALSE, UNKNOWN) of BOOLEAN type. It will confuse users here, what's the relationship between them. Best, Jark On Mon, 21 Oct 2019 at 17:53, Paul Lam wrote: Hi, IMHO, `UNKNOWN` does not fully reflects the situation here, because the types are actually “known” to users, and users just want to leave them out of Flink type system. +1 for `RAW`, for it's more intuitive than `OPAQUE`. Best, Paul Lam 在 2019年10月21日，16:43，Kurt Young 写道： OPAQUE seems to be a little bit advanced to a lot non-english speakers (including me). I think Xuefu raised a good alternative: UNKNOWN. What do you think about it? Best, Kurt On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek wrote: I prefer OPAQUE compared to ANY because any is often the root object in an object hierarchy and would indicate to users the wrong thing. Aljoscha On 18. Oct 2019, at 18:41, Xuefu Z wrote: Thanks to Timo for bringing up an interesting topic. Personally, "OPAQUE" doesn't seem very intuitive with respect to types. (It suits pretty well to glasses, thought. :)) Anyway, could we just use "UNKNOWN", which is more explicit and true reflects its nature? Thanks, Xuefu On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: Hi everyone, Stephan pointed out that our naming of a generic/blackbox/opaque type in SQL might be not intuitive for users. As the term ANY rather describes a "super-class of all types" which is not the case in our type system. Our current ANY type stands for a type that is just a blackbox within SQL, serialized by some custom serializer, that can only be modified within UDFs. I also gathered feedback from a training instructor and native English speaker (David in CC) where I received the following: "The way I’m thinking about this is this: there’s a concept here that people have to become aware of, which is that Flink SQL is able to operate generically on opaquely typed things — and folks need to be able to connect what they see in code examples, etc. with this concept (which they may be unaware of initially). I feel like ANY misses the mark a little bit, but isn’t particularly bad. I do worry that it may cause some confusion about its purpose and power. I think OPAQUE would more clearly express what’s going on." Also resources like Wikipedia [1] show that this terminology is common: "a data type whose concrete data structure is not defined [...] its values can only be manipulated by calling subroutines that have access to the missing information" I would therefore vote for refactoring the type name because it is not used much yet. Implications are: - a new parser keyword "OPAQUE" and changed SQL parser - changes for logical type root, logical type visitors, and their usages What do you think? Thanks, Timo [1] https://en.wikipedia.org/wiki/Opaque_data_type -- Xuefu Zhang "In Honey We Trust!"
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
+1 to RAW, if there's no better candidate comes up. Best, Kurt On Mon, Oct 21, 2019 at 9:25 PM Timo Walther wrote: > I would also avoid `UNKNOWN` because of the mentioned reasons. > > I'm fine with `RAW`. I will wait another day or two until I conclude the > discussion. > > Thanks, > Timo > > > On 21.10.19 12:23, Jark Wu wrote: > > I also think `UNKNOWN` is not suitable here. > > Because we already have `UNKNOWN` value in SQL, i.e. the three-valued > logic > > (TRUE, FALSE, UNKNOWN) of BOOLEAN type. > > It will confuse users here, what's the relationship between them. > > > > Best, > > Jark > > > > On Mon, 21 Oct 2019 at 17:53, Paul Lam wrote: > > > >> Hi, > >> > >> IMHO, `UNKNOWN` does not fully reflects the situation here, because the > >> types are > >> actually “known” to users, and users just want to leave them out of > Flink > >> type system. > >> > >> +1 for `RAW`, for it's more intuitive than `OPAQUE`. > >> > >> Best, > >> Paul Lam > >> > >>> 在 2019年10月21日，16:43，Kurt Young 写道： > >>> > >>> OPAQUE seems to be a little bit advanced to a lot non-english > >>> speakers (including me). I think Xuefu raised a good alternative: > >>> UNKNOWN. What do you think about it? > >>> > >>> Best, > >>> Kurt > >>> > >>> > >>> On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek > >>> wrote: > >>> > I prefer OPAQUE compared to ANY because any is often the root object > in > >> an > object hierarchy and would indicate to users the wrong thing. > > Aljoscha > > > On 18. Oct 2019, at 18:41, Xuefu Z wrote: > > > > Thanks to Timo for bringing up an interesting topic. > > > > Personally, "OPAQUE" doesn't seem very intuitive with respect to > types. > (It > > suits pretty well to glasses, thought. :)) Anyway, could we just use > > "UNKNOWN", which is more explicit and true reflects its nature? > > > > Thanks, > > Xuefu > > > > > > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther > >> wrote: > >> Hi everyone, > >> > >> Stephan pointed out that our naming of a generic/blackbox/opaque > type > >> in > >> SQL might be not intuitive for users. As the term ANY rather > >> describes a > >> "super-class of all types" which is not the case in our type system. > >> Our > >> current ANY type stands for a type that is just a blackbox within > SQL, > >> serialized by some custom serializer, that can only be modified > within > >> UDFs. > >> > >> I also gathered feedback from a training instructor and native > English > >> speaker (David in CC) where I received the following: > >> > >> "The way I’m thinking about this is this: there’s a concept here > that > >> people have to become aware of, which is that Flink SQL is able to > >> operate generically on opaquely typed things — and folks need to be > >> able > >> to connect what they see in code examples, etc. with this concept > >> (which > >> they may be unaware of initially). > >> I feel like ANY misses the mark a little bit, but isn’t particularly > >> bad. I do worry that it may cause some confusion about its purpose > and > >> power. I think OPAQUE would more clearly express what’s going on." > >> > >> Also resources like Wikipedia [1] show that this terminology is > >> common: > >> "a data type whose concrete data structure is not defined [...] its > >> values can only be manipulated by calling subroutines that have > access > >> to the missing information" > >> > >> I would therefore vote for refactoring the type name because it is > not > >> used much yet. > >> > >> Implications are: > >> > >> - a new parser keyword "OPAQUE" and changed SQL parser > >> > >> - changes for logical type root, logical type visitors, and their > >> usages > >> What do you think? > >> > >> Thanks, > >> > >> Timo > >> > >> [1] https://en.wikipedia.org/wiki/Opaque_data_type > >> > >> > >> > > -- > > Xuefu Zhang > > > > "In Honey We Trust!" > > >> > >
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
“OPAQUE” seems a little strange to me. + 1 for ‘RAW’. Best, Terry Wang > 2019年10月22日 09:19，Kurt Young 写道： > > +1 to RAW, if there's no better candidate comes up. > > Best, > Kurt > > > On Mon, Oct 21, 2019 at 9:25 PM Timo Walther wrote: > >> I would also avoid `UNKNOWN` because of the mentioned reasons. >> >> I'm fine with `RAW`. I will wait another day or two until I conclude the >> discussion. >> >> Thanks, >> Timo >> >> >> On 21.10.19 12:23, Jark Wu wrote: >>> I also think `UNKNOWN` is not suitable here. >>> Because we already have `UNKNOWN` value in SQL, i.e. the three-valued >> logic >>> (TRUE, FALSE, UNKNOWN) of BOOLEAN type. >>> It will confuse users here, what's the relationship between them. >>> >>> Best, >>> Jark >>> >>> On Mon, 21 Oct 2019 at 17:53, Paul Lam wrote: >>> Hi, IMHO, `UNKNOWN` does not fully reflects the situation here, because the types are actually “known” to users, and users just want to leave them out of >> Flink type system. +1 for `RAW`, for it's more intuitive than `OPAQUE`. Best, Paul Lam > 在 2019年10月21日，16:43，Kurt Young 写道： > > OPAQUE seems to be a little bit advanced to a lot non-english > speakers (including me). I think Xuefu raised a good alternative: > UNKNOWN. What do you think about it? > > Best, > Kurt > > > On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek > wrote: > >> I prefer OPAQUE compared to ANY because any is often the root object >> in an >> object hierarchy and would indicate to users the wrong thing. >> >> Aljoscha >> >>> On 18. Oct 2019, at 18:41, Xuefu Z wrote: >>> >>> Thanks to Timo for bringing up an interesting topic. >>> >>> Personally, "OPAQUE" doesn't seem very intuitive with respect to >> types. >> (It >>> suits pretty well to glasses, thought. :)) Anyway, could we just use >>> "UNKNOWN", which is more explicit and true reflects its nature? >>> >>> Thanks, >>> Xuefu >>> >>> >>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: Hi everyone, Stephan pointed out that our naming of a generic/blackbox/opaque >> type in SQL might be not intuitive for users. As the term ANY rather describes a "super-class of all types" which is not the case in our type system. Our current ANY type stands for a type that is just a blackbox within >> SQL, serialized by some custom serializer, that can only be modified >> within UDFs. I also gathered feedback from a training instructor and native >> English speaker (David in CC) where I received the following: "The way I’m thinking about this is this: there’s a concept here >> that people have to become aware of, which is that Flink SQL is able to operate generically on opaquely typed things — and folks need to be able to connect what they see in code examples, etc. with this concept (which they may be unaware of initially). I feel like ANY misses the mark a little bit, but isn’t particularly bad. I do worry that it may cause some confusion about its purpose >> and power. I think OPAQUE would more clearly express what’s going on." Also resources like Wikipedia [1] show that this terminology is common: "a data type whose concrete data structure is not defined [...] its values can only be manipulated by calling subroutines that have >> access to the missing information" I would therefore vote for refactoring the type name because it is >> not used much yet. Implications are: - a new parser keyword "OPAQUE" and changed SQL parser - changes for logical type root, logical type visitors, and their usages What do you think? Thanks, Timo [1] https://en.wikipedia.org/wiki/Opaque_data_type >>> -- >>> Xuefu Zhang >>> >>> "In Honey We Trust!" >> >> >>
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
+1 for RAW. I agree that this is clearer than OPAQUE (which I initially proposed). On Mon, Oct 21, 2019 at 10:33 AM Jark Wu wrote: > > +1 to rename ANY. > > I don't have strong opinion on the new name. I think "OPAQUE" is fine, > because it is introduced in IBM Informix and Oracle. > In Informix, it says [1]: > > "An opaque data type is fully encapsulated; the database server does not know > about the internal format of an opaque data type. > Therefore, the database server cannot make assumptions about how to access a > column having an opaque data type. > The database developer defines a data structure that holds the opaque-type > information and support functions > that tell the database server how to access this data structure." > > So, I think "opaque" is fine here. > > Another option is "RAW" which is introduced by Oracle [2] represents data > that is not to be interpreted by system . > > Best, > Jark > > [1]: > https://www.ibm.com/support/knowledgecenter/en/SSGU8G_11.70.0/com.ibm.esqlc.doc/ids_esqlc_0357.htm > [2]: > https://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#CNCPT613 > > On Mon, 21 Oct 2019 at 15:49, Aljoscha Krettek wrote: >> >> I prefer OPAQUE compared to ANY because any is often the root object in an >> object hierarchy and would indicate to users the wrong thing. >> >> Aljoscha >> >> > On 18. Oct 2019, at 18:41, Xuefu Z wrote: >> > >> > Thanks to Timo for bringing up an interesting topic. >> > >> > Personally, "OPAQUE" doesn't seem very intuitive with respect to types. (It >> > suits pretty well to glasses, thought. :)) Anyway, could we just use >> > "UNKNOWN", which is more explicit and true reflects its nature? >> > >> > Thanks, >> > Xuefu >> > >> > >> > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: >> > >> >> Hi everyone, >> >> >> >> Stephan pointed out that our naming of a generic/blackbox/opaque type in >> >> SQL might be not intuitive for users. As the term ANY rather describes a >> >> "super-class of all types" which is not the case in our type system. Our >> >> current ANY type stands for a type that is just a blackbox within SQL, >> >> serialized by some custom serializer, that can only be modified within >> >> UDFs. >> >> >> >> I also gathered feedback from a training instructor and native English >> >> speaker (David in CC) where I received the following: >> >> >> >> "The way I’m thinking about this is this: there’s a concept here that >> >> people have to become aware of, which is that Flink SQL is able to >> >> operate generically on opaquely typed things — and folks need to be able >> >> to connect what they see in code examples, etc. with this concept (which >> >> they may be unaware of initially). >> >> I feel like ANY misses the mark a little bit, but isn’t particularly >> >> bad. I do worry that it may cause some confusion about its purpose and >> >> power. I think OPAQUE would more clearly express what’s going on." >> >> >> >> Also resources like Wikipedia [1] show that this terminology is common: >> >> >> >> "a data type whose concrete data structure is not defined [...] its >> >> values can only be manipulated by calling subroutines that have access >> >> to the missing information" >> >> >> >> I would therefore vote for refactoring the type name because it is not >> >> used much yet. >> >> >> >> Implications are: >> >> >> >> - a new parser keyword "OPAQUE" and changed SQL parser >> >> >> >> - changes for logical type root, logical type visitors, and their usages >> >> >> >> What do you think? >> >> >> >> Thanks, >> >> >> >> Timo >> >> >> >> [1] https://en.wikipedia.org/wiki/Opaque_data_type >> >> >> >> >> >> >> > >> > -- >> > Xuefu Zhang >> > >> > "In Honey We Trust!" >>
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
+1 for RAW type, which is easy to guess its actual meaning, even for a new user. Best, Weike On Tue, Oct 22, 2019 at 5:31 PM David Anderson wrote: > +1 for RAW. > > I agree that this is clearer than OPAQUE (which I initially proposed). > > On Mon, Oct 21, 2019 at 10:33 AM Jark Wu wrote: > > > > +1 to rename ANY. > > > > I don't have strong opinion on the new name. I think "OPAQUE" is fine, > because it is introduced in IBM Informix and Oracle. > > In Informix, it says [1]: > > > > "An opaque data type is fully encapsulated; the database server does not > know about the internal format of an opaque data type. > > Therefore, the database server cannot make assumptions about how to > access a column having an opaque data type. > > The database developer defines a data structure that holds the > opaque-type information and support functions > > that tell the database server how to access this data structure." > > > > So, I think "opaque" is fine here. > > > > Another option is "RAW" which is introduced by Oracle [2] represents > data that is not to be interpreted by system . > > > > Best, > > Jark > > > > [1]: > https://www.ibm.com/support/knowledgecenter/en/SSGU8G_11.70.0/com.ibm.esqlc.doc/ids_esqlc_0357.htm > > [2]: > https://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#CNCPT613 > > > > On Mon, 21 Oct 2019 at 15:49, Aljoscha Krettek > wrote: > >> > >> I prefer OPAQUE compared to ANY because any is often the root object in > an object hierarchy and would indicate to users the wrong thing. > >> > >> Aljoscha > >> > >> > On 18. Oct 2019, at 18:41, Xuefu Z wrote: > >> > > >> > Thanks to Timo for bringing up an interesting topic. > >> > > >> > Personally, "OPAQUE" doesn't seem very intuitive with respect to > types. (It > >> > suits pretty well to glasses, thought. :)) Anyway, could we just use > >> > "UNKNOWN", which is more explicit and true reflects its nature? > >> > > >> > Thanks, > >> > Xuefu > >> > > >> > > >> > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther > wrote: > >> > > >> >> Hi everyone, > >> >> > >> >> Stephan pointed out that our naming of a generic/blackbox/opaque > type in > >> >> SQL might be not intuitive for users. As the term ANY rather > describes a > >> >> "super-class of all types" which is not the case in our type system. > Our > >> >> current ANY type stands for a type that is just a blackbox within > SQL, > >> >> serialized by some custom serializer, that can only be modified > within > >> >> UDFs. > >> >> > >> >> I also gathered feedback from a training instructor and native > English > >> >> speaker (David in CC) where I received the following: > >> >> > >> >> "The way I’m thinking about this is this: there’s a concept here that > >> >> people have to become aware of, which is that Flink SQL is able to > >> >> operate generically on opaquely typed things — and folks need to be > able > >> >> to connect what they see in code examples, etc. with this concept > (which > >> >> they may be unaware of initially). > >> >> I feel like ANY misses the mark a little bit, but isn’t particularly > >> >> bad. I do worry that it may cause some confusion about its purpose > and > >> >> power. I think OPAQUE would more clearly express what’s going on." > >> >> > >> >> Also resources like Wikipedia [1] show that this terminology is > common: > >> >> > >> >> "a data type whose concrete data structure is not defined [...] its > >> >> values can only be manipulated by calling subroutines that have > access > >> >> to the missing information" > >> >> > >> >> I would therefore vote for refactoring the type name because it is > not > >> >> used much yet. > >> >> > >> >> Implications are: > >> >> > >> >> - a new parser keyword "OPAQUE" and changed SQL parser > >> >> > >> >> - changes for logical type root, logical type visitors, and their > usages > >> >> > >> >> What do you think? > >> >> > >> >> Thanks, > >> >> > >> >> Timo > >> >> > >> >> [1] https://en.wikipedia.org/wiki/Opaque_data_type > >> >> > >> >> > >> >> > >> > > >> > -- > >> > Xuefu Zhang > >> > > >> > "In Honey We Trust!" > >> >
Re: [DISCUSS] Rename the SQL ANY type to OPAQUE type
Thanks everyone for the feedback! We have a clear favorite which is the RAW type. I will make sure that this change is applied to the Flink code base. Thanks again, Timo On 22.10.19 04:07, Terry Wang wrote: “OPAQUE” seems a little strange to me. + 1 for ‘RAW’. Best, Terry Wang 2019年10月22日 09:19，Kurt Young 写道： +1 to RAW, if there's no better candidate comes up. Best, Kurt On Mon, Oct 21, 2019 at 9:25 PM Timo Walther wrote: I would also avoid `UNKNOWN` because of the mentioned reasons. I'm fine with `RAW`. I will wait another day or two until I conclude the discussion. Thanks, Timo On 21.10.19 12:23, Jark Wu wrote: I also think `UNKNOWN` is not suitable here. Because we already have `UNKNOWN` value in SQL, i.e. the three-valued logic (TRUE, FALSE, UNKNOWN) of BOOLEAN type. It will confuse users here, what's the relationship between them. Best, Jark On Mon, 21 Oct 2019 at 17:53, Paul Lam wrote: Hi, IMHO, `UNKNOWN` does not fully reflects the situation here, because the types are actually “known” to users, and users just want to leave them out of Flink type system. +1 for `RAW`, for it's more intuitive than `OPAQUE`. Best, Paul Lam 在 2019年10月21日，16:43，Kurt Young 写道： OPAQUE seems to be a little bit advanced to a lot non-english speakers (including me). I think Xuefu raised a good alternative: UNKNOWN. What do you think about it? Best, Kurt On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek wrote: I prefer OPAQUE compared to ANY because any is often the root object in an object hierarchy and would indicate to users the wrong thing. Aljoscha On 18. Oct 2019, at 18:41, Xuefu Z wrote: Thanks to Timo for bringing up an interesting topic. Personally, "OPAQUE" doesn't seem very intuitive with respect to types. (It suits pretty well to glasses, thought. :)) Anyway, could we just use "UNKNOWN", which is more explicit and true reflects its nature? Thanks, Xuefu On Fri, Oct 18, 2019 at 7:51 AM Timo Walther wrote: Hi everyone, Stephan pointed out that our naming of a generic/blackbox/opaque type in SQL might be not intuitive for users. As the term ANY rather describes a "super-class of all types" which is not the case in our type system. Our current ANY type stands for a type that is just a blackbox within SQL, serialized by some custom serializer, that can only be modified within UDFs. I also gathered feedback from a training instructor and native English speaker (David in CC) where I received the following: "The way I’m thinking about this is this: there’s a concept here that people have to become aware of, which is that Flink SQL is able to operate generically on opaquely typed things — and folks need to be able to connect what they see in code examples, etc. with this concept (which they may be unaware of initially). I feel like ANY misses the mark a little bit, but isn’t particularly bad. I do worry that it may cause some confusion about its purpose and power. I think OPAQUE would more clearly express what’s going on." Also resources like Wikipedia [1] show that this terminology is common: "a data type whose concrete data structure is not defined [...] its values can only be manipulated by calling subroutines that have access to the missing information" I would therefore vote for refactoring the type name because it is not used much yet. Implications are: - a new parser keyword "OPAQUE" and changed SQL parser - changes for logical type root, logical type visitors, and their usages What do you think? Thanks, Timo [1] https://en.wikipedia.org/wiki/Opaque_data_type -- Xuefu Zhang "In Honey We Trust!"
