Re: Confusion about the GeoFunctions

[Julian Hyde]

I can see why you want a Geom in Flink’s namespace. I imagine it’s mainly about 
branding — it’s hard to explain to people why they’re using an interface from a 
module they have not heard of — and a little about control.

You probably want to let people write UDFs that take geometries as arguments 
and return them as results. We haven’t done that in Calcite yet. One of the 
questions in my mind was whether we would have Java types for sub-classes of 
geometry (e.g. point, line, polygon).

A concern is that Flink users will write UDFs that should really be built-in 
functions. Another concern is that Flink developers will implement built-in 
OpenGIS functions in Flink rather than Calcite. Adding a Flink-specific Geom 
type makes it more difficult to migrate functions from Flink to Calcite.

I am fine with you making a protected method public, or whatever. But since 
you're are adding a new use case — I made the method protected because I didn’t 
imagine any method outside the org.apache.calcite.runtime package would call it 
— let’s describe the use case and write tests for it. And while we’re at it, 
let’s talk about the ‘@Experimental’ tag against the GeoFunctions class.

Julian
 

> On May 24, 2018, at 7:22 AM, Xingcan Cui  wrote:
> 
> Hi Julian,
> 
> Sorry for the misunderstanding caused. As Timo said, we don’t really mean to 
> reimplement the functions, but just think of exposing another data type. The 
> protected method seems to be a little bit misleading for us. 
> 
> Besides, if there’s something needs improvement, we will definitely raise 
> issues and try to make contributions here.
> 
> Regards,
> Xingcan
> 
>> On May 24, 2018, at 8:06 PM, Timo Walther  wrote:
>> 
>> Hi Julian,
>> 
>> the Flink community is very thankful for the OpenGIS efforts done by the 
>> Calcite community and I think both project can benefit from it. As Xingcan 
>> mentioned, we are thinking about contributing a GeoOperatorTable similar to 
>> SqlStdOperatorTable. We don't want to reimplement the functions. The only 
>> concern that I raised was about exposing the data type through Flink. 
>> Instead of exposing `org.apache.calcite.runtime.GeoFunctions.Geom` as a type 
>> to our users, I was thinking about a Flink specific type that implements the 
>> interface. Flink also needs to implement serializers for this type.
>> 
>> We saw that GeoFunctions has a `protected bind()` method and were wondering 
>> if one could open the binding/type creation and make it more flexible.
>> 
>> Regards,
>> Timo
>> 
>> 
>> Am 23.05.18 um 19:19 schrieb Julian Hyde:
>>> I am wondering why it is necessary to have a different geometry type in 
>>> Flink than in Calcite.
>>> 
>>> If you think that the implementation in Calcite sucks, then rather than 
>>> making a different one for Flink, how about making a different one for 
>>> Calcite, and use that in Flink? I’m not super-proud of the implementation 
>>> in Calcite; it was the best I could do given the time available.
>>> 
>>> In Calcite we are extremely short of development resources. All of the 
>>> spatial work in Calcite has been done by me, on my own time. It is 
>>> depressing to see someone use it and immediately decide they are going to 
>>> re-implement it all in their own project.
>>> 
>>> Julian
>>> 
>>> 
>>> 
 On May 22, 2018, at 8:28 PM, Xingcan Cui  wrote:
 
 Hi all,
 
 Recently, the flink community aims to add some OpenGIS functions (see
 FLINK-9219 ) provided in
 CALCITE-1968 . For some
 reasons, we plan to implement some Flink Geom types (as illustrated in this
 comment
 ),
 but the private constructor in `o.a.c.r.GeoFunctions` makes that 
 impossible.
 
 We are confused about the protected method `bind()`, as well as other
 private modifiers in `o.a.c.r.GeoFunctions`. I wonder if someone could help
 to give some explanations about that. Do we in the right direction?
 
 Besides, maybe there should be a new Geom operand type in
 `o.a.c.q.t.OperandTypes`?
 
 Thanks,
 Xingcan
>> 
>> 
> 

Re: Confusion about the GeoFunctions

[Xingcan Cui]

Hi Julian,

Sorry for the misunderstanding caused. As Timo said, we don’t really mean to 
reimplement the functions, but just think of exposing another data type. The 
protected method seems to be a little bit misleading for us. 

Besides, if there’s something needs improvement, we will definitely raise 
issues and try to make contributions here.

Regards,
Xingcan

> On May 24, 2018, at 8:06 PM, Timo Walther  wrote:
> 
> Hi Julian,
> 
> the Flink community is very thankful for the OpenGIS efforts done by the 
> Calcite community and I think both project can benefit from it. As Xingcan 
> mentioned, we are thinking about contributing a GeoOperatorTable similar to 
> SqlStdOperatorTable. We don't want to reimplement the functions. The only 
> concern that I raised was about exposing the data type through Flink. Instead 
> of exposing `org.apache.calcite.runtime.GeoFunctions.Geom` as a type to our 
> users, I was thinking about a Flink specific type that implements the 
> interface. Flink also needs to implement serializers for this type.
> 
> We saw that GeoFunctions has a `protected bind()` method and were wondering 
> if one could open the binding/type creation and make it more flexible.
> 
> Regards,
> Timo
> 
> 
> Am 23.05.18 um 19:19 schrieb Julian Hyde:
>> I am wondering why it is necessary to have a different geometry type in 
>> Flink than in Calcite.
>> 
>> If you think that the implementation in Calcite sucks, then rather than 
>> making a different one for Flink, how about making a different one for 
>> Calcite, and use that in Flink? I’m not super-proud of the implementation in 
>> Calcite; it was the best I could do given the time available.
>> 
>> In Calcite we are extremely short of development resources. All of the 
>> spatial work in Calcite has been done by me, on my own time. It is 
>> depressing to see someone use it and immediately decide they are going to 
>> re-implement it all in their own project.
>> 
>> Julian
>> 
>> 
>> 
>>> On May 22, 2018, at 8:28 PM, Xingcan Cui  wrote:
>>> 
>>> Hi all,
>>> 
>>> Recently, the flink community aims to add some OpenGIS functions (see
>>> FLINK-9219 ) provided in
>>> CALCITE-1968 . For some
>>> reasons, we plan to implement some Flink Geom types (as illustrated in this
>>> comment
>>> ),
>>> but the private constructor in `o.a.c.r.GeoFunctions` makes that impossible.
>>> 
>>> We are confused about the protected method `bind()`, as well as other
>>> private modifiers in `o.a.c.r.GeoFunctions`. I wonder if someone could help
>>> to give some explanations about that. Do we in the right direction?
>>> 
>>> Besides, maybe there should be a new Geom operand type in
>>> `o.a.c.q.t.OperandTypes`?
>>> 
>>> Thanks,
>>> Xingcan
> 
> 

Re: Confusion about the GeoFunctions

[Timo Walther]

Hi Julian,

the Flink community is very thankful for the OpenGIS efforts done by the 
Calcite community and I think both project can benefit from it. As 
Xingcan mentioned, we are thinking about contributing a GeoOperatorTable 
similar to SqlStdOperatorTable. We don't want to reimplement the 
functions. The only concern that I raised was about exposing the data 
type through Flink. Instead of exposing 
`org.apache.calcite.runtime.GeoFunctions.Geom` as a type to our users, I 
was thinking about a Flink specific type that implements the interface. 
Flink also needs to implement serializers for this type.


We saw that GeoFunctions has a `protected bind()` method and were 
wondering if one could open the binding/type creation and make it more 
flexible.


Regards,
Timo


Am 23.05.18 um 19:19 schrieb Julian Hyde:

I am wondering why it is necessary to have a different geometry type in Flink 
than in Calcite.

If you think that the implementation in Calcite sucks, then rather than making 
a different one for Flink, how about making a different one for Calcite, and 
use that in Flink? I’m not super-proud of the implementation in Calcite; it was 
the best I could do given the time available.

In Calcite we are extremely short of development resources. All of the spatial 
work in Calcite has been done by me, on my own time. It is depressing to see 
someone use it and immediately decide they are going to re-implement it all in 
their own project.

Julian




On May 22, 2018, at 8:28 PM, Xingcan Cui  wrote:

Hi all,

Recently, the flink community aims to add some OpenGIS functions (see
FLINK-9219 ) provided in
CALCITE-1968 . For some
reasons, we plan to implement some Flink Geom types (as illustrated in this
comment
),
but the private constructor in `o.a.c.r.GeoFunctions` makes that impossible.

We are confused about the protected method `bind()`, as well as other
private modifiers in `o.a.c.r.GeoFunctions`. I wonder if someone could help
to give some explanations about that. Do we in the right direction?

Besides, maybe there should be a new Geom operand type in
`o.a.c.q.t.OperandTypes`?

Thanks,
Xingcan

Re: Confusion about the GeoFunctions

[Julian Hyde]

I am wondering why it is necessary to have a different geometry type in Flink 
than in Calcite.

If you think that the implementation in Calcite sucks, then rather than making 
a different one for Flink, how about making a different one for Calcite, and 
use that in Flink? I’m not super-proud of the implementation in Calcite; it was 
the best I could do given the time available.

In Calcite we are extremely short of development resources. All of the spatial 
work in Calcite has been done by me, on my own time. It is depressing to see 
someone use it and immediately decide they are going to re-implement it all in 
their own project.

Julian



> On May 22, 2018, at 8:28 PM, Xingcan Cui  wrote:
> 
> Hi all,
> 
> Recently, the flink community aims to add some OpenGIS functions (see
> FLINK-9219 ) provided in
> CALCITE-1968 . For some
> reasons, we plan to implement some Flink Geom types (as illustrated in this
> comment
> ),
> but the private constructor in `o.a.c.r.GeoFunctions` makes that impossible.
> 
> We are confused about the protected method `bind()`, as well as other
> private modifiers in `o.a.c.r.GeoFunctions`. I wonder if someone could help
> to give some explanations about that. Do we in the right direction?
> 
> Besides, maybe there should be a new Geom operand type in
> `o.a.c.q.t.OperandTypes`?
> 
> Thanks,
> Xingcan