Re: CsvAdapter (Json content) from String / InputStream
Pls see CALCITE-3560 [1] which was merged recently. You can now instantiate
Source from a generic CharSource [2]:
Source source = Sources.fromCharSource(CharSource.wrap("hello"));
This should be released with 1.22
[1] https://issues.apache.org/jira/browse/CALCITE-3560
[2]
https://guava.dev/releases/19.0/api/docs/com/google/common/io/CharSource.html
On Tue, Nov 19, 2019 at 1:07 PM Yanna elina
wrote:
> Hi guys ,
>
> i have another question about this adapter.
> I have a json sample like this :
> [
> {
> "field": "test",
> "properties": [
> "a",
> "b",
> "c"
> ]
> }
> ]
>
> JsonTable look like to convert Array to OTHER ColumType .
> the column "properties" is converted on ' JAVA.UTIL.ARRAYLIST)>
> when i check JsonEnumerator i can see this function code :*RelDataType
> type = typeFactory.createJavaType(jsonFieldMap.get(key).getClass());*
>
> *About WHERE / INis it supposed to work? on JavaType(ArrayList) ? *
> if i try to make this query "SELECT * FROM TABLE_A WHERE
> properties=any('a') i will have alway an Exception
> : org.apache.calcite.sql.validate.SqlValidatorException: Values passed to
> IN operator must have compatible type
>
> *if its not supposed to work i guess i need to re-implement this one to
> convert *JAVA.UTIL.ARRAYLIST on SQL_ARRAY type right ?
>
> thank
>
>
>
> Le lun. 18 nov. 2019 à 19:15, Yanna elina a
> écrit :
>
> > Big Thank Guys !!!
> >
> > Even if this CSVAdapter is an "example" its still really usefull :) and
> > with this update it will be a good "out of box" tools usable in many
> > scenario/workflow
> >
> >
> > Yanna
> >
> > Le jeu. 14 nov. 2019 à 21:20, Andrei Sereda a écrit :
> >
> >> > I see that this feature request relates to Source.java and
> Sources.java,
> >> which are in org.apache.calcite.util in core.
> >> I'm not planning to change Source.java it already exposes Reader /
> >> InputStream
> >>
> >> > If you add some capability, it is fine to add It to the CSV adapter
> >> example, but it is much more important that that capability exists in
> the
> >> file adapter.
> >> I will add to both. The general idea behind this change is that
> currently
> >> CSV / File Adapter(s) require inputs to be File(s) or URL(s) which
> forces
> >> users to create temporal resources manually (when their content is
> already
> >> in-memory). If input to CSV / File adapter(s) is generic Readable [1] /
> >> Reader [2] or InputStream it gives more flexibility to users.
> >>
> >> [1] https://docs.oracle.com/javase/7/docs/api/java/lang/Readable.html
> >> [2] https://docs.oracle.com/javase/7/docs/api/java/io/Reader.html
> >>
> >>
> >> On Thu, Nov 14, 2019 at 2:45 PM Julian Hyde wrote:
> >>
> >> > I see that this feature request relates to Source.java and
> Sources.java,
> >> > which are in org.apache.calcite.util in core.
> >> >
> >> > If you add some capability, it is fine to add It to the CSV adapter
> >> > example, but it is much more important that that capability exists in
> >> the
> >> > file adapter.
> >> >
> >> > Julian
> >> >
> >> >
> >> > > On Nov 14, 2019, at 11:36 AM, Andrei Sereda
> wrote:
> >> > >
> >> > > I think the change is straightforward (will not add complexity).
> >> > >
> >> > > On Thu, Nov 14, 2019 at 1:24 PM Julian Hyde
> wrote:
> >> > >
> >> > >> Remember that CsvAdapter is in the “example” module. Keep it
> simple.
> >> > >>
> >> > >> The file adapter can also parse CSV files.
> >> > >>
> >> > >> Julian
> >> > >>
> >> > >>
> >> > >>
> >> > >>> On Nov 14, 2019, at 9:40 AM, Andrei Sereda
> >> wrote:
> >> > >>>
> >> > >>> Hello,
> >> > >>>
> >> > >>> Source object already exposes Reader / InputStream API. Probably
> >> > >>> JsonEnumerator can be changed to use those methods.
> >> > >>>
> >> > >>> Do you mind creating a JIRA ticket ? I'll take a look.
> >> > >>>
> >> > >>> Thanks,
> >> > >>> Andrei.
> >> > >>>
> >> > >>> On Thu, Nov 14, 2019 at 7:45 AM Yanna elina <
> >> > yannaelinasul...@gmail.com>
> >> > >>> wrote:
> >> > >>>
> >> > Hi guys ,
> >> > I saw in the code that this nice adapter makes it possible to
> make
> >> SQL
> >> > queries on the data JSON
> >> >
> >> >
> >> > >>
> >> >
> >>
> https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv
> >> >
> >> > But it's limited on File / URL.
> >> > JsonTable constructor accepte only a Source object and this
> Source
> >> > >> object
> >> > can be construct only accross a File / URL.
> >> >
> >> > it could be nice to have the possibility to make this source from
> >> > ImputStream too .
> >> >
> >> > Creating a temp-file from an InputStream or String can be
> excesive.
> >> >
> >> > Thanks
> >> >
> >> > >>
> >> > >>
> >> >
> >> >
> >>
> >
>
Re: CsvAdapter (Json content) from String / InputStream
Hi guys ,
i have another question about this adapter.
I have a json sample like this :
[
{
"field": "test",
"properties": [
"a",
"b",
"c"
]
}
]
JsonTable look like to convert Array to OTHER ColumType .
the column "properties" is converted on '
when i check JsonEnumerator i can see this function code :*RelDataType
type = typeFactory.createJavaType(jsonFieldMap.get(key).getClass());*
*About WHERE / INis it supposed to work? on JavaType(ArrayList) ? *
if i try to make this query "SELECT * FROM TABLE_A WHERE
properties=any('a') i will have alway an Exception
: org.apache.calcite.sql.validate.SqlValidatorException: Values passed to
IN operator must have compatible type
*if its not supposed to work i guess i need to re-implement this one to
convert *JAVA.UTIL.ARRAYLIST on SQL_ARRAY type right ?
thank
Le lun. 18 nov. 2019 à 19:15, Yanna elina a
écrit :
> Big Thank Guys !!!
>
> Even if this CSVAdapter is an "example" its still really usefull :) and
> with this update it will be a good "out of box" tools usable in many
> scenario/workflow
>
>
> Yanna
>
> Le jeu. 14 nov. 2019 à 21:20, Andrei Sereda a écrit :
>
>> > I see that this feature request relates to Source.java and Sources.java,
>> which are in org.apache.calcite.util in core.
>> I'm not planning to change Source.java it already exposes Reader /
>> InputStream
>>
>> > If you add some capability, it is fine to add It to the CSV adapter
>> example, but it is much more important that that capability exists in the
>> file adapter.
>> I will add to both. The general idea behind this change is that currently
>> CSV / File Adapter(s) require inputs to be File(s) or URL(s) which forces
>> users to create temporal resources manually (when their content is already
>> in-memory). If input to CSV / File adapter(s) is generic Readable [1] /
>> Reader [2] or InputStream it gives more flexibility to users.
>>
>> [1] https://docs.oracle.com/javase/7/docs/api/java/lang/Readable.html
>> [2] https://docs.oracle.com/javase/7/docs/api/java/io/Reader.html
>>
>>
>> On Thu, Nov 14, 2019 at 2:45 PM Julian Hyde wrote:
>>
>> > I see that this feature request relates to Source.java and Sources.java,
>> > which are in org.apache.calcite.util in core.
>> >
>> > If you add some capability, it is fine to add It to the CSV adapter
>> > example, but it is much more important that that capability exists in
>> the
>> > file adapter.
>> >
>> > Julian
>> >
>> >
>> > > On Nov 14, 2019, at 11:36 AM, Andrei Sereda wrote:
>> > >
>> > > I think the change is straightforward (will not add complexity).
>> > >
>> > > On Thu, Nov 14, 2019 at 1:24 PM Julian Hyde wrote:
>> > >
>> > >> Remember that CsvAdapter is in the “example” module. Keep it simple.
>> > >>
>> > >> The file adapter can also parse CSV files.
>> > >>
>> > >> Julian
>> > >>
>> > >>
>> > >>
>> > >>> On Nov 14, 2019, at 9:40 AM, Andrei Sereda
>> wrote:
>> > >>>
>> > >>> Hello,
>> > >>>
>> > >>> Source object already exposes Reader / InputStream API. Probably
>> > >>> JsonEnumerator can be changed to use those methods.
>> > >>>
>> > >>> Do you mind creating a JIRA ticket ? I'll take a look.
>> > >>>
>> > >>> Thanks,
>> > >>> Andrei.
>> > >>>
>> > >>> On Thu, Nov 14, 2019 at 7:45 AM Yanna elina <
>> > yannaelinasul...@gmail.com>
>> > >>> wrote:
>> > >>>
>> > Hi guys ,
>> > I saw in the code that this nice adapter makes it possible to make
>> SQL
>> > queries on the data JSON
>> >
>> >
>> > >>
>> >
>> https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv
>> >
>> > But it's limited on File / URL.
>> > JsonTable constructor accepte only a Source object and this Source
>> > >> object
>> > can be construct only accross a File / URL.
>> >
>> > it could be nice to have the possibility to make this source from
>> > ImputStream too .
>> >
>> > Creating a temp-file from an InputStream or String can be excesive.
>> >
>> > Thanks
>> >
>> > >>
>> > >>
>> >
>> >
>>
>
Re: CsvAdapter (Json content) from String / InputStream
Big Thank Guys !!! Even if this CSVAdapter is an "example" its still really usefull :) and with this update it will be a good "out of box" tools usable in many scenario/workflow Yanna Le jeu. 14 nov. 2019 à 21:20, Andrei Sereda a écrit : > > I see that this feature request relates to Source.java and Sources.java, > which are in org.apache.calcite.util in core. > I'm not planning to change Source.java it already exposes Reader / > InputStream > > > If you add some capability, it is fine to add It to the CSV adapter > example, but it is much more important that that capability exists in the > file adapter. > I will add to both. The general idea behind this change is that currently > CSV / File Adapter(s) require inputs to be File(s) or URL(s) which forces > users to create temporal resources manually (when their content is already > in-memory). If input to CSV / File adapter(s) is generic Readable [1] / > Reader [2] or InputStream it gives more flexibility to users. > > [1] https://docs.oracle.com/javase/7/docs/api/java/lang/Readable.html > [2] https://docs.oracle.com/javase/7/docs/api/java/io/Reader.html > > > On Thu, Nov 14, 2019 at 2:45 PM Julian Hyde wrote: > > > I see that this feature request relates to Source.java and Sources.java, > > which are in org.apache.calcite.util in core. > > > > If you add some capability, it is fine to add It to the CSV adapter > > example, but it is much more important that that capability exists in the > > file adapter. > > > > Julian > > > > > > > On Nov 14, 2019, at 11:36 AM, Andrei Sereda wrote: > > > > > > I think the change is straightforward (will not add complexity). > > > > > > On Thu, Nov 14, 2019 at 1:24 PM Julian Hyde wrote: > > > > > >> Remember that CsvAdapter is in the “example” module. Keep it simple. > > >> > > >> The file adapter can also parse CSV files. > > >> > > >> Julian > > >> > > >> > > >> > > >>> On Nov 14, 2019, at 9:40 AM, Andrei Sereda wrote: > > >>> > > >>> Hello, > > >>> > > >>> Source object already exposes Reader / InputStream API. Probably > > >>> JsonEnumerator can be changed to use those methods. > > >>> > > >>> Do you mind creating a JIRA ticket ? I'll take a look. > > >>> > > >>> Thanks, > > >>> Andrei. > > >>> > > >>> On Thu, Nov 14, 2019 at 7:45 AM Yanna elina < > > yannaelinasul...@gmail.com> > > >>> wrote: > > >>> > > Hi guys , > > I saw in the code that this nice adapter makes it possible to make > SQL > > queries on the data JSON > > > > > > >> > > > https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv > > > > But it's limited on File / URL. > > JsonTable constructor accepte only a Source object and this Source > > >> object > > can be construct only accross a File / URL. > > > > it could be nice to have the possibility to make this source from > > ImputStream too . > > > > Creating a temp-file from an InputStream or String can be excesive. > > > > Thanks > > > > >> > > >> > > > > >
Re: CsvAdapter (Json content) from String / InputStream
> I see that this feature request relates to Source.java and Sources.java, which are in org.apache.calcite.util in core. I'm not planning to change Source.java it already exposes Reader / InputStream > If you add some capability, it is fine to add It to the CSV adapter example, but it is much more important that that capability exists in the file adapter. I will add to both. The general idea behind this change is that currently CSV / File Adapter(s) require inputs to be File(s) or URL(s) which forces users to create temporal resources manually (when their content is already in-memory). If input to CSV / File adapter(s) is generic Readable [1] / Reader [2] or InputStream it gives more flexibility to users. [1] https://docs.oracle.com/javase/7/docs/api/java/lang/Readable.html [2] https://docs.oracle.com/javase/7/docs/api/java/io/Reader.html On Thu, Nov 14, 2019 at 2:45 PM Julian Hyde wrote: > I see that this feature request relates to Source.java and Sources.java, > which are in org.apache.calcite.util in core. > > If you add some capability, it is fine to add It to the CSV adapter > example, but it is much more important that that capability exists in the > file adapter. > > Julian > > > > On Nov 14, 2019, at 11:36 AM, Andrei Sereda wrote: > > > > I think the change is straightforward (will not add complexity). > > > > On Thu, Nov 14, 2019 at 1:24 PM Julian Hyde wrote: > > > >> Remember that CsvAdapter is in the “example” module. Keep it simple. > >> > >> The file adapter can also parse CSV files. > >> > >> Julian > >> > >> > >> > >>> On Nov 14, 2019, at 9:40 AM, Andrei Sereda wrote: > >>> > >>> Hello, > >>> > >>> Source object already exposes Reader / InputStream API. Probably > >>> JsonEnumerator can be changed to use those methods. > >>> > >>> Do you mind creating a JIRA ticket ? I'll take a look. > >>> > >>> Thanks, > >>> Andrei. > >>> > >>> On Thu, Nov 14, 2019 at 7:45 AM Yanna elina < > yannaelinasul...@gmail.com> > >>> wrote: > >>> > Hi guys , > I saw in the code that this nice adapter makes it possible to make SQL > queries on the data JSON > > > >> > https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv > > But it's limited on File / URL. > JsonTable constructor accepte only a Source object and this Source > >> object > can be construct only accross a File / URL. > > it could be nice to have the possibility to make this source from > ImputStream too . > > Creating a temp-file from an InputStream or String can be excesive. > > Thanks > > >> > >> > >
Re: CsvAdapter (Json content) from String / InputStream
I see that this feature request relates to Source.java and Sources.java, which are in org.apache.calcite.util in core. If you add some capability, it is fine to add It to the CSV adapter example, but it is much more important that that capability exists in the file adapter. Julian > On Nov 14, 2019, at 11:36 AM, Andrei Sereda wrote: > > I think the change is straightforward (will not add complexity). > > On Thu, Nov 14, 2019 at 1:24 PM Julian Hyde wrote: > >> Remember that CsvAdapter is in the “example” module. Keep it simple. >> >> The file adapter can also parse CSV files. >> >> Julian >> >> >> >>> On Nov 14, 2019, at 9:40 AM, Andrei Sereda wrote: >>> >>> Hello, >>> >>> Source object already exposes Reader / InputStream API. Probably >>> JsonEnumerator can be changed to use those methods. >>> >>> Do you mind creating a JIRA ticket ? I'll take a look. >>> >>> Thanks, >>> Andrei. >>> >>> On Thu, Nov 14, 2019 at 7:45 AM Yanna elina >>> wrote: >>> Hi guys , I saw in the code that this nice adapter makes it possible to make SQL queries on the data JSON >> https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv But it's limited on File / URL. JsonTable constructor accepte only a Source object and this Source >> object can be construct only accross a File / URL. it could be nice to have the possibility to make this source from ImputStream too . Creating a temp-file from an InputStream or String can be excesive. Thanks >> >>
Re: CsvAdapter (Json content) from String / InputStream
I think the change is straightforward (will not add complexity). On Thu, Nov 14, 2019 at 1:24 PM Julian Hyde wrote: > Remember that CsvAdapter is in the “example” module. Keep it simple. > > The file adapter can also parse CSV files. > > Julian > > > > > On Nov 14, 2019, at 9:40 AM, Andrei Sereda wrote: > > > > Hello, > > > > Source object already exposes Reader / InputStream API. Probably > > JsonEnumerator can be changed to use those methods. > > > > Do you mind creating a JIRA ticket ? I'll take a look. > > > > Thanks, > > Andrei. > > > > On Thu, Nov 14, 2019 at 7:45 AM Yanna elina > > wrote: > > > >> Hi guys , > >> I saw in the code that this nice adapter makes it possible to make SQL > >> queries on the data JSON > >> > >> > https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv > >> > >> But it's limited on File / URL. > >> JsonTable constructor accepte only a Source object and this Source > object > >> can be construct only accross a File / URL. > >> > >> it could be nice to have the possibility to make this source from > >> ImputStream too . > >> > >> Creating a temp-file from an InputStream or String can be excesive. > >> > >> Thanks > >> > >
Re: CsvAdapter (Json content) from String / InputStream
Remember that CsvAdapter is in the “example” module. Keep it simple. The file adapter can also parse CSV files. Julian > On Nov 14, 2019, at 9:40 AM, Andrei Sereda wrote: > > Hello, > > Source object already exposes Reader / InputStream API. Probably > JsonEnumerator can be changed to use those methods. > > Do you mind creating a JIRA ticket ? I'll take a look. > > Thanks, > Andrei. > > On Thu, Nov 14, 2019 at 7:45 AM Yanna elina > wrote: > >> Hi guys , >> I saw in the code that this nice adapter makes it possible to make SQL >> queries on the data JSON >> >> https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv >> >> But it's limited on File / URL. >> JsonTable constructor accepte only a Source object and this Source object >> can be construct only accross a File / URL. >> >> it could be nice to have the possibility to make this source from >> ImputStream too . >> >> Creating a temp-file from an InputStream or String can be excesive. >> >> Thanks >>
Re: CsvAdapter (Json content) from String / InputStream
Hello, Source object already exposes Reader / InputStream API. Probably JsonEnumerator can be changed to use those methods. Do you mind creating a JIRA ticket ? I'll take a look. Thanks, Andrei. On Thu, Nov 14, 2019 at 7:45 AM Yanna elina wrote: > Hi guys , > I saw in the code that this nice adapter makes it possible to make SQL > queries on the data JSON > > https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv > > But it's limited on File / URL. > JsonTable constructor accepte only a Source object and this Source object > can be construct only accross a File / URL. > > it could be nice to have the possibility to make this source from > ImputStream too . > > Creating a temp-file from an InputStream or String can be excesive. > > Thanks >
CsvAdapter (Json content) from String / InputStream
Hi guys , I saw in the code that this nice adapter makes it possible to make SQL queries on the data JSON https://github.com/apache/calcite/tree/ab71c4cae5a5c3c7d979337a2d38ddaf271aa206/example/csv/src/main/java/org/apache/calcite/adapter/csv But it's limited on File / URL. JsonTable constructor accepte only a Source object and this Source object can be construct only accross a File / URL. it could be nice to have the possibility to make this source from ImputStream too . Creating a temp-file from an InputStream or String can be excesive. Thanks
