Re: Query hints visible to DSV2 connectors?

2023-08-03 Thread Ryan Blue
You probably want to use data source options. Those get passed through but
can't be set in SQL.

On Wed, Aug 2, 2023 at 5:39 PM Alex Cruise  wrote:

> Hey folks,
>
> I'm adding an optional feature to my DSV2 connector where it can choose
> between a row-based or columnar PartitionReader dynamically depending on a
> query's schema. I'd like to be able to supply a hint at query time that's
> visible to the connector, but at the moment I can't see any way to
> accomplish that.
>
> From what I can see the artifacts produced by the existing hint system [
> https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-hints.html
> or sql("select 1").hint("foo").show()] aren't visible from the
> TableCatalog/Table/ScanBuilder.
>
> I guess I could set a config parameter but I'd rather do this on a
> per-query basis. Any tips?
>
> Thanks!
>
> -0xe1a
>


-- 
Ryan Blue
Tabular


Query hints visible to DSV2 connectors?

2023-08-02 Thread Alex Cruise
Hey folks,

I'm adding an optional feature to my DSV2 connector where it can choose
between a row-based or columnar PartitionReader dynamically depending on a
query's schema. I'd like to be able to supply a hint at query time that's
visible to the connector, but at the moment I can't see any way to
accomplish that.

>From what I can see the artifacts produced by the existing hint system [
https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-hints.html
or sql("select 1").hint("foo").show()] aren't visible from the
TableCatalog/Table/ScanBuilder.

I guess I could set a config parameter but I'd rather do this on a
per-query basis. Any tips?

Thanks!

-0xe1a