Platform providers may want to disable hints completely for security
reasons.
Currently, there is a configuration to disable OPTIONS hint -
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/config/#table-dynamic-table-options-enabled
However, there is no configuration available
Hi,
Thanks for driving this proposal.
Can you explain why you would need to disable query hints because of
security issues? I don't really understand why query hints affects security.
Best,
Ron
Bonnie Arogyam Varghese 于2023年8月16日周三
23:59写道：
> Platform providers may want to disable hints compl
Hi Liu,
Options hints could be a security concern since users can override
settings. However, query hints specifically could affect performance.
Since we have a config to disable Options hint, I'm suggesting we also have
a config to disable Query hints.
On Wed, Aug 16, 2023 at 9:41 AM liu ron wr
Hi Bonnie,
this makes sense to me, in particular, given that we already have this
toggle for a different type of hints.
Best,
Konstantin
Am Mi., 16. Aug. 2023 um 19:38 Uhr schrieb Bonnie Arogyam Varghese
:
> Hi Liu,
> Options hints could be a security concern since users can override
> settin
+1 for this proposal.
Not every data team would like to enable hints. Also because they are an
extension to the SQL standard. It might also be the case that custom
rules would be overwritten otherwise. Setting hints could also be the
exclusive task of a DevOp team.
Regards,
Timo
On 17.08.2
Sorry, I still don't understand why we need to disable the query hint.
It doesn't have the security problems as options hint. Bonnie said it
could affect performance, but that depends on users using it explicitly.
If there is any performance problem, users can remove the hint.
If we want to disabl
Hi, Bonnie
> Options hints could be a security concern since users can override
settings.
I think this still doesn't answer my question
Best,
Ron
Jark Wu 于2023年8月17日周四 19:51写道：
> Sorry, I still don't understand why we need to disable the query hint.
> It doesn't have the security problems as
> lots of the streaming SQL syntax are extensions of SQL standard
That is true. But hints are kind of a special case because they are not
even "part of Flink SQL" that's why they are written in a comment syntax.
Anyway, I feel hints could be sometimes confusing for users because most
of them
It looks like it will be nice to have a config to disable hints. Any other
thoughts/concerns before we can close this discussion?
On Fri, Aug 18, 2023 at 7:43 AM Timo Walther wrote:
> > lots of the streaming SQL syntax are extensions of SQL standard
>
> That is true. But hints are kind of a spe
Hi Liu Ron,
To answer your question,
Security might not be the main reason for disabling this option but
other arguments brought forward by Timo. Let me know if you have any
further questions or concerns.
On Tue, Sep 5, 2023 at 9:35 PM Bonnie Arogyam Varghese <
bvargh...@confluent.io> wrote:
Hi, Boonie
I'm with Jark on why disable hint is needed if it won't affect security. If
users don't need to use hint, then they won't care about it and I don't
think it's going to be a nuisance. On top of that, Lookup Join Hint is very
useful for streaming jobs, and disabling the hint would result
Hi Liu,
The default will be set to enabled which is the current behavior. The
option will allow users/platform providers to disable it if they want to.
On Wed, Sep 6, 2023 at 6:39 PM liu ron wrote:
> Hi, Boonie
>
> I'm with Jark on why disable hint is needed if it won't affect security. If
> us
Hi,
Thanks for bringing this to our attention. At the first glance, it looks
reasonable to offer a new configuration to enable/disable SQL hints
globally. However, IMHO, it is not the right timing to do it now, because
we should not only think as platform providers but also as end users(the
number
I agree with Jing,
My biggest concern is this makes the boundary of adding an option very
unclear.
It's not a strong reason to add a config just because of it doesn't affect
existing
users. Does this mean that in the future we might add an option to disable
each feature?
Flink already has a very
Hi Jing and Jark!
I can definitely appreciate the desire to have fewer configurations.
Do you have a suggested alternative for platform providers to limit or
restrict the hints that Bonnie is talking about?
As one possibility, maybe one configuration could be set to control all
hints.
Cheers,
Hi all,
I think Jark has a valid point with:
> Does this mean that in the future we might add an option to disable each
feature?
I don't think that's a reasonable outcome indeed, but we are currently in a
situation where we don't have clear guidelines on when to add a
configuration option, and w
Hi Martijn,
I can understand that.
Is there any database/system that supports disabling/enabling query hints
with a configuration? This might help us to understand the use case,
and follow the approach.
Best,
Jark
On Fri, 15 Sept 2023 at 15:34, Martijn Visser
wrote:
> Hi all,
>
> I think Jark
Hi Jark,
Oracle has/had a generic "OPTIMIZER_IGNORE_HINTS" [1]. It looks like MSSQL
has configuration options to disable specific hints [2] instead of a
generic solution.
[1]
https://docs.oracle.com/en/database/oracle/oracle-database/23/refrn/OPTIMIZER_IGNORE_HINTS.html#GUID-D62CA6D8-D0D8-4A20-93
Hi Martijn,
Thanks for the investigation. I found the blog[1] shows a use case
that they use "OPTIMIZER_IGNORE_HINTS" to check if embedded
hints can be removed in legacy code. I think this is a useful tool to
improve queries without complex hints strewn throughout the code.
Therefore, I'm fine to
Hey Jark,
Sounds fine to me.
@Bonnie does that also work for you?
Best regards,
Martijn
On Fri, Sep 15, 2023 at 1:28 PM Jark Wu wrote:
>
> Hi Martijn,
>
> Thanks for the investigation. I found the blog[1] shows a use case
> that they use "OPTIMIZER_IGNORE_HINTS" to check if embedded
> hints ca
Hi Martjin,
Yes, the suggestion for the configuration name made by Jark sounds good.
Also, thanks to everyone who participated in this discussion.
On Tue, Sep 19, 2023 at 2:40 PM Martijn Visser
wrote:
> Hey Jark,
>
> Sounds fine to me.
> @Bonnie does that also work for you?
>
> Best regards,
>
Hi Jark,
A minor suggestion. Would it be ok if we changed the config name to `
table.optimizer.query-options.enabled`?
This is inline with other existing configurations such as:
table.dynamic-table-options.enabled
table.optimizer.distinct-agg.split.enabled
table.optimizer.dynamic-filtering.enable
Hi Bonnie,
I think changing it to something like .enabled
could lead to misunderstanding
for instance when we set this flag to false what should it mean?
1. Just ignore hints and execute a query like the same query however with
removed hints
2. Fail on validation because hints are disabled
3. some
Hi,
I have the same concern as Sergey and would like to make sure the purpose
of this discussion is to globally ignore hints without changing any other
behaviours, if I am not mistaken. Thanks!
Best regards,
Jing
On Mon, Oct 2, 2023 at 3:40 PM Sergey Nuyanzin wrote:
> Hi Bonnie,
>
> I think ch
Hey all,
My understanding was that from the first message we were discussing
throwing an exception. Oracle was only shown as an example of a system
that have a flag for hints behaviour.
Let's get back to the discussion and agree on the behavior. My
suggestion is to introduce an enum instead o
Hi Dawid,
Thanks for bringing this.
I would agree with enum approach
ignored option would allow to follow Oracle's behavior as well
>table.optimizer.query-options = ENABLED/DISABLED/IGNORED
nit: Can we have "hint" in config option name
e.g. table.optimizer.query-options-hints ?
On Tue, Oct 3,
Hi Dawid,
Thanks for the clarification. If you could go through the discussion, you
would be aware that the focus has been moved from "disable" to "ignore".
There was an alignment only on "ignore hints". Your suggestion bypassed the
alignment and mixed everything together. That confused me a bit.
Hey Jing,
If you went through the discussion, you would see it has never been
shifted towards "ignore". The only concern in the discussion was we'd
have too many options and that lookup joins require them. It was never
questioned we should not throw an exception that was suggested in the
firs
Hi Dawid,
Please don't get me wrong. I just described the facts, shared different
opinions, and tried to make sure we are on the same page. My intention is
clearly not to block your effort. If you, after hearing all the different
opinions, still think your solution is the right approach, please go
29 matches
Mail list logo
