[
https://issues.apache.org/jira/browse/IMPALA-10370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17243882#comment-17243882
]
Yongle Zhang commented on IMPALA-10370:
---------------------------------------
Hi [~tarmstrong],
Thanks for the response.
Is there a way that we can exclude the thrift definitions for the internal
communications you mentioned, but only keep the result for those that matter
(e.g., the TAggregateFunction/TFunction serialized to hive store)? Maybe our
checker would provide more helpful results that way.
> Thrift Message Incompatibility Detector
> ---------------------------------------
>
> Key: IMPALA-10370
> URL: https://issues.apache.org/jira/browse/IMPALA-10370
> Project: IMPALA
> Issue Type: Improvement
> Reporter: junwen yang
> Priority: Major
> Attachments: impala_thrift_incompatibility.txt
>
>
> Regarding the issue IMPALA-8243 caused by the incompatibility of thrift
> message, we have created a static checker which keeps track of the thrift
> file change, and detect potential incompatibility:
> # Add/delete required field. The thrift guidelines suggests _Any new fields
> that you add should be optional_.
> # The tag number of a field has been changed. Also, the thrift guidelines
> suggests _Don’t change the numeric tags for any existing fields_.
> # A required field has been changed to optional, or an optional field has
> been changed to required. According to the guidelines , _*Required Is
> Forever* You should be very careful about marking fields as required. If at
> some point you wish to stop writing or sending a required field, it will be
> problematic to change the field to an optional field — old readers will
> consider messages without this field to be incomplete and may reject or drop
> them unintentionally. You should consider writing application-specific custom
> validation routines for your buffers instead. Some have come to the
> conclusion that using required does more harm than good; they prefer to use
> only optional. However, this view is not universal._
> We have applied our checker on the frequently maintained IMPALA versions:
> refs/tags/2.10.0, refs/tags/2.11.0, refs/tags/2.12.0, refs/tags/2.7.0,
> refs/tags/2.8.0, refs/tags/2.9.0, refs/tags/3.0.0, refs/tags/3.0.1,
> refs/tags/3.1.0, refs/tags/3.2.0, refs/tags/3.3.0, refs/tags/3.4.0, we found
> more than 1000 problems as attached.
> The results reported by our checker got confirmed by developers of HBASE and
> our checker is requested by them, which can be found at HBASE-25340.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
