[
https://issues.apache.org/jira/browse/DRILL-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17246002#comment-17246002
]
ASF GitHub Bot commented on DRILL-7817:
---------------------------------------
cgivre commented on pull request #2122:
URL: https://github.com/apache/drill/pull/2122#issuecomment-740768976
> So I tried to fix the test which was really hard because the ordering of
the columns in case of a wildcard was based on the hash (i.e. visually random).
>
> * I made the ordering of the columns in case of the wildcard a stable
textual ordering (by means of a TreeMap instead of a HashMap).
> * I found that the Casts were the same for all columns (which is
incorrect, so I fixed that).
Can you clarify this? The unit tests do verify that Drill is producing the
correct data types and they seemed to be working.
> * As a consequence the tests become much easier to read because everything
is sorted.
:-)
>
> At this point I think the type remapping doesn't work.
> I haven't seen any testing code or documentation so I'm not sure.
Can you clarify which type remapping you're referring to? When we did the
refactoring, I added unit tests that check for data types. Most fields are
strings, so they get mapped to `VARCHAR` types. However, the timestamps and
dates get mapped to the appropriate data types. Also, the pieces of the times
also get mapped to `DOUBLE` data types. (I think).
You can confirm that in each unit test where it constructs a schema.
https://github.com/apache/drill/blob/7a614d8ce7a0018c6ffaa12955d56366ee62a6d6/contrib/format-httpd/src/test/java/org/apache/drill/exec/store/httpd/TestHTTPDLogReader.java#L224-L228
> Because of the lack of testing I'm pretty sure one of my changes broke it.
> @cgivre If you can please give me a hint on how you intended it to work I
can pick that up for a test to include.
I actually liked your original implementation with the config option. What
I was thinking however was that if the option was enabled, dissector would only
be added if the user executed a star query OR they included one of the raw
`user-agent` fields in the query.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Add direct Yauaa support for HTTPD Format Plugin.
> -------------------------------------------------
>
> Key: DRILL-7817
> URL: https://issues.apache.org/jira/browse/DRILL-7817
> Project: Apache Drill
> Issue Type: New Feature
> Reporter: Niels Basjes
> Assignee: Niels Basjes
> Priority: Minor
>
> Enhancement of having the Yauaa useragent parser immediately integrated with
> the HTTPD logparser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
