[
https://issues.apache.org/jira/browse/DRILL-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532697#comment-16532697
]
ASF GitHub Bot commented on DRILL-6519:
---------------------------------------
arina-ielchiieva commented on issue #1331: DRILL-6519: Add String Distance and
Phonetic Functions
URL: https://github.com/apache/drill/pull/1331#issuecomment-402465726
@cgivre the following test has failed:
```
04:58:58.570 [main] ERROR org.apache.drill.TestReporter - Test Failed (d: 0
B(2 B), h: 11.1 MiB(462.5 MiB), nh: -122.3 KiB(198.7 MiB)):
testSoundsLike(org.apache.drill.exec.fn.impl.TestPhoneticFunctions)
java.lang.IndexOutOfBoundsException: DrillBuf[45723], udle: [30229 61..62],
index: 0, length: 4 (expected: range(0, 1))
ledger[34623] allocator: ROOT), isOwning: true, size: 64, references: 1,
life: 19420096047899443..0, allocatorManager: [30228, life:
19420096047893613..0] holds 5 buffers.
DrillBuf[45712], udle: [30229 61..62]
event log for: DrillBuf[45712]
19420096048520530 create()
at org.apache.drill.common.StackTrace.<init>(StackTrace.java:33)
at
org.apache.drill.common.HistoricalLog$Event.<init>(HistoricalLog.java:39)
at
org.apache.drill.common.HistoricalLog.recordEvent(HistoricalLog.java:95)
at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:85)
at
org.apache.drill.exec.memory.AllocationManager$BufferLedger.newDrillBuf(AllocationManager.java:389)
at
org.apache.drill.exec.memory.AllocationManager$BufferLedger.newDrillBuf(AllocationManager.java:368)
at io.netty.buffer.DrillBuf.slice(DrillBuf.java:342)
at io.netty.buffer.DrillBuf.slice(DrillBuf.java:317)
at org.apache.drill.exec.rpc.RpcDecoder.decode(RpcDecoder.java:96)
at org.apache.drill.exec.rpc.RpcDecoder.decode(RpcDecoder.java:35)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
19420096048701912 retain(1)
at org.apache.drill.common.StackTrace.<init>(StackTrace.java:33)
at
org.apache.drill.common.HistoricalLog$Event.<init>(HistoricalLog.java:39)
at
org.apache.drill.common.HistoricalLog.recordEvent(HistoricalLog.java:95)
at io.netty.buffer.DrillBuf.retain(DrillBuf.java:447)
at org.apache.drill.exec.rpc.RpcDecoder.decode(RpcDecoder.java:97)
at org.apache.drill.exec.rpc.RpcDecoder.decode(RpcDecoder.java:35)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
```
Please make sure you run tests before making PR and after each round of
changes after code review. Your tests in Slow category so Travis did not run
them. Since it's UDFs that you are adding, you need build project first and
them run your unit tests (since they are cached) or run all tests during build
which I suspect was not done. I had to redo batch commits because of this
failure :( Now you commit is excluded.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Add String Distance and Phonetic Functions
> ------------------------------------------
>
> Key: DRILL-6519
> URL: https://issues.apache.org/jira/browse/DRILL-6519
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Charles Givre
> Assignee: Charles Givre
> Priority: Major
> Labels: doc-impacting
> Fix For: 1.14.0
>
>
> From a recent project, this collection of functions makes it possible to do
> fuzzy string matching as well as phonetic matching on strings.
>
> The following functions are all phonetic functions and map text to a number
> or string based on how the word sounds. For instance "Jayme" and "Jaime"
> have the same soundex values and hence these functions can be used to match
> similar sounding words.
> * caverphone1( <string> )
> * caverphone2( <string> )
> * cologne_phonetic( <string> )
> * dm_soundex( <string> )
> * double_metaphone(<string>)
> * match_rating_encoder( <string> )
> * metaphone(<string>)
> * nysiis( <string> )
> * refined_soundex(<string>)
> * soundex(<string>)
> Additionally, there is the
> {code:java}
> sounds_like(<string1>,<string2>){code}
> function which can be used to find strings that sound similar. For instance:
>
> {code:java}
> SELECT *
> FROM <data>
> WHERE sounds_like( last_name, 'Gretsky' )
> {code}
> h2. String Distance Functions
> In addition to the phonetic functions, there are a series of distance
> functions which measure the difference between two strings. The functions
> include:
> * cosine_distance(<string1>,<string2>)
> * fuzzy_score(<string1>,<string2>)
> * hamming_distance (<string1>,<string2>)
> * jaccard_distance (<string1>,<string2>)
> * jaro_distance (<string1>,<string2>)
> * levenshtein_distance (<string1>,<string2>)
> * longest_common_substring_distance(<string1>,<string2>)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
