Hi Paul, That was what I thought as well. The function doesn't seem to work properly either setting positive as 1 or -1. I looked at other UDFs that I’ve written in the past that return Boolean values and the implementation is almost exactly the same, so I’m a bit at a loss on this one. — C
> On Jul 5, 2018, at 16:01, Paul Rogers <[email protected]> wrote: > > Hi Tim, > > My understanding is that Booleans are stored, as you said, as a UInt1 vector. > But, the values are 0 and (positive) 1. As you said, singletonInt will read > this value. If the value is coming back as -1, then this is a bug. I'd be > surprised if it is since I wrote lots of code that assumed the value is 1. > > Given this, it would be a simple addition to the test QueryBuilder to add a > "singletonBoolean" that returns singletonInt() != 0. > > Charles, > You should not set the BitHolder to -1, you should set it to either 0 or > (positive) 1. > > Thanks, > - Paul > > > > On Thursday, July 5, 2018, 8:39:58 AM PDT, Charles Givre > <[email protected]> wrote: > > Hi Timothy, > I am still having issues with the unit tests for this particular function. > The code is below. This function takes the soundex values of two strings and > if they are equal is supposed to return true, if not returns false. I do > this by setting the value of a BitHolder to either -1 or 0. > > The code for the function appears to work, however, when I think what is > causing the error here was the sounds_like function itself. > If I run the query: > select * > from > WHERE soundex( ‘brian’ ) = soundex(`first_name` ) > > This returns results > > However, if I run the query > SELECT * > FROM <Data> > WEHRE sounds_like( ‘brian’, `first_name` ) > I get no results. However, if I run this query: > > SELECT sounds_like( ‘brian’, `first_name` ) > FROM cp.. > > I get trues and falses. I’m not sure why this isn’t working properly and in > the interests of not holding up the works, I’m removing the sounds_like from > the PR, but I’d like to understand how to fix this. > > > @FunctionTemplate(name = "sounds_like", scope = > FunctionTemplate.FunctionScope.SIMPLE, nulls = > FunctionTemplate.NullHandling.NULL_IF_NULL) > public static class SoundsLikeFunction implements DrillSimpleFunc { > > @Param > VarCharHolder rawInput1; > > @Param > VarCharHolder rawInput2; > > @Output > BitHolder out; > > @Override > public void setup() { > } > > @Override > public void eval() { > > String input1 = > org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput1.start, > rawInput1.end, rawInput1.buffer); > String input2 = > org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput2.start, > rawInput2.end, rawInput2.buffer); > > String soundex1 = new > org.apache.commons.codec.language.Soundex().soundex(input1); > String soundex2 = new > org.apache.commons.codec.language.Soundex().soundex(input2); > > boolean result = soundex1.equals(soundex2); > if (result) { > out.value = -1; > } else { > out.value = 0; > } > } > } > > > > > jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) AS > steve FROM (VALUES(1)); > Error: SYSTEM ERROR: IllegalArgumentException: Can not set > org.apache.drill.exec.vector.complex.reader.FieldReader field > org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to > org.apache.drill.exec.expr.holders.BitHolder > > > [Error Id: f15a91b9-ae19-481a-b145-1024709547e9 on 10.1.9.176:31010] > (state=,code=0) > 0: jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) > FROM (VALUES(1)); > Error: SYSTEM ERROR: IllegalArgumentException: Can not set > org.apache.drill.exec.vector.complex.reader.FieldReader field > org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to > org.apache.drill.exec.expr.holders.BitHolder > > > [Error Id: 870a2471-8ab7-4a0f-ab7f-9ff873ba767a on 10.1.9.176:31010] > (state=,code=0) > 0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'Stephen' ) FROM > (VALUES(1)); > +---------+ > | EXPR$0 | > +---------+ > | true | > +---------+ > 1 row selected (0.689 seconds) > 0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'bob' ) FROM > (VALUES(1)); > +---------+ > | EXPR$0 | > +---------+ > | false | > +---------+ > 1 row selected (0.175 seconds) > > > > >> On Jun 28, 2018, at 19:46, Timothy Farkas <[email protected]> wrote: >> >> Hi Charles, >> >> So it is actually supported. Drill's boolean vector is BitVector. >> Internally bits are stored efficiently, but when you fetch a bit from the >> vector it becomes an int, -1 for true and 0 for false. So currently you can >> check this by using singletonInt and comparing against -1 and 0. >> >> Here is a test snippet that does it. We should probably add some >> convenience methods specifically for booleans so this is not so confusing. >> >> public class SimpleQueryTest extends ClusterTest { >> @Rule >> public final BaseDirTestWatcher baseDirTestWatcher = new >> BaseDirTestWatcher(); >> >> @Before >> public void start() throws Exception { >> startCluster(new ClusterFixtureBuilder(baseDirTestWatcher)); >> } >> >> @Test >> public void testBoolean() throws RpcException { >> RowSet rowSet = this.queryBuilder().sql("select position_id, >> cast(false as boolean) from cp.`employee.json`").rowSet(); >> RowSetReader reader = rowSet.reader(); >> reader.next(); >> System.out.println(reader.column(1).scalar().getInt()); >> rowSet.clear(); >> >> rowSet = this.queryBuilder().sql("select position_id, cast(true as >> boolean) from cp.`employee.json`").rowSet(); >> reader = rowSet.reader(); >> reader.next(); >> System.out.println(reader.column(1).scalar().getInt()); >> rowSet.clear(); >> } >> } >> >> >> >> On Thu, Jun 28, 2018 at 4:09 PM, Timothy Farkas <[email protected]> wrote: >> >>> Hi Charles, >>> >>> Currently the RowSetReader doesn't support booleans, but it can be added. >>> I'll try to add it now, and see if it could be done quickly. I'll update >>> you on my progress. >>> >>> Tim >>> >>> On Thu, Jun 28, 2018 at 2:38 PM, Charles Givre <[email protected]> wrote: >>> >>>> Hi Tim, >>>> Could post some sample code as to how to test a SQL query that returns a >>>> Boolean? >>>> —C >>>> >>>>> On Jun 28, 2018, at 17:30, Timothy Farkas <[email protected]> wrote: >>>>> >>>>> - We would have to add a boolean column reader to ColumnAccessors and >>>> wire >>>>> it in and add a getBoolean method to ScalarReader. >>>>> >>>>> - Your example should work as is, ClusterTest has a testBuilder method >>>>> that allows you to use the traditional test builder. Is there something >>>> not >>>>> working with the test builder? >>>>> >>>>> Tim >>>>> >>>>> >>>>> On Thu, Jun 28, 2018 at 12:39 PM, Arina Yelchiyeva < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Tim, >>>>>> >>>>>> it looks like deprecating BaseTestQuery was a little bit pre-mature. >>>>>> For example, from in this PR - https://urldefense.proofpoint. >>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd >>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH >>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb- >>>> BDFaZkRaKQKwt_zl2HA&e=. >>>>>> com/v2/url?u=https-3A__github.com_apache_drill_pull_1331&d=DwIBaQ&c= >>>>>> cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m= >>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=_uxhA_ >>>>>> qiMBTjbit6DDw-DvZNRGesfeA5g-FQjkl7f10&e= - >>>>>> Charles is trying to re-work BaseTestQuery usage to ClusterTest. >>>>>> First, it did not contain getSigletonDouble method which Charles has >>>>>> implemented. Now he has troubles with implementing getSigletonBoolean >>>>>> method which might be due to reader limitations. >>>>>> Also I am not quite clear how we can verify columns names and multiple >>>>>> columns in the result. >>>>>> For example: >>>>>> >>>>>> testBuilder() >>>>>> .sqlQuery("select (mi || lname) as CONCATOperator, mi, lname, >>>>>> concat(mi, lname) as CONCAT from concatNull") >>>>>> .ordered() >>>>>> .baselineColumns("CONCATOperator", "mi", "lname", "CONCAT") >>>>>> .baselineValues("A.Nowmer", "A.", "Nowmer", "A.Nowmer") >>>>>> .baselineValues("I.Whelply", "I.", "Whelply", "I.Whelply") >>>>>> .baselineValues(null, null, "Derry", "Derry") >>>>>> .baselineValues("J.Spence", "J.", "Spence", "J.Spence") >>>>>> .build().run(); >>>>>> >>>>>> Can you please suggest how this example can be re-written? >>>>>> >>>>>> Kind regards, >>>>>> Arina >>>>>> >>>>>> On Mon, Jun 25, 2018 at 11:10 PM Timothy Farkas <[email protected]> >>>> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> BaseTestQuery was deprecated a while ago. Keeping it short and sweet >>>> :), >>>>>> if >>>>>>> you want to use BaseTestQuery directly, don't. Use ClusterTest >>>> instead. >>>>>> If >>>>>>> you are using PlanTestBase for planner tests, continue to do so. >>>>>> Eventually >>>>>>> PlanTestBase will be changed to extend ClusterTest instead. There is a >>>>>> JIRA >>>>>>> to track that issue https://urldefense.proofpoint. >>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd >>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH >>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb- >>>> BDFaZkRaKQKwt_zl2HA&e=. >>>>>> com/v2/url?u=https-3A__issues.apache.org_jira_browse_DRILL- >>>>>> 2D6536&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m= >>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s= >>>>>> BPAlenAq0k1kjAz7fUYXyEQMaJM1IPOgmdeySMMY84U&e=. >>>>>>> >>>>>>> Thanks, >>>>>>> Tim >>>>>>> >>>>>> >>>> >>>> >>>
