Hi Tim,
My understanding is that Booleans are stored, as you said, as a UInt1 vector.
But, the values are 0 and (positive) 1. As you said, singletonInt will read
this value. If the value is coming back as -1, then this is a bug. I'd be
surprised if it is since I wrote lots of code that assumed the value is 1.
Given this, it would be a simple addition to the test QueryBuilder to add a
"singletonBoolean" that returns singletonInt() != 0.
Charles,
You should not set the BitHolder to -1, you should set it to either 0 or
(positive) 1.
Thanks,
- Paul
On Thursday, July 5, 2018, 8:39:58 AM PDT, Charles Givre <[email protected]>
wrote:
Hi Timothy,
I am still having issues with the unit tests for this particular function. The
code is below. This function takes the soundex values of two strings and if
they are equal is supposed to return true, if not returns false. I do this by
setting the value of a BitHolder to either -1 or 0.
The code for the function appears to work, however, when I think what is
causing the error here was the sounds_like function itself.
If I run the query:
select *
from
WHERE soundex( ‘brian’ ) = soundex(`first_name` )
This returns results
However, if I run the query
SELECT *
FROM <Data>
WEHRE sounds_like( ‘brian’, `first_name` )
I get no results. However, if I run this query:
SELECT sounds_like( ‘brian’, `first_name` )
FROM cp..
I get trues and falses. I’m not sure why this isn’t working properly and in
the interests of not holding up the works, I’m removing the sounds_like from
the PR, but I’d like to understand how to fix this.
@FunctionTemplate(name = "sounds_like", scope =
FunctionTemplate.FunctionScope.SIMPLE, nulls =
FunctionTemplate.NullHandling.NULL_IF_NULL)
public static class SoundsLikeFunction implements DrillSimpleFunc {
@Param
VarCharHolder rawInput1;
@Param
VarCharHolder rawInput2;
@Output
BitHolder out;
@Override
public void setup() {
}
@Override
public void eval() {
String input1 =
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput1.start,
rawInput1.end, rawInput1.buffer);
String input2 =
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.toStringFromUTF8(rawInput2.start,
rawInput2.end, rawInput2.buffer);
String soundex1 = new
org.apache.commons.codec.language.Soundex().soundex(input1);
String soundex2 = new
org.apache.commons.codec.language.Soundex().soundex(input2);
boolean result = soundex1.equals(soundex2);
if (result) {
out.value = -1;
} else {
out.value = 0;
}
}
}
jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) AS steve
FROM (VALUES(1));
Error: SYSTEM ERROR: IllegalArgumentException: Can not set
org.apache.drill.exec.vector.complex.reader.FieldReader field
org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to
org.apache.drill.exec.expr.holders.BitHolder
[Error Id: f15a91b9-ae19-481a-b145-1024709547e9 on 10.1.9.176:31010]
(state=,code=0)
0: jdbc:drill:zk=local> SELECT typeof(sounds_like( 'Steven', 'Stephen' )) FROM
(VALUES(1));
Error: SYSTEM ERROR: IllegalArgumentException: Can not set
org.apache.drill.exec.vector.complex.reader.FieldReader field
org.apache.drill.exec.expr.fn.impl.UnionFunctions$GetType.input to
org.apache.drill.exec.expr.holders.BitHolder
[Error Id: 870a2471-8ab7-4a0f-ab7f-9ff873ba767a on 10.1.9.176:31010]
(state=,code=0)
0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'Stephen' ) FROM
(VALUES(1));
+---------+
| EXPR$0 |
+---------+
| true |
+---------+
1 row selected (0.689 seconds)
0: jdbc:drill:zk=local> SELECT sounds_like( 'Steven', 'bob' ) FROM (VALUES(1));
+---------+
| EXPR$0 |
+---------+
| false |
+---------+
1 row selected (0.175 seconds)
> On Jun 28, 2018, at 19:46, Timothy Farkas <[email protected]> wrote:
>
> Hi Charles,
>
> So it is actually supported. Drill's boolean vector is BitVector.
> Internally bits are stored efficiently, but when you fetch a bit from the
> vector it becomes an int, -1 for true and 0 for false. So currently you can
> check this by using singletonInt and comparing against -1 and 0.
>
> Here is a test snippet that does it. We should probably add some
> convenience methods specifically for booleans so this is not so confusing.
>
> public class SimpleQueryTest extends ClusterTest {
> @Rule
> public final BaseDirTestWatcher baseDirTestWatcher = new
>BaseDirTestWatcher();
>
> @Before
> public void start() throws Exception {
> startCluster(new ClusterFixtureBuilder(baseDirTestWatcher));
> }
>
> @Test
> public void testBoolean() throws RpcException {
> RowSet rowSet = this.queryBuilder().sql("select position_id,
> cast(false as boolean) from cp.`employee.json`").rowSet();
> RowSetReader reader = rowSet.reader();
> reader.next();
> System.out.println(reader.column(1).scalar().getInt());
> rowSet.clear();
>
> rowSet = this.queryBuilder().sql("select position_id, cast(true as
> boolean) from cp.`employee.json`").rowSet();
> reader = rowSet.reader();
> reader.next();
> System.out.println(reader.column(1).scalar().getInt());
> rowSet.clear();
> }
> }
>
>
>
> On Thu, Jun 28, 2018 at 4:09 PM, Timothy Farkas <[email protected]> wrote:
>
>> Hi Charles,
>>
>> Currently the RowSetReader doesn't support booleans, but it can be added.
>> I'll try to add it now, and see if it could be done quickly. I'll update
>> you on my progress.
>>
>> Tim
>>
>> On Thu, Jun 28, 2018 at 2:38 PM, Charles Givre <[email protected]> wrote:
>>
>>> Hi Tim,
>>> Could post some sample code as to how to test a SQL query that returns a
>>> Boolean?
>>> —C
>>>
>>>> On Jun 28, 2018, at 17:30, Timothy Farkas <[email protected]> wrote:
>>>>
>>>> - We would have to add a boolean column reader to ColumnAccessors and
>>> wire
>>>> it in and add a getBoolean method to ScalarReader.
>>>>
>>>> - Your example should work as is, ClusterTest has a testBuilder method
>>>> that allows you to use the traditional test builder. Is there something
>>> not
>>>> working with the test builder?
>>>>
>>>> Tim
>>>>
>>>>
>>>> On Thu, Jun 28, 2018 at 12:39 PM, Arina Yelchiyeva <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi Tim,
>>>>>
>>>>> it looks like deprecating BaseTestQuery was a little bit pre-mature.
>>>>> For example, from in this PR - https://urldefense.proofpoint.
>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd
>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH
>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb-
>>> BDFaZkRaKQKwt_zl2HA&e=.
>>>>> com/v2/url?u=https-3A__github.com_apache_drill_pull_1331&d=DwIBaQ&c=
>>>>> cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=
>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=_uxhA_
>>>>> qiMBTjbit6DDw-DvZNRGesfeA5g-FQjkl7f10&e= -
>>>>> Charles is trying to re-work BaseTestQuery usage to ClusterTest.
>>>>> First, it did not contain getSigletonDouble method which Charles has
>>>>> implemented. Now he has troubles with implementing getSigletonBoolean
>>>>> method which might be due to reader limitations.
>>>>> Also I am not quite clear how we can verify columns names and multiple
>>>>> columns in the result.
>>>>> For example:
>>>>>
>>>>> testBuilder()
>>>>> .sqlQuery("select (mi || lname) as CONCATOperator, mi, lname,
>>>>> concat(mi, lname) as CONCAT from concatNull")
>>>>> .ordered()
>>>>> .baselineColumns("CONCATOperator", "mi", "lname", "CONCAT")
>>>>> .baselineValues("A.Nowmer", "A.", "Nowmer", "A.Nowmer")
>>>>> .baselineValues("I.Whelply", "I.", "Whelply", "I.Whelply")
>>>>> .baselineValues(null, null, "Derry", "Derry")
>>>>> .baselineValues("J.Spence", "J.", "Spence", "J.Spence")
>>>>> .build().run();
>>>>>
>>>>> Can you please suggest how this example can be re-written?
>>>>>
>>>>> Kind regards,
>>>>> Arina
>>>>>
>>>>> On Mon, Jun 25, 2018 at 11:10 PM Timothy Farkas <[email protected]>
>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> BaseTestQuery was deprecated a while ago. Keeping it short and sweet
>>> :),
>>>>> if
>>>>>> you want to use BaseTestQuery directly, don't. Use ClusterTest
>>> instead.
>>>>> If
>>>>>> you are using PlanTestBase for planner tests, continue to do so.
>>>>> Eventually
>>>>>> PlanTestBase will be changed to extend ClusterTest instead. There is a
>>>>> JIRA
>>>>>> to track that issue https://urldefense.proofpoint.
>>> com/v2/url?u=https-3A__urldefense.proofpoint&d=DwIFaQ&c=cskd
>>> kSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=D9i95wxPKk0aPH
>>> ip2Rj6Xju0J151UakaSe6OnMGTW5s&s=z5tYyOUI_Cyx1cwhMEMfnCb-
>>> BDFaZkRaKQKwt_zl2HA&e=.
>>>>> com/v2/url?u=https-3A__issues.apache.org_jira_browse_DRILL-
>>>>> 2D6536&d=DwIBaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=4eQVr8zB8ZBff-yxTimdOQ&m=
>>>>> zoNJPdWKxMX9-jbR2bftzwkX-CSrihIbpCirhHM1kA0&s=
>>>>> BPAlenAq0k1kjAz7fUYXyEQMaJM1IPOgmdeySMMY84U&e=.
>>>>>>
>>>>>> Thanks,
>>>>>> Tim
>>>>>>
>>>>>
>>>
>>>
>>
