Sometime this month. Thanks,
On Mon, Feb 3, 2014 at 9:14 AM, Justin Workman <[email protected]>wrote: > Thanks. Is there an ETA on the 3.0 release? > > > On Mon, Feb 3, 2014 at 9:52 AM, James Taylor <[email protected]>wrote: > >> There will be an upgrade step required to go from 2.x to 3.0, as the >> system table has changed (and probably will a bit more still before we >> release). >> >> For now, you can do the following if you want to test out 3.0.0-snapshot: >> - Remove com.salesforce.* coprocessors on existing tables. If you haven't >> added any of your own, probably easiest to just remove all coprocessors. >> - Re-issue your DDL commands. If you have existing data against that >> table, it'd be best to open a connection at a timestamp earlier than any of >> your data using the CURRENT_SCN connection property. If you don't care >> about doing point-in-time queries at an earlier timestamp (or flash-back >> queries), than you don't need to worry about this, and you can just >> re-issue the DDL. >> >> Thanks, >> James >> >> >> On Mon, Feb 3, 2014 at 8:40 AM, Justin Workman >> <[email protected]>wrote: >> >>> We updated to the 3.0.0-SNAPSHOT in an effort to also test the Flume >>> component, and we are not able to query any of our existing tables now >>> through sqlline or a java jdbc connection. However the Flume component >>> works fine writing to a new table. Here is the error we are getting when >>> doing a select count(1) from keywords; >>> >>> Error: org.apache.hadoop.hbase.DoNotRetryIOException: keywords: at index >>> 4 >>> at >>> com.salesforce.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:83) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.getTable(MetaDataEndpointImpl.java:1034) >>> at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:5482) >>> at >>> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3720) >>> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at >>> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308) >>> at >>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) >>> Caused by: java.lang.NullPointerException: at index 4 >>> at >>> com.google.common.collect.ImmutableList.checkElementNotNull(ImmutableList.java:305) >>> at >>> com.google.common.collect.ImmutableList.construct(ImmutableList.java:296) >>> at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:272) >>> at com.salesforce.phoenix.schema.PTableImpl.init(PTableImpl.java:290) >>> at com.salesforce.phoenix.schema.PTableImpl.<init>(PTableImpl.java:219) >>> at >>> com.salesforce.phoenix.schema.PTableImpl.makePTable(PTableImpl.java:212) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.getTable(MetaDataEndpointImpl.java:436) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.buildTable(MetaDataEndpointImpl.java:254) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.doGetTable(MetaDataEndpointImpl.java:1082) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.addIndexToTable(MetaDataEndpointImpl.java:279) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.getTable(MetaDataEndpointImpl.java:430) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.buildTable(MetaDataEndpointImpl.java:254) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.doGetTable(MetaDataEndpointImpl.java:1082) >>> at >>> com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl.getTable(MetaDataEndpointImpl.java:1028) >>> ... 10 more (state=08000,code=101) >>> >>> >>> On Thu, Jan 30, 2014 at 4:01 PM, Justin Workman < >>> [email protected]> wrote: >>> >>>> I will test with the latest master build. When this table goes live we >>>> will shorten the cf name, that was a mistake. Thanks for all the info. I do >>>> think going forward we will be creating these tables via Phoenix. We are >>>> still testing the flume sink and pig handlers before completely committing. >>>> >>>> I'll update the list once I've had a chance to test with the latest >>>> build and file a Jira if the problem persists. >>>> >>>> Thanks! >>>> Justin >>>> >>>> Sent from my iPhone >>>> >>>> On Jan 30, 2014, at 1:25 PM, James Taylor <[email protected]> >>>> wrote: >>>> >>>> Thanks for all the detail, Justin. Based on this, it looks like a bug >>>> related to using case sensitive column names. Maryann checked in a fix >>>> related to this, so it might be fixed in the latest on master. >>>> >>>> If it's not fixed, would you mind filing a JIRA? >>>> >>>> FWIW, you may want to consider a shorter column family name, like "k" >>>> or "kw" as that'll make your table smaller. Also, did you know you can >>>> provide your HBase table and column family config parameters in your CREATE >>>> TABLE statement and it'll create the HBase table and the column families, >>>> like below? >>>> >>>> CREATE TABLE SEO.KEYWORDIDEAS ( >>>> "pk" VARCHAR PRIMARY KEY, >>>> "keyword"."jobId" VARCHAR, >>>> "keyword"."jobName" VARCHAR, >>>> "keyword"."jobType" VARCHAR, >>>> "keyword"."keywordText" VARCHAR, >>>> "keyword"."parentKeywordText" VARCHAR, >>>> "keyword"."refinementName" VARCHAR, >>>> "keyword"."refinementValue" VARCHAR, >>>> "keyword"."relatedKeywordRank" VARCHAR >>>> ) IMMUTABLE_ROWS=true, COMPRESSION='SNAPPY' ; >>>> >>>> >>>> >>>> >>>> On Thu, Jan 30, 2014 at 8:50 AM, Justin Workman < >>>> [email protected]> wrote: >>>> >>>>> I don't think that is the issue we are hitting. Details are below. The >>>>> Hbase table does have more columns than we are defining the phoenix table. >>>>> We were hoping to just be able to use the dynamic column features for >>>>> if/when we need to access data in other columns in the underlying table. >>>>> As >>>>> you can see from the output of the explain statement below, a simple query >>>>> does not use the index. >>>>> >>>>> However if I create another identical table using Phoenix and upsert >>>>> into that new table from the table below, create the same index on that >>>>> table and then run the same select query, it does use the index on that >>>>> table. >>>>> >>>>> So I am still very confused as to why the index is not invoked when >>>>> the table is created on top of an existing Hbase table. >>>>> >>>>> Hbase Create Table >>>>> > create 'SEO.KEYWORDIDEAS', { NAME=>'keyword', COMPRESSION=>'SNAPPY' } >>>>> >>>>> Phoenix Create Table >>>>> CREATE TABLE SEO.KEYWORDIDEAS ( >>>>> "pk" VARCHAR PRIMARY KEY, >>>>> "keyword"."jobId" VARCHAR, >>>>> "keyword"."jobName" VARCHAR, >>>>> "keyword"."jobType" VARCHAR, >>>>> "keyword"."keywordText" VARCHAR, >>>>> "keyword"."parentKeywordText" VARCHAR, >>>>> "keyword"."refinementName" VARCHAR, >>>>> "keyword"."refinementValue" VARCHAR, >>>>> "keyword"."relatedKeywordRank" VARCHAR >>>>> ) IMMUTABLE_ROWS=true; >>>>> >>>>> Create Index >>>>> CREATE INDEX KWDIDX ON SEO.KEYWORDIDEAS ("parentKeywordText"); >>>>> >>>>> Show and count indexes >>>>> >>>>> +-----------+-------------+------------+------------+-----------------+------------+------+------------------+-------------+-------------+-------------+--------+------------------+-----------+-----------+ >>>>> | TABLE_CAT | TABLE_SCHEM | TABLE_NAME | NON_UNIQUE | INDEX_QUALIFIER >>>>> | INDEX_NAME | TYPE | ORDINAL_POSITION | COLUMN_NAME | ASC_OR_DESC | >>>>> CARDINALITY | PAGES | FILTER_CONDITION | DATA_TYPE | TYPE_NAME | >>>>> >>>>> +-----------+-------------+------------+------------+-----------------+------------+------+------------------+-------------+-------------+-------------+--------+------------------+-----------+-----------+ >>>>> | null | SEO | KEYWORDIDEAS | true | null >>>>> | KWDIDX | 3 | 1 | keyword:parentKeywordText | A >>>>> | null | null | null | | >>>>> | null | SEO | KEYWORDIDEAS | true | null >>>>> | KWDIDX | 3 | 2 | :pk | A | null >>>>> | null | null | 12 | V | >>>>> | null | SEO | KEYWORDIDEAS | true | null >>>>> | RA_TEST_ID | 3 | 1 | keyword:jobId | A | >>>>> null | null | null | 12 | | >>>>> | null | SEO | KEYWORDIDEAS | true | null >>>>> | RA_TEST_ID | 3 | 2 | :pk | A | null >>>>> | null | null | 12 | V | >>>>> >>>>> +-----------+-------------+------------+------------+-----------------+------------+------+------------------+-------------+-------------+-------------+--------+------------------+-----------+-----------+ >>>>> >>>>> > select count(1) from seo.keywordideas; >>>>> +----------+ >>>>> | COUNT(1) | >>>>> +----------+ >>>>> | 423229 | >>>>> +----------+ >>>>> > select count(1) from seo.kwdidx; >>>>> +----------+ >>>>> | COUNT(1) | >>>>> +----------+ >>>>> | 423229 | >>>>> +----------+ >>>>> >>>>> > explain select count(1) from seo.keywords where "parentKeywordText" >>>>> = 'table'; >>>>> +------------+ >>>>> | PLAN | >>>>> +------------+ >>>>> | CLIENT PARALLEL 18-WAY FULL SCAN OVER SEO.KEYWORDIDEAS | >>>>> | SERVER FILTER BY keyword.parentKeywordText = 'sheets' | >>>>> | SERVER AGGREGATE INTO SINGLE ROW | >>>>> +------------+ >>>>> >>>>> Now here is where I can get the index to be invoked. >>>>> > CREATE TABLE SEO.NEW_KEYWORDIDEAS ( >>>>> PK VARCHAR PRIMARY KEY, >>>>> JOB_ID VARCHAR >>>>> JOB_NAME VARCHAR, >>>>> JOB_TYPE VARCHAR, >>>>> KEYWORD_TEXT VARCHAR, >>>>> PARENT_KEYWORD_TEXT VARCHAR, >>>>> REFINEMENT_NAME VARCHAR, >>>>> REFINEMENT_VALUE VARCHAR, >>>>> RELATED_KEYWORD_RANK VARCHAR >>>>> ) IMMUTABLE_ROWS=true; >>>>> >>>>> > UPSERT INTO SEO.NEW_KEYWORDIEAS SELECT * FROM SEO.KEYWORDIDEAS; >>>>> >>>>> > CREATE INDEX NEW_KWD_IDX ON SEO.NEW_KEYWORDIDEAS >>>>> (PARENT_KEYWORD_TEXT); >>>>> >>>>> > explain select count(1) from seo.new_keywordideas where >>>>> parent_keyword_text = 'table'; >>>>> >>>>> +------------+ >>>>> >>>>> | PLAN | >>>>> >>>>> +------------+ >>>>> >>>>> | CLIENT PARALLEL 1-WAY RANGE SCAN OVER SEO.NEW_KWD_IDX ['table'] | >>>>> >>>>> | SERVER AGGREGATE INTO SINGLE ROW | >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jan 29, 2014 at 5:21 PM, James Taylor >>>>> <[email protected]>wrote: >>>>> >>>>>> Hi Justin, >>>>>> Please take a look at this FAQ: >>>>>> http://phoenix.incubator.apache.org/faq.html#/Why_isnnullt_my_secondary_index_being_used >>>>>> >>>>>> If that's not the case for you, can you include your CREATE TABLE, >>>>>> CREATE INDEX, SELECT statement, and EXPLAIN plan? >>>>>> >>>>>> Thanks, >>>>>> James >>>>>> >>>>>> >>>>>> On Wed, Jan 29, 2014 at 4:13 PM, Justin Workman < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> I am seeing some odd behavior with indexes and want some >>>>>>> clarification on how they are used. >>>>>>> >>>>>>> When I create an table in phoenix on top of an existing Hbase table, >>>>>>> and then create an index on this table, I can see the index get built >>>>>>> and >>>>>>> populated properly, however no queries show that they are using this >>>>>>> index >>>>>>> when I run an explain on the query. >>>>>>> >>>>>>> However, if I create an seperete table in Phoenix and do an upsert >>>>>>> from my hbase table into the new table that I created, and create the >>>>>>> same >>>>>>> index as on the previous table. Then my queries show that they would use >>>>>>> the index when running them through the explain plan. >>>>>>> >>>>>>> Are we not able to create or use an index on a table we create over >>>>>>> an exiting Hbase table, or am I doing something wrong? >>>>>>> >>>>>>> Thanks in advance for any help. >>>>>>> Justin >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
