Re: Kerberos ticket renewal

This is the error in the log when it fails ERROR org.apache.hadoop.security.UserGroupInformation - PriviledgedActionException as: (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find

Adding table compression

All Are there any known issues if we use the hbase shell to alter a phoenix table to apply compression? We're currently using Phoenix 4.4 on HDP 2.3.4. I plan on testing, but also want to double check for any gotchas. Michael McAllister Staff Data Warehouse Engineer | Decision Systems

Re: Does phoenix CsvBulkLoadTool write to WAL/Memstore

Hi Vamsi, How many number of rows your expecting out of your transformation and what is the frequency of job? If there are less number of row (< ~100K and this depends on cluster size as well), you can go ahead with phoenix-spark plug-in , increase batch size to accommodate more rows, else use

Re: Problem Updating Stats

Ankit, I did not see any problems when connecting with the phoenix sqlline client. So, below is the what you asked for. I hope that you can give us insight into fixing this. hbase(main):005:0> describe 'SYSTEM.STATS' Table SYSTEM.STATS is ENABLED

Re: Problem Updating Stats

ok , or you could have drop SYSTEM.STATS table from sql client at CURRENT_SCN=7 and reconnect the client. If client doesn't see this table , it will create it automatically. On Thu, Mar 17, 2016 at 3:14 AM, Benjamin Kim wrote: > I got it to work by uninstalling Phoenix and

Re: Implement Custom Aggregate Functions in Phoenix

No, custom UDFs can be added dynamically as described here: https://phoenix.apache.org/udf.html. No need to re-build Phoenix. It's just custom aggregates that would require rebuilding. FYI, we have support for UPPER and LOWER already. On Thu, Mar 17, 2016 at 6:09 PM, Swapna Swapna

Re: array support issue

How can users know what to expect when they're using an undocumented, unsupported, non public API? On Thu, Mar 17, 2016 at 6:20 PM, Nick Dimiduk wrote: > > Applications should never query the SYSTEM.CATALOG directly. Instead > they should go through the DatabaseMetaData

Re: Kerberos ticket renewal

Where do you see this error? Is it the client side? Ideally you don't need to renew ticket since Phoenix Driver gets the required information (principal name and keytab path) from jdbc connection string and performs User.login itself. Thanks, Sergey On Wed, Mar 16, 2016 at 11:02 AM, Sanooj

Re: Kerberos ticket renewal

Hi Anil Thanks for your reply. We do not do anything explicitly in the code to do the ticket renwal , what we do is run a cron job for the user for which the ticket has to be renewed. But with this approach we need a restart to get the thing going after the ticket expiry We use the following

Re: Implement Custom Aggregate Functions in Phoenix

Yes, we do have support UPPER and LOWER. I just provided as an example to refer to UDF. For custom UDF's, i understand that we can go ahead and create custom UDF jar. but how do we register that function? As per the blog, i found the below lines: *Finally, we'll need to register our new

Re: Does phoenix CsvBulkLoadTool write to WAL/Memstore

Thanks Gabriel & Ravi. I have a data processing job wirtten in Spark-Scala. I do a join on data from 2 data files (CSV files) and do data transformation on the resulting data. Finally load the transformed data into phoenix table using Phoenix-Spark plugin. On seeing that Phoenix-Spark plugin goes

how to tune phoenix CsvBulkLoadTool job

Hi, I'm using CsvBulkLoadTool to load a csv data file into Phoenix/HBase table. HDP Version : 2.3.2 (Phoenix Version : 4.4.0, HBase Version: 1.1.2) CSV file size: 97.6 GB No. of records: 1,439,000,238 Cluster: 13 node Phoenix table salt-buckets: 13 Phoenix table compression: snappy HBase table

Re: array support issue

> Applications should never query the SYSTEM.CATALOG directly. Instead they should go through the DatabaseMetaData interface from Connection.getMetaData(). I may have this detail wrong, but the point remains: applications are getting an incorrect value, or misinterpreting the correct value they

Re: Implement Custom Aggregate Functions in Phoenix

Thank you James for swift response. is the process (adding to phoenix-core and rebuild the jar) remains the same for custom UDF's as well (like as for custom aggregate functions)? ex: we have UDF's like UPPER, LOWER ,etc On Thu, Mar 17, 2016 at 5:53 PM, James Taylor

Re: Adding table compression

Nope, it should be transparent. New data will be compressed on flush and old data will be compressed during next compaction. -Vlad On Fri, Mar 18, 2016 at 12:55 PM, Michael McAllister < mmcallis...@homeaway.com> wrote: > All > > > > Are there any known issues if we use the hbase shell to alter

Question about in-flight new rows while index creation in progress

Hi Community, I want to understand and confirm whether it is expected behavior that a long running index creation will capture all in-flight new rows to the data table while the index creation is still in progress. i.e. when I issue CREATE INDEX there are only 1 million rows after I issued

Re: how to decode phoenix data under hbase

Hi Kevin, You can access the data created using phoenix with java hbase api .. Use the sample code below.. Keep in mind for varchar (i.e. for columns whose size is unknown phoenix uses separator) based columns we need to use QueryConstants.SEPARATOR_BYTE_ARRAY as the separator and for number

Re: how to decode phoenix data under hbase

Hi Kevin, You should use Phoenix commandline(squirrel) or Phoenix api to read data written via Phoenix. One of the biggest advantage of Phoenix is that it converts long, int, date, etc into a human readable format at the time of displaying data(unlike binary in HBase). Have a look at Phoenix

Re: Problem Updating Stats

I got it to work by uninstalling Phoenix and reinstalling it again. I had to wipe clean all components. Thanks, Ben > On Mar 16, 2016, at 10:47 AM, Ankit Singhal wrote: > > It seems from the attached logs that you have upgraded phoenix to 4.7 version > and now you

Re: Implement Custom Aggregate Functions in Phoenix

No need to register your custom UDFs. Did you see these directions: https://phoenix.apache.org/udf.html#How_to_write_custom_UDF? Have you tried it yet? On Thu, Mar 17, 2016 at 6:49 PM, Swapna Swapna wrote: > Yes, we do have support UPPER and LOWER. I just provided as an

Re: how to tune phoenix CsvBulkLoadTool job

Hi Vamsi, I see from your counters that the number of map spill records is double the number of map output records, so I think that raising the mapreduce.task.io.sort.mb setting on the job should improve the shuffle throughput. However, like I said before, I think that the first thing to try is

Re: how to tune phoenix CsvBulkLoadTool job

Hi Vamsi, The first thing that I notice looking at the info that you've posted is that you have 13 nodes and 13 salt buckets (which I assume also means that you have 13 regions). A single region is the unit of parallelism that is used for reducers in the CsvBulkLoadTool (or HFile-writing

Re: Implement Custom Aggregate Functions in Phoenix

Hi, I found this in Phoenix UDF documentation: - After compiling your code to a jar, you need to deploy the jar into the HDFS. It would be better to add the jar to HDFS folder configured for hbase.dynamic.jars.dir. My question is, can that be any 'udf-user-specific' jar which need to