Re: multiple data versions vs. multiple rows?

2015-01-19 Thread Serega Sheypak
does performance should differ significantly if row value size is small and we don't have too much versions. Assume, that a pack of versions for key is less than recommended HFile block (8KB to 1MB https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/io/hfile/HFile.html), which is minimal "read

Re: IllegalArgumentException: Connection is null or closed when calling HConnection.getTable()

2015-01-19 Thread Nick Dimiduk
Hi Calvin, An HConnection created via HConnectionManager#createConnection(Configuration) is an "unmanaged" connection, meaning it's lifecycle is managed by your code. Are you calling HConnection#close() on that instance someplace? Please notice that these are different semantics from the previous

Re: IllegalArgumentException: Connection is null or closed when calling HConnection.getTable()

2015-01-19 Thread Calvin Lei
Thanks. I was more curious why the connection would be closed. On Mon, Jan 19, 2015 at 5:22 PM, Ted Yu wrote: > Here is related code from HTable ctor: > > if (connection == null || connection.isClosed()) { > throw new IllegalArgumentException("Connection is null or closed."); > } >

Re: IllegalArgumentException: Connection is null or closed when calling HConnection.getTable()

2015-01-19 Thread Ted Yu
Here is related code from HTable ctor: if (connection == null || connection.isClosed()) { throw new IllegalArgumentException("Connection is null or closed."); } It was likely that connection was closed (from your description of your code). If HConnectionImplementation were to check

Re: IllegalArgumentException: Connection is null or closed when calling HConnection.getTable()

2015-01-19 Thread Calvin Lei
I upgraded to 0.98.0.2.1.1.0-385-hadoop2. The exception from hbase is: java.lang.IllegalArgumentException: Connection is null or closed. at org.apache.hadoop.hbase.client.HTable.(HTable.java:302) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getTable(HConnectio

Re: IllegalArgumentException: Connection is null or closed when calling HConnection.getTable()

2015-01-19 Thread Ted Yu
Which 0.98 release did you upgrade to ? Can you pastebin the whole stack trace ? Thanks > On Jan 19, 2015, at 1:25 PM, Calvin Lei wrote: > > Dear all, > I recently upgrade to HBase 0.0.98 and I have started seeing the error > "Connection is null or closed" when calling HConnection.getTable

Re: multiple data versions vs. multiple rows?

2015-01-19 Thread Jean-Marc Spaggiari
Hi Yong, If you want to compare the performances, you need to run way bigger and longer tests. Dont run them in parallete. Run them at least 10 time each to make sure you have a good trend. Is the difference between the 2 significant? It should not. JM 2015-01-19 15:17 GMT-05:00 yonghu : > Hi,

IllegalArgumentException: Connection is null or closed when calling HConnection.getTable()

2015-01-19 Thread Calvin Lei
Dear all, I recently upgrade to HBase 0.0.98 and I have started seeing the error "Connection is null or closed" when calling HConnection.getTable(). As recommended by the documentation, I create a HConnection using HConnectionManager.createConnection(config) and at app start and close the con

Re: multiple data versions vs. multiple rows?

2015-01-19 Thread yonghu
Hi, Thanks for your suggestion. I have already considered the first issue that one row is not allowed to be split between 2 regions. However, I have made a small scan-test with MapReduce. I first created a table t1 with 1 million rows and allowed each column to store 10 data versions. Then, I tr

Re: multiple data versions vs. multiple rows?

2015-01-19 Thread Jean-Marc Spaggiari
Hi Yong, A row will not split between 2 regions. If you plan having thousands of versions, based on the size of your data, you might end up having a row bigger than your preferred region size. If you plan just keep few versions of the history to have a look at it, I will say go with it. If you pl

multiple data versions vs. multiple rows?

2015-01-19 Thread yonghu
Dear all, I want to record the user history data. I know there exists two options, one is to store user events in a single row with multiple data versions and the other one is to use multiple rows. I wonder which one is better for performance? Thanks! Yong

Re: HBase Restful Start Error ->Port in use: 0.0.0.0:8085

2015-01-19 Thread Ted Yu
Have you found out how the process below was started ? If the discussion involves HDP, I suggest continuing on related mailing list. Cheers On Mon, Jan 19, 2015 at 2:18 AM, anil gupta wrote: > Ok figured out that by default the port is 8080. But, it seems like restful > server is not running i

Re: HBase Restful Start Error ->Port in use: 0.0.0.0:8085

2015-01-19 Thread anil gupta
Ok figured out that by default the port is 8080. But, it seems like restful server is not running in readonly mode. Is there a way to modify the startup(in HDP) and make sure that this run in ReadOnly mode? On Mon, Jan 19, 2015 at 2:12 AM, anil gupta wrote: > /usr/java/default/bin/java -Dproc_re

Re: HBase Restful Start Error ->Port in use: 0.0.0.0:8085

2015-01-19 Thread anil gupta
/usr/java/default/bin/java -Dproc_rest -XX:OnOutOfMemoryError=kill -9 %p -Xmx1000m -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hbase/hs_err_pid%p.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/hbase/gc.log-201411272023 -Dhbase.log.dir=/var/log/hbase -Dhbase.log.file=

Re: HBase Restful Start Error ->Port in use: 0.0.0.0:8085

2015-01-19 Thread Ted Yu
Can you find out which process owns port 8085 ? If it is REST process, maybe someone has started gateway already ? Cheers > On Jan 19, 2015, at 1:53 AM, anil gupta wrote: > > Hi Ted, > > hbase.rest.info.port = 8085 > We are using HBase0.98.x (HDP 2.1.5) > > Thanks, > Anil > >> On Mon, Jan

Re: HBase Restful Start Error ->Port in use: 0.0.0.0:8085

2015-01-19 Thread anil gupta
Hi Ted, hbase.rest.info.port = 8085 We are using HBase0.98.x (HDP 2.1.5) Thanks, Anil On Mon, Jan 19, 2015 at 1:49 AM, Ted Yu wrote: > What's the value for config 'hbase.rest.info.port' (default 8085) ? > > What hbase release are you using ? > > Cheers > > On Sun, Jan 18, 2015 at 10:47 PM, ani

Re: HBase Restful Start Error ->Port in use: 0.0.0.0:8085

2015-01-19 Thread Ted Yu
What's the value for config 'hbase.rest.info.port' (default 8085) ? What hbase release are you using ? Cheers On Sun, Jan 18, 2015 at 10:47 PM, anil gupta wrote: > Hi, > > We are trying to start HBase startgate on one the node that is running > HBase master. Some rest stuff of HBase has occupi

Fwd: Does 'online region merge' make regions unavailable for some time?

2015-01-19 Thread Vladimir Tretyakov
Hi, I have one question about 'online region merge' ( https://issues.apache.org/jira/browse/HBASE-7403). How I've understood regions which will be passed to merge method will be unavailable for some time. That means: 1. Some data will be unavailable some time. 2. If client will try to write data t