[jira] [Created] (HBASE-12793) [hbck] closeRegionSilentlyAndWait() should log cause of IOException and retry until hbase.hbck.close.timeout expires
Esteban Gutierrez created HBASE-12793: - Summary: [hbck] closeRegionSilentlyAndWait() should log cause of IOException and retry until hbase.hbck.close.timeout expires Key: HBASE-12793 URL: https://issues.apache.org/jira/browse/HBASE-12793 Project: HBase Issue Type: Bug Components: hbck Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Minor This is subtask on HBASE-12131 in order to handle gracefully network partitions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12792) [backport] HBASE-5835: Catch and handle NotServingRegionException when close region attempt fails
Esteban Gutierrez created HBASE-12792: - Summary: [backport] HBASE-5835: Catch and handle NotServingRegionException when close region attempt fails Key: HBASE-12792 URL: https://issues.apache.org/jira/browse/HBASE-12792 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.94.26 Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial Fix For: 0.94.27 This one is around in 0.94 and its a low hanging fruit when we get a NotServerRegionException if the region is not found when we attempt to close. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12791) HBase does not attempt to clean up an aborted split when the regionserver shutting down
Rajeshbabu Chintaguntla created HBASE-12791:
---
Summary: HBase does not attempt to clean up an aborted split when
the regionserver shutting down
Key: HBASE-12791
URL: https://issues.apache.org/jira/browse/HBASE-12791
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.98.0
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
Priority: Critical
Fix For: 2.0.0, 0.98.10, 1.0.1
HBase not cleaning the daughter region directories from HDFS if region server
shut down after creating the daughter region directories during the split.
Here the logs.
-> RS shutdown after creating the daughter regions.
{code}
2014-12-31 09:05:41,406 DEBUG [regionserver60020-splits-1419996941385]
zookeeper.ZKAssign: regionserver:60020-0x14a9701e53100d1,
quorum=localhost:2181, baseZNode=/hbase Transitioned node
80c665138d4fa32da4d792d8ed13206f from RS_ZK_REQUEST_REGION_SPLIT to
RS_ZK_REQUEST_REGION_SPLIT
2014-12-31 09:05:41,514 DEBUG [regionserver60020-splits-1419996941385]
regionserver.HRegion: Closing
t,,1419996880699.80c665138d4fa32da4d792d8ed13206f.: disabling compactions &
flushes
2014-12-31 09:05:41,514 DEBUG [regionserver60020-splits-1419996941385]
regionserver.HRegion: Updates disabled for region
t,,1419996880699.80c665138d4fa32da4d792d8ed13206f.
2014-12-31 09:05:41,516 INFO
[StoreCloserThread-t,,1419996880699.80c665138d4fa32da4d792d8ed13206f.-1]
regionserver.HStore: Closed f
2014-12-31 09:05:41,518 INFO [regionserver60020-splits-1419996941385]
regionserver.HRegion: Closed t,,1419996880699.80c665138d4fa32da4d792d8ed13206f.
2014-12-31 09:05:49,922 DEBUG [regionserver60020-splits-1419996941385]
regionserver.MetricsRegionSourceImpl: Creating new MetricsRegionSourceImpl for
table t dd9731ee43b104da565257ca1539aa8c
2014-12-31 09:05:49,922 DEBUG [regionserver60020-splits-1419996941385]
regionserver.HRegion: Instantiated
t,,1419996941401.dd9731ee43b104da565257ca1539aa8c.
2014-12-31 09:05:49,929 DEBUG [regionserver60020-splits-1419996941385]
regionserver.MetricsRegionSourceImpl: Creating new MetricsRegionSourceImpl for
table t 2e40a44511c0e187d357d651f13a1dab
2014-12-31 09:05:49,929 DEBUG [regionserver60020-splits-1419996941385]
regionserver.HRegion: Instantiated
t,row2,1419996941401.2e40a44511c0e187d357d651f13a1dab.
Wed Dec 31 09:06:30 IST 2014 Terminating regionserver
2014-12-31 09:06:30,465 INFO [Thread-8] regionserver.ShutdownHook: Shutdown
hook starting; hbase.shutdown.hook=true;
fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@42d2282e
{code}
-> Skipping rollback if RS stopped or stopping so we end up in dirty daughter
regions in HDFS.
{code}
2014-12-31 09:07:49,547 INFO [regionserver60020-splits-1419996941385]
regionserver.SplitRequest: Skip rollback/cleanup of failed split of
t,,1419996880699.80c665138d4fa32da4d792d8ed13206f. because server is stopped
java.io.InterruptedIOException: Interrupted after 0 tries on 350
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:156)
{code}
Because of this hbck always showing inconsistencies.
{code}
ERROR: Region { meta => null, hdfs =>
hdfs://localhost:9000/hbase/data/default/t/2e40a44511c0e187d357d651f13a1dab,
deployed => } on HDFS, but not listed in hbase:meta or deployed on any region
server
ERROR: Region { meta => null, hdfs =>
hdfs://localhost:9000/hbase/data/default/t/dd9731ee43b104da565257ca1539aa8c,
deployed => } on HDFS, but not listed in hbase:meta or deployed on any region
server
{code}
If we try to repair then we end up in overlap regions in hbase:meta.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12790) Support fairness across parallelized scans
James Taylor created HBASE-12790: Summary: Support fairness across parallelized scans Key: HBASE-12790 URL: https://issues.apache.org/jira/browse/HBASE-12790 Project: HBase Issue Type: Bug Reporter: James Taylor Some HBase clients parallelize the execution of a scan to reduce latency in getting back results. This can lead to starvation with a loaded cluster and interleaved scans, since the RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients, A & B that submit largish scans at the same time. Say each scan is broken down into 100 scans by the client (broken down into equal depth chunks along the row key), and the 100 scans of client A are queued first, followed immediately by the 100 scans of client B. In this case, client B will be starved out of getting any results back until the scans for client A complete. One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new config parameter. Using this queue would require the client to have the same identifier for all of the 100 parallel scans that represent a single logical scan from the clients point of view. With this information, the round robin queue would pick off a task from the queue in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved parallelized scans. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Considering a RpcSchedulerFactory change in 0.98 for HBASE-12028
Thanks, Andrew. I filed PHOENIX-1569 as well. On Tue, Dec 30, 2014 at 4:34 PM, Andrew Purtell wrote: > Thanks for the feedback James. I filed HBASE-12787 in response. > > On Tue, Dec 30, 2014 at 3:49 PM, James Taylor > wrote: > >> bq. How many versions of HBase >= 0.98.10 do you think would need to >> be binary compatible with 4.2.2? >> >> Good question. Do you have an opinion? We have a compatibility check >> that we do on first connection to a cluster. Perhaps we can add a >> check of Phoenix server version vs HBase server version to detect a >> "breakage" scenario? In this case, we'd require the server-side >> Phoenix version to be bumped up (maybe do this in 4.4?). We can doc it >> as well, but it's been my experience that folks just don't read this. >> >> So perhaps have the reflection in place in HBase long enough for us to >> get 4.4 out? >> >> Thanks for asking! >> >> James >> >> On Tue, Dec 30, 2014 at 3:36 PM, Andrew Purtell >> wrote: >> > It would be a binary compatibility break unless we detect by reflection >> that it's an older factory missing the new 'create' method and therefore >> call the old one. >> > >> > We could add that. >> > >> > How many versions of HBase >= 0.98.10 do you think would need to be >> binary compatible with 4.2.2? >> > >> > >> >> On Dec 30, 2014, at 3:23 PM, James Taylor >> wrote: >> >> >> >> Would our 4.2.2 binaries continue to work with releases of HBase >> >> containing this change? >> >> >> >> >> >>> On Tue, Dec 30, 2014 at 3:14 PM, Enis Söztutar >> wrote: >> >>> Thanks Andrew, >> >>> >> >>> Once HBASE-12028 is committed it should be easy enough to make the >> changes >> >>> in Phoenix to be able to compile with HBase versions pre or post >> >>> HBASE-12028. But we need a PHOENIX issue for that. >> >>> >> >>> We should also make Abortable a LimitedPrivate it seems. >> >>> >> >>> Enis >> >>> >> >>> On Tue, Dec 30, 2014 at 2:49 PM, Andrew Purtell < >> andrew.purt...@gmail.com> >> >>> wrote: >> >>> >> Hi Phoenix, >> >> Please see https://issues.apache.org/jira/browse/HBASE-12028 >> >> The proposed change if committed into 0.98 branch would introduce a >> new >> 'create' method into the RpcSchedulerFactory interface that receives >> an >> Abortable as an additional parameter. Thus, the factory can pass this >> on to >> schedulers and workers and if something terrible happens in or to a >> RPC >> handler they can trigger a server abort. Due to a design oversight we >> don't >> otherwise have this capability. In my opinion it is important to fix >> this >> oversight. (Phoenix can also potentially make use of the Abortable for >> fatal issues involving indexes.) Otherwise RPC handlers can silently >> terminate upon receiving an unhandled throwable, potentially leaving >> behind >> bad state, certainly impacting performance and availability. However >> because RpcSchedulerFactory is an interface any implementor will not >> compile after this change, until updated. >> >> HBase could include this change in the next 0.98 release or not. >> Please >> advise. >> >> >> >> >> > > > > -- > Best regards, > >- Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White)
[jira] [Resolved] (HBASE-12270) A bug in the bucket cache, with cache blocks on write enabled
[ https://issues.apache.org/jira/browse/HBASE-12270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-12270. Resolution: Fixed Something weird happened with TestReplicationEndpoint in the build where the addendum went in but it could be spurious and the compile problem is definitely fixed. > A bug in the bucket cache, with cache blocks on write enabled > - > > Key: HBASE-12270 > URL: https://issues.apache.org/jira/browse/HBASE-12270 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.11, 0.98.6.1 > Environment: I can reproduce it on a simple 2 node cluster, one > running the master and another running a RS. I was testing on ec2. > I used the following configurations for the cluster. > hbase-env:HBASE_REGIONSERVER_OPTS=-Xmx2G -XX:MaxDirectMemorySize=5G > -XX:CMSInitiatingOccupancyFraction=88 -XX:+AggressiveOpts -verbose:gc > -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xlog > gc:/tmp/hbase-regionserver-gc.log > hbase-site: > hbase.bucketcache.ioengine=offheap > hbase.bucketcache.size=4196 > hbase.rs.cacheblocksonwrite=true > hfile.block.index.cacheonwrite=true > hfile.block.bloom.cacheonwrite=true >Reporter: Khaled Elmeleegy >Assignee: Liu Shaohui >Priority: Critical > Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0 > > Attachments: HBASE-12270-v1.diff, HBASE-12270-v2.diff, > HBASE-12270-v2.patch, TestHBase.java, TestKey.java > > > In my experiments, I have writers streaming their output to HBase. The reader > powers a web page and does this scatter/gather, where it reads 1000 keys > written last and passes them the the front end. With this workload, I get the > exception below at the region server. Again, I am using HBAse (0.98.6.1). Any > help is appreciated. > 2014-10-10 15:06:44,173 ERROR > [B.DefaultRpcServer.handler=62,queue=2,port=60020] ipc.RpcServer: Unexpected > throwable object > java.lang.IllegalArgumentException > at java.nio.Buffer.position(Buffer.java:236) > at > org.apache.hadoop.hbase.util.ByteBufferUtils.skip(ByteBufferUtils.java:434) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readKeyValueLen(HFileReaderV2.java:849) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:760) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:248) >at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:152) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:317) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:176) > at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1780) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3758) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1950) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1936) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1913) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3157) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29587) >at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) >at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) > at java.lang.Thread.run(Thread.java:744) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: A face-lift for 1.0
Any chance of fixing this by way of a contribution back to the maven-skin maintainers? We can avoid bringing back the site.vm if we leave this in their hands. On Tue, Dec 30, 2014 at 12:11 PM, Dima Spivak wrote: > Ooh, good call, guys. Yeah, looks like all we'd need to do is move > bootstrap.min.css and bootstrap-responsive.min.css onto our server and link > to it relatively instead of with absolute links it has now. Seeing as how > both of those CSS files are Apache License, should be easy. > > -Dima > > On Tue, Dec 30, 2014 at 11:54 AM, Elliott Clark wrote: > > > Yeah the js and css are all loaded though a cdn. It should be pretty easy > > to change the urls to be protocol relative and then things should work. > > > > On Tue, Dec 30, 2014 at 11:17 AM, Enis Söztutar > > wrote: > > > > > I had the same problem. We link to CSS which is http instead of forking > > and > > > serving it in the site (I believe this is the default practice). My > > chrome > > > extension refuses to connect to http links causing rendering problems. > > > > > > Enis > > > > > > On Tue, Dec 30, 2014 at 11:00 AM, Dima Spivak > > > wrote: > > > > > > > Not sure if this is something to address in this thread or if I > should > > > file > > > > a separate JIRA, but for some reason, accessing the site over https > > gets > > > > rid of the pretty CSS. Seems to happen in Chrome and Firefox. Anyone > > > have a > > > > clue why? > > > > > > > > -Dima > > > > > > > > On Wed, Dec 17, 2014 at 10:53 AM, Nick Dimiduk > > > wrote: > > > > > > > > > Tweaking the layout of the website goes one of two ways. Either > > > there's a > > > > > configuration point exposed from the maven skin or there isn't. If > > > there > > > > > is, it's simply editing the appropriate custom tag in site.xml. If > > not, > > > > > we'll need to bring back the velocity site.vm template. > > Configurations > > > > are > > > > > documented here: > > > > > http://andriusvelykis.github.io/reflow-maven-skin/skin/config.html > > > > > > > > > > Tweaking visuals is now based on the theme. We can either choose > > > another > > > > > bootswatch theme, or write our own. That stuff is documented here: > > > > > http://andriusvelykis.github.io/reflow-maven-skin/skin/themes/ > > > > > > > > > > Right now the site is separate from the book. In theory we could > move > > > > book > > > > > content over to become site content, in which case it would be > > managed > > > by > > > > > this same plugin. Short of that, we get into applying style via the > > > > docbook > > > > > plugin. I've looked into this a bit; it's not nearly as easy as > > > applying > > > > > reflow-maven-skin. The current leads I have for this are > > > > > https://github.com/lcahlander/docbook and > > > > > > > > > > > > > > > > > > > > https://github.com/pmartin/phing-documentation-docbook/commit/8d8f0b81951c4e07420083b204599a6b9ef00537 > > > > > . > > > > > > > > > > Honestly though, if we're looking at moving away from docbook, I > > think > > > we > > > > > should consider a more modern toolset, Sphinx-doc, for example: > > > > > http://sphinx-doc.org > > > > > > > > > > On Tue, Dec 16, 2014 at 8:21 PM, Misty Stanley-Jones < > > > > > mstanleyjo...@cloudera.com> wrote: > > > > > > > > > > > > Absolutely fantastic!!! Got pointers to how tweaking the website > > > would > > > > > > work? > > > > > > > > > > > > In the new year I hope to really get started on a restructure of > > the > > > > Ref > > > > > > Guide. I've thought about asking to move to AsciiDOC or MarkDown > > but > > > > I'd > > > > > > like to do a proof-of-concept and see if it really gets us > anything > > > we > > > > > > don't already have with Docbook. It's less of a potential value > > since > > > > we > > > > > > are not hosted on Github, which has great integration with > AsciiDOC > > > and > > > > > > MarkDown. I'm very interested in improving the look and feel of > the > > > > doc. > > > > > I > > > > > > think a lot of it will be structural -- reorganizing and > > re-chunking > > > > > > content. > > > > > > > > > > > > In the meantime, theming the Docbook is not as hard as you might > > > think. > > > > > > Have a look at the XSLT files. (I know, famous last words). > > > > > > > > > > > > On Wed, Dec 17, 2014 at 1:21 PM, Nick Dimiduk < > ndimi...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > For those not following closely in JIRAland, I've two changes > > that > > > > are > > > > > > > ready to go. First, there's transition over to a > bootstrap-based > > > > layout > > > > > > for > > > > > > > our home page. This work is done in HBASE-12688. This improves > > the > > > > > looks > > > > > > of > > > > > > > things and will make it pretty easy to update our look-and-feel > > in > > > > the > > > > > > > future. Second, I fixed a build problem where the online book > > > wasn't > > > > > > > getting its intended styling. This is done in HBASE-12687. I'd > > like > > > > to > > > > > > > update the book to look the
[jira] [Resolved] (HBASE-12789) TestCacheOnWrite#testStoreFileCacheOnWrite sometimes fails with HFileBlock cached assertion failure
[
https://issues.apache.org/jira/browse/HBASE-12789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-12789.
---
Resolution: Invalid
Copy/paste of our build box output with no input.
> TestCacheOnWrite#testStoreFileCacheOnWrite sometimes fails with HFileBlock
> cached assertion failure
> ---
>
> Key: HBASE-12789
> URL: https://issues.apache.org/jira/browse/HBASE-12789
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Priority: Minor
>
> From
> https://builds.apache.org/job/HBase-TRUNK/5980/testReport/junit/org.apache.hadoop.hbase.io.hfile/TestCacheOnWrite/testStoreFileCacheOnWrite_106_/
> :
> {code}
> java.lang.AssertionError: shouldBeCached: false
> isCached: true
> Test description: [cacheOnWrite=INDEX_BLOCKS, compress=GZ,
> encoderType=BLOCK_ENCODING_EVERYWHERE, cacheCompressedData=false]
> block: HFileBlock [ fileOffset=12046 headerSize()=33 blockType=ENCODED_DATA
> onDiskSizeWithoutHeader=1325 uncompressedSizeWithoutHeader=1591
> prevBlockOffset=10457 isUseHBaseChecksum()=true checksumType=NULL
> bytesPerChecksum=512 onDiskDataSizeWithHeader=1346
> getOnDiskSizeWithHeader()=1358 totalChecksumBytes()=12 isUnpacked()=true
> buf=[ java.nio.HeapByteBuffer[pos=0 lim=1636 cap=1669] ]
> dataBeginsWith=\x00\x02\x00\x00\x08N.\x1D\x00\x00 a
> fileContext=HFileContext [ usesHBaseChecksum=true checksumType=NULL
> bytesPerChecksum=512 blocksize=65536 encoding=NONE includesMvcc=true
> includesTags=true compressAlgo=NONE compressTags=false cryptoContext=[
> cipher=NONE keyHash=NONE ] ] ]
> encodingInCache: PREFIX
> blockCacheKey: bfdf56eb18464371a17ed05500e189d6_12046
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite.readStoreFile(TestCacheOnWrite.java:285)
> at
> org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite.testStoreFileCacheOnWriteInternals(TestCacheOnWrite.java:244)
> at
> org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite.testStoreFileCacheOnWrite(TestCacheOnWrite.java:239)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12789) TestCacheOnWrite#testStoreFileCacheOnWrite sometimes fails with HFileBlock cached assertion failure
Ted Yu created HBASE-12789:
--
Summary: TestCacheOnWrite#testStoreFileCacheOnWrite sometimes
fails with HFileBlock cached assertion failure
Key: HBASE-12789
URL: https://issues.apache.org/jira/browse/HBASE-12789
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Priority: Minor
>From
>https://builds.apache.org/job/HBase-TRUNK/5980/testReport/junit/org.apache.hadoop.hbase.io.hfile/TestCacheOnWrite/testStoreFileCacheOnWrite_106_/
> :
{code}
java.lang.AssertionError: shouldBeCached: false
isCached: true
Test description: [cacheOnWrite=INDEX_BLOCKS, compress=GZ,
encoderType=BLOCK_ENCODING_EVERYWHERE, cacheCompressedData=false]
block: HFileBlock [ fileOffset=12046 headerSize()=33 blockType=ENCODED_DATA
onDiskSizeWithoutHeader=1325 uncompressedSizeWithoutHeader=1591
prevBlockOffset=10457 isUseHBaseChecksum()=true checksumType=NULL
bytesPerChecksum=512 onDiskDataSizeWithHeader=1346
getOnDiskSizeWithHeader()=1358 totalChecksumBytes()=12 isUnpacked()=true buf=[
java.nio.HeapByteBuffer[pos=0 lim=1636 cap=1669] ]
dataBeginsWith=\x00\x02\x00\x00\x08N.\x1D\x00\x00 a
fileContext=HFileContext [ usesHBaseChecksum=true checksumType=NULL
bytesPerChecksum=512 blocksize=65536 encoding=NONE includesMvcc=true
includesTags=true compressAlgo=NONE compressTags=false cryptoContext=[
cipher=NONE keyHash=NONE ] ] ]
encodingInCache: PREFIX
blockCacheKey: bfdf56eb18464371a17ed05500e189d6_12046
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite.readStoreFile(TestCacheOnWrite.java:285)
at
org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite.testStoreFileCacheOnWriteInternals(TestCacheOnWrite.java:244)
at
org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite.testStoreFileCacheOnWrite(TestCacheOnWrite.java:239)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12788) Promote Abortable to LimitedPrivate
Andrew Purtell created HBASE-12788: -- Summary: Promote Abortable to LimitedPrivate Key: HBASE-12788 URL: https://issues.apache.org/jira/browse/HBASE-12788 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0 After HBASE-12028, we will be passing Abortable around in a LimitedPrivate interface, with the intent of its use in implementors and subclasses of related types (RPC schedulers). Therefore we should promote Abortable to LimitedPrivate as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Considering a RpcSchedulerFactory change in 0.98 for HBASE-12028
If we would like to port this change to 0.98, another option is not changing RpcSchedulerFactory, but add a SET method for RpcScheduler that set the Abortable afterwards. Thus there will be no backward compatible issue. New code need to know to call the SET method. Old code will have a null abortable. Alicia On Tue, Dec 30, 2014 at 3:49 PM, James Taylor wrote: > bq. How many versions of HBase >= 0.98.10 do you think would need to > be binary compatible with 4.2.2? > > Good question. Do you have an opinion? We have a compatibility check > that we do on first connection to a cluster. Perhaps we can add a > check of Phoenix server version vs HBase server version to detect a > "breakage" scenario? In this case, we'd require the server-side > Phoenix version to be bumped up (maybe do this in 4.4?). We can doc it > as well, but it's been my experience that folks just don't read this. > > So perhaps have the reflection in place in HBase long enough for us to > get 4.4 out? > > Thanks for asking! > > James > > On Tue, Dec 30, 2014 at 3:36 PM, Andrew Purtell > wrote: > > It would be a binary compatibility break unless we detect by reflection > that it's an older factory missing the new 'create' method and therefore > call the old one. > > > > We could add that. > > > > How many versions of HBase >= 0.98.10 do you think would need to be > binary compatible with 4.2.2? > > > > > >> On Dec 30, 2014, at 3:23 PM, James Taylor > wrote: > >> > >> Would our 4.2.2 binaries continue to work with releases of HBase > >> containing this change? > >> > >> > >>> On Tue, Dec 30, 2014 at 3:14 PM, Enis Söztutar > wrote: > >>> Thanks Andrew, > >>> > >>> Once HBASE-12028 is committed it should be easy enough to make the > changes > >>> in Phoenix to be able to compile with HBase versions pre or post > >>> HBASE-12028. But we need a PHOENIX issue for that. > >>> > >>> We should also make Abortable a LimitedPrivate it seems. > >>> > >>> Enis > >>> > >>> On Tue, Dec 30, 2014 at 2:49 PM, Andrew Purtell < > andrew.purt...@gmail.com> > >>> wrote: > >>> > Hi Phoenix, > > Please see https://issues.apache.org/jira/browse/HBASE-12028 > > The proposed change if committed into 0.98 branch would introduce a > new > 'create' method into the RpcSchedulerFactory interface that receives > an > Abortable as an additional parameter. Thus, the factory can pass this > on to > schedulers and workers and if something terrible happens in or to a > RPC > handler they can trigger a server abort. Due to a design oversight we > don't > otherwise have this capability. In my opinion it is important to fix > this > oversight. (Phoenix can also potentially make use of the Abortable for > fatal issues involving indexes.) Otherwise RPC handlers can silently > terminate upon receiving an unhandled throwable, potentially leaving > behind > bad state, certainly impacting performance and availability. However > because RpcSchedulerFactory is an interface any implementor will not > compile after this change, until updated. > > HBase could include this change in the next 0.98 release or not. > Please > advise. > > > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Considering a RpcSchedulerFactory change in 0.98 for HBASE-12028
Thanks for the feedback James. I filed HBASE-12787 in response. On Tue, Dec 30, 2014 at 3:49 PM, James Taylor wrote: > bq. How many versions of HBase >= 0.98.10 do you think would need to > be binary compatible with 4.2.2? > > Good question. Do you have an opinion? We have a compatibility check > that we do on first connection to a cluster. Perhaps we can add a > check of Phoenix server version vs HBase server version to detect a > "breakage" scenario? In this case, we'd require the server-side > Phoenix version to be bumped up (maybe do this in 4.4?). We can doc it > as well, but it's been my experience that folks just don't read this. > > So perhaps have the reflection in place in HBase long enough for us to > get 4.4 out? > > Thanks for asking! > > James > > On Tue, Dec 30, 2014 at 3:36 PM, Andrew Purtell > wrote: > > It would be a binary compatibility break unless we detect by reflection > that it's an older factory missing the new 'create' method and therefore > call the old one. > > > > We could add that. > > > > How many versions of HBase >= 0.98.10 do you think would need to be > binary compatible with 4.2.2? > > > > > >> On Dec 30, 2014, at 3:23 PM, James Taylor > wrote: > >> > >> Would our 4.2.2 binaries continue to work with releases of HBase > >> containing this change? > >> > >> > >>> On Tue, Dec 30, 2014 at 3:14 PM, Enis Söztutar > wrote: > >>> Thanks Andrew, > >>> > >>> Once HBASE-12028 is committed it should be easy enough to make the > changes > >>> in Phoenix to be able to compile with HBase versions pre or post > >>> HBASE-12028. But we need a PHOENIX issue for that. > >>> > >>> We should also make Abortable a LimitedPrivate it seems. > >>> > >>> Enis > >>> > >>> On Tue, Dec 30, 2014 at 2:49 PM, Andrew Purtell < > andrew.purt...@gmail.com> > >>> wrote: > >>> > Hi Phoenix, > > Please see https://issues.apache.org/jira/browse/HBASE-12028 > > The proposed change if committed into 0.98 branch would introduce a > new > 'create' method into the RpcSchedulerFactory interface that receives > an > Abortable as an additional parameter. Thus, the factory can pass this > on to > schedulers and workers and if something terrible happens in or to a > RPC > handler they can trigger a server abort. Due to a design oversight we > don't > otherwise have this capability. In my opinion it is important to fix > this > oversight. (Phoenix can also potentially make use of the Abortable for > fatal issues involving indexes.) Otherwise RPC handlers can silently > terminate upon receiving an unhandled throwable, potentially leaving > behind > bad state, certainly impacting performance and availability. However > because RpcSchedulerFactory is an interface any implementor will not > compile after this change, until updated. > > HBase could include this change in the next 0.98 release or not. > Please > advise. > > > > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
[jira] [Created] (HBASE-12787) Backport HBASE-12028 (Abort the RegionServer when it's handler threads die) to 0.98
Andrew Purtell created HBASE-12787: -- Summary: Backport HBASE-12028 (Abort the RegionServer when it's handler threads die) to 0.98 Key: HBASE-12787 URL: https://issues.apache.org/jira/browse/HBASE-12787 Project: HBase Issue Type: Task Reporter: Andrew Purtell Assignee: Andrew Purtell Backport HBASE-12028 (Abort the RegionServer when it's handler threads die) to 0.98. There are two 0.98-specific changes that should be addressed: # The default configuration should preserve current behavior # The interface change to RpcSchedulerFactory should be binary compatible for Phoenix if possible, and we can do this by detecting with reflection if an older implementation of the interface is missing the new 'create' method and by invoking the older deprecated 'create' method instead. We can cache the decision after making it once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Considering a RpcSchedulerFactory change in 0.98 for HBASE-12028
bq. How many versions of HBase >= 0.98.10 do you think would need to be binary compatible with 4.2.2? Good question. Do you have an opinion? We have a compatibility check that we do on first connection to a cluster. Perhaps we can add a check of Phoenix server version vs HBase server version to detect a "breakage" scenario? In this case, we'd require the server-side Phoenix version to be bumped up (maybe do this in 4.4?). We can doc it as well, but it's been my experience that folks just don't read this. So perhaps have the reflection in place in HBase long enough for us to get 4.4 out? Thanks for asking! James On Tue, Dec 30, 2014 at 3:36 PM, Andrew Purtell wrote: > It would be a binary compatibility break unless we detect by reflection that > it's an older factory missing the new 'create' method and therefore call the > old one. > > We could add that. > > How many versions of HBase >= 0.98.10 do you think would need to be binary > compatible with 4.2.2? > > >> On Dec 30, 2014, at 3:23 PM, James Taylor wrote: >> >> Would our 4.2.2 binaries continue to work with releases of HBase >> containing this change? >> >> >>> On Tue, Dec 30, 2014 at 3:14 PM, Enis Söztutar wrote: >>> Thanks Andrew, >>> >>> Once HBASE-12028 is committed it should be easy enough to make the changes >>> in Phoenix to be able to compile with HBase versions pre or post >>> HBASE-12028. But we need a PHOENIX issue for that. >>> >>> We should also make Abortable a LimitedPrivate it seems. >>> >>> Enis >>> >>> On Tue, Dec 30, 2014 at 2:49 PM, Andrew Purtell >>> wrote: >>> Hi Phoenix, Please see https://issues.apache.org/jira/browse/HBASE-12028 The proposed change if committed into 0.98 branch would introduce a new 'create' method into the RpcSchedulerFactory interface that receives an Abortable as an additional parameter. Thus, the factory can pass this on to schedulers and workers and if something terrible happens in or to a RPC handler they can trigger a server abort. Due to a design oversight we don't otherwise have this capability. In my opinion it is important to fix this oversight. (Phoenix can also potentially make use of the Abortable for fatal issues involving indexes.) Otherwise RPC handlers can silently terminate upon receiving an unhandled throwable, potentially leaving behind bad state, certainly impacting performance and availability. However because RpcSchedulerFactory is an interface any implementor will not compile after this change, until updated. HBase could include this change in the next 0.98 release or not. Please advise.
[jira] [Reopened] (HBASE-12270) A bug in the bucket cache, with cache blocks on write enabled
[ https://issues.apache.org/jira/browse/HBASE-12270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-12270: This broke the 0.98 build [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/jenkins/jenkins-slave/workspace/HBase-0.98-on-Hadoop-1.1/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java:[224,8] cannot find symbol symbol : constructor CacheConfig(org.apache.hadoop.hbase.io.hfile.BlockCache,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean,boolean) location: class org.apache.hadoop.abase.io.hfile.CacheConfig [INFO] 1 error > A bug in the bucket cache, with cache blocks on write enabled > - > > Key: HBASE-12270 > URL: https://issues.apache.org/jira/browse/HBASE-12270 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.11, 0.98.6.1 > Environment: I can reproduce it on a simple 2 node cluster, one > running the master and another running a RS. I was testing on ec2. > I used the following configurations for the cluster. > hbase-env:HBASE_REGIONSERVER_OPTS=-Xmx2G -XX:MaxDirectMemorySize=5G > -XX:CMSInitiatingOccupancyFraction=88 -XX:+AggressiveOpts -verbose:gc > -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xlog > gc:/tmp/hbase-regionserver-gc.log > hbase-site: > hbase.bucketcache.ioengine=offheap > hbase.bucketcache.size=4196 > hbase.rs.cacheblocksonwrite=true > hfile.block.index.cacheonwrite=true > hfile.block.bloom.cacheonwrite=true >Reporter: Khaled Elmeleegy >Assignee: Liu Shaohui >Priority: Critical > Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0 > > Attachments: HBASE-12270-v1.diff, HBASE-12270-v2.diff, > HBASE-12270-v2.patch, TestHBase.java, TestKey.java > > > In my experiments, I have writers streaming their output to HBase. The reader > powers a web page and does this scatter/gather, where it reads 1000 keys > written last and passes them the the front end. With this workload, I get the > exception below at the region server. Again, I am using HBAse (0.98.6.1). Any > help is appreciated. > 2014-10-10 15:06:44,173 ERROR > [B.DefaultRpcServer.handler=62,queue=2,port=60020] ipc.RpcServer: Unexpected > throwable object > java.lang.IllegalArgumentException > at java.nio.Buffer.position(Buffer.java:236) > at > org.apache.hadoop.hbase.util.ByteBufferUtils.skip(ByteBufferUtils.java:434) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readKeyValueLen(HFileReaderV2.java:849) > at > org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:760) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:248) >at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:152) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:317) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:176) > at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1780) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3758) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1950) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1936) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1913) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3157) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29587) >at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) >at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) > at java.lang.Thread.run(Thread.java:744) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Considering a RpcSchedulerFactory change in 0.98 for HBASE-12028
It would be a binary compatibility break unless we detect by reflection that it's an older factory missing the new 'create' method and therefore call the old one. We could add that. How many versions of HBase >= 0.98.10 do you think would need to be binary compatible with 4.2.2? > On Dec 30, 2014, at 3:23 PM, James Taylor wrote: > > Would our 4.2.2 binaries continue to work with releases of HBase > containing this change? > > >> On Tue, Dec 30, 2014 at 3:14 PM, Enis Söztutar wrote: >> Thanks Andrew, >> >> Once HBASE-12028 is committed it should be easy enough to make the changes >> in Phoenix to be able to compile with HBase versions pre or post >> HBASE-12028. But we need a PHOENIX issue for that. >> >> We should also make Abortable a LimitedPrivate it seems. >> >> Enis >> >> On Tue, Dec 30, 2014 at 2:49 PM, Andrew Purtell >> wrote: >> >>> Hi Phoenix, >>> >>> Please see https://issues.apache.org/jira/browse/HBASE-12028 >>> >>> The proposed change if committed into 0.98 branch would introduce a new >>> 'create' method into the RpcSchedulerFactory interface that receives an >>> Abortable as an additional parameter. Thus, the factory can pass this on to >>> schedulers and workers and if something terrible happens in or to a RPC >>> handler they can trigger a server abort. Due to a design oversight we don't >>> otherwise have this capability. In my opinion it is important to fix this >>> oversight. (Phoenix can also potentially make use of the Abortable for >>> fatal issues involving indexes.) Otherwise RPC handlers can silently >>> terminate upon receiving an unhandled throwable, potentially leaving behind >>> bad state, certainly impacting performance and availability. However >>> because RpcSchedulerFactory is an interface any implementor will not >>> compile after this change, until updated. >>> >>> HBase could include this change in the next 0.98 release or not. Please >>> advise. >>> >>> >>> >>>
Re: Considering a RpcSchedulerFactory change in 0.98 for HBASE-12028
Would our 4.2.2 binaries continue to work with releases of HBase containing this change? On Tue, Dec 30, 2014 at 3:14 PM, Enis Söztutar wrote: > Thanks Andrew, > > Once HBASE-12028 is committed it should be easy enough to make the changes > in Phoenix to be able to compile with HBase versions pre or post > HBASE-12028. But we need a PHOENIX issue for that. > > We should also make Abortable a LimitedPrivate it seems. > > Enis > > On Tue, Dec 30, 2014 at 2:49 PM, Andrew Purtell > wrote: > >> Hi Phoenix, >> >> Please see https://issues.apache.org/jira/browse/HBASE-12028 >> >> The proposed change if committed into 0.98 branch would introduce a new >> 'create' method into the RpcSchedulerFactory interface that receives an >> Abortable as an additional parameter. Thus, the factory can pass this on to >> schedulers and workers and if something terrible happens in or to a RPC >> handler they can trigger a server abort. Due to a design oversight we don't >> otherwise have this capability. In my opinion it is important to fix this >> oversight. (Phoenix can also potentially make use of the Abortable for >> fatal issues involving indexes.) Otherwise RPC handlers can silently >> terminate upon receiving an unhandled throwable, potentially leaving behind >> bad state, certainly impacting performance and availability. However >> because RpcSchedulerFactory is an interface any implementor will not >> compile after this change, until updated. >> >> HBase could include this change in the next 0.98 release or not. Please >> advise. >> >> >> >>
Re: Considering a RpcSchedulerFactory change in 0.98 for HBASE-12028
Thanks Andrew, Once HBASE-12028 is committed it should be easy enough to make the changes in Phoenix to be able to compile with HBase versions pre or post HBASE-12028. But we need a PHOENIX issue for that. We should also make Abortable a LimitedPrivate it seems. Enis On Tue, Dec 30, 2014 at 2:49 PM, Andrew Purtell wrote: > Hi Phoenix, > > Please see https://issues.apache.org/jira/browse/HBASE-12028 > > The proposed change if committed into 0.98 branch would introduce a new > 'create' method into the RpcSchedulerFactory interface that receives an > Abortable as an additional parameter. Thus, the factory can pass this on to > schedulers and workers and if something terrible happens in or to a RPC > handler they can trigger a server abort. Due to a design oversight we don't > otherwise have this capability. In my opinion it is important to fix this > oversight. (Phoenix can also potentially make use of the Abortable for > fatal issues involving indexes.) Otherwise RPC handlers can silently > terminate upon receiving an unhandled throwable, potentially leaving behind > bad state, certainly impacting performance and availability. However > because RpcSchedulerFactory is an interface any implementor will not > compile after this change, until updated. > > HBase could include this change in the next 0.98 release or not. Please > advise. > > > >
Considering a RpcSchedulerFactory change in 0.98 for HBASE-12028
Hi Phoenix, Please see https://issues.apache.org/jira/browse/HBASE-12028 The proposed change if committed into 0.98 branch would introduce a new 'create' method into the RpcSchedulerFactory interface that receives an Abortable as an additional parameter. Thus, the factory can pass this on to schedulers and workers and if something terrible happens in or to a RPC handler they can trigger a server abort. Due to a design oversight we don't otherwise have this capability. In my opinion it is important to fix this oversight. (Phoenix can also potentially make use of the Abortable for fatal issues involving indexes.) Otherwise RPC handlers can silently terminate upon receiving an unhandled throwable, potentially leaving behind bad state, certainly impacting performance and availability. However because RpcSchedulerFactory is an interface any implementor will not compile after this change, until updated. HBase could include this change in the next 0.98 release or not. Please advise.
[jira] [Created] (HBASE-12786) List the zookeeper quorum nodes on different lines in the Master web ui
Rishit Shroff created HBASE-12786: - Summary: List the zookeeper quorum nodes on different lines in the Master web ui Key: HBASE-12786 URL: https://issues.apache.org/jira/browse/HBASE-12786 Project: HBase Issue Type: Task Components: master Affects Versions: 0.89-fb Reporter: Rishit Shroff Priority: Trivial HBase uses zookeeper to store the current state of the cluster and uses it to maintain the overall consistency of the cluster. The zookeeper is quorum is hosted on set of nodes (5 typically) in the same hbase cluster. The names of these hosts are accessible in the HMaster web ui page. However, they are all ',' separated. It would be nice to list them on different lines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: A face-lift for 1.0
Ooh, good call, guys. Yeah, looks like all we'd need to do is move bootstrap.min.css and bootstrap-responsive.min.css onto our server and link to it relatively instead of with absolute links it has now. Seeing as how both of those CSS files are Apache License, should be easy. -Dima On Tue, Dec 30, 2014 at 11:54 AM, Elliott Clark wrote: > Yeah the js and css are all loaded though a cdn. It should be pretty easy > to change the urls to be protocol relative and then things should work. > > On Tue, Dec 30, 2014 at 11:17 AM, Enis Söztutar > wrote: > > > I had the same problem. We link to CSS which is http instead of forking > and > > serving it in the site (I believe this is the default practice). My > chrome > > extension refuses to connect to http links causing rendering problems. > > > > Enis > > > > On Tue, Dec 30, 2014 at 11:00 AM, Dima Spivak > > wrote: > > > > > Not sure if this is something to address in this thread or if I should > > file > > > a separate JIRA, but for some reason, accessing the site over https > gets > > > rid of the pretty CSS. Seems to happen in Chrome and Firefox. Anyone > > have a > > > clue why? > > > > > > -Dima > > > > > > On Wed, Dec 17, 2014 at 10:53 AM, Nick Dimiduk > > wrote: > > > > > > > Tweaking the layout of the website goes one of two ways. Either > > there's a > > > > configuration point exposed from the maven skin or there isn't. If > > there > > > > is, it's simply editing the appropriate custom tag in site.xml. If > not, > > > > we'll need to bring back the velocity site.vm template. > Configurations > > > are > > > > documented here: > > > > http://andriusvelykis.github.io/reflow-maven-skin/skin/config.html > > > > > > > > Tweaking visuals is now based on the theme. We can either choose > > another > > > > bootswatch theme, or write our own. That stuff is documented here: > > > > http://andriusvelykis.github.io/reflow-maven-skin/skin/themes/ > > > > > > > > Right now the site is separate from the book. In theory we could move > > > book > > > > content over to become site content, in which case it would be > managed > > by > > > > this same plugin. Short of that, we get into applying style via the > > > docbook > > > > plugin. I've looked into this a bit; it's not nearly as easy as > > applying > > > > reflow-maven-skin. The current leads I have for this are > > > > https://github.com/lcahlander/docbook and > > > > > > > > > > > > > > https://github.com/pmartin/phing-documentation-docbook/commit/8d8f0b81951c4e07420083b204599a6b9ef00537 > > > > . > > > > > > > > Honestly though, if we're looking at moving away from docbook, I > think > > we > > > > should consider a more modern toolset, Sphinx-doc, for example: > > > > http://sphinx-doc.org > > > > > > > > On Tue, Dec 16, 2014 at 8:21 PM, Misty Stanley-Jones < > > > > mstanleyjo...@cloudera.com> wrote: > > > > > > > > > > Absolutely fantastic!!! Got pointers to how tweaking the website > > would > > > > > work? > > > > > > > > > > In the new year I hope to really get started on a restructure of > the > > > Ref > > > > > Guide. I've thought about asking to move to AsciiDOC or MarkDown > but > > > I'd > > > > > like to do a proof-of-concept and see if it really gets us anything > > we > > > > > don't already have with Docbook. It's less of a potential value > since > > > we > > > > > are not hosted on Github, which has great integration with AsciiDOC > > and > > > > > MarkDown. I'm very interested in improving the look and feel of the > > > doc. > > > > I > > > > > think a lot of it will be structural -- reorganizing and > re-chunking > > > > > content. > > > > > > > > > > In the meantime, theming the Docbook is not as hard as you might > > think. > > > > > Have a look at the XSLT files. (I know, famous last words). > > > > > > > > > > On Wed, Dec 17, 2014 at 1:21 PM, Nick Dimiduk > > > > wrote: > > > > > > > > > > > For those not following closely in JIRAland, I've two changes > that > > > are > > > > > > ready to go. First, there's transition over to a bootstrap-based > > > layout > > > > > for > > > > > > our home page. This work is done in HBASE-12688. This improves > the > > > > looks > > > > > of > > > > > > things and will make it pretty easy to update our look-and-feel > in > > > the > > > > > > future. Second, I fixed a build problem where the online book > > wasn't > > > > > > getting its intended styling. This is done in HBASE-12687. I'd > like > > > to > > > > > > update the book to look the same as the front-page, but getting > > > styles > > > > > into > > > > > > docbook is proving non-trivial. > > > > > > > > > > > > Anyway, both of these changes are pushed to master and we're > ready > > > for > > > > a > > > > > > site re-build with the new look. I'll make that happen tomorrow > > > morning > > > > > > unless I hear actionable objections. > > > > > > > > > > > > You can get a preview of things to come over one > > > > > > http://people.apache.org/~ndimiduk/site/ > > > > > > > > > > > > Than
Re: A face-lift for 1.0
Yeah the js and css are all loaded though a cdn. It should be pretty easy to change the urls to be protocol relative and then things should work. On Tue, Dec 30, 2014 at 11:17 AM, Enis Söztutar wrote: > I had the same problem. We link to CSS which is http instead of forking and > serving it in the site (I believe this is the default practice). My chrome > extension refuses to connect to http links causing rendering problems. > > Enis > > On Tue, Dec 30, 2014 at 11:00 AM, Dima Spivak > wrote: > > > Not sure if this is something to address in this thread or if I should > file > > a separate JIRA, but for some reason, accessing the site over https gets > > rid of the pretty CSS. Seems to happen in Chrome and Firefox. Anyone > have a > > clue why? > > > > -Dima > > > > On Wed, Dec 17, 2014 at 10:53 AM, Nick Dimiduk > wrote: > > > > > Tweaking the layout of the website goes one of two ways. Either > there's a > > > configuration point exposed from the maven skin or there isn't. If > there > > > is, it's simply editing the appropriate custom tag in site.xml. If not, > > > we'll need to bring back the velocity site.vm template. Configurations > > are > > > documented here: > > > http://andriusvelykis.github.io/reflow-maven-skin/skin/config.html > > > > > > Tweaking visuals is now based on the theme. We can either choose > another > > > bootswatch theme, or write our own. That stuff is documented here: > > > http://andriusvelykis.github.io/reflow-maven-skin/skin/themes/ > > > > > > Right now the site is separate from the book. In theory we could move > > book > > > content over to become site content, in which case it would be managed > by > > > this same plugin. Short of that, we get into applying style via the > > docbook > > > plugin. I've looked into this a bit; it's not nearly as easy as > applying > > > reflow-maven-skin. The current leads I have for this are > > > https://github.com/lcahlander/docbook and > > > > > > > > > https://github.com/pmartin/phing-documentation-docbook/commit/8d8f0b81951c4e07420083b204599a6b9ef00537 > > > . > > > > > > Honestly though, if we're looking at moving away from docbook, I think > we > > > should consider a more modern toolset, Sphinx-doc, for example: > > > http://sphinx-doc.org > > > > > > On Tue, Dec 16, 2014 at 8:21 PM, Misty Stanley-Jones < > > > mstanleyjo...@cloudera.com> wrote: > > > > > > > > Absolutely fantastic!!! Got pointers to how tweaking the website > would > > > > work? > > > > > > > > In the new year I hope to really get started on a restructure of the > > Ref > > > > Guide. I've thought about asking to move to AsciiDOC or MarkDown but > > I'd > > > > like to do a proof-of-concept and see if it really gets us anything > we > > > > don't already have with Docbook. It's less of a potential value since > > we > > > > are not hosted on Github, which has great integration with AsciiDOC > and > > > > MarkDown. I'm very interested in improving the look and feel of the > > doc. > > > I > > > > think a lot of it will be structural -- reorganizing and re-chunking > > > > content. > > > > > > > > In the meantime, theming the Docbook is not as hard as you might > think. > > > > Have a look at the XSLT files. (I know, famous last words). > > > > > > > > On Wed, Dec 17, 2014 at 1:21 PM, Nick Dimiduk > > > wrote: > > > > > > > > > For those not following closely in JIRAland, I've two changes that > > are > > > > > ready to go. First, there's transition over to a bootstrap-based > > layout > > > > for > > > > > our home page. This work is done in HBASE-12688. This improves the > > > looks > > > > of > > > > > things and will make it pretty easy to update our look-and-feel in > > the > > > > > future. Second, I fixed a build problem where the online book > wasn't > > > > > getting its intended styling. This is done in HBASE-12687. I'd like > > to > > > > > update the book to look the same as the front-page, but getting > > styles > > > > into > > > > > docbook is proving non-trivial. > > > > > > > > > > Anyway, both of these changes are pushed to master and we're ready > > for > > > a > > > > > site re-build with the new look. I'll make that happen tomorrow > > morning > > > > > unless I hear actionable objections. > > > > > > > > > > You can get a preview of things to come over one > > > > > http://people.apache.org/~ndimiduk/site/ > > > > > > > > > > Thanks, > > > > > Nick > > > > > > > > > > On Fri, Dec 5, 2014 at 12:29 PM, Stack wrote: > > > > > > > > > > > > Thanks. I've been having a dialog w/ your crew that has been > going > > > on a > > > > > > while... was wondering if any progress. > > > > > > Grand, > > > > > > St.Ack > > > > > > > > > > > > On Fri, Dec 5, 2014 at 11:55 AM, Ryan Rawson > > > > > wrote: > > > > > > > > > > > > > I'm checking to see if our marketing and web team can help. > The > > > > > > > primary requirement is going to be ditching the mvn:site from > the > > > > > > > front page. "reskinning it" might not be so easy. > > > > > > >
Re: A face-lift for 1.0
I had the same problem. We link to CSS which is http instead of forking and serving it in the site (I believe this is the default practice). My chrome extension refuses to connect to http links causing rendering problems. Enis On Tue, Dec 30, 2014 at 11:00 AM, Dima Spivak wrote: > Not sure if this is something to address in this thread or if I should file > a separate JIRA, but for some reason, accessing the site over https gets > rid of the pretty CSS. Seems to happen in Chrome and Firefox. Anyone have a > clue why? > > -Dima > > On Wed, Dec 17, 2014 at 10:53 AM, Nick Dimiduk wrote: > > > Tweaking the layout of the website goes one of two ways. Either there's a > > configuration point exposed from the maven skin or there isn't. If there > > is, it's simply editing the appropriate custom tag in site.xml. If not, > > we'll need to bring back the velocity site.vm template. Configurations > are > > documented here: > > http://andriusvelykis.github.io/reflow-maven-skin/skin/config.html > > > > Tweaking visuals is now based on the theme. We can either choose another > > bootswatch theme, or write our own. That stuff is documented here: > > http://andriusvelykis.github.io/reflow-maven-skin/skin/themes/ > > > > Right now the site is separate from the book. In theory we could move > book > > content over to become site content, in which case it would be managed by > > this same plugin. Short of that, we get into applying style via the > docbook > > plugin. I've looked into this a bit; it's not nearly as easy as applying > > reflow-maven-skin. The current leads I have for this are > > https://github.com/lcahlander/docbook and > > > > > https://github.com/pmartin/phing-documentation-docbook/commit/8d8f0b81951c4e07420083b204599a6b9ef00537 > > . > > > > Honestly though, if we're looking at moving away from docbook, I think we > > should consider a more modern toolset, Sphinx-doc, for example: > > http://sphinx-doc.org > > > > On Tue, Dec 16, 2014 at 8:21 PM, Misty Stanley-Jones < > > mstanleyjo...@cloudera.com> wrote: > > > > > > Absolutely fantastic!!! Got pointers to how tweaking the website would > > > work? > > > > > > In the new year I hope to really get started on a restructure of the > Ref > > > Guide. I've thought about asking to move to AsciiDOC or MarkDown but > I'd > > > like to do a proof-of-concept and see if it really gets us anything we > > > don't already have with Docbook. It's less of a potential value since > we > > > are not hosted on Github, which has great integration with AsciiDOC and > > > MarkDown. I'm very interested in improving the look and feel of the > doc. > > I > > > think a lot of it will be structural -- reorganizing and re-chunking > > > content. > > > > > > In the meantime, theming the Docbook is not as hard as you might think. > > > Have a look at the XSLT files. (I know, famous last words). > > > > > > On Wed, Dec 17, 2014 at 1:21 PM, Nick Dimiduk > > wrote: > > > > > > > For those not following closely in JIRAland, I've two changes that > are > > > > ready to go. First, there's transition over to a bootstrap-based > layout > > > for > > > > our home page. This work is done in HBASE-12688. This improves the > > looks > > > of > > > > things and will make it pretty easy to update our look-and-feel in > the > > > > future. Second, I fixed a build problem where the online book wasn't > > > > getting its intended styling. This is done in HBASE-12687. I'd like > to > > > > update the book to look the same as the front-page, but getting > styles > > > into > > > > docbook is proving non-trivial. > > > > > > > > Anyway, both of these changes are pushed to master and we're ready > for > > a > > > > site re-build with the new look. I'll make that happen tomorrow > morning > > > > unless I hear actionable objections. > > > > > > > > You can get a preview of things to come over one > > > > http://people.apache.org/~ndimiduk/site/ > > > > > > > > Thanks, > > > > Nick > > > > > > > > On Fri, Dec 5, 2014 at 12:29 PM, Stack wrote: > > > > > > > > > > Thanks. I've been having a dialog w/ your crew that has been going > > on a > > > > > while... was wondering if any progress. > > > > > Grand, > > > > > St.Ack > > > > > > > > > > On Fri, Dec 5, 2014 at 11:55 AM, Ryan Rawson > > > wrote: > > > > > > > > > > > I'm checking to see if our marketing and web team can help. The > > > > > > primary requirement is going to be ditching the mvn:site from the > > > > > > front page. "reskinning it" might not be so easy. > > > > > > > > > > > > -ryan > > > > > > > > > > > > On Thu, Dec 4, 2014 at 9:52 AM, lars hofhansl > > > > wrote: > > > > > > > +1 > > > > > > > I just came across one of the various HBase vs. Cassandra > > articles, > > > > and > > > > > > one of the main tenants of the articles was how much better the > > > > Cassandra > > > > > > documentation was.Thank god we have Misty now. :) > > > > > > > > > > > > > > (not sure how much just a skin would help, but it surely won't > > > hurt
Re: A face-lift for 1.0
Not sure if this is something to address in this thread or if I should file a separate JIRA, but for some reason, accessing the site over https gets rid of the pretty CSS. Seems to happen in Chrome and Firefox. Anyone have a clue why? -Dima On Wed, Dec 17, 2014 at 10:53 AM, Nick Dimiduk wrote: > Tweaking the layout of the website goes one of two ways. Either there's a > configuration point exposed from the maven skin or there isn't. If there > is, it's simply editing the appropriate custom tag in site.xml. If not, > we'll need to bring back the velocity site.vm template. Configurations are > documented here: > http://andriusvelykis.github.io/reflow-maven-skin/skin/config.html > > Tweaking visuals is now based on the theme. We can either choose another > bootswatch theme, or write our own. That stuff is documented here: > http://andriusvelykis.github.io/reflow-maven-skin/skin/themes/ > > Right now the site is separate from the book. In theory we could move book > content over to become site content, in which case it would be managed by > this same plugin. Short of that, we get into applying style via the docbook > plugin. I've looked into this a bit; it's not nearly as easy as applying > reflow-maven-skin. The current leads I have for this are > https://github.com/lcahlander/docbook and > > https://github.com/pmartin/phing-documentation-docbook/commit/8d8f0b81951c4e07420083b204599a6b9ef00537 > . > > Honestly though, if we're looking at moving away from docbook, I think we > should consider a more modern toolset, Sphinx-doc, for example: > http://sphinx-doc.org > > On Tue, Dec 16, 2014 at 8:21 PM, Misty Stanley-Jones < > mstanleyjo...@cloudera.com> wrote: > > > > Absolutely fantastic!!! Got pointers to how tweaking the website would > > work? > > > > In the new year I hope to really get started on a restructure of the Ref > > Guide. I've thought about asking to move to AsciiDOC or MarkDown but I'd > > like to do a proof-of-concept and see if it really gets us anything we > > don't already have with Docbook. It's less of a potential value since we > > are not hosted on Github, which has great integration with AsciiDOC and > > MarkDown. I'm very interested in improving the look and feel of the doc. > I > > think a lot of it will be structural -- reorganizing and re-chunking > > content. > > > > In the meantime, theming the Docbook is not as hard as you might think. > > Have a look at the XSLT files. (I know, famous last words). > > > > On Wed, Dec 17, 2014 at 1:21 PM, Nick Dimiduk > wrote: > > > > > For those not following closely in JIRAland, I've two changes that are > > > ready to go. First, there's transition over to a bootstrap-based layout > > for > > > our home page. This work is done in HBASE-12688. This improves the > looks > > of > > > things and will make it pretty easy to update our look-and-feel in the > > > future. Second, I fixed a build problem where the online book wasn't > > > getting its intended styling. This is done in HBASE-12687. I'd like to > > > update the book to look the same as the front-page, but getting styles > > into > > > docbook is proving non-trivial. > > > > > > Anyway, both of these changes are pushed to master and we're ready for > a > > > site re-build with the new look. I'll make that happen tomorrow morning > > > unless I hear actionable objections. > > > > > > You can get a preview of things to come over one > > > http://people.apache.org/~ndimiduk/site/ > > > > > > Thanks, > > > Nick > > > > > > On Fri, Dec 5, 2014 at 12:29 PM, Stack wrote: > > > > > > > > Thanks. I've been having a dialog w/ your crew that has been going > on a > > > > while... was wondering if any progress. > > > > Grand, > > > > St.Ack > > > > > > > > On Fri, Dec 5, 2014 at 11:55 AM, Ryan Rawson > > wrote: > > > > > > > > > I'm checking to see if our marketing and web team can help. The > > > > > primary requirement is going to be ditching the mvn:site from the > > > > > front page. "reskinning it" might not be so easy. > > > > > > > > > > -ryan > > > > > > > > > > On Thu, Dec 4, 2014 at 9:52 AM, lars hofhansl > > > wrote: > > > > > > +1 > > > > > > I just came across one of the various HBase vs. Cassandra > articles, > > > and > > > > > one of the main tenants of the articles was how much better the > > > Cassandra > > > > > documentation was.Thank god we have Misty now. :) > > > > > > > > > > > > (not sure how much just a skin would help, but it surely won't > > hurt) > > > > > > > > > > > > -- Lars > > > > > > From: Nick Dimiduk > > > > > > To: hbase-dev > > > > > > Sent: Tuesday, December 2, 2014 9:46 AM > > > > > > Subject: A face-lift for 1.0 > > > > > > > > > > > > Heya, > > > > > > > > > > > > In mind of the new release, I was thinking we should clean up our > > > act a > > > > > > little bit in regard to hbase.apache.org and our book. Just > > because > > > > the > > > > > > project started in 2007 doesn't mean we need a site that looks > like > > > > it's > > > > > >
[jira] [Resolved] (HBASE-12762) Region with no hfiles will have the highest locality cost in LocalityCostFunction
[
https://issues.apache.org/jira/browse/HBASE-12762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-12762.
Resolution: Fixed
> Region with no hfiles will have the highest locality cost in
> LocalityCostFunction
> -
>
> Key: HBASE-12762
> URL: https://issues.apache.org/jira/browse/HBASE-12762
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Affects Versions: 0.99.2
>Reporter: cuijianwei
>Assignee: cuijianwei
>Priority: Minor
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12762-trunk.patch
>
>
> The locality cost of region will be computed in LocalityCostFunction.cost as:
> {code}
> double cost() {
> ...
> int index = -1;
> for (int j = 0; j < regionLocations.length; j++) {
> if (regionLocations[j] >= 0 && regionLocations[j] == serverIndex) {
> index = j;
> break;
> }
> }
> if (index < 0) {
> cost += 1; // ==> region with no hfiles will have the highest cost
> } else {
> cost += (double) index / (double) regionLocations.length;
> }
> ...
> }
> {code}
> The region with no hfiles(such as empty region) will have the highest cost
> which represents the worst case that region located in the server with no
> locality for hfiles. However, this might be the best case because there are
> no hlogs for the region. Although the absolute cost value won't affect the
> balance process, will it be more reasonable to have zero cost for such
> regions? such as:
> {code}
>...
> if (index < 0) {
> if (regionLocation.length > 0) { // ==> only consider regions with
> hfiles
> cost += 1;
> }
> } else {
> cost += (double) index / (double) regionLocations.length;
> }
>...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12785) Use FutureTask to timeout the attempt to get the lock for hbck
Ted Yu created HBASE-12785: -- Summary: Use FutureTask to timeout the attempt to get the lock for hbck Key: HBASE-12785 URL: https://issues.apache.org/jira/browse/HBASE-12785 Project: HBase Issue Type: Task Reporter: Ted Yu Priority: Minor In reviewing HBASE-12607, Sean pointed out: It would be nice if we used a [FutureTask|http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/FutureTask.html] to timeout the attempt to get the lock rather than wait the whole period and then fail. This issue is to address Sean's review comment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12784) TestFastFailWithoutTestUtil#testPreemptiveFastFailException50Times sometimes hangs
Ted Yu created HBASE-12784:
--
Summary:
TestFastFailWithoutTestUtil#testPreemptiveFastFailException50Times sometimes
hangs
Key: HBASE-12784
URL: https://issues.apache.org/jira/browse/HBASE-12784
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Priority: Minor
I was running test suite against hadoop 2.7.0-SNAPSHOT and saw this:
{code}
testPreemptiveFastFailException50Times(org.apache.hadoop.hbase.client.TestFastFailWithoutTestUtil)
Time elapsed: 120.013 sec <<< ERROR!
java.lang.Exception: test timed out after 12 milliseconds
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248)
at java.util.concurrent.FutureTask.get(FutureTask.java:111)
at
org.apache.hadoop.hbase.client.TestFastFailWithoutTestUtil.testPreemptiveFastFailException(TestFastFailWithoutTestUtil.java:450)
at
org.apache.hadoop.hbase.client.TestFastFailWithoutTestUtil.testPreemptiveFastFailException50Times(TestFastFailWithoutTestUtil.java:338)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12783) Create efficient RegionLocator implementation
Solomon Duskis created HBASE-12783: -- Summary: Create efficient RegionLocator implementation Key: HBASE-12783 URL: https://issues.apache.org/jira/browse/HBASE-12783 Project: HBase Issue Type: Bug Affects Versions: 1.0.0, 2.0.0 Reporter: Solomon Duskis Assignee: Solomon Duskis A new HRegionLocator that only implements RegionLocator functionality will be more efficient to instantiate than a full HTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-12762) Region with no hfiles will have the highest locality cost in LocalityCostFunction
[
https://issues.apache.org/jira/browse/HBASE-12762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-12762:
No fix version for 0.98 was set on this issue but it was committed there.
TestShell is failing on 0.98 since this change, please see
https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/728 and
https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/729
> Region with no hfiles will have the highest locality cost in
> LocalityCostFunction
> -
>
> Key: HBASE-12762
> URL: https://issues.apache.org/jira/browse/HBASE-12762
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Affects Versions: 0.99.2
>Reporter: cuijianwei
>Assignee: cuijianwei
>Priority: Minor
> Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0
>
> Attachments: HBASE-12762-trunk.patch
>
>
> The locality cost of region will be computed in LocalityCostFunction.cost as:
> {code}
> double cost() {
> ...
> int index = -1;
> for (int j = 0; j < regionLocations.length; j++) {
> if (regionLocations[j] >= 0 && regionLocations[j] == serverIndex) {
> index = j;
> break;
> }
> }
> if (index < 0) {
> cost += 1; // ==> region with no hfiles will have the highest cost
> } else {
> cost += (double) index / (double) regionLocations.length;
> }
> ...
> }
> {code}
> The region with no hfiles(such as empty region) will have the highest cost
> which represents the worst case that region located in the server with no
> locality for hfiles. However, this might be the best case because there are
> no hlogs for the region. Although the absolute cost value won't affect the
> balance process, will it be more reasonable to have zero cost for such
> regions? such as:
> {code}
>...
> if (index < 0) {
> if (regionLocation.length > 0) { // ==> only consider regions with
> hfiles
> cost += 1;
> }
> } else {
> cost += (double) index / (double) regionLocations.length;
> }
>...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
