[jira] [Updated] (HBASE-11399) Improve Quickstart chapter and move Pseudo-distributed and distributed to it
[ https://issues.apache.org/jira/browse/HBASE-11399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11399: Attachment: Chapters_1_and_2.pdf This PDF should not have the spacing issues but is not styled at all, and is just provided as a quick review tool if you don't want to build the docs locally. > Improve Quickstart chapter and move Pseudo-distributed and distributed to it > > > Key: HBASE-11399 > URL: https://issues.apache.org/jira/browse/HBASE-11399 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: Chapters_1_and_2.pdf, Chapters_1_and_2.pdf, > HBASE-11399-1.patch, HBASE-11399.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4931) CopyTable instructions could be improved.
[
https://issues.apache.org/jira/browse/HBASE-4931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14047367#comment-14047367
]
Misty Stanley-Jones commented on HBASE-4931:
bq. The rs.impl and rs.class do not default to ReplicationXYZ. The command line
in this jira is just an example of what you can do. and as far as I know we
don't have those ReplicationXYZ classes in the codebase.
OK, I took that out. Thanks for clearing it up.
bq. the export referenced in the Import, should be with the
first letter uppercase.
Fixed.
bq. At the end "For performance consider the following general options
scanner.caching=100 speculative.execution=false"
bq. speculative execution should be always off and not for performance reason
but to avoid to write data twice which may produce the wrong result.
bq. For the caching maybe say something like >= 100, or some explanation like
an higher value uses more memory but does less round trip to the server which
may improve performance (something like that)
I made this modification. Please double-check. If there is a parameter that
should always be false, why don't we just set it to false in the code?
bq. then there is another reference to the ReplicationRegionInterface and
ReplicationRegionServer which I'm not able to find anywhere in our code base
Removed that one. I'm not even sure why it was there as it was an empty
paragraph.
> CopyTable instructions could be improved.
> -
>
> Key: HBASE-4931
> URL: https://issues.apache.org/jira/browse/HBASE-4931
> Project: HBase
> Issue Type: Bug
> Components: documentation, mapreduce
>Affects Versions: 0.90.4, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-4931.patch
>
>
> The book and the usage instructions could be improved to include more
> details, things caveats and to better explain usage.
> One example in particular, could be updated to refer to
> ReplicationRegionInterface and ReplicationRegionServer in thier current
> locations (o.a.h.h.client.replication and o.a.h.h.replication.regionserver),
> and better explain why one would use particular arguments.
> {code}
> $ bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable
> --rs.class=org.apache.hadoop.hbase.ipc.ReplicationRegionInterface
> --rs.impl=org.apache.hadoop.hbase.regionserver.replication.ReplicationRegionServer
> --starttime=1265875194289 --endtime=1265878794289
> --peer.adr=server1,server2,server3:2181:/hbase TestTable
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-4931) CopyTable instructions could be improved.
[
https://issues.apache.org/jira/browse/HBASE-4931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-4931:
---
Attachment: HBASE-4931-1.patch
> CopyTable instructions could be improved.
> -
>
> Key: HBASE-4931
> URL: https://issues.apache.org/jira/browse/HBASE-4931
> Project: HBase
> Issue Type: Bug
> Components: documentation, mapreduce
>Affects Versions: 0.90.4, 0.92.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-4931-1.patch, HBASE-4931.patch
>
>
> The book and the usage instructions could be improved to include more
> details, things caveats and to better explain usage.
> One example in particular, could be updated to refer to
> ReplicationRegionInterface and ReplicationRegionServer in thier current
> locations (o.a.h.h.client.replication and o.a.h.h.replication.regionserver),
> and better explain why one would use particular arguments.
> {code}
> $ bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable
> --rs.class=org.apache.hadoop.hbase.ipc.ReplicationRegionInterface
> --rs.impl=org.apache.hadoop.hbase.regionserver.replication.ReplicationRegionServer
> --starttime=1265875194289 --endtime=1265878794289
> --peer.adr=server1,server2,server3:2181:/hbase TestTable
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11399) Improve Quickstart chapter and move Pseudo-distributed and distributed to it
[ https://issues.apache.org/jira/browse/HBASE-11399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11399: Attachment: HBASE-11399-2.patch Sorry about missing that. This patch fixes the port errors. > Improve Quickstart chapter and move Pseudo-distributed and distributed to it > > > Key: HBASE-11399 > URL: https://issues.apache.org/jira/browse/HBASE-11399 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: Chapters_1_and_2.pdf, Chapters_1_and_2.pdf, > HBASE-11399-1.patch, HBASE-11399-2.patch, HBASE-11399.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11399) Improve Quickstart chapter and move Pseudo-distributed and distributed to it
[ https://issues.apache.org/jira/browse/HBASE-11399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11399: Attachment: HBASE-11399-2.patch Rebased and regenerated patch. > Improve Quickstart chapter and move Pseudo-distributed and distributed to it > > > Key: HBASE-11399 > URL: https://issues.apache.org/jira/browse/HBASE-11399 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: Chapters_1_and_2.pdf, Chapters_1_and_2.pdf, > HBASE-11399-1.patch, HBASE-11399-2.patch, HBASE-11399.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11399) Improve Quickstart chapter and move Pseudo-distributed and distributed to it
[ https://issues.apache.org/jira/browse/HBASE-11399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11399: Attachment: (was: HBASE-11399-2.patch) > Improve Quickstart chapter and move Pseudo-distributed and distributed to it > > > Key: HBASE-11399 > URL: https://issues.apache.org/jira/browse/HBASE-11399 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: Chapters_1_and_2.pdf, Chapters_1_and_2.pdf, > HBASE-11399-1.patch, HBASE-11399-2.patch, HBASE-11399.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048371#comment-14048371 ] Misty Stanley-Jones commented on HBASE-11316: - Working on this today. Do you prefer HFile or StoreFile? I didn't actually realize they were the same thing when I was originally writing this. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: 9.7.pdf, HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316.patch > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11316: Attachment: HBASE-11316-4.patch * Fixed typos I introduced into hbase-default.xml * Fixed some invalid XML that was introduced by other patches and came in via a rebase * Tried to unify terminology but I need clarification here. The old compaction docs used StoreFile, but [~stack]'s previous comment seemed to imply that he preferred HFile. Same thing with Store vs ColumnFamily. It's weird to say that a ColumnFamily has HFiles. Isn't it more that the HFiles represent the ColumnFamily's data? Anyway I played around with this and tried to be consistent in using HFile instead of StoreFile, Column Family instead of Store. This is easy to revert if I need to -- please give me your opinions. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: 9.7.pdf, HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316.patch > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048483#comment-14048483 ] Misty Stanley-Jones commented on HBASE-11316: - Got clarification from JD that it is most appropriate to talk about StoreFiles and Stores, not HFiles and ColumnFamilies. I'll update that patch to reflect that. Sorry about the churn. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: 9.7.pdf, HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316.patch > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048503#comment-14048503 ] Misty Stanley-Jones commented on HBASE-11316: - Ready for review (again). > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11316: Attachment: (was: 9.7.pdf) > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11316: Attachment: HBASE-11316-5.patch > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11316: Attachment: ch9_compactions.pdf PDF attached for easier review -- did not include the generated hbase-default.xml contents. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones reassigned HBASE-10202: --- Assignee: Misty Stanley-Jones (was: Jean-Marc Spaggiari) > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.98.0, 0.94.14, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049354#comment-14049354 ] Misty Stanley-Jones commented on HBASE-11316: - So, [~jdcryans], [~esteban], and I talked about this yesterday and he seemed to have the opinion that we should be using StoreFile and Store, since we could support other types of StoreFiles besides HFile later on. Based on that, I changed it all back. Now I will hold off on further changes until I get more consistent advice about how I Should be referring to these things. :) > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049405#comment-14049405 ] Misty Stanley-Jones commented on HBASE-11316: - OK, then how do they relate? Especially Store / Column Family. It seems really strange to me to talk about an HFile being "in" a Column Family. Can you help me out? > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11321) Restructure Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11321: Attachment: HBase Reference Guide 2.0 -- Proposed Structure Version 2 - incorporating feedback so far. > Restructure Ref Guide > - > > Key: HBASE-11321 > URL: https://issues.apache.org/jira/browse/HBASE-11321 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBase Reference Guide 2.0 -- Proposed Structure, HBase > Reference Guide 2.0 -- Proposed Structure > > > Reorganize Ref Guide to make the important things more prominent and easier > to find. This will be a big multi-step task. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-10202: Status: Patch Available (was: Open) Let me know if this is sufficient. > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.96.1.1, 0.94.14, 0.98.0, 0.99.0 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-10202: Attachment: HBASE-10202.patch > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.98.0, 0.94.14, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-2328) Make important configurations more obvious to new users
[ https://issues.apache.org/jira/browse/HBASE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones reassigned HBASE-2328: -- Assignee: Misty Stanley-Jones > Make important configurations more obvious to new users > --- > > Key: HBASE-2328 > URL: https://issues.apache.org/jira/browse/HBASE-2328 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Jean-Daniel Cryans >Assignee: Misty Stanley-Jones > Labels: moved_from_0_20_5 > Attachments: HBASE-2328.txt, notsoquick.html > > > Over the last 2 weeks, I encountered many situations where people didn't set > file descriptors and xcievers higher and that was causing a ton of problems > that are hard to debug if you're not used to them. To improve that we should: > - Refuse to start HBase if ulimit -n returns some small number smaller than > 2048, or at least print out in big red blinking letters that the current > configuration is bad and then link to a simple troubleshooting entry on the > wiki. > - Write a clearer Getting Started document where we don't give as much > explanations but add more stuff like "this is what your > hbase-site.xml/hdfs-site/xml should look like now" and give a complete file > example. At this point we don't even give a number for xcievers and we expect > new users to come up with one. > Any other low hanging fruit others can think of? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-10303) Have snappy support properly documented would be helpful to hadoop and hbase users
[ https://issues.apache.org/jira/browse/HBASE-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones reassigned HBASE-10303: --- Assignee: Misty Stanley-Jones > Have snappy support properly documented would be helpful to hadoop and hbase > users > -- > > Key: HBASE-10303 > URL: https://issues.apache.org/jira/browse/HBASE-10303 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Rural Hunter >Assignee: Misty Stanley-Jones > > The currentl document for configuring snappy > support(http://hbase.apache.org/book/snappy.compression.html) is not complete > and it's a bit obscure. IMO, there are several improvments can be made: > 1. Describe the relationship among hadoop,hbase,snappy. Is the snappy > actually needed by hadoop hdfs or hbase itself? That's to make clear if you > need to configure snappy support in hbase or hadoop. > 2. It didn't mention the default hadoop binary package is compiled without > snappy support and you need to compile it with snappy option manually. > Actually it didn't work with any native libs on 64 bits OS as the > libhadoop.so in the binary package is only for 32 bits OS(this of course is a > hadoop issue not hbase. but it's good to mention it.). > 3. In my experience, I actually need to install both snappy and > hadoop-snappy. So the doc lack of the steps to install hadoop-snappy. > 4. During my set up, I found difference where hadoop and hbase to pick up the > native lib files. hadoop picks those files in ./lib while hbase picks in > ./lib/[PLATFORM]. If it's correct, it can also be mentioned. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-2328) Make important configurations more obvious to new users
[ https://issues.apache.org/jira/browse/HBASE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones resolved HBASE-2328. Resolution: Implemented > Make important configurations more obvious to new users > --- > > Key: HBASE-2328 > URL: https://issues.apache.org/jira/browse/HBASE-2328 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Jean-Daniel Cryans >Assignee: Misty Stanley-Jones > Labels: moved_from_0_20_5 > Attachments: HBASE-2328.txt, notsoquick.html > > > Over the last 2 weeks, I encountered many situations where people didn't set > file descriptors and xcievers higher and that was causing a ton of problems > that are hard to debug if you're not used to them. To improve that we should: > - Refuse to start HBase if ulimit -n returns some small number smaller than > 2048, or at least print out in big red blinking letters that the current > configuration is bad and then link to a simple troubleshooting entry on the > wiki. > - Write a clearer Getting Started document where we don't give as much > explanations but add more stuff like "this is what your > hbase-site.xml/hdfs-site/xml should look like now" and give a complete file > example. At this point we don't even give a number for xcievers and we expect > new users to come up with one. > Any other low hanging fruit others can think of? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-2328) Make important configurations more obvious to new users
[ https://issues.apache.org/jira/browse/HBASE-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049508#comment-14049508 ] Misty Stanley-Jones commented on HBASE-2328: These things appear to all be addressed now. Please reopen or open a new JIRA if this is not the case. > Make important configurations more obvious to new users > --- > > Key: HBASE-2328 > URL: https://issues.apache.org/jira/browse/HBASE-2328 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Jean-Daniel Cryans > Labels: moved_from_0_20_5 > Attachments: HBASE-2328.txt, notsoquick.html > > > Over the last 2 weeks, I encountered many situations where people didn't set > file descriptors and xcievers higher and that was causing a ton of problems > that are hard to debug if you're not used to them. To improve that we should: > - Refuse to start HBase if ulimit -n returns some small number smaller than > 2048, or at least print out in big red blinking letters that the current > configuration is bad and then link to a simple troubleshooting entry on the > wiki. > - Write a clearer Getting Started document where we don't give as much > explanations but add more stuff like "this is what your > hbase-site.xml/hdfs-site/xml should look like now" and give a complete file > example. At this point we don't even give a number for xcievers and we expect > new users to come up with one. > Any other low hanging fruit others can think of? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-10303) Have snappy support properly documented would be helpful to hadoop and hbase users
[ https://issues.apache.org/jira/browse/HBASE-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones resolved HBASE-10303. - Resolution: Fixed > Have snappy support properly documented would be helpful to hadoop and hbase > users > -- > > Key: HBASE-10303 > URL: https://issues.apache.org/jira/browse/HBASE-10303 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Rural Hunter >Assignee: Misty Stanley-Jones > > The currentl document for configuring snappy > support(http://hbase.apache.org/book/snappy.compression.html) is not complete > and it's a bit obscure. IMO, there are several improvments can be made: > 1. Describe the relationship among hadoop,hbase,snappy. Is the snappy > actually needed by hadoop hdfs or hbase itself? That's to make clear if you > need to configure snappy support in hbase or hadoop. > 2. It didn't mention the default hadoop binary package is compiled without > snappy support and you need to compile it with snappy option manually. > Actually it didn't work with any native libs on 64 bits OS as the > libhadoop.so in the binary package is only for 32 bits OS(this of course is a > hadoop issue not hbase. but it's good to mention it.). > 3. In my experience, I actually need to install both snappy and > hadoop-snappy. So the doc lack of the steps to install hadoop-snappy. > 4. During my set up, I found difference where hadoop and hbase to pick up the > native lib files. hadoop picks those files in ./lib while hbase picks in > ./lib/[PLATFORM]. If it's correct, it can also be mentioned. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10303) Have snappy support properly documented would be helpful to hadoop and hbase users
[ https://issues.apache.org/jira/browse/HBASE-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049511#comment-14049511 ] Misty Stanley-Jones commented on HBASE-10303: - I rewrote that whole chapter and I'm awaiting review of it at HBASE-11400 so closing this one since Stack already did most of the work (years ago). > Have snappy support properly documented would be helpful to hadoop and hbase > users > -- > > Key: HBASE-10303 > URL: https://issues.apache.org/jira/browse/HBASE-10303 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Rural Hunter >Assignee: Misty Stanley-Jones > > The currentl document for configuring snappy > support(http://hbase.apache.org/book/snappy.compression.html) is not complete > and it's a bit obscure. IMO, there are several improvments can be made: > 1. Describe the relationship among hadoop,hbase,snappy. Is the snappy > actually needed by hadoop hdfs or hbase itself? That's to make clear if you > need to configure snappy support in hbase or hadoop. > 2. It didn't mention the default hadoop binary package is compiled without > snappy support and you need to compile it with snappy option manually. > Actually it didn't work with any native libs on 64 bits OS as the > libhadoop.so in the binary package is only for 32 bits OS(this of course is a > hadoop issue not hbase. but it's good to mention it.). > 3. In my experience, I actually need to install both snappy and > hadoop-snappy. So the doc lack of the steps to install hadoop-snappy. > 4. During my set up, I found difference where hadoop and hbase to pick up the > native lib files. hadoop picks those files in ./lib while hbase picks in > ./lib/[PLATFORM]. If it's correct, it can also be mentioned. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049588#comment-14049588 ] Misty Stanley-Jones commented on HBASE-11316: - Well, it seems like you and JD are in disagreement on this so I am unsure how to proceed. :) > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11316: Attachment: HBASE-11316-6.patch Added a little explanation of StoreFile / HFile and Store/ColumnFamily at the beginning of the section on compaction! > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316-6.patch, HBASE-11316.patch, ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050762#comment-14050762 ] Misty Stanley-Jones commented on HBASE-10202: - Perhaps my instinct to label the other stuff 'legacy' was wrong. I will have another crack and submit another patch in a few hours. > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.98.0, 0.94.14, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10202) Documentation is lacking information about rolling-restart.sh script.
[ https://issues.apache.org/jira/browse/HBASE-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-10202: Attachment: HBASE-10202-1.patch Refactored this section into three sub-sections: rolling-restart.sh, manual rolling restart, and logic to consider when creating your own rolling restart script. Lost the "legacy" label and gave some info about when you would and wouldn't use the rolling-restart.sh. > Documentation is lacking information about rolling-restart.sh script. > - > > Key: HBASE-10202 > URL: https://issues.apache.org/jira/browse/HBASE-10202 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.98.0, 0.94.14, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Misty Stanley-Jones > Attachments: HBASE-10202-1.patch, HBASE-10202.patch > > > Current documentation is talking about graceful_stop.sh and how to do a > rolling restart but is not talking about the rolling-restart.sh script. We > need to document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11317) Mockito section of HBase Book missing examples
[
https://issues.apache.org/jira/browse/HBASE-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14053237#comment-14053237
]
Misty Stanley-Jones commented on HBASE-11317:
-
http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/
good resource.
> Mockito section of HBase Book missing examples
> --
>
> Key: HBASE-11317
> URL: https://issues.apache.org/jira/browse/HBASE-11317
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Mike Drob
>Assignee: Misty Stanley-Jones
>Priority: Trivial
>
> The section at http://hbase.apache.org/book.html#mockito only has a {{TODO}}
> where examples should go.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (HBASE-11317) Mockito section of HBase Book missing examples
[
https://issues.apache.org/jira/browse/HBASE-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones reassigned HBASE-11317:
---
Assignee: Misty Stanley-Jones
> Mockito section of HBase Book missing examples
> --
>
> Key: HBASE-11317
> URL: https://issues.apache.org/jira/browse/HBASE-11317
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Mike Drob
>Assignee: Misty Stanley-Jones
>Priority: Trivial
>
> The section at http://hbase.apache.org/book.html#mockito only has a {{TODO}}
> where examples should go.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-1.patch Had to rebase. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400.patch > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11317) Expand unit testing to cover Mockito and MRUnit and give more examples
[
https://issues.apache.org/jira/browse/HBASE-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-11317:
Summary: Expand unit testing to cover Mockito and MRUnit and give more
examples (was: Mockito section of HBase Book missing examples)
> Expand unit testing to cover Mockito and MRUnit and give more examples
> --
>
> Key: HBASE-11317
> URL: https://issues.apache.org/jira/browse/HBASE-11317
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Mike Drob
>Assignee: Misty Stanley-Jones
>Priority: Trivial
>
> The section at http://hbase.apache.org/book.html#mockito only has a {{TODO}}
> where examples should go.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11317) Expand unit testing to cover Mockito and MRUnit and give more examples
[
https://issues.apache.org/jira/browse/HBASE-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-11317:
Attachment: HBASE-11317.patch
Integrated info taken from
http://blog.cloudera.com/blog/2013/09/how-to-test-hbase-applications-using-popular-tools/
> Expand unit testing to cover Mockito and MRUnit and give more examples
> --
>
> Key: HBASE-11317
> URL: https://issues.apache.org/jira/browse/HBASE-11317
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Mike Drob
>Assignee: Misty Stanley-Jones
>Priority: Trivial
> Attachments: HBASE-11317.patch
>
>
> The section at http://hbase.apache.org/book.html#mockito only has a {{TODO}}
> where examples should go.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11317) Expand unit testing to cover Mockito and MRUnit and give more examples
[
https://issues.apache.org/jira/browse/HBASE-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-11317:
Status: Patch Available (was: Open)
Care to have a look, [~mdrob]
> Expand unit testing to cover Mockito and MRUnit and give more examples
> --
>
> Key: HBASE-11317
> URL: https://issues.apache.org/jira/browse/HBASE-11317
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Mike Drob
>Assignee: Misty Stanley-Jones
>Priority: Trivial
> Attachments: HBASE-11317.patch
>
>
> The section at http://hbase.apache.org/book.html#mockito only has a {{TODO}}
> where examples should go.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11321) Restructure Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11321: Attachment: HBase Reference Guide 2.0 -- Proposed Structure Renamed "Antipatterns" for more clarity and expanded upon some other sections. Sorry I haven't done these as Diffs -- that might have been better. > Restructure Ref Guide > - > > Key: HBASE-11321 > URL: https://issues.apache.org/jira/browse/HBASE-11321 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBase Reference Guide 2.0 -- Proposed Structure, HBase > Reference Guide 2.0 -- Proposed Structure, HBase Reference Guide 2.0 -- > Proposed Structure > > > Reorganize Ref Guide to make the important things more prominent and easier > to find. This will be a big multi-step task. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11101) Documentation review
[ https://issues.apache.org/jira/browse/HBASE-11101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14053354#comment-14053354 ] Misty Stanley-Jones commented on HBASE-11101: - This is kind of nebulous. I will do my best. I'd appreciate pointers to the highest-priority areas. > Documentation review > > > Key: HBASE-11101 > URL: https://issues.apache.org/jira/browse/HBASE-11101 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Misty Stanley-Jones > Fix For: 0.99.0 > > > It would be good to go over the documentation once and update missing parts > and clean up for 1.0. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones reassigned HBASE-9005: -- Assignee: Misty Stanley-Jones > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0 > > Attachments: 9005.txt > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14053363#comment-14053363 ] Misty Stanley-Jones commented on HBASE-9005: I don't know why this didn't go in, it seemed like it was done. I'll integrate this into the current state of docs and make a new patch. > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0 > > Attachments: 9005.txt > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-9005: --- Attachment: HBASE-11317.patch Replicated Lars' patch (couldn't get it to apply automatically) and massaged the wording a bit. Please review. > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0 > > Attachments: 9005.txt, HBASE-11317.patch > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-9005: --- Status: Patch Available (was: Open) > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0 > > Attachments: 9005.txt, HBASE-11317.patch > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054187#comment-14054187 ] Misty Stanley-Jones commented on HBASE-9005: That's because I uploaded the wrong patch. Fixed. Please review HBASE-11317 about the changes related to unit testing. > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0 > > Attachments: 9005.txt, HBASE-11317.patch > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-9005: --- Attachment: HBASE-9005-1.patch > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0 > > Attachments: 9005.txt, HBASE-9005-1.patch > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-9005) Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete markers
[ https://issues.apache.org/jira/browse/HBASE-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-9005: --- Attachment: (was: HBASE-11317.patch) > Improve documentation around KEEP_DELETED_CELLS, time range scans, and delete > markers > - > > Key: HBASE-9005 > URL: https://issues.apache.org/jira/browse/HBASE-9005 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Lars Hofhansl >Assignee: Misty Stanley-Jones >Priority: Minor > Fix For: 0.99.0 > > Attachments: 9005.txt, HBASE-9005-1.patch > > > Without KEEP_DELETED_CELLS all timerange queries are broken if their range > covers a delete marker. > As some internal discussions with colleagues showed, this feature is not well > understand and documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
Misty Stanley-Jones created HBASE-11476: --- Summary: Expand 'Conceptual View' section of Data Model chapter Key: HBASE-11476 URL: https://issues.apache.org/jira/browse/HBASE-11476 Project: HBase Issue Type: Bug Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Could use some updating and expansion to emphasize the differences between HBase and an RDBMS. I found http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable which is just excellent and we should link to it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
[ https://issues.apache.org/jira/browse/HBASE-11476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11476: Component/s: documentation > Expand 'Conceptual View' section of Data Model chapter > --- > > Key: HBASE-11476 > URL: https://issues.apache.org/jira/browse/HBASE-11476 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > Could use some updating and expansion to emphasize the differences between > HBase and an RDBMS. I found > http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable > which is just excellent and we should link to it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
[ https://issues.apache.org/jira/browse/HBASE-11476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11476: Attachment: HBASE-11476.patch > Expand 'Conceptual View' section of Data Model chapter > --- > > Key: HBASE-11476 > URL: https://issues.apache.org/jira/browse/HBASE-11476 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11476.patch > > > Could use some updating and expansion to emphasize the differences between > HBase and an RDBMS. I found > http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable > which is just excellent and we should link to it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11477) book.xml has Docbook validity issues (again)
Misty Stanley-Jones created HBASE-11477: --- Summary: book.xml has Docbook validity issues (again) Key: HBASE-11477 URL: https://issues.apache.org/jira/browse/HBASE-11477 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11476) Expand 'Conceptual View' section of Data Model chapter
[ https://issues.apache.org/jira/browse/HBASE-11476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11476: Status: Patch Available (was: Open) I added some links to resources that explain it better (IMO) than the Ref Guide does. Then I tried to expand the existing doc (the Data Model chapter introduction, Conceptual View, and Physical View sections) to be more clear, including adding a second row to the table along with all the versions of com.cnn.www, because it wasn't very clear to me that the "rows" there were just versions of the same row. Review and feedback is appreciated. I definitely think we can do better here. I think we should lose the tabular view altogether -- I find the JSON-esque view to be much clearer. But it is also possible I have really misunderstood something and gotten something horribly wrong. By the way I started delving into this when I wanted to expand the docs about gets and scans and realized I didn't understand enough about the data model yet. I suspect I am not the only one! > Expand 'Conceptual View' section of Data Model chapter > --- > > Key: HBASE-11476 > URL: https://issues.apache.org/jira/browse/HBASE-11476 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11476.patch > > > Could use some updating and expansion to emphasize the differences between > HBase and an RDBMS. I found > http://jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable > which is just excellent and we should link to it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11477) book.xml has Docbook validity issues (again)
[ https://issues.apache.org/jira/browse/HBASE-11477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11477: Attachment: HBASE-11477.patch No changes to content, only Docbook validity. > book.xml has Docbook validity issues (again) > > > Key: HBASE-11477 > URL: https://issues.apache.org/jira/browse/HBASE-11477 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11477.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11477) book.xml has Docbook validity issues (again)
[ https://issues.apache.org/jira/browse/HBASE-11477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11477: Status: Patch Available (was: Open) > book.xml has Docbook validity issues (again) > > > Key: HBASE-11477 > URL: https://issues.apache.org/jira/browse/HBASE-11477 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11477.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11483) Check the rest of the book for new XML validity errors and fix
Misty Stanley-Jones created HBASE-11483: --- Summary: Check the rest of the book for new XML validity errors and fix Key: HBASE-11483 URL: https://issues.apache.org/jira/browse/HBASE-11483 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11483) Check the rest of the book for new XML validity errors and fix
[ https://issues.apache.org/jira/browse/HBASE-11483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11483: Attachment: HBASE-11483.patch Just a couple. Someone got really creative with Docbook by adding tags and a few others. > Check the rest of the book for new XML validity errors and fix > -- > > Key: HBASE-11483 > URL: https://issues.apache.org/jira/browse/HBASE-11483 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11483.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11483) Check the rest of the book for new XML validity errors and fix
[ https://issues.apache.org/jira/browse/HBASE-11483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11483: Status: Patch Available (was: Open) Should be an easy review -- no content changes. > Check the rest of the book for new XML validity errors and fix > -- > > Key: HBASE-11483 > URL: https://issues.apache.org/jira/browse/HBASE-11483 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11483.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-6626) Add a chapter on HDFS in the troubleshooting section of the HBase reference guide.
[ https://issues.apache.org/jira/browse/HBASE-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones reassigned HBASE-6626: -- Assignee: Misty Stanley-Jones (was: Doug Meil) > Add a chapter on HDFS in the troubleshooting section of the HBase reference > guide. > -- > > Key: HBASE-6626 > URL: https://issues.apache.org/jira/browse/HBASE-6626 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Misty Stanley-Jones >Priority: Blocker > Attachments: troubleshooting.txt > > > I looked mainly at the major failure case, but here is what I have: > New sub chapter in the existing chapter "Troubleshooting and Debugging > HBase": "HDFS & HBASE" > 1) HDFS & HBase > 2) Connection related settings > 2.1) Number of retries > 2.2) Timeouts > 3) Log samples > 1) HDFS & HBase > HBase uses HDFS to store its HFile, i.e. the core HBase files and the > Write-Ahead-Logs, i.e. the files that will be used to restore the data after > a crash. > In both cases, the reliability of HBase comes from the fact that HDFS writes > the data to multiple locations. To be efficient, HBase needs the data to be > available locally, hence it's highly recommended to have the HDFS datanode on > the same machines as the HBase Region Servers. > Detailed information on how HDFS works can be found at [1]. > Important features are: > - HBase is a client application of HDFS, i.e. uses the HDFS DFSClient class. > This class can appears in HBase logs with other HDFS client related logs. > - Some HDFS settings are HDFS-server-side, i.e. must be set on the HDFS > side, while some other are HDFS-client-side, i.e. must be set in HBase, while > some other must be set in both places. > - the HDFS writes are pipelined from one datanode to another. When writing, > there are communications between: > - HBase and HDFS namenode, through the HDFS client classes. > - HBase and HDFS datanodes, through the HDFS client classes. > - HDFS datanode between themselves: issues on these communications are in > HDFS logs, not HBase. HDFS writes are always local when possible. As a > consequence, there should not be much write error in HBase Region Servers: > they write to the local datanode. If this datanode can't replicate the > blocks, it will appear in its logs, not in the region servers logs. > - datanodes can be contacted through the ipc.Client interface (once again > this class can shows up in HBase logs) and the data transfer interface > (usually shows up as the DataNode class in the HBase logs). There are on > different ports (defaults being: 50010 and 50020). > - To understand exactly what's going on, you must look that the HDFS log > files as well: HBase logs represent the client side. > - With the default setting, HDFS needs 630s to mark a datanode as dead. For > this reason, this node will still be tried by HBase or by other datanodes > when writing and reading until HDFS definitively decides it's dead. This will > add some extras lines in the logs. This monitoring is performed by the > NameNode. > - The HDFS clients (i.e. HBase using HDFS client code) don't fully rely on > the NameNode, but can mark temporally a node as dead if they had an error > when they tried to use it. > 2) Settings for retries and timeouts > 2.1) Retries > ipc.client.connect.max.retries > Default 10 > Indicates the number of retries a client will make to establish a server > connection. Not taken into account if the error is a SocketTimeout. In this > case the number of retries is 45 (fixed on branch, HADOOP-7932 or in > HADOOP-7397). For SASL, the number of retries is hard-coded to 15. Can be > increased, especially if the socket timeouts have been lowered. > ipc.client.connect.max.retries.on.timeouts > Default 45 > If you have HADOOP-7932, max number of retries on timeout. Counter is > different than ipc.client.connect.max.retries so if you mix the socket errors > you will get 55 retries with the default values. Could be lowered, once it is > available. With HADOOP-7397 ipc.client.connect.max.retries is reused so there > would be 10 tries. > dfs.client.block.write.retries > Default 3 > Number of tries for the client when writing a block. After a failure, will > connect to the namenode a get a new location, sending the list of the > datanodes already tried without success. Could be increased, especially if > the socket timeouts have been lowered. See HBASE-6490. > dfs.client.block.write.locateFollowingBlock.retries > Default 5 > Number of retries to the namenode when the client got > NotReplicatedYetException, i.e. the existing nodes of the files are not yet > replicated to dfs.r
[jira] [Updated] (HBASE-6626) Add a chapter on HDFS in the troubleshooting section of the HBase reference guide.
[ https://issues.apache.org/jira/browse/HBASE-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-6626: --- Attachment: HBASE-6626.patch I made an attempt. I did not integrate the info in the comments but I did check the initial content and updated the Hadoop parameters and defaults where needed. I left a couple of the parameters out because they didn't seem to exist anymore or were marked as 'expert' in the HDFS config docs. I would consider 'expert' parameters for HDFS to be out of scope and possibly dangerous for HBase to recommend tweaking. WDYT? > Add a chapter on HDFS in the troubleshooting section of the HBase reference > guide. > -- > > Key: HBASE-6626 > URL: https://issues.apache.org/jira/browse/HBASE-6626 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Misty Stanley-Jones >Priority: Blocker > Attachments: HBASE-6626.patch, troubleshooting.txt > > > I looked mainly at the major failure case, but here is what I have: > New sub chapter in the existing chapter "Troubleshooting and Debugging > HBase": "HDFS & HBASE" > 1) HDFS & HBase > 2) Connection related settings > 2.1) Number of retries > 2.2) Timeouts > 3) Log samples > 1) HDFS & HBase > HBase uses HDFS to store its HFile, i.e. the core HBase files and the > Write-Ahead-Logs, i.e. the files that will be used to restore the data after > a crash. > In both cases, the reliability of HBase comes from the fact that HDFS writes > the data to multiple locations. To be efficient, HBase needs the data to be > available locally, hence it's highly recommended to have the HDFS datanode on > the same machines as the HBase Region Servers. > Detailed information on how HDFS works can be found at [1]. > Important features are: > - HBase is a client application of HDFS, i.e. uses the HDFS DFSClient class. > This class can appears in HBase logs with other HDFS client related logs. > - Some HDFS settings are HDFS-server-side, i.e. must be set on the HDFS > side, while some other are HDFS-client-side, i.e. must be set in HBase, while > some other must be set in both places. > - the HDFS writes are pipelined from one datanode to another. When writing, > there are communications between: > - HBase and HDFS namenode, through the HDFS client classes. > - HBase and HDFS datanodes, through the HDFS client classes. > - HDFS datanode between themselves: issues on these communications are in > HDFS logs, not HBase. HDFS writes are always local when possible. As a > consequence, there should not be much write error in HBase Region Servers: > they write to the local datanode. If this datanode can't replicate the > blocks, it will appear in its logs, not in the region servers logs. > - datanodes can be contacted through the ipc.Client interface (once again > this class can shows up in HBase logs) and the data transfer interface > (usually shows up as the DataNode class in the HBase logs). There are on > different ports (defaults being: 50010 and 50020). > - To understand exactly what's going on, you must look that the HDFS log > files as well: HBase logs represent the client side. > - With the default setting, HDFS needs 630s to mark a datanode as dead. For > this reason, this node will still be tried by HBase or by other datanodes > when writing and reading until HDFS definitively decides it's dead. This will > add some extras lines in the logs. This monitoring is performed by the > NameNode. > - The HDFS clients (i.e. HBase using HDFS client code) don't fully rely on > the NameNode, but can mark temporally a node as dead if they had an error > when they tried to use it. > 2) Settings for retries and timeouts > 2.1) Retries > ipc.client.connect.max.retries > Default 10 > Indicates the number of retries a client will make to establish a server > connection. Not taken into account if the error is a SocketTimeout. In this > case the number of retries is 45 (fixed on branch, HADOOP-7932 or in > HADOOP-7397). For SASL, the number of retries is hard-coded to 15. Can be > increased, especially if the socket timeouts have been lowered. > ipc.client.connect.max.retries.on.timeouts > Default 45 > If you have HADOOP-7932, max number of retries on timeout. Counter is > different than ipc.client.connect.max.retries so if you mix the socket errors > you will get 55 retries with the default values. Could be lowered, once it is > available. With HADOOP-7397 ipc.client.connect.max.retries is reused so there > would be 10 tries. > dfs.client.block.write.retries > Default 3 > Number of tries for the client when writing a block. After a failure, will > connect to
[jira] [Updated] (HBASE-6626) Add a chapter on HDFS in the troubleshooting section of the HBase reference guide.
[ https://issues.apache.org/jira/browse/HBASE-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-6626: --- Status: Patch Available (was: Open) > Add a chapter on HDFS in the troubleshooting section of the HBase reference > guide. > -- > > Key: HBASE-6626 > URL: https://issues.apache.org/jira/browse/HBASE-6626 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Misty Stanley-Jones >Priority: Blocker > Attachments: HBASE-6626.patch, troubleshooting.txt > > > I looked mainly at the major failure case, but here is what I have: > New sub chapter in the existing chapter "Troubleshooting and Debugging > HBase": "HDFS & HBASE" > 1) HDFS & HBase > 2) Connection related settings > 2.1) Number of retries > 2.2) Timeouts > 3) Log samples > 1) HDFS & HBase > HBase uses HDFS to store its HFile, i.e. the core HBase files and the > Write-Ahead-Logs, i.e. the files that will be used to restore the data after > a crash. > In both cases, the reliability of HBase comes from the fact that HDFS writes > the data to multiple locations. To be efficient, HBase needs the data to be > available locally, hence it's highly recommended to have the HDFS datanode on > the same machines as the HBase Region Servers. > Detailed information on how HDFS works can be found at [1]. > Important features are: > - HBase is a client application of HDFS, i.e. uses the HDFS DFSClient class. > This class can appears in HBase logs with other HDFS client related logs. > - Some HDFS settings are HDFS-server-side, i.e. must be set on the HDFS > side, while some other are HDFS-client-side, i.e. must be set in HBase, while > some other must be set in both places. > - the HDFS writes are pipelined from one datanode to another. When writing, > there are communications between: > - HBase and HDFS namenode, through the HDFS client classes. > - HBase and HDFS datanodes, through the HDFS client classes. > - HDFS datanode between themselves: issues on these communications are in > HDFS logs, not HBase. HDFS writes are always local when possible. As a > consequence, there should not be much write error in HBase Region Servers: > they write to the local datanode. If this datanode can't replicate the > blocks, it will appear in its logs, not in the region servers logs. > - datanodes can be contacted through the ipc.Client interface (once again > this class can shows up in HBase logs) and the data transfer interface > (usually shows up as the DataNode class in the HBase logs). There are on > different ports (defaults being: 50010 and 50020). > - To understand exactly what's going on, you must look that the HDFS log > files as well: HBase logs represent the client side. > - With the default setting, HDFS needs 630s to mark a datanode as dead. For > this reason, this node will still be tried by HBase or by other datanodes > when writing and reading until HDFS definitively decides it's dead. This will > add some extras lines in the logs. This monitoring is performed by the > NameNode. > - The HDFS clients (i.e. HBase using HDFS client code) don't fully rely on > the NameNode, but can mark temporally a node as dead if they had an error > when they tried to use it. > 2) Settings for retries and timeouts > 2.1) Retries > ipc.client.connect.max.retries > Default 10 > Indicates the number of retries a client will make to establish a server > connection. Not taken into account if the error is a SocketTimeout. In this > case the number of retries is 45 (fixed on branch, HADOOP-7932 or in > HADOOP-7397). For SASL, the number of retries is hard-coded to 15. Can be > increased, especially if the socket timeouts have been lowered. > ipc.client.connect.max.retries.on.timeouts > Default 45 > If you have HADOOP-7932, max number of retries on timeout. Counter is > different than ipc.client.connect.max.retries so if you mix the socket errors > you will get 55 retries with the default values. Could be lowered, once it is > available. With HADOOP-7397 ipc.client.connect.max.retries is reused so there > would be 10 tries. > dfs.client.block.write.retries > Default 3 > Number of tries for the client when writing a block. After a failure, will > connect to the namenode a get a new location, sending the list of the > datanodes already tried without success. Could be increased, especially if > the socket timeouts have been lowered. See HBASE-6490. > dfs.client.block.write.locateFollowingBlock.retries > Default 5 > Number of retries to the namenode when the client got > NotReplicatedYetException, i.e. the existing nodes of the files are not yet > replicated to dfs.
[jira] [Updated] (HBASE-11316) Expand info about compactions beyond HBASE-11120
[ https://issues.apache.org/jira/browse/HBASE-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11316: Attachment: HBASE-11316-6-rebased.patch Had to rebase. > Expand info about compactions beyond HBASE-11120 > > > Key: HBASE-11316 > URL: https://issues.apache.org/jira/browse/HBASE-11316 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-11316-1.patch, HBASE-11316-2.patch, > HBASE-11316-3.patch, HBASE-11316-4.patch, HBASE-11316-5.patch, > HBASE-11316-6-rebased.patch, HBASE-11316-6.patch, HBASE-11316.patch, > ch9_compactions.pdf > > > Round 2 - expand info about the algorithms, talk about stripe compaction, and > talk more about configuration. Hopefully find some rules of thumb. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-6626) Add a chapter on HDFS in the troubleshooting section of the HBase reference guide.
[ https://issues.apache.org/jira/browse/HBASE-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-6626: --- Attachment: (was: HBASE-6626.patch) > Add a chapter on HDFS in the troubleshooting section of the HBase reference > guide. > -- > > Key: HBASE-6626 > URL: https://issues.apache.org/jira/browse/HBASE-6626 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Misty Stanley-Jones >Priority: Blocker > Attachments: troubleshooting.txt > > > I looked mainly at the major failure case, but here is what I have: > New sub chapter in the existing chapter "Troubleshooting and Debugging > HBase": "HDFS & HBASE" > 1) HDFS & HBase > 2) Connection related settings > 2.1) Number of retries > 2.2) Timeouts > 3) Log samples > 1) HDFS & HBase > HBase uses HDFS to store its HFile, i.e. the core HBase files and the > Write-Ahead-Logs, i.e. the files that will be used to restore the data after > a crash. > In both cases, the reliability of HBase comes from the fact that HDFS writes > the data to multiple locations. To be efficient, HBase needs the data to be > available locally, hence it's highly recommended to have the HDFS datanode on > the same machines as the HBase Region Servers. > Detailed information on how HDFS works can be found at [1]. > Important features are: > - HBase is a client application of HDFS, i.e. uses the HDFS DFSClient class. > This class can appears in HBase logs with other HDFS client related logs. > - Some HDFS settings are HDFS-server-side, i.e. must be set on the HDFS > side, while some other are HDFS-client-side, i.e. must be set in HBase, while > some other must be set in both places. > - the HDFS writes are pipelined from one datanode to another. When writing, > there are communications between: > - HBase and HDFS namenode, through the HDFS client classes. > - HBase and HDFS datanodes, through the HDFS client classes. > - HDFS datanode between themselves: issues on these communications are in > HDFS logs, not HBase. HDFS writes are always local when possible. As a > consequence, there should not be much write error in HBase Region Servers: > they write to the local datanode. If this datanode can't replicate the > blocks, it will appear in its logs, not in the region servers logs. > - datanodes can be contacted through the ipc.Client interface (once again > this class can shows up in HBase logs) and the data transfer interface > (usually shows up as the DataNode class in the HBase logs). There are on > different ports (defaults being: 50010 and 50020). > - To understand exactly what's going on, you must look that the HDFS log > files as well: HBase logs represent the client side. > - With the default setting, HDFS needs 630s to mark a datanode as dead. For > this reason, this node will still be tried by HBase or by other datanodes > when writing and reading until HDFS definitively decides it's dead. This will > add some extras lines in the logs. This monitoring is performed by the > NameNode. > - The HDFS clients (i.e. HBase using HDFS client code) don't fully rely on > the NameNode, but can mark temporally a node as dead if they had an error > when they tried to use it. > 2) Settings for retries and timeouts > 2.1) Retries > ipc.client.connect.max.retries > Default 10 > Indicates the number of retries a client will make to establish a server > connection. Not taken into account if the error is a SocketTimeout. In this > case the number of retries is 45 (fixed on branch, HADOOP-7932 or in > HADOOP-7397). For SASL, the number of retries is hard-coded to 15. Can be > increased, especially if the socket timeouts have been lowered. > ipc.client.connect.max.retries.on.timeouts > Default 45 > If you have HADOOP-7932, max number of retries on timeout. Counter is > different than ipc.client.connect.max.retries so if you mix the socket errors > you will get 55 retries with the default values. Could be lowered, once it is > available. With HADOOP-7397 ipc.client.connect.max.retries is reused so there > would be 10 tries. > dfs.client.block.write.retries > Default 3 > Number of tries for the client when writing a block. After a failure, will > connect to the namenode a get a new location, sending the list of the > datanodes already tried without success. Could be increased, especially if > the socket timeouts have been lowered. See HBASE-6490. > dfs.client.block.write.locateFollowingBlock.retries > Default 5 > Number of retries to the namenode when the client got > NotReplicatedYetException, i.e. the existing nodes of the files are not yet > replicated to dfs.replication.min
[jira] [Updated] (HBASE-6626) Add a chapter on HDFS in the troubleshooting section of the HBase reference guide.
[ https://issues.apache.org/jira/browse/HBASE-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-6626: --- Attachment: HBASE-6626.patch Re-generated the patch. I can apply it to the current master so I'm not sure what is wrong. > Add a chapter on HDFS in the troubleshooting section of the HBase reference > guide. > -- > > Key: HBASE-6626 > URL: https://issues.apache.org/jira/browse/HBASE-6626 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Misty Stanley-Jones >Priority: Blocker > Attachments: HBASE-6626.patch, troubleshooting.txt > > > I looked mainly at the major failure case, but here is what I have: > New sub chapter in the existing chapter "Troubleshooting and Debugging > HBase": "HDFS & HBASE" > 1) HDFS & HBase > 2) Connection related settings > 2.1) Number of retries > 2.2) Timeouts > 3) Log samples > 1) HDFS & HBase > HBase uses HDFS to store its HFile, i.e. the core HBase files and the > Write-Ahead-Logs, i.e. the files that will be used to restore the data after > a crash. > In both cases, the reliability of HBase comes from the fact that HDFS writes > the data to multiple locations. To be efficient, HBase needs the data to be > available locally, hence it's highly recommended to have the HDFS datanode on > the same machines as the HBase Region Servers. > Detailed information on how HDFS works can be found at [1]. > Important features are: > - HBase is a client application of HDFS, i.e. uses the HDFS DFSClient class. > This class can appears in HBase logs with other HDFS client related logs. > - Some HDFS settings are HDFS-server-side, i.e. must be set on the HDFS > side, while some other are HDFS-client-side, i.e. must be set in HBase, while > some other must be set in both places. > - the HDFS writes are pipelined from one datanode to another. When writing, > there are communications between: > - HBase and HDFS namenode, through the HDFS client classes. > - HBase and HDFS datanodes, through the HDFS client classes. > - HDFS datanode between themselves: issues on these communications are in > HDFS logs, not HBase. HDFS writes are always local when possible. As a > consequence, there should not be much write error in HBase Region Servers: > they write to the local datanode. If this datanode can't replicate the > blocks, it will appear in its logs, not in the region servers logs. > - datanodes can be contacted through the ipc.Client interface (once again > this class can shows up in HBase logs) and the data transfer interface > (usually shows up as the DataNode class in the HBase logs). There are on > different ports (defaults being: 50010 and 50020). > - To understand exactly what's going on, you must look that the HDFS log > files as well: HBase logs represent the client side. > - With the default setting, HDFS needs 630s to mark a datanode as dead. For > this reason, this node will still be tried by HBase or by other datanodes > when writing and reading until HDFS definitively decides it's dead. This will > add some extras lines in the logs. This monitoring is performed by the > NameNode. > - The HDFS clients (i.e. HBase using HDFS client code) don't fully rely on > the NameNode, but can mark temporally a node as dead if they had an error > when they tried to use it. > 2) Settings for retries and timeouts > 2.1) Retries > ipc.client.connect.max.retries > Default 10 > Indicates the number of retries a client will make to establish a server > connection. Not taken into account if the error is a SocketTimeout. In this > case the number of retries is 45 (fixed on branch, HADOOP-7932 or in > HADOOP-7397). For SASL, the number of retries is hard-coded to 15. Can be > increased, especially if the socket timeouts have been lowered. > ipc.client.connect.max.retries.on.timeouts > Default 45 > If you have HADOOP-7932, max number of retries on timeout. Counter is > different than ipc.client.connect.max.retries so if you mix the socket errors > you will get 55 retries with the default values. Could be lowered, once it is > available. With HADOOP-7397 ipc.client.connect.max.retries is reused so there > would be 10 tries. > dfs.client.block.write.retries > Default 3 > Number of tries for the client when writing a block. After a failure, will > connect to the namenode a get a new location, sending the list of the > datanodes already tried without success. Could be increased, especially if > the socket timeouts have been lowered. See HBASE-6490. > dfs.client.block.write.locateFollowingBlock.retries > Default 5 > Number of retries to the namenode when the client got > NotReplicat
[jira] [Resolved] (HBASE-5426) How to Set Up a Pseudo-Distributed Mode for HBase
[ https://issues.apache.org/jira/browse/HBASE-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones resolved HBASE-5426. Resolution: Duplicate Please reopen if you feel that this information is still relevant and not covered by HBASE-11399. > How to Set Up a Pseudo-Distributed Mode for HBase > - > > Key: HBASE-5426 > URL: https://issues.apache.org/jira/browse/HBASE-5426 > Project: HBase > Issue Type: Improvement > Components: documentation >Affects Versions: 0.92.0 > Environment: RedHat 7, Ubuntu 10 >Reporter: Bing Li >Assignee: Doug Meil > Labels: documentation > Fix For: 0.92.0 > > Original Estimate: 48h > Remaining Estimate: 48h > > Hi, all, > I just made a summary about the experiences to set up a pseudo-distributed > mode HBase. > 1) RedHat 9 is not suitable for running HBase and Hadoop. I don't know the > reasons. Now Ubuntu is my choice. > 2) After the pseudo-distributed mode of HDFS is configured, it is required to > configure the hbase-env.sh and hbase-site.xml. The book, HBase the Definitive > Guide, does not mentions hbase.env.xml. > 3) It should set up JAVA_HOME, HBASE_CLASSPATH and HBASE_MANAGES_ZK. My > hbase-env.sh is as follows. > export JAVA_HOME=/opt/jdk1.6.1/ > export HBASE_CLASSPATH=/opt/hbase-0.92.0/conf > export HBASE_OPTS="-XX:+UseConcMarkSweepGC" > export HBASE_MANAGES_ZK=true > 4) When configuring hbase-site.xml, the property, hbase.cluster.distributed, > must be set also. The book, HBase the Definitive Guide, does not do that > either. My hbase-site.xml is as follows. > > > > >hbase.rootdir >hdfs://localhost:9000/hbase > > >dfs.replication >1 > > >hbase.cluster.distributed >true > > > I am a new user of HBase. Your suggestions are highly appreciated. > Best regards, > Bing -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-8473) add note to ref guide about snapshots and ec2 reverse dns requirements.
[
https://issues.apache.org/jira/browse/HBASE-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones reassigned HBASE-8473:
--
Assignee: Misty Stanley-Jones (was: Jonathan Hsieh)
> add note to ref guide about snapshots and ec2 reverse dns requirements.
> ---
>
> Key: HBASE-8473
> URL: https://issues.apache.org/jira/browse/HBASE-8473
> Project: HBase
> Issue Type: Bug
> Components: documentation, snapshots
>Affects Versions: 0.98.0, 0.94.6.1, 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
>
> From IRC from mighty Jeremy Carroll.
> {code}
> 17:10 jmhsieh: I think I found the root cuase. All my region
> servers reach the barrier, but it does not continue.
> 17:11 jmhsieh: All RS have this in their logs: 2013-05-01
> 00:04:56,356 DEBUG org.apache.hadoop.hbase.procedure.Subprocedure:
> Subprocedure 'backup1' coordinator notified of 'acquire', waiting on
> 'reached' or 'abort' from coordinator.
> 17:11 jmhsieh: Then the coordinator (Master) never sends
> anything. They just sit until the timeout.
> 17:12 jmhsieh: So basically 'reached' is never obtained.
> Then abort it set, and it fails.
> ...
> 17:24 jmhsieh: Found the bug. The hostnames dont match the
> master due to DNS resolution
> 17:25 jmhsieh: The barrier aquired is putting in the local
> hostname from the regionservers. In EC2 (Where reverse DNS does not work
> well), the master hands the internal name to the client.
> 17:25 jmhsieh:
> https://s3.amazonaws.com/uploads.hipchat.com/23947/185789/au94meik0h3y5ii/Screen%20Shot%202013-04-30%20at%2017.25.50.png
>
> 17:26 jmhsieh: So it's waiting for something like
> 'ip-10-155-208-202.ec2.internal,60020,1367366580066' zNode to show up, but
> instead 'hbasemetaclustera-d1b0a484,60020,1367366580066,' is being inserted.
> Barrier is not reached
> 17:27 jmhsieh: Reason being in our environment the master
> does not have a reverse DNS entry. So we get stuff like this on RegionServer
> startup in our logs.
> 17:27 jmhsieh: 2013-05-01 00:03:00,614 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us hostname
> to use. Was=hbasemetaclustera-d1b0a484, Now=ip-10-155-208-202.ec2.internal
> 17:54 jmhsieh: That was it. Verified. Now that Reverse DNS
> is working, snapshots are working. Now how to figure out how to get Reverse
> DNS working on Route53. I wished there was something like 'slave.host.name'
> inside of Hadoop for this. Looking at source code.
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-8473) add note to ref guide about snapshots and ec2 reverse dns requirements.
[
https://issues.apache.org/jira/browse/HBASE-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-8473:
---
Attachment: HBASE-8473.patch
Is this good enough and in the right place?
> add note to ref guide about snapshots and ec2 reverse dns requirements.
> ---
>
> Key: HBASE-8473
> URL: https://issues.apache.org/jira/browse/HBASE-8473
> Project: HBase
> Issue Type: Bug
> Components: documentation, snapshots
>Affects Versions: 0.98.0, 0.94.6.1, 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-8473.patch
>
>
> From IRC from mighty Jeremy Carroll.
> {code}
> 17:10 jmhsieh: I think I found the root cuase. All my region
> servers reach the barrier, but it does not continue.
> 17:11 jmhsieh: All RS have this in their logs: 2013-05-01
> 00:04:56,356 DEBUG org.apache.hadoop.hbase.procedure.Subprocedure:
> Subprocedure 'backup1' coordinator notified of 'acquire', waiting on
> 'reached' or 'abort' from coordinator.
> 17:11 jmhsieh: Then the coordinator (Master) never sends
> anything. They just sit until the timeout.
> 17:12 jmhsieh: So basically 'reached' is never obtained.
> Then abort it set, and it fails.
> ...
> 17:24 jmhsieh: Found the bug. The hostnames dont match the
> master due to DNS resolution
> 17:25 jmhsieh: The barrier aquired is putting in the local
> hostname from the regionservers. In EC2 (Where reverse DNS does not work
> well), the master hands the internal name to the client.
> 17:25 jmhsieh:
> https://s3.amazonaws.com/uploads.hipchat.com/23947/185789/au94meik0h3y5ii/Screen%20Shot%202013-04-30%20at%2017.25.50.png
>
> 17:26 jmhsieh: So it's waiting for something like
> 'ip-10-155-208-202.ec2.internal,60020,1367366580066' zNode to show up, but
> instead 'hbasemetaclustera-d1b0a484,60020,1367366580066,' is being inserted.
> Barrier is not reached
> 17:27 jmhsieh: Reason being in our environment the master
> does not have a reverse DNS entry. So we get stuff like this on RegionServer
> startup in our logs.
> 17:27 jmhsieh: 2013-05-01 00:03:00,614 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us hostname
> to use. Was=hbasemetaclustera-d1b0a484, Now=ip-10-155-208-202.ec2.internal
> 17:54 jmhsieh: That was it. Verified. Now that Reverse DNS
> is working, snapshots are working. Now how to figure out how to get Reverse
> DNS working on Route53. I wished there was something like 'slave.host.name'
> inside of Hadoop for this. Looking at source code.
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-8473) add note to ref guide about snapshots and ec2 reverse dns requirements.
[
https://issues.apache.org/jira/browse/HBASE-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-8473:
---
Status: Patch Available (was: Open)
> add note to ref guide about snapshots and ec2 reverse dns requirements.
> ---
>
> Key: HBASE-8473
> URL: https://issues.apache.org/jira/browse/HBASE-8473
> Project: HBase
> Issue Type: Bug
> Components: documentation, snapshots
>Affects Versions: 0.95.0, 0.94.6.1, 0.98.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
> Attachments: HBASE-8473.patch
>
>
> From IRC from mighty Jeremy Carroll.
> {code}
> 17:10 jmhsieh: I think I found the root cuase. All my region
> servers reach the barrier, but it does not continue.
> 17:11 jmhsieh: All RS have this in their logs: 2013-05-01
> 00:04:56,356 DEBUG org.apache.hadoop.hbase.procedure.Subprocedure:
> Subprocedure 'backup1' coordinator notified of 'acquire', waiting on
> 'reached' or 'abort' from coordinator.
> 17:11 jmhsieh: Then the coordinator (Master) never sends
> anything. They just sit until the timeout.
> 17:12 jmhsieh: So basically 'reached' is never obtained.
> Then abort it set, and it fails.
> ...
> 17:24 jmhsieh: Found the bug. The hostnames dont match the
> master due to DNS resolution
> 17:25 jmhsieh: The barrier aquired is putting in the local
> hostname from the regionservers. In EC2 (Where reverse DNS does not work
> well), the master hands the internal name to the client.
> 17:25 jmhsieh:
> https://s3.amazonaws.com/uploads.hipchat.com/23947/185789/au94meik0h3y5ii/Screen%20Shot%202013-04-30%20at%2017.25.50.png
>
> 17:26 jmhsieh: So it's waiting for something like
> 'ip-10-155-208-202.ec2.internal,60020,1367366580066' zNode to show up, but
> instead 'hbasemetaclustera-d1b0a484,60020,1367366580066,' is being inserted.
> Barrier is not reached
> 17:27 jmhsieh: Reason being in our environment the master
> does not have a reverse DNS entry. So we get stuff like this on RegionServer
> startup in our logs.
> 17:27 jmhsieh: 2013-05-01 00:03:00,614 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us hostname
> to use. Was=hbasemetaclustera-d1b0a484, Now=ip-10-155-208-202.ec2.internal
> 17:54 jmhsieh: That was it. Verified. Now that Reverse DNS
> is working, snapshots are working. Now how to figure out how to get Reverse
> DNS working on Route53. I wished there was something like 'slave.host.name'
> inside of Hadoop for this. Looking at source code.
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-2.patch Addressed feedback from [~j...@cloudera.com], including adding the images (permission pending). As part of adding the images, I also fixed the "mvn site" target so that it is actually possible to use images in the book again. I realize that is a little out of scope for this JIRA but I left it in the patch. I suspect there are lots of broken and thus invisible images in the book currently and will address those elsewhere. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400.patch > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058116#comment-14058116 ] Misty Stanley-Jones commented on HBASE-11400: - How do I actually get the images into the diff? Do I need to upload them here and you will put them in the images directory on your end? > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400.patch > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11502) Track down broken images in Ref Guide
Misty Stanley-Jones created HBASE-11502: --- Summary: Track down broken images in Ref Guide Key: HBASE-11502 URL: https://issues.apache.org/jira/browse/HBASE-11502 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones I realized that image support was broken in the Ref Guide. I fixed it by making some changes to the pom.xml (in HBASE-11400). I need to track down what images are broken besides the new ones I added, find out where they are, and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058223#comment-14058223 ] Misty Stanley-Jones commented on HBASE-11502: - Mistys-MacBook-Pro:docbkx misty$ grep "fileref" * book.xml: fileref="hbase_logo.png" /> book.xml:fileref="timeline_consistency.png" /> book.xml: book.xml: book.xml.orig: fileref="hbase_logo.png" /> book.xml.orig: book.xml.orig: book.xml.orig: > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058224#comment-14058224 ] Misty Stanley-Jones commented on HBASE-11502: - The timeline_consistency.png file is missing. I'll see if I can find it elsewhere and put it back. The others are just in the site/ directory and need to be copied to the docbkx/ directory. It's best not to try to single-source images between the website and the book for simplicity. > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058234#comment-14058234 ] Misty Stanley-Jones commented on HBASE-11502: - The timeline_consistency.png should actually be hfile.png but was changed at some point (I verified at commit 8943f02ff8bc57b1580364f01e3559c793e9cad8 which was in Sep 2013). Going to do a bit of checking and see when it changed but will probalby just change it back. > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058253#comment-14058253 ] Misty Stanley-Jones commented on HBASE-11502: - Change was introduced in commit e50811a7ab21069ad941eeebd81c2d3d9ee98c00 Author: Enis Soztutar Date: Thu May 15 23:38:45 2014 + HBASE-10513 Provide user documentation for region replicas git-svn-id: https://svn.apache.org/repos/asf/hbase/branches/hbase-10070@1595077 13f79535-47bb-0310-9956-ffa450edef68 I will explore this more. > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058256#comment-14058256 ] Misty Stanley-Jones commented on HBASE-11502: - [~enis] did you make that change on purpose? Did you want to add a new image instead of the hfile.png? > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058257#comment-14058257 ] Misty Stanley-Jones commented on HBASE-11502: - I got it, it's a typo. The file is called ./src/main/site/resources/images/timeline_consitency.png. I will rename it. > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-3.patch Added a section to talk about implications of https://issues.apache.org/jira/browse/HBASE-6370. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400.patch > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060239#comment-14060239 ] Misty Stanley-Jones commented on HBASE-11502: - Yes it might be cleaner if you do it in the relevant JIRA than here. > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: data_block_diff_encoding.png data_block_no_encoding.png data_block_prefix_encoding.png HBASE-11400-4.patch Got permission from Matteo to use his images, and also got some clarification on why an image is not needed for Fast Diff and is difficult to do for Prefix Tree. The attached images belong in the src/main/docbkx/images/ directory if that is not clear from the diff itself. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch, > data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: (was: HBASE-11400-4.patch) > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400.patch, data_block_diff_encoding.png, > data_block_no_encoding.png, data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-4.patch Regenerated version 4 of the patch because I had mistakenly removed the modifications to pom.xml for image support. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch, > data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060299#comment-14060299 ] Misty Stanley-Jones commented on HBASE-11400: - Hmm. How have I done that? Let me explore and see. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400.patch, > data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11508) Document changes to IPC config parameters from HBASE-11492
Misty Stanley-Jones created HBASE-11508: --- Summary: Document changes to IPC config parameters from HBASE-11492 Key: HBASE-11508 URL: https://issues.apache.org/jira/browse/HBASE-11508 Project: HBase Issue Type: Sub-task Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-5.patch I put back the LZ4 stuff. [~stack], the images should work fine in HTML and HTML single. Please test my changes to pom.xml. I think they might break other images. If so, the images need to be copied from site/images/ to docbkx/images/. I know there is a maintenance cost but it's better for the long term and makes it easier to add more images in future. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400-5.patch, > HBASE-11400.patch, data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-8473) add note to ref guide about snapshots and ec2 reverse dns requirements.
[
https://issues.apache.org/jira/browse/HBASE-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060333#comment-14060333
]
Misty Stanley-Jones commented on HBASE-8473:
Thanks, I saw the other section but didn't de-duplicate for now because it
needs more re-working. At some point I will stop re-working things and start
working on the new guide. Almost to that point.
> add note to ref guide about snapshots and ec2 reverse dns requirements.
> ---
>
> Key: HBASE-8473
> URL: https://issues.apache.org/jira/browse/HBASE-8473
> Project: HBase
> Issue Type: Bug
> Components: documentation, snapshots
>Affects Versions: 0.98.0, 0.94.6.1, 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-8473.patch
>
>
> From IRC from mighty Jeremy Carroll.
> {code}
> 17:10 jmhsieh: I think I found the root cuase. All my region
> servers reach the barrier, but it does not continue.
> 17:11 jmhsieh: All RS have this in their logs: 2013-05-01
> 00:04:56,356 DEBUG org.apache.hadoop.hbase.procedure.Subprocedure:
> Subprocedure 'backup1' coordinator notified of 'acquire', waiting on
> 'reached' or 'abort' from coordinator.
> 17:11 jmhsieh: Then the coordinator (Master) never sends
> anything. They just sit until the timeout.
> 17:12 jmhsieh: So basically 'reached' is never obtained.
> Then abort it set, and it fails.
> ...
> 17:24 jmhsieh: Found the bug. The hostnames dont match the
> master due to DNS resolution
> 17:25 jmhsieh: The barrier aquired is putting in the local
> hostname from the regionservers. In EC2 (Where reverse DNS does not work
> well), the master hands the internal name to the client.
> 17:25 jmhsieh:
> https://s3.amazonaws.com/uploads.hipchat.com/23947/185789/au94meik0h3y5ii/Screen%20Shot%202013-04-30%20at%2017.25.50.png
>
> 17:26 jmhsieh: So it's waiting for something like
> 'ip-10-155-208-202.ec2.internal,60020,1367366580066' zNode to show up, but
> instead 'hbasemetaclustera-d1b0a484,60020,1367366580066,' is being inserted.
> Barrier is not reached
> 17:27 jmhsieh: Reason being in our environment the master
> does not have a reverse DNS entry. So we get stuff like this on RegionServer
> startup in our logs.
> 17:27 jmhsieh: 2013-05-01 00:03:00,614 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us hostname
> to use. Was=hbasemetaclustera-d1b0a484, Now=ip-10-155-208-202.ec2.internal
> 17:54 jmhsieh: That was it. Verified. Now that Reverse DNS
> is working, snapshots are working. Now how to figure out how to get Reverse
> DNS working on Route53. I wished there was something like 'slave.host.name'
> inside of Hadoop for this. Looking at source code.
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14060335#comment-14060335 ] Misty Stanley-Jones commented on HBASE-11400: - @stack I think that HBASE-11459 might have collided with my work and I rewrote a lot of that stuff so it's a little hard to parse whether everything from it is there. Can you look it over? I can do a more detailed comparison and make sure I got everything tomorrow if not. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400-5.patch, > HBASE-11400.patch, data_block_diff_encoding.png, data_block_no_encoding.png, > data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-11492:
Release Note:
If the Hadoop configuration is read after the HBase configuration, Hadoop's
settings can override HBase's settings if the names of the settings are the
same. To avoid the risk of override, HBase has renamed the following settings
(by prepending 'hbase.') so that you can set them independent of your setting
for Hadoop. If you do not use the HBase-specific variants, the Hadoop settings
will be used.
Old Name New Name
==
ipc.server.listen.queue.size hbase.ipc.server.listen.queue.size
ipc.server.max.callqueue.sizehbase.ipc.server.max.callqueue.size
ipc.server.read.threadpool.size hbase.ipc.server.read.threadpool.size
ipc.server.tcpkeepalive hbase.ipc.server.tcpkeepalive
ipc.server.tcpnodelayhbase.ipc.server.tcpnodelay
ipc.client.call.purge.timeouthbase.ipc.client.call.purge.timeout
ipc.client.connection.maxidletimehbase.ipc.client.connection.maxidletime
ipc.client.idlethreshold hbase.ipc.client.idlethreshold
ipc.client.kill.max hbase.ipc.client.kill.max
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-11492:
Release Note:
If the Hadoop configuration is read after the HBase configuration, Hadoop's
settings can override HBase's settings if the names of the settings are the
same. To avoid the risk of override, HBase has renamed the following settings
(by prepending 'hbase.') so that you can set them independent of your setting
for Hadoop. If you do not use the HBase-specific variants, the Hadoop settings
will be used.
Old Name --> New Name
==
ipc.server.listen.queue.size --> hbase.ipc.server.listen.queue.size
ipc.server.max.callqueue.size --> hbase.ipc.server.max.callqueue.size
ipc.server.read.threadpool.size --> hbase.ipc.server.read.threadpool.size
ipc.server.tcpkeepalive --> hbase.ipc.server.tcpkeepalive
ipc.server.tcpnodelay --> hbase.ipc.server.tcpnodelay
ipc.client.call.purge.timeout --> hbase.ipc.client.call.purge.timeout
ipc.client.connection.maxidletime -->
hbase.ipc.client.connection.maxidletime
ipc.client.idlethreshold --> hbase.ipc.client.idlethreshold
ipc.client.kill.max --> hbase.ipc.client.kill.max
was:
If the Hadoop configuration is read after the HBase configuration, Hadoop's
settings can override HBase's settings if the names of the settings are the
same. To avoid the risk of override, HBase has renamed the following settings
(by prepending 'hbase.') so that you can set them independent of your setting
for Hadoop. If you do not use the HBase-specific variants, the Hadoop settings
will be used.
Old Name --> New Name
ipc.server.listen.queue.size --> hbase.ipc.server.listen.queue.size
ipc.server.max.callqueue.size --> hbase.ipc.server.max.callqueue.size
ipc.server.read.threadpool.size --> hbase.ipc.server.read.threadpool.size
ipc.server.tcpkeepalive --> hbase.ipc.server.tcpkeepalive
ipc.server.tcpnodelay --> hbase.ipc.server.tcpnodelay
ipc.client.call.purge.timeout --> hbase.ipc.client.call.purge.timeout
ipc.client.connection.maxidletime -->
hbase.ipc.client.connection.maxidletime
ipc.client.idlethreshold --> hbase.ipc.client.idlethreshold
ipc.client.kill.max --> hbase.ipc.client.kill.max
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11492) Hadoop configuration overrides some ipc parameters including tcpNoDelay
[
https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-11492:
Release Note:
If the Hadoop configuration is read after the HBase configuration, Hadoop's
settings can override HBase's settings if the names of the settings are the
same. To avoid the risk of override, HBase has renamed the following settings
(by prepending 'hbase.') so that you can set them independent of your setting
for Hadoop. If you do not use the HBase-specific variants, the Hadoop settings
will be used.
Old Name --> New Name
ipc.server.listen.queue.size --> hbase.ipc.server.listen.queue.size
ipc.server.max.callqueue.size --> hbase.ipc.server.max.callqueue.size
ipc.server.read.threadpool.size --> hbase.ipc.server.read.threadpool.size
ipc.server.tcpkeepalive --> hbase.ipc.server.tcpkeepalive
ipc.server.tcpnodelay --> hbase.ipc.server.tcpnodelay
ipc.client.call.purge.timeout --> hbase.ipc.client.call.purge.timeout
ipc.client.connection.maxidletime -->
hbase.ipc.client.connection.maxidletime
ipc.client.idlethreshold --> hbase.ipc.client.idlethreshold
ipc.client.kill.max --> hbase.ipc.client.kill.max
was:
If the Hadoop configuration is read after the HBase configuration, Hadoop's
settings can override HBase's settings if the names of the settings are the
same. To avoid the risk of override, HBase has renamed the following settings
(by prepending 'hbase.') so that you can set them independent of your setting
for Hadoop. If you do not use the HBase-specific variants, the Hadoop settings
will be used.
Old Name New Name
==
ipc.server.listen.queue.size hbase.ipc.server.listen.queue.size
ipc.server.max.callqueue.sizehbase.ipc.server.max.callqueue.size
ipc.server.read.threadpool.size hbase.ipc.server.read.threadpool.size
ipc.server.tcpkeepalive hbase.ipc.server.tcpkeepalive
ipc.server.tcpnodelayhbase.ipc.server.tcpnodelay
ipc.client.call.purge.timeouthbase.ipc.client.call.purge.timeout
ipc.client.connection.maxidletimehbase.ipc.client.connection.maxidletime
ipc.client.idlethreshold hbase.ipc.client.idlethreshold
ipc.client.kill.max hbase.ipc.client.kill.max
> Hadoop configuration overrides some ipc parameters including tcpNoDelay
> ---
>
> Key: HBASE-11492
> URL: https://issues.apache.org/jira/browse/HBASE-11492
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Critical
> Fix For: 0.99.0, 0.98.4, 2.0.0
>
> Attachments: 11492.v1.patch, 11492.v1.withp1.patch,
> 11492.v2-0.98.patch, 11492.v2.patch, 11492.v2.patch
>
>
> There is an option to set tcpNoDelay, defaulted to true, but the socket
> channel is actually not changed. As a consequence, the server works with
> nagle enabled. This leads to very degraded behavior when a single connection
> is shared between threads. We enter into conflicts with nagle and tcp delayed
> ack.
> Here is an example of performance with the PE tool plus HBASE-11491:
> {noformat}
> oneCon #client sleep exeTime (seconds)
> avg latency, sleep excluded (microseconds)
> true 1 031
> 310
> false 1 031
> 310
> true 2 050
> 500
> false 2 0 31
> 310
> true 25 488 (including 200s sleeping)
> 2880
> false 2 5 246 (including 200s sleeping)
> 460
> {noformat}
> The latency is multiple by 5 (2880 vs 460) when the connection is shared.
> This is the delayed ack kicking in. This can be fixed by really using tcp no
> delay.
> Any application sharing the tcp connection between threads has the issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11508) Document changes to IPC config parameters from HBASE-11492
[ https://issues.apache.org/jira/browse/HBASE-11508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11508: Attachment: HBASE-11492.patch Added a section to Troubleshooting and looked through the rest of the docs to find instances of the old name and changed to the new name. > Document changes to IPC config parameters from HBASE-11492 > -- > > Key: HBASE-11508 > URL: https://issues.apache.org/jira/browse/HBASE-11508 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 2.0.0 > > Attachments: HBASE-11492.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11508) Document changes to IPC config parameters from HBASE-11492
[ https://issues.apache.org/jira/browse/HBASE-11508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11508: Status: Patch Available (was: Open) > Document changes to IPC config parameters from HBASE-11492 > -- > > Key: HBASE-11508 > URL: https://issues.apache.org/jira/browse/HBASE-11508 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 2.0.0 > > Attachments: HBASE-11492.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11515) Rewrite HBase Ref Guide for HBase 2.0
Misty Stanley-Jones created HBASE-11515: --- Summary: Rewrite HBase Ref Guide for HBase 2.0 Key: HBASE-11515 URL: https://issues.apache.org/jira/browse/HBASE-11515 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Fix For: 2.0.0 Rewrite the HBase Ref Guide, using the existing guide as reference material. To be commenced after 1.0 is out the door. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11321) Restructure Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11321: Issue Type: Sub-task (was: Bug) Parent: HBASE-11515 > Restructure Ref Guide > - > > Key: HBASE-11321 > URL: https://issues.apache.org/jira/browse/HBASE-11321 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBase Reference Guide 2.0 -- Proposed Structure, HBase > Reference Guide 2.0 -- Proposed Structure, HBase Reference Guide 2.0 -- > Proposed Structure > > > Reorganize Ref Guide to make the important things more prominent and easier > to find. This will be a big multi-step task. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11321) Restructure Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones resolved HBASE-11321. - Resolution: Fixed This will be the rough structure of the new guide for 2.0. If anyone else has feedback, keep leaving it, but it seems not much more is forthcoming and it will be easier once a basic structure is in place. > Restructure Ref Guide > - > > Key: HBASE-11321 > URL: https://issues.apache.org/jira/browse/HBASE-11321 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBase Reference Guide 2.0 -- Proposed Structure, HBase > Reference Guide 2.0 -- Proposed Structure, HBase Reference Guide 2.0 -- > Proposed Structure > > > Reorganize Ref Guide to make the important things more prominent and easier > to find. This will be a big multi-step task. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10398) HBase book updates for Replication after HBASE-10322
[ https://issues.apache.org/jira/browse/HBASE-10398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061611#comment-14061611 ] Misty Stanley-Jones commented on HBASE-10398: - [~anoop.hbase] Do you mind providing some source material for these docs? I tried parsing HBASE-10322 but I need some help understanding it enough to document. > HBase book updates for Replication after HBASE-10322 > > > Key: HBASE-10398 > URL: https://issues.apache.org/jira/browse/HBASE-10398 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062767#comment-14062767 ] Misty Stanley-Jones commented on HBASE-11502: - It looks fine now, thank you! I am waiting on a commit from HBASE-11400 to fix some problems with building the book itself, and I have just discovered another problem with CSS that I will fix there as well. > Track down broken images in Ref Guide > - > > Key: HBASE-11502 > URL: https://issues.apache.org/jira/browse/HBASE-11502 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > > I realized that image support was broken in the Ref Guide. I fixed it by > making some changes to the pom.xml (in HBASE-11400). I need to track down > what images are broken besides the new ones I added, find out where they are, > and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-6.patch I removed the changes from pom.xml and realized I had put the images in the wrong place in the build, so fixed those things in this patch. I will open a separate JIRA to handle the pom.xml changes as there is another change needed for CSS. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400-5.patch, > HBASE-11400-6.patch, HBASE-11400.patch, data_block_diff_encoding.png, > data_block_no_encoding.png, data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
Misty Stanley-Jones created HBASE-11521: --- Summary: Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly Key: HBASE-11521 URL: https://issues.apache.org/jira/browse/HBASE-11521 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Priority: Critical Currently, images are broken in the html-single version of the Ref Guide and a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
[ https://issues.apache.org/jira/browse/HBASE-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11521: Attachment: HBASE-11521.patch > Modify pom.xml to copy the images/ and css/ directories to the right location > for the Ref Guide to see them correctly > - > > Key: HBASE-11521 > URL: https://issues.apache.org/jira/browse/HBASE-11521 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Critical > > Currently, images are broken in the html-single version of the Ref Guide and > a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
[ https://issues.apache.org/jira/browse/HBASE-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11521: Attachment: (was: HBASE-11521.patch) > Modify pom.xml to copy the images/ and css/ directories to the right location > for the Ref Guide to see them correctly > - > > Key: HBASE-11521 > URL: https://issues.apache.org/jira/browse/HBASE-11521 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Critical > > Currently, images are broken in the html-single version of the Ref Guide and > a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
[ https://issues.apache.org/jira/browse/HBASE-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11521: Attachment: HBASE-11521.patch > Modify pom.xml to copy the images/ and css/ directories to the right location > for the Ref Guide to see them correctly > - > > Key: HBASE-11521 > URL: https://issues.apache.org/jira/browse/HBASE-11521 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Critical > Attachments: HBASE-11521.patch > > > Currently, images are broken in the html-single version of the Ref Guide and > a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
[ https://issues.apache.org/jira/browse/HBASE-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11521: Status: Patch Available (was: Open) > Modify pom.xml to copy the images/ and css/ directories to the right location > for the Ref Guide to see them correctly > - > > Key: HBASE-11521 > URL: https://issues.apache.org/jira/browse/HBASE-11521 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Critical > Attachments: HBASE-11521.patch > > > Currently, images are broken in the html-single version of the Ref Guide and > a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062984#comment-14062984 ] Misty Stanley-Jones commented on HBASE-11400: - POM changes moved to HBASE-11521. > Edit, consolidate, and update Compression and data encoding docs > > > Key: HBASE-11400 > URL: https://issues.apache.org/jira/browse/HBASE-11400 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones >Priority: Minor > Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, > HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400-5.patch, > HBASE-11400-6.patch, HBASE-11400.patch, data_block_diff_encoding.png, > data_block_no_encoding.png, data_block_prefix_encoding.png > > > Current docs are here: http://hbase.apache.org/book.html#compression.test > It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
