Re: [VOTE] Apache Accumulo 1.6.1 RC1
I haven't had an opportunity to try it again since my +1, but prior to that it has been consistently failing. - I tried extending the timeout on the test, but it would still time out. - I see the behavior on Mac OS X and under CentOS. (I wonder if it's a JVM thing?) On Wed, Sep 24, 2014 at 9:06 PM, Corey Nolet cjno...@gmail.com wrote: Vote passes with 4 +1's and no -1's. Bill, were you able to get the IT to run yet? I'm still having timeouts on my end as well. On Wed, Sep 24, 2014 at 1:41 PM, Josh Elser josh.el...@gmail.com wrote: The crux of it is that both of the errors in the CRC where single bit variants. y instead of 9 and p instead of 0 Both of these cases are a '1' in the most significant bit of the byte instead of a '0'. We recognized these because y and p are outside of the hex range. Fixing both of these fixes the CRC error (manually verified). That's all we know right now. I'm currently running memtest86. I do not have ECC ram, so it *is* theoretically possible that was the cause. After running memtest for a day or so (or until I need my desktop functional again), I'll go back and see if I can reproduce this again. Mike Drob wrote: Any chance the IRC chats can make it only the ML for posterity? Mike On Wed, Sep 24, 2014 at 12:04 PM, Keith Turnerke...@deenlo.com wrote: On Wed, Sep 24, 2014 at 12:44 PM, Russ Weeksrwe...@newbrightidea.com wrote: Interesting that y (0x79) and 9 (0x39) are one bit away from each other. I blame cosmic rays! It is interesting, and thats only half of the story. Its been interesting chatting w/ Josh about this on irc and hearing about his findings. On Wed, Sep 24, 2014 at 9:05 AM, Josh Elserjosh.el...@gmail.com wrote: The offending keys are: 389a85668b6ebf8e 2ff6:4a78 [] 1411499115242 3a10885b-d481-4d00-be00-0477e231ey65:8576b169: 0cd98965c9ccc1d0:ba15529e The careful eye will notice that the UUID in the first component of the value has a different suffix than the next corrupt key/value (ends with ey65 instead of e965). Fixing this in the Value and re-running the CRC makes it pass. and 7e56b58a0c7df128 5fa0:6249 [] 1411499311578 3a10885b-d481-4d00-be00-0477e231e965:p000872d60eb: 499fa72752d82a7c:5c5f19e8 -- // Bill Havanki // Solutions Architect, Cloudera Govt Solutions // 443.686.9283
Re: [VOTE] Apache Accumulo 1.6.1 RC1
I'm seeing the behavior under Max OS X and Fedora 19 and they have been consistently failing for me. I'm thinking ACCUMULO-3073. Since others are able to get it to pass, I did not think it should fail the vote solely on that but I do think it needs attention, quickly. On Thu, Sep 25, 2014 at 10:43 AM, Bill Havanki bhava...@clouderagovt.com wrote: I haven't had an opportunity to try it again since my +1, but prior to that it has been consistently failing. - I tried extending the timeout on the test, but it would still time out. - I see the behavior on Mac OS X and under CentOS. (I wonder if it's a JVM thing?) On Wed, Sep 24, 2014 at 9:06 PM, Corey Nolet cjno...@gmail.com wrote: Vote passes with 4 +1's and no -1's. Bill, were you able to get the IT to run yet? I'm still having timeouts on my end as well. On Wed, Sep 24, 2014 at 1:41 PM, Josh Elser josh.el...@gmail.com wrote: The crux of it is that both of the errors in the CRC where single bit variants. y instead of 9 and p instead of 0 Both of these cases are a '1' in the most significant bit of the byte instead of a '0'. We recognized these because y and p are outside of the hex range. Fixing both of these fixes the CRC error (manually verified). That's all we know right now. I'm currently running memtest86. I do not have ECC ram, so it *is* theoretically possible that was the cause. After running memtest for a day or so (or until I need my desktop functional again), I'll go back and see if I can reproduce this again. Mike Drob wrote: Any chance the IRC chats can make it only the ML for posterity? Mike On Wed, Sep 24, 2014 at 12:04 PM, Keith Turnerke...@deenlo.com wrote: On Wed, Sep 24, 2014 at 12:44 PM, Russ Weeks rwe...@newbrightidea.com wrote: Interesting that y (0x79) and 9 (0x39) are one bit away from each other. I blame cosmic rays! It is interesting, and thats only half of the story. Its been interesting chatting w/ Josh about this on irc and hearing about his findings. On Wed, Sep 24, 2014 at 9:05 AM, Josh Elserjosh.el...@gmail.com wrote: The offending keys are: 389a85668b6ebf8e 2ff6:4a78 [] 1411499115242 3a10885b-d481-4d00-be00-0477e231ey65:8576b169: 0cd98965c9ccc1d0:ba15529e The careful eye will notice that the UUID in the first component of the value has a different suffix than the next corrupt key/value (ends with ey65 instead of e965). Fixing this in the Value and re-running the CRC makes it pass. and 7e56b58a0c7df128 5fa0:6249 [] 1411499311578 3a10885b-d481-4d00-be00-0477e231e965:p000872d60eb: 499fa72752d82a7c:5c5f19e8 -- // Bill Havanki // Solutions Architect, Cloudera Govt Solutions // 443.686.9283
Re: [VOTE] Apache Accumulo 1.6.1 RC1
Please make a ticket for it and supply the MAC directories for the test and the failsafe output. It doesn't fail for me. It's possible that there is some edge case that you and Bill are hitting that I'm not. Corey Nolet wrote: I'm seeing the behavior under Max OS X and Fedora 19 and they have been consistently failing for me. I'm thinking ACCUMULO-3073. Since others are able to get it to pass, I did not think it should fail the vote solely on that but I do think it needs attention, quickly. On Thu, Sep 25, 2014 at 10:43 AM, Bill Havankibhava...@clouderagovt.com wrote: I haven't had an opportunity to try it again since my +1, but prior to that it has been consistently failing. - I tried extending the timeout on the test, but it would still time out. - I see the behavior on Mac OS X and under CentOS. (I wonder if it's a JVM thing?) On Wed, Sep 24, 2014 at 9:06 PM, Corey Noletcjno...@gmail.com wrote: Vote passes with 4 +1's and no -1's. Bill, were you able to get the IT to run yet? I'm still having timeouts on my end as well. On Wed, Sep 24, 2014 at 1:41 PM, Josh Elserjosh.el...@gmail.com wrote: The crux of it is that both of the errors in the CRC where single bit variants. y instead of 9 and p instead of 0 Both of these cases are a '1' in the most significant bit of the byte instead of a '0'. We recognized these because y and p are outside of the hex range. Fixing both of these fixes the CRC error (manually verified). That's all we know right now. I'm currently running memtest86. I do not have ECC ram, so it *is* theoretically possible that was the cause. After running memtest for a day or so (or until I need my desktop functional again), I'll go back and see if I can reproduce this again. Mike Drob wrote: Any chance the IRC chats can make it only the ML for posterity? Mike On Wed, Sep 24, 2014 at 12:04 PM, Keith Turnerke...@deenlo.com wrote: On Wed, Sep 24, 2014 at 12:44 PM, Russ Weeks rwe...@newbrightidea.com wrote: Interesting that y (0x79) and 9 (0x39) are one bit away from each other. I blame cosmic rays! It is interesting, and thats only half of the story. Its been interesting chatting w/ Josh about this on irc and hearing about his findings. On Wed, Sep 24, 2014 at 9:05 AM, Josh Elserjosh.el...@gmail.com wrote: The offending keys are: 389a85668b6ebf8e 2ff6:4a78 [] 1411499115242 3a10885b-d481-4d00-be00-0477e231ey65:8576b169: 0cd98965c9ccc1d0:ba15529e The careful eye will notice that the UUID in the first component of the value has a different suffix than the next corrupt key/value (ends with ey65 instead of e965). Fixing this in the Value and re-running the CRC makes it pass. and 7e56b58a0c7df128 5fa0:6249 [] 1411499311578 3a10885b-d481-4d00-be00-0477e231e965:p000872d60eb: 499fa72752d82a7c:5c5f19e8 -- // Bill Havanki // Solutions Architect, Cloudera Govt Solutions // 443.686.9283
Re: [VOTE] Apache Accumulo 1.6.1 RC1
I'm a little concerned we had two +1's that mention failures. The one time when we're supposed to have a clean run through, we have 50% of the participators noticing failure. It doesn't instill much confidence in me. On Thu, Sep 25, 2014 at 2:18 PM, Josh Elser josh.el...@gmail.com wrote: Please make a ticket for it and supply the MAC directories for the test and the failsafe output. It doesn't fail for me. It's possible that there is some edge case that you and Bill are hitting that I'm not. Corey Nolet wrote: I'm seeing the behavior under Max OS X and Fedora 19 and they have been consistently failing for me. I'm thinking ACCUMULO-3073. Since others are able to get it to pass, I did not think it should fail the vote solely on that but I do think it needs attention, quickly. On Thu, Sep 25, 2014 at 10:43 AM, Bill Havankibhava...@clouderagovt.com wrote: I haven't had an opportunity to try it again since my +1, but prior to that it has been consistently failing. - I tried extending the timeout on the test, but it would still time out. - I see the behavior on Mac OS X and under CentOS. (I wonder if it's a JVM thing?) On Wed, Sep 24, 2014 at 9:06 PM, Corey Noletcjno...@gmail.com wrote: Vote passes with 4 +1's and no -1's. Bill, were you able to get the IT to run yet? I'm still having timeouts on my end as well. On Wed, Sep 24, 2014 at 1:41 PM, Josh Elserjosh.el...@gmail.com wrote: The crux of it is that both of the errors in the CRC where single bit variants. y instead of 9 and p instead of 0 Both of these cases are a '1' in the most significant bit of the byte instead of a '0'. We recognized these because y and p are outside of the hex range. Fixing both of these fixes the CRC error (manually verified). That's all we know right now. I'm currently running memtest86. I do not have ECC ram, so it *is* theoretically possible that was the cause. After running memtest for a day or so (or until I need my desktop functional again), I'll go back and see if I can reproduce this again. Mike Drob wrote: Any chance the IRC chats can make it only the ML for posterity? Mike On Wed, Sep 24, 2014 at 12:04 PM, Keith Turnerke...@deenlo.com wrote: On Wed, Sep 24, 2014 at 12:44 PM, Russ Weeks rwe...@newbrightidea.com wrote: Interesting that y (0x79) and 9 (0x39) are one bit away from each other. I blame cosmic rays! It is interesting, and thats only half of the story. Its been interesting chatting w/ Josh about this on irc and hearing about his findings. On Wed, Sep 24, 2014 at 9:05 AM, Josh Elserjosh.el...@gmail.com wrote: The offending keys are: 389a85668b6ebf8e 2ff6:4a78 [] 1411499115242 3a10885b-d481-4d00-be00-0477e231ey65:8576b169: 0cd98965c9ccc1d0:ba15529e The careful eye will notice that the UUID in the first component of the value has a different suffix than the next corrupt key/value (ends with ey65 instead of e965). Fixing this in the Value and re-running the CRC makes it pass. and 7e56b58a0c7df128 5fa0:6249 [] 1411499311578 3a10885b-d481-4d00-be00-0477e231e965:p000872d60eb: 499fa72752d82a7c:5c5f19e8 -- // Bill Havanki // Solutions Architect, Cloudera Govt Solutions // 443.686.9283
Re: [VOTE] Apache Accumulo 1.6.1 RC1
I ran 24 hr of random walk against 1.6.1. I saw ACCUMULO-3169, ACCUMULO-3170, and ACCUMULO-3171. I feel like these are not new in 1.6.1, but have not investigated in depth yet. On Fri, Sep 19, 2014 at 10:49 PM, Corey Nolet cjno...@gmail.com wrote: Devs, Please consider the following candidate for Apache Accumulo 1.6.1 Branch: 1.6.1-rc1 SHA1: 88c5473b3b49d797d3dabebd12fe517e9b248ba2 Staging Repository: * https://repository.apache.org/content/repositories/orgapacheaccumulo-1017/ https://repository.apache.org/content/repositories/orgapacheaccumulo-1017/ * Source tarball: * http://repository.apache.org/content/repositories/orgapacheaccumulo-1017/org/apache/accumulo/accumulo/1.6.1/accumulo-1.6.1-src.tar.gz http://repository.apache.org/content/repositories/orgapacheaccumulo-1017/org/apache/accumulo/accumulo/1.6.1/accumulo-1.6.1-src.tar.gz * Binary tarball: * http://repository.apache.org/content/repositories/orgapacheaccumulo-1017/org/apache/accumulo/accumulo/1.6.1/accumulo-1.6.1-bin.tar.gz http://repository.apache.org/content/repositories/orgapacheaccumulo-1017/org/apache/accumulo/accumulo/1.6.1/accumulo-1.6.1-bin.tar.gz * (Append .sha1, .md5 or .asc to download the signature/hash for a given artifact.) Signing keys available at: https://www.apache.org/dist/accumulo/KEYS Over 1.6.1, we have 188 issues resolved * https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob;f=CHANGES;h=91b9d31e3b9dc53f1a576cc49bbc061919eb0070;hb=1.6.1-rc1 https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=blob;f=CHANGES;h=91b9d31e3b9dc53f1a576cc49bbc061919eb0070;hb=1.6.1-rc1 * Testing: All unit and functional tests are passing. Vote will be open until Thursday, September 25th 12:00AM UTC (9/24 8:00PM ET, 9/24 5:00PM PT)
Re: [accumulo] your /dist/ artifacts - 1 BAD signature
[note: thread moved to dev@] Okay, I just confirmed that the current files in dist are the same ones in Maven Central are the same ones that we voted on. So, that issue is resolved. I double checked and saw that the gpg-signed tag hasn't been created for 1.6.1 (git tag -s 1.6.1 origin/1.6.1-rc1). I guess technically anybody could do this, and merge it (along with the version bump to 1.6.2-SNAPSHOT commit) to 1.6.2-SNAPSHOT branch (and forward, with -sours), if Corey doesn't have time/gets busy. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 2:21 PM, Corey Nolet cjno...@gmail.com wrote: There's still a few things I need to do before announcing the release to the user list. Merging the rc into the next version branch was one of them and creating the official release tag was another. I'll do these tonight as well as writing up the release notes for the site. On Thu, Sep 25, 2014 at 1:59 PM, Christopher ctubb...@apache.org wrote: Also, we can move this list to dev@. There's no reason for it to be private@ . -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 1:59 PM, Christopher ctubb...@apache.org wrote: There's one more problem that Keith and I found... it doesn't look like the rc1 branch got merged to 1.6.2-SNAPSHOT. I don't know if some other branch got accidentally merged instead. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 1:40 PM, Josh Elser josh.el...@gmail.com wrote: Things look good to me now. I checked the artifacts on dist/ against what I have from evaluating the RC and they appear to match. Anything else we need to do here? Christopher wrote: I was able to confirm the signature is bad. When I checked the RC, the signature was good, so I'm guessing the wrong one just got uploaded. I don't have a copy of the RC that I had previously downloaded, but I was able to grab a copy of what was deployed to Maven central and fix the dist sigs/checksums from that. Now, it's possible that the wrong artifacts were uploaded to Maven central (perhaps the wrong staging repo was promoted?) I can't know that for sure, until I can get to work and check my last download from the RC vote and compare with what's in Maven central now. If that is the case, then we need to determine precisely what is different from this upload and what was voted on and see if we need to immediately re-release as 1.6.2 to fix the problems. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 3:12 AM, Henk Penninghe...@apache.org wrote: Hi PMC accumulo, I watch 'www.apache.org/dist/', and I noticed that : -- you have 1 BAD pgp signature accumulo/1.6.1/accumulo-1.6.1-src.tar.gz.asc Please fix this problem soon ; for details, see http://people.apache.org/~henkp/checker/sig.html#project-accumulo http://people.apache.org/~henkp/checker/md5.html For information on how to fix problems, see the faq : http://people.apache.org/~henkp/checker/faq.html Thanks a lot, regards, Henk Penning -- apache.org infrastructure PS. The contents of this message is generated, but the mail itself is sent by hand. PS. Please cc me on all relevant emails. - _ Henk P. Penning, ICT-beta R Uithof WISK-412 _/ _ Faculty of Science, Utrecht University T +31 30 253 4106 / _/ Budapestlaan 6, 3584CD Utrecht, NL F +31 30 253 4553 _/ _/ http://people.cs.uu.nl/henkp/ M penn...@uu.nl _/
Re: [accumulo] your /dist/ artifacts - 1 BAD signature
I see what happened. I was expecting the mvn:release plugin to push the prepare for next development iteration which it did not. I just pushed it up and created the tag. I'll work on the release notes in a bit. On Thu, Sep 25, 2014 at 3:33 PM, Christopher ctubb...@apache.org wrote: [note: thread moved to dev@] Okay, I just confirmed that the current files in dist are the same ones in Maven Central are the same ones that we voted on. So, that issue is resolved. I double checked and saw that the gpg-signed tag hasn't been created for 1.6.1 (git tag -s 1.6.1 origin/1.6.1-rc1). I guess technically anybody could do this, and merge it (along with the version bump to 1.6.2-SNAPSHOT commit) to 1.6.2-SNAPSHOT branch (and forward, with -sours), if Corey doesn't have time/gets busy. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 2:21 PM, Corey Nolet cjno...@gmail.com wrote: There's still a few things I need to do before announcing the release to the user list. Merging the rc into the next version branch was one of them and creating the official release tag was another. I'll do these tonight as well as writing up the release notes for the site. On Thu, Sep 25, 2014 at 1:59 PM, Christopher ctubb...@apache.org wrote: Also, we can move this list to dev@. There's no reason for it to be private@ . -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 1:59 PM, Christopher ctubb...@apache.org wrote: There's one more problem that Keith and I found... it doesn't look like the rc1 branch got merged to 1.6.2-SNAPSHOT. I don't know if some other branch got accidentally merged instead. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 1:40 PM, Josh Elser josh.el...@gmail.com wrote: Things look good to me now. I checked the artifacts on dist/ against what I have from evaluating the RC and they appear to match. Anything else we need to do here? Christopher wrote: I was able to confirm the signature is bad. When I checked the RC, the signature was good, so I'm guessing the wrong one just got uploaded. I don't have a copy of the RC that I had previously downloaded, but I was able to grab a copy of what was deployed to Maven central and fix the dist sigs/checksums from that. Now, it's possible that the wrong artifacts were uploaded to Maven central (perhaps the wrong staging repo was promoted?) I can't know that for sure, until I can get to work and check my last download from the RC vote and compare with what's in Maven central now. If that is the case, then we need to determine precisely what is different from this upload and what was voted on and see if we need to immediately re-release as 1.6.2 to fix the problems. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 3:12 AM, Henk Penninghe...@apache.org wrote: Hi PMC accumulo, I watch 'www.apache.org/dist/', and I noticed that : -- you have 1 BAD pgp signature accumulo/1.6.1/accumulo-1.6.1-src.tar.gz.asc Please fix this problem soon ; for details, see http://people.apache.org/~henkp/checker/sig.html#project-accumulo http://people.apache.org/~henkp/checker/md5.html For information on how to fix problems, see the faq : http://people.apache.org/~henkp/checker/faq.html Thanks a lot, regards, Henk Penning -- apache.org infrastructure PS. The contents of this message is generated, but the mail itself is sent by hand. PS. Please cc me on all relevant emails. - _ Henk P. Penning, ICT-beta R Uithof WISK-412 _/ _ Faculty of Science, Utrecht University T +31 30 253 4106 / _/ Budapestlaan 6, 3584CD Utrecht, NL F +31 30 253 4553 _/ _/ http://people.cs.uu.nl/henkp/ M penn...@uu.nl _/
Re: [VOTE] Apache Accumulo 1.6.1 RC1
That seems like a reason to vote -1 (and perhaps to encourage others to do so also). I'm not sure this can be helped so long as people have different criteria for their vote, though. If we can fix those issues, I'm ready to vote on a 1.6.2 :) -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 2:42 PM, William Slacum wilhelm.von.cl...@accumulo.net wrote: I'm a little concerned we had two +1's that mention failures. The one time when we're supposed to have a clean run through, we have 50% of the participators noticing failure. It doesn't instill much confidence in me. On Thu, Sep 25, 2014 at 2:18 PM, Josh Elser josh.el...@gmail.com wrote: Please make a ticket for it and supply the MAC directories for the test and the failsafe output. It doesn't fail for me. It's possible that there is some edge case that you and Bill are hitting that I'm not. Corey Nolet wrote: I'm seeing the behavior under Max OS X and Fedora 19 and they have been consistently failing for me. I'm thinking ACCUMULO-3073. Since others are able to get it to pass, I did not think it should fail the vote solely on that but I do think it needs attention, quickly. On Thu, Sep 25, 2014 at 10:43 AM, Bill Havanki bhava...@clouderagovt.com wrote: I haven't had an opportunity to try it again since my +1, but prior to that it has been consistently failing. - I tried extending the timeout on the test, but it would still time out. - I see the behavior on Mac OS X and under CentOS. (I wonder if it's a JVM thing?) On Wed, Sep 24, 2014 at 9:06 PM, Corey Noletcjno...@gmail.com wrote: Vote passes with 4 +1's and no -1's. Bill, were you able to get the IT to run yet? I'm still having timeouts on my end as well. On Wed, Sep 24, 2014 at 1:41 PM, Josh Elserjosh.el...@gmail.com wrote: The crux of it is that both of the errors in the CRC where single bit variants. y instead of 9 and p instead of 0 Both of these cases are a '1' in the most significant bit of the byte instead of a '0'. We recognized these because y and p are outside of the hex range. Fixing both of these fixes the CRC error (manually verified). That's all we know right now. I'm currently running memtest86. I do not have ECC ram, so it *is* theoretically possible that was the cause. After running memtest for a day or so (or until I need my desktop functional again), I'll go back and see if I can reproduce this again. Mike Drob wrote: Any chance the IRC chats can make it only the ML for posterity? Mike On Wed, Sep 24, 2014 at 12:04 PM, Keith Turnerke...@deenlo.com wrote: On Wed, Sep 24, 2014 at 12:44 PM, Russ Weeks rwe...@newbrightidea.com wrote: Interesting that y (0x79) and 9 (0x39) are one bit away from each other. I blame cosmic rays! It is interesting, and thats only half of the story. Its been interesting chatting w/ Josh about this on irc and hearing about his findings. On Wed, Sep 24, 2014 at 9:05 AM, Josh Elserjosh.el...@gmail.com wrote: The offending keys are: 389a85668b6ebf8e 2ff6:4a78 [] 1411499115242 3a10885b-d481-4d00-be00-0477e231ey65:8576b169: 0cd98965c9ccc1d0:ba15529e The careful eye will notice that the UUID in the first component of the value has a different suffix than the next corrupt key/value (ends with ey65 instead of e965). Fixing this in the Value and re-running the CRC makes it pass. and 7e56b58a0c7df128 5fa0:6249 [] 1411499311578 3a10885b-d481-4d00-be00-0477e231e965:p000872d60eb: 499fa72752d82a7c:5c5f19e8 -- // Bill Havanki // Solutions Architect, Cloudera Govt Solutions // 443.686.9283
[GitHub] accumulo pull request: ACCUMULO-2942 Fixes ShardedTableDistributio...
Github user asfgit closed the pull request at: https://github.com/apache/accumulo/pull/10 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] accumulo pull request: ACCUMULO-2943 Fixing failures where no RNG ...
Github user asfgit closed the pull request at: https://github.com/apache/accumulo/pull/11 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: [VOTE] Apache Accumulo 1.6.1 RC1
Christopher, are you referring to Keith's last comment or Bill Slacum's? On Thu, Sep 25, 2014 at 9:13 PM, Christopher ctubb...@apache.org wrote: That seems like a reason to vote -1 (and perhaps to encourage others to do so also). I'm not sure this can be helped so long as people have different criteria for their vote, though. If we can fix those issues, I'm ready to vote on a 1.6.2 :) -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Sep 25, 2014 at 2:42 PM, William Slacum wilhelm.von.cl...@accumulo.net wrote: I'm a little concerned we had two +1's that mention failures. The one time when we're supposed to have a clean run through, we have 50% of the participators noticing failure. It doesn't instill much confidence in me. On Thu, Sep 25, 2014 at 2:18 PM, Josh Elser josh.el...@gmail.com wrote: Please make a ticket for it and supply the MAC directories for the test and the failsafe output. It doesn't fail for me. It's possible that there is some edge case that you and Bill are hitting that I'm not. Corey Nolet wrote: I'm seeing the behavior under Max OS X and Fedora 19 and they have been consistently failing for me. I'm thinking ACCUMULO-3073. Since others are able to get it to pass, I did not think it should fail the vote solely on that but I do think it needs attention, quickly. On Thu, Sep 25, 2014 at 10:43 AM, Bill Havanki bhava...@clouderagovt.com wrote: I haven't had an opportunity to try it again since my +1, but prior to that it has been consistently failing. - I tried extending the timeout on the test, but it would still time out. - I see the behavior on Mac OS X and under CentOS. (I wonder if it's a JVM thing?) On Wed, Sep 24, 2014 at 9:06 PM, Corey Noletcjno...@gmail.com wrote: Vote passes with 4 +1's and no -1's. Bill, were you able to get the IT to run yet? I'm still having timeouts on my end as well. On Wed, Sep 24, 2014 at 1:41 PM, Josh Elserjosh.el...@gmail.com wrote: The crux of it is that both of the errors in the CRC where single bit variants. y instead of 9 and p instead of 0 Both of these cases are a '1' in the most significant bit of the byte instead of a '0'. We recognized these because y and p are outside of the hex range. Fixing both of these fixes the CRC error (manually verified). That's all we know right now. I'm currently running memtest86. I do not have ECC ram, so it *is* theoretically possible that was the cause. After running memtest for a day or so (or until I need my desktop functional again), I'll go back and see if I can reproduce this again. Mike Drob wrote: Any chance the IRC chats can make it only the ML for posterity? Mike On Wed, Sep 24, 2014 at 12:04 PM, Keith Turnerke...@deenlo.com wrote: On Wed, Sep 24, 2014 at 12:44 PM, Russ Weeks rwe...@newbrightidea.com wrote: Interesting that y (0x79) and 9 (0x39) are one bit away from each other. I blame cosmic rays! It is interesting, and thats only half of the story. Its been interesting chatting w/ Josh about this on irc and hearing about his findings. On Wed, Sep 24, 2014 at 9:05 AM, Josh Elser josh.el...@gmail.com wrote: The offending keys are: 389a85668b6ebf8e 2ff6:4a78 [] 1411499115242 3a10885b-d481-4d00-be00-0477e231ey65:8576b169: 0cd98965c9ccc1d0:ba15529e The careful eye will notice that the UUID in the first component of the value has a different suffix than the next corrupt key/value (ends with ey65 instead of e965). Fixing this in the Value and re-running the CRC makes it pass. and 7e56b58a0c7df128 5fa0:6249 [] 1411499311578 3a10885b-d481-4d00-be00-0477e231e965:p000872d60eb: 499fa72752d82a7c:5c5f19e8 -- // Bill Havanki // Solutions Architect, Cloudera Govt Solutions // 443.686.9283
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request: https://github.com/apache/accumulo/pull/16#discussion_r18073259 --- Diff: core/src/main/java/org/apache/accumulo/core/client/admin/TableOperations.java --- @@ -103,6 +103,25 @@ void create(String tableName, boolean versioningIter, TimeType timeType) throws AccumuloException, AccumuloSecurityException, TableExistsException; /** + * @param tableName + * the name of the table + * @param limitVersion + * Enables/disables the versioning iterator, which will limit the number of Key versions kept. + * @param timeType + * specifies logical or real-time based time recording for entries in the table + * @param properties + * initial table properties the user wants + * @throws AccumuloException + * if a general error occurs + * @throws AccumuloSecurityException + * if the user does not have permission + * @throws TableExistsException + * if the table already exists + */ + void create(String tableName, boolean limitVersion, TimeType timeType, MapString,String properties) throws AccumuloException, AccumuloSecurityException, --- End diff -- Why are we adding more constructors at all? It seems unrelated to the nature of the title. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request: https://github.com/apache/accumulo/pull/16#discussion_r18073371 --- Diff: core/src/main/java/org/apache/accumulo/core/client/admin/TableOperations.java --- @@ -103,6 +103,25 @@ void create(String tableName, boolean versioningIter, TimeType timeType) throws AccumuloException, AccumuloSecurityException, TableExistsException; /** + * @param tableName + * the name of the table + * @param limitVersion + * Enables/disables the versioning iterator, which will limit the number of Key versions kept. + * @param timeType + * specifies logical or real-time based time recording for entries in the table + * @param properties + * initial table properties the user wants + * @throws AccumuloException + * if a general error occurs + * @throws AccumuloSecurityException + * if the user does not have permission + * @throws TableExistsException + * if the table already exists + */ + void create(String tableName, boolean limitVersion, TimeType timeType, MapString,String properties) throws AccumuloException, AccumuloSecurityException, --- End diff -- That's a use case, but it isn't a necessity. Why can't the existing table property configuration methods be used? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073474
--- Diff: test/src/test/java/org/apache/accumulo/test/VolumeChooserIT.java
---
@@ -0,0 +1,670 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.test;
+
+import static org.junit.Assert.*;
+
+import java.io.File;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.SortedSet;
+import java.util.TreeSet;
+import java.util.Map.Entry;
+
+import org.apache.accumulo.core.client.BatchWriter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.admin.TimeType;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Mutation;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import
org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.DataFileColumnFamily;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.minicluster.impl.MiniAccumuloConfigImpl;
+import org.apache.accumulo.test.functional.ConfigurableMacIT;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RawLocalFileSystem;
+import org.apache.hadoop.io.Text;
+import org.junit.Test;
+
+/**
+ *
+ */
+public class VolumeChooserIT extends ConfigurableMacIT {
+
+ private static final Text EMPTY = new Text();
+ private static final Value EMPTY_VALUE = new Value(new byte[] {});
+ private File volDirBase;
+ private Path v1, v2, v3, v4;
+
+ @Override
+ public void configure(MiniAccumuloConfigImpl cfg, Configuration
hadoopCoreSite) {
+// Get 2 tablet servers
+cfg.setNumTservers(2);
+
+// Set the general volume chooser to the GeneralVolumeChooser so that
different choosers can be specified
+MapString,String siteConfig = new HashMapString,String();
+siteConfig.put(Property.GENERAL_VOLUME_CHOOSER.getKey(),
org.apache.accumulo.server.fs.GeneralVolumeChooser.class.getName());
+cfg.setSiteConfig(siteConfig);
+
+// Set up 4 different volume paths
+File baseDir = cfg.getDir();
+volDirBase = new File(baseDir, volumes);
+File v1f = new File(volDirBase, v1);
+File v2f = new File(volDirBase, v2);
+File v3f = new File(volDirBase, v3);
+File v4f = new File(volDirBase, v4);
+v1f.mkdir();
+v2f.mkdir();
+v4f.mkdir();
+v1 = new Path(file:// + v1f.getAbsolutePath());
+v2 = new Path(file:// + v2f.getAbsolutePath());
+v3 = new Path(file:// + v3f.getAbsolutePath());
+v4 = new Path(file:// + v4f.getAbsolutePath());
+
+// Only add volumes 1, 2, and 4 to the list of instance volumes to
have one volume that isn't in the options list when they are choosing
+cfg.setProperty(Property.INSTANCE_VOLUMES, v1.toString() + , +
v2.toString() + , + v4.toString());
+
+// use raw local file system so walogs sync and flush will work
+hadoopCoreSite.set(fs.file.impl, RawLocalFileSystem.class.getName());
+
+super.configure(cfg, hadoopCoreSite);
+
+ }
+
+ // Test that uses two tables with 10 split points each. They each use
the StaticVolumeChooser to choose volumes.
+ @Test(timeout = 60 * 1000)
+ public void twoTablesStaticVolumeChooser() throws Exception {
+log.info(Starting StaticVolumeChooser);
+
+// Create and populate initial properties map for creating table 1
+MapString,String properties = new HashMapString,String();
+String propertyName = table.custom.chooser;
+String volume =
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073475
--- Diff: test/src/test/java/org/apache/accumulo/test/VolumeChooserIT.java
---
@@ -0,0 +1,670 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.test;
+
+import static org.junit.Assert.*;
+
+import java.io.File;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.SortedSet;
+import java.util.TreeSet;
+import java.util.Map.Entry;
+
+import org.apache.accumulo.core.client.BatchWriter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.admin.TimeType;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Mutation;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import
org.apache.accumulo.core.metadata.schema.MetadataSchema.TabletsSection.DataFileColumnFamily;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.minicluster.impl.MiniAccumuloConfigImpl;
+import org.apache.accumulo.test.functional.ConfigurableMacIT;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.RawLocalFileSystem;
+import org.apache.hadoop.io.Text;
+import org.junit.Test;
+
+/**
+ *
+ */
+public class VolumeChooserIT extends ConfigurableMacIT {
+
+ private static final Text EMPTY = new Text();
+ private static final Value EMPTY_VALUE = new Value(new byte[] {});
+ private File volDirBase;
+ private Path v1, v2, v3, v4;
+
+ @Override
+ public void configure(MiniAccumuloConfigImpl cfg, Configuration
hadoopCoreSite) {
+// Get 2 tablet servers
+cfg.setNumTservers(2);
+
+// Set the general volume chooser to the GeneralVolumeChooser so that
different choosers can be specified
+MapString,String siteConfig = new HashMapString,String();
+siteConfig.put(Property.GENERAL_VOLUME_CHOOSER.getKey(),
org.apache.accumulo.server.fs.GeneralVolumeChooser.class.getName());
+cfg.setSiteConfig(siteConfig);
+
+// Set up 4 different volume paths
+File baseDir = cfg.getDir();
+volDirBase = new File(baseDir, volumes);
+File v1f = new File(volDirBase, v1);
+File v2f = new File(volDirBase, v2);
+File v3f = new File(volDirBase, v3);
+File v4f = new File(volDirBase, v4);
+v1f.mkdir();
+v2f.mkdir();
+v4f.mkdir();
+v1 = new Path(file:// + v1f.getAbsolutePath());
+v2 = new Path(file:// + v2f.getAbsolutePath());
+v3 = new Path(file:// + v3f.getAbsolutePath());
+v4 = new Path(file:// + v4f.getAbsolutePath());
+
+// Only add volumes 1, 2, and 4 to the list of instance volumes to
have one volume that isn't in the options list when they are choosing
+cfg.setProperty(Property.INSTANCE_VOLUMES, v1.toString() + , +
v2.toString() + , + v4.toString());
+
+// use raw local file system so walogs sync and flush will work
+hadoopCoreSite.set(fs.file.impl, RawLocalFileSystem.class.getName());
+
+super.configure(cfg, hadoopCoreSite);
+
+ }
+
+ // Test that uses two tables with 10 split points each. They each use
the StaticVolumeChooser to choose volumes.
+ @Test(timeout = 60 * 1000)
+ public void twoTablesStaticVolumeChooser() throws Exception {
+log.info(Starting StaticVolumeChooser);
+
+// Create and populate initial properties map for creating table 1
+MapString,String properties = new HashMapString,String();
+String propertyName = table.custom.chooser;
+String volume =
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073494
--- Diff:
server/base/src/main/java/org/apache/accumulo/server/fs/StaticVolumeChooser.java
---
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.fs;
+
+import java.util.Map;
+
+import org.apache.accumulo.core.conf.AccumuloConfiguration.AllFilter;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.PropertyFilter;
+import org.apache.log4j.Logger;
+
+public class StaticVolumeChooser implements VolumeChooser {
--- End diff --
If this is being provided for users to leverage, it needs unit tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073509
--- Diff:
server/base/src/main/java/org/apache/accumulo/server/fs/RandomVolumeChooser.java
---
@@ -16,14 +16,65 @@
*/
package org.apache.accumulo.server.fs;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
import java.util.Random;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.AllFilter;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.PropertyFilter;
+import org.apache.log4j.Logger;
+
public class RandomVolumeChooser implements VolumeChooser {
+ private static final Logger log =
Logger.getLogger(RandomVolumeChooser.class);
Random random = new Random();
-
+
+ public RandomVolumeChooser() {}
+
@Override
- public String choose(String[] options) {
-return options[random.nextInt(options.length)];
+ public String choose(VolumeChooserEnvironment env, String[] options) {
+// Get the current table's properties, and find the preferred volumes
property
+PropertyFilter filter = new AllFilter();
+MapString,String props = new java.util.HashMapString,String();
+ArrayListString prefVol = new ArrayListString();
+env.getProperties(props, filter);
+String volumes = props.get(table.custom.preferredVolumes);
--- End diff --
Should be an element in Property, not a hard-coded, inline'd string.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073546
--- Diff:
server/base/src/main/java/org/apache/accumulo/server/fs/RandomVolumeChooser.java
---
@@ -16,14 +16,65 @@
*/
package org.apache.accumulo.server.fs;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
import java.util.Random;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.AllFilter;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.PropertyFilter;
+import org.apache.log4j.Logger;
+
public class RandomVolumeChooser implements VolumeChooser {
+ private static final Logger log =
Logger.getLogger(RandomVolumeChooser.class);
Random random = new Random();
-
+
+ public RandomVolumeChooser() {}
--- End diff --
Actually, this is really confusing because when the table property is set,
this isn't a random volume chooser at all -- it's a choose a volume from
this set chooser. It would make more sense to me for this code to live in its
own volume chooser implementation rather than push itself into the existing
RandomVolumeChooser.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073654
--- Diff:
server/base/src/main/java/org/apache/accumulo/server/fs/GeneralVolumeChooser.java
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.fs;
+
+import java.util.Map;
+
+import org.apache.accumulo.core.conf.AccumuloConfiguration.AllFilter;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.PropertyFilter;
+import org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader;
+import org.apache.log4j.Logger;
+
+public class GeneralVolumeChooser implements VolumeChooser {
+ private static final Logger log =
Logger.getLogger(GeneralVolumeChooser.class);
+
+ public GeneralVolumeChooser() {}
+
+ @Override
+ public String choose(VolumeChooserEnvironment env, String[] options) {
+String clazzName = new String();
+try {
+ // Get the current table's properties, and find the chooser property
+ PropertyFilter filter = new AllFilter();
+ MapString,String props = new java.util.HashMapString,String();
+ env.getProperties(props, filter);
+
+ clazzName = props.get(table.custom.chooser);
+ log.info(TableID: + env.getTableId() + ClassName: +
clazzName);
+
+ // Load the correct chooser and create an instance of it
+ Class? extends VolumeChooser clazz =
AccumuloVFSClassLoader.loadClass(clazzName, VolumeChooser.class);
--- End diff --
This is going to be rather expensive to load the class and instantiate it
every single time we choose a volume for a file WRT how quick it should be.
Make sure the implementations are thread-safe and aren't sharing state and you
can keep a cache of implementations around. (although this has issues when
considering dynamic classloading)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073661
--- Diff:
server/base/src/main/java/org/apache/accumulo/server/fs/GeneralVolumeChooser.java
---
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.fs;
+
+import java.util.Map;
+
+import org.apache.accumulo.core.conf.AccumuloConfiguration.AllFilter;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.PropertyFilter;
+import org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader;
+import org.apache.log4j.Logger;
+
+public class GeneralVolumeChooser implements VolumeChooser {
+ private static final Logger log =
Logger.getLogger(GeneralVolumeChooser.class);
+
+ public GeneralVolumeChooser() {}
+
+ @Override
+ public String choose(VolumeChooserEnvironment env, String[] options) {
+String clazzName = new String();
+try {
+ // Get the current table's properties, and find the chooser property
+ PropertyFilter filter = new AllFilter();
+ MapString,String props = new java.util.HashMapString,String();
+ env.getProperties(props, filter);
+
+ clazzName = props.get(table.custom.chooser);
--- End diff --
Again, should be some public constant, likely in Property and something
that ties it to the `GeneralVolumeChooser` class
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073696
--- Diff:
server/base/src/main/java/org/apache/accumulo/server/fs/StaticVolumeChooser.java
---
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.fs;
+
+import java.util.Map;
+
+import org.apache.accumulo.core.conf.AccumuloConfiguration.AllFilter;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.PropertyFilter;
+import org.apache.log4j.Logger;
+
+public class StaticVolumeChooser implements VolumeChooser {
+ private static final Logger log =
Logger.getLogger(StaticVolumeChooser.class);
+
+ public StaticVolumeChooser() {}
--- End diff --
Need some unit tests for `StaticVolumeChooser`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073713
--- Diff:
server/base/src/main/java/org/apache/accumulo/server/fs/StaticVolumeChooser.java
---
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.server.fs;
+
+import java.util.Map;
+
+import org.apache.accumulo.core.conf.AccumuloConfiguration.AllFilter;
+import org.apache.accumulo.core.conf.AccumuloConfiguration.PropertyFilter;
+import org.apache.log4j.Logger;
+
+public class StaticVolumeChooser implements VolumeChooser {
+ private static final Logger log =
Logger.getLogger(StaticVolumeChooser.class);
+
+ public StaticVolumeChooser() {}
+
+ @Override
+ public String choose(VolumeChooserEnvironment env, String[] options) {
+// Get the current table's properties, and find the preferred volumes
property
+PropertyFilter filter = new AllFilter();
+MapString,String props = new java.util.HashMapString,String();
+env.getProperties(props, filter);
+String volume = props.get(table.custom.preferredVolumes);
+log.info(In custom chooser);
--- End diff --
Fine for debugging, but these need to be removed or pulled back to trace
logging.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request: ACCUMULO-3089: Create volume chooser from t...
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/16#discussion_r18073825
--- Diff:
server/base/src/main/java/org/apache/accumulo/server/util/FileUtil.java ---
@@ -79,7 +81,7 @@ public Text getLastRow() {
private static final Logger log = Logger.getLogger(FileUtil.class);
private static Path createTmpDir(AccumuloConfiguration acuConf,
VolumeManager fs) throws IOException {
-String accumuloDir = fs.choose(ServerConstants.getTemporaryDirs());
+String accumuloDir = fs.choose(Optional.Stringabsent(),
ServerConstants.getTemporaryDirs()) + /tables/;
--- End diff --
Why the extra tables in the path?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
