Re: 1.6.2 candidates
We are running 1.6.1 w/patches in production already. I would much rather have a 1.6.2 official release. I may have temporary access to a small cluster (3-ish racks) to run some of the long running tests on bare metal. Testing sooner, rather than later is preferable. On Tue, Dec 16, 2014 at 7:18 PM, Corey Nolet wrote: > > I have cycles to spin the RCs- I wouldn't mind finishing the updates (per > my notes) of the release documentation as well. > > On Tue, Dec 16, 2014 at 7:11 PM, Christopher wrote: > > > > I think it'd be good to let somebody else exercise the process a bit, > but I > > can make the RCs if nobody else volunteers. My primary concern is that > > people will have time to test. > > > > > > -- > > Christopher L Tubbs II > > http://gravatar.com/ctubbsii > > > > On Tue, Dec 16, 2014 at 6:37 PM, Josh Elser > wrote: > > > > > > +1 There are lots of good bug fixes in 1.6.2 already. > > > > > > I can make some time to test, document, etc. Are you volunteering to > spin > > > the RCs as well? > > > > > > > > > Christopher wrote: > > > > > >> I'm thinking we should look at releasing 1.6.2 in January. I'd say > > sooner, > > >> but I don't know if people will have time to test if we start putting > > >> together RCs this week or next. > > >> > > >> -- > > >> Christopher L Tubbs II > > >> http://gravatar.com/ctubbsii > > >> > > >> > > >
Re: Accumulo Map Reduce
If you're still getting connection refused errors, make sure that you started the YARN Resource Manager and Node Manager processes, typically via $HADOOP_PREFIX/sbin/start-yarn.sh. panqing...@163.com wrote: I using Hadoop 2.5.0 ,I want to use MapReduce in online, do the table associated operation. Accumulo provides an AccumuloInputFormat which allows Accumulo to be the source of data for a MapReduce job. We also provide a few OutputFormat implementations, the easiest to use being the AccumuloOutputFormat which will use BatchWriters to write mutations that you generate into Accumulo. You can look at https://github.com/apache/accumulo/blob/1.6/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TeraSortIngest.java for an end-to-end example of a MapReduce job. If you're trying to run your MapReduce job in local mode and getting a ConnectException, it sounds like you're not actually running in local mode as you expect. Can you provide more information as to what you're actually invoking? Also, are you using Hadoop 1 or Hadoop 2? On Mon, Dec 15, 2014 at 3:52 AM, panqing...@163.com wrote: Hi: There are several problems on MapReduce 1、I wanted to know more about the Accumulo Map Reduce is a specific function is to achieve what function? Use in what time? 2、I want to do a MapReduce test in the local, there is always wrong. MapReduce write their own should be how to run the test? Caused by: java.net.ConnectException: Connection refused: no further information panqing...@163.com Quoted from: http://apache-accumulo.1065345.n5.nabble.com/Accumulo-Map-Reduce-tp12575p12576.html _ Sent from http://apache-accumulo.1065345.n5.nabble.com
Re: Namespaces & Application Classpath interplay
Thanks - I had completely glossed over the "config -ns" option. So not an edge case - rather just pure mea culpa.I'll test it tomorrow but looks like it should do exactly what I need. Chris On Tue, Dec 16, 2014 at 12:59 AM, Christopher wrote: > > I'd imagine you can set the context classpath for the entire namespace > with: > > # this in the accumulo-site.xml > general.vfs.context.classpath.mycontext=jar1,jar2,jar3,etc. > and > # this in the namespace config > table.classpath.context=mycontext > > > -- > Christopher L Tubbs II > http://gravatar.com/ctubbsii > > On Mon, Dec 15, 2014 at 11:46 PM, Josh Elser wrote: > > > > In essence, you're asking to apply some context classloader to a > namespace > > and have it propagate down to each table in that namespace as opposed to > > setting it on each table? By doing so, you wouldn't have to ensure that > > each table for your logical "application" is configured with the same > > context classloader. > > > > If that's the case, +1 for the change -- nearly all of our properties > > already "delegate" in that fashion: per-table inherits from namespace > which > > inherits from system. I'm a little sad if it doesn't actually already do > it > > (because that means classloader configuration is an edge case > internally), > > but happy that you'd like to work on it :) > > > > > > Chris Bennight wrote: > > > >> (Note - my assumption here is that I can't currently assign an isolated > >> classpath as "default" for a particular namespace; if I missed > something > >> apologies) > >> > >> So I get the ability to assign an application specific classpath on a > per > >> table basis. In my case I create (and sometimes remove) tables via the > >> API > >> (as opposed to the shell) - which scope normally mediated by a > namespace. > >> > >> I can attach an application classpath at creation to each table, but > this > >> requires sharing information (name of application classpath somehow > >> configured/set for application), or convention (always call it "xyz") > >> between the accumulo config and client application. > >> > >> I think a cleaner solution in this part would be to let the classpath be > >> tied through accumulo configuration to the namespace (this would also > make > >> classpath isolation easier for other programs - they don't have to add > in > >> hooks in the case of programatically creating tables). > >> > >> So my questions: > >> -- Is this something that is already in the works? (I swear, I looked > in > >> JIRA) > >> -- Is it a terrible idea (why?) > >> -- If it's not already done, and not a bad idea, should I create a > ticket, > >> propose a design, and start coding after vetting? > >> > >> Thanks, > >> Chris > >> > >> >
[GitHub] accumulo pull request: 1.6
Github user tim-to closed the pull request at: https://github.com/apache/accumulo/pull/20 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] accumulo pull request: 1.6
GitHub user tim-to opened a pull request: https://github.com/apache/accumulo/pull/20 1.6 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/accumulo 1.6 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/accumulo/pull/20.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20 commit 22de8d5237ff14b9c46638df0f998368f80a5cc9 Author: Christopher Tubbs Date: 2014-09-12T21:36:39Z ACCUMULO-3078 Fix ConditionalWriterIT trace test Discovered by "dead code" warning. commit d3e17f4582634e0f4985c1f3a92afcd67e999427 Author: Christopher Tubbs Date: 2014-09-12T22:16:09Z ACCUMULO-3119 Cleanup javadocs in 1.6.x branch commit 939ed96bb31a5473a4eaf22e072c84ab76b29fbf Author: Corey J. Nolet Date: 2014-09-13T01:19:31Z Merge branch '1.5.2-SNAPSHOT' into 1.6.1-SNAPSHOT commit bc53d3a4be7466524f9106d9b97399d6f6ba2119 Author: Josh Elser Date: 2014-09-15T15:17:47Z Merge branch '1.5.2-SNAPSHOT' into 1.6.1-SNAPSHOT Conflicts: CHANGES commit 4556ec4bd1d7057fa485e2531348ae4e57b4cc88 Author: Josh Elser Date: 2014-09-15T21:05:34Z ACCUMULO-3128 Reduce zookeeper log spam Also reduce SslConnectionParams because that was just as spammy commit e455005f6e9a33f82eb25d6b89816704ad8663de Author: Josh Elser Date: 2014-09-15T22:49:16Z ACCUMULO-3128 Screwed up the conditional commit 7f62ec1b37dc1e8fe7512358360dcb591b0ba224 Author: Josh Elser Date: 2014-09-16T00:26:02Z ACCUMULO-3129 Pass credential provider into client configuration and set non-empty truststore password. Making a non-empty password for the truststore makes things a bit more reliable to actually work (I think empty passwords are disallowed by the CredentialProvider impls). If the client is also using a CredentialProvider, the ClientConfiguration also needs this property. Adding the extra value to the enum passes it from accumulo-site.xml into the generate client.conf by AbstractMacIT. commit 0055bab8b94bb87c1b65db6223a5ae8840d2d52c Author: Eric C. Newton Date: 2014-09-16T21:22:53Z ACCUMULO-3132 kludge the deserialization to work with 1.6.0 FATE objects commit 2e273505d9ca2d32db48d2052f38be2625119bec Author: Josh Elser Date: 2014-09-16T21:54:42Z ACCUMULO-3136 Be a little more rigid on the ActiveScans we observe. We might get scans by the system to the metadata table instead of the scan we run in the test case. When this happens, the test will timeout because the interrupt happens before the scan is running, and then the scan just sits on the SlowIterator. commit 3e4d07b2fd967328fe1161b0dfad1fdf72191829 Author: Josh Elser Date: 2014-09-17T04:56:25Z ACCUMULO-3138 Pass down the SSL/CredentialProvider ClientConfiguration into the input format commit 597b787cee82a0cb3a8c39e70de312878d0182fc Author: Eric C. Newton Date: 2014-09-17T15:14:04Z ACCUMULO-3132 setting TInfo's serialization value is much simpler After discussing it with [~kturner], this change seems simpler and supports 1.6.1 -> 1.6.0 revert of an upgrade. The unit test will catch any re-generation of the file. commit 0bd5d438578f45e66eaa0dcd7cfb81f8693841ee Author: Eric C. Newton Date: 2014-09-17T15:18:37Z Merge remote branch 'origin/1.6.1-SNAPSHOT' into 1.6.1-SNAPSHOT commit 4cf467b926d7beae17b47103105d892f24e75e70 Author: Josh Elser Date: 2014-09-17T19:06:41Z ACCUMULO-3139 Reflect on UNIXProcess to get the pid and use kill instead of pkill. pkill has the potential to overmatch and SIGSTOP+SIGCONT other tablet servers on the same node. If it tries to match tservers started by other users, this will also fail, AFAICT. commit d0f95f85a484e190ddaeb753ab141ce7c3d84ff1 Author: Josh Elser Date: 2014-09-17T21:07:30Z ACCUMULO-3139 Some extra test stabilizations for BalanceAfterCommsFailureIT commit 255ce5f25164dc1c429ee57bbeecfdb1f976cacb Author: Josh Elser Date: 2014-09-18T16:23:26Z ACCUMULO-3078 ACCUMULO-3138 Remove the getStaticCluster reference again The AccumuloConfiguration from the Instance is at least the ClientConfiguration. We can use that to remove the reliance on needing the shared MiniAccumuloCluster. commit 32570d5a554503d7af6c76379edd8d46175637e2 Author: Josh Elser Date: 2014-09-18T18:33:21Z ACCUMULO-3144 Give the servers some time to flush out their buffers to the MAC log pipe. commit 992414ebba8109b601846b5c279bf72b8165485d Author: Josh Elser Date: 2014-09-18T19:01:02Z ACCUMULO-3145 Fix loop invariants in listscans Added some extra logging and (automatically) fixed formatting. commit 7a40f7bfa4c6e30467a24105da974687ac442cdf Author: Josh Elser Date: 2014-09-18T22:12:12Z ACCUMULO-3146 Try to sta
Re: 1.6.2 candidates
I have cycles to spin the RCs- I wouldn't mind finishing the updates (per my notes) of the release documentation as well. On Tue, Dec 16, 2014 at 7:11 PM, Christopher wrote: > > I think it'd be good to let somebody else exercise the process a bit, but I > can make the RCs if nobody else volunteers. My primary concern is that > people will have time to test. > > > -- > Christopher L Tubbs II > http://gravatar.com/ctubbsii > > On Tue, Dec 16, 2014 at 6:37 PM, Josh Elser wrote: > > > > +1 There are lots of good bug fixes in 1.6.2 already. > > > > I can make some time to test, document, etc. Are you volunteering to spin > > the RCs as well? > > > > > > Christopher wrote: > > > >> I'm thinking we should look at releasing 1.6.2 in January. I'd say > sooner, > >> but I don't know if people will have time to test if we start putting > >> together RCs this week or next. > >> > >> -- > >> Christopher L Tubbs II > >> http://gravatar.com/ctubbsii > >> > >> >
Re: 1.6.2 candidates
I think it'd be good to let somebody else exercise the process a bit, but I can make the RCs if nobody else volunteers. My primary concern is that people will have time to test. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Tue, Dec 16, 2014 at 6:37 PM, Josh Elser wrote: > > +1 There are lots of good bug fixes in 1.6.2 already. > > I can make some time to test, document, etc. Are you volunteering to spin > the RCs as well? > > > Christopher wrote: > >> I'm thinking we should look at releasing 1.6.2 in January. I'd say sooner, >> but I don't know if people will have time to test if we start putting >> together RCs this week or next. >> >> -- >> Christopher L Tubbs II >> http://gravatar.com/ctubbsii >> >>
Re: 1.6.2 candidates
+1 There are lots of good bug fixes in 1.6.2 already. I can make some time to test, document, etc. Are you volunteering to spin the RCs as well? Christopher wrote: I'm thinking we should look at releasing 1.6.2 in January. I'd say sooner, but I don't know if people will have time to test if we start putting together RCs this week or next. -- Christopher L Tubbs II http://gravatar.com/ctubbsii
1.6.2 candidates
I'm thinking we should look at releasing 1.6.2 in January. I'd say sooner, but I don't know if people will have time to test if we start putting together RCs this week or next. -- Christopher L Tubbs II http://gravatar.com/ctubbsii
