Re: Accumulo Quarterly Report - Due Wed, July 10th.

2024-07-08 Thread Dave Marion
LGTM. Thanks Ed! On Mon, Jul 8, 2024 at 6:35 AM Mark Owens wrote: > LGTM > > On Mon, Jul 8, 2024 at 1:49 AM Ed Coleman wrote: > > > The Accumulo community decided to draft the Apache community reports on > the > > Accumulo dev list – and this is a draft of the April report for review / > >

Re: [DISCUSS] Classic GitHub Projects sunsetting

2024-07-03 Thread Dave Marion
reating a new Project would be overkill. I > just don't think it's a great option for milestone tracking. > > On Wed, Jul 3, 2024 at 2:15 PM Dave Marion wrote: > > > > Another option would be to do it manually by creating an issue where the > > top comment is a checklist

Re: [DISCUSS] Classic GitHub Projects sunsetting

2024-07-03 Thread Dave Marion
Another option would be to do it manually by creating an issue where the top comment is a checklist of all the related items in that release. Much like the post-vote release checklist[1] issues that we have. Yes, this means that the developer is going to have to update some other number of tickets

Re: Accumulo Quarterly Report - Due Wednesday, April 10th.

2024-04-05 Thread Dave Marion
;here are some things you may find helpful" - maybe the > proxy is more than that? But I don't recall ever mention the proxy > specifically in a project report. > > Would if help if the sub-heading (it's at the same level as releases) read > "Other Releases:" ? > > O

Re: Accumulo Quarterly Report - Due Wednesday, April 10th.

2024-04-05 Thread Dave Marion
is the consensus. Same as considering it as a release - > but as you mentioned we don't seem to treat other sub-projects that way. > > > On 2024/04/05 17:59:22 Dave Marion wrote: > > Just curious why the Accumulo Access library mentions is in "Other". I > > think an alt

Re: Accumulo Quarterly Report - Due Wednesday, April 10th.

2024-04-05 Thread Dave Marion
Just curious why the Accumulo Access library mentions is in "Other". I think an alternate version of this could describe Accumulo Access in the "Project Status" or "Project Activity" section and then add the release to the "Releases" section. Does it get relegated to "Other" because it's a

[TEST][ANNOUNCE] Apache Accumulo Access 1.0.0-beta

2024-02-20 Thread Dave Marion
The Apache Accumulo project is pleased to announce the release of Apache Accumulo Access 1.0.0-beta. Accumulo Access is a standalone java library that provides the same functionality, semantics, and syntax as the Accumulo ColumnVisibility and VisibilityEvaluator classes. The Accumulo Access

Re: [DRAFT][ANNOUNCE] Apache Accumulo Access 1.0.0-beta

2024-02-20 Thread Dave Marion
of Accumulo Access is designated as beta because the API may change as > we work to integrate it into Accumulo. > > On Tue, Feb 20, 2024 at 8:06 AM Dave Marion wrote: > > > All, > > > > Please review and provide feedback on the draft release announcement > > belo

[DRAFT][ANNOUNCE] Apache Accumulo Access 1.0.0-beta

2024-02-20 Thread Dave Marion
All, Please review and provide feedback on the draft release announcement below. As this is the initial introduction of Accumulo Access I have included more information than we would normally put into a release announcement. --- The Apache Accumulo project is pleased to announce the release

[RESULT][VOTE] Apache Accumulo-Access 1.0.0-beta-rc4

2024-02-17 Thread Dave Marion
This vote passes with 6 +1s and no other votes. Thanks all for verifying this release candidate. The post-vote release steps are being tracked at: https://github.com/apache/accumulo-access/issues/63

Re: [VOTE] Apache Accumulo-Access 1.0.0-beta-rc4

2024-02-16 Thread Dave Marion
repo when doing releases. It's late tonight as > I write this, but I created the upstream issue. We'll see if it gains > traction, and if a solution arises upstream. If not, I have an idea for a > workaround locally in our project that might work for the next release or > release can

Re: [VOTE] Apache Accumulo-Access 1.0.0-beta-rc4

2024-02-15 Thread Dave Marion
in the README On Wed, Feb 14, 2024 at 9:27 AM Dave Marion wrote: > Accumulo Developers, > > Please consider the following candidate for Apache Accumulo-Access > 1.0.0-beta. > > Git Commit: > 1c6051d4f450d620e3594659ed8f00c107348003 > Branch: > 1.0.0-beta-rc4 > > If t

[VOTE] Apache Accumulo-Access 1.0.0-beta-rc4

2024-02-14 Thread Dave Marion
Accumulo Developers, Please consider the following candidate for Apache Accumulo-Access 1.0.0-beta. Git Commit: 1c6051d4f450d620e3594659ed8f00c107348003 Branch: 1.0.0-beta-rc4 If this vote passes, a gpg-signed tag will be created using: git tag -f -s -m 'Apache Accumulo-Access

Re: [VOTE] Apache Accumulo-Access 1.0.0-rc3

2024-02-13 Thread Dave Marion
is working on) and include any other changes that stem from that in a 1.0.0 release. I have created PR #49 which changes the version to beta. My plan is to merge that and then create another RC in the morning for 1.0.0-beta. On Tue, Feb 13, 2024 at 10:58 AM Dave Marion wrote: > > Thi

Re: [VOTE] Apache Accumulo-Access 1.0.0-rc3

2024-02-13 Thread Dave Marion
violates the release policy and needs to be > updated. > > On Tue, Feb 13, 2024 at 8:28 AM Dave Marion wrote: > > > -1 (binding) > > > > I performed the following checks and I think I need to vote -1 because of > > the language at > > https://apache.org/lega

Re: [VOTE] Apache Accumulo-Access 1.0.0-rc3

2024-02-13 Thread Dave Marion
ulo-access/pull/45 > > > On Mon, Feb 12, 2024 at 9:57 AM Dave Marion wrote: > > > Accumulo Developers, > > > > Please consider the following candidate for Apache Accumulo-Access 1.0.0. > > > > Git Commit: > > 276dfe1bfb3868812da5414d79f4d995bade9

[VOTE] Apache Accumulo-Access 1.0.0-rc3

2024-02-12 Thread Dave Marion
Accumulo Developers, Please consider the following candidate for Apache Accumulo-Access 1.0.0. Git Commit: 276dfe1bfb3868812da5414d79f4d995bade9a36 Branch: 1.0.0-rc3 If this vote passes, a gpg-signed tag will be created using: git tag -f -s -m 'Apache Accumulo-Access 1.0.0'

Re: Accumulo Board Report - due by Wed Jan 10th

2024-01-05 Thread Dave Marion
Looks good Ed. Thanks! On Fri, Jan 5, 2024 at 10:20 AM dev1 wrote: > > > The Accumulo community decided to draft the Apache community reports on > the Accumulo dev list – and this is a draft of the January report for > review / comments. The report is due by Jan 10th. > > The text below is

Re: [DRAFT][ANNOUNCE] Apache Accumulo 1.10.4

2023-11-16 Thread Dave Marion
Looks good. On Thu, Nov 16, 2023 at 5:37 PM Christopher wrote: > The following is the draft of the announcement for 1.10.4 > *** > > The Apache Accumulo project is pleased to announce the release of > Apache Accumulo 1.10.4! Apache Accumulo 1.10.4 is the final bug fix > release of the 1.10 LTM

Re: MutationsRejectedException

2023-10-23 Thread Dave Marion
By design, we typically don't send user data back in error messages so that we are not leaking data into a client side log or something. On Mon, Oct 23, 2023 at 8:43 AM Logan Jones wrote: > Hi Ed / Josef: > > Yes, the violation summaries are available, but I was hoping that the > actual

Re: New Accumulo git repository

2023-09-07 Thread Dave Marion
I don't think the risk of not creating a release from the new repo exists. I believe we have to use non-snapshot dependencies when creating an Accumulo release. Right? I have no issue with creating the repo now and no issue with the name, since we can rename it. On Wed, Sep 6, 2023 at 3:18 PM

Re: [VOTE] Apache Accumulo 3.0.0-rc1

2023-08-15 Thread Dave Marion
+1 * verified signatures and hashes * built from source tarball with no issues * Sunny day tests passed * Started instance locally. Ran ci until the disk was almost full. Ran scan, walk, batchwalk with no issues. * Looked at diff between 3.0 and 2.1 to look for changes that were not in the

Re: [VOTE] Apache Accumulo 2.1.2-rc1

2023-08-14 Thread Dave Marion
+1 I have: * verified hashes and signatures * confirmed errorprone completes successfully with the assert statement fixed in AccumuloInputFormatIT, so no other issues * ran sunny day tests * built with thrift profile, confirmed no changes On Mon, Aug 14, 2023 at 4:08 AM Christopher wrote: >

Re: New committer - Dan Roberts

2023-08-10 Thread Dave Marion
Congratulations Dan! On Thu, Aug 10, 2023 at 7:26 AM Christopher Shannon < christopher.l.shan...@gmail.com> wrote: > Congrats Dan! > > On Thu, Aug 10, 2023 at 12:08 AM Christopher wrote: > > > Welcome Dan, and congrats > > > > On Wed, Aug 9, 2023, 18:27 dev1 wrote: > > > > > The Project

Re: Accumulo Quaterly Report - Due by 7/12

2023-07-11 Thread Dave Marion
Looks good. On Tue, Jul 11, 2023 at 9:34 AM dev1 wrote: > I updated the section that Mark pointed out. It now reads (as pasted from > the reported tool): > > Work continues on the evolution of the Accumulo processing model to support > dynamic scaling and provide elasticity. The goal for

Re: [VOTE] Apache Accumulo 2.1.1-rc2

2023-06-16 Thread Dave Marion
+1 Verified signatures (asc, md5, sha1 and sha512) match expected Ran SunnyDay ITs Ran CI for 6+ hours with no issues on a 10-node cluster On Thu, Jun 15, 2023 at 2:15 PM dev1 wrote: > +1 > > Verified signatures (asc, md5, sha1 and sha512) match expected > Sunny tests passed with no failures.

Re: [VOTE] Apache Accumulo 2.1.1-rc1

2023-06-13 Thread Dave Marion
-1 - Because https://github.com/apache/accumulo/issues/3491 was found during testing which prevents normal compaction operation on a Tablet with a large row. I was able to do the following: * Verified hashes and signatures * Confirmed unit tests pass (mvn clean package) * Confirmed SunnyDay ITs

Re: [DISCUSS] Should we have Jekyll build directly to asf-site branch?

2023-04-25 Thread Dave Marion
> > > > 1. Initial PR to main branch > > > > > > > 2. Auto build and publish on PR merge > > > > > > > 3. View on published site to evaluate if additional changes > need to > > > > be > > > > > > > made and

Re: [VOTE] Apache Accumulo 1.10.3-rc1

2023-04-12 Thread Dave Marion
FoundError: > > > > > > org/apache/commons/beanutils/BeanIntrospector > > > > > > at > > > > > > > > > > > > > > > > > > > org.apache.accumulo.core.conf.CredentialProviderFactoryShimTest.extractFromHdfs(CredentialProviderFacto

Re: [VOTE] Apache Accumulo 1.10.3-rc1

2023-04-11 Thread Dave Marion
Chris Shannon pointed out that this test was failing and the resolution was to give it more memory (see https://github.com/apache/accumulo/issues/3281). On Mon, Apr 10, 2023 at 6:04 PM Dave Marion wrote: > *mvn clean verify -Phadoop3,sunny* failed in ExamplesIT with the > aforemen

Re: [VOTE] Apache Accumulo 1.10.3-rc1

2023-04-10 Thread Dave Marion
*mvn clean verify -Phadoop3,sunny* failed in ExamplesIT with the aforementioned changes. Specifically, ExamplesIT.testScansWithInterference failed with a server error when closing the batch writer. No further information in the logs. On Mon, Apr 10, 2023 at 5:06 PM Dave Marion wrote: > Updat

Re: [VOTE] Apache Accumulo 1.10.3-rc1

2023-04-10 Thread Dave Marion
/pom.xml @@ -212,7 +212,7 @@ commons-configuration commons-configuration -1.6 +1.10 commons-io On Mon, Apr 10, 2023 at 12:16 PM Dave Marion wrote: > From what I can tell: > > 1. commons-configuration:1.6 has a commons-beanut

Re: [VOTE] Apache Accumulo 1.10.3-rc1

2023-04-10 Thread Dave Marion
ust not be on the classpath for the test. I have no idea why this is failing on my machine and not for others. I'm also seeing general slowness building 1.10.3 on my machine, but not building other git branches. On Mon, Apr 10, 2023 at 8:24 AM Dave Marion wrote: > I built the source tarball using the hadoop3 p

Re: [VOTE] Apache Accumulo 1.10.3-rc1

2023-04-10 Thread Dave Marion
I built the source tarball using the hadoop3 profile again and encountered the issue below. The version of hadoop (3.0.3) has not changed, so I assume that this is related to a change in our dependencies. [ERROR] Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.856 s <<<

Re: [DISCUSS] Should we have Jekyll build directly to asf-site branch?

2023-04-09 Thread Dave Marion
streamlined. > > (For reference, the local build is documented in the website README at > > https://github.com/apache/accumulo-website/blob/main/README.md#local-builds-for-testing > ) > > On Fri, Apr 7, 2023 at 7:24 PM Dave Marion wrote: > > > > I find the stagi

Re: [DISCUSS] Should we have Jekyll build directly to asf-site branch?

2023-04-07 Thread Dave Marion
I find the staging site useful for reviewing changes before publishing. There are some things that are not the easiest to review in a markdown diff. Especially since the markdown is transformed to what is ultimately displayed. I have no issue with streamlining this, I'll just need to remember to

Re: [TEST][VOTE] Apache Accumulo 1.10.3-rc0

2023-04-06 Thread Dave Marion
oaderTest [ERROR] org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest Time elapsed: 1.15 s <<< ERROR! java.lang.NoSuchMethodError: 'boolean org.eclipse.jetty.servlet.ServletMapping.containsPathSpec(java.lang.String)' On Thu, Apr 6, 2023 at 1:36 PM Dave Mari

Re: [TEST][VOTE] Apache Accumulo 1.10.3-rc0

2023-04-06 Thread Dave Marion
`mvn -Dtimeout.factor=2 clean package` worked. Thanks! On Thu, Apr 6, 2023 at 1:19 PM Christopher wrote: > Might just be a coincidence. If it's still failing with a higher timeout, I > can look into it. > > On Thu, Apr 6, 2023, 12:51 Dave Marion wrote: > > > I ju

Re: [TEST][VOTE] Apache Accumulo 1.10.3-rc0

2023-04-06 Thread Dave Marion
pen for any of our branches. So > overriding the arbitrary timeout by some factor can help. > > On Thu, Apr 6, 2023, 12:41 Dave Marion wrote: > > > Verified sha1 & md5 signatures > > Verified signing key > > > > Ran into an issue trying to build from the source

Re: [TEST][VOTE] Apache Accumulo 1.10.3-rc0

2023-04-06 Thread Dave Marion
Verified sha1 & md5 signatures Verified signing key Ran into an issue trying to build from the source tarball. I tried 3 times with the command `mvn clean package` and the build failed in the same spot every time (see below). Note that on the same machine I ran `mvn clean package` on my local git

Re: Dynamic Scaling of Accumulo

2023-04-05 Thread Dave Marion
ior (see #3) for a range > of Tablets on a table. > > > On Tue, Apr 4, 2023 at 1:08 PM Keith Turner wrote: > >> On Mon, Apr 3, 2023 at 3:33 PM Dave Marion wrote: >> > >> > I could see that working initially, but I think you would get some drift >> > ove

Re: Dynamic Scaling of Accumulo

2023-04-05 Thread Dave Marion
023 at 3:33 PM Dave Marion wrote: > > > > I could see that working initially, but I think you would get some drift > > over time as splits or merges happen. In your example, what happens when > > Drift is definitely something to consider in the design and users > would

Re: [DISCUSS] Accumulo quarterly report (due Saturday 4/8)

2023-04-05 Thread Dave Marion
LGTM. Thanks for putting this together Ed. On Tue, Apr 4, 2023 at 4:27 PM dev1 wrote: > The Accumulo community decided to draft the Apache community reports on > the Accumulo dev list – and this is a draft of the January report for > review / comments. > > The text below is copied from the

Re: Dynamic Scaling of Accumulo

2023-04-03 Thread Dave Marion
and new splits are added daily or weekly, the range would have to be updated at the same time that splits are created to keep the last N days hosted. On Mon, Apr 3, 2023 at 2:13 PM Keith Turner wrote: > On Mon, Apr 3, 2023 at 10:45 AM Dave Marion wrote: > > > > Looking through the c

Re: Dynamic Scaling of Accumulo

2023-04-03 Thread Dave Marion
mpler user experience is outweighed by the complexity added to the code to achieve it. Personally, I don't think it's that hard to reason about, especially if the user reads the docs and it is explained well. On Wed, Mar 29, 2023 at 1:04 PM Christopher wrote: > On Wed, Mar 29, 2023 at 5:33 A

Re: Dynamic Scaling of Accumulo

2023-03-29 Thread Dave Marion
> I think we should deprecate support for offline table scanning, since it shouldn't be needed with the availability of ScanServers. Just making sure I understand your suggestion - you mean removing the OfflineScanner and the ability to scan over offline tables in the MapReduce code, but we

Re: Dynamic Scaling of Accumulo

2023-03-23 Thread Dave Marion
gt; there to use that feature? Do you mean simply never being brought > online? > > Would it be possible to support (external) compactions for an offline > table? > > I feel like that's a pretty useful feature to revert, and would want > to consider alternatives. > > On Thu,

Re: Dynamic Scaling of Accumulo

2023-03-23 Thread Dave Marion
continue to grow because compactions are never run on the table. On Mon, Mar 20, 2023 at 9:37 AM Dave Marion wrote: > Following up on this. Discussion and design documents are up on the > wiki[1]. There is a GitHub project[2] for planning out some of the tasks, > which are then turned in

Re: Dynamic Scaling of Accumulo

2023-03-20 Thread Dave Marion
/Elasticity [2] https://github.com/orgs/apache/projects/164 On Wed, Feb 22, 2023 at 2:35 PM Dave Marion wrote: > Except for the new bulk import code, Accumulo requires that tables are in > an online state to work with them (ingest, scan, compact, split, etc.). In > some cases this could be

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-16 Thread Dave Marion
> On Wed, Mar 15, 2023 at 2:55 PM Dave Marion wrote: > > > > It looks like we could generate the GH wiki from a folder in the source > > code. This would allow for issues and PRs. Just a thought. > > > > Ref: https://nimblehq.co/blog/create-github-wiki-pull-r

Re: slow ingest speeds

2023-03-15 Thread Dave Marion
; > does. > > > > We did already lose a node on the cluster a few days ago. I'm currently > > waiting for the system administrators to replace a disk. > > > > Thanks, > > > > On Wed, Mar 15, 2023 at 5:59 PM Dave Marion wrote: > > > >

Re: slow ingest speeds

2023-03-15 Thread Dave Marion
sounds like you have a hot-spot on those two datanode hosts. Either because the blocks that it's writing to are all (or a majority) located there, or there is some type of issue with the host. Stopping the DN processes on those two hosts should confirm this, unless the hot spot moves. Do you have

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Dave Marion
e really challenged at all. However, since I seem to be > the > > > only > > > > one advocating for using the website, to keep things centralized, as > > per > > > > previous attempts to consolidate documentation, I won't fight the use > > of > > >

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-15 Thread Dave Marion
Apache confluence issue is sorted out. > > Ed Coleman > > -----Original Message- > From: Dave Marion > Sent: Wednesday, March 8, 2023 11:09 AM > To: dev@accumulo.apache.org > Subject: Re: [DISCUSS] Enable Github wiki in asf.yaml? > > Looking at the Infra slack channe

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-08 Thread Dave Marion
Looking at the Infra slack channel response, one of the responses in the channel said that "it's some sort of db corruption according to Atlassian". Doesn't sound good On Wed, Mar 8, 2023 at 10:55 AM Dave Marion wrote: > https://issues.apache.org/jira/browse/INFRA-24291 is sti

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-08 Thread Dave Marion
tes for > now. > > -Original Message----- > From: Dave Marion > Sent: Friday, March 3, 2023 12:06 PM > To: dev@accumulo.apache.org > Subject: Re: [DISCUSS] Enable Github wiki in asf.yaml? > > Right, I was just curious if there was any follow-up as I think Ed said >

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-03 Thread Dave Marion
manually. On Fri, Mar 3, 2023 at 11:57 AM Christopher wrote: > You can track that issue at > https://issues.apache.org/jira/browse/INFRA-24291 > > On Fri, Mar 3, 2023 at 10:31 AM Dave Marion wrote: > > > > Ed, > > > > Any update from INFRA on being able to cre

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-03 Thread Dave Marion
opinion about the website being more accessible, via > our current GitHub PR/issue/Markdown workflows, and I wonder how many > other potential contributors would feel similarly. It's hard to know, > since it seems like a lot of this is subjective, and is going to come > down to a consens

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-02 Thread Dave Marion
wanted to participate they could. On Thu, Mar 2, 2023 at 2:53 PM Christopher wrote: > On Thu, Mar 2, 2023 at 1:34 PM Dave Marion wrote: > > > > I'm opposed to using the website for the reasons I specified earlier, so > it > > Your reasons that I saw were: > > >

Re: [DISCUSS] Enable Github wiki in asf.yaml?

2023-03-02 Thread Dave Marion
> > > location for our documentation. I don't even think infra is backing > up > > > > wiki repos, so there wouldn't even be a record of the wiki contents > in > > > > ASF spaces (vs. the main repo, which is backed up to GitBox and the > > > > issue t

Dynamic Scaling of Accumulo

2023-02-22 Thread Dave Marion
Except for the new bulk import code, Accumulo requires that tables are in an online state to work with them (ingest, scan, compact, split, etc.). In some cases this could become cost prohibitive and resource inefficient as resources necessary to keep the tables online might be unused. I'd like to

Re: jdk 17 and accumulo testing

2022-12-13 Thread Dave Marion
There is no guide. You are using implementation classes (see clientImpl in the package name) vs. using the client api. If you can use the client api directly, then this should insulate you from changes in the future (except during major versions). We can try and find where things might have moved,

Re: Fixing the 2.1 branch - a brief analysis

2022-12-01 Thread Dave Marion
Christopher, Thanks for cleaning up my mess, I appreciate your help here. Your git-foo is stronger than mine :-) Lesson learned regarding the GitHub UI merge between major branches. The GH UI in this case is lacking several important features - 1) a warning, 2) it didn't show in the UI all of

Re: Rollback to 1.9.3

2022-11-04 Thread Dave Marion
ing to identify whether or not rollback is even an option. > > - Logan > > On Fri, Nov 4, 2022 at 3:49 PM Dave Marion wrote: > > > Are you running into an error or some other issue that is making you > > think that you have to rollback? I don't know that rolling back

Re: Rollback to 1.9.3

2022-11-04 Thread Dave Marion
Are you running into an error or some other issue that is making you think that you have to rollback? I don't know that rolling back has been tested. On Fri, Nov 4, 2022 at 3:40 PM Logan Jones wrote: > Hello: > > We recently upgraded from Accumulo 1.9.3 to 1.10.2. Is it safe to roll back > to

Re: [DRAFT][ANNOUNCE] Apache Accumulo 2.1.0

2022-11-02 Thread Dave Marion
Christopher, Thanks for spear-heading the release. A few questions: 1. Given the number of changes and that not all of them are documented in the release notes, should we also suggest that users look at the user guide in addition to the release notes? 2. For the description at the bottom,

Re: [DRAFT][ANNOUNCE] Apache Accumulo 2.1.0

2022-11-02 Thread Dave Marion
I don't see it either. I know that it is documented in the user guide. I can certainly add a one-liner under "Other notable changes" if you like. On Wed, Nov 2, 2022 at 6:53 AM Keith Turner wrote: > In the release notes I do not see mention of the new cluster yaml file > that replaced the

Re: [VOTE] Apache Accumulo 2.1.0-rc4

2022-10-31 Thread Dave Marion
+1 (binding) Validated checksums of src and binary tarballs Created a 20 node cluster using: CentOS 7.9 ZooKeeper 3.8.0 Hadoop 3.3.4 Accumulo 2.1.0-RC4 Performed the following tests: 1. Created the ci table configured with 200 splits, without encryption, and using external

Re: Hadoop Metrics2 and JMX

2022-10-12 Thread Dave Marion
Looking at [1], specifically the overview section, I think they are the same metrics just accessible via JMX instead of configuring a sink. [1] https://hadoop.apache.org/docs/r3.3.4/api/org/apache/hadoop/metrics2/package-summary.html On Wed, Oct 12, 2022 at 1:39 PM Christopher wrote: > I don't

Re: forceably failable master node

2022-10-11 Thread Dave Marion
The secondary master is waiting to get the lock in ZooKeeper for it to become active. The only way to do that is to stop the current one. On Tue, Oct 11, 2022 at 10:38 AM Vincent Russell wrote: > Hello, > > We are running accumulo 2.0.1 with high availability? Is there a way to > force

Re: New committer / PMC member: Chris Shannon

2022-09-29 Thread Dave Marion
Congrats Chris! On Thu, Sep 29, 2022 at 9:26 AM Dominic Garguilo wrote: > Congrats and welcome, Chris! > > On Thu, Sep 29, 2022 at 9:15 AM dev1 wrote: > > > The Project Management Committee (PMC) for Apache Accumulo has invited > > Chris to become a committer and PMC member and we are pleased

Re: Re: Re: Re: Question about Accumulo Tracer

2022-07-29 Thread Dave Marion
't affect the tracer server. `-a` > should still work fine there. > > On Fri, Jul 29, 2022 at 3:00 PM Dave Marion wrote: > > > https://github.com/apache/accumulo/pull/2119 fixed a bug in 2.1 where > the > > -a argument was not correctly setting the hostname. It looks like

Re: Re: Re: Re: Question about Accumulo Tracer

2022-07-29 Thread Dave Marion
https://github.com/apache/accumulo/pull/2119 fixed a bug in 2.1 where the -a argument was not correctly setting the hostname. It looks like 2.0.1 is affected by this too. On Fri, Jul 29, 2022 at 12:57 PM Christopher wrote: > The tracer should be advertising its own address in ZK. By default,

Re: {DISCUSS} Accumulo Quarterly Report - due Wed 7/13

2022-07-08 Thread Dave Marion
Looks good. On Fri, Jul 8, 2022 at 10:08 AM dev1 wrote: > The Accumulo quarterly report is due Wednesday, July 13, 2022. . The > community decided to publicly prepare the report on the dev mailing list. > Below is the current draft. (note: This is a cut-n-paste from the report > wizard, so

Re: Behavior of Fates on Failed Compactions

2022-07-07 Thread Dave Marion
I think FaTE ensures that the transaction is started and it waits for it to finish. It must be the case that a failure is not being propagated back up to fail the transaction. Are you seeing FaTE restarting the same compaction over and over again, or are the multiple IN_PROGRESS transactions from

Re: Intro

2022-05-03 Thread Dave Marion
It has been done. On Tue, May 3, 2022 at 1:28 PM Christopher wrote: > The slack invite process changed. We have to explicitly invite people now. > I can do it. > > On Tue, May 3, 2022, 12:33 Dave Marion wrote: > > > See https://accumulo.apache.org/contact-us/#slack, there

Re: Intro

2022-05-03 Thread Dave Marion
> in depth. But we're confident that given what Accumulo is capable of, > > there > > > will be good content for us to share with the community in the future. > We > > > will continue to keep our eyes out for how we can engage, and excited > to > > do > > &

Re: Intro

2022-04-27 Thread Dave Marion
Welcome! And +1 for hearing more about how you are using Accumulo. I visited the Ghost's website earlier this morning - awesome stuff. On Wed, Apr 27, 2022 at 3:59 PM Christopher wrote: > Hi Nikita! > > Welcome to our community. I'm curious to hear more about how Ghost is > using Accumulo. Have

Re: Major compactions during map reduce

2022-04-19 Thread Dave Marion
using the new MapReduce API, but we are not setting any > > settings for isolated scan so we are using whatever the default is. > > > > Thanks, > > Vincent > > > > On Mon, Apr 18, 2022 at 3:12 PM Dave Marion wrote: > > > > > Major compactions s

Re: Major compactions during map reduce

2022-04-18 Thread Dave Marion
Major compactions should not move rows to new tablets, but a tablet split could. Are you using the new MapReduce API introduced in 2.0? Are you setting it to use an isolated scan? On Mon, Apr 18, 2022 at 3:01 PM Vincent Russell wrote: > Hello All, > > Could major compactions that occur while a

Re: [DISCUSS] Draft Accumulo quarterly report - due Wednesday 4/13.

2022-04-08 Thread Dave Marion
LGTM. On Fri, Apr 8, 2022 at 12:57 PM dev1 wrote: > The Accumulo quarterly report is due Wednesday, April 13, 2022. . The > community decided to publicly prepare the report on the dev mailing list. > Below is the current draft. > > (note: This is a cut-n-paster from the report wizard, so

Re: Scan Server discussion [WAS: Re: 2.1 Release TODO]

2022-04-04 Thread Dave Marion
I understand the desire to see less coupling for the optional features, but getting to that point for ScanServers (and less so for ExternalCompactions) would be a ton of work I think. The concern that I brought up in the "2.1 Release TODOs" thread regarding planning has not been addressed. If

Re: 2.1 Release TODO

2022-04-04 Thread Dave Marion
mulo/pull/2574 > [3] https://github.com/apache/accumulo/issues/2406 > [4] https://github.com/apache/accumulo/pull/2215 > > On Fri, Apr 1, 2022 at 2:21 PM Dave Marion wrote: > > > I think it would be useful to do some release planning so that we know > what > > features we

Re: 2.1 Release TODO

2022-04-01 Thread Dave Marion
I think it would be useful to do some release planning so that we know what features we are working towards and in which release they will be in. This would be helpful for determining what existing PRs need to make it into 2.1.0. 2.1.0 is the LTM release, so patches for existing features will be

Re: [DISCUSS] 1.10.2 release with reload4j

2022-02-03 Thread Dave Marion
I'd like to try and include https://github.com/apache/accumulo/pull/2221. A little more testing needs to be done, do you have a schedule for the 1.10.2 release? On Thu, Feb 3, 2022 at 1:55 PM Christopher wrote: > I'm interested in putting together a 1.10.2 release with the changes in >

Re: Accumulo quarterly report. Due 1/12/2022

2022-01-10 Thread Dave Marion
Looks good. On Mon, Jan 10, 2022 at 9:30 AM dev1 wrote: > The Accumulo community quarterly report for January is due Wednesday > 1/12/2022. The community decided to publicly prepare the report on the dev > mailing list. Below is the current draft. This is a simple cut'n'paste > from the report

Re: Accumulo 2.0.1 init with hdfs running with SSL

2021-12-15 Thread Dave Marion
it. That interface is not in the > > public API and should avoid being used. > > > > On Wed, Dec 15, 2021 at 10:45 AM Dave Marion > wrote: > > > > > Is that datanode configured correctly? I wonder why it's excluded. > > > > > > On Wed

Re: Accumulo 2.0.1 init with hdfs running with SSL

2021-12-15 Thread Dave Marion
Is that datanode configured correctly? I wonder why it's excluded. On Wed, Dec 15, 2021 at 9:45 AM Vincent Russell wrote: > Thank you Christopher, > > I was able to determine that the ssl settings in core-site.xml are being > picked up and used. In fact when accumulo init is run, accumulo is

Re: [DISCUSS] The current state of replication and the way forward?

2021-10-29 Thread Dave Marion
https://github.com/apache/accumulo/pull/2335 has been created to deprecate the replication classes, properties, etc. and fix-up the references to them. On Wed, Oct 20, 2021 at 1:29 AM Christopher wrote: > For reference, our last conversation about the state of replication > was >

Re: 1.10 <-> 2.0 shim

2021-10-22 Thread Dave Marion
I created a simple maven pom file that when run with *mvn clean package* will generate a report that shows the differences in the public API between 2.0.0 and 1.10.0. https://gist.github.com/dlmarion/b1063c334d519f637cc78d81ba9e15ef On Wed, Oct 20, 2021 at 6:20 PM Jeremy Kepner wrote: > Seeme

Re: [DISCUSS] Version number of next release?

2021-10-21 Thread Dave Marion
I'd like to make the case for staying with 2.1. My main motivation for this is the slow speed at which users upgrade and the perceived risk of the number 3.0 (vs 2.1). I think users would see "3.0" and think that it would require a lot more work to upgrade to it than "2.1". Moving to 3.0 has a

Re: Metrics Replacement

2021-10-04 Thread Dave Marion
erent approach, where we > continue to use Hadoop Metrics2 internally and attempt to write a > Micrometer sink for the Metrics2 framework for 2.x and move to Micrometer > for the next major release. Based on the Hadoop JIRA, it does not appear > that they have plans to move away from thi

Re: Accumulo quarterly report. Due 10/13/2021

2021-09-28 Thread Dave Marion
Typo in last line, "Jira to Gibhub" On Tue, Sep 28, 2021 at 8:10 AM dev1 wrote: > The Accumulo community quarterly report for October is due Wednesday > 10/13/2021. The community decided to publicly prepare the report on the > dev mailing list. Below is the current draft. > > Ed Coleman > >

Re: Metrics Replacement

2021-09-27 Thread Dave Marion
the target system - but those > should be documented in each module and is not within our scope. > > > > -----Original Message- > > From: Keith Turner > > Sent: Tuesday, September 21, 2021 5:07 PM > > To: Accumulo Dev List > > Subject: Re: Metrics Replacemen

Metrics Replacement

2021-09-21 Thread Dave Marion
There is a WIP pull request against 2.1.0-SNAPSHOT for replacing the Hadoop Metrics2 framework with Micrometer[1]. Micrometer suggests using a naming pattern[2] for the metrics internally where words are all lowercase separated by a period. Micrometer output formats then rewrite the metric names

Re: new committer: Dominic Garguilo

2021-07-29 Thread Dave Marion
Congrats! On Thu, Jul 29, 2021 at 2:34 PM Harjit Singh wrote: > Welcome !!! > > > On Jul 29, 2021, at 2:08 PM, Gall, Deeanna wrote: > > > > Congrats!!! > > > > -Original Message- > > From: Faulkner, Tanisha A > > Sent: Thursday, July 29, 2021 1:45 PM > > To: dev@accumulo.apache.org >

Re: Accumulo 1.10.1 performance

2021-06-07 Thread Dave Marion
megabytes. I also just tested on some older hardware that is > > closer to what was used in the 2014 paper, and the single process ingest > > rate is ~8x slower. > > > > Has anyone done any recent benchmarking of Accumulo 1.10+? > > > > Regards. -Jeremy >

Re: Accumulo 1.10.1 performance

2021-06-05 Thread Dave Marion
Jeremy, Are you able to share any details about the hardware and the Accumulo configuration? Is the Accumulo/Hadoop configuration the same as the prior test (no replication, WAL turned off, batch writer configuration, etc.) Dave On Sat, Jun 5, 2021 at 6:12 PM Kepner, Jeremy - LLSC - MITLL <

External Compactions

2021-05-11 Thread Dave Marion
Keith and I have been working on a solution for issue #1451 - being able to run major compactions outside the tablet server. This would enable compactions to run when tables are offline, tablet servers die, and tablets are balancing. We have created two pull requests, one for the code[1] changes

Re: Which String deduplication option?

2021-02-08 Thread Dave Marion
String.intern() would seem to provide better coverage considering that some users may not use the G1 collector. On Mon, Feb 8, 2021 at 3:19 PM Keith Turner wrote: > Recently while running some large map reduce jobs I learned that > Hadoop uses String.intern() in its RPC code (below is a link to

  1   2   >