Accumulo-1.8 - Build # 258 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Accumulo-1.8 (build #258)

Status: Still Failing

Check console output at https://builds.apache.org/job/Accumulo-1.8/258/ to view 
the results.

[jira] [Updated] (ACCUMULO-4803) Remove automatic KEK creation if SecretKeyEncryptionStrategies cannot find KEK file

[Nick Felts (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Felts updated ACCUMULO-4803:
-
Fix Version/s: 2.0.0

> Remove automatic KEK creation if SecretKeyEncryptionStrategies cannot find 
> KEK file
> ---
>
> Key: ACCUMULO-4803
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4803
> Project: Accumulo
>  Issue Type: Improvement
>Reporter: Nick Felts
>Assignee: Nick Felts
>Priority: Minor
> Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ACCUMULO-4803) Remove automatic KEK creation if SecretKeyEncryptionStrategies cannot find KEK file

[Nick Felts (JIRA)]

Nick Felts created ACCUMULO-4803:


 Summary: Remove automatic KEK creation if 
SecretKeyEncryptionStrategies cannot find KEK file
 Key: ACCUMULO-4803
 URL: https://issues.apache.org/jira/browse/ACCUMULO-4803
 Project: Accumulo
  Issue Type: Improvement
Reporter: Nick Felts
Assignee: Nick Felts






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Accumulo-Master - Build # 2252 - Fixed

[Apache Jenkins Server]

The Apache Jenkins build system has built Accumulo-Master (build #2252)

Status: Fixed

Check console output at https://builds.apache.org/job/Accumulo-Master/2252/ to 
view the results.

[GitHub] keith-turner closed pull request #374: ACCUMULO-4790 Avoid creating FileRef

[GitBox]

keith-turner closed pull request #374: ACCUMULO-4790 Avoid creating FileRef
URL: https://github.com/apache/accumulo/pull/374
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/server/tserver/src/main/java/org/apache/accumulo/tserver/FileManager.java 
b/server/tserver/src/main/java/org/apache/accumulo/tserver/FileManager.java
index 8bc0fedc0a..a707251463 100644
--- a/server/tserver/src/main/java/org/apache/accumulo/tserver/FileManager.java
+++ b/server/tserver/src/main/java/org/apache/accumulo/tserver/FileManager.java
@@ -510,6 +510,14 @@ public void setInterruptFlag(AtomicBoolean flag) {
 
   ArrayList iters = new ArrayList<>();
 
+  boolean sawTimeSet = false;
+  for (DataFileValue dfv : files.values()) {
+if (dfv.isTimeSet()) {
+  sawTimeSet = true;
+  break;
+}
+  }
+
   for (FileSKVIterator reader : newlyReservedReaders) {
 String filename = getReservedReadeFilename(reader);
 InterruptibleIterator iter;
@@ -530,9 +538,13 @@ public void setInterruptFlag(AtomicBoolean flag) {
 } else {
   iter = new ProblemReportingIterator(context, tablet.getTableId(), 
filename, continueOnFailure, source);
 }
-DataFileValue value = files.get(new FileRef(filename));
-if (value.isTimeSet()) {
-  iter = new TimeSettingIterator(iter, value.getTime());
+
+if (sawTimeSet) {
+  // constucting FileRef is expensive so avoid if not needed
+  DataFileValue value = files.get(new FileRef(filename));
+  if (value.isTimeSet()) {
+iter = new TimeSettingIterator(iter, value.getTime());
+  }
 }
 
 iters.add(iter);


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Resolved] (ACCUMULO-4772) Update Accumulo shell to utilize new NewTableConfiguration methods

[Mark Owens (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Owens resolved ACCUMULO-4772.
--
Resolution: Resolved

> Update Accumulo shell to utilize new NewTableConfiguration methods
> --
>
> Key: ACCUMULO-4772
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4772
> Project: Accumulo
>  Issue Type: Improvement
>  Components: shell
>Reporter: Mark Owens
>Assignee: Mark Owens
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 2.0.0
>
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> ACCUMULO-4732 adds the capability for NewTableConfiguration to preconfigure 
> iterators and locality groups prior to table creation. Update the Accumulo 
> shell to allow the same capability. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create builder for Connector

[GitBox]

keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create 
builder for Connector
URL: https://github.com/apache/accumulo/pull/361#discussion_r166730012
 
 

 ##
 File path: 
core/src/main/java/org/apache/accumulo/core/conf/ClientConfigGenerate.java
 ##
 @@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.conf;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.FileNotFoundException;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.util.Objects;
+import java.util.Set;
+import java.util.TreeMap;
+
+import com.google.common.collect.Sets;
+
+/**
+ * Generates client-properties.md for documentation on Accumulo website and 
accumulo-client.properties for Accumulo distribution tarball
+ */
+class ClientConfigGenerate {
+
+  private abstract class Format {
+
+abstract void beginSection(String section);
+
+abstract void pageHeader();
+
+abstract void property(ClientProperty prop);
+
+void generate() {
+  pageHeader();
+
+  generateSection("Instance", "instance.");
+  generateSection("Authentication", "auth.", "auth.method", 
"auth.username");
+  generateSection("Batch Writer", "batch.writer.");
+  generateSection("SSL", "ssl.");
+  generateSection("SASL", "sasl.");
+  generateSection("Tracing", "trace.");
+
+  doc.close();
+}
+
+void generateSection(String section, String prefix, String... prefixProps) 
{
+  beginSection(section);
+  for (String prop : prefixProps) {
+ClientProperty cp = sortedProps.get(prop);
+if (cp != null) {
+  property(cp);
+}
+  }
+  Set prefixSet = Sets.newHashSet(prefixProps);
+  for (ClientProperty prop : sortedProps.values()) {
+if (prop.getKey().startsWith(prefix) && 
!prefixSet.contains(prop.getKey())) {
+  property(prop);
+}
+  }
+}
+
+void generateSection(String section, String prefix) {
+  generateSection(section, prefix, "");
+}
+  }
+
+  private class Markdown extends Format {
+
+@Override
+void beginSection(String section) {}
+
+@Override
+void pageHeader() {
+  doc.println("---");
+  doc.println("title: Client Properties");
+  doc.println("category: development");
+  doc.println("order: 9");
+  doc.println("---\n");
+  doc.println("\n");
 
 Review comment:
   This whole automated doc generation is nice.  Could also add another comment 
like.
   
   ```java
doc.println("");
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create builder for Connector

[GitBox]

keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create 
builder for Connector
URL: https://github.com/apache/accumulo/pull/361#discussion_r166725393
 
 

 ##
 File path: 
core/src/main/java/org/apache/accumulo/core/client/ConnectionInfo.java
 ##
 @@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.client;
+
+import java.util.Properties;
+
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+
+/**
+ * Accumulo client connection information. Can be built using {@link 
Connector#builder()}
+ */
+public interface ConnectionInfo {
 
 Review comment:
   needs a since tag


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create builder for Connector

[GitBox]

keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create 
builder for Connector
URL: https://github.com/apache/accumulo/pull/361#discussion_r166726135
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/client/Connector.java
 ##
 @@ -128,6 +151,19 @@ public abstract BatchDeleter createBatchDeleter(String 
tableName, Authorizations
 
   public abstract BatchWriter createBatchWriter(String tableName, 
BatchWriterConfig config) throws TableNotFoundException;
 
 Review comment:
   Could document the new merge behavior in javadocs.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create builder for Connector

[GitBox]

keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create 
builder for Connector
URL: https://github.com/apache/accumulo/pull/361#discussion_r166736816
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/client/Connector.java
 ##
 @@ -237,4 +282,271 @@ public abstract BatchDeleter createBatchDeleter(String 
tableName, Authorizations
* @since 1.7.0
*/
   public abstract ReplicationOperations replicationOperations();
+
+  /**
+   * Builds ConnectionInfo after all options have been specified
+   *
+   * @since 2.0.0
+   */
+  public interface ConnInfoFactory {
+
+/**
+ * Builds ConnectionInfo after all options have been specified
+ *
+ * @return ConnectionInfo
+ */
+ConnectionInfo info();
+  }
+
+  /**
+   * Builds Connector
+   *
+   * @since 2.0.0
+   */
+  public interface ConnectorFactory extends ConnInfoFactory {
+
+/**
+ * Builds Connector after all options have been specified
+ *
+ * @return Connector
+ */
+Connector build() throws AccumuloException, AccumuloSecurityException;
+
+  }
+
+  /**
+   * Builder method for setting Accumulo instance and zookeepers
+   *
+   * @since 2.0.0
+   */
+  public interface InstanceArgs {
+AuthenticationArgs forInstance(String instanceName, String zookeepers);
+  }
+
+  /**
+   * Builder methods for creating Connector using properties
+   *
+   * @since 2.0.0
+   */
+  public interface PropertyOptions extends InstanceArgs {
+
+/**
+ * Build using properties file
+ *
+ * @param propertiesFile
+ *  Path to properties file
+ * @return this builder
+ */
+ConnectorFactory usingProperties(String propertiesFile);
 
 Review comment:
   Some APIs will take a File (or in java 8 a Path) type for a method like 
this.  I find those more cumbersome to use (I seem to always have a string when 
going use something like this). From a design perspective I wonder if there is 
any advantage to this.  Seems like its only better when you already have a File 
object in that you get better type checking.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create builder for Connector

[GitBox]

keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create 
builder for Connector
URL: https://github.com/apache/accumulo/pull/361#discussion_r166735759
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/client/Connector.java
 ##
 @@ -237,4 +282,271 @@ public abstract BatchDeleter createBatchDeleter(String 
tableName, Authorizations
* @since 1.7.0
*/
   public abstract ReplicationOperations replicationOperations();
+
+  /**
+   * Builds ConnectionInfo after all options have been specified
+   *
+   * @since 2.0.0
+   */
+  public interface ConnInfoFactory {
+
+/**
+ * Builds ConnectionInfo after all options have been specified
+ *
+ * @return ConnectionInfo
+ */
+ConnectionInfo info();
+  }
+
+  /**
+   * Builds Connector
+   *
+   * @since 2.0.0
+   */
+  public interface ConnectorFactory extends ConnInfoFactory {
+
+/**
+ * Builds Connector after all options have been specified
+ *
+ * @return Connector
+ */
+Connector build() throws AccumuloException, AccumuloSecurityException;
+
+  }
+
+  /**
+   * Builder method for setting Accumulo instance and zookeepers
+   *
+   * @since 2.0.0
+   */
+  public interface InstanceArgs {
+AuthenticationArgs forInstance(String instanceName, String zookeepers);
+  }
+
+  /**
+   * Builder methods for creating Connector using properties
+   *
+   * @since 2.0.0
+   */
+  public interface PropertyOptions extends InstanceArgs {
+
+/**
+ * Build using properties file
 
 Review comment:
   Would be nice to point user to docs for these.  Maybe a stable path on the 
website?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create builder for Connector

[GitBox]

keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create 
builder for Connector
URL: https://github.com/apache/accumulo/pull/361#discussion_r166725291
 
 

 ##
 File path: 
core/src/main/java/org/apache/accumulo/core/client/BatchWriterConfig.java
 ##
 @@ -309,6 +311,26 @@ public boolean equals(Object o) {
 return false;
   }
 
+  private static  T merge(T o1, T o2) {
+if (o1 != null)
+  return o1;
+return o2;
+  }
+
+  public BatchWriterConfig merge(BatchWriterConfig other) {
 
 Review comment:
   This needs javadoc describing what it does and what it returns.  Also needs 
a since tag.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create builder for Connector

[GitBox]

keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create 
builder for Connector
URL: https://github.com/apache/accumulo/pull/361#discussion_r166725644
 
 

 ##
 File path: 
core/src/main/java/org/apache/accumulo/core/client/ConnectionInfo.java
 ##
 @@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.client;
+
+import java.util.Properties;
+
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+
+/**
+ * Accumulo client connection information. Can be built using {@link 
Connector#builder()}
+ */
+public interface ConnectionInfo {
+
+  /**
+   * @return Accumulo instance name
+   */
+  String getInstanceName();
+
+  /**
+   * @return Zookeeper connection information for Accumulo instance
+   */
+  String getZookeepers();
+
+  /**
+   * @return Accumulo principal/username
+   */
+  String getPrincipal();
+
+  /**
+   * @return {@link AuthenticationToken} used for this connection
+   */
+  AuthenticationToken getAuthenticationToken();
+
+  /**
+   * Retrieves property value given key
+   *
+   * @param key
+   *  Property key
+   * @return Value of property
 
 Review comment:
   Should describe behavior if not present.  Why have this method?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create builder for Connector

[GitBox]

keith-turner commented on a change in pull request #361: ACCUMULO-4784 - Create 
builder for Connector
URL: https://github.com/apache/accumulo/pull/361#discussion_r166737783
 
 

 ##
 File path: 
core/src/main/java/org/apache/accumulo/core/conf/ClientConfigGenerate.java
 ##
 @@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.conf;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.FileNotFoundException;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.util.Objects;
+import java.util.Set;
+import java.util.TreeMap;
+
+import com.google.common.collect.Sets;
+
+/**
+ * Generates client-properties.md for documentation on Accumulo website and 
accumulo-client.properties for Accumulo distribution tarball
+ */
+class ClientConfigGenerate {
+
+  private abstract class Format {
+
+abstract void beginSection(String section);
+
+abstract void pageHeader();
+
+abstract void property(ClientProperty prop);
+
+void generate() {
+  pageHeader();
+
+  generateSection("Instance", "instance.");
+  generateSection("Authentication", "auth.", "auth.method", 
"auth.username");
+  generateSection("Batch Writer", "batch.writer.");
+  generateSection("SSL", "ssl.");
+  generateSection("SASL", "sasl.");
+  generateSection("Tracing", "trace.");
+
+  doc.close();
+}
+
+void generateSection(String section, String prefix, String... prefixProps) 
{
+  beginSection(section);
+  for (String prop : prefixProps) {
+ClientProperty cp = sortedProps.get(prop);
+if (cp != null) {
+  property(cp);
+}
+  }
+  Set prefixSet = Sets.newHashSet(prefixProps);
+  for (ClientProperty prop : sortedProps.values()) {
+if (prop.getKey().startsWith(prefix) && 
!prefixSet.contains(prop.getKey())) {
+  property(prop);
+}
+  }
+}
+
+void generateSection(String section, String prefix) {
+  generateSection(section, prefix, "");
+}
+  }
+
+  private class Markdown extends Format {
+
+@Override
+void beginSection(String section) {}
+
+@Override
+void pageHeader() {
+  doc.println("---");
+  doc.println("title: Client Properties");
+  doc.println("category: development");
+  doc.println("order: 9");
+  doc.println("---\n");
+  doc.println("\n");
+  doc.println("Below are properties set in `accumulo-client.properties` 
that configure Accumulo clients:\n");
+  doc.println("| Property | Default value | Description |");
 
 Review comment:
   A since column would be nice.  Maybe not now, but in the future.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Accumulo-Pull-Requests - Build # 1025 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Accumulo-Pull-Requests (build #1025)

Status: Still Failing

Check console output at 
https://builds.apache.org/job/Accumulo-Pull-Requests/1025/ to view the results.

[jira] [Updated] (ACCUMULO-4800) Preparse iterator configuration

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ACCUMULO-4800:
-
Labels: pull-request-available  (was: )

> Preparse iterator configuration
> ---
>
> Key: ACCUMULO-4800
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4800
> Project: Accumulo
>  Issue Type: Improvement
>Affects Versions: 1.7.3, 1.8.1
>Reporter: Keith Turner
>Assignee: Keith Turner
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.9.0, 2.0.0
>
>
> I am noticing that for small scans a good bit of time is spent parsing 
> iterator config.  It would be nice to pre-parse iterator config and only 
> reparse when table config changes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] keith-turner opened a new pull request #378: ACCUMULO-4800 Cache parsing of iterator config

[GitBox]

keith-turner opened a new pull request #378: ACCUMULO-4800 Cache parsing of 
iterator config
URL: https://github.com/apache/accumulo/pull/378
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (ACCUMULO-4802) Add kill command to accumulo-cluster script

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated ACCUMULO-4802:
-
Labels: pull-request-available  (was: )

> Add kill command to accumulo-cluster script
> ---
>
> Key: ACCUMULO-4802
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4802
> Project: Accumulo
>  Issue Type: Improvement
>  Components: scripts
>Reporter: Mike Walch
>Assignee: Mike Walch
>Priority: Major
>  Labels: pull-request-available
>
> Would helpful to be able to kill cluster quickly



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] mikewalch opened a new pull request #377: ACCUMULO-4802 Added kill command to accumulo-cluster script

[GitBox]

mikewalch opened a new pull request #377: ACCUMULO-4802 Added kill command to 
accumulo-cluster script
URL: https://github.com/apache/accumulo/pull/377
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #373: ACCUMULO-4709 sanity check in Mutation

[GitBox]

keith-turner commented on a change in pull request #373: ACCUMULO-4709 sanity 
check in Mutation
URL: https://github.com/apache/accumulo/pull/373#discussion_r166659412
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/data/Mutation.java
 ##
 @@ -306,6 +340,9 @@ private void put(byte[] cf, int cfLength, byte[] cq, int 
cqLength, byte[] cv, bo
 if (buffer == null) {
 
 Review comment:
   After a mutation is serialized, put methods can not be called.  Therefore I 
do not think the serialization needs to change to transfer the total length 
information.  Also modifying the serialization may have far reaching 
implications as mutations are serialized in write ahead logs and replication 
data.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] keith-turner commented on a change in pull request #373: ACCUMULO-4709 sanity check in Mutation

[GitBox]

keith-turner commented on a change in pull request #373: ACCUMULO-4709 sanity 
check in Mutation
URL: https://github.com/apache/accumulo/pull/373#discussion_r166650901
 
 

 ##
 File path: core/src/main/java/org/apache/accumulo/core/data/Mutation.java
 ##
 @@ -306,6 +340,9 @@ private void put(byte[] cf, int cfLength, byte[] cq, int 
cqLength, byte[] cv, bo
 if (buffer == null) {
   throw new IllegalStateException("Can not add to mutation after 
serializing it");
 }
+long length = cfLength + cqLength + valLength;
 
 Review comment:
   This does not capture the complete picture.  When data is serialized, the 
length is written in addition to the data.  For example if a 20 byte family is 
written it may actually write 21 bytes including the length.  The length is 
written as a variable length long, so its hard to know how many bytes it will 
take beforehand.  Because this hard to know beforehand, I would suggest just 
doing the worst case and assuming 5 bytes.  
   
   Also could take the strategy of only tracking the length of long values and 
asking the buffer for the size of everything else.  This way an accurate size 
for what is already serialized is obtained and we only need to estimated what 
is about to be serialized. 
   
   ```java
 long estimatedNewSize = (cfLength + cqLength + valLength + cv.lenght() + 
4* 5 + 1 + (hasTs ? 8 : 0)) + buffer.currentSize() + valueLenghts;
 //sanity check estimated new size
   
   // if large value, then increment valueLenghts
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Accumulo-Master - Build # 2251 - Still Failing

[Apache Jenkins Server]

The Apache Jenkins build system has built Accumulo-Master (build #2251)

Status: Still Failing

Check console output at https://builds.apache.org/job/Accumulo-Master/2251/ to 
view the results.

[GitHub] keith-turner closed pull request #370: ACCUMULO-4772 Update shell to use NewTableConfiguration methods

[GitBox]

keith-turner closed pull request #370: ACCUMULO-4772 Update shell to use 
NewTableConfiguration methods
URL: https://github.com/apache/accumulo/pull/370
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/shell/src/main/java/org/apache/accumulo/shell/commands/CreateTableCommand.java
 
b/shell/src/main/java/org/apache/accumulo/shell/commands/CreateTableCommand.java
index eac16fa73d..5a66de0c6f 100644
--- 
a/shell/src/main/java/org/apache/accumulo/shell/commands/CreateTableCommand.java
+++ 
b/shell/src/main/java/org/apache/accumulo/shell/commands/CreateTableCommand.java
@@ -17,7 +17,11 @@
 package org.apache.accumulo.shell.commands;
 
 import java.io.IOException;
+import java.util.Arrays;
+import java.util.EnumSet;
 import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
 import java.util.Map;
 import java.util.Map.Entry;
 import java.util.Set;
@@ -26,6 +30,7 @@
 
 import org.apache.accumulo.core.client.AccumuloException;
 import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.IteratorSetting;
 import org.apache.accumulo.core.client.TableExistsException;
 import org.apache.accumulo.core.client.TableNotFoundException;
 import org.apache.accumulo.core.client.admin.NewTableConfiguration;
@@ -34,6 +39,7 @@
 import org.apache.accumulo.core.conf.Property;
 import org.apache.accumulo.core.constraints.VisibilityConstraint;
 import org.apache.accumulo.core.iterators.IteratorUtil;
+import org.apache.accumulo.core.iterators.IteratorUtil.IteratorScope;
 import org.apache.accumulo.shell.Shell;
 import org.apache.accumulo.shell.Shell.Command;
 import org.apache.accumulo.shell.ShellUtil;
@@ -56,6 +62,8 @@
   private Option base64Opt;
   private Option createTableOptFormatter;
   private Option createTableOptInitProp;
+  private Option createTableOptLocalityProps;
+  private Option createTableOptIteratorProps;
 
   @Override
   public int execute(final String fullCommand, final CommandLine cl, final 
Shell shellState) throws AccumuloException, AccumuloSecurityException,
@@ -63,6 +71,7 @@ public int execute(final String fullCommand, final 
CommandLine cl, final Shell s
 
 final String testTableName = cl.getArgs()[0];
 final HashMap props = new HashMap<>();
+NewTableConfiguration ntc = new NewTableConfiguration();
 
 if (!testTableName.matches(Tables.VALID_NAME_REGEX)) {
   shellState.getReader().println("Only letters, numbers and underscores 
are allowed for use in table names.");
@@ -106,8 +115,18 @@ public int execute(final String fullCommand, final 
CommandLine cl, final Shell s
   }
 }
 
+// Set iterator if supplied
+if (cl.hasOption(createTableOptIteratorProps.getOpt())) {
+  ntc = attachIteratorToNewTable(cl, shellState, ntc);
+}
+
+// Set up locality groups, if supplied
+if (cl.hasOption(createTableOptLocalityProps.getOpt())) {
+  ntc = setLocalityForNewTable(cl, ntc);
+}
+
 // create table
-shellState.getConnector().tableOperations().create(tableName, new 
NewTableConfiguration().setTimeType(timeType).setProperties(props));
+shellState.getConnector().tableOperations().create(tableName, 
ntc.setTimeType(timeType).setProperties(props));
 if (partitions.size() > 0) {
   shellState.getConnector().tableOperations().addSplits(tableName, 
partitions);
 }
@@ -150,9 +169,100 @@ public int execute(final String fullCommand, final 
CommandLine cl, final Shell s
 return 0;
   }
 
+  /**
+   * Add supplied locality groups information to a NewTableConfiguration 
object.
+   *
+   * Used in conjunction with createtable shell command to allow locality 
groups to be configured upon table creation.
+   */
+  private NewTableConfiguration setLocalityForNewTable(CommandLine cl, 
NewTableConfiguration ntc) {
+HashMap localityGroupMap = new HashMap<>();
+String[] options = 
cl.getOptionValues(createTableOptLocalityProps.getOpt());
+for (String localityInfo : options) {
+  final String parts[] = localityInfo.split("=", 2);
+  if (parts.length < 2)
+throw new IllegalArgumentException("Missing '=' or there are spaces 
between entries");
+  final String groupName = parts[0];
+  final HashSet colFams = new HashSet<>();
+  for (String family : parts[1].split(","))
+colFams.add(new Text(family.getBytes(Shell.CHARSET)));
+  // check that group names are not duplicated on usage line
+  if (localityGroupMap.put(groupName, colFams) != null)
+throw new IllegalArgumentException("Duplicate locality group name 
found. Group names must be unique");
+}
+ntc.setLocalityGroups(localityGroupMap);
+return ntc;
+  }
+
+  /**
+   * Add supplied iterator information to