Accumulo-Integration-Tests - Build # 543 - Still unstable! -- 1.6

[elserj]

Accumulo-Integration-Tests - Build # 543 - Still unstable:

Check console output at 
https://secure.penguinsinabox.com/jenkins/job/Accumulo-Integration-Tests/543/ 
to view the results.

[jira] [Updated] (ACCUMULO-4063) TServer should verify its lock is held before answering any RPC

[Josh Elser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ACCUMULO-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated ACCUMULO-4063:
-
Fix Version/s: 1.8.0
   1.7.2

> TServer should verify its lock is held before answering any RPC
> ---
>
> Key: ACCUMULO-4063
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4063
> Project: Accumulo
>  Issue Type: Bug
>  Components: rpc, tserver
>Affects Versions: 1.7.0
>Reporter: Josh Elser
> Fix For: 1.7.2, 1.8.0
>
>
> Noticing the following:
> {noformat}
> 2015-11-19 22:42:47,130 [thrift.ProcessFunction] ERROR: Internal error 
> processing flush
> org.apache.thrift.TException: Lock not acquired
> at 
> org.apache.accumulo.server.rpc.RpcWrapper$1.invoke(RpcWrapper.java:51)
> at com.sun.proxy.$Proxy21.flush(Unknown Source)
> at 
> org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$flush.getResult(TabletClientService.java:2671)
> at 
> org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$flush.getResult(TabletClientService.java:2657)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at 
> org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:63)
> at 
> org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:516)
> at 
> org.apache.accumulo.server.rpc.CustomNonBlockingServer$1.run(CustomNonBlockingServer.java:78)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at 
> org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
> at java.lang.Thread.run(Thread.java:745)
> 2015-11-19 22:42:47,402 [tserver.TabletServer$ThriftClientHandler] ERROR: 
> org.apache.accumulo.tserver.session.MultiScanSession cannot be cast to 
> org.apache.accumulo.tserver.session.ScanSession
> java.lang.ClassCastException: 
> org.apache.accumulo.tserver.session.MultiScanSession cannot be cast to 
> org.apache.accumulo.tserver.session.ScanSession
> at 
> org.apache.accumulo.tserver.TabletServer$ThriftClientHandler.closeScan(TabletServer.java:574)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at 
> org.apache.accumulo.core.trace.wrappers.RpcServerInvocationHandler.invoke(RpcServerInvocationHandler.java:46)
> at 
> org.apache.accumulo.server.rpc.RpcWrapper$1.invoke(RpcWrapper.java:47)
> at com.sun.proxy.$Proxy21.closeScan(Unknown Source)
> at 
> org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$closeScan.getResult(TabletClientService.java:2309)
> at 
> org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$closeScan.getResult(TabletClientService.java:2295)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at 
> org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:63)
> at 
> org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:516)
> at 
> org.apache.accumulo.server.rpc.CustomNonBlockingServer$1.run(CustomNonBlockingServer.java:78)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at 
> org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> a {{flush}} rpc came in and was denied, but then a {{closeScan}} came in and 
> threw an unrelated error (ignore the classcast for now).
> Are there _any_ RPCs that a tabletserver should answer before it holds its 
> lock?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ACCUMULO-2493) BinaryFormatter needs to be refactored

[Matt Dailey (JIRA)]

[ 
https://issues.apache.org/jira/browse/ACCUMULO-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15080707#comment-15080707
 ] 

Matt Dailey commented on ACCUMULO-2493:
---

I liked the Supplier interface idea, so I refactored the class to be called 
{{DateFormatSupplier extends ... implements Supplier}}.

The FormatterEnv change I was on the fence about, and ended up not including it 
in my pull request.  I like the idea of extra power for the Formatter via 
context, but adding a dependency to a Connector ties Formatters to only be used 
in that context.  Formatters wouldn't be usable to just format entries, like in 
a MapReduce context where there is no active Connector.

> BinaryFormatter needs to be refactored
> --
>
> Key: ACCUMULO-2493
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2493
> Project: Accumulo
>  Issue Type: Bug
>  Components: client
>Reporter: Mike Drob
>Assignee: Matt Dailey
>  Labels: newbie
> Fix For: 1.7.1, 1.8.0
>
>
> BinaryFormatter is currently used in a couple places in the shell, but the 
> code is hard to read and understand. There is a static getlength, which is 
> actually a setter, and all the instance calls end up going through 
> unnecessary static methods.
> This combination makes it hard to reuse BinaryFormatter objects, or even use 
> multiple, since the static state is likely to conflict.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ACCUMULO-3509) Scanner lock cause Tablet lock, hence preventing idle scans from being swept, hence blocking SimpleTimer thread

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/ACCUMULO-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15080690#comment-15080690
 ] 

Josh Elser commented on ACCUMULO-3509:
--

[~phrocker] is this relevant for 1.6.5 and 1.7.1 too?

> Scanner lock cause Tablet lock, hence preventing idle scans from being swept, 
> hence blocking SimpleTimer thread 
> 
>
> Key: ACCUMULO-3509
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3509
> Project: Accumulo
>  Issue Type: Bug
>  Components: tserver
>Affects Versions: 1.6.0
>Reporter: marco polo
>Assignee: marco polo
> Fix For: 1.8.0
>
>
> Synchronization with Tablet$Scanner via a read() will block close() being 
> called via the sweep method in TabletServer. As a result, the SimpleTimer 
> thread does not continue, and idle threads grow until the scan completes. 
> My patch, which is forthcoming, converts synchronized methods to use a fair 
> lock. If the lock is held by a read call, the close call will attempt to 
> obtain it, time out, and return indicating a close was not successful. The 
> sweep will continue, and the SimpleTimer thread will respawn later, 
> attempting closure on those Tablets at a later time. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ACCUMULO-2493) BinaryFormatter needs to be refactored

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ACCUMULO-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15080703#comment-15080703
 ] 

ASF GitHub Bot commented on ACCUMULO-2493:
--

GitHub user matthew-dailey opened a pull request:

https://github.com/apache/accumulo/pull/61

ACCUMULO-2493: Deprecated BinaryFormatter in favor of DefaultFormatter

with FormatterConfig.

* New class FormatterConfig handles configuration of Formatter objects
* FormatterConfig is taken by Formatter.initialize, which broke the
interface for many classes throughout the non-public API
* Added DateFormatSupplier to let Formatters use DateFormat in a
Thread-safe way
* Removed code from ScanCommand and Shell tied to BinaryFormatter to use
properly configured DefaultFormatter instead
* Fixed bug where `scan -f [num] -fm [class]` would ignore Formatter
class used in `-fm` and be overridden with BinaryFormatter

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/matthew-dailey/accumulo 
ACCUMULO-2493-1.7-squashed

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/accumulo/pull/61.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #61


commit 8f60b4b1cb87f6bc315479d240e9e17b212a16a9
Author: Matt Dailey 
Date:   2015-12-05T02:50:58Z

ACCUMULO-2493: Deprecated BinaryFormatter in favor of DefaultFormatter
with FormatterConfig.

* New class FormatterConfig handles configuration of Formatter objects
* FormatterConfig is taken by Formatter.initialize, which broke the
interface for many classes throughout the non-public API
* Added DateFormatSupplier to let Formatters use DateFormat in a
Thread-safe way
* Removed code from ScanCommand and Shell tied to BinaryFormatter to use
properly configured DefaultFormatter instead
* Fixed bug where `scan -f [num] -fm [class]` would ignore Formatter
class used in `-fm` and be overridden with BinaryFormatter




> BinaryFormatter needs to be refactored
> --
>
> Key: ACCUMULO-2493
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2493
> Project: Accumulo
>  Issue Type: Bug
>  Components: client
>Reporter: Mike Drob
>Assignee: Matt Dailey
>  Labels: newbie
> Fix For: 1.7.1, 1.8.0
>
>
> BinaryFormatter is currently used in a couple places in the shell, but the 
> code is hard to read and understand. There is a static getlength, which is 
> actually a setter, and all the instance calls end up going through 
> unnecessary static methods.
> This combination makes it hard to reuse BinaryFormatter objects, or even use 
> multiple, since the static state is likely to conflict.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Accumulo-Integration-Tests - Build # 542 - Aborted! -- master

[elserj]

Accumulo-Integration-Tests - Build # 542 - Aborted:

Check console output at 
https://secure.penguinsinabox.com/jenkins/job/Accumulo-Integration-Tests/542/ 
to view the results.