[
https://issues.apache.org/jira/browse/USERGRID-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391846#comment-14391846
]
George Reyes commented on USERGRID-537:
---------------------------------------
I am only seeing this specific issue under load, but when it happens it happens
a lot. This ticket will require a profiling and debug run of the
writeUniqueVerify stage. It is possible that we are simply overwhelming
hysterix and seeing unexpected behavior from it since it has caused some
problems in the past. We have also seen the replay commands fail it is because
usergrid is just too slow in returning the entity. But we have a specific error
that we would return in that case. In this case we are returning from loading
the entity in cassandra, it just doesn't have a version associated with it.
This ticket should also accompany a document with the read write path detailed
so that others who see this will know where to look.
> Write Unique Verify - could not get uniqueValue from the field name
> -------------------------------------------------------------------
>
> Key: USERGRID-537
> URL: https://issues.apache.org/jira/browse/USERGRID-537
> Project: Usergrid
> Issue Type: Story
> Reporter: George Reyes
> Assignee: George Reyes
>
> 2015-04-01 18:57:54,440 ERROR (http-bio-8080-exec-224)
> AbstractExceptionMapper - Server Error (500):
> {"error":"hystrix_runtime","timestamp":1427929074440,"duration":0,"error_description":"ConsistentReplayCommand
> failed and failed retrieving
> fallback.","exception":"com.netflix.hystrix.exception.HystrixRuntimeException"}
> 2015-04-01 18:57:54,440 ERROR (hystrix-user-94) WriteUniqueVerify - Could not
> get uniqueValue from the field name.
> 2015-04-01 18:57:54,440 ERROR (http-bio-8080-exec-325) ThrowableMapper - An
> uncaught exception occurred during HTTP invocation
> com.netflix.hystrix.exception.HystrixRuntimeException:
> ConsistentReplayCommand failed and failed retrieving fallback.
> at
> com.netflix.hystrix.HystrixCommand.getFallbackOrThrowException(HystrixCommand.java:1660)
> at
> com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1341)
> at
> com.netflix.hystrix.HystrixCommand.access$2300(HystrixCommand.java:103)
> at com.netflix.hystrix.HystrixCommand$5.call(HystrixCommand.java:1186)
> at
> com.netflix.hystrix.strategy.concurrency.HystrixContextCallable.call(HystrixContextCallable.java:51)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: Could not retrieve unique value for
> field name, unable to verify
> at
> org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify$ConsistentReplayCommand.executeStrategy(WriteUniqueVerify.java:207)
> at
> org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify$ConsistentReplayCommand.run(WriteUniqueVerify.java:178)
> at
> org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify$ConsistentReplayCommand.run(WriteUniqueVerify.java:159)
> at
> com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1281)
> ... 7 more
> Caused by: java.lang.Throwable: Calling Thread included as the last 'caused
> by' on the chain.
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1033)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:282)
> at
> rx.internal.operators.BlockingOperatorToFuture$2.get(BlockingOperatorToFuture.java:111)
> at
> com.netflix.hystrix.HystrixCommand$1.performBlockingGetWithTimeout(HystrixCommand.java:623)
> at com.netflix.hystrix.HystrixCommand$1.get(HystrixCommand.java:522)
> at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:431)
> at
> org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify.call(WriteUniqueVerify.java:151)
> at
> org.apache.usergrid.persistence.collection.mvcc.stage.write.WriteUniqueVerify.call(WriteUniqueVerify.java:68)
> at rx.Observable$12.onNext(Observable.java:4036)
> at
> rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:61)
> at rx.Observable$3.call(Observable.java:1551)
> at rx.Observable$3.call(Observable.java:1546)
> at rx.Observable.unsafeSubscribe(Observable.java:6839)
> at
> rx.internal.operators.OperatorSubscribeOn$1$1.call(OperatorSubscribeOn.java:60)
> at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:43)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> ... 3 more
> Caused by: rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting
> onNext value: CollectionIoEvent.class
> at
> rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:98)
> at
> rx.internal.operators.OperatorDoOnEach$1.onNext(OperatorDoOnEach.java:63)
> ... 12 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
