[
https://issues.apache.org/jira/browse/DRILL-5481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003073#comment-16003073
]
ASF GitHub Bot commented on DRILL-5481:
---------------------------------------
Github user kkhatua commented on a diff in the pull request:
https://github.com/apache/drill/pull/827#discussion_r115546712
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/store/EphemeralPersistentStore.java
---
@@ -15,28 +15,42 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.drill.exec.testing.store;
+package org.apache.drill.exec.store.sys.store;
import java.util.Iterator;
import java.util.Map;
import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
-import com.google.common.collect.Iterables;
-import com.google.common.collect.Maps;
import org.apache.drill.common.concurrent.AutoCloseableLock;
+import org.apache.drill.exec.exception.StoreException;
import org.apache.drill.exec.exception.VersionMismatchException;
import org.apache.drill.exec.store.sys.BasePersistentStore;
+import org.apache.drill.exec.store.sys.PersistentStoreConfig;
import org.apache.drill.exec.store.sys.PersistentStoreMode;
-import org.apache.drill.exec.store.sys.store.DataChangeVersion;
-public class NoWriteLocalStore<V> extends BasePersistentStore<V> {
+import com.google.common.collect.Iterables;
+
+public class EphemeralPersistentStore<V> extends BasePersistentStore<V> {
+ private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(EphemeralPersistentStore.class);
+
private final ReadWriteLock readWriteLock = new ReentrantReadWriteLock();
private final AutoCloseableLock readLock = new
AutoCloseableLock(readWriteLock.readLock());
private final AutoCloseableLock writeLock = new
AutoCloseableLock(readWriteLock.writeLock());
- private final ConcurrentMap<String, V> store = Maps.newConcurrentMap();
+ private ConcurrentMap<String, V> store;
private int version = -1;
+ private int maxCapacity;
+ private AtomicInteger currentSize = new AtomicInteger();
--- End diff --
There is already a writelock. However, with a SkipList, the call of using
size() is expensive. To minimize this computation, we have an atomic counter.
With the writeLock, if I hit the max "virtual" capacity, I am simply inserting
the latest entry and popping out the last (oldest entry). Unlike Concurrent
Queues, ConcurrentSkipListMap allows natural ordering of the keys (which is
required for serving via the WebUI).
> Allow Drill to persist profiles in-memory only with a max capacity
> ------------------------------------------------------------------
>
> Key: DRILL-5481
> URL: https://issues.apache.org/jira/browse/DRILL-5481
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.10.0
> Reporter: Kunal Khatua
> Assignee: Kunal Khatua
>
> To allow for fast persistence of profiles on a temporary basis (i.e. till the
> life of the Drillbit), an existing test class
> {{org.apache.drill.exec.testing.store.NoWriteLocalStore.java}} was refactored
> to {{org.apache.drill.exec.store.sys.store.EphemeralPersistentStore}} and
> given the ability to maintain a max capacity.
> This should allow query profiles to be available for as long as the Drillbit
> process' lifespan.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
