[GitHub] [incubator-iotdb] JackieTien97 commented on a change in pull request #802: Cached chunk
JackieTien97 commented on a change in pull request #802: Cached chunk
URL: https://github.com/apache/incubator-iotdb/pull/802#discussion_r379276321
##
File path:
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkCache.java
##
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.engine.cache;
+
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.tsfile.file.metadata.ChunkMetaData;
+import org.apache.iotdb.tsfile.read.TsFileSequenceReader;
+import org.apache.iotdb.tsfile.read.common.Chunk;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * This class is used to cache Chunk of
ChunkMetaData in IoTDB. The caching
+ * strategy is LRU.
+ */
+public class ChunkCache {
+
+ private static final Logger logger =
LoggerFactory.getLogger(ChunkCache.class);
+ private static final IoTDBConfig config =
IoTDBDescriptor.getInstance().getConfig();
+ private static final long MEMORY_THRESHOLD_IN_CHUNK_CACHE =
config.getAllocateMemoryForChunkCache();
+ private static boolean cacheEnable = config.isMetaDataCacheEnable();
+
+ private final LRULinkedHashMap lruCache;
+
+ private AtomicLong cacheHitNum = new AtomicLong();
+ private AtomicLong cacheRequestNum = new AtomicLong();
+
+ private final ReadWriteLock lock = new ReentrantReadWriteLock();
+ private final Lock readLock = lock.readLock();
+ private final Lock writeLock = lock.writeLock();
+
+
+ private ChunkCache() {
+lruCache = new LRULinkedHashMap(MEMORY_THRESHOLD_IN_CHUNK_CACHE, true) {
+ @Override
+ protected long calEntrySize(ChunkMetaData key, Chunk value) {
+return RamUsageEstimator.sizeOf(value) +
RamUsageEstimator.sizeOf(value);
+ }
+};
+ }
+
+ public static ChunkCache getInstance() {
+return ChunkCacheHolder.INSTANCE;
+ }
+
+ public Chunk get(ChunkMetaData chunkMetaData, TsFileSequenceReader reader)
throws IOException {
+if (!cacheEnable) {
+ Chunk chunk = reader.readMemChunk(chunkMetaData);
+ return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
+}
+
+cacheRequestNum.incrementAndGet();
+
+try {
+ readLock.lock();
+ if (lruCache.containsKey(chunkMetaData)) {
+cacheHitNum.incrementAndGet();
+printCacheLog(true);
+Chunk chunk = lruCache.get(chunkMetaData);
+return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
Review comment:
The chunk has a field byte buffer, it should be reused.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #802: Cached chunk
qiaojialin commented on a change in pull request #802: Cached chunk URL: https://github.com/apache/incubator-iotdb/pull/802#discussion_r379260931 ## File path: server/src/assembly/resources/conf/iotdb-engine.properties ## @@ -275,9 +275,9 @@ chunk_merge_point_threshold=20480 # whether to cache meta data(ChunkMetaData and TsFileMetaData) or not. meta_data_cache_enable=true -# Read memory Allocation Ratio: FileMetaDataCache, ChunkMetaDataCache, and Free Memory Used in Query. -# The parameter form is a:b:c, where a, b and c are integers. for example: 1:1:1 , 3:6:10 -filemeta_chunkmeta_free_memory_proportion=3:6:10 +# Read memory Allocation Ratio: FileMetaDataCache, ChunkMetaDataCache, ChunkCache and Free Memory Used in Query. +# The parameter form is a:b:c:d, where a, b, c and d are integers. for example: 1:1:1:1 , 3:6:10:20 +filemeta_chunkmeta_free_memory_proportion=3:6:10:20 Review comment: this name lacks "chunk", how about remove this parameter and dynamically adjust in the server This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #802: Cached chunk
qiaojialin commented on a change in pull request #802: Cached chunk
URL: https://github.com/apache/incubator-iotdb/pull/802#discussion_r379261558
##
File path:
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkCache.java
##
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.engine.cache;
+
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.tsfile.file.metadata.ChunkMetaData;
+import org.apache.iotdb.tsfile.read.TsFileSequenceReader;
+import org.apache.iotdb.tsfile.read.common.Chunk;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * This class is used to cache Chunk of
ChunkMetaData in IoTDB. The caching
+ * strategy is LRU.
+ */
+public class ChunkCache {
+
+ private static final Logger logger =
LoggerFactory.getLogger(ChunkCache.class);
+ private static final IoTDBConfig config =
IoTDBDescriptor.getInstance().getConfig();
+ private static final long MEMORY_THRESHOLD_IN_CHUNK_CACHE =
config.getAllocateMemoryForChunkCache();
+ private static boolean cacheEnable = config.isMetaDataCacheEnable();
+
+ private final LRULinkedHashMap lruCache;
+
+ private AtomicLong cacheHitNum = new AtomicLong();
+ private AtomicLong cacheRequestNum = new AtomicLong();
+
+ private final ReadWriteLock lock = new ReentrantReadWriteLock();
+ private final Lock readLock = lock.readLock();
+ private final Lock writeLock = lock.writeLock();
Review comment:
I suggest using lock.readLock to replace the readLock field
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #802: Cached chunk
qiaojialin commented on a change in pull request #802: Cached chunk
URL: https://github.com/apache/incubator-iotdb/pull/802#discussion_r379262597
##
File path:
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkCache.java
##
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.engine.cache;
+
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.tsfile.file.metadata.ChunkMetaData;
+import org.apache.iotdb.tsfile.read.TsFileSequenceReader;
+import org.apache.iotdb.tsfile.read.common.Chunk;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * This class is used to cache Chunk of
ChunkMetaData in IoTDB. The caching
+ * strategy is LRU.
+ */
+public class ChunkCache {
+
+ private static final Logger logger =
LoggerFactory.getLogger(ChunkCache.class);
+ private static final IoTDBConfig config =
IoTDBDescriptor.getInstance().getConfig();
+ private static final long MEMORY_THRESHOLD_IN_CHUNK_CACHE =
config.getAllocateMemoryForChunkCache();
+ private static boolean cacheEnable = config.isMetaDataCacheEnable();
+
+ private final LRULinkedHashMap lruCache;
+
+ private AtomicLong cacheHitNum = new AtomicLong();
+ private AtomicLong cacheRequestNum = new AtomicLong();
+
+ private final ReadWriteLock lock = new ReentrantReadWriteLock();
+ private final Lock readLock = lock.readLock();
+ private final Lock writeLock = lock.writeLock();
+
+
+ private ChunkCache() {
+lruCache = new LRULinkedHashMap(MEMORY_THRESHOLD_IN_CHUNK_CACHE, true) {
+ @Override
+ protected long calEntrySize(ChunkMetaData key, Chunk value) {
+return RamUsageEstimator.sizeOf(value) +
RamUsageEstimator.sizeOf(value);
+ }
+};
+ }
+
+ public static ChunkCache getInstance() {
+return ChunkCacheHolder.INSTANCE;
+ }
+
+ public Chunk get(ChunkMetaData chunkMetaData, TsFileSequenceReader reader)
throws IOException {
+if (!cacheEnable) {
+ Chunk chunk = reader.readMemChunk(chunkMetaData);
+ return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
+}
+
+cacheRequestNum.incrementAndGet();
+
+try {
+ readLock.lock();
+ if (lruCache.containsKey(chunkMetaData)) {
+cacheHitNum.incrementAndGet();
+printCacheLog(true);
+Chunk chunk = lruCache.get(chunkMetaData);
+return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
Review comment:
why not return the chunk got from cache?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #802: Cached chunk
qiaojialin commented on a change in pull request #802: Cached chunk
URL: https://github.com/apache/incubator-iotdb/pull/802#discussion_r379263215
##
File path:
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkCache.java
##
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.engine.cache;
+
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.tsfile.file.metadata.ChunkMetaData;
+import org.apache.iotdb.tsfile.read.TsFileSequenceReader;
+import org.apache.iotdb.tsfile.read.common.Chunk;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * This class is used to cache Chunk of
ChunkMetaData in IoTDB. The caching
+ * strategy is LRU.
+ */
+public class ChunkCache {
+
+ private static final Logger logger =
LoggerFactory.getLogger(ChunkCache.class);
+ private static final IoTDBConfig config =
IoTDBDescriptor.getInstance().getConfig();
+ private static final long MEMORY_THRESHOLD_IN_CHUNK_CACHE =
config.getAllocateMemoryForChunkCache();
+ private static boolean cacheEnable = config.isMetaDataCacheEnable();
+
+ private final LRULinkedHashMap lruCache;
+
+ private AtomicLong cacheHitNum = new AtomicLong();
+ private AtomicLong cacheRequestNum = new AtomicLong();
+
+ private final ReadWriteLock lock = new ReentrantReadWriteLock();
+ private final Lock readLock = lock.readLock();
+ private final Lock writeLock = lock.writeLock();
+
+
+ private ChunkCache() {
+lruCache = new LRULinkedHashMap(MEMORY_THRESHOLD_IN_CHUNK_CACHE, true) {
+ @Override
+ protected long calEntrySize(ChunkMetaData key, Chunk value) {
+return RamUsageEstimator.sizeOf(value) +
RamUsageEstimator.sizeOf(value);
+ }
+};
+ }
+
+ public static ChunkCache getInstance() {
+return ChunkCacheHolder.INSTANCE;
+ }
+
+ public Chunk get(ChunkMetaData chunkMetaData, TsFileSequenceReader reader)
throws IOException {
+if (!cacheEnable) {
+ Chunk chunk = reader.readMemChunk(chunkMetaData);
+ return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
+}
+
+cacheRequestNum.incrementAndGet();
+
+try {
+ readLock.lock();
+ if (lruCache.containsKey(chunkMetaData)) {
+cacheHitNum.incrementAndGet();
+printCacheLog(true);
+Chunk chunk = lruCache.get(chunkMetaData);
+return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
+ }
+} finally {
+ readLock.unlock();
+}
+
+writeLock.lock();
+if (lruCache.containsKey(chunkMetaData)) {
+ writeLock.unlock();
+ readLock.lock();
+ cacheHitNum.incrementAndGet();
+ printCacheLog(true);
+ Chunk chunk = lruCache.get(chunkMetaData);
+ readLock.unlock();
+ return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
+}
+printCacheLog(false);
+Chunk chunk = reader.readMemChunk(chunkMetaData);
+lruCache.put(chunkMetaData, chunk);
+writeLock.unlock();
+return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
Review comment:
```suggestion
return chunk;
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #802: Cached chunk
qiaojialin commented on a change in pull request #802: Cached chunk
URL: https://github.com/apache/incubator-iotdb/pull/802#discussion_r379263035
##
File path:
server/src/main/java/org/apache/iotdb/db/engine/cache/ChunkCache.java
##
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.engine.cache;
+
+import org.apache.iotdb.db.conf.IoTDBConfig;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
+import org.apache.iotdb.tsfile.file.metadata.ChunkMetaData;
+import org.apache.iotdb.tsfile.read.TsFileSequenceReader;
+import org.apache.iotdb.tsfile.read.common.Chunk;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+
+/**
+ * This class is used to cache Chunk of
ChunkMetaData in IoTDB. The caching
+ * strategy is LRU.
+ */
+public class ChunkCache {
+
+ private static final Logger logger =
LoggerFactory.getLogger(ChunkCache.class);
+ private static final IoTDBConfig config =
IoTDBDescriptor.getInstance().getConfig();
+ private static final long MEMORY_THRESHOLD_IN_CHUNK_CACHE =
config.getAllocateMemoryForChunkCache();
+ private static boolean cacheEnable = config.isMetaDataCacheEnable();
+
+ private final LRULinkedHashMap lruCache;
+
+ private AtomicLong cacheHitNum = new AtomicLong();
+ private AtomicLong cacheRequestNum = new AtomicLong();
+
+ private final ReadWriteLock lock = new ReentrantReadWriteLock();
+ private final Lock readLock = lock.readLock();
+ private final Lock writeLock = lock.writeLock();
+
+
+ private ChunkCache() {
+lruCache = new LRULinkedHashMap(MEMORY_THRESHOLD_IN_CHUNK_CACHE, true) {
+ @Override
+ protected long calEntrySize(ChunkMetaData key, Chunk value) {
+return RamUsageEstimator.sizeOf(value) +
RamUsageEstimator.sizeOf(value);
+ }
+};
+ }
+
+ public static ChunkCache getInstance() {
+return ChunkCacheHolder.INSTANCE;
+ }
+
+ public Chunk get(ChunkMetaData chunkMetaData, TsFileSequenceReader reader)
throws IOException {
+if (!cacheEnable) {
+ Chunk chunk = reader.readMemChunk(chunkMetaData);
+ return new Chunk(chunk.getHeader(), chunk.getData().duplicate(),
chunk.getDeletedAt(), reader.getEndianType());
Review comment:
why new a Chunk?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] sonarcloud[bot] commented on issue #802: Cached chunk
sonarcloud[bot] commented on issue #802: Cached chunk URL: https://github.com/apache/incubator-iotdb/pull/802#issuecomment-586095283 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=VULNERABILITY) (and [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=SECURITY_HOTSPOT) to review) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=802&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=802) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=802&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=802&metric=new_duplicated_lines_density&view=list) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] JackieTien97 opened a new pull request #802: Cached chunk
JackieTien97 opened a new pull request #802: Cached chunk URL: https://github.com/apache/incubator-iotdb/pull/802 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] sonarcloud[bot] removed a comment on issue #793: [IOTDB-274] Refactor MManager
sonarcloud[bot] removed a comment on issue #793: [IOTDB-274] Refactor MManager URL: https://github.com/apache/incubator-iotdb/pull/793#issuecomment-585576160 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=VULNERABILITY) (and [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=SECURITY_HOTSPOT) to review) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=CODE_SMELL) [6 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=793) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=793&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=793&metric=new_duplicated_lines_density&view=list) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] sonarcloud[bot] commented on issue #793: [IOTDB-274] Refactor MManager
sonarcloud[bot] commented on issue #793: [IOTDB-274] Refactor MManager URL: https://github.com/apache/incubator-iotdb/pull/793#issuecomment-586091974 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=VULNERABILITY) (and [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=SECURITY_HOTSPOT) to review) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=CODE_SMELL) [4 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=793&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=793) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=793&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=793&metric=new_duplicated_lines_density&view=list) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] sonarcloud[bot] commented on issue #801: Add more log of compression ratio
sonarcloud[bot] commented on issue #801: Add more log of compression ratio URL: https://github.com/apache/incubator-iotdb/pull/801#issuecomment-586081273 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=VULNERABILITY) (and [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=SECURITY_HOTSPOT) to review) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=801&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=801) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=801&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=801&metric=new_duplicated_lines_density&view=list) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] fanhualta opened a new pull request #801: Add more log of compression ratio
fanhualta opened a new pull request #801: Add more log of compression ratio URL: https://github.com/apache/incubator-iotdb/pull/801 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] liutaohua commented on issue #798: del ChunkLoaderImpl LruCache
liutaohua commented on issue #798: del ChunkLoaderImpl LruCache URL: https://github.com/apache/incubator-iotdb/pull/798#issuecomment-586064909 > Hi, the reason for discarding the cache? Chunkloader has a problem with the use it , it is `new` in most cases, so cache will not be used! And a huge cache will cause OOM. This issue has been fixed and submitted by Jackie tien This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] sonarcloud[bot] removed a comment on issue #713: [IOTDB-418] New series reader
sonarcloud[bot] removed a comment on issue #713: [IOTDB-418] New series reader URL: https://github.com/apache/incubator-iotdb/pull/713#issuecomment-585585475 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=VULNERABILITY) (and [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=SECURITY_HOTSPOT) to review) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=CODE_SMELL) [17 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=713) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=713&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=713&metric=new_duplicated_lines_density&view=list) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] sonarcloud[bot] commented on issue #713: [IOTDB-418] New series reader
sonarcloud[bot] commented on issue #713: [IOTDB-418] New series reader URL: https://github.com/apache/incubator-iotdb/pull/713#issuecomment-586054107 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=VULNERABILITY) (and [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=SECURITY_HOTSPOT) to review) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=CODE_SMELL) [17 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=713&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=713) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=713&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=713&metric=new_duplicated_lines_density&view=list) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] jixuan1989 commented on issue #798: del ChunkLoaderImpl LruCache
jixuan1989 commented on issue #798: del ChunkLoaderImpl LruCache URL: https://github.com/apache/incubator-iotdb/pull/798#issuecomment-586044556 Hi, the reason for discarding the cache? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] jack870131 commented on a change in pull request #713: [IOTDB-418] New series reader
jack870131 commented on a change in pull request #713: [IOTDB-418] New series
reader
URL: https://github.com/apache/incubator-iotdb/pull/713#discussion_r378857095
##
File path: server/src/main/java/org/apache/iotdb/db/conf/IoTDBDescriptor.java
##
@@ -309,6 +309,11 @@ private void loadProps() {
conf.setMemtableNumInEachStorageGroup(
Integer.parseInt(properties.getProperty("memtable_num_in_each_storage_group",
String.valueOf(conf.getMemtableNumInEachStorageGroup();
+ // the num of memtables in each storage group
Review comment:
Fixed. Sorry for misunderstanding. This should be "the default fill interval
in LinearFill and PreviousFill".
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] jack870131 commented on a change in pull request #713: [IOTDB-418] New series reader
jack870131 commented on a change in pull request #713: [IOTDB-418] New series
reader
URL: https://github.com/apache/incubator-iotdb/pull/713#discussion_r378856713
##
File path:
server/src/main/java/org/apache/iotdb/db/query/aggregation/AggregateResult.java
##
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.aggregation;
+
+import org.apache.iotdb.db.exception.query.QueryProcessException;
+import org.apache.iotdb.db.query.reader.series.IReaderByTimestamp;
+import org.apache.iotdb.tsfile.exception.write.UnSupportedDataTypeException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.file.metadata.statistics.Statistics;
+import org.apache.iotdb.tsfile.read.common.BatchData;
+import org.apache.iotdb.tsfile.utils.Binary;
+
+import java.io.IOException;
+
+public abstract class AggregateResult {
+
+ protected TSDataType dataType;
+
+ private boolean booleanRet;
+ private int intRet;
+ private long longRet;
+ private float floatRet;
+ private double doubleRet;
+ private Binary binaryRet;
+
+ private boolean hasResult;
+
+ /**
+ * construct.
+ *
+ * @param dataType result data type.
+ */
+ public AggregateResult(TSDataType dataType) {
+this.dataType = dataType;
+this.hasResult = false;
+ }
+
+ public abstract Object getResult();
+
+ /**
+ * Calculate the aggregation using Statistics
+ *
+ * @param statistics chunkStatistics or pageStatistics
+ */
+ public abstract void updateResultFromStatistics(Statistics statistics)
+ throws QueryProcessException;
+
+ /**
+ * Aggregate results cannot be calculated using Statistics directly, using
the data in each page
+ *
+ * @param dataInThisPage the data in Page
+ */
+ public abstract void updateResultFromPageData(BatchData dataInThisPage)
throws IOException;
+ /**
+ * Aggregate results cannot be calculated using Statistics directly, using
the data in each page
+ *
+ * @param dataInThisPage the data in Page
+ * @param bound calculate points whose time < bound
+ */
+ public abstract void updateResultFromPageData(BatchData dataInThisPage, long
bound)
+ throws IOException;
+
+ /**
+ * This method is calculate the aggregation using the common timestamps
of cross series
Review comment:
Fixed. Thx.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-iotdb] liutaohua closed pull request #798: del ChunkLoaderImpl LruCache
liutaohua closed pull request #798: del ChunkLoaderImpl LruCache URL: https://github.com/apache/incubator-iotdb/pull/798 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin merged pull request #800: [IOTDB-471] fix and try to locate latestTimeForEachDevice null pointer bug
qiaojialin merged pull request #800: [IOTDB-471] fix and try to locate latestTimeForEachDevice null pointer bug URL: https://github.com/apache/incubator-iotdb/pull/800 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] qiaojialin merged pull request #797: Enable auto create schema
qiaojialin merged pull request #797: Enable auto create schema URL: https://github.com/apache/incubator-iotdb/pull/797 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-iotdb] SilverNarcissus opened a new pull request #800: Iotdb 471 fix bug
SilverNarcissus opened a new pull request #800: Iotdb 471 fix bug URL: https://github.com/apache/incubator-iotdb/pull/800 Add more info when encounter error This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
