ibessonov commented on a change in pull request #562:
URL: https://github.com/apache/ignite-3/pull/562#discussion_r789462135
##########
File path:
modules/storage-rocksdb/src/main/java/org/apache/ignite/internal/storage/rocksdb/RocksDbPartitionStorage.java
##########
@@ -436,21 +476,47 @@ public boolean hasNext() {
@Override
protected DataRow decodeEntry(byte[] key, byte[] value) {
- return new SimpleDataRow(key, value);
+ byte[] rowKey = new byte[key.length - PARTITION_KEY_PREFIX_SIZE];
+
+ System.arraycopy(key, PARTITION_KEY_PREFIX_SIZE, rowKey, 0,
rowKey.length);
+
+ return new SimpleDataRow(rowKey, value);
}
}
+ /**
+ * Creates a key used in this partition storage by prepending a partition
ID (to distinguish between different partition data)
+ * and the key's hash (an optimisation).
+ */
+ private byte[] partitionKey(byte[] key) {
+ byte[] partitionKey = new byte[PARTITION_KEY_PREFIX_SIZE + key.length];
+
+ int pos = 0;
+
+ intToBytes(partId, partitionKey, pos, Integer.BYTES);
Review comment:
Why don't you create a ByteBuffer with corresponding ordering and then
invoke some `putInt`s?
##########
File path:
modules/storage-rocksdb/src/main/java/org/apache/ignite/internal/storage/rocksdb/RocksDbPartitionStorage.java
##########
@@ -46,20 +47,27 @@
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.rocksdb.IngestExternalFileOptions;
+import org.rocksdb.ReadOptions;
import org.rocksdb.RocksDB;
import org.rocksdb.RocksDBException;
import org.rocksdb.RocksIterator;
+import org.rocksdb.Slice;
import org.rocksdb.Snapshot;
import org.rocksdb.WriteBatch;
import org.rocksdb.WriteOptions;
/**
* Storage implementation based on a single RocksDB instance.
*/
-public class RocksDbPartitionStorage implements PartitionStorage {
+class RocksDbPartitionStorage implements PartitionStorage {
/** Suffix for the temporary snapshot folder. */
private static final String TMP_SUFFIX = ".tmp";
+ /**
+ * Size of the overhead for all keys in the storage: partition ID ({@code
int}) + key hash ({@code int}).
Review comment:
Hmm... Partition ID is unsigned short, prefix can be 6 bytes
##########
File path:
modules/storage-rocksdb/src/main/java/org/apache/ignite/internal/storage/rocksdb/RocksDbTableStorage.java
##########
@@ -238,65 +254,94 @@ public void destroy() throws StorageException {
/** {@inheritDoc} */
@Override
public PartitionStorage getOrCreatePartition(int partId) throws
StorageException {
- PartitionStorage partition = getPartition(partId);
+ assert !stopped : "Storage has been stopped";
- if (partition != null) {
- return partition;
- }
+ synchronized (partitionsLock) {
Review comment:
Will this prevent us from concurrent partitions creation? I guess so,
this is not good. Ignite 2.x uses
`org.apache.ignite.internal.util.GridStripedLock` for similar purposes, you
could port it :)
Although I'm not sure that `Runtime.getRuntime().availableProcessors()` is
good choice for stripes count
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
