Hello all, I would like to understand the usage of leveldb in the Warp10 standalone. I created 315360000 points in the database (1 signal sampled ate 1 sample / second for 10 years) and it creates a lot of level 3 files (LOG file indicate it is level 3, all files are arround 2.1 Mb) but it never reach upper levels. I got 2118 files of level 0 to 3. In leveldb folder there is 4Gb data:
./warp10/leveldb$ ls -l | more total 4331748 -rw-r--r-- 1 warp10 warp10 2111426 Jan 18 16:52 002708.sst -rw-r--r-- 1 warp10 warp10 2112858 Jan 18 16:52 002710.sst -rw-r--r-- 1 warp10 warp10 2110567 Jan 18 16:52 002714.sst -rw-r--r-- 1 warp10 warp10 2112793 Jan 18 16:52 002718.sst -rw-r--r-- 1 warp10 warp10 2111804 Jan 18 16:52 002720.sst -rw-r--r-- 1 warp10 warp10 2111263 Jan 18 16:52 002724.sst ... -rw-r--r-- 1 warp10 warp10 2111727 Jan 18 16:55 004641.sst -rw-r--r-- 1 warp10 warp10 2112784 Jan 18 16:55 004643.sst -rw-r--r-- 1 warp10 warp10 2110597 Jan 18 16:55 004647.sst -rw-r--r-- 1 warp10 warp10 2112826 Jan 18 16:55 004649.sst -rw-r--r-- 1 warp10 warp10 2111476 Jan 18 16:55 004654.sst -rw-r--r-- 1 warp10 warp10 2111578 Jan 18 16:55 004656.sst ... -rw-r--r-- 1 warp10 warp10 2111374 Jan 18 17:20 020859.sst -rw-r--r-- 1 warp10 warp10 2031616 Jan 18 17:29 020865.log -rw-r--r-- 1 warp10 warp10 2113084 Jan 18 17:20 020867.sst -rw-r--r-- 1 warp10 warp10 1661397 Jan 18 17:20 020868.sst -rw-r--r-- 1 warp10 warp10 2112236 Jan 18 17:20 020869.sst -rw-r--r-- 1 warp10 warp10 2107283 Jan 18 17:20 020870.sst -rw-r--r-- 1 warp10 warp10 16 Jan 18 16:35 CURRENT -rw-r--r-- 1 warp10 warp10 0 Jan 18 16:35 LOCK -rw-r--r-- 1 warp10 warp10 4240723 Jan 18 17:20 LOG -rw-r--r-- 1 warp10 warp10 57 Jan 18 16:35 LOG.old -rw-r--r-- 1 warp10 warp10 2031616 Jan 18 17:20 MANIFEST-000004 After a delete with the WebAPI, the files are not deleted, others are created (for tombstone I think). Then when I FETCH data with WarpScript, it is very long. It seams DELETE Warpscript or FETCH ask time to time a clean on the database. It takes a lot of seconds. When I run a manual compaction, it seams compaction is ok. (in docker, manual compaction command is /usr/lib/jvm/java-1.8-openjdk/jre/bin/java -cp /opt/warp10/bin/warp10-2.7.2.jar -Dfile.encoding=UTF-8 io.warp10.standalone.WarpCompact /data/warp10/leveldb "" "" ) Then FETCH command is 200ms. Why Warp10 stop compaction at level3 ? Is it possible to run a compaction manually ? It is possible to raise the compaction level of leveldb with Warp10 ? If I create a Function Extension of WarpScript to manually start a compaction, does it freeze the ingestion possibilities of Warp10 ? What does it means "key range" parameters of the function compactRange(byte[] begin, byte[] end)? Best regards, Fabien. -- You received this message because you are subscribed to the Google Groups "Warp 10 users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/warp10-users/2e3665e0-8958-454a-82d0-1e92bd769f99n%40googlegroups.com.
