I believe if you name your stores, the directory name will reflect that. If
you see any directories in your state.dir corresponding to the store names,
that's rocksdb. As the name suggest, in memory stores are held (completely)
in memory
On Fri, Jul 19, 2019 at 3:21 AM Muhammed Ashik <ashi...@gmail.com> wrote:
> Hi If I’m not wrong I remember seeing the streams code ., that default
> rocksdb state.dir is rocksdb itself. Any content would go under
> /tmp/Kafka-streams/rocksdb
>
> On Fri, 19 Jul 2019 at 1:55 AM, Sophie Blee-Goldman <sop...@confluent.io>
> wrote:
>
> > And all four stores (BucketData, CacheData, StationKeyValue,
> > StationCacheData) are definitely explicitly added as
> > Stores.inMemoryKeyValueStore("name")? Hm.
> >
> > Does du -h show any contents within the
> > ./buzzard.MonitoringSeWlanStatsAggregator/0_1 directories that match any
> of
> > these store names? Also if you rerun the app with a different state.dir,
> do
> > all the same directories get created?
> >
> > On Thu, Jul 18, 2019 at 12:15 AM Muhammed Ashik <ashi...@gmail.com>
> wrote:
> >
> > > Hi, my Topology description..
> > >
> > > - Topologies:
> > > Sub-topology: 0
> > > Source: SeSource (topics: [buzzard.decoded.amon.stats.ap,
> > > buzzard.app.monitoring.internal.clientsestats])
> > > --> SeProcess
> > > Processor: SeProcess (stores: [BucketData, CacheData])
> > > --> AeSink
> > > <-- SeSource
> > > Sink: AeSink (topic: buzzard.app.monitoring.internal.stats)
> > > <-- SeProcess
> > >
> > > Sub-topology: 1
> > > Source: StationSource (topics:
> [buzzard.decoded.amon.stats.station])
> > > --> StationProcess
> > > Processor: StationProcess (stores: [StationKeyValue,
> > StationCacheData])
> > > --> StationSink
> > > <-- StationSource
> > > Sink: StationSink (topic:
> > > buzzard.app.monitoring.internal.clientsestats)
> > > <-- StationProcess
> > >
> > > Regards
> > > Ashik
> > >
> > > On Thu, Jul 18, 2019 at 1:31 AM Sophie Blee-Goldman <
> sop...@confluent.io
> > >
> > > wrote:
> > >
> > > > Hm. These directories shouldn't be created if using only an in memory
> > > > store. Can you print your topology?
> > > >
> > > > On Wed, Jul 17, 2019 at 11:02 AM Muhammed Ashik <ashi...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi I just did `du -mh` on `\tmp\kafka-streams` below are the
> folders
> > > > listed
> > > > > with some .lock files inside.
> > > > > not sure if these are coming from rocksdb.. and looks like the
> sizes
> > of
> > > > > these files are less.
> > > > >
> > > > > 4.0K ./buzzard.MonitoringSeWlanStatsAggregator/0_5
> > > > > 4.0K ./buzzard.MonitoringSeWlanStatsAggregator/0_3
> > > > > 4.0K ./buzzard.MonitoringSeWlanStatsAggregator/0_2
> > > > > 4.0K ./buzzard.MonitoringSeWlanStatsAggregator/1_4
> > > > > 4.0K ./buzzard.MonitoringSeWlanStatsAggregator/0_1
> > > > > 4.0K ./buzzard.MonitoringSeWlanStatsAggregator/0_4
> > > > > 28K ./buzzard.MonitoringSeWlanStatsAggregator
> > > > > 32K .
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Jul 17, 2019 at 11:18 PM Sophie Blee-Goldman <
> > > > sop...@confluent.io>
> > > > > wrote:
> > > > >
> > > > > > Hm. Just to be absolutely sure, could you try throwing an
> exception
> > > or
> > > > > > something in your RocksDBConfigSetter?
> > > > > >
> > > > > > On Wed, Jul 17, 2019 at 10:43 AM Muhammed Ashik <
> ashi...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > I can confirm the /tmp/kafka-streams doesn't have any data
> > related
> > > to
> > > > > > > rocksdb.
> > > > > > >
> > > > > > > On Wed, Jul 17, 2019 at 11:11 PM Sophie Blee-Goldman <
> > > > > > sop...@confluent.io>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > You can describe your topology to see if there are any state
> > > stores
> > > > > in
> > > > > > it
> > > > > > > > that you aren't aware of. Alternatively you could check out
> the
> > > > state
> > > > > > > > directory (/tmp/kafka-streams by default) and see if there is
> > any
> > > > > data
> > > > > > in
> > > > > > > > there
> > > > > > > >
> > > > > > > > On Wed, Jul 17, 2019 at 10:36 AM Muhammed Ashik <
> > > ashi...@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Thanks How can I verify If there is some data really going
> on
> > > > > rocksdb
> > > > > > > > > I tried printing the statistics with no success.
> > > > > > > > >
> > > > > > > > > class CustomRocksDBConfig extends RocksDBConfigSetter {
> > > > > > > > > override def setConfig(storeName: String, options:
> Options,
> > > > > > configs:
> > > > > > > > > util.Map[String, AnyRef]): Unit = {
> > > > > > > > >
> > > > > > > > > val stats = new Statistics
> > > > > > > > > stats.setStatsLevel(StatsLevel.ALL)
> > > > > > > > > options.setStatistics(stats)
> > > > > > > > > .setStatsDumpPeriodSec(600)
> > > > > > > > > options
> > > > > > > > > .setInfoLogLevel(InfoLogLevel.INFO_LEVEL)
> > > > > > > > > options.setDbLogDir("/tmp/dump")
> > > > > > > > >
> > > > > > > > > }
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > and included in the stream config ..
> > > > > > > > >
> > > > > > > > >
> > settings.put(StreamsConfig.ROCKSDB_CONFIG_SETTER_CLASS_CONFIG,
> > > > > > > > > classOf[CustomRocksDBConfig])
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Regards
> > > > > > > > > Ashik
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Jul 17, 2019 at 10:52 PM Sophie Blee-Goldman <
> > > > > > > > sop...@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Sorry, didn't see the "off-heap" part of the email. Are
> you
> > > > using
> > > > > > any
> > > > > > > > > > stateful DSL operators? The default stores are
> persistent,
> > so
> > > > you
> > > > > > may
> > > > > > > > > have
> > > > > > > > > > a RocksDB store in your topology without explicitly using
> > > one.
> > > > > > > > > >
> > > > > > > > > > On Wed, Jul 17, 2019 at 10:12 AM Sophie Blee-Goldman <
> > > > > > > > > sop...@confluent.io>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > If you are using inMemoryKeyValueStore, the records are
> > > > stored
> > > > > by
> > > > > > > > > > > definition in memory. RocksDB is not used at all. This
> > > store
> > > > > will
> > > > > > > > > > continue
> > > > > > > > > > > to grow proportionally to your keyspace. If you do not
> > have
> > > > > > > > sufficient
> > > > > > > > > > > memory to hold your entire dataset in memory, consider
> > > adding
> > > > > > > another
> > > > > > > > > > > instance or switching to the RocksDB store
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jul 17, 2019 at 6:22 AM Muhammed Ashik <
> > > > > > ashi...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > >> Kafka Streams version - 2.0.0
> > > > > > > > > > >>
> > > > > > > > > > >> Hi, in our streaming instance we are observing a
> steady
> > > > growth
> > > > > > in
> > > > > > > > the
> > > > > > > > > > >> off-heap memory (out of 2gb allocated memory 1.3 is
> > > reserved
> > > > > for
> > > > > > > > heap
> > > > > > > > > > >> memory and the ~700mb memory is utilised over a time
> of
> > > > ~6hrs
> > > > > > and
> > > > > > > > the
> > > > > > > > > > >> process is OOM killed eventually).
> > > > > > > > > > >>
> > > > > > > > > > >> we are using only the inMemoryKeyValueStore and not
> > doing
> > > > any
> > > > > > > > > > persistence.
> > > > > > > > > > >> as suggested the iterators are closed at the places it
> > is
> > > > > > > used(using
> > > > > > > > > it
> > > > > > > > > > in
> > > > > > > > > > >> only once).
> > > > > > > > > > >>
> > > > > > > > > > >> Some forums were relating such issues with rocksdb but
> > we
> > > > are
> > > > > > not
> > > > > > > > > > >> specifying rocksdb in the config explicitly though. I
> > was
> > > > not
> > > > > > sure
> > > > > > > > > > whether
> > > > > > > > > > >> it is used as a default in memory store by kafka
> > streams.
> > > > > > > > > > >>
> > > > > > > > > > >> Regards
> > > > > > > > > > >> Ashik
> > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
