Hi Kevin,

You can use the Processor API to create multiple state stores and associate
these stores to your processors, some guidance can be found here:

http://kafka.apache.org/0100/documentation.html#streams_processor

As for your specific case, unfortunately I cannot tell much from your code
snippet and the stack trace, if you could update your program sketch
defining the topology I can help you defining what king of apis you may
want to use.

Guozhang


On Tue, Apr 12, 2016 at 8:28 AM, Kevin Niemann <[email protected]
> wrote:

> I've built Kafka from trunk 0.10 intending to build a streaming
> application and I'm wondering it if supports Windows. I tried increasing
> the Maven dependency to RocksDB 4.2.0.
>
> Jay's post mentions Kafka support multiple state stores. Is it documented
> how to configure these or do you recommend running my app in Linux?
>
>   public static String getJniLibraryName(final String name) {
>     if (isUnix()) {
>       final String arch = (is64Bit()) ? "64" : "32";
>       return String.format("%sjni-linux%s", name, arch);
>     } else if (isMac()) {
>       return String.format("%sjni-osx", name);
>     }
>     throw new UnsupportedOperationException();
>   }
>
> Exception in thread "StreamThread-1" java.lang.ExceptionInInitializerError
>                 at org.rocksdb.RocksDB.loadLibrary(RocksDB.java:47)
>                 at org.rocksdb.RocksDB.<clinit>(RocksDB.java:23)
>                 at org.rocksdb.Options.<clinit>(Options.java:21)
>                 at
> org.apache.kafka.streams.state.internals.RocksDBStore.<init>(RocksDBStore.java:113)
>                 at
> org.apache.kafka.streams.state.internals.RocksDBStore.<init>(RocksDBStore.java:99)
>                 at
> org.apache.kafka.streams.state.internals.RocksDBKeyValueStoreSupplier.get(RocksDBKeyValueStoreSupplier.java:56)
>                 at
> org.apache.kafka.streams.processor.internals.AbstractTask.initializeStateStores(AbstractTask.java:77)
>                 at
> org.apache.kafka.streams.processor.internals.StreamTask.<init>(StreamTask.java:115)
>                 at
> org.apache.kafka.streams.processor.internals.StreamThread.createStreamTask(StreamThread.java:579)
>                 at
> org.apache.kafka.streams.processor.internals.StreamThread.addStreamTasks(StreamThread.java:606)
>                 at
> org.apache.kafka.streams.processor.internals.StreamThread.access$000(StreamThread.java:71)
>                 at
> org.apache.kafka.streams.processor.internals.StreamThread$1.onPartitionsAssigned(StreamThread.java:126)
>                 at
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:205)
>                 at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:226)
>                 at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$1.onSuccess(AbstractCoordinator.java:221)
>                 at
> org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:133)
>                 at
> org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:107)
>                 at
> org.apache.kafka.clients.consumer.internals.RequestFuture$2.onSuccess(RequestFuture.java:182)
>                 at
> org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:133)
>                 at
> org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:107)
>                 at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupRequestHandler.handle(AbstractCoordinator.java:430)
>                 at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupRequestHandler.handle(AbstractCoordinator.java:416)
>                 at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:673)
>                 at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:652)
>                 at
> org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:167)
>                 at
> org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:133)
>                 at
> org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:107)
>                 at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(ConsumerNetworkClient.java:397)
>                 at
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:279)
>                 at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:338)
>                 at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
>                 at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:191)
>                 at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:161)
>                 at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:237)
>                 at
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.ensurePartitionAssignment(ConsumerCoordinator.java:323)
>                 at
> org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:900)
>                 at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:862)
>                 at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:325)
>                 at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:248)
> Caused by: java.lang.UnsupportedOperationException
>                 at
> org.rocksdb.util.Environment.getJniLibraryName(Environment.java:40)
>                 at
> org.rocksdb.NativeLibraryLoader.<clinit>(NativeLibraryLoader.java:19)
>                 ... 39 more
>
> Thanks,
> Kevin
>



-- 
-- Guozhang

Reply via email to