applike-ss opened a new issue, #17789:
URL: https://github.com/apache/druid/issues/17789
Druid fails to start when a memcached host can not be resolved.
### Affected Version
v32.0.0
### Description
I am setting up a 2-layer cache on my test druid system.
When setting it up, i had a typo in my memcached hosts config, so the
address was not resolvable.
The result was that druid wouldn't start up.
This exact behavior would also happen, when a memcached pod would get
re-scheduled at the same time that a druid pod is re-scheduled.
That makes it undesirable to have druid not starting up when remote cache is
not available.
Under normal circumstances the memcached would be available very quickly
again, however there can be issues like volumes not attachable when the cache
would not be available quickly again.
Please include as much detailed information about the problem as possible.
- Cluster size: Doesn't matter
- Configurations in use
- Steps to reproduce the problem:
- configure hybrid mode cache with memcached as L2
- set dns to something that is not resolvable
- see error in logs
- The error message or stack traces encountered. Providing more context,
such as nearby log messages or even entire logs, can be helpful.
- Any debugging that you have already done
logs:
```
Exception in thread "main" java.lang.RuntimeException:
com.google.inject.CreationException: Unable to create injector, see the
following errors:
1) Error in custom provider, java.nio.channels.UnresolvedAddressException
while locating org.apache.druid.client.cache.CacheProvider annotated with
interface org.apache.druid.guice.annotations.Global
at org.apache.druid.guice.CacheModule.configure(CacheModule.java:51) (via
modules: com.google.inject.util.Modules$OverrideModule ->
com.google.inject.util.Modules$OverrideModule ->
org.apache.druid.cli.CliHistorical$$Lambda$211/0x0000001001346d38 ->
org.apache.druid.guice.CacheModule)
while locating org.apache.druid.client.cache.Cache
for the 1st parameter of
org.apache.druid.client.cache.CacheMonitor.setCache(CacheMonitor.java:48)
while locating org.apache.druid.client.cache.CacheMonitor
at
org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:112)
(via modules: com.google.inject.util.Modules$OverrideModule ->
com.google.inject.util.Modules$OverrideModule ->
org.apache.druid.server.metrics.MetricsModule)
at
org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:112)
(via modules: com.google.inject.util.Modules$OverrideModule ->
com.google.inject.util.Modules$OverrideModule ->
org.apache.druid.server.metrics.MetricsModule)
while locating org.apache.druid.java.util.metrics.MonitorScheduler
at
org.apache.druid.server.metrics.MetricsModule.configure(MetricsModule.java:97)
(via modules: com.google.inject.util.Modules$OverrideModule ->
com.google.inject.util.Modules$OverrideModule ->
org.apache.druid.server.metrics.MetricsModule)
while locating org.apache.druid.java.util.metrics.MonitorScheduler
annotated with @com.google.inject.name.Named(value=ForTheEagerness)
1 error
at
org.apache.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:88)
at org.apache.druid.cli.ServerRunnable.run(ServerRunnable.java:69)
at org.apache.druid.cli.Main.main(Main.java:112)
Caused by: com.google.inject.CreationException: Unable to create injector,
see the following errors:
1) Error in custom provider, java.nio.channels.UnresolvedAddressException
while locating org.apache.druid.client.cache.CacheProvider annotated with
interface org.apache.druid.guice.annotations.Global
at org.apache.druid.guice.CacheModule.configure(CacheModule.java:51) (via
modules: com.google.inject.util.Modules$OverrideModule ->
com.google.inject.util.Modules$OverrideModule ->
org.apache.druid.cli.CliHistorical$$Lambda$211/0x0000001001346d38 ->
org.apache.druid.guice.CacheModule)
while locating org.apache.druid.client.cache.Cache
for the 1st parameter of
org.apache.druid.client.cache.CacheMonitor.setCache(CacheMonitor.java:48)
while locating org.apache.druid.client.cache.CacheMonitor
at
org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:112)
(via modules: com.google.inject.util.Modules$OverrideModule ->
com.google.inject.util.Modules$OverrideModule ->
org.apache.druid.server.metrics.MetricsModule)
at
org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:112)
(via modules: com.google.inject.util.Modules$OverrideModule ->
com.google.inject.util.Modules$OverrideModule ->
org.apache.druid.server.metrics.MetricsModule)
while locating org.apache.druid.java.util.metrics.MonitorScheduler
at
org.apache.druid.server.metrics.MetricsModule.configure(MetricsModule.java:97)
(via modules: com.google.inject.util.Modules$OverrideModule ->
com.google.inject.util.Modules$OverrideModule ->
org.apache.druid.server.metrics.MetricsModule)
while locating org.apache.druid.java.util.metrics.MonitorScheduler
annotated with @com.google.inject.name.Named(value=ForTheEagerness)
1 error
at
com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:470)
at
com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)
at
com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
at com.google.inject.Guice.createInjector(Guice.java:99)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at
org.apache.druid.initialization.ExtensionInjectorBuilder.build(ExtensionInjectorBuilder.java:49)
at
org.apache.druid.initialization.ServerInjectorBuilder.build(ServerInjectorBuilder.java:118)
at
org.apache.druid.initialization.ServerInjectorBuilder.makeServerInjector(ServerInjectorBuilder.java:73)
at
org.apache.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:85)
... 2 more
Caused by: java.nio.channels.UnresolvedAddressException
at java.base/sun.nio.ch.Net.checkAddress(Net.java:149)
at java.base/sun.nio.ch.Net.checkAddress(Net.java:157)
at
java.base/sun.nio.ch.SocketChannelImpl.checkRemote(SocketChannelImpl.java:816)
at
java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:839)
at
net.spy.memcached.MemcachedConnection.createConnections(MemcachedConnection.java:479)
at
net.spy.memcached.MemcachedConnection.<init>(MemcachedConnection.java:369)
at
net.spy.memcached.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:253)
at
net.spy.memcached.MemcachedClient.setupConnection(MemcachedClient.java:350)
at net.spy.memcached.MemcachedClient.<init>(MemcachedClient.java:297)
at net.spy.memcached.MemcachedClient.<init>(MemcachedClient.java:229)
at
org.apache.druid.client.cache.MemcachedCache$5.get(MemcachedCache.java:365)
at
org.apache.druid.client.cache.MemcachedCache$5.get(MemcachedCache.java:360)
at
org.apache.druid.client.cache.MemcacheClientPool.<init>(MemcacheClientPool.java:68)
at
org.apache.druid.client.cache.MemcachedCache.create(MemcachedCache.java:358)
at
org.apache.druid.client.cache.MemcachedCacheProvider.get(MemcachedCacheProvider.java:27)
at
org.apache.druid.client.cache.MemcachedCacheProvider.get(MemcachedCacheProvider.java:22)
at
org.apache.druid.client.cache.HybridCacheProvider.get(HybridCacheProvider.java:51)
at
org.apache.druid.client.cache.HybridCacheProvider.get(HybridCacheProvider.java:27)
at
com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)
at
com.google.inject.internal.BoundProviderFactory.provision(BoundProviderFactory.java:72)
at
com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:61)
at
com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:62)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at org.apache.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68)
at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
at
com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
at
com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
at
com.google.inject.internal.SingleMethodInjector.inject(SingleMethodInjector.java:82)
at
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:132)
at
com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:120)
at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)
at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)
at
com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1019)
at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1015)
at
com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1054)
at
org.apache.druid.server.metrics.MetricsModule.getMonitorScheduler(MetricsModule.java:118)
at
org.apache.druid.server.metrics.MetricsModule$$FastClassByGuice$$99ddce1b.invoke(<generated>)
at
com.google.inject.internal.ProviderMethod$FastClassProviderMethod.doProvision(ProviderMethod.java:264)
at
com.google.inject.internal.ProviderMethod$Factory.provision(ProviderMethod.java:401)
at
com.google.inject.internal.ProviderMethod$Factory.get(ProviderMethod.java:376)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at org.apache.druid.guice.LifecycleScope$1.get(LifecycleScope.java:68)
at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at
com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)
at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
at
com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:205)
at
com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:199)
at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)
at
com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:199)
at
com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:180)
... 10 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
