Hi Javier, thank you for the prompt response! I attach the log file of the execution.
Before the initialization of the container df output was: # df -hT # just printing the root fs here /dev/disk/by-label/cloudimg-rootfs ext4 20G 4.0G 16G 21% / and /var/lib/docker lies into the same partition. the container execution fails with the same output (as you can also see from the log file). df after the execution is: # df -hT # just printing the root fs here /dev/disk/by-label/cloudimg-rootfs ext4 20G 12G 8.0G 60% / I can verify that the VM has enough disk (20G) and memory (24G). Best, Giannis On 04/08/2016 03:07 PM, Javier Picorel wrote: > Hi Giannis, > > First of all, thanks for your interest in CloudSuite. > > I believe you are running out of disk space in your host machine. The index > occupies around 12GB > of space. You can check the disk occupancy of your machine while the > container is running by executing > the “df” command in your host machine. You should be able to see whether you > are running out of space > or not. Check that and let me know. > > Regards, > Javier > >> On 07 Apr 2016, at 14:39, Giannis Giannakopoulos <[email protected]> >> wrote: >> >> Hello, >> >> I have a problem with the execution of the web search benchmark. I am >> executing: >> >> docker run -it --name server -p 8983:8983 cloudsuite/web-search:server 12g 1 >> >> on a 16G machine with 20G disk and I get the following output: >> >> tar: data/index/_ne.fdt: Cannot write: No space left on device >> (during wget) >> and >> >> log4j:ERROR Failed to flush writer, >> java.io.IOException: No space left on device >> at java.io.FileOutputStream.writeBytes(Native Method) >> at java.io.FileOutputStream.write(FileOutputStream.java:345) >> (when trying to start solr) >> >> (I provide the full log trace into the attachment). >> >> Thank you in advance, >> Giannis >> >> -- >> Ioannis Giannakopoulos >> PhD Candidate, Systems Administrator >> Computing Systems Laboratory, School of ECE >> National Technical University of Athens >> e-mail: [email protected] >> site: http://www.cslab.ece.ntua.gr/~ggian/ >> skype name: giagiannis >> tel: +302107721544 (int. 200) >> <trace.log> -- Ioannis Giannakopoulos PhD Candidate, Systems Administrator Computing Systems Laboratory, School of ECE National Technical University of Athens e-mail: [email protected] site: http://www.cslab.ece.ntua.gr/~ggian/ skype name: giagiannis tel: +302107721544 (int. 200)
Waiting to see Solr listening on port 8983 [\]
Started Solr server on port 8983 (pid=63). Happy searching!
Found 1 Solr nodes:
Solr process 63 running on port 8983
{
"solr_home":"/usr/src/solr_cores/",
"version":"5.2.1 1684708 - shalin - 2015-06-10 23:20:13",
"startTime":"2016-04-08T13:19:40.815Z",
"uptime":"0 days, 0 hours, 0 minutes, 21 seconds",
"memory":"1 GB (%8.7) of 11.5 GB",
"cloud":{
"ZooKeeper":"localhost:9983",
"liveNodes":"1",
"collections":"0"}}
Connecting to ZooKeeper at localhost:9983
Uploading /usr/src/solr-5.2.1/server/solr/configsets/basic_configs/conf for config cloudsuite_web_search to ZooKeeper at localhost:9983
Creating new collection 'cloudsuite_web_search' using command:
http://172.17.0.1:8983/solr/admin/collections?action=CREATE&name=cloudsuite_web_search&numShards=1&replicationFactor=1&maxShardsPerNode=1&collection.configName=cloudsuite_web_search
{
"responseHeader":{
"status":0,
"QTime":5213},
"success":{"":{
"responseHeader":{
"status":0,
"QTime":3993},
"core":"cloudsuite_web_search_shard1_replica1"}}}
--2016-04-08 13:20:11-- http://lsi-www.epfl.ch/parsa/index
Resolving lsi-www.epfl.ch (lsi-www.epfl.ch)... 128.178.72.70
Connecting to lsi-www.epfl.ch (lsi-www.epfl.ch)|128.178.72.70|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12093085208 (11G) [application/octet-stream]
Saving to: 'STDOUT'
0% [ ] 0 --.-K/s data/
data/index/
data/index/_ne.nvd
0% [ ] 3,722,288 4.41MB/s data/index/write.lock
data/index/_ne.fnm
data/index/_ne.fdx
0% [ ] 6,064,304 5.76MB/s data/index/_ne_Lucene50_0.pos
34% [==========================================================> ] 4,204,402,632 11.2MB/s eta 11m 20sdata/index/_ne_Lucene50_0.pay
54% [===========================================================================================> ] 6,584,075,664 11.2MB/s eta 7m 57s data/index/_ne.si
data/index/_ne.nvm
data/index/_ne_Lucene50_0.doc
58% [==================================================================================================> ] 7,104,280,984 11.2MB/s eta 7m 12s tar: data/index/_ne_Lucene50_0.doc: Wrote only 9216 of 10240 bytes
66% [================================================================================================================> ] 8,075,458,640 11.2MB/s eta 5m 48s data/index/_ne.fdt
tar: data/index/_ne.fdt: Cannot write: No space left on device
96% [=================================================================================================================================================================> ] 11,653,002,120 11.2MB/s eta 38s data/index/_ne_Lucene50_0.tim
tar: data/index/_ne_Lucene50_0.tim: Cannot write: No space left on device
99% [=======================================================================================================================================================================> ] 12,082,826,168 11.2MB/s eta 2s data/index/segments_an
tar: data/index/segments_an: Cannot write: No space left on device
data/index/_ne_Lucene50_0.tip
tar: data/index/_ne_Lucene50_0.tip: Cannot write: No space left on device
99% [=======================================================================================================================================================================> ] 12,092,190,192 11.2MB/s eta 1s data/tlog/
data/tlog/tlog.0000000000000000380
tar: data/tlog/tlog.0000000000000000380: Cannot write: No space left on device
100%[========================================================================================================================================================================>] 12,093,085,208 11.2MB/s in 17m 24s
2016-04-08 13:37:35 (11.1 MB/s) - written to stdout [12093085208/12093085208]
tar: Exiting with failure status due to previous errors
=================================
Index Node IP Address: 172.17.0.1
=================================
Starting Solr in SolrCloud mode on port 8983 from /usr/src/solr-5.2.1/server
OpenJDK 64-Bit Server VM warning: Insufficient space for shared memory file:
518
Try using the -Djava.io.tmpdir= option to select an alternate temp location.
log4j:ERROR Failed to flush writer,
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.RollingFileAppender.subAppend(RollingFileAppender.java:276)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.log(Log4jLoggerAdapter.java:601)
at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:620)
at org.eclipse.jetty.util.log.JettyAwareLogger.info(JettyAwareLogger.java:314)
at org.eclipse.jetty.util.log.Slf4jLog.info(Slf4jLog.java:74)
at org.eclipse.jetty.util.log.Log.initialized(Log.java:186)
at org.eclipse.jetty.util.log.Log.getLogger(Log.java:288)
at org.eclipse.jetty.util.log.Log.getLogger(Log.java:278)
at org.eclipse.jetty.xml.XmlConfiguration.<clinit>(XmlConfiguration.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:321)
at org.eclipse.jetty.start.Main.start(Main.java:817)
at org.eclipse.jetty.start.Main.main(Main.java:112)
0 [main] INFO org.eclipse.jetty.util.log [ ] ? Logging initialized @396ms
236 [main] INFO org.eclipse.jetty.server.Server [ ] ? jetty-9.2.10.v20150310
273 [main] WARN org.eclipse.jetty.server.handler.RequestLogHandler [ ] ? !RequestLog
277 [main] INFO org.eclipse.jetty.deploy.providers.ScanningAppProvider [ ] ? Deployment monitor [file:/usr/src/solr-5.2.1/server/contexts/] at interval 0
852 [main] INFO org.eclipse.jetty.webapp.StandardDescriptorProcessor [ ] ? NO JSP Support for /solr, did not find org.apache.jasper.servlet.JspServlet
883 [main] WARN org.eclipse.jetty.security.SecurityHandler [ ] ? [email protected]@614c97d4{/solr,file:/usr/src/solr-5.2.1/server/solr-webapp/webapp/,STARTING}{/solr.war} has uncovered http methods for path: /
920 [main] INFO org.apache.solr.servlet.SolrDispatchFilter [ ] ? SolrDispatchFilter.init()WebAppClassLoader=1631310339@613bd203
942 [main] INFO org.apache.solr.core.SolrResourceLoader [ ] ? JNDI not configured for solr (NoInitialContextEx)
943 [main] INFO org.apache.solr.core.SolrResourceLoader [ ] ? using system property solr.solr.home: /usr/src/solr_cores
944 [main] INFO org.apache.solr.core.SolrResourceLoader [ ] ? new SolrResourceLoader for directory: '/usr/src/solr_cores/'
1110 [main] INFO org.apache.solr.core.SolrXmlConfig [ ] ? Loading container configuration from /usr/src/solr_cores/solr.xml
1219 [main] INFO org.apache.solr.core.CoresLocator [ ] ? Config-defined core root directory: /usr/src/solr_cores
1251 [main] INFO org.apache.solr.core.CoreContainer [ ] ? New CoreContainer 1876800513
1251 [main] INFO org.apache.solr.core.CoreContainer [ ] ? Loading cores into CoreContainer [instanceDir=/usr/src/solr_cores/]
1252 [main] INFO org.apache.solr.core.CoreContainer [ ] ? loading shared library: /usr/src/solr_cores/lib
1253 [main] WARN org.apache.solr.core.SolrResourceLoader [ ] ? Can't find (or read) directory to add to classloader: lib (resolved as: /usr/src/solr_cores/lib).
1283 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory [ ] ? created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
1741 [main] INFO org.apache.solr.update.UpdateShardHandler [ ] ? Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
1744 [main] INFO org.apache.solr.logging.LogWatcher [ ] ? SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
1747 [main] INFO org.apache.solr.logging.LogWatcher [ ] ? Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
1749 [main] INFO org.apache.solr.core.CoreContainer [ ] ? Node Name:
1756 [main] INFO org.apache.solr.cloud.SolrZkServer [ ] ? STARTING EMBEDDED STANDALONE ZOOKEEPER SERVER at port 9983
2257 [main] INFO org.apache.solr.core.ZkContainer [ ] ? Zookeeper client=localhost:9983
2300 [main] INFO org.apache.solr.common.cloud.ConnectionManager [ ] ? Waiting for client to connect to ZooKeeper
2317 [SyncThread:0] ERROR org.apache.zookeeper.server.SyncRequestProcessor [ ] ? Severe unrecoverable error, exiting
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at org.apache.zookeeper.server.persistence.FileTxnLog.append(FileTxnLog.java:211)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.append(FileTxnSnapLog.java:314)
at org.apache.zookeeper.server.ZKDatabase.append(ZKDatabase.java:476)
at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:140)
signature.asc
Description: OpenPGP digital signature
