Re: [Dev] [Stratos] java.io.IOException: No space left on device in Docker containers
In your local setup go with non autoscaling mode, which is equal min and
max.
On Wed, Jul 15, 2015 at 9:30 AM, Punnadi Gunarathna
wrote:
> Hi Imesh,
>
> I had a chat with Reka and found out due to high loadAverage, more
> containers are spawned by AS. This causes the "No space left on device".
> Since I am working with 1 minion, at first there is 3 applications for
> dev,test and prod and when they started to scale the VM left with no space.
>
> On Wed, Jul 15, 2015 at 6:33 AM, Imesh Gunaratne wrote:
>
>> May be you need more disk space on each minion host.
>>
>> On Tue, Jul 14, 2015 at 12:00 PM, Punnadi Gunarathna
>> wrote:
>>
>>> Hi Manisha,
>>>
>>> No it can't be done with puppet scripts at the moment. I was referring
>>> to kubernets minion which had space issue. But sure I will have to figure
>>> out a way to solve this.
>>>
>>> On Tue, Jul 14, 2015 at 7:26 AM, Manisha Gayathri
>>> wrote:
>>>
If we are doing a new setup, we need to clean the old existing
instances, don't we?
Hope we can automate this step via puppet scripts.
Thanks
Manisha
On Mon, Jul 13, 2015 at 7:22 PM, Punnadi Gunarathna
wrote:
> Hi All,
>
> Despite cleaning pods via kubernetes client, I noticed there are
> number of old docker containers running on node-01 with below command:
>
> docker ps -a
>
>
> So I deleted them with below command:
>
> docker ps -a | grep '' | awk '{print $1}' | xargs
> --no-run-if-empty docker rm -f
>
> e.g:
>
> docker ps -a | grep '3 hours ago' | awk '{print $1}' | xargs
> --no-run-if-empty docker rm -f
>
> Hope this will solve the problem.
>
> On Mon, Jul 13, 2015 at 6:59 PM, Punnadi Gunarathna
> wrote:
>
>> hi All,
>>
>> I got below error in all the docker containers for the setup i just
>> did and haven't seen it before.
>>
>> In WSO2AS:
>>
>> TID: [2015-07-13 13:09:38,746] INFO
>> {org.wso2.carbon.server.extensions.PatchInstaller} - Patch changes
>> detected {org.wso2.carbon.server.extensions.PatchInstaller}
>> TID: [2015-07-13 13:09:57,267] INFO
>> {org.wso2.carbon.server.util.PatchUtils.console} - Backed up plugins to
>> patch {org.wso2.carbon.server.util.PatchUtils.console}
>> TID: [2015-07-13 13:10:05,378] ERROR
>> {org.wso2.carbon.server.extensions.PatchInstaller} - Error occurred
>> while
>> applying patches {org.wso2.carbon.server.extensions.PatchInstaller}
>> java.io.IOException: No space left on device
>> at java.io.FileOutputStream.writeBytes(Native Method)
>> at java.io.FileOutputStream.write(FileOutputStream.java:345)
>> at org.wso2.carbon.server.util.FileUtils.copy(FileUtils.java:230)
>> at
>> org.wso2.carbon.server.util.FileUtils.copyFile(FileUtils.java:152)
>> at
>> org.wso2.carbon.server.util.PatchUtils.copyServicepacksAndPatches(PatchUtils.java:90)
>> at
>> org.wso2.carbon.server.util.PatchUtils.applyServicepacksAndPatches(PatchUtils.java:45)
>> at
>> org.wso2.carbon.server.extensions.PatchInstaller.perform(PatchInstaller.java:58)
>> at org.wso2.carbon.server.Main.invokeExtensions(Main.java:152)
>> at org.wso2.carbon.server.Main.main(Main.java:94)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.wso2.carbon.bootstrap.Bootstrap.loadClass(Bootstrap.java:63)
>> at org.wso2.carbon.bootstrap.Bootstrap.main(Bootstrap.java:45)
>>
>> CA:
>> > [2015-07-13 13:10:29,565] DEBUG
>> {eventhandler.py:on_complete_tenant_event} - Processing Complete tenant
>> event...
>> > Traceback (most recent call last):
>> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
>> > self.flush()
>> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
>> > self.stream.flush()
>> > IOError: [Errno 28] No space left on device
>> > Logged from file eventhandler.py, line 221
>> > [2015-07-13 13:10:29,565] DEBUG
>> {eventhandler.py:on_complete_tenant_event} - Complete tenants:[]
>> > Traceback (most recent call last):
>> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
>> > self.flush()
>> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
>> > self.stream.flush()
>> > IOError: [Errno 28] No space left on device
>>
>>
>> Whenever I do a new setup, I always clean all the pods with
>> kubernetes client. Any thoughts on this?
>> --
>> Thanks and Regards,
>>
>> Punnadi Gunarathna
>> Senior
Re: [Dev] [Stratos] java.io.IOException: No space left on device in Docker containers
Hi Imesh,
I had a chat with Reka and found out due to high loadAverage, more
containers are spawned by AS. This causes the "No space left on device".
Since I am working with 1 minion, at first there is 3 applications for
dev,test and prod and when they started to scale the VM left with no space.
On Wed, Jul 15, 2015 at 6:33 AM, Imesh Gunaratne wrote:
> May be you need more disk space on each minion host.
>
> On Tue, Jul 14, 2015 at 12:00 PM, Punnadi Gunarathna
> wrote:
>
>> Hi Manisha,
>>
>> No it can't be done with puppet scripts at the moment. I was referring to
>> kubernets minion which had space issue. But sure I will have to figure out
>> a way to solve this.
>>
>> On Tue, Jul 14, 2015 at 7:26 AM, Manisha Gayathri
>> wrote:
>>
>>> If we are doing a new setup, we need to clean the old existing
>>> instances, don't we?
>>>
>>> Hope we can automate this step via puppet scripts.
>>>
>>> Thanks
>>> Manisha
>>>
>>> On Mon, Jul 13, 2015 at 7:22 PM, Punnadi Gunarathna
>>> wrote:
>>>
Hi All,
Despite cleaning pods via kubernetes client, I noticed there are number
of old docker containers running on node-01 with below command:
docker ps -a
So I deleted them with below command:
docker ps -a | grep '' | awk '{print $1}' | xargs
--no-run-if-empty docker rm -f
e.g:
docker ps -a | grep '3 hours ago' | awk '{print $1}' | xargs
--no-run-if-empty docker rm -f
Hope this will solve the problem.
On Mon, Jul 13, 2015 at 6:59 PM, Punnadi Gunarathna
wrote:
> hi All,
>
> I got below error in all the docker containers for the setup i just
> did and haven't seen it before.
>
> In WSO2AS:
>
> TID: [2015-07-13 13:09:38,746] INFO
> {org.wso2.carbon.server.extensions.PatchInstaller} - Patch changes
> detected {org.wso2.carbon.server.extensions.PatchInstaller}
> TID: [2015-07-13 13:09:57,267] INFO
> {org.wso2.carbon.server.util.PatchUtils.console} - Backed up plugins to
> patch {org.wso2.carbon.server.util.PatchUtils.console}
> TID: [2015-07-13 13:10:05,378] ERROR
> {org.wso2.carbon.server.extensions.PatchInstaller} - Error occurred while
> applying patches {org.wso2.carbon.server.extensions.PatchInstaller}
> java.io.IOException: No space left on device
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:345)
> at org.wso2.carbon.server.util.FileUtils.copy(FileUtils.java:230)
> at
> org.wso2.carbon.server.util.FileUtils.copyFile(FileUtils.java:152)
> at
> org.wso2.carbon.server.util.PatchUtils.copyServicepacksAndPatches(PatchUtils.java:90)
> at
> org.wso2.carbon.server.util.PatchUtils.applyServicepacksAndPatches(PatchUtils.java:45)
> at
> org.wso2.carbon.server.extensions.PatchInstaller.perform(PatchInstaller.java:58)
> at org.wso2.carbon.server.Main.invokeExtensions(Main.java:152)
> at org.wso2.carbon.server.Main.main(Main.java:94)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.wso2.carbon.bootstrap.Bootstrap.loadClass(Bootstrap.java:63)
> at org.wso2.carbon.bootstrap.Bootstrap.main(Bootstrap.java:45)
>
> CA:
> > [2015-07-13 13:10:29,565] DEBUG
> {eventhandler.py:on_complete_tenant_event} - Processing Complete tenant
> event...
> > Traceback (most recent call last):
> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
> > self.flush()
> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
> > self.stream.flush()
> > IOError: [Errno 28] No space left on device
> > Logged from file eventhandler.py, line 221
> > [2015-07-13 13:10:29,565] DEBUG
> {eventhandler.py:on_complete_tenant_event} - Complete tenants:[]
> > Traceback (most recent call last):
> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
> > self.flush()
> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
> > self.stream.flush()
> > IOError: [Errno 28] No space left on device
>
>
> Whenever I do a new setup, I always clean all the pods with kubernetes
> client. Any thoughts on this?
> --
> Thanks and Regards,
>
> Punnadi Gunarathna
> Senior Software Engineer,
> WSO2, Inc.; http://wso2.com
> Blog: http://hi-my-world.blogspot.com/
> Tel : 94 11 214 5345
> Fax :94 11 2145300
>
>
>
>
Re: [Dev] [Stratos] java.io.IOException: No space left on device in Docker containers
May be you need more disk space on each minion host.
On Tue, Jul 14, 2015 at 12:00 PM, Punnadi Gunarathna
wrote:
> Hi Manisha,
>
> No it can't be done with puppet scripts at the moment. I was referring to
> kubernets minion which had space issue. But sure I will have to figure out
> a way to solve this.
>
> On Tue, Jul 14, 2015 at 7:26 AM, Manisha Gayathri
> wrote:
>
>> If we are doing a new setup, we need to clean the old existing instances,
>> don't we?
>>
>> Hope we can automate this step via puppet scripts.
>>
>> Thanks
>> Manisha
>>
>> On Mon, Jul 13, 2015 at 7:22 PM, Punnadi Gunarathna
>> wrote:
>>
>>> Hi All,
>>>
>>> Despite cleaning pods via kubernetes client, I noticed there are number
>>> of old docker containers running on node-01 with below command:
>>>
>>> docker ps -a
>>>
>>>
>>> So I deleted them with below command:
>>>
>>> docker ps -a | grep '' | awk '{print $1}' | xargs
>>> --no-run-if-empty docker rm -f
>>>
>>> e.g:
>>>
>>> docker ps -a | grep '3 hours ago' | awk '{print $1}' | xargs
>>> --no-run-if-empty docker rm -f
>>>
>>> Hope this will solve the problem.
>>>
>>> On Mon, Jul 13, 2015 at 6:59 PM, Punnadi Gunarathna
>>> wrote:
>>>
hi All,
I got below error in all the docker containers for the setup i just did
and haven't seen it before.
In WSO2AS:
TID: [2015-07-13 13:09:38,746] INFO
{org.wso2.carbon.server.extensions.PatchInstaller} - Patch changes
detected {org.wso2.carbon.server.extensions.PatchInstaller}
TID: [2015-07-13 13:09:57,267] INFO
{org.wso2.carbon.server.util.PatchUtils.console} - Backed up plugins to
patch {org.wso2.carbon.server.util.PatchUtils.console}
TID: [2015-07-13 13:10:05,378] ERROR
{org.wso2.carbon.server.extensions.PatchInstaller} - Error occurred while
applying patches {org.wso2.carbon.server.extensions.PatchInstaller}
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at org.wso2.carbon.server.util.FileUtils.copy(FileUtils.java:230)
at
org.wso2.carbon.server.util.FileUtils.copyFile(FileUtils.java:152)
at
org.wso2.carbon.server.util.PatchUtils.copyServicepacksAndPatches(PatchUtils.java:90)
at
org.wso2.carbon.server.util.PatchUtils.applyServicepacksAndPatches(PatchUtils.java:45)
at
org.wso2.carbon.server.extensions.PatchInstaller.perform(PatchInstaller.java:58)
at org.wso2.carbon.server.Main.invokeExtensions(Main.java:152)
at org.wso2.carbon.server.Main.main(Main.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.wso2.carbon.bootstrap.Bootstrap.loadClass(Bootstrap.java:63)
at org.wso2.carbon.bootstrap.Bootstrap.main(Bootstrap.java:45)
CA:
> [2015-07-13 13:10:29,565] DEBUG
{eventhandler.py:on_complete_tenant_event} - Processing Complete tenant
event...
> Traceback (most recent call last):
> File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
> self.flush()
> File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
> self.stream.flush()
> IOError: [Errno 28] No space left on device
> Logged from file eventhandler.py, line 221
> [2015-07-13 13:10:29,565] DEBUG
{eventhandler.py:on_complete_tenant_event} - Complete tenants:[]
> Traceback (most recent call last):
> File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
> self.flush()
> File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
> self.stream.flush()
> IOError: [Errno 28] No space left on device
Whenever I do a new setup, I always clean all the pods with kubernetes
client. Any thoughts on this?
--
Thanks and Regards,
Punnadi Gunarathna
Senior Software Engineer,
WSO2, Inc.; http://wso2.com
Blog: http://hi-my-world.blogspot.com/
Tel : 94 11 214 5345
Fax :94 11 2145300
>>> Blog: http://hi-my-world.blogspot.com/
>>> Tel : 94 11 214 5345
>>> Fax :94 11 2145300
>>>
>>>
>>>
>>>
Re: [Dev] [Stratos] java.io.IOException: No space left on device in Docker containers
Hi Manisha,
No it can't be done with puppet scripts at the moment. I was referring to
kubernets minion which had space issue. But sure I will have to figure out
a way to solve this.
On Tue, Jul 14, 2015 at 7:26 AM, Manisha Gayathri wrote:
> If we are doing a new setup, we need to clean the old existing instances,
> don't we?
>
> Hope we can automate this step via puppet scripts.
>
> Thanks
> Manisha
>
> On Mon, Jul 13, 2015 at 7:22 PM, Punnadi Gunarathna
> wrote:
>
>> Hi All,
>>
>> Despite cleaning pods via kubernetes client, I noticed there are number
>> of old docker containers running on node-01 with below command:
>>
>> docker ps -a
>>
>>
>> So I deleted them with below command:
>>
>> docker ps -a | grep '' | awk '{print $1}' | xargs
>> --no-run-if-empty docker rm -f
>>
>> e.g:
>>
>> docker ps -a | grep '3 hours ago' | awk '{print $1}' | xargs
>> --no-run-if-empty docker rm -f
>>
>> Hope this will solve the problem.
>>
>> On Mon, Jul 13, 2015 at 6:59 PM, Punnadi Gunarathna
>> wrote:
>>
>>> hi All,
>>>
>>> I got below error in all the docker containers for the setup i just did
>>> and haven't seen it before.
>>>
>>> In WSO2AS:
>>>
>>> TID: [2015-07-13 13:09:38,746] INFO
>>> {org.wso2.carbon.server.extensions.PatchInstaller} - Patch changes
>>> detected {org.wso2.carbon.server.extensions.PatchInstaller}
>>> TID: [2015-07-13 13:09:57,267] INFO
>>> {org.wso2.carbon.server.util.PatchUtils.console} - Backed up plugins to
>>> patch {org.wso2.carbon.server.util.PatchUtils.console}
>>> TID: [2015-07-13 13:10:05,378] ERROR
>>> {org.wso2.carbon.server.extensions.PatchInstaller} - Error occurred while
>>> applying patches {org.wso2.carbon.server.extensions.PatchInstaller}
>>> java.io.IOException: No space left on device
>>> at java.io.FileOutputStream.writeBytes(Native Method)
>>> at java.io.FileOutputStream.write(FileOutputStream.java:345)
>>> at org.wso2.carbon.server.util.FileUtils.copy(FileUtils.java:230)
>>> at org.wso2.carbon.server.util.FileUtils.copyFile(FileUtils.java:152)
>>> at
>>> org.wso2.carbon.server.util.PatchUtils.copyServicepacksAndPatches(PatchUtils.java:90)
>>> at
>>> org.wso2.carbon.server.util.PatchUtils.applyServicepacksAndPatches(PatchUtils.java:45)
>>> at
>>> org.wso2.carbon.server.extensions.PatchInstaller.perform(PatchInstaller.java:58)
>>> at org.wso2.carbon.server.Main.invokeExtensions(Main.java:152)
>>> at org.wso2.carbon.server.Main.main(Main.java:94)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.wso2.carbon.bootstrap.Bootstrap.loadClass(Bootstrap.java:63)
>>> at org.wso2.carbon.bootstrap.Bootstrap.main(Bootstrap.java:45)
>>>
>>> CA:
>>> > [2015-07-13 13:10:29,565] DEBUG
>>> {eventhandler.py:on_complete_tenant_event} - Processing Complete tenant
>>> event...
>>> > Traceback (most recent call last):
>>> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
>>> > self.flush()
>>> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
>>> > self.stream.flush()
>>> > IOError: [Errno 28] No space left on device
>>> > Logged from file eventhandler.py, line 221
>>> > [2015-07-13 13:10:29,565] DEBUG
>>> {eventhandler.py:on_complete_tenant_event} - Complete tenants:[]
>>> > Traceback (most recent call last):
>>> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
>>> > self.flush()
>>> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
>>> > self.stream.flush()
>>> > IOError: [Errno 28] No space left on device
>>>
>>>
>>> Whenever I do a new setup, I always clean all the pods with kubernetes
>>> client. Any thoughts on this?
>>> --
>>> Thanks and Regards,
>>>
>>> Punnadi Gunarathna
>>> Senior Software Engineer,
>>> WSO2, Inc.; http://wso2.com
>>> Blog: http://hi-my-world.blogspot.com/
>>> Tel : 94 11 214 5345
>>> Fax :94 11 2145300
>>>
>>>
>>>
>>>
>> Blog: http://hi-my-world.blogspot.com/
>> Tel : 94 11 214 5345
>> Fax :94 11 2145300
>>
>>
>>
>>
Re: [Dev] [Stratos] java.io.IOException: No space left on device in Docker containers
If we are doing a new setup, we need to clean the old existing instances,
don't we?
Hope we can automate this step via puppet scripts.
Thanks
Manisha
On Mon, Jul 13, 2015 at 7:22 PM, Punnadi Gunarathna
wrote:
> Hi All,
>
> Despite cleaning pods via kubernetes client, I noticed there are number of
> old docker containers running on node-01 with below command:
>
> docker ps -a
>
>
> So I deleted them with below command:
>
> docker ps -a | grep '' | awk '{print $1}' | xargs
> --no-run-if-empty docker rm -f
>
> e.g:
>
> docker ps -a | grep '3 hours ago' | awk '{print $1}' | xargs
> --no-run-if-empty docker rm -f
>
> Hope this will solve the problem.
>
> On Mon, Jul 13, 2015 at 6:59 PM, Punnadi Gunarathna
> wrote:
>
>> hi All,
>>
>> I got below error in all the docker containers for the setup i just did
>> and haven't seen it before.
>>
>> In WSO2AS:
>>
>> TID: [2015-07-13 13:09:38,746] INFO
>> {org.wso2.carbon.server.extensions.PatchInstaller} - Patch changes
>> detected {org.wso2.carbon.server.extensions.PatchInstaller}
>> TID: [2015-07-13 13:09:57,267] INFO
>> {org.wso2.carbon.server.util.PatchUtils.console} - Backed up plugins to
>> patch {org.wso2.carbon.server.util.PatchUtils.console}
>> TID: [2015-07-13 13:10:05,378] ERROR
>> {org.wso2.carbon.server.extensions.PatchInstaller} - Error occurred while
>> applying patches {org.wso2.carbon.server.extensions.PatchInstaller}
>> java.io.IOException: No space left on device
>> at java.io.FileOutputStream.writeBytes(Native Method)
>> at java.io.FileOutputStream.write(FileOutputStream.java:345)
>> at org.wso2.carbon.server.util.FileUtils.copy(FileUtils.java:230)
>> at org.wso2.carbon.server.util.FileUtils.copyFile(FileUtils.java:152)
>> at
>> org.wso2.carbon.server.util.PatchUtils.copyServicepacksAndPatches(PatchUtils.java:90)
>> at
>> org.wso2.carbon.server.util.PatchUtils.applyServicepacksAndPatches(PatchUtils.java:45)
>> at
>> org.wso2.carbon.server.extensions.PatchInstaller.perform(PatchInstaller.java:58)
>> at org.wso2.carbon.server.Main.invokeExtensions(Main.java:152)
>> at org.wso2.carbon.server.Main.main(Main.java:94)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.wso2.carbon.bootstrap.Bootstrap.loadClass(Bootstrap.java:63)
>> at org.wso2.carbon.bootstrap.Bootstrap.main(Bootstrap.java:45)
>>
>> CA:
>> > [2015-07-13 13:10:29,565] DEBUG
>> {eventhandler.py:on_complete_tenant_event} - Processing Complete tenant
>> event...
>> > Traceback (most recent call last):
>> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
>> > self.flush()
>> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
>> > self.stream.flush()
>> > IOError: [Errno 28] No space left on device
>> > Logged from file eventhandler.py, line 221
>> > [2015-07-13 13:10:29,565] DEBUG
>> {eventhandler.py:on_complete_tenant_event} - Complete tenants:[]
>> > Traceback (most recent call last):
>> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
>> > self.flush()
>> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
>> > self.stream.flush()
>> > IOError: [Errno 28] No space left on device
>>
>>
>> Whenever I do a new setup, I always clean all the pods with kubernetes
>> client. Any thoughts on this?
>> --
>> Thanks and Regards,
>>
>> Punnadi Gunarathna
>> Senior Software Engineer,
>> WSO2, Inc.; http://wso2.com
>> Blog: http://hi-my-world.blogspot.com/
>> Tel : 94 11 214 5345
>> Fax :94 11 2145300
>>
>>
>>
>>
> Blog: http://hi-my-world.blogspot.com/
> Tel : 94 11 214 5345
> Fax :94 11 2145300
>
>
>
>
Re: [Dev] [Stratos] java.io.IOException: No space left on device in Docker containers
Hi All,
Despite cleaning pods via kubernetes client, I noticed there are number of
old docker containers running on node-01 with below command:
docker ps -a
So I deleted them with below command:
docker ps -a | grep '' | awk '{print $1}' | xargs
--no-run-if-empty docker rm -f
e.g:
docker ps -a | grep '3 hours ago' | awk '{print $1}' | xargs
--no-run-if-empty docker rm -f
Hope this will solve the problem.
On Mon, Jul 13, 2015 at 6:59 PM, Punnadi Gunarathna
wrote:
> hi All,
>
> I got below error in all the docker containers for the setup i just did
> and haven't seen it before.
>
> In WSO2AS:
>
> TID: [2015-07-13 13:09:38,746] INFO
> {org.wso2.carbon.server.extensions.PatchInstaller} - Patch changes
> detected {org.wso2.carbon.server.extensions.PatchInstaller}
> TID: [2015-07-13 13:09:57,267] INFO
> {org.wso2.carbon.server.util.PatchUtils.console} - Backed up plugins to
> patch {org.wso2.carbon.server.util.PatchUtils.console}
> TID: [2015-07-13 13:10:05,378] ERROR
> {org.wso2.carbon.server.extensions.PatchInstaller} - Error occurred while
> applying patches {org.wso2.carbon.server.extensions.PatchInstaller}
> java.io.IOException: No space left on device
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:345)
> at org.wso2.carbon.server.util.FileUtils.copy(FileUtils.java:230)
> at org.wso2.carbon.server.util.FileUtils.copyFile(FileUtils.java:152)
> at
> org.wso2.carbon.server.util.PatchUtils.copyServicepacksAndPatches(PatchUtils.java:90)
> at
> org.wso2.carbon.server.util.PatchUtils.applyServicepacksAndPatches(PatchUtils.java:45)
> at
> org.wso2.carbon.server.extensions.PatchInstaller.perform(PatchInstaller.java:58)
> at org.wso2.carbon.server.Main.invokeExtensions(Main.java:152)
> at org.wso2.carbon.server.Main.main(Main.java:94)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.wso2.carbon.bootstrap.Bootstrap.loadClass(Bootstrap.java:63)
> at org.wso2.carbon.bootstrap.Bootstrap.main(Bootstrap.java:45)
>
> CA:
> > [2015-07-13 13:10:29,565] DEBUG
> {eventhandler.py:on_complete_tenant_event} - Processing Complete tenant
> event...
> > Traceback (most recent call last):
> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
> > self.flush()
> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
> > self.stream.flush()
> > IOError: [Errno 28] No space left on device
> > Logged from file eventhandler.py, line 221
> > [2015-07-13 13:10:29,565] DEBUG
> {eventhandler.py:on_complete_tenant_event} - Complete tenants:[]
> > Traceback (most recent call last):
> > File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
> > self.flush()
> > File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
> > self.stream.flush()
> > IOError: [Errno 28] No space left on device
>
>
> Whenever I do a new setup, I always clean all the pods with kubernetes
> client. Any thoughts on this?
> --
> Thanks and Regards,
>
> Punnadi Gunarathna
> Senior Software Engineer,
> WSO2, Inc.; http://wso2.com
> Blog: http://hi-my-world.blogspot.com/
> Tel : 94 11 214 5345
> Fax :94 11 2145300
>
>
>
>
Blog: http://hi-my-world.blogspot.com/
Tel : 94 11 214 5345
Fax :94 11 2145300
[Dev] [Stratos] java.io.IOException: No space left on device in Docker containers
hi All,
I got below error in all the docker containers for the setup i just did and
haven't seen it before.
In WSO2AS:
TID: [2015-07-13 13:09:38,746] INFO
{org.wso2.carbon.server.extensions.PatchInstaller} - Patch changes
detected {org.wso2.carbon.server.extensions.PatchInstaller}
TID: [2015-07-13 13:09:57,267] INFO
{org.wso2.carbon.server.util.PatchUtils.console} - Backed up plugins to
patch {org.wso2.carbon.server.util.PatchUtils.console}
TID: [2015-07-13 13:10:05,378] ERROR
{org.wso2.carbon.server.extensions.PatchInstaller} - Error occurred while
applying patches {org.wso2.carbon.server.extensions.PatchInstaller}
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at org.wso2.carbon.server.util.FileUtils.copy(FileUtils.java:230)
at org.wso2.carbon.server.util.FileUtils.copyFile(FileUtils.java:152)
at
org.wso2.carbon.server.util.PatchUtils.copyServicepacksAndPatches(PatchUtils.java:90)
at
org.wso2.carbon.server.util.PatchUtils.applyServicepacksAndPatches(PatchUtils.java:45)
at
org.wso2.carbon.server.extensions.PatchInstaller.perform(PatchInstaller.java:58)
at org.wso2.carbon.server.Main.invokeExtensions(Main.java:152)
at org.wso2.carbon.server.Main.main(Main.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.wso2.carbon.bootstrap.Bootstrap.loadClass(Bootstrap.java:63)
at org.wso2.carbon.bootstrap.Bootstrap.main(Bootstrap.java:45)
CA:
> [2015-07-13 13:10:29,565] DEBUG
{eventhandler.py:on_complete_tenant_event} - Processing Complete tenant
event...
> Traceback (most recent call last):
> File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
> self.flush()
> File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
> self.stream.flush()
> IOError: [Errno 28] No space left on device
> Logged from file eventhandler.py, line 221
> [2015-07-13 13:10:29,565] DEBUG
{eventhandler.py:on_complete_tenant_event} - Complete tenants:[]
> Traceback (most recent call last):
> File "/usr/lib/python2.7/logging/__init__.py", line 874, in emit
> self.flush()
> File "/usr/lib/python2.7/logging/__init__.py", line 834, in flush
> self.stream.flush()
> IOError: [Errno 28] No space left on device
Whenever I do a new setup, I always clean all the pods with kubernetes
client. Any thoughts on this?
--
Thanks and Regards,
Punnadi Gunarathna
Senior Software Engineer,
WSO2, Inc.; http://wso2.com
Blog: http://hi-my-world.blogspot.com/
Tel : 94 11 214 5345
Fax :94 11 2145300
