Thanks for the feedback Andrei, In my case the problem seems unrelated to S3 as when I configure NFS as my secondary storage I get the same problem. My problem seems to be that the system tries to write the snapshot to directory path like /mnt/e5c41.../snapshots/2/3/ but as for some reason it finds that /mnt/e5c41.../snapshots/2/3/ doesn't exist as the /mnt/e5c41... is not actually mounting secondary storage at that time. If I mount it manually, the path is there but the system is not doing it automatically for me.
Tom. On Tue, 2014-04-01 at 22:41 +0100, Andrei Mikhailovsky wrote: > From what I remember, there was an issue with S3 and snapshots. > > I might be mixing version numbers, but it could be related to the size of the > image. In the past, there was no support for multi-part upload I was not able > to snapshot anything larger than 5gb. Not sure if this is the same problem > that you are experiencing. > > I would suggest trying 4.3 and letting us know how it works for you. > > I am planning to upgrade to 4.3 and test snapshotting with nfs / s3 / ceph. > Will update on the progress. > > Andrei > > > ----- Original Message ----- > From: "Thomas O'Dowd" <tpod...@cloudian.com> > To: users@cloudstack.apache.org > Sent: Tuesday, 1 April, 2014 6:44:28 AM > Subject: Re: Snapshot exceptions on 4.2.1 with NFS and S3 > > Anyone any hints on this? I've installed it a few times and always have > the same issue. Wondering if it's some configuration problem or a known > bug. > > Thanks, > > Tom. > > On Thu, 2014-03-27 at 11:55 +0900, Thomas O'Dowd wrote: > > I remember this used to work on Cloudstack 4.2 when I was testing it > > with devcloud but with CloudPlatform 4.2.1 and KVM I can't get snapshots > > to work. > > > > I've setup a single node using KVM and management server, db and agent. > > > > I can start an instance using a template stored on NFS secondary > > storage. If I stop the instance and goto storage, I can see it's disk > > and then I can select "take snapshot". This always seems to fail for me > > and I'm not sure why. > > > > The error seems to be: > > Failed to create snapshot > > com.cloud.utils.exception.CloudRuntimeException: Failed to > > backup 4d3ea6fa-6902-4037-be67-166e34575ee9 for > > disk > > /mnt/8a2181f7-86a0-3d7d-a5be-10caa89b3d27/f0615600-d806-486e-bbea-d408de0225e4 > > to /mnt/e5c41d36-90d2-3d5e-bdbf-8a0140dd330e/snapshots/2/3 > > > > /mnt/8a218... is mounted fine if I look at df. > > /mnt/e5c41... is not mounted. > > > > If I browse to /mnt, the e5c41... directory (used for mounting) exists. > > If I manually mount it, I can see the > > directory /mnt/e5c41.../snapshots/2/3 so it exists. If I leave it > > mounted and try to take a snapshot again, it fails again with the same > > error and again when I check the /mnt/e5c41... directory is not mounted. > > > > As I mentioned above, the NFS server I'm using is actually the same > > node. I'm using KVM but I don't think that is causing the issue? I'm > > using CentOS 6.4. > > > > If I setup S3 as secondary storage and use NFS as the staging server, I > > get the same error. Any hints on what I am doing wrong is greatly > > appreciated. > > > > Here is more of the exception log: > > 2014-03-24 16:12:33,450 DEBUG [cloud.api.ApiServlet] > > (catalina-exec-10:null) ===START=== 10.181.164.198 -- GET > > command=createSnapshot&volumeid=96f59602-8a84-4731-91a5-218d4f45ee70&response=json&sessionkey=TnWYMJmMIZRly5DfGPNX%2BpZeT%2BI%3D&_=1395645153401 > > 2014-03-24 16:12:33,819 DEBUG [cloud.async.AsyncJobManagerImpl] > > (catalina-exec-10:null) submit async job-10 = [ > > 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ], details: AsyncJobVO {id:10, userId: > > 2, accountId: 2, sessionKey: null, instanceType: Snapshot, instanceId: 1, > > cmd: org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd, > > cmdOriginator: null, cmdInfo: > > {"id":"1","response":"json","sessionkey":"TnWYMJmMIZRly5DfGPNX+pZeT+I\u003d","cmdEventType":"SNAPSHOT.CREATE","ctxUserId":"2","httpmethod":"GET","_":"1395645153401","volumeid":"96f59602-8a84-4731-91a5-218d4f45ee70","ctxAccountId":"2","ctxStartEventId":"45"}, > > cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, > > processStatus: 0, resultCode: 0, result: null, initMsid: 143618408992, > > completeMsid: null, lastUpdated: null, lastPolled: null, created: null} > > 2014-03-24 16:12:33,820 DEBUG [cloud.async.AsyncJobManagerImpl] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) > > Executing org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd > > for job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ] > > 2014-03-24 16:12:33,821 DEBUG [cloud.api.ApiServlet] > > (catalina-exec-10:null) ===END=== 10.181.164.198 -- GET > > command=createSnapshot&volumeid=96f59602-8a84-4731-91a5-218d4f45ee70&response=json&sessionkey=TnWYMJmMIZRly5DfGPNX%2BpZeT%2BI%3D&_=1395645153401 > > 2014-03-24 16:12:33,847 INFO [user.snapshot.CreateSnapshotCmd] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) VOLSS: > > createSnapshotCmd starts:1395645153847 > > 2014-03-24 16:12:34,179 DEBUG [agent.transport.Request] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Seq > > 1-249561236: Sending { Cmd , MgmtId: 143618408992, via: 1, Ver: v1, Flags: > > 100011, > > [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"96f59602-8a84-4731-91a5-218d4f45ee70","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8a2181f7-86a0-3d7d-a5be-10caa89b3d27","id":1,"poolType":"NetworkFilesystem","host":"10.181.168.230","path":"/home/primary","port":2049}},"name":"ROOT-3","size":13147648,"path":"f0615600-d806-486e-bbea-d408de0225e4","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"format":"QCOW2","id":3,"hypervisorType":"KVM"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8a2181f7-86a0-3d7d-a5be-10caa89b3d27","id":1,"poolType":"NetworkFilesystem","host":"10.181.168.230","path":"/home/primary","port":2049}},"vmName":"i-2-3-VM","name":"test1_ROOT-3_20140324071233","hypervisorType":"KVM","id":1}},"wait":0}}] > > } > > 2014-03-24 16:12:34,374 DEBUG [agent.transport.Request] > > (AgentManager-Handler-8:null) Seq 1-249561236: Processing: { Ans: , > > MgmtId: 143618408992, via: 1, Ver: v1, Flags: 10, > > [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/8a2181f7-86a0-3d7d-a5be-10caa89b3d27/f0615600-d806-486e-bbea-d408de0225e4/4d3ea6fa-6902-4037-be67-166e34575ee9","id":0}},"result":true,"wait":0}}] > > } > > 2014-03-24 16:12:34,374 DEBUG [agent.transport.Request] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Seq > > 1-249561236: Received: { Ans: , MgmtId: 143618408992, via: 1, Ver: v1, > > Flags: 10, { CreateObjectAnswer } } > > 2014-03-24 16:12:34,784 DEBUG > > [storage.motion.AncientDataMotionStrategy] (Job-Executor-10:job-10 = [ > > 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) copyAsync inspecting src type > > SNAPSHOT copyAsync inspecting dest type SNAPSHOT > > 2014-03-24 16:12:34,802 DEBUG > > [storage.motion.AncientDataMotionStrategy] (Job-Executor-10:job-10 = [ > > 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) needCacheStorage true, dest at > > snapshots/2/3 dest role > > Image/mnt/8a2181f7-86a0-3d7d-a5be-10caa89b3d27/f0615600-d806-486e-bbea-d408de0225e4/4d3ea6fa-6902-4037-be67-166e34575ee9 > > src role Primary > > 2014-03-24 16:12:34,938 DEBUG [agent.transport.Request] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Seq > > 1-249561237: Sending { Cmd , MgmtId: 143618408992, via: 1, Ver: v1, Flags: > > 100111, > > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/8a2181f7-86a0-3d7d-a5be-10caa89b3d27/f0615600-d806-486e-bbea-d408de0225e4/4d3ea6fa-6902-4037-be67-166e34575ee9","volume":{"uuid":"96f59602-8a84-4731-91a5-218d4f45ee70","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8a2181f7-86a0-3d7d-a5be-10caa89b3d27","id":1,"poolType":"NetworkFilesystem","host":"10.181.168.230","path":"/home/primary","port":2049}},"name":"ROOT-3","size":13147648,"path":"f0615600-d806-486e-bbea-d408de0225e4","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"format":"QCOW2","id":3,"hypervisorType":"KVM"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8a2181f7-86a0-3d7d-a5be-10caa89b3d27","id":1,"poolType":"NetworkFilesystem","host":"10.181.168.230","path":"/home/primary","port":2049}},"vmName":"i-2-3-VM","name":"test1_ROOT-3_20140324071233","hypervisorType":"KVM","id":1}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/3","volume":{"uuid":"96f59602-8a84-4731-91a5-218d4f45ee70","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8a2181f7-86a0-3d7d-a5be-10caa89b3d27","id":1,"poolType":"NetworkFilesystem","host":"10.181.168.230","path":"/home/primary","port":2049}},"name":"ROOT-3","size":13147648,"path":"f0615600-d806-486e-bbea-d408de0225e4","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"format":"QCOW2","id":3,"hypervisorType":"KVM"},"dataStore":{"com.cloud.agent.api.to.S3TO":{"id":2,"uuid":"c18a1785-f578-4102-bb94-0298289786e0","endPoint":"s3.cloudian.com:18080","bucketName":"cloudstack","httpsFlag":false,"created":"Mar > > 24, 2014 3:39:20 > > PM","enableRRS":false,"maxSingleUploadSizeInBytes":5368709120}},"vmName":"i-2-3-VM","name":"test1_ROOT-3_20140324071233","hypervisorType":"KVM","id":1}},"cacheTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/3","volume":{"uuid":"96f59602-8a84-4731-91a5-218d4f45ee70","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8a2181f7-86a0-3d7d-a5be-10caa89b3d27","id":1,"poolType":"NetworkFilesystem","host":"10.181.168.230","path":"/home/primary","port":2049}},"name":"ROOT-3","size":13147648,"path":"f0615600-d806-486e-bbea-d408de0225e4","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"format":"QCOW2","id":3,"hypervisorType":"KVM"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.181.168.230/home/staging","_role":"ImageCache"}},"vmName":"i-2-3-VM","name":"test1_ROOT-3_20140324071233","hypervisorType":"KVM","id":1}},"executeInSequence":true,"wait":21600}}] > > } > > 2014-03-24 16:12:35,090 DEBUG [agent.transport.Request] > > (AgentManager-Handler-7:null) Seq 1-249561237: Processing: { Ans: , > > MgmtId: 143618408992, via: 1, Ver: v1, Flags: 110, > > [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Failed > > to backup 4d3ea6fa-6902-4037-be67-166e34575ee9 for disk > > /mnt/8a2181f7-86a0-3d7d-a5be-10caa89b3d27/f0615600-d806-486e-bbea-d408de0225e4 > > to /mnt/e5c41d36-90d2-3d5e-bdbf-8a0140dd330e/snapshots/2/3","wait":0}}] } > > 2014-03-24 16:12:35,090 DEBUG [agent.manager.AgentAttache] > > (AgentManager-Handler-7:null) Seq 1-249561237: No more commands found > > 2014-03-24 16:12:35,090 DEBUG [agent.transport.Request] > > (AgentManager-Handler-7:null) Seq 1-249561237: Processing: { Ans: , > > MgmtId: 143618408992, via: 1, Ver: v1, Flags: 110, > > [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Failed > > to backup 4d3ea6fa-6902-4037-be67-166e34575ee9 for disk > > /mnt/8a2181f7-86a0-3d7d-a5be-10caa89b3d27/f0615600-d806-486e-bbea-d408de0225e4 > > to /mnt/e5c41d36-90d2-3d5e-bdbf-8a0140dd330e/snapshots/2/3","wait":0}}] } > > 2014-03-24 16:12:35,090 DEBUG [agent.manager.AgentAttache] > > (AgentManager-Handler-7:null) Seq 1-249561237: No more commands found > > 2014-03-24 16:12:35,090 DEBUG [agent.transport.Request] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Seq > > 1-249561237: Received: { Ans: , MgmtId: 143618408992, via: 1, Ver: v1, > > Flags: 110, { CopyCmdAnswer } } > > 2014-03-24 16:12:35,098 DEBUG [agent.transport.Request] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Seq > > 3-1564409959: Sending { Cmd , MgmtId: 143618408992, via: 3, Ver: v1, > > Flags: 100011, > > [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/3","volume":{"uuid":"96f59602-8a84-4731-91a5-218d4f45ee70","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8a2181f7-86a0-3d7d-a5be-10caa89b3d27","id":1,"poolType":"NetworkFilesystem","host":"10.181.168.230","path":"/home/primary","port":2049}},"name":"ROOT-3","size":13147648,"path":"f0615600-d806-486e-bbea-d408de0225e4","volumeId":3,"vmName":"i-2-3-VM","accountId":2,"format":"QCOW2","id":3,"hypervisorType":"KVM"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.181.168.230/home/staging","_role":"ImageCache"}},"vmName":"i-2-3-VM","name":"test1_ROOT-3_20140324071233","hypervisorType":"KVM","id":1}},"wait":0}}] > > } > > 2014-03-24 16:12:35,260 DEBUG [agent.transport.Request] > > (AgentManager-Handler-9:null) Seq 3-1564409959: Processing: { Ans: , > > MgmtId: 143618408992, via: 3, Ver: v1, Flags: 10, > > [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] } > > 2014-03-24 16:12:35,260 DEBUG [agent.transport.Request] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Seq > > 3-1564409959: Received: { Ans: , MgmtId: 143618408992, via: 3, Ver: v1, > > Flags: 10, { Answer } } > > 2014-03-24 16:12:35,468 DEBUG > > [storage.snapshot.SnapshotManagerImpl] (Job-Executor-10:job-10 = [ > > 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Failed to create snapshot > > com.cloud.utils.exception.CloudRuntimeException: Failed to backup > > 4d3ea6fa-6902-4037-be67-166e34575ee9 for disk > > /mnt/8a2181f7-86a0-3d7d-a5be-10caa89b3d27/f0615600-d806-486e-bbea-d408de0225e4 > > to /mnt/e5c41d36-90d2-3d5e-bdbf-8a0140dd330e/snapshots/2/3 > > at > > org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280) > > at > > org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:136) > > at > > org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:279) > > at > > com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1001) > > at > > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > > at > > org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1311) > > at > > com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2779) > > at > > org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:170) > > at > > com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > > at > > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:722) > > 2014-03-24 16:12:35,635 DEBUG [storage.volume.VolumeServiceImpl] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Take > > snapshot: 3 failed > > com.cloud.utils.exception.CloudRuntimeException: Failed to create > > snapshot > > at > > com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1028) > > at > > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > > at > > org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1311) > > at > > com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2779) > > at > > org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:170) > > at > > com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > > at > > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:722) > > Caused by: com.cloud.utils.exception.CloudRuntimeException: Failed > > to backup 4d3ea6fa-6902-4037-be67-166e34575ee9 for disk > > /mnt/8a2181f7-86a0-3d7d-a5be-10caa89b3d27/f0615600-d806-486e-bbea-d408de0225e4 > > to /mnt/e5c41d36-90d2-3d5e-bdbf-8a0140dd330e/snapshots/2/3 > > at > > org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280) > > at > > org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:136) > > at > > org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:279) > > at > > com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1001) > > ... 16 more > > 2014-03-24 16:12:35,635 DEBUG [user.snapshot.CreateSnapshotCmd] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) Failed > > to create snapshot > > org.apache.cloudstack.api.ServerApiException > > at > > org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:176) > > at > > com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > > at > > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at > > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:722) > > 2014-03-24 16:12:35,637 DEBUG [cloud.async.AsyncJobManagerImpl] > > (Job-Executor-10:job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ]) > > Complete async job-10 = [ 4f6ffdef-f66d-405c-81cc-8a09cb39f5da ], > > jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed > > to create snapshot due to an internal error creating snapshot for volume 3 > > > > > > Thanks, > > > > Tom. > -- Cloudian KK - http://www.cloudian.com/get-started.html Fancy 10TB of full featured S3 Storage? Checkout the Cloudian® Community Edition!