[Openstack] [SWIFT] raising network traffic o the storage node

[Klaus Schürmann]

Hi,
I use a swift storage as a mail-store. Now I have about  1.000.000 objects 
stored in the cluster.

I'm wondering about the raising network traffic on my storage nodes. The 
traffic from the proxy-server has a normal 
characteristic.

Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png
Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png

Can someone explain such behavior?

Thanks
Klaus
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [SWIFT] raising network traffic o the storage node

[Peter Portante]

Can you zoom in past the spike, most recent 2 or three weeks and see
how it looks?

My guess is that the proxy traffic is also rising.

On Mon, Jul 8, 2013 at 9:50 AM, Klaus Schürmann
 wrote:
> Hi,
>
> I use a swift storage as a mail-store. Now I have about  1.000.000 objects
> stored in the cluster.
>
>
>
> I’m wondering about the raising network traffic on my storage nodes. The
> traffic from the proxy-server has a normal characteristic.
>
>
>
> Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png
>
> Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png
>
>
>
> Can someone explain such behavior?
>
>
>
> Thanks
>
> Klaus
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [SWIFT] raising network traffic o the storage node

[Klaus Schürmann]

On Monday some more mailboxes store their Mails in the objectstorage.
But that only increased the raising.
 
Traffic Storagenode: http://www.schuermann.net/temp/storagenode2.png
Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver2.png


-Ursprüngliche Nachricht-
Von: Peter Portante [mailto:peter.a.porta...@gmail.com] 
Gesendet: Montag, 8. Juli 2013 16:04
An: Klaus Schürmann
Cc: openstack@lists.launchpad.net
Betreff: Re: [Openstack] [SWIFT] raising network traffic o the storage node

Can you zoom in past the spike, most recent 2 or three weeks and see
how it looks?

My guess is that the proxy traffic is also rising.

On Mon, Jul 8, 2013 at 9:50 AM, Klaus Schürmann
 wrote:
> Hi,
>
> I use a swift storage as a mail-store. Now I have about  1.000.000 objects
> stored in the cluster.
>
>
>
> I'm wondering about the raising network traffic on my storage nodes. The
> traffic from the proxy-server has a normal characteristic.
>
>
>
> Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png
>
> Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png
>
>
>
> Can someone explain such behavior?
>
>
>
> Thanks
>
> Klaus
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [SWIFT] raising network traffic o the storage node

[Kuo Hugo]

Hi Klaus,

Would you please grep your swift log for object-replicator?  thx



+Hugo Kuo+
h...@swiftstack.com
tonyt...@gmail.com
+886 935004793


2013/7/8 Klaus Schürmann 

> On Monday some more mailboxes store their Mails in the objectstorage.
> But that only increased the raising.
>
> Traffic Storagenode: http://www.schuermann.net/temp/storagenode2.png
> Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver2.png
>
>
> -Ursprüngliche Nachricht-
> Von: Peter Portante [mailto:peter.a.porta...@gmail.com]
> Gesendet: Montag, 8. Juli 2013 16:04
> An: Klaus Schürmann
> Cc: openstack@lists.launchpad.net
> Betreff: Re: [Openstack] [SWIFT] raising network traffic o the storage node
>
> Can you zoom in past the spike, most recent 2 or three weeks and see
> how it looks?
>
> My guess is that the proxy traffic is also rising.
>
> On Mon, Jul 8, 2013 at 9:50 AM, Klaus Schürmann
>  wrote:
> > Hi,
> >
> > I use a swift storage as a mail-store. Now I have about  1.000.000
> objects
> > stored in the cluster.
> >
> >
> >
> > I'm wondering about the raising network traffic on my storage nodes. The
> > traffic from the proxy-server has a normal characteristic.
> >
> >
> >
> > Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png
> >
> > Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png
> >
> >
> >
> > Can someone explain such behavior?
> >
> >
> >
> > Thanks
> >
> > Klaus
> >
> >
> > ___
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [SWIFT] raising network traffic o the storage node

[Pete Zaitcev]

On Mon, 8 Jul 2013 13:50:38 +
Klaus Schürmann  wrote:

> I use a swift storage as a mail-store. Now I have about  1.000.000 objects 
> stored in the cluster.

> Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png
> Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png
> 
> Can someone explain such behavior?

At a guess, raising number of objects makes number of partitions to
increase, and that increases the replication traffic, specifically
the number of sent MD5s for partitions.

It would be interesting to corellate the number of objects and
number of non-empty and empty partitions with the amounts of traffic.
If the increasing transfer of hashes is the reason, you could also
verify by graphing the traffic to port 873 separately. Swift never
replicates object bodies through its own HTTP interface, so this
splits control traffic from data traffic for you. Data traffic
should be driven by customer and node failures, not consistency
checking.

Be prepared to split up storage nodes, however. Even if there's no
bug with replication, its aggregate traffic increases with the
increases in object counts.

-- Pete

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp