Re: warm restart
weird... I suspect that's something in your environment. It's certainly
not that way anywhere I've ever run it. I'm using it right now with a
space :)
On Mon, 9 Dec 2019, David Karlsen wrote:
> -e /cache_state/memory_file <-- fails-e/cache_state/memory_file <-- OK
>
> man. 9. des. 2019 kl. 22:58 skrev dormando :
> example?
>
> On Mon, 9 Dec 2019, David Karlsen wrote:
>
> > OK, found it, apparently -e needs to be followed by path w/o any
> whitespace
> >
> > man. 9. des. 2019 kl. 21:54 skrev dormando :
> > I'm not sure offhand. From my perspective I'd triple check what
> the
> > path/file it's trying to open is (add a printf or something),
> else you're
> > in container/kube territory and that's a bit beyond me. I
> assume it logs
> > something on policy violations, or can be made to do so.
> >
> > On Mon, 9 Dec 2019, David Karlsen wrote:
> >
> > > No. It is there and writable
> > >
> > > man. 9. des. 2019 kl. 21:32 skrev dormando
> :
> > > Is the directory missing?
> > >
> > > On Mon, 9 Dec 2019, David Karlsen wrote:
> > >
> > > > OK, I hacked it together apt-installing some shared
> libs.With the patch applied I get:
> > > >
> > > > k logs test-memcached-0
> > > > failed to open file for mmap: No such file or
> directory
> > > >
> > > > Which is a bit strange - should not the file be
> created dynamically if it does not exist?
> > > >
> > > > fredag 6. desember 2019 23.51.38 UTC+1 skrev Dormando
> følgende:
> > > > It's going to use some caps (opening files,
> mmap'ing them, shared memory,
> > > > etc). I don't know what maps to which specific
> thing.
> > > >
> > > > That error looks like an omission on my part..
> > > >
> > > > mmap_fd = open(file, O_RDWR|O_CREAT,
> S_IRWXU);
> > > > if (ftruncate(mmap_fd, limit) != 0) {
> > > > perror("ftruncate failed");
> > > > abort();
> > > > }
> > > >
> > > > missing the error check after open.
> > > >
> > > > Try adding a:
> > > >
> > > > if (mmap_fd == -1) {
> > > > perror("failed to open file for mmap");
> > > > abort();
> > > > }
> > > >
> > > > between the open and if (ftruncate) lines,
> which will give you the real
> > > > error. I'll get that fixed upstream.
> > > >
> > > > On Fri, 6 Dec 2019, David Karlsen wrote:
> > > >
> > > > > Does memcached use any of these
>
> capabilities:https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/policy/container-capabilities/
> ?
> > > > >
> > > > >
> > > > > fre. 6. des. 2019 kl. 16:39 skrev David
> Karlsen :
> > > > > So far I am stuck on:
> > > > > k logs test-memcached-0
> > > > > ftruncate failed: Bad file descriptor
> > > > >
> > > > >
> > > > > - memcached
> > > > > - -m 768m
> > > > > - -I 1m
> > > > > - -v
> > > > > - -e /cache-state/memory_file
> > > > >
> > > > > -vvv does not reveal anything interesting.
> >
Re: warm restart
example?
On Mon, 9 Dec 2019, David Karlsen wrote:
> OK, found it, apparently -e needs to be followed by path w/o any whitespace
>
> man. 9. des. 2019 kl. 21:54 skrev dormando :
> I'm not sure offhand. From my perspective I'd triple check what the
> path/file it's trying to open is (add a printf or something), else
> you're
> in container/kube territory and that's a bit beyond me. I assume it logs
> something on policy violations, or can be made to do so.
>
> On Mon, 9 Dec 2019, David Karlsen wrote:
>
> > No. It is there and writable
> >
> > man. 9. des. 2019 kl. 21:32 skrev dormando :
> > Is the directory missing?
> >
> > On Mon, 9 Dec 2019, David Karlsen wrote:
> >
> > > OK, I hacked it together apt-installing some shared libs.With
> the patch applied I get:
> > >
> > > k logs test-memcached-0
> > > failed to open file for mmap: No such file or directory
> > >
> > > Which is a bit strange - should not the file be created
> dynamically if it does not exist?
> > >
> > > fredag 6. desember 2019 23.51.38 UTC+1 skrev Dormando
> følgende:
> > > It's going to use some caps (opening files, mmap'ing
> them, shared memory,
> > > etc). I don't know what maps to which specific thing.
> > >
> > > That error looks like an omission on my part..
> > >
> > > mmap_fd = open(file, O_RDWR|O_CREAT, S_IRWXU);
> > > if (ftruncate(mmap_fd, limit) != 0) {
> > > perror("ftruncate failed");
> > > abort();
> > > }
> > >
> > > missing the error check after open.
> > >
> > > Try adding a:
> > >
> > > if (mmap_fd == -1) {
> > > perror("failed to open file for mmap");
> > > abort();
> > > }
> > >
> > > between the open and if (ftruncate) lines, which will
> give you the real
> > > error. I'll get that fixed upstream.
> > >
> > > On Fri, 6 Dec 2019, David Karlsen wrote:
> > >
> > > > Does memcached use any of these
> capabilities:https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/policy/container-capabilities/
> ?
> > > >
> > > >
> > > > fre. 6. des. 2019 kl. 16:39 skrev David Karlsen
> :
> > > > So far I am stuck on:
> > > > k logs test-memcached-0
> > > > ftruncate failed: Bad file descriptor
> > > >
> > > >
> > > > - memcached
> > > > - -m 768m
> > > > - -I 1m
> > > > - -v
> > > > - -e /cache-state/memory_file
> > > >
> > > > -vvv does not reveal anything interesting.
> > > > What could be the cause of this?
> > > >
> > > > lørdag 30. november 2019 18.03.23 UTC+1 skrev David
> Karlsen følgende:
> > > > Reading
> https://github.com/memcached/memcached/wiki/WarmRestart it is a bit unclear
> to me if the mount *has* to be tmpfs backed, or it can be
> a
> > normal
> > > > fileystem like xfs.
> > > > We are looking into running memcached through
> Kubernetes/containers - and as a tmpfs volume would be wiped on pod-recreation
> > > >
> > > > --
> > > >
> > > > ---
> > > > You received this message because you are subscribed
> to the Google Groups "memcached" group.
> > > > To unsubscribe from this group and stop receiving
> emails from it, send an email to memc
Re: warm restart
Is the directory missing?
On Mon, 9 Dec 2019, David Karlsen wrote:
> OK, I hacked it together apt-installing some shared libs.With the patch
> applied I get:
>
> k logs test-memcached-0
> failed to open file for mmap: No such file or directory
>
> Which is a bit strange - should not the file be created dynamically if it
> does not exist?
>
> fredag 6. desember 2019 23.51.38 UTC+1 skrev Dormando følgende:
> It's going to use some caps (opening files, mmap'ing them, shared
> memory,
> etc). I don't know what maps to which specific thing.
>
> That error looks like an omission on my part..
>
> mmap_fd = open(file, O_RDWR|O_CREAT, S_IRWXU);
> if (ftruncate(mmap_fd, limit) != 0) {
> perror("ftruncate failed");
> abort();
> }
>
> missing the error check after open.
>
> Try adding a:
>
> if (mmap_fd == -1) {
> perror("failed to open file for mmap");
> abort();
> }
>
> between the open and if (ftruncate) lines, which will give you the real
> error. I'll get that fixed upstream.
>
> On Fri, 6 Dec 2019, David Karlsen wrote:
>
> > Does memcached use any of these
> capabilities:https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/policy/container-capabilities/
> ?
> >
> >
> > fre. 6. des. 2019 kl. 16:39 skrev David Karlsen
> :
> > So far I am stuck on:
> > k logs test-memcached-0
> > ftruncate failed: Bad file descriptor
> >
> >
> > - memcached
> > - -m 768m
> > - -I 1m
> > - -v
> > - -e /cache-state/memory_file
> >
> > -vvv does not reveal anything interesting.
> > What could be the cause of this?
> >
> > lørdag 30. november 2019 18.03.23 UTC+1 skrev David Karlsen følgende:
> > Reading https://github.com/memcached/memcached/wiki/WarmRestart
> it is a bit unclear to me if the mount *has* to be tmpfs backed, or it can be
> a normal
> > fileystem like xfs.
> > We are looking into running memcached through
> Kubernetes/containers - and as a tmpfs volume would be wiped on pod-recreation
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups "memcached" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send an email to memc...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/memcached/f39374e2-d603-43e6-928a-a9a0fc9e93ed%40googlegroups.com.
> >
> >
> >
> > --
> > --
> > David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups "memcached" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send an email to memc...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/memcached/CAGO7Ob1d002j0ve-aN6hBGncZWF7jR9ygpaz7B54wbQUGDA%2Beg%40mail.gmail.com.
> >
> >
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "memcached" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to memcached+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/memcached/8defad58-9248-466d-8e4a-14f9e4afd9af%40googlegroups.com.
>
>
--
---
You received this message because you are subscribed to the Google Groups
"memcached" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to memcached+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1912091232060.19126%40dskull.
Re: warm restart
It's going to use some caps (opening files, mmap'ing them, shared memory,
etc). I don't know what maps to which specific thing.
That error looks like an omission on my part..
mmap_fd = open(file, O_RDWR|O_CREAT, S_IRWXU);
if (ftruncate(mmap_fd, limit) != 0) {
perror("ftruncate failed");
abort();
}
missing the error check after open.
Try adding a:
if (mmap_fd == -1) {
perror("failed to open file for mmap");
abort();
}
between the open and if (ftruncate) lines, which will give you the real
error. I'll get that fixed upstream.
On Fri, 6 Dec 2019, David Karlsen wrote:
> Does memcached use any of these
> capabilities:https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/policy/container-capabilities/
> ?
>
>
> fre. 6. des. 2019 kl. 16:39 skrev David Karlsen :
> So far I am stuck on:
> k logs test-memcached-0
> ftruncate failed: Bad file descriptor
>
>
> - memcached
> - -m 768m
> - -I 1m
> - -v
> - -e /cache-state/memory_file
>
> -vvv does not reveal anything interesting.
> What could be the cause of this?
>
> lørdag 30. november 2019 18.03.23 UTC+1 skrev David Karlsen følgende:
> Reading https://github.com/memcached/memcached/wiki/WarmRestart it is a
> bit unclear to me if the mount *has* to be tmpfs backed, or it can be a normal
> fileystem like xfs.
> We are looking into running memcached through Kubernetes/containers -
> and as a tmpfs volume would be wiped on pod-recreation
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "memcached" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to memcached+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/memcached/f39374e2-d603-43e6-928a-a9a0fc9e93ed%40googlegroups.com.
>
>
>
> --
> --
> David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "memcached" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to memcached+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/memcached/CAGO7Ob1d002j0ve-aN6hBGncZWF7jR9ygpaz7B54wbQUGDA%2Beg%40mail.gmail.com.
>
>
--
---
You received this message because you are subscribed to the Google Groups
"memcached" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to memcached+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1912061449210.19126%40dskull.
Re: warm restart
If you succeed you should share with the class :) On Sun, 1 Dec 2019, David Karlsen wrote: > Thank you - that explains it well. I'll look around if I can create a > "durable" tmpfs in k8s via a storageclass :) > > søn. 1. des. 2019 kl. 04:25 skrev dormando : > The disk file is memory mapped; that is the actual memory, now external > to > memcached. There's no flush at shutdown, it just gracefully stops all > in-flight actions and then does a fast data fixup on restart. > > So it does continually read/write to that file. As I said earlier you > can > create an "equivalent" to writing the file at shutdown by moving the > file > after shutdown :) > > On Sun, 1 Dec 2019, David Karlsen wrote: > > > Won’t the cache be written to file at shutdown and not contionously > while running? > > > > søn. 1. des. 2019 kl. 03:58 skrev dormando : > > Hey, > > > > It's only guaranteed to work in a ram disk. It will "work" on > anything > > else, but you'll lose deterministic performance. Worst case > it'll burn out > > whatever device is underlying because it's not optimized for > anything but > > RAM. > > > > So, two options for this situation: > > > > 1) I'd hope there's some way to bind mount an underlying tmpfs. > With > > almost all container systems there's some method of exposing an > underlying > > path, though I have a low opinion of Kube so manybe not. > > > > 2) It does just create two normal files: the path you give it + > .meta file > > that appears during graceful shutdown. After shutdown you can > copy these > > (perhaps with pigz or something) to a filesystem then restore > to in-pod > > tmpfs before starting up again. It'll increase the downtime but > it'll > > work. > > > > I guess.. 3) For completeness it also works on a DCPMM dax > mount, which > > survive reboots and act as "filesystems". You'd need to have > the right > > system and memory and etc. > > > > -Dormando > > > > On Sat, 30 Nov 2019, David Karlsen wrote: > > > > > Reading > https://github.com/memcached/memcached/wiki/WarmRestart it is a bit unclear > to me if the mount *has* to be tmpfs backed, or it can be a normal > fileystem > > like xfs. > > > We are looking into running memcached through > Kubernetes/containers - and as a tmpfs volume would be wiped on pod-recreation > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the > Google Groups "memcached" group. > > > To unsubscribe from this group and stop receiving emails from > it, send an email to memcached+unsubscr...@googlegroups.com. > > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/2ed07578-4aff-4704-83b1-3cd7d56de59f%40googlegroups.com. > > > > > > > > > > -- > > > > --- > > You received this message because you are subscribed to the > Google Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from > it, send an email to memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1911301854320.5300%40dskull. > > > > -- > > -- > > David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/CAGO7Ob36THu%2BJTo3uxFMW6t6RV4BUQ38WMuUVUetY9TqGwDP7w%40mail.gmail.com. > > > > > > -- > > --- > You received this message because you are subscribed to the Google > Groups "
Re: warm restart
The disk file is memory mapped; that is the actual memory, now external to memcached. There's no flush at shutdown, it just gracefully stops all in-flight actions and then does a fast data fixup on restart. So it does continually read/write to that file. As I said earlier you can create an "equivalent" to writing the file at shutdown by moving the file after shutdown :) On Sun, 1 Dec 2019, David Karlsen wrote: > Won’t the cache be written to file at shutdown and not contionously while > running? > > søn. 1. des. 2019 kl. 03:58 skrev dormando : > Hey, > > It's only guaranteed to work in a ram disk. It will "work" on anything > else, but you'll lose deterministic performance. Worst case it'll burn > out > whatever device is underlying because it's not optimized for anything > but > RAM. > > So, two options for this situation: > > 1) I'd hope there's some way to bind mount an underlying tmpfs. With > almost all container systems there's some method of exposing an > underlying > path, though I have a low opinion of Kube so manybe not. > > 2) It does just create two normal files: the path you give it + .meta > file > that appears during graceful shutdown. After shutdown you can copy these > (perhaps with pigz or something) to a filesystem then restore to in-pod > tmpfs before starting up again. It'll increase the downtime but it'll > work. > > I guess.. 3) For completeness it also works on a DCPMM dax mount, which > survive reboots and act as "filesystems". You'd need to have the right > system and memory and etc. > > -Dormando > > On Sat, 30 Nov 2019, David Karlsen wrote: > > > Reading https://github.com/memcached/memcached/wiki/WarmRestart it is > a bit unclear to me if the mount *has* to be tmpfs backed, or it can be a > normal fileystem > like xfs. > > We are looking into running memcached through Kubernetes/containers - > and as a tmpfs volume would be wiped on pod-recreation > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/2ed07578-4aff-4704-83b1-3cd7d56de59f%40googlegroups.com. > > > > > > -- > > --- > You received this message because you are subscribed to the Google > Groups "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1911301854320.5300%40dskull. > > -- > -- > David J. M. Karlsen - http://www.linkedin.com/in/davidkarlsen > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/CAGO7Ob36THu%2BJTo3uxFMW6t6RV4BUQ38WMuUVUetY9TqGwDP7w%40mail.gmail.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1911301923260.5300%40dskull.
Re: warm restart
Hey, It's only guaranteed to work in a ram disk. It will "work" on anything else, but you'll lose deterministic performance. Worst case it'll burn out whatever device is underlying because it's not optimized for anything but RAM. So, two options for this situation: 1) I'd hope there's some way to bind mount an underlying tmpfs. With almost all container systems there's some method of exposing an underlying path, though I have a low opinion of Kube so manybe not. 2) It does just create two normal files: the path you give it + .meta file that appears during graceful shutdown. After shutdown you can copy these (perhaps with pigz or something) to a filesystem then restore to in-pod tmpfs before starting up again. It'll increase the downtime but it'll work. I guess.. 3) For completeness it also works on a DCPMM dax mount, which survive reboots and act as "filesystems". You'd need to have the right system and memory and etc. -Dormando On Sat, 30 Nov 2019, David Karlsen wrote: > Reading https://github.com/memcached/memcached/wiki/WarmRestart it is a bit > unclear to me if the mount *has* to be tmpfs backed, or it can be a normal > fileystem like xfs. > We are looking into running memcached through Kubernetes/containers - and as > a tmpfs volume would be wiped on pod-recreation > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/2ed07578-4aff-4704-83b1-3cd7d56de59f%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1911301854320.5300%40dskull.
Re: limits
It's fine. Just make sure you actually have enough RAM to handle it. On Fri, 8 Nov 2019, Ramesh wrote: > Thanks, how about the performance if increased to 512MB? > > On Friday, November 8, 2019 at 2:45:22 PM UTC-5, Dormando wrote: > default is 1MB. You can increase that to 1G via the -I start argument. > > On Fri, 8 Nov 2019, Ramesh wrote: > > > Hello, > > > > I can breakdown what is the limit on the size of the value per key? I > see Redis has limitation of 512MB or how about Memcached? > > Thanks > > Ramesh > > > > On Wednesday, November 6, 2019 at 2:23:12 PM UTC-5, Ramesh wrote: > > Hello, > > I want to store binary streams of value size in the orderof 1GB per > one object, are there any limitations on the size of the value? > > > > Thanks > > Ramesh > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memc...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/69cb3a9b-a52b-45fb-b1a0-0aa9700d08d9%40googlegroups.com. > > > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/e857729f-c4e8-4fb9-950a-b06e79de28c3%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1911081457210.29389%40dskull.
Re: limits
default is 1MB. You can increase that to 1G via the -I start argument. On Fri, 8 Nov 2019, Ramesh wrote: > Hello, > > I can breakdown what is the limit on the size of the value per key? I see > Redis has limitation of 512MB or how about Memcached? > Thanks > Ramesh > > On Wednesday, November 6, 2019 at 2:23:12 PM UTC-5, Ramesh wrote: > Hello, > I want to store binary streams of value size in the orderof 1GB per one > object, are there any limitations on the size of the value? > > Thanks > Ramesh > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/69cb3a9b-a52b-45fb-b1a0-0aa9700d08d9%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1911081144540.29389%40dskull.
non-production shadow testing?
Just in case: https://github.com/memcached/memcached/pull/557 - is a pretty major patch. it can potentially hugely reduce CPU usage on both clients and servers, cut memory usage for large numbers of connections, fix memory bloats in some specific situations/etc. ...but it's in a pretty crufty part of the codebase. If any of you have production systems where you can run a memcached build non-live (debug/development or ideally "shadowing" production traffic), I'd appreciate you trying it out ahead of me merging it. Thanks! -dormando -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1910221156310.5481%40dskull.
Discord chat
Trying an experiment: https://discord.gg/8yVxuJY (this link won't work forever). The freenode IRC channel has been dying slowly. I hate slack's multi-server interface and their profit model makes public/OSS chats awful. -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1910171513060.3754%40dskull.
Re: Memcached performance numbers
the high end numbers are due to pipelining responses. ie; ascii multiget, which reduces the syscalls. you can see how the tests were run via the links to the source code in the blog. I was running some pure get tests on dual 8 core machine yesterday with memcached pinned to one numa node. Without any pipelining it was doing ~1.8m ops/sec. With heavy pipelining it should be much more than that. In extreme and contrived cases I've gotten the pure get throughput above 50 million keys/sec. So I know that part scales... sets would as well but nobody really asks for it so I've not focused on it. Latency is probably not great at that throughput though :) On Mon, 7 Oct 2019, Pradeep Fernando wrote: > Thanks for the article link. That is some comprehensive benchmarking. > Compared to article numbers, my latency numbers are sane enough. I hit ~120 > us while you get similar/closer numbers at 99 th percentile. > > However, my throughput numbers seems to be wrong. I hit a throughput > kneepoint at 500K ops/sec while yours is around ~6000K. > Order of magnitude difference. Can you please comment on it. :) > > thanks > --Pradeep > > On Mon, Oct 7, 2019 at 4:46 PM dormando wrote: > It'll depend on your hardware/test/etc. > > https://memcached.org/blog/persistent-memory/ - a thorough performance > test with some higher end numbers on both throughput and latency along > with 50th/90th/95/99/etc percentiles and latency point clouds for each > sub > test. That was a big machine though. > > ...and no I'm not going to ignore your 50/50 ratio :) the ratio changes > the results too much. people will have to test with what they expect to > see. if you do 100% get it scales linearly with the number of worker > threads/cores. Anything below 100% gets will slowly scale down to 500-1m > ops/s depending on the hardware and size of the objects. > > On Mon, 7 Oct 2019, Pradeep Fernando wrote: > > > Hi, > > Thanks for the help!. > > After a couple of trial and error configs, I figured out 'concurrency > parameter' used in memaslap as the culprit. > > In my configs I was using 16 (constant) as the concurrency input. > Scaling the value along with thread count gave me sane numbers. > > > > The average latency is 120 us for get/set workload ( please ignore my > 50/50 ratio) and throughput max out around 500K ops/second. > > graph attached. > > > > I know that the benchmarking numbers are heavily dependent on the > setup and other things. But Is my numbers are faithful enough to be quoted > for > > memcached single server numbers? In other words, are these numbers > are way off from a typical memcached performance numbers? > > > > thanks > > --Pradeep > > > > > > On Mon, Oct 7, 2019 at 2:42 PM dormando wrote: > > Hey, > > > > Sorry; I'm not going to have any other major insights :) I'd > have to sit > > here playing 20 questions to figure out your test setup. If > you're running > > memaslap from another box, that one needs to be cpu pinned as > well. If > > it's a VM, the governor/etc might not even matter. > > > > Also I don't use memaslap at all, so I can't attest to it. I use > > https://github.com/memcached/mc-crusher with the external > latency sampling > > util it comes with. it's not as easy to use though. > > > > On Mon, 7 Oct 2019, Pradeep Fernando wrote: > > > > > Hi Dormando, > > > That is great insight.!. > > > However, it did not solve the problem. I disabled turbo, as > per your instructions. > > > I even, set the CPU to operate with maximum performance, with > > > > cpupower frequency-set --governor performance ( i verified > this by monitoring cpu freq) > > > > > > Still the same unexplained behavior. :(. Do you have any > other suggestions? > > > > > > thanks > > > --Pradeep > > > > > > On Mon, Oct 7, 2019 at 1:08 PM dormando > wrote: > > > Hi, > > > > > > First as an aside; 1/1 get/set ratio is unusual for mc. > The gets scale a > > > lot better than sets. If you get into testing more > "realistic" perf > >
Re: Memcached performance numbers
It'll depend on your hardware/test/etc. https://memcached.org/blog/persistent-memory/ - a thorough performance test with some higher end numbers on both throughput and latency along with 50th/90th/95/99/etc percentiles and latency point clouds for each sub test. That was a big machine though. ...and no I'm not going to ignore your 50/50 ratio :) the ratio changes the results too much. people will have to test with what they expect to see. if you do 100% get it scales linearly with the number of worker threads/cores. Anything below 100% gets will slowly scale down to 500-1m ops/s depending on the hardware and size of the objects. On Mon, 7 Oct 2019, Pradeep Fernando wrote: > Hi, > Thanks for the help!. > After a couple of trial and error configs, I figured out 'concurrency > parameter' used in memaslap as the culprit. > In my configs I was using 16 (constant) as the concurrency input. Scaling the > value along with thread count gave me sane numbers. > > The average latency is 120 us for get/set workload ( please ignore my 50/50 > ratio) and throughput max out around 500K ops/second. > graph attached. > > I know that the benchmarking numbers are heavily dependent on the setup and > other things. But Is my numbers are faithful enough to be quoted for > memcached single server numbers? In other words, are these numbers are way > off from a typical memcached performance numbers? > > thanks > --Pradeep > > > On Mon, Oct 7, 2019 at 2:42 PM dormando wrote: > Hey, > > Sorry; I'm not going to have any other major insights :) I'd have to sit > here playing 20 questions to figure out your test setup. If you're > running > memaslap from another box, that one needs to be cpu pinned as well. If > it's a VM, the governor/etc might not even matter. > > Also I don't use memaslap at all, so I can't attest to it. I use > https://github.com/memcached/mc-crusher with the external latency > sampling > util it comes with. it's not as easy to use though. > > On Mon, 7 Oct 2019, Pradeep Fernando wrote: > > > Hi Dormando, > > That is great insight.!. > > However, it did not solve the problem. I disabled turbo, as per your > instructions. > > I even, set the CPU to operate with maximum performance, with > > > cpupower frequency-set --governor performance ( i verified this by > monitoring cpu freq) > > > > Still the same unexplained behavior. :(. Do you have any other > suggestions? > > > > thanks > > --Pradeep > > > > On Mon, Oct 7, 2019 at 1:08 PM dormando wrote: > > Hi, > > > > First as an aside; 1/1 get/set ratio is unusual for mc. The > gets scale a > > lot better than sets. If you get into testing more "realistic" > perf > > numbers make sure to increase the get rate. > > > > You're probably just running into CPU scaling. OS's come with a > "battery > > saver" or "ondemand" performance scheduler by default. They > also have > > turbo. Once you start loading it up more the CPU will stay in > the higher > > frequency states or begin to issue turbo, which will lower the > latency. > > > > /usr/bin/echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo > > cpupower frequency-set -g performance > > > > ... or whatever works for your platform. > > > > On Mon, 7 Oct 2019, Pradeep Fernando wrote: > > > > > Hi Devs, > > > I run memaslap to understand the performance characteristics > of memcached, > > > > > > My setup : both memcached and memaslap running on a single > machine with NUMA. memcached is bound to NUMA 1. Gave 3GB of memory to > memcached. > > > workload : get/set 0.5/0.5 > > > > > > I increase number of thread from memaslap and observe > throughput latency numbers. > > > > > > I see increase in throughput (expected) but latency drops as > I crease the load. > > > The initial average latency is 83 us and it drops to 30us > with number of threads = 8, this is an unexpected number.-- I expected the > latency to go up. > > > Am I reading the output wrong? > > > > > > Apologies, if this question does not qualify for this mailing > list. I
Re: Memcached performance numbers
Hey, Sorry; I'm not going to have any other major insights :) I'd have to sit here playing 20 questions to figure out your test setup. If you're running memaslap from another box, that one needs to be cpu pinned as well. If it's a VM, the governor/etc might not even matter. Also I don't use memaslap at all, so I can't attest to it. I use https://github.com/memcached/mc-crusher with the external latency sampling util it comes with. it's not as easy to use though. On Mon, 7 Oct 2019, Pradeep Fernando wrote: > Hi Dormando, > That is great insight.!. > However, it did not solve the problem. I disabled turbo, as per your > instructions. > I even, set the CPU to operate with maximum performance, with > > cpupower frequency-set --governor performance ( i verified this by > > monitoring cpu freq) > > Still the same unexplained behavior. :(. Do you have any other suggestions? > > thanks > --Pradeep > > On Mon, Oct 7, 2019 at 1:08 PM dormando wrote: > Hi, > > First as an aside; 1/1 get/set ratio is unusual for mc. The gets scale a > lot better than sets. If you get into testing more "realistic" perf > numbers make sure to increase the get rate. > > You're probably just running into CPU scaling. OS's come with a "battery > saver" or "ondemand" performance scheduler by default. They also have > turbo. Once you start loading it up more the CPU will stay in the higher > frequency states or begin to issue turbo, which will lower the latency. > > /usr/bin/echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo > cpupower frequency-set -g performance > > ... or whatever works for your platform. > > On Mon, 7 Oct 2019, Pradeep Fernando wrote: > > > Hi Devs, > > I run memaslap to understand the performance characteristics of > memcached, > > > > My setup : both memcached and memaslap running on a single machine > with NUMA. memcached is bound to NUMA 1. Gave 3GB of memory to memcached. > > workload : get/set 0.5/0.5 > > > > I increase number of thread from memaslap and observe throughput > latency numbers. > > > > I see increase in throughput (expected) but latency drops as I crease > the load. > > The initial average latency is 83 us and it drops to 30us with number > of threads = 8, this is an unexpected number.-- I expected the latency to go > up. > > Am I reading the output wrong? > > > > Apologies, if this question does not qualify for this mailing list. > If so, please direct me to correct list I can get help. :) > > > > --Pradeep > > > > > > > > Thread count = 1 > > > > > > Total Statistics (11447336 events) > > Min: 11 > > Max: 1663 > > Avg: 83 > > Geo: 79.83 > > Std: 36.39 > > Log2 Dist: > > 4: 42 594 351733 9527982 > > 8: 1551101 7451 7103 1330 > > > > cmd_get: 5723671 > > cmd_set: 5723681 > > get_misses: 0 > > written_bytes: 394933167 > > read_bytes: 343419948 > > object_bytes: 183157792 > > > > Run time: 60.0s Ops: 11447352 TPS: 190765 Net_rate: 11.7M/s > > > > Thread count = 2 > > > > Total Statistics (30888799 events) > > Min: 12 > > Max: 2011 > > Avg: 30 > > Geo: 29.68 > > Std: 15.32 > > Log2 Dist: > > 4: 170225 25862674 478 66398 > > 8: 154 5017 17493 170 > > > > cmd_get: 1504 > > cmd_set: 1511 > > get_misses: 0 > > written_bytes: 1065663678 > > read_bytes: 926663772 > > object_bytes: 494221152 > > > > Run time: 60.0s Ops: 3015 TPS: 514751 Net_rate: 31.7M/s > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/cd593815-c27b-4995-bb7f-21859d9a3187%40googlegroups.com. > >
Re: Memcached performance numbers
Hi, First as an aside; 1/1 get/set ratio is unusual for mc. The gets scale a lot better than sets. If you get into testing more "realistic" perf numbers make sure to increase the get rate. You're probably just running into CPU scaling. OS's come with a "battery saver" or "ondemand" performance scheduler by default. They also have turbo. Once you start loading it up more the CPU will stay in the higher frequency states or begin to issue turbo, which will lower the latency. /usr/bin/echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo cpupower frequency-set -g performance ... or whatever works for your platform. On Mon, 7 Oct 2019, Pradeep Fernando wrote: > Hi Devs, > I run memaslap to understand the performance characteristics of memcached, > > My setup : both memcached and memaslap running on a single machine with NUMA. > memcached is bound to NUMA 1. Gave 3GB of memory to memcached. > workload : get/set 0.5/0.5 > > I increase number of thread from memaslap and observe throughput latency > numbers. > > I see increase in throughput (expected) but latency drops as I crease the > load. > The initial average latency is 83 us and it drops to 30us with number of > threads = 8, this is an unexpected number.-- I expected the latency to go up. > Am I reading the output wrong? > > Apologies, if this question does not qualify for this mailing list. If so, > please direct me to correct list I can get help. :) > > --Pradeep > > > > Thread count = 1 > > > Total Statistics (11447336 events) > Min: 11 > Max: 1663 > Avg: 83 > Geo: 79.83 > Std: 36.39 > Log2 Dist: > 4: 42 594 351733 9527982 > 8: 1551101 7451 7103 1330 > > cmd_get: 5723671 > cmd_set: 5723681 > get_misses: 0 > written_bytes: 394933167 > read_bytes: 343419948 > object_bytes: 183157792 > > Run time: 60.0s Ops: 11447352 TPS: 190765 Net_rate: 11.7M/s > > Thread count = 2 > > Total Statistics (30888799 events) > Min: 12 > Max: 2011 > Avg: 30 > Geo: 29.68 > Std: 15.32 > Log2 Dist: > 4: 170225 25862674 478 66398 > 8: 154 5017 17493 170 > > cmd_get: 1504 > cmd_set: 1511 > get_misses: 0 > written_bytes: 1065663678 > read_bytes: 926663772 > object_bytes: 494221152 > > Run time: 60.0s Ops: 3015 TPS: 514751 Net_rate: 31.7M/s > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/cd593815-c27b-4995-bb7f-21859d9a3187%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1910071005340.21578%40dskull.
Re: Is SASL always file-based?
That all depends on your SASL backend service; you should be able to connect it to whatever you want. However actually doing that can be complicated. You'll have to find tutorials online, since it isn't memcached specific. On Tue, 1 Oct 2019, Jiuming Shao wrote: > Hi there, > I have a quick question on SASL configuration and mechanism choice. > I noticed that for SASL DB, I always need to pass in a username and password > (either in a db file or using `saslpasswd2`) as the credential to auth > connection. I am wondering if > there is any plugins or mechanisms which can accept dynamic credential. > I don't want to store anything on disk nor send any credential in plain text > on wire. I already have a credential provider daemon ready, I can always > query it for the latest > credential that I should use(shared by both cache client and cache server.). > > Thanks in advance! > > Jiuming > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/7427f813-b2c5-417e-b018-4b493443eff5%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1910011220210.31828%40dskull.
Re: get value, but only if cas token is not the provided one
> > > That is a really nice idea - I'll definitely have to explore it. An > effective hot cache like this may reduce or eliminate the need for mitigating > specific keys. We could > incorporate size into it so that larger values get a higher weighting. > First one's free :) I found some other weirdness with the last access time, but I'm pretty sure it's fixable so this pattern should work once released. -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1909240012050.620%40dskull.
Re: get value, but only if cas token is not the provided one
Gotcha. Thanks a ton for reaching out and putting up with my questions :)
I'm still a little concerned that this setup would be slower since you're
blocking on the check to mc before using the local cache, but you know
your workload best :) This would still be a win if the cache objects are
very large, or CPU intensive to deserialize and the local cache held
deserialized objects.
I'll take a look today at the value flag. It's come up enough times, just
need to keep the logic clean internally.
One other thing mget might get you here is an easy probabilistic hot
cache. I really like probabilistic algorithms since they require low or
zero coordination :)
With mget, you could:
`mget foo slhvfc` etc
s = size, required kind of :)
l = last access time (fixing this in the PR today)
h = if the item's been hit before (also being fixed today)
v = value
f = client flags (if you need them)
c = cas value, for your later comparison/check
Then for your hot cache:
if (hit && "last access is < 5 seconds" && random(1000) == 1) {
insert_into_localcache(obj);
}
to autotune, that 1000 becomes a variable you sync out periodically (or
simply stick in the local cache for 60 seconds unconditionally),
decreasing or increasing to match a target hit ratio on the local cache.
This is similar to how extstore decides to recache objects. It's been
working well in production workloads.
It should also be possible to make that check only against objects which
you think could be hot (so not user level stuff I'd guess), reducing the
bytes on wire a bit.
On Mon, 23 Sep 2019, John Reilly wrote:
> I am on the caching team in Box and I'm thinking about this potentially as
> part of a mechanism for hot key mitigation rather than for general use for
> all gets. Assuming I have
> a mechanism for detecting hot keys and notifying clients about them, I
> believe the clients could mitigate the impact of hot keys using a small local
> cache for these cache
> entries but I would still want to ensure the clients are not using stale data
> from the local cache. Using mget in either of the ways we talked about
> before would work for this
> scenario, but having the "fetch if cas does not match" flag would avoid the
> second round trip in the case that the value changes.
> Cheers,
> John
>
> On Sun, Sep 22, 2019 at 12:20 AM dormando wrote:
> Can you give me more background on your workload?
>
> Correct me if I'm wrong but your aim was to use a local cache but check
> the memcached server on every fetch to find out if it's valid?
>
> If your objects are less than... like a kilobyte that's actually slower
> than not having the local cache at all. Think you need to be in the 32k+
> range to start seeing benefits.
>
> Best is to time bound it, at any rate. Max 1 validations per second for
> hot keys that get X requests per second each.
>
> I'm going to draft out two possible mget flags to see if they'd not be
> overcomplex:
>
> - Return value if CAS match
> - Return value if CAS not match.
>
> ... the latter of which would do what you want, but lets just take a
> quick
> look at your workload first so you don't end up with something more
> complicated and slower :) I'm not sure I can get the flags to make sense
> but I'll try.
>
> On Sat, 21 Sep 2019, John Reilly wrote:
>
> > Hi dormando,Sorry for the delayed response. I finally got a chance
> to read through https://github.com/memcached/memcached/pull/484 . It sounds
> great.
> >
> > In my case, I was thinking about using a local cache to mitigate the
> network impact of hot keys rather than per-request performance reasons, but I
> was hoping to do
> that without
> > the clients potentially using stale data from their local cache. It
> might still be nice to have a flag on mget to fetch the value if it does not
> match a provided
> cas, but in
> > the absence of this flag I think it would work fine using mget to
> only get the cas, and doing a full fetch on cas mismatch.
> >
> > Cheers,
> > John
> >
> >
> >
> > On Tue, Sep 17, 2019 at 5:43 PM dormando wrote:
> > Hey,
> >
> > Check this out: https://github.com/memcached/memcached/pull/484
> >
> > You can't quite do this with the way metaget is now, though
> it's feasible
> > to add some "value if cas match on mget" flag. I'd have to
> think it
> > through first.
> >
> > For local caches though, unless your object is hug
Re: get value, but only if cas token is not the provided one
Can you give me more background on your workload? Correct me if I'm wrong but your aim was to use a local cache but check the memcached server on every fetch to find out if it's valid? If your objects are less than... like a kilobyte that's actually slower than not having the local cache at all. Think you need to be in the 32k+ range to start seeing benefits. Best is to time bound it, at any rate. Max 1 validations per second for hot keys that get X requests per second each. I'm going to draft out two possible mget flags to see if they'd not be overcomplex: - Return value if CAS match - Return value if CAS not match. ... the latter of which would do what you want, but lets just take a quick look at your workload first so you don't end up with something more complicated and slower :) I'm not sure I can get the flags to make sense but I'll try. On Sat, 21 Sep 2019, John Reilly wrote: > Hi dormando,Sorry for the delayed response. I finally got a chance to read > through https://github.com/memcached/memcached/pull/484 . It sounds great. > > In my case, I was thinking about using a local cache to mitigate the network > impact of hot keys rather than per-request performance reasons, but I was > hoping to do that without > the clients potentially using stale data from their local cache. It might > still be nice to have a flag on mget to fetch the value if it does not match > a provided cas, but in > the absence of this flag I think it would work fine using mget to only get > the cas, and doing a full fetch on cas mismatch. > > Cheers, > John > > > > On Tue, Sep 17, 2019 at 5:43 PM dormando wrote: > Hey, > > Check this out: https://github.com/memcached/memcached/pull/484 > > You can't quite do this with the way metaget is now, though it's > feasible > to add some "value if cas match on mget" flag. I'd have to think it > through first. > > For local caches though, unless your object is huge, simply waiting on a > round trip to memcached to see if it's up to date removes most of the > value of having the local cache. With a local cache you have to check it > first, then check if it's fresh, then use it. It's likely the same speed > to just not have the local cache at that point so you can avoid the CPU > burn of the initial hash/test or trade it for CPU/network used in > pulling > in the value and having a simple system. > > However! If you have a limited size "hot cache" and you can > asynchronously > test if they need to update, you could (say once per second or whatever > makes sense for how hot your objects are), kick off an async test which > runs mget with options for no-bump (optionally), no value, and cas (no > flags, size, etc) for a lightweight response of just the cas value. > > If the cas doesn't match, re-issue for a full fetch. This works okay for > high frequency items since an update would only leave them out of sync > briefly. Polling kind of sucks but you'd only do this when it would > reduce > the number of requests back to origin anyway :) > > I'm hoping to get metaget in mainline ASAP. Been hunting around for > feedback :) Should be finishing up the code very soon, with merge once a > bit more confident. > > On Tue, 17 Sep 2019, John Reilly wrote: > > > Hi all,I was just thinking it would be great to be able to cache the > most used items in a local cache on the client side and I think this would be > possible if > there was a way > > for the client to send a request to get a key, but only if the cas > value is not the same as the cas token of the value I already know about > locally in the client. > I don't think > > this is possible with either protocol today, but would be happy if > told otherwise :) > > > > Also, can anyone think of a reason why this would not work - if it is > not supported today, it might be a nice feature to add. > > > > Thanks, > > John > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/CAJ__CS_0CaWLU-fqTV%2BeYRU6V3Pg6D8Rix%2B7Lbg_YyDs5tjxPg%40mail.gmail.com. > > > > > > -- > > --- > You received this message because you are subs
restartable cache mode (1.5.18)
https://github.com/memcached/memcached/wiki/ReleaseNotes1518 enjoy -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1909172230270.1888%40dskull.
Re: get value, but only if cas token is not the provided one
Hey, Check this out: https://github.com/memcached/memcached/pull/484 You can't quite do this with the way metaget is now, though it's feasible to add some "value if cas match on mget" flag. I'd have to think it through first. For local caches though, unless your object is huge, simply waiting on a round trip to memcached to see if it's up to date removes most of the value of having the local cache. With a local cache you have to check it first, then check if it's fresh, then use it. It's likely the same speed to just not have the local cache at that point so you can avoid the CPU burn of the initial hash/test or trade it for CPU/network used in pulling in the value and having a simple system. However! If you have a limited size "hot cache" and you can asynchronously test if they need to update, you could (say once per second or whatever makes sense for how hot your objects are), kick off an async test which runs mget with options for no-bump (optionally), no value, and cas (no flags, size, etc) for a lightweight response of just the cas value. If the cas doesn't match, re-issue for a full fetch. This works okay for high frequency items since an update would only leave them out of sync briefly. Polling kind of sucks but you'd only do this when it would reduce the number of requests back to origin anyway :) I'm hoping to get metaget in mainline ASAP. Been hunting around for feedback :) Should be finishing up the code very soon, with merge once a bit more confident. On Tue, 17 Sep 2019, John Reilly wrote: > Hi all,I was just thinking it would be great to be able to cache the most > used items in a local cache on the client side and I think this would be > possible if there was a way > for the client to send a request to get a key, but only if the cas value is > not the same as the cas token of the value I already know about locally in > the client. I don't think > this is possible with either protocol today, but would be happy if told > otherwise :) > > Also, can anyone think of a reason why this would not work - if it is not > supported today, it might be a nice feature to add. > > Thanks, > John > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/CAJ__CS_0CaWLU-fqTV%2BeYRU6V3Pg6D8Rix%2B7Lbg_YyDs5tjxPg%40mail.gmail.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1909171732430.1888%40dskull.
Re: HIPAA eligible/Compliance
It can be depending on how you use it; you need to discuss this with your compliance officer. On Fri, 6 Sep 2019, 'Jeganath James' via memcached wrote: > Is Memcached HIPAA eligible/Compliant service. We have frequent asks from > customers on similar lines. Any help is much appreciated. > > The content of this e-mail is confidential and is intended solely for the use > of the individual or entity to whom it is addressed. If you have received > this e-mail by mistake, > please reply to this e-mail and follow with its deletion. If you are not the > intended recipient, please note that it shall be considered unlawful to copy, > forward or in any > manner reveal the contents of this e-mail or any part thereof to anyone. > Although Freshworks has taken reasonable precautions to ensure no malware is > present in this e-mail, > Freshworks cannot accept responsibility for any loss or damage arising from > the use of this e-mail or attachments. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/7e1502c4-06ff-40ef-9dbe-ab03b46d4a8d%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1909071600360.28461%40dskull.
Re: persistent memory blog
Yo, it's just this branch: https://github.com/memcached/memcached/pull/342 pass "-e /path/to/dax/mount/memcache" and it'll use the persistent memory mount. The graceful restart code isn't complete yet, but you can ignore it if all you want to do is test using persistent memory for capacity reasons. Should be in the mainline soon. I'm pushing to get it complete. -Dormando On Tue, 3 Sep 2019, Pradeep Fernando wrote: > Hi Dormando, > Thanks for the post!. I want to run the same benchmark/configuration for one > of my workloads. > In the post, you mentioned, > > "Persistent memory is enabled using a DAX filesystem and a prototype patch > for memcached’s restartable mode. The persistent memory is accessed via a > mmap’ed file." > > I have a pmem DAX file system and your mmap memory allocation patch would > help me. Can you please share it -- I could not find a link in your post. > Further, I do not understand the additional requirement of 'memcached > restartable mode' and how it relates to persistent memory. > > Thanks in advance, > --Pradeep > > > > > On Tuesday, April 30, 2019 at 3:23:30 PM UTC-4, Dormando wrote: > https://memcached.org/blog/persistent-memory/ > > did some extensive testing with the new persistent memory modules from > intel. If you can get them, they're pretty good. A lot less work to use > than I was expecting. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/79cd2011-987f-4712-8675-c305c6d68601%40googlegroups.com. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1909031615000.28461%40dskull.
mget/mset/mdelete feedback
Hey, Looking again for some client author feedback for a pretty major protocol change: https://github.com/memcached/memcached/pull/484 - this may completely supplant the binary protocol, adding a lot of features/flexibility and cleanups along the way. If you've written a client, or get down in the details frequently, would be happy to hear from. thanks, -Dormando -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1908281341050.18948%40dskull.
Re: Corner case with timestamp TTLs
Yo,
In general we don't recommend using timestamps for short TTL's. They were
really meant for dates far off in the future (months, where the accuracy
wouldn't matter too much). There're zero use cases for TIMESTAMP + TTL
where ttl is short. simply stop adding time() to it, or run it through a
function to convert if the requested TTL is huge.
That said, your problem isn't specifically "VM's" or not VM's, it's that
your VM sucks at keeping time!
memcached uses a monotonic clock; which will only ever go forward. This
ensures that if the system clock bounces backwards that an item doesn't
end up becoming immortal due to underflow.
Even on a good system the monotonic clock will drift over a long period of
uptime, usually after a few months. Just by a couple seconds. CPU's with
buggier clocks could shift by minutes I guess. Or in this case VM's.
There isn't too much that could be done about this simply, which is why we
highly recommend people stick to relative TTL's. If I could remove the
date feature I would, but we can't for backwards compatibility reasons.
On Mon, 12 Aug 2019, Slawomir Pryczek wrote:
> Hi Everyone, i have just came upon very strange corner case when doing SET
> using UNIX_TIMESTAMP plus TTL in memcached running in vmware container. It
> seems that on barebone
> servers there is no such issue.
>
> So if the container is running for some time, eg. a week or more, then it
> seems that timestamps between PHP and memcached are de-synchronizing, so the
> code like this will no
> longer work as expected.
>
> $mc = __getMemcache();
> $mc->set("key", "value", 0, time()+10);
> var_dump($mc->get("key"));
> Will return false.
>
> The workaround is to restart memcached from time to time, then it again
> starts working correctly and will return "value".
> Doing $mc->set("key", "value", 0, 10) will not result in same issue, so
> problem might be that:
> 1. vmware keeps timestamps counted per process and the errors are aggregating
> over time
> 2. there's some cache for timestamp build into memcached so it's using some
> internal code for getting timestamp and the errors are aggregating there
>
> Any comments on that?
> Basically it'd be good to put into docs if using TIMESTAMP+time is safe to do
> on barebone servers and on VMs. Maybe some info that it shouldn't be used for
> small TTLs because fo
> r higher ones 1-2 minute differences are probably irrelevant anyway...
>
> Thanks,
> Slawomir.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "memcached" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to memcached+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/memcached/72771e82-79aa-4b23-9563-1b4099930e68%40googlegroups.com.
>
>
--
---
You received this message because you are subscribed to the Google Groups
"memcached" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to memcached+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1908120210240.18661%40dskull.
Re: building issue
you're sure it was a clean compile after the NEED_ALIGN change? Any idea what the most recent version to build/pass tests is? I'm not sure how to fix this without access to HPPA hardware. It could take ages to back and forth about it :( On Sun, 28 Jul 2019, mike hosken wrote: > Still fails log attached > Mike > > On Saturday, July 27, 2019 at 11:26:56 PM UTC+12, mike hosken wrote: > > Hi, > > > > The output files are attached > > > > > > > > The output of the syslog > > root@atlantis-isp:/srv# cd memcached-1.5.16 > > root@atlantis-isp:/srv/memcached-1.5.16# prove t/*.t &>/srv/prove2 > > [ 1744.125238] memcached-debug(2582): unaligned access to > 0xf8cfffd2 at ip=0x0001b2c3 > > [ 1744.150158] memcached-debug(2582): unaligned access to > 0xf8cfffd2 at ip=0x0001b2c3 > > [ 1744.163229] memcached-debug(2582): unaligned access to > 0xf8cfff82 at ip=0x0001b2c3 > > [ 1745.325421] memcached-debug(2591): unaligned access to > 0xf8cfefd2 at ip=0x00039ecb > > [ 1745.325728] memcached-debug(2591): unaligned access to > 0xf8cfefd6 at ip=0x00039ecf > > [ 1755.206857] handle_unaligned: 2619 callbacks suppressed > > [ 1755.206875] memcached-debug(2582): unaligned access to > 0xf8cfff3a at ip=0x0001b2c3 > > [ 1755.230171] memcached-debug(2582): unaligned access to > 0xf8cffeea at ip=0x0001b2c3 > > [ 1755.250607] memcached-debug(2582): unaligned access to > 0xf8cfffde at ip=0x0001b2c3 > > [ 1755.265211] memcached-debug(2582): unaligned access to > 0xf8cfff8e at ip=0x0001b2c3 > > [ 1755.647999] memcached-debug(2582): unaligned access to > 0xf8cffe42 at ip=0x0001b2c3 > > [ 1760.705014] handle_unaligned: 10 callbacks suppressed > > [ 1760.705032] memcached-debug(2582): unaligned access to > 0xf8d23289 at ip=0x0001eab7 > > [ 1760.705713] memcached-debug(2582): unaligned access to > 0xf8d2328d at ip=0x0001eabb > > [ 1760.711934] memcached-debug(2582): unaligned access to > 0xf8d23291 at ip=0x0001eacf > > [ 1760.716485] memcached-debug(2582): unaligned access to > 0xf8d23295 at ip=0x0001ead3 > > [ 1760.721071] memcached-debug(2582): unaligned access to > 0xf8d23299 at ip=0x0001eadf > > [ 1765.738790] handle_unaligned: 7767 callbacks suppressed > > [ 1765.738809] memcached-debug(2582): unaligned access to > 0xf8d232cf at ip=0x0001eab7 > > [ 1765.739522] memcached-debug(2582): unaligned access to > 0xf8d232d3 at ip=0x0001eabb > > [ 1765.745671] memcached-debug(2582): unaligned access to > 0xf8d232d7 at ip=0x0001eacf > > [ 1765.750246] memcached-debug(2582): unaligned access to > 0xf8d232db at ip=0x0001ead3 > > [ 1765.754858] memcached-debug(2582): unaligned access to > 0xf8d232df at ip=0x0001eadf > > [ 1770.834862] handle_unaligned: 7692 callbacks suppressed > > [ 1770.834877] memcached-debug(2582): unaligned access to > 0xf1fec11e at ip=0x0001b2c3 > > [ 1770.845364] memcached-debug(2582): unaligned access to > 0xf8cfe3c2 at ip=0x0001b2c3 > > [ 1770.910709] memcached-debug(2582): unaligned access to > 0xf8d22b25 at ip=0x0001eab7 > > [ 1770.911011] memcached-debug(2582): unaligned access to > 0xf8d22b29 at ip=0x0001eabb > > [ 1770.911522] memcached-debug(2582): unaligned access to > 0xf8d22b2d at ip=0x0001eacf > > [ 1775.838744] handle_unaligned: 10050 callbacks suppressed > > [ 1775.838759] memcached-debug(2582): unaligned access to > 0xf8d22b25 at ip=0x0001eab7 > > [ 1775.839501] memcached-debug(2582): unaligned access to > 0xf8d22b29 at ip=0x0001eabb > > [ 1775.845809] memcached-debug(2582): unaligned access to > 0xf8d22b2d at ip=0x0001eacf > > [ 1775.850395] memcached-debug(2582): unaligned access to > 0xf8d22b31 at ip=0x0001ead3 > > [ 1775.855029] memcached-debug(2582): unaligned access to > 0xf8d22b35 at ip=0x0001eadf > > [ 1780.878981] handle_unaligned: 10900 callbacks suppressed > > [ 1780.878996] memcached-debug(2582): unaligned access to > 0xf8d22b25 at ip=0x0001eab7 > > [ 1780.879705] memcached-debug(2582): unaligned access to > 0xf8d22b29 at ip=0x0001eabb > > [ 1780.885682] memcached-debug(2582): unaligned access to > 0xf8d22b2d at ip=0x0001eacf > > [ 1780.890234] memcached-debug(2582): unaligned access to > 0xf8d22b31 at ip=0x0001ead3 > > [
Re: building issue
Smells like NEED_ALIGN isn't getting set... after configure, but before make: edit config.h: /* Machine need alignment */ /* #undef NEED_ALIGN */ ^ change this to: #define NEED_ALIGN 1 ... then run make and make test? On Sat, 27 Jul 2019, Mike Hosken wrote: > > Hi, > > > > The output files are attached > > > > > > > > The output of the syslog > > root@atlantis-isp:/srv# cd memcached-1.5.16 > > root@atlantis-isp:/srv/memcached-1.5.16# prove t/*.t &>/srv/prove2 > > [ 1744.125238] memcached-debug(2582): unaligned access to 0xf8cfffd2 > at ip=0x0001b2c3 > > [ 1744.150158] memcached-debug(2582): unaligned access to 0xf8cfffd2 > at ip=0x0001b2c3 > > [ 1744.163229] memcached-debug(2582): unaligned access to 0xf8cfff82 > at ip=0x0001b2c3 > > [ 1745.325421] memcached-debug(2591): unaligned access to 0xf8cfefd2 > at ip=0x00039ecb > > [ 1745.325728] memcached-debug(2591): unaligned access to 0xf8cfefd6 > at ip=0x00039ecf > > [ 1755.206857] handle_unaligned: 2619 callbacks suppressed > > [ 1755.206875] memcached-debug(2582): unaligned access to 0xf8cfff3a > at ip=0x0001b2c3 > > [ 1755.230171] memcached-debug(2582): unaligned access to 0xf8cffeea > at ip=0x0001b2c3 > > [ 1755.250607] memcached-debug(2582): unaligned access to 0xf8cfffde > at ip=0x0001b2c3 > > [ 1755.265211] memcached-debug(2582): unaligned access to 0xf8cfff8e > at ip=0x0001b2c3 > > [ 1755.647999] memcached-debug(2582): unaligned access to 0xf8cffe42 > at ip=0x0001b2c3 > > [ 1760.705014] handle_unaligned: 10 callbacks suppressed > > [ 1760.705032] memcached-debug(2582): unaligned access to 0xf8d23289 > at ip=0x0001eab7 > > [ 1760.705713] memcached-debug(2582): unaligned access to 0xf8d2328d > at ip=0x0001eabb > > [ 1760.711934] memcached-debug(2582): unaligned access to 0xf8d23291 > at ip=0x0001eacf > > [ 1760.716485] memcached-debug(2582): unaligned access to 0xf8d23295 > at ip=0x0001ead3 > > [ 1760.721071] memcached-debug(2582): unaligned access to 0xf8d23299 > at ip=0x0001eadf > > [ 1765.738790] handle_unaligned: 7767 callbacks suppressed > > [ 1765.738809] memcached-debug(2582): unaligned access to 0xf8d232cf > at ip=0x0001eab7 > > [ 1765.739522] memcached-debug(2582): unaligned access to 0xf8d232d3 > at ip=0x0001eabb > > [ 1765.745671] memcached-debug(2582): unaligned access to 0xf8d232d7 > at ip=0x0001eacf > > [ 1765.750246] memcached-debug(2582): unaligned access to 0xf8d232db > at ip=0x0001ead3 > > [ 1765.754858] memcached-debug(2582): unaligned access to 0xf8d232df > at ip=0x0001eadf > > [ 1770.834862] handle_unaligned: 7692 callbacks suppressed > > [ 1770.834877] memcached-debug(2582): unaligned access to 0xf1fec11e > at ip=0x0001b2c3 > > [ 1770.845364] memcached-debug(2582): unaligned access to 0xf8cfe3c2 > at ip=0x0001b2c3 > > [ 1770.910709] memcached-debug(2582): unaligned access to 0xf8d22b25 > at ip=0x0001eab7 > > [ 1770.911011] memcached-debug(2582): unaligned access to 0xf8d22b29 > at ip=0x0001eabb > > [ 1770.911522] memcached-debug(2582): unaligned access to 0xf8d22b2d > at ip=0x0001eacf > > [ 1775.838744] handle_unaligned: 10050 callbacks suppressed > > [ 1775.838759] memcached-debug(2582): unaligned access to 0xf8d22b25 > at ip=0x0001eab7 > > [ 1775.839501] memcached-debug(2582): unaligned access to 0xf8d22b29 > at ip=0x0001eabb > > [ 1775.845809] memcached-debug(2582): unaligned access to 0xf8d22b2d > at ip=0x0001eacf > > [ 1775.850395] memcached-debug(2582): unaligned access to 0xf8d22b31 > at ip=0x0001ead3 > > [ 1775.855029] memcached-debug(2582): unaligned access to 0xf8d22b35 > at ip=0x0001eadf > > [ 1780.878981] handle_unaligned: 10900 callbacks suppressed > > [ 1780.878996] memcached-debug(2582): unaligned access to 0xf8d22b25 > at ip=0x0001eab7 > > [ 1780.879705] memcached-debug(2582): unaligned access to 0xf8d22b29 > at ip=0x0001eabb > > [ 1780.885682] memcached-debug(2582): unaligned access to 0xf8d22b2d > at ip=0x0001eacf > > [ 1780.890234] memcached-debug(2582): unaligned access to 0xf8d22b31 > at ip=0x0001ead3 > > [ 1780.895011] memcached-debug(2582): unaligned access to 0xf8d22b35 > at ip=0x0001eadf > > [ 1785.882967] handle_unaligned: 11530 callbacks suppressed > > [ 1785.882982] memcached-debug(2582): unaligned access to 0xf8d22b25 > at ip=0x0001eab7 > > [ 1785.883692] memcached-debug(2582): unaligned access to 0xf8d22b29 > at ip=0x0001eabb > > [ 1785.889671] memcached-debug(2582):
Re: memcached fails to build
looks like it actually built I guess. I've never had an hppa machine. I do build test on ARM, but that's about it for non-x86 CPU's lately. after configure and make, what's the output of: $ prove t/00-startup.t If that passes, try: $ prove t/*.t .. and see which of those fails. On Sat, 27 Jul 2019, mike hosken wrote: > root@atlantis-isp:~# gcc -v > Using built-in specs. > COLLECT_GCC=gcc > COLLECT_LTO_WRAPPER=/usr/lib/gcc/hppa-linux-gnu/8/lto-wrapper > Target: hppa-linux-gnu > Configured with: ../src/configure -v --with-pkgversion='Debian 8.3.0-7' > --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs > --enable-languages=c,ada,c++,d,fortran,objc,obj-c++ > --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 > --program-prefix=hppa-linux-gnu- --enable-shared --enable-linker-build-id > --libexecdir=/usr/lib > --without-included-gettext --enable-threads=posix --libdir=/usr/lib > --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug > --enable-libstdcxx-time=yes > --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libssp > --disable-libitm --disable-libsanitizer --disable-libquadmath > --disable-libquadmath-support > --enable-plugin --with-system-zlib --disable-libphobos --enable-objc-gc=auto > --enable-multiarch --disable-werror --disable-libstdcxx-pch > --enable-checking=release > --build=hppa-linux-gnu --host=hppa-linux-gnu --target=hppa-linux-gnu > Thread model: posix > gcc version 8.3.0 (Debian 8.3.0-7) > > > On Saturday, July 27, 2019 at 6:22:38 PM UTC+12, mike hosken wrote: > Hi > I have been trying to build memcached on debian hppa linux > > it fails to build in debian see log here > > https://debian-stable.parisc-linux.org/build/memcached_1.5.6-1.1_hppa.build > > it also fails to build from source > root@atlantis-isp:~/memcached-1.5.16# ./configure > --prefix=/usr/local/memcached > checking build system type... hppa64-unknown-linux-gnu > checking host system type... hppa64-unknown-linux-gnu > checking for a BSD-compatible install... /usr/bin/install -c > checking whether build environment is sane... yes > checking for a thread-safe mkdir -p... /usr/bin/mkdir -p > checking for gawk... gawk > checking whether make sets $(MAKE)... yes > checking whether make supports nested variables... yes > checking for gcc... gcc > checking whether the C compiler works... yes > checking for C compiler default output file name... a.out > checking for suffix of executables... > checking whether we are cross compiling... no > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether gcc accepts -g... yes > checking for gcc option to accept ISO C89... none needed > checking whether gcc understands -c and -o together... yes > checking whether make supports the include directive... yes (GNU style) > checking dependency style of gcc... gcc3 > checking how to run the C preprocessor... gcc -E > checking for grep that handles long lines and -e... /usr/bin/grep > checking for egrep... /usr/bin/grep -E > checking for icc in use... no > checking for clang in use... no > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for unistd.h... yes > checking whether __SUNPRO_C is declared... no > checking for gcc option to accept ISO C99... none needed > checking sasl/sasl.h usability... yes > checking sasl/sasl.h presence... yes > checking for sasl/sasl.h... yes > checking for gcov... /usr/bin/gcov > checking for main in -lgcov... yes > checking for library containing clock_gettime... none required > checking for library containing socket... none required > checking for library containing gethostbyname... none required > checking for libevent directory... (system) > checking for library containing umem_cache_create... no > checking for library containing gethugepagesizes... no > checking for stdbool.h that conforms to C99... yes > checking for _Bool... yes > checking for inttypes.h... (cached) yes > checking for sasl_callback_ft... yes > checking for print macros for integers (C99 section 7.8.1)... yes > checking for an ANSI C-conforming const... yes > checking for socklen_t... yes > checking for endianness... big > checking for htonll... no > checking for library containing pthread_create... none required > checking for mlockall... yes > checking for getpagesizes... no > checking for memcntl... no > checking for sigignore... yes > checking for clock_gettime... yes > checking for accept4... yes > checking for getopt_long... yes > checking for alignment... none > checking for GCC atomics... yes > checking for GCC 64bit atomics... yes > checking for setppriv... no > checking for pledge... no > checking for cap_enter... no > checking umem.h usability... no > checking
Re: memcached fails to build
What does "make clean ; make" look like in that directory? make isn't doing anything, is there even a memcached binary in the directory? On Fri, 26 Jul 2019, mike hosken wrote: > Hi > I have been trying to build memcached on debian hppa linux > > it fails to build in debian see log here > > https://debian-stable.parisc-linux.org/build/memcached_1.5.6-1.1_hppa.build > > it also fails to build from source > root@atlantis-isp:~/memcached-1.5.16# ./configure > --prefix=/usr/local/memcached > checking build system type... hppa64-unknown-linux-gnu > checking host system type... hppa64-unknown-linux-gnu > checking for a BSD-compatible install... /usr/bin/install -c > checking whether build environment is sane... yes > checking for a thread-safe mkdir -p... /usr/bin/mkdir -p > checking for gawk... gawk > checking whether make sets $(MAKE)... yes > checking whether make supports nested variables... yes > checking for gcc... gcc > checking whether the C compiler works... yes > checking for C compiler default output file name... a.out > checking for suffix of executables... > checking whether we are cross compiling... no > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether gcc accepts -g... yes > checking for gcc option to accept ISO C89... none needed > checking whether gcc understands -c and -o together... yes > checking whether make supports the include directive... yes (GNU style) > checking dependency style of gcc... gcc3 > checking how to run the C preprocessor... gcc -E > checking for grep that handles long lines and -e... /usr/bin/grep > checking for egrep... /usr/bin/grep -E > checking for icc in use... no > checking for clang in use... no > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for unistd.h... yes > checking whether __SUNPRO_C is declared... no > checking for gcc option to accept ISO C99... none needed > checking sasl/sasl.h usability... yes > checking sasl/sasl.h presence... yes > checking for sasl/sasl.h... yes > checking for gcov... /usr/bin/gcov > checking for main in -lgcov... yes > checking for library containing clock_gettime... none required > checking for library containing socket... none required > checking for library containing gethostbyname... none required > checking for libevent directory... (system) > checking for library containing umem_cache_create... no > checking for library containing gethugepagesizes... no > checking for stdbool.h that conforms to C99... yes > checking for _Bool... yes > checking for inttypes.h... (cached) yes > checking for sasl_callback_ft... yes > checking for print macros for integers (C99 section 7.8.1)... yes > checking for an ANSI C-conforming const... yes > checking for socklen_t... yes > checking for endianness... big > checking for htonll... no > checking for library containing pthread_create... none required > checking for mlockall... yes > checking for getpagesizes... no > checking for memcntl... no > checking for sigignore... yes > checking for clock_gettime... yes > checking for accept4... yes > checking for getopt_long... yes > checking for alignment... none > checking for GCC atomics... yes > checking for GCC 64bit atomics... yes > checking for setppriv... no > checking for pledge... no > checking for cap_enter... no > checking umem.h usability... no > checking umem.h presence... no > checking for umem.h... no > checking for xml2rfc... no > checking for xsltproc... /usr/bin/xsltproc > checking that generated files are newer than configure... done > configure: creating ./config.status > config.status: creating Makefile > config.status: creating doc/Makefile > config.status: creating config.h > config.status: config.h is unchanged > config.status: executing depfiles commands > root@atlantis-isp:~/memcached-1.5.16# > > root@atlantis-isp:~/memcached-1.5.16# make > make all-recursive > make[1]: Entering directory '/root/memcached-1.5.16' > Making all in doc > make[2]: Entering directory '/root/memcached-1.5.16/doc' > make all-am > make[3]: Entering directory '/root/memcached-1.5.16/doc' > make[3]: Nothing to be done for 'all-am'. > make[3]: Leaving directory '/root/memcached-1.5.16/doc' > make[2]: Leaving directory '/root/memcached-1.5.16/doc' > make[2]: Entering directory '/root/memcached-1.5.16' > make[2]: Leaving directory '/root/memcached-1.5.16' > make[1]: Leaving directory '/root/memcached-1.5.16' > root@atlantis-isp:~/memcached-1.5.16# > > root@atlantis-isp:~/memcached-1.5.16# make test > ./sizes > Slab Stats 64 > Thread stats -6464 > Global stats 160 > Settings 224 > Item (no cas) 32 > Item (cas) 40 > Libevent thread 96 > Connection 352 > >
1.5.15 - ascii authentication mode
https://github.com/memcached/memcached/wiki/ReleaseNotes1515 -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905201402120.28905%40dskull. For more options, visit https://groups.google.com/d/optout.
ascii auth mode
https://github.com/memcached/memcached/pull/489 only user/pass in case users should be useful roles in the future, but for now they're just blind tokens; everyone gets the same access. need to decide on the SET method or switching to a GET command, else the rest are small TODO items. I'd like to release this ASAP. In a few months or a year `-o modern` should default to auth being enabled and looking for a file in some random path. Distro's should be free to default to using the option with a randomly generated token (or however they handle it for similar systems). Thoughts? -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905112204300.29273%40dskull. For more options, visit https://groups.google.com/d/optout.
Re: binary protocol?
flatbuffers are unrelated to protocol, and are definitely not backwards compatible with ascii :p On Thu, 9 May 2019, Roberto Spadim wrote: > maybe could be used something like flatbuffers? > > Em qui, 9 de mai de 2019 às 20:15, dormando escreveu: > To that end, I think SASL's confused everyone who's ever met it, and I > don't think anyone actually asked for it back in binprot days. > > Said I was going to do this before but hadn't... Lets try something > simple, and if someone asks for SASL I think it's actually easy: > > client connect: > set auth 0 0 [length] > username password > \r\n > > ... and if you get back a STORED, you're good to go. if you try to run > any > other command you'll get: > > CLIENT_ERROR unauthorized > > password database is a username:password\n file on disk, reloaded when > it > changes. no fancy datastructures so will get slow if people use a > billion > users. user's don't do anything, but putting them there now in case we > ever decide to. > > if we ever decide to support SASL again: > > client connect: > get mech-list > MECH CRAM-MD5 PLAIN ETC > END > > set sasl-step 0 0 [length] > data > STORED > > get authenticated > [indicates done or STEP if SASL requires more steps] > > ... extra roundtrip in the protocol but this could be solved with > mget/mset. > > On Thu, 9 May 2019, dormando wrote: > > > Indeed we would. SASL is actually a text based protocol, as it's used > in > > SMTP and similar. That shouldn't be too bad. > > > > On Thu, 9 May 2019, Om Kale wrote: > > > > > Hey Dormando,As of now only binary protocol supports SASL. > > > Correct me if I am wrong but to ensure compatibility with SASL and > libmemcached clients we would need binary protocol support right? > > > > > > > > > Thanks and Regards,Om Kale > > > > > > > > > > > > On Thu, May 9, 2019 at 2:58 PM dormando wrote: > > > Yo, > > > > > > Any of you out there really _like_ the binary protocol? Aside > from the > > > features it gives (CAS everywhere, pipelining, etc). > > > > > > just thinkin' through some things. > > > > > > Thanks, > > > -Dormando > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the > Google Groups "memcached" group. > > > To unsubscribe from this group and stop receiving emails from > it, send an email to memcached+unsubscr...@googlegroups.com. > > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905091457160.27082%40dskull. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+unsubscr...@googlegroups.com. > > > To view this discussion on the web visit > > > > https://groups.google.com/d/msgid/memcached/CANvBZ%3DvraaTi48Yu-a07gfc-OCAJ%3DfJfA_z97XA3UvjFretK5g%40mail.gmail.com. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905091527370.27082%40dskull. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > --- > You received this message because you are subscribed to the Google > Groups "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to memcached+unsubscr...@goo
Re: binary protocol?
To that end, I think SASL's confused everyone who's ever met it, and I don't think anyone actually asked for it back in binprot days. Said I was going to do this before but hadn't... Lets try something simple, and if someone asks for SASL I think it's actually easy: client connect: set auth 0 0 [length] username password \r\n ... and if you get back a STORED, you're good to go. if you try to run any other command you'll get: CLIENT_ERROR unauthorized password database is a username:password\n file on disk, reloaded when it changes. no fancy datastructures so will get slow if people use a billion users. user's don't do anything, but putting them there now in case we ever decide to. if we ever decide to support SASL again: client connect: get mech-list MECH CRAM-MD5 PLAIN ETC END set sasl-step 0 0 [length] data STORED get authenticated [indicates done or STEP if SASL requires more steps] ... extra roundtrip in the protocol but this could be solved with mget/mset. On Thu, 9 May 2019, dormando wrote: > Indeed we would. SASL is actually a text based protocol, as it's used in > SMTP and similar. That shouldn't be too bad. > > On Thu, 9 May 2019, Om Kale wrote: > > > Hey Dormando,As of now only binary protocol supports SASL. > > Correct me if I am wrong but to ensure compatibility with SASL and > > libmemcached clients we would need binary protocol support right? > > > > > > Thanks and Regards,Om Kale > > > > > > > > On Thu, May 9, 2019 at 2:58 PM dormando wrote: > > Yo, > > > > Any of you out there really _like_ the binary protocol? Aside from the > > features it gives (CAS everywhere, pipelining, etc). > > > > just thinkin' through some things. > > > > Thanks, > > -Dormando > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > > send an email to memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905091457160.27082%40dskull. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > > > --- > > You received this message because you are subscribed to the Google Groups > > "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to memcached+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/memcached/CANvBZ%3DvraaTi48Yu-a07gfc-OCAJ%3DfJfA_z97XA3UvjFretK5g%40mail.gmail.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905091527370.27082%40dskull. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905091608320.27082%40dskull. For more options, visit https://groups.google.com/d/optout.
Re: binary protocol?
Indeed we would. SASL is actually a text based protocol, as it's used in SMTP and similar. That shouldn't be too bad. On Thu, 9 May 2019, Om Kale wrote: > Hey Dormando,As of now only binary protocol supports SASL. > Correct me if I am wrong but to ensure compatibility with SASL and > libmemcached clients we would need binary protocol support right? > > > Thanks and Regards,Om Kale > > > > On Thu, May 9, 2019 at 2:58 PM dormando wrote: > Yo, > > Any of you out there really _like_ the binary protocol? Aside from the > features it gives (CAS everywhere, pipelining, etc). > > just thinkin' through some things. > > Thanks, > -Dormando > > -- > > --- > You received this message because you are subscribed to the Google > Groups "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905091457160.27082%40dskull. > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/memcached/CANvBZ%3DvraaTi48Yu-a07gfc-OCAJ%3DfJfA_z97XA3UvjFretK5g%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905091527370.27082%40dskull. For more options, visit https://groups.google.com/d/optout.
binary protocol?
Yo, Any of you out there really _like_ the binary protocol? Aside from the features it gives (CAS everywhere, pipelining, etc). just thinkin' through some things. Thanks, -Dormando -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/memcached/alpine.DEB.2.21.1905091457160.27082%40dskull. For more options, visit https://groups.google.com/d/optout.
metaget (mget) PR
A smallish patch with some significant feature change: https://github.com/memcached/memcached/pull/484 1) all of the potential fetch modes (do/don't bump LRU, etc) 2) currently hidden metadata (TTL remaining, last access time, etc) 3) human-readable (or easily script-readable) dump of all possible metadata 4) thundering-herd lease-like passive lock feature. 5) stale-while-revalidate semantics 6) etc. fleshing it out over the next couple days. hoping I haven't screwed it up conceptually. -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
persistent memory blog
https://memcached.org/blog/persistent-memory/ did some extensive testing with the new persistent memory modules from intel. If you can get them, they're pretty good. A lot less work to use than I was expecting. -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
TLS support is out
https://github.com/memcached/memcached/wiki/ReleaseNotes1513 enjoy. -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
memcached TLS support branch
Anyone want to take a look at/spin with this? https://github.com/memcached/memcached/pull/440 no clients actually support TLS/SSL yet; you'd have to use an stunnel/socat wrapper or modify a client. curious to hear any feedback about usability/features/etc. Thanks! -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: "Operation now in progress" error on delete
Well I don't know what that is offhand. Not much of a python user myself Some quick googling shows it's likely a timeout: https://github.com/lericson/pylibmc/issues/89 - there's nothing about deletes that would cause that specifically, so it might be happening all the time. Or you're calling deletes on the wrong object, batching them too much, causing protocol desync with bad keys (spaces/newlines) hopefully someone else has something more insightful to say. On Tue, 9 Apr 2019, Alex Bliskovsky wrote: > I'm using Python 3.4, Django 1.11 and Django's version of PyLibMC. > > Alexander Bliskovsky, Software Engineer > fusionbox▝ > o: 303.952.7490 > > www.fusionbox.com > > This information is confidential and intended solely for the use > of the individual or entity to whom it is addressed. If you have > received this email in error please notify the sender. > > On Tue, Apr 9, 2019 at 2:49 PM dormando wrote: > > > > That's not a protocol error. What client/language/etc are you using? > > > > On Tue, 9 Apr 2019, abliskov...@fusionbox.com wrote: > > > > > Hi, > > > > > > We are frequently getting the error "operation now in progress" when > > > attempting to delete keys from memcached, but I can find little > > > documentation about how to resolve them. Can anyone provide any > > > suggestions as to what to investigate to attempt to solve this issue? > > > > > > Thanks, > > > Alex > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the Google Groups > > > "memcached" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to memcached+unsubscr...@googlegroups.com. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > -- > > > > --- > > You received this message because you are subscribed to a topic in the > > Google Groups "memcached" group. > > To unsubscribe from this topic, visit > > https://groups.google.com/d/topic/memcached/51-8XUOBkpc/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > > memcached+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: "Operation now in progress" error on delete
That's not a protocol error. What client/language/etc are you using? On Tue, 9 Apr 2019, abliskov...@fusionbox.com wrote: > Hi, > > We are frequently getting the error "operation now in progress" when > attempting to delete keys from memcached, but I can find little > documentation about how to resolve them. Can anyone provide any suggestions > as to what to investigate to attempt to solve this issue? > > Thanks, > Alex > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Extend data tiering
On Mon, 8 Apr 2019, Luca Giacchino wrote: > Hi, > > Would it make sense to extend the concept of data tiers beyond extstore? > Instead of having a coupled architecture, an interface would be > defined between Memcached “core” and extended storage. The overall behavior > remains unchanged (extstore remains the de facto > implementation), but one could plug in more complex backend engines and > enable further tiering of the data based on access patterns. > More or less yeah. The interface is already pretty well defined, so it'd be easy to swap out the code and experiment. A bit of work to make it pluggable or compile/start time generic at least. Thing you have to keep in mind is the extstore design is "half tiered", ie; flat single authoritative hash table. Pointer objects into the "other" system, whatever it is. Otherwise the complexity balloons and the latency/write performance ends up worse than just using the underlying database by itself (sometimes, anyway). The extstore design itself could be extended for some types of tiering as well: https://github.com/memcached/memcached/pull/432 Objects are written into arbitrary "Buckets", that the memcached server has some general idea of what to do with. In theory bucket A could be extstore and bucket B could be different storage code, rather than simply a different data path/device. -Dormando -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Beginner issue
Yo,
-m doesn't take units, and apparently doesn't error if you give it a bad
string.
> b'limit_maxbytes': 67108864,
Sadly, it's gone to its default of 64 megabytes of memory. Sorry about
that :(
try: "-m 64000" - you should be able to confirm with 'ps' that you're not
using the memory as-is.
On Wed, 27 Mar 2019, Jerome Kieffer wrote:
> Hi Dormado,
>
> Thanks for your prompt feed-back.
>
> On Tue, 26 Mar 2019 11:06:10 -0700 (PDT)
> dormando wrote:
>
> > Seems like this is a borderline use case, but it might still work for you.
>
> From what I read on the internet, it looks like we are miss-using the
> tool ... but on the other hand I don't see why it shouldn't work ...
>
>
> > How did you verify you found the cause? Can you share snapshots from
> > "stats items" and "stats slabs" output after your test was run?
> >
> > Memory isn't evenly distributed; it's assigned where objects actually
> > exist. so either you've filled the server with other objects or
> > there's been a miscalculation. It's possible you're hitting client
> > timeouts or something.
>
> This is how I performed the test, using one of the many Python bindings (I
> tested 3 of them with the same effect)
>
>
> On the server side:
> memcached -m 64G -I 16m -vv
>
> On the client side:
> Generate the data:
>
> import numpy
> shape = (2048,2048)
> nframes, scan = 1024, 0
> data = [numpy.random.randint(0,65530,
> numpy.prod(shape)).reshape(shape).astype("uint16") for i in range(nframes)]
> print(len(data[0].tostring()), type(data[0].tostring()))
>
>
> --> 8388608
>
> Connection:
> from pymemcache.client.base import Client
> client = Client(('127.0.0.1', 11211))
>
> The test for writing:
> %time scan+=1;res=[client.set('scan%d_frame%d'%(scan,idx),
> frame.tobytes()) for idx,frame in enumerate(data)]
> --> 8.5s for 8G of data
>
> client.stats()
>
> {b'pid': 12731,
> b'uptime': 390,
> b'time': 1553627843,
> b'version': b'1.5.6',
> b'libevent': b'2.0.21-stable',
> b'pointer_size': 64,
> b'rusage_user': 0.174296,
> b'rusage_system': 2.338119,
> b'max_connections': 1024,
> b'curr_connections': 1,
> b'total_connections': 2,
> b'rejected_connections': 0,
> b'connection_structures': 2,
> b'reserved_fds': 20,
> b'cmd_get': 0,
> b'cmd_set': 1024,
> b'cmd_flush': 0,
> b'cmd_touch': 0,
> b'get_hits': 0,
> b'get_misses': 0,
> b'get_expired': 0,
> b'get_flushed': 0,
> b'delete_misses': 0,
> b'delete_hits': 0,
> b'incr_misses': 0,
> b'incr_hits': 0,
> b'decr_misses': 0,
> b'decr_hits': 0,
> b'cas_misses': 0,
> b'cas_hits': 0,
> b'cas_badval': 0,
> b'touch_hits': 0,
> b'touch_misses': 0,
> b'auth_cmds': 0,
> b'auth_errors': 0,
> b'bytes_read': 8589977522,
> b'bytes_written': 0,
> b'limit_maxbytes': 67108864,
> b'accepting_conns': 1,
> b'listen_disabled_num': 0,
> b'time_in_listen_disabled_us': 0,
> b'threads': 4,
> b'conn_yields': 0,
> b'hash_power_level': 16,
> b'hash_bytes': 524288,
> b'hash_is_expanding': False,
> b'slab_reassign_rescues': 0,
> b'slab_reassign_chunk_rescues': 2,
> b'slab_reassign_evictions_nomem': 0,
> b'slab_reassign_inline_reclaim': 0,
> b'slab_reassign_busy_items': 0,
> b'slab_reassign_busy_deletes': 0,
> b'slab_reassign_running': False,
> b'slabs_moved': 1,
> b'lru_crawler_running': 0,
> b'lru_crawler_starts': 1020,
> b'lru_maintainer_juggles': 5931,
> b'malloc_fails': 0,
> b'log_worker_dropped': 0,
> b'log_worker_written': 0,
> b'log_watcher_skipped': 0,
> b'log_watcher_sent': 0,
> b'bytes': 58720774,
> b'curr_items': 7,
> b'total_items': 1024,
> b'slab_global_page_pool': 0,
> b'expired_unfetched': 0,
> b'evicted_unfetched': 1017, <==
> b'evicted_active': 0,
> b'evictions': 1017,
> b'reclaimed': 0,
> b'crawler_reclaimed': 0,
> b'crawler_items_checked': 6,
> b'lrutail_reflocked': 4,
> b'moves_to_cold': 1024,
> b'moves_to_warm': 0,
> b'moves_within_lru': 0,
> b'direct_reclaims': 1197,
> b'lru_bumps_dropped': 0}
>
> client.stats("items")
>
> {b'items:39:number': 7,
> b'items:39:number_hot': 0,
> b'items:39:number_warm': 0,
> b'items:39:number_cold': 7,
> b'items:39:age_hot': 0,
> b'items:39:age_warm': 0,
> b'items:39:age': 71,
> b'items:39:evicted': 1017,
Re: Beginner issue
Seems like this is a borderline use case, but it might still work for you. How did you verify you found the cause? Can you share snapshots from "stats items" and "stats slabs" output after your test was run? Memory isn't evenly distributed; it's assigned where objects actually exist. so either you've filled the server with other objects or there's been a miscalculation. It's possible you're hitting client timeouts or something. On Tue, 26 Mar 2019, Jerome Kieffer wrote: > Hi, > > I am completely new to memcached but it looks like the right tool for what I > need to do: > I would like to do temporary storage of some data coming from a detector at a > pretty high data-rate and share them with other computer > via memcached. The image size varies from 512x512 to 4096x4096 which makes > the raw data size from 0.5 to 32 MB (hence compatible with > memcached). I wish to use some fast (nvme) SSD as extra memory as well. > > I setup a quick benchmark, initially to measure speeds (memcached 1.5.8 from > debian10): > * Memcached is setup to accept object up to 128m and use up to 64G of RAM > (optional SSD storage coming later ...) > * Store 1024 images of 2048x2048 as uint16 which represents ~8G of RAM. > > Actually only a limited number of frames were still available in the cache > after the end of the write (between 7 and 120 frames). > > I found the cause: all frames have the same size hence fall into the same > slab (the one for the largest objects), which is of limited > size because chunk slabs are evenly distributed over object sizes. > > This raises 2 questions: > * How can I assign "most" of the available memory to a given size avoid > dropping frames ? (I know in advance what will be the size of > the objects) > * Maybe memcached is not the right tool and you could indicate another tool > better adapted ? > > Thanks in advance for your help. > > Cheers, > > Jérôme > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: how to get find out key expiration time after it was put in the cache?
Got it. stats cachedump might find it, but it's limited to "a megabyte" worth of keys/metadata per slab class. it'll also hang your server while it fills up the buffer. in newer versions you can definitely find it with lru_crawler metadump [slabclass]. I'll be improving those interfaces too. Questions like this help me figure out which direction to go :) On Fri, 22 Mar 2019, 'Nikhil' via memcached wrote: > Got it. Thank you Dormando and Daniel for your prompt responses. > Dormando, yes I am debugging a corner case, where I deleted an unused feature > that queried a key from memcache (that key is not in the > db, so cannot be in memcache after TTL expired). But it happens to be there, > so I want to know the TTL to be sure that it was put there > incorrectly (possibly with 0 ie never expire). > > Thanks > Nikhil > > On Friday, March 22, 2019 at 12:32:12 PM UTC-7, Dormando wrote: > There's no way to retrieve the metadata back from an item aside from the > client flags as part of a normal command. > > What people typically do is embed the original TTL into the object > you've > stored. that way the application can read it or any other metadata you > want to make decisions with. > > Though I see you said "by hand" - are you debugging something or > designing > a flow in your app to remove stale data? > > If you're debugging, "lru_crawler metadump" will get you the data, but > you > need a newer version I think. > > On Fri, 22 Mar 2019, Daniel Ellis wrote: > > > As far as I know, cachedump is the only way. The unfortunate > downside is that you can only dump one page per slab, > according to this. > > I bet you could get fancy and dump the entire process memory and dig > through it yourself... That could be fun. > > > > On Fri, Mar 22, 2019 at 11:27 AM nagashe via memcached > wrote: > > Hi All, > > > > I have a `key` which was put in the cache, the database does not have > the key anymore, but since it was in the cache our > app gets > > the value. I want to know when it would expire before I go in and try > to delete it by hand. What is the recommended way to > find > > the key expiration time after the fact? We are running 1.4.20 (i know > its a few years old). I tried to figure out by > searching on > > google, some people recommended `stats cachedump ` > > > (https://stackoverflow.com/questions/2558706/how-can-i-get-the-expire-time-for-the-particular-item-in-memcached). > I wanted > to > > check here if there is some better or "recommended" way > > > > Thanks > > Nikhil > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: how to get find out key expiration time after it was put in the cache?
There's no way to retrieve the metadata back from an item aside from the client flags as part of a normal command. What people typically do is embed the original TTL into the object you've stored. that way the application can read it or any other metadata you want to make decisions with. Though I see you said "by hand" - are you debugging something or designing a flow in your app to remove stale data? If you're debugging, "lru_crawler metadump" will get you the data, but you need a newer version I think. On Fri, 22 Mar 2019, Daniel Ellis wrote: > As far as I know, cachedump is the only way. The unfortunate downside is > that you can only dump one page per slab, according to this. > I bet you could get fancy and dump the entire process memory and dig through > it yourself... That could be fun. > > On Fri, Mar 22, 2019 at 11:27 AM nagashe via memcached > wrote: > Hi All, > > I have a `key` which was put in the cache, the database does not have the key > anymore, but since it was in the cache our app gets > the value. I want to know when it would expire before I go in and try to > delete it by hand. What is the recommended way to find > the key expiration time after the fact? We are running 1.4.20 (i know its a > few years old). I tried to figure out by searching on > google, some people recommended `stats cachedump ` > (https://stackoverflow.com/questions/2558706/how-can-i-get-the-expire-time-for-the-particular-item-in-memcached). > I wanted to > check here if there is some better or "recommended" way > > Thanks > Nikhil > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Memcached client with murmur3
ketama's just fine. Optimize for problems you have or can prove. The default in newer versions is murmur. I keep forgetting that AWS elasticache is stuck on that old version. Also beware they seem to have some bugs. If you run into trouble just run your own instances; it's not too hard. On Wed, 20 Mar 2019, Eranda Rajapakshe wrote: > Thanks a lot, Dormando! > I'm using KETAMA_HASH at the client side, didn't go for CRC_HASH as per the > Java Doc it says CRC_HASH can be slower in performance (and > we are only having Java client, so no need to support for multiple > platforms). We only have max 10 nodes of Memcached, but we want to > have a fair distribution of values across (at the moment we see 2 nodes are > getting selected in a higher frequency). That's the main > reason for trying out Murmur3. Is there any default algorithm you can > recommend to solve this issue? > > Server-side I will stick for Jenkins for the moment then! Thanks for the > quick response Dormando. > > Thanks, > Eranda > > On Thursday, March 21, 2019 at 1:25:13 PM UTC+8, Dormando wrote: > You don't need to worry about the server one at all. They don't need to > match up either; on the server side it's just for the hash table. The > default is fine. I only left jenkins in as a "just in case", too. > > I don't think it matters that much for clients either. There're very few > buckets involved for clients (just the number of servers) so it'd have > to > be a very poor algorithm to display a significant bias. Most original > clients use crc32 and they did fine. > > On Wed, 20 Mar 2019, Eranda Rajapakshe wrote: > > > Hi, > > > > I'm quite an amateur on Memcached, please excuse me if my > understanding is not correct. I'm trying to connect to AWS > ElastiCache > > Memcached servers from my Spring service. As per my understanding > cache key hashing is done at two locations.1. client > side to decide > > the Memcache node > > 2. at Memcache node to get the value corresponding to the key > > > > For hashing in Memcached server, I found that two possible algorithms > are Jenkins and Murmur. [1] For client-side hashing, > I see several > > default algorithms provided in net.spy.memcached.DefaultHashAlgorithm > class. But I cannot see Murmur algorithm in the Java > client > > library as a default algorithm. I'm trying to add Murmur as a custom > implementation and load it to my application. > > > > Can you please let me know, whether there is a reason for not > including Murmur in client-side even though its there in the > Server side. > > I have read that Murmur3 is a better algorithm to use for non-crypto > hashing but has it proved otherwise for this "node > selection" > > scenario. Also please correct me if my understanding is not correct. > > > > [1]. > https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/ParameterGroups.Memcached.html > > > > Thanks, > > Eranda > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Memcached client with murmur3
You don't need to worry about the server one at all. They don't need to match up either; on the server side it's just for the hash table. The default is fine. I only left jenkins in as a "just in case", too. I don't think it matters that much for clients either. There're very few buckets involved for clients (just the number of servers) so it'd have to be a very poor algorithm to display a significant bias. Most original clients use crc32 and they did fine. On Wed, 20 Mar 2019, Eranda Rajapakshe wrote: > Hi, > > I'm quite an amateur on Memcached, please excuse me if my understanding is > not correct. I'm trying to connect to AWS ElastiCache > Memcached servers from my Spring service. As per my understanding cache key > hashing is done at two locations.1. client side to decide > the Memcache node > 2. at Memcache node to get the value corresponding to the key > > For hashing in Memcached server, I found that two possible algorithms are > Jenkins and Murmur. [1] For client-side hashing, I see several > default algorithms provided in net.spy.memcached.DefaultHashAlgorithm class. > But I cannot see Murmur algorithm in the Java client > library as a default algorithm. I'm trying to add Murmur as a custom > implementation and load it to my application. > > Can you please let me know, whether there is a reason for not including > Murmur in client-side even though its there in the Server side. > I have read that Murmur3 is a better algorithm to use for non-crypto hashing > but has it proved otherwise for this "node selection" > scenario. Also please correct me if my understanding is not correct. > > [1]. > https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/ParameterGroups.Memcached.html > > Thanks, > Eranda > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: memcached + SASL: Password verification failed
t/binary-sasl.t under memcached/memcached should show you examples of how
to authenticate. You should be able to just hack up the test to get more
information about what the password files look like/etc. it writes it out
to tmp.
seems some systems require the @hostname and some don't (mine doesn't, I
haven't looked into why)
On Tue, 19 Mar 2019, Jiuming Shao wrote:
> Thanks for getting back to me! I referred to that because memcached/memcached
> does not tell me how the binary protocol packets for SASL
> AUTH looks like. For all the server configuration and db setup, I followed
> https://github.com/memcached/memcached/wiki/SASLHowto
> and https://github.com/memcached/memcached/wiki/SASLAuthProtocol
> Please let me know when you have time to take a closer look.
>
> Cheers!
> Jiuming
>
> dormando 于2019年3月18日周一 下午4:10写道：
> Hey,
>
> Can look more closely later, but a few quick things that might help:
>
> 1) stick to memcached/memcached on github - that's an old couchbase fork
> you linked to. If you're using couchbase you need to talk to them
> instead.
>
> 2) in the t/ dir there're some unit tests for SASL which might help you
> understand the workflow better.
>
> On Mon, 18 Mar 2019, Jiuming Shao wrote:
>
> > Hey all,
> > I am writing my own implementation of a memcachedClient within which
> I want to add authentication. I just started with
> PLAIN auth but
> > failed.
> >
> > My main reference is this one
> https://github.com/couchbase/memcached/blob/master/docs/sasl.md
> > My guess is that the binary message I am sending through the wire was
> wrong, thus it could never match with the secret I
> store in the db
> > file.
> > After searching around, I found out the SASL_AUTH(0X21) is also a
> key-value like operation, where the key is the auth
> mechanism, and the
> > value being auth data. The tricky part is how I put them in the
> outgoing request.
> >
> > Please correct me if i am wrong, below is an example of PLAIN auth
> request
> > 1. The auth mechanism comes right after the header. in this case
> 'PLAIN'
> > 2. A NULL byte comes after the "key" -> "PLAIN". In this case byte #
> 29.
> > 3. Then comes the user@hostName
> > 4. A NULL bytes comes after user@hostname. In this case byte # 34
> > 5. The last part is the password
> >
> > Byte/ 0 | 1 | 2 | 3 |
> > / | | | |
> > |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
> > +---+---+---+---+
> > 0| 0x80 | 0x21 | 0x00 | 0x05 |
> > +---+---+---+---+
> > 4| 0x00 | 0x00 | 0x00 | 0x00 |
> > +---+---+---+---+
> > 8| 0x00 | 0x00 | 0x00 | 0x11 |
> > +---+---+---+---+
> > 12| 0x00 | 0x00 | 0x00 | 0x00 |
> > +---+---+---+---+
> > 16| 0x00 | 0x00 | 0x00 | 0x00 |
> > +---+---+---+---+
> > 20| 0x00 | 0x00 | 0x00 | 0x00 |
> > +---+---+---+---+
> > 24| 0x50 ('P') | 0x4c ('L') | 0x41 ('A') | 0x49 ('I') |
> > +---+---+---+---+
> > 28| 0x4e ('N') | 0x00 | 0x75 ('u') | 0x73 ('s') |
> > +---+---+---+---+
> > 32| 0x65 ('e') | 0x72 ('r') | 0x00 | 0x70 ('p') |
> > +---+---+---+---+
> > 36| 0x65 ('e') | 0x6e ('n') | 0x63 ('c') | 0x69 ('i') |
> > +---+---+---+---+
> > 40| 0x6c ('l') |
> > +---+
> > Total 41 bytes (24 bytes header, 5 bytes key and 12 value)
> >
> > Field (offset) (value)
> > Magic
Re: memcached + SASL: Password verification failed
Hey,
Can look more closely later, but a few quick things that might help:
1) stick to memcached/memcached on github - that's an old couchbase fork
you linked to. If you're using couchbase you need to talk to them instead.
2) in the t/ dir there're some unit tests for SASL which might help you
understand the workflow better.
On Mon, 18 Mar 2019, Jiuming Shao wrote:
> Hey all,
> I am writing my own implementation of a memcachedClient within which I want
> to add authentication. I just started with PLAIN auth but
> failed.
>
> My main reference is this one
> https://github.com/couchbase/memcached/blob/master/docs/sasl.md
> My guess is that the binary message I am sending through the wire was wrong,
> thus it could never match with the secret I store in the db
> file.
> After searching around, I found out the SASL_AUTH(0X21) is also a key-value
> like operation, where the key is the auth mechanism, and the
> value being auth data. The tricky part is how I put them in the outgoing
> request.
>
> Please correct me if i am wrong, below is an example of PLAIN auth request
> 1. The auth mechanism comes right after the header. in this case 'PLAIN'
> 2. A NULL byte comes after the "key" -> "PLAIN". In this case byte # 29.
> 3. Then comes the user@hostName
> 4. A NULL bytes comes after user@hostname. In this case byte # 34
> 5. The last part is the password
>
> Byte/ 0 | 1 | 2 | 3 |
> / | | | |
> |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
> +---+---+---+---+
>0| 0x80 | 0x21 | 0x00 | 0x05 |
> +---+---+---+---+
>4| 0x00 | 0x00 | 0x00 | 0x00 |
> +---+---+---+---+
>8| 0x00 | 0x00 | 0x00 | 0x11 |
> +---+---+---+---+
> 12| 0x00 | 0x00 | 0x00 | 0x00 |
> +---+---+---+---+
> 16| 0x00 | 0x00 | 0x00 | 0x00 |
> +---+---+---+---+
> 20| 0x00 | 0x00 | 0x00 | 0x00 |
> +---+---+---+---+
> 24| 0x50 ('P')| 0x4c ('L')| 0x41 ('A')| 0x49 ('I')|
> +---+---+---+---+
> 28| 0x4e ('N')| 0x00 | 0x75 ('u')| 0x73 ('s')|
> +---+---+---+---+
> 32| 0x65 ('e')| 0x72 ('r')| 0x00 | 0x70 ('p')|
> +---+---+---+---+
> 36| 0x65 ('e')| 0x6e ('n')| 0x63 ('c')| 0x69 ('i')|
> +---+---+---+---+
> 40| 0x6c ('l')|
> +---+
> Total 41 bytes (24 bytes header, 5 bytes key and 12 value)
>
> Field(offset) (value)
> Magic(0): 0x80
> Opcode (1): 0x21
> Key length (2,3) : 0x0005
> Extra length (4): 0x00
> Data type(5): 0x00
> Vbucket (6,7) : 0x
> Total body (8-11) : 0x0011
> Opaque (12-15): 0x
> CAS (16-23): 0x
>
> What could be wrong?
> * In my memcached-sasl-db, should I store userName:password or
> username@hostName:password?
> * Does the TotalLength of the message include the NULL bytes being added
> between authKey/authData and username/password?
> * In my authData should I use \0x00userName\0x00password or
> \0x00userName@hostNname\0x00password?
> * Any other suggestions?
>
> Best regards,
> Jiuming
>
> Below are Logs and configurations for your references
>
> memcached logs: Below you will find that I did a LIST_MECH(0x20) and then did
> a SASL_AUTH(0X21)
>
> LRU crawler thread sleeping
> <28 new binary client connection.
> 28: going from conn_new_cmd to conn_waiting
> 28: going from conn_waiting to conn_read
> 28: going from conn_read to conn_closing
> <28 connection closed.
> 28: going from conn_closing to conn_closed
> <28 new binary client connection.
> 28: going from conn_new_cmd to conn_waiting
> 28: going from conn_waiting to conn_read
> 28: going from conn_read to conn_parse_cmd
> <28 Read binary protocol data:
> <280x80 0x20 0x00 0x00
> <280x00 0x00 0x00 0x00
> <280x00 0x00 0x00 0x00
> <280x00 0x00 0x00 0x01
> <280x00 0x00 0x00 0x00
> <280x00 0x00 0x00 0x00
> authenticated() in cmd 0x20 is true
> >28 Writing bin response:
> >28 0x81 0x20 0x00 0x00
> >28 0x00 0x00 0x00 0x00
> >28 0x00 0x00 0x00 0x15
> >28 0x00 0x00 0x00 0x01
> >28 0x00 0x00 0x00 0x00
> >28 0x00 0x00 0x00 0x00
>
Re: Alternate hashing algorithm for higher performance?
Sounds good! fwiw binprot simply lacks a method for batching responses. I'll be fixing it this year but that's going to be a big refactor of the old frontend code. There's some use to it but there should be an option for batching anyway. VM's fine. I do most of my quick tests on a NUC, which is essentially a laptop with better cooling. Have some donated hardware from packet.net for bigger tests, thankfully :) mc-crusher is nice since it uses very little CPU compared to something like memtier. A lot of "memcached" benchmarks end up benchmarking the benchmark, and are only useful in measuring relative latency of changes in the daemon vs scalability/throughput. Just uploaded some fixes and an updated README file. pretty sure I'm the only regular user of the thing so please ask away if anything is weird/confusing. have fun, -Dormando On Sun, 17 Mar 2019, 'Eamonn Nugent' via memcached wrote: > Hi, > > I've been meaning to get myself dedicated test environments at one point, for > my own stuff as well. Guess now is as good a time as any > to try it out. > I'll start using mc-crusher for benchmarks, and I'm going to throw it on at > least a VM for now, run some long-term benchmarks, and > likely try out a couple algos to figure out which one runs best. Good to know > about ascii being a different workload, I wasn't sure if > they were essentially aliased or some sort of individually unique protocol. > Maybe if I get annoyed enough at syscalls, I'll find a way > of writing an OS that allows user applications to be run safely without the > same overhead. Tempting, tempting... > > No worries about it being a pain in the ass - this is the process behind good > and reliable software, and I personally wouldn't want it > any other way. I just wanted to get back with early results to get your > feedback. > > Thanks, > > Eamonn > > On Sun, Mar 17, 2019 at 4:16 PM dormando wrote: > Yo, > > Fwiw, I use mc-crusher as the "official" benchmark: > https://github.com/memcached/mc-crusher tho I need to update the README > slightly. will do that in a bit. > > I also test on hardware uninterrupted by other things with turbo > disabled > :) testing on laptops can be really tough since you'll get different > frequencies minute to minute. You have to interleave test runs > A/B/A/B/A/B > then average to get through the noise. > > Also, make sure to test both binprot/ascii. with ascii multiget you can > deeply pipeline requests and get more hashing vs syscalls. > > Also, test short vs long keys. a for loop and some scripting should get > you there. :) > > I don't really want to add a ton of algo's. More or less a "best > average" > is fine. Given the syscall overhead, hashing is barely measurable for a > lot of workloads. When I switched from jenkins to murmur I did a > somewhat comprehensive set of tests then swapped default + left old one > in just in case. I highly value good defaults! Libmemcached ended up > kitchen-sinking hash algo's and I think that didn't work out well in the > long run. > > I did also test hash chain bucket depth a bit. Finally, loading up > different counts of keys (1m, 10m, 100m, etc) and re-running uniform > random benchmarks since fairness will affect the bucket depth and thus > latency. > > Sorry if that's a pain in the ass, but this thing is quite widely used > and > there aren't really beta testers :) Have to be thorough these days. > > -Dormando > > On Sun, 17 Mar 2019, 'Eamonn Nugent' via memcached wrote: > > > Reporting back with very preliminary benchmarks. Somehow, xxh64 is > actually faster than xxh3 on my machine. One thing I > forgot to > > mention before - I also looked at latencies with xxh32/64, and saw > the 99th percentile latency lowered by about half > compared to mmh3. > > So it could be beneficial in that sense. Latencies with xxh3 are in > the 3.6ms 99% range, xxh64 go down to about 3.0 (I saw > 2.5 > > yesterday, maybe testing on a laptop with about a billion chrome tabs > open isn't a brilliant idea), and mmh3 were in the > 4.xms range. > > All of this with modern options, but with non-modern, xxh64 shone > quite a bit. I was doing my testing there yesterday. > > I used the following memtier_benchmark command to stress test: > > > > ./memtier_benchmark -P memcache_binary -p 11211 --key-prefix="" > --key-maximum=999 > > > > A lot of this seems to be very architecture dependent. Mayb
Re: Alternate hashing algorithm for higher performance?
Yo, Fwiw, I use mc-crusher as the "official" benchmark: https://github.com/memcached/mc-crusher tho I need to update the README slightly. will do that in a bit. I also test on hardware uninterrupted by other things with turbo disabled :) testing on laptops can be really tough since you'll get different frequencies minute to minute. You have to interleave test runs A/B/A/B/A/B then average to get through the noise. Also, make sure to test both binprot/ascii. with ascii multiget you can deeply pipeline requests and get more hashing vs syscalls. Also, test short vs long keys. a for loop and some scripting should get you there. :) I don't really want to add a ton of algo's. More or less a "best average" is fine. Given the syscall overhead, hashing is barely measurable for a lot of workloads. When I switched from jenkins to murmur I did a somewhat comprehensive set of tests then swapped default + left old one in just in case. I highly value good defaults! Libmemcached ended up kitchen-sinking hash algo's and I think that didn't work out well in the long run. I did also test hash chain bucket depth a bit. Finally, loading up different counts of keys (1m, 10m, 100m, etc) and re-running uniform random benchmarks since fairness will affect the bucket depth and thus latency. Sorry if that's a pain in the ass, but this thing is quite widely used and there aren't really beta testers :) Have to be thorough these days. -Dormando On Sun, 17 Mar 2019, 'Eamonn Nugent' via memcached wrote: > Reporting back with very preliminary benchmarks. Somehow, xxh64 is actually > faster than xxh3 on my machine. One thing I forgot to > mention before - I also looked at latencies with xxh32/64, and saw the 99th > percentile latency lowered by about half compared to mmh3. > So it could be beneficial in that sense. Latencies with xxh3 are in the 3.6ms > 99% range, xxh64 go down to about 3.0 (I saw 2.5 > yesterday, maybe testing on a laptop with about a billion chrome tabs open > isn't a brilliant idea), and mmh3 were in the 4.xms range. > All of this with modern options, but with non-modern, xxh64 shone quite a > bit. I was doing my testing there yesterday. > I used the following memtier_benchmark command to stress test: > > ./memtier_benchmark -P memcache_binary -p 11211 --key-prefix="" > --key-maximum=999 > > A lot of this seems to be very architecture dependent. Maybe it would make > sense to include a lot of hash algos long term, and let power > users figure out which they feel like using? Not sure, though, and you're the > expert here :P > > Thanks, > > Eamonn > > On Sun, Mar 17, 2019 at 2:59 PM Eamonn Nugent > wrote: > Hiya, > Last night, I was running memtier_benchmark on my laptop (mid-2015 15" MBP, > 2.5GHz 4c i7) and achieved about a 10-15% throughput > improvement on both modern and non-modern settings on the 64 bit variant. 32 > bit variant was about equal in performance (the > results showed them to be within about 3% of each other, but most of the > difference was probably just typical entropy). I was able > to solve the 32/64 bit compile time problem by adding in a wrapper and some > compile-time declarations, so I'd say that's about 50% > solved for x86-based systems. But yeah, with ARM, it could turn interesting. > > As a next-ish step, I'm going to attempt to drop in xxh3, but since it's > still in active development, it's probably not good as > anything more than a tech demo. I'm happy, if it would help, just to go nuts > adding a dozen different algos into hash.c, though > (cityhash/farmhash, as you mentioned). In xxhash's implementation, though, I > played with some compile-time flags to make it a bit > faster, and I've been toying with the idea of modifying it so no seed logic > ever occurs, to maybe gain a couple cycles of speed > increase. I'm also looking into seeing if I can find a pure assembly version > to squeeze a bit more speed out of x86 and ARM > versions. I should probably get one of my ARM systems running and test the > difference... > > But hey, thanks for humoring me. Maybe next I'll take a look at the reading & > processing command steps, and see if there's > anything I can do. Or maybe parallelizing rotl... Hm. I'll keep on with > trying it out :) > > Thanks, > > Eamonn > > > On Sun, Mar 17, 2019 at 2:46 PM dormando wrote: > Hey, > > What exact test did you do? > > Well to be honest I've been wanting to swap in xxhash for a long time, > but > in my own profiling other things show up higher than murmur so I keep > deprioritizing it :) > > One big problem with the hash algo is mc keys can be short and are > hashed one at a time. xxhash is more optimized fo
Re: Alternate hashing algorithm for higher performance?
Hey, What exact test did you do? Well to be honest I've been wanting to swap in xxhash for a long time, but in my own profiling other things show up higher than murmur so I keep deprioritizing it :) One big problem with the hash algo is mc keys can be short and are hashed one at a time. xxhash is more optimized for longer data (kilobytes to megabytes). The original author tries to address this with an updated algorithm: https://fastcompression.blogspot.com/2019/03/presenting-xxh3.html xxhash makes significant use of instruction parallelism, such that if a key is 8 bytes or less you could end up waiting for the pipeline more than murmur. Other algos like cityhash/farmhash are better at short keys IIRC. Also xx's 32bit algo is a bit slower on 64bit machines... so if I wanted to use it I was going to test both 32bit and 64bit hashes and then have to do compile time testing to figure out which to use. It's also heavily x86 optimized so we might have to default something else for ARM. Sorry, not debated on the list, just in my own head :) It's not quite as straightforward as just dropping it in. If you're willing to get all the conditions tested go nuts! :) -Dormando On Sat, 16 Mar 2019, eamonn.nugent via memcached wrote: > Hi there, > I started using memcached in prod a week or two ago, and am loving it. I > wanted to give back, and took a look through the issues board, > but most of them looked solved. So, in my usual "it's never fast enough" > style, I went and profiled its performance, and had some fun. > > After seeing that MurmurHash3 was taking a good amount of the execution time, > I decided to run a test integrating one of my old favorite > hash functions, xxhash. My guess is that Memcached could benefit from using > the hash function, as it is faster than MMH3 and has several > native variants. I ran some of my own tests, and found roughly equal > performance, but with no tuning performed on xxhash. For example, > using an assembly (x86/arm/etc) version could likely speed up hashing, along > with properly implementing it in memcached. However, I was > also running this on a much older Nehalem CPU, so there could be unseen > advantages to one or both of the algorithms by running them on a > newer CPU. I'm in the process of fighting with my newer systems to get > libevent installed properly, so I'll report back with more > up-to-date tests later. > > I did a cursory search, but didn't find any mention of the algo in the > mailing list. If this has been discussed, though, apologies for > bringing it up again. On the other hand, I would be happy to write a PR to > add it, using the `hash_algorithm` CLI arg. > > Thanks, > Eamonn > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
updates
Yo, Apologies, guess I stopped sending updates to this list. It's not actually clear to me how many people still read it. :) Releases quieted down at the end of last year but will start up again shortly. Still aiming at monthly cuts with at least small updates. I started a blog on the main site: https://memcached.org/blog - gets semi-regular updates about memcached internals, test benchmarks, and so on. I'll link new posts to the ML going forward as well. have fun, -Dormando -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Configuring memcached in VM and load test from host
You can use ssh port forwarding (-R/-L/etc), but that's going to be pretty slow. you'll find a better answer by looking up ways to network qemu guests with the host. there're better guides and that's outside the scope of this mailing list, I think. You should be able to just forward the memcached port to the host, similar to how ssh is being forwarded. On Wed, 7 Nov 2018, Probir Roy wrote: > I am trying to benchmark memcached running in a hosted virtual machine > (Qemu). For this I want to run memcslap from the host machine. > > The memcached running in VM is configured as following: > > > /etc/init.d/memcached status > > > ● memcached.service - memcached daemon > > Loaded: loaded (/lib/systemd/system/memcached.service; enabled; vendor > preset: enabled) > > Active: active (running) since Tue 2018-11-06 11:14:06 CST; 6s ago > > Main PID: 2434 (memcached) > > Tasks: 6 > > Memory: 3.7M > > CPU: 18ms > > CGroup: /system.slice/memcached.service > > └─2434 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1 > > > > > I can run memcslap from VM using following command: > memcslap --servers=127.0.0.1:11211 --concurrency=100 --execute-number=1 > --initial-load=1 --flush > > > But I would like to run the memcslap from host machine. The host can ssh to > VM through: > > ssh -X -p user@192.168.122.1 > > > My question is how can I run memcslap from the host to the VM memcache. > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Disabling compression when reading large values >1MB
telnet has no concept of compression. if you're intending to modify the server code, you should study it a bit more. Sorry :( Since clients have compression built in, you shouldn't have to do this. Compressing/decompressing in the server won't be easy to accomplish and it will take a lot more bandwidth to ship uncompressed data to clients. On Sun, 4 Nov 2018, Pradeep Sivakumar wrote: > Thanks for the quick response. I have unlined my answers below: > > > On Nov 4, 2018, at 11:50 AM, dormando wrote: > > > > Not entirely clear on what you're doing. Are you modifying memcached > > itself to do internal compression? > > > Correct. I am doing internal compression inside memcached. > > > > Some clients autocompress after threshholds. there's no internal > > compression. > > > Which tells me that the compression happens in the telnet client or > pymemcache. Is there a way to disable compression in telnet? > > > >> On Sun, 4 Nov 2018, Pradeep Sivakumar wrote: > >> > >> I am writing an internal compression routine that will compress large > >> values with a Huffman table in memcached [version 1.5.10]. However I am > >> finding that > >> after reading large values and printing them inside memcached, I am > >> getting garbled output. For example, I am reading one large value csv file > >> in csv > >> format where the value length is 1981182 [~2MB]. When I print the value > >> inside the assoc_insert function I am getting output such as ▒L$ or ▒▒ . > >> It only > >> happens for large values and I have tried a telnet client as well as > >> pymemcache [https://pymemcache.readthedocs.io/en/latest/index.html]. Are > >> the large > >> values getting compressed in memcached [or the client for that matter]? If > >> so, is there a way to disable this compression? > >> > >> -- > >> > >> --- > >> You received this message because you are subscribed to the Google Groups > >> "memcached" group. > >> To unsubscribe from this group and stop receiving emails from it, send an > >> email to memcached+unsubscr...@googlegroups.com. > >> For more options, visit https://groups.google.com/d/optout. > >> > >> > > > > -- > > > > --- > > You received this message because you are subscribed to the Google Groups > > "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to memcached+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Disabling compression when reading large values >1MB
Not entirely clear on what you're doing. Are you modifying memcached itself to do internal compression? Some clients autocompress after threshholds. there's no internal compression. On Sun, 4 Nov 2018, Pradeep Sivakumar wrote: > I am writing an internal compression routine that will compress large values > with a Huffman table in memcached [version 1.5.10]. However I am finding that > after reading large values and printing them inside memcached, I am getting > garbled output. For example, I am reading one large value csv file in csv > format where the value length is 1981182 [~2MB]. When I print the value > inside the assoc_insert function I am getting output such as ▒L$ or ▒▒ . It > only > happens for large values and I have tried a telnet client as well as > pymemcache [https://pymemcache.readthedocs.io/en/latest/index.html]. Are the > large > values getting compressed in memcached [or the client for that matter]? If > so, is there a way to disable this compression? > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: memcached connection reset by peer
Check the stats on the daemon to see if it's rejected any client connections? That can happen if you're out of local ports, if the server's hit the connection limit, etc. On Fri, 2 Nov 2018, capoluc01 wrote: > I sometimes get 0 Ops/sec in memtiers json output if run against memcached. > Here is the setup: > I have 3 client VMs on Azure, each running a single instance of memtier > version 1.2.14. Then I have a single server VM on Azure running an instance of > memcached version 1.5.6. > I start each memtier instance with the following options: > --server= --port= > --protocol=memcache_text --ratio=1:0 --expiry-range=-1 > --key-maximum=1 > --hide-histogram --clients=56 --threads=2 --test-time=75 --data-size=4096 > --json-out-file=client.json &> mt1.txt > > memcached is started like this: > memcached -p -t 1 -v &> mc1.log > > It happens that often (not always) at least one memtier client ouputs a json > with 0 Ops/sec. In this case, the corresponding mt1.txt (where stdout & > stderr are redirected to) shows the error message "read error: connection > reset by peer". I have no idea why this happens and would appreciate your > help. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: event_queue_remove not on queue 8
Version 1.2.8 is too old and is not supported, sorry! Newer versions do not have that problem. On Mon, 15 Oct 2018, huan ling wrote: > Does the event_queue_remove problem resolved？ > My SUSE linux server with libevent 1.4 and memcached 1.2.8 have this problem > too > > 在 2010年2月23日星期二 UTC+8上午2:16:58，Beier写道： > My memcached server has been running smoothly in the past few months, > but last Friday and this morning, during peak traffic time memcached > server went down without much error message. the only message logged > is: > > [err] event_queue_remove: 0x62fad8(fd 27) not on queue 8 > > my guess is something wrong with libevent? but no idea what made this > happening. any help will be really appreciated > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Thousands of CLOSE_WAIT connections
inline responses. also, I dunno if I missed it but what version were you on originally? are the start arguments the same? On Sat, 13 Oct 2018, Jim Jones wrote: > The "-n" and "ext_item_size" options were picked to help handle the large > volume of small key values our service generates. But if increasing either > will prevent potential deadlocks, I'm sure we'd prefer to accept the > efficiency hit. Those values don't necessarily translate to doing what you want; I'll illustrate that once I get a gander at your stats. > The two servers that stopped accepting connections were only getting write > operations, that's correct. They are operating as shadow copies of the > cache during this parallel testing period. I do see trivial numbers of > "get" requests, but it's a tiny number compared to the number of "set" and > "delete" commands. Got it. > The code generating the memcache requests is PHP, with "twemproxy" (v0.4.1) > handling all the communications with the memcache servers. I can dig into > the code to get more specific information on the PHP setup if that's > helpful. Ok, so ascii only, then. Just need to know the general commands being used and the protocol, at this point. > I'll relay stats from one of the servers via private e-mail in case those > counters help suggest things we can try. > > And we'll definitely gather the info you described in the event we hit this > problem again. Thanks! Appreciate your patience. > -jj > > On Sat, Oct 13, 2018 at 10:07 AM dormando wrote: > Sounds like the daemon hardlocked. Some of your start arguments > are fairly > aggressive (ext_item_size and -n especially), I'll double check > that those > won't cause problems like this. > > First, to confirm: these two hung machines were only getting > writes the > whole time? no reads? > > Any info you can share about the client? What library, protocol > (ascii or > binary, etc)? > > If you can privately share with me any of your stats snapshots > that would > help a lot as well, since I can better determine which features > are being > exercised. > > Aside from that, there don't seem to be many hints here. If it > happens > again, please do: > > 1) see if you can still connect to it and run commands, grab > stats if so. > 2) grab a GDB backtrace before killing the daemon: > gdb -p $(pidof memcached) > thread apply all bt > ^ full thread backtrace. > > If it's hardlocked the backtrace *should* immediately tell me > what's going > on. > > Sorry for the trouble! > > On Fri, 12 Oct 2018, Jim Jones wrote: > > > The commandline arguments used are: > > > > -u memcached -m 236544 -c 64000 -p 11211 -t 32 -C -n 5 -f 1.05 > -o > > >ext_path=/mnt/memcache:1700G,ext_path=/mnt1/memcache:1700G,ext_path=/mnt2/m > emcache:1700G,ext_path=/mnt3/memcache:1700G,ext_threads=32,ext_item_size=6 > 4 > > > > And we have some data, but frankly when the issue was > happening we focused on the Memcache servers late in the > process. The initial errors suggested the > > problem was in a very different part of the collection of > technical components in our larger service. When we realized > the "memcached" processes were not > > accepting new connections, we wanted to correct the behavior > quickly since a fair amount of time had already passed. > > > > First, sockets in use on the servers... > > > > One system, call it server-248, shows TCP sockets on the > system hovering around 1900 after traffic ramped up for the > day. If held at that level from > > ~6:45am until 10:06am. We collect SAR data every 2 minutes, > so the next reading was at 10:08 and the TCP sockets jumped to > 63842. Meaning it didn't grow > > slowly over time, it jumped frlmo 1937 to 63842 in a 2 minute > window. That number was 63842-63844 until 12:06pm when we > restarted the "memcached" > > process. After that the number dropped over time back to a > more typical level. > > > > 10:02am 1937 > > 10:04am 1937 > > 10:06am 1937 > > 10:08am 63842 > > 10:10am 63842 > > ...etc... > > 12:08pm 63843 > > 12:06pm 63844 > > 12:08pm 18415 > > 12:10pm 17202 > > 12:12pm 1633
Re: Thousands of CLOSE_WAIT connections
Sounds like the daemon hardlocked. Some of your start arguments are fairly aggressive (ext_item_size and -n especially), I'll double check that those won't cause problems like this. First, to confirm: these two hung machines were only getting writes the whole time? no reads? Any info you can share about the client? What library, protocol (ascii or binary, etc)? If you can privately share with me any of your stats snapshots that would help a lot as well, since I can better determine which features are being exercised. Aside from that, there don't seem to be many hints here. If it happens again, please do: 1) see if you can still connect to it and run commands, grab stats if so. 2) grab a GDB backtrace before killing the daemon: gdb -p $(pidof memcached) thread apply all bt ^ full thread backtrace. If it's hardlocked the backtrace *should* immediately tell me what's going on. Sorry for the trouble! On Fri, 12 Oct 2018, Jim Jones wrote: > The commandline arguments used are: > > -u memcached -m 236544 -c 64000 -p 11211 -t 32 -C -n 5 -f 1.05 -o > ext_path=/mnt/memcache:1700G,ext_path=/mnt1/memcache:1700G,ext_path=/mnt2/memcache:1700G,ext_path=/mnt3/memcache:1700G,ext_threads=32,ext_item_size=64 > > And we have some data, but frankly when the issue was happening we focused on > the Memcache servers late in the process. The initial errors suggested the > problem was in a very different part of the collection of technical > components in our larger service. When we realized the "memcached" processes > were not > accepting new connections, we wanted to correct the behavior quickly since a > fair amount of time had already passed. > > First, sockets in use on the servers... > > One system, call it server-248, shows TCP sockets on the system hovering > around 1900 after traffic ramped up for the day. If held at that level from > ~6:45am until 10:06am. We collect SAR data every 2 minutes, so the next > reading was at 10:08 and the TCP sockets jumped to 63842. Meaning it didn't > grow > slowly over time, it jumped frlmo 1937 to 63842 in a 2 minute window. That > number was 63842-63844 until 12:06pm when we restarted the "memcached" > process. After that the number dropped over time back to a more typical > level. > > 10:02am 1937 > 10:04am 1937 > 10:06am 1937 > 10:08am 63842 > 10:10am 63842 > ...etc... > 12:08pm 63843 > 12:06pm 63844 > 12:08pm 18415 > 12:10pm 17202 > 12:12pm 16333 > 12:14pm 16197 > 12:16pm 16134 > 12:18pm 16099 > 12:20pm 1617 > > The other system which ran into trouble, which I'll call server-85, exhibited > similar behavior but started later. Here's a sample of TCP socket counts > from that server. > > 11:30am 1805 > 11:32am 1801 > 11:34am 1830 > 11:36am 1817 > 11:38am 63905 > 11:40am 63905 > ...etc... > 12:20pm 63908 > 12:22pm 63908 > 12:24pm 1708 > 12:26pm 1720 > 12:28pm 1747 > > There were other network-centric datapoints that show the systems grind to a > halt in terms of accepting new connections, like the bandwidth going in/out > of the NIC's, etc... But it's all in support of the same idea, that the > "memcached" server stopped accepting new connections. > > Second, details about the sockets... > > During the incident, we did capture summary information on the socket states > from various servers on the network, and a full "netstat -an" listing from > one of the servers. Both server-248 and server-85 showed tens of thousands > of sockets in a CLOSE_WAIT state, hundreds in a SYN_RCVD state, and a small > number of ESTABLISHED sockets. > > There may continue to be traffic on the ESTABLISHED connections to the > "memcached" servers, but if there is it's a trivial amount. Multiple people > were > running "memkeys" at the time and report seeing no activity. > > Third, "stats" from the incapacitated "memcached" processes... > > We do not have stats from either server-248 or server-85 during the time they > were in trouble. In hindsight, that was a big oversight. > It's not clear that we could have gotten a connection to the server to pull > the stats, but I'd really like to know what those counters said! > > I do have the results from "stats", "stats slabs", "stats items" and "stats > conns" from 17:10 the previous evening. That doesn't show any obvious > errors/problems slowly building up, waiting for some event to trigger a > massive failure. But it's from ~15 hours before the server got into trouble > so I > don't think it's all that helpful. > > -jj > > On Fri, Oct 12, 2018 at 6:24 PM dormando wrote: > Hey, >
Re: Thousands of CLOSE_WAIT connections
Hey, Probably unrelated to the original thread. I'm highly interested in understanding what's going on here though, and would appreciate any help you can spare. What data did you collect? The more information you can share the better: 1) start arguments for the daemon 2) stats output (stats, stats items, stats slabs, and importantly "stats conns" - you will probably have to redact some of the output. 3) were you still able to connect to and run commands on those daemons while they had a mass of CLOSE_WAIT? Thanks, -Dormando On Fri, 12 Oct 2018, jjo...@smugmug.com wrote: > We're testing verison 1.5.10 as a possible upgrade candidate for out older > memcached servers, using a pool 9 servers. They are running in parallel > with the production pool, also 9 servers. For the test all read requests > are going to the production pool, and all updates (set, delete, etc...) are > sent to one server in production pool and one server in the 1.5.10 pool via > the key hashing algorithm. > > That setup had been running without incident for about 12 days then > yesterday two of the servers experienced the mass of CLOSE_WAIT connections > similar to what's been described here. We were able to collect some data, > but not enough to figure out what's happening. So I'm hoping to kickstart a > discussion here about how to diagnose what's going on. Until we can find > way to explain (and prevent) another problem like this, we're unable to > upgrade. > > I can provide more information about our configuration. I'm just not sure > what bits are useful/interesting. I will note that we're using "extstore" > functionality on the new servers. > > -jj > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: memcached using more memory for storage items than I set
Hey, Sadly there isn't much I can do about such an old version, there're too many bugs fixed since then. :/ Why can't you upgrade it? anything we can do to help there? On Fri, 12 Oct 2018, Vlad Lobanov wrote: > Hi, > > I use memcached 1.4.5 and have a question about two stats: bytes and > limit_maxbytes. > memcached using more memory for storing items than I > set(limit_maxbytes Could you please help me understand why it happens? > P.S. It is a really old version and unfortunately I can't update it. > stats > STAT pid 980 > STAT uptime 5125559 > STAT time 1539365261 > STAT version 1.4.5 > STAT pointer_size 64 > STAT rusage_user 2534.966425 > STAT rusage_system 6581.831338 > STAT curr_connections 100 > STAT total_connections 19810 > STAT connection_structures 467 > STAT cmd_get 1445605 > STAT cmd_set 139140 > STAT cmd_flush 1 > STAT get_hits 1278210 > STAT get_misses 167395 > STAT delete_misses 0 > STAT delete_hits 7385 > STAT incr_misses 0 > STAT incr_hits 0 > STAT decr_misses 0 > STAT decr_hits 0 > STAT cas_misses 0 > STAT cas_hits 0 > STAT cas_badval 0 > STAT auth_cmds 0 > STAT auth_errors 0 > STAT bytes_read 139148014 > STAT bytes_written 779491913 > STAT limit_maxbytes 536870912 > STAT accepting_conns 1 > STAT listen_disabled_num 0 > STAT threads 4 > STAT conn_yields 29 > STAT bytes 644661305 > STAT curr_items 1103484 > STAT total_items 139140 > STAT evictions 118130 > STAT reclaimed 0 > END > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Thousands of "memp_trickle thread: writing 0 dirty pages" messages in my memcachedb.log
that is the normal version... however it doesn't make log entries that start with [memcachedb], that's a different program called memcachedb. check your process list? make sure you didn't accidentally install and start memcachedb instead. On Wed, 22 Aug 2018, Werner Naseweis wrote: > I thought this is the Normal > ~ # memcached -V > memcached 1.5.6 > > > Am Mi., 22. Aug. 2018 um 00:19 Uhr schrieb dormando : > can you just use normal memcached with it instead? > > if you want to extend onto disk, there's extstore: > https://memcached.org/blog/extstore-cloud/ > > On Wed, 22 Aug 2018, Werner Naseweis wrote: > > > Only nextcloud. > > > > Am Mi., 22. Aug. 2018 um 00:02 Uhr schrieb dormando > : > > memcachedb isn't supported software AFAIK. what're you using it > for? > > > > On Tue, 21 Aug 2018, Werner Naseweis wrote: > > > > > [memcachedb] [Tue Aug 21 23:38:28 2018] "checkpoint thread: a > txn_checkpoint is done" > > > [memcachedb] [Tue Aug 21 23:38:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:38:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:39:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:39:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:40:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:40:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:41:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:41:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:42:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:42:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:43:28 2018] "checkpoint thread: a > txn_checkpoint is done" > > > [memcachedb] [Tue Aug 21 23:43:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:43:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:44:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:44:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:45:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:45:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:46:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:46:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:47:29 2018] "memp_trickle thread: > writing 0 dirty pages" > > > [memcachedb] [Tue Aug 21 23:47:59 2018] "memp_trickle thread: > writing 0 dirty pages" > > > > > > Does anyone know what to do about it? > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the > Google Groups "memcached" group. > > > To unsubscribe from this group and stop receiving emails from > it, send an email to memcached+unsubscr...@googlegroups.com. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > -- > > > > --- > > You received this message because you are subscribed to the > Google Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from > it, send an email to memcached+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > > > --- >
Re: Thousands of "memp_trickle thread: writing 0 dirty pages" messages in my memcachedb.log
can you just use normal memcached with it instead? if you want to extend onto disk, there's extstore: https://memcached.org/blog/extstore-cloud/ On Wed, 22 Aug 2018, Werner Naseweis wrote: > Only nextcloud. > > Am Mi., 22. Aug. 2018 um 00:02 Uhr schrieb dormando : > memcachedb isn't supported software AFAIK. what're you using it for? > > On Tue, 21 Aug 2018, Werner Naseweis wrote: > > > [memcachedb] [Tue Aug 21 23:38:28 2018] "checkpoint thread: a > txn_checkpoint is done" > > [memcachedb] [Tue Aug 21 23:38:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:38:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:39:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:39:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:40:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:40:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:41:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:41:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:42:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:42:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:43:28 2018] "checkpoint thread: a > txn_checkpoint is done" > > [memcachedb] [Tue Aug 21 23:43:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:43:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:44:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:44:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:45:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:45:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:46:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:46:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:47:29 2018] "memp_trickle thread: writing > 0 dirty pages" > > [memcachedb] [Tue Aug 21 23:47:59 2018] "memp_trickle thread: writing > 0 dirty pages" > > > > Does anyone know what to do about it? > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to memcached+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > --- > You received this message because you are subscribed to the Google > Groups "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Thousands of "memp_trickle thread: writing 0 dirty pages" messages in my memcachedb.log
memcachedb isn't supported software AFAIK. what're you using it for? On Tue, 21 Aug 2018, Werner Naseweis wrote: > [memcachedb] [Tue Aug 21 23:38:28 2018] "checkpoint thread: a txn_checkpoint > is done" > [memcachedb] [Tue Aug 21 23:38:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:38:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:39:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:39:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:40:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:40:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:41:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:41:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:42:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:42:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:43:28 2018] "checkpoint thread: a txn_checkpoint > is done" > [memcachedb] [Tue Aug 21 23:43:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:43:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:44:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:44:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:45:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:45:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:46:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:46:59 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:47:29 2018] "memp_trickle thread: writing 0 dirty > pages" > [memcachedb] [Tue Aug 21 23:47:59 2018] "memp_trickle thread: writing 0 dirty > pages" > > Does anyone know what to do about it? > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Getting hostnames of servers where a particular key is stored when ketama hashing and replication is enabled
You have the list of all servers right? Just query them individually? if it's just for debugging it should take you a few minutes to code soemthing yup; you could even use a bash script. On Thu, 2 Aug 2018, Om Kale wrote: > The reason I am trying to do this is for better debugging. In case something > goes wrong in the system and I am unable to get the value stored for a > particular key, I can go ahead and use this functionality to understand on > which server that particular key is stored. > Also, the second question about why the printing doesn't work. Is there any > other way to check on which server a key is stored. > Additionally, when you say brute force, how would I be able to do it on > memcached server as I will not know on which server instance the key is > stored. > Could you please elaborate on this? > > Thanks and Regards,Om Kale > > On Thu, Aug 2, 2018 at 4:27 PM, dormando wrote: > Why are you trying to do this? That's generally a bad sign, the > replication is for redundancy. if you're just trying to confirm it > works, > you can do a brute force search for the key > > On Thu, 2 Aug 2018, Om Kale wrote: > > > Hi All,I have enabled replication (using --NUMBER_OF_REPLICAS) and > ketama hashing. > > This means, I will have replicas of keys stored on different servers > memntioned in my server list. > > > > My question is, is there any way of retrieving the list of servers > where replicas of a particular key lie, in libmemcached? > > Example: If I give a key, I need all the servers where replicas of > that key are stored. > > > > For getting one server where key is stored, I use this: > > const struct memcached_instance_st *server;server = const > memcached_instance_st * memcached_server_by_key(memcached_st *ptr, const > > char *key, size_t key_length, memcached_return_t *error); > > > > > > However, I am unable to print the hostname from the returned > structure using server->hostname > > > > in a .c file. Can this info not be printed? > > > > Thanks and Regards, > > > > Om Kale > > > > > > > > -- > > > > --- > > You received this message because you are subscribed to the Google Groups > > "memcached" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to memcached+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Getting hostnames of servers where a particular key is stored when ketama hashing and replication is enabled
Why are you trying to do this? That's generally a bad sign, the replication is for redundancy. if you're just trying to confirm it works, you can do a brute force search for the key On Thu, 2 Aug 2018, Om Kale wrote: > Hi All,I have enabled replication (using --NUMBER_OF_REPLICAS) and ketama > hashing. > This means, I will have replicas of keys stored on different servers > memntioned in my server list. > > My question is, is there any way of retrieving the list of servers where > replicas of a particular key lie, in libmemcached? > Example: If I give a key, I need all the servers where replicas of that key > are stored. > > For getting one server where key is stored, I use this: > const struct memcached_instance_st *server;server = const > memcached_instance_st * memcached_server_by_key(memcached_st *ptr, const > char *key, size_t key_length, memcached_return_t *error); > > > However, I am unable to print the hostname from the returned structure > using server->hostname > > in a .c file. Can this info not be printed? > > Thanks and Regards, > > Om Kale > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: server requirements
https://github.com/memcached/memcached/wiki/Hardware On Wed, 1 Aug 2018, server requirements wrote: > What are the server requirements of Memcached? > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
supporting memcached (quick survey)
If you use memcached in production currently, I'd appreciate your taking a couple minutes to answer this survey: https://goo.gl/forms/37zynfK0PbUfTf1I3 thanks! -Dormando -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Same server/nodes for multiple instance of application
Hey, You should talk with the authors of the application, since that depends on how they implemented their cache. ie; if they call flush_all, or can't add a namespace prefix to the cache keys, the instances cannot be shared. On Mon, 9 Jul 2018, Pablo Ortiz wrote: > Hello, > I have an application that can integrate with memcached, and I intend to > install it in multiple servers to have different instances of the app; now my > doubt is if I can use the same memcached servers/nodes for all the app > instances (provided the scaling is right) or if I need to keep them separate. > Would you be able to shed some light on that matter? :) > > Thanks in advance for your time. > > Kind regards, > Pablo, > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
linux seccomp issue
https://github.com/memcached/memcached/issues/402 linking here for a (hopefully?) wider audience. been getting a constant trickle of reports of broken distro builds, so I've made it a priority to fix-for-sure for the 1.5.9 release I'm planning on cutting late tonight or tomorrow. Thanks! -Dormando -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Add .swp/.swo to .gitignore #398
What kind of things do you find interesting? There're a number of open issues right now which could use some work or research; ones marked "help wanted" tend to be on the easier side. There're some odd platform test failures that're likely race conditions in extstore tests. I've fixed a few but getting the last to work has taken some brain power. They'd at least give you an entrypoint to learn more about some of the features. If you don't care I could start assigning ones at random :P As far as my personal process, I try to stamp down open PR's/issue's before I go hunting for more work. Right now there're a few more open than average, though not terribly glorious work the users certainly appreciate timely fixes :) That's a long-winded way of saying I'd probably find more to do once the outstanding issues are cut down a bit. On Wed, 27 Jun 2018, Vadim Pushtaev wrote: > I just want to mess with some decent libevent project. However, now once I've > learned how to build memcached, I have no idea what exactly to do. If you > have any moderately difficult task to assign, I would be glad to be an > assignee. > > -- > Vadim Pushtaev > > Thanks for these! Are you just cleaning up a few things, or are you > working on anything else? Just out of curiosity :) > > On Mon, 18 Jun 2018, Vadim Pushtaev wrote: > > > Hello. > > It's me running the test suite again :). > > `t/whitespace.t` fails for me because I have `.swo` and `.swp` > files created by `vim` in the project root. I believe vim is > pretty common and it can be a > good idea to add them to `.gitignore`. > > Also, we probably don't need to explicitly exempt files in the > test that are mentioned in `.gitignore`. > > https://github.com/memcached/memcached/pull/398 > > -- > Vadim Pushtaev > > > -- > > --- > You received this message because you are subscribed to the > Google Groups "memcached" group. > To unsubscribe from this group and stop receiving emails from > it, send an email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > > --- > You received this message because you are subscribed to the Google > Groups "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Add .swp/.swo to .gitignore #398
Thanks for these! Are you just cleaning up a few things, or are you working on anything else? Just out of curiosity :) On Mon, 18 Jun 2018, Vadim Pushtaev wrote: > Hello. > > It's me running the test suite again :). > > `t/whitespace.t` fails for me because I have `.swo` and `.swp` files created > by `vim` in the project root. I believe vim is pretty common and it can be a > good idea to add them to `.gitignore`. > > Also, we probably don't need to explicitly exempt files in the test that are > mentioned in `.gitignore`. > > https://github.com/memcached/memcached/pull/398 > > -- > Vadim Pushtaev > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
On Mon, 7 May 2018, Om Kale wrote: > Okcouple of follow up questions on the same:1. Inorder to enable/set up > stunnel on memcached server, I need to create certificates using openssl. How > do I execute the openssl certificate generation on memcached server? Also, > after this how could I distribute this to client? There are lots of guides online about how to manage certificates; that is beyond the scope of this mailing list. I will give you a hint though: that you don't need to generate the certificates from any particular place. > 2. Additionally, when you say 'you can modify libmemcached to use OpenSSL > directly', you mean setting up the socket connections in client to support > SSL/TLS, corect? Yes. > > Thanks and Regards,Om Kale > > > On Mon, May 7, 2018 at 1:11 PM, dormando <dorma...@rydia.net> wrote: > hmm. I guess so... > > re: stunnel, as I detailed you still have to get the client > (libmemcached) > to talk over TLS. For the server, no change. > > For the client, you could prototype by having stunnel local to the > client > and connect through that. so you have stunnel talking to stunnel. If > that's not something you can deploy for clients, you can modify > libmemcached to use OpenSSL directly, which should be easier than > modifying the server. > > On Mon, 7 May 2018, Om Kale wrote: > > > The problem with libsasl2 was regarding license. Also, I am unsure if > libsasl2 will give me an ability to perform some sort of certificate > based > > authentication.One more question I had was, would the use of stunnel > need any code change with memached codebase? > > > > Thanks and Regards,Om Kale > > > > > > On Mon, May 7, 2018 at 12:40 PM, dormando <dorma...@rydia.net> wrote: > > Hey, > > > > Just to be clear: I'm completely positive you can make this > work with just > > the libsasl2 that comes with openwrt, you don't need to rebuild > it. the > > problem is you can't use sasl over an untrusted network: SASL > is supposed > > to be used underneath TLS or a trusted network. > > > > Either way, try stunnel. that might just make your life easier > in both > > directions, it's fairly simple. > > > > On Mon, 7 May 2018, Om Kale wrote: > > > > > Hi Dormando and Trond,I think I will first try Dormando's > suggestion of stunnel before delving into changing the memcached code > itself. I > > haven't read > > > much about stunnel, so will need to look into it in some > detail. > > > Again, thanks a lot for the support. It would have been very > good if I could have used sasl (using libsasl2) directly but because > of the > > GPLV3 license > > > requirements that is a problem. > > > I will keep you updated with my progress. > > > > > > > > > Thanks and Regards,Om Kale > > > > > > On Sat, May 5, 2018 at 4:53 PM, dormando <dorma...@rydia.net> > wrote: > > > > On Fri, May 4, 2018 at 10:46 PM dormando > <dorma...@rydia.net> wrote: > > > > > > > > The closest would be SCRAM-SHA-256/512 > mechanism, but the RFC for that states "in combination with TLS" up front, and > I'd be > > wary of > > > using it > > > > over the internet as well. > > > > > > > > > > > > If we ignore TLS for a second and just look at SCRAM > it is fairly easy to implement a minimalistic support for those > mechanisms > > within > > > SASL. There is > > > > however one huge problem by using them in memcached > without doing major refactoring in the SASL support in memcached. By > design > > SCRAM use a > > > hashing > > > > function with an iteration count, which should be set > high enough to burn enough CPU on both the client and the server to > make > > brute force > > > attacks > > > > "impossible" (the RFC states that for SCRAM-SHA1 i
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
hmm. I guess so... re: stunnel, as I detailed you still have to get the client (libmemcached) to talk over TLS. For the server, no change. For the client, you could prototype by having stunnel local to the client and connect through that. so you have stunnel talking to stunnel. If that's not something you can deploy for clients, you can modify libmemcached to use OpenSSL directly, which should be easier than modifying the server. On Mon, 7 May 2018, Om Kale wrote: > The problem with libsasl2 was regarding license. Also, I am unsure if > libsasl2 will give me an ability to perform some sort of certificate based > authentication.One more question I had was, would the use of stunnel need any > code change with memached codebase? > > Thanks and Regards,Om Kale > > > On Mon, May 7, 2018 at 12:40 PM, dormando <dorma...@rydia.net> wrote: > Hey, > > Just to be clear: I'm completely positive you can make this work with > just > the libsasl2 that comes with openwrt, you don't need to rebuild it. the > problem is you can't use sasl over an untrusted network: SASL is > supposed > to be used underneath TLS or a trusted network. > > Either way, try stunnel. that might just make your life easier in both > directions, it's fairly simple. > > On Mon, 7 May 2018, Om Kale wrote: > > > Hi Dormando and Trond,I think I will first try Dormando's suggestion > of stunnel before delving into changing the memcached code itself. I > haven't read > > much about stunnel, so will need to look into it in some detail. > > Again, thanks a lot for the support. It would have been very good if > I could have used sasl (using libsasl2) directly but because of the > GPLV3 license > > requirements that is a problem. > > I will keep you updated with my progress. > > > > > > Thanks and Regards,Om Kale > > > > On Sat, May 5, 2018 at 4:53 PM, dormando <dorma...@rydia.net> wrote: > > > On Fri, May 4, 2018 at 10:46 PM dormando <dorma...@rydia.net> > wrote: > > > > > > The closest would be SCRAM-SHA-256/512 mechanism, but > the RFC for that states "in combination with TLS" up front, and I'd be > wary of > > using it > > > over the internet as well. > > > > > > > > > If we ignore TLS for a second and just look at SCRAM it is > fairly easy to implement a minimalistic support for those mechanisms > within > > SASL. There is > > > however one huge problem by using them in memcached without > doing major refactoring in the SASL support in memcached. By design > SCRAM use a > > hashing > > > function with an iteration count, which should be set high > enough to burn enough CPU on both the client and the server to make > brute force > > attacks > > > "impossible" (the RFC states that for SCRAM-SHA1 it should be > _at least 4096_). Given that the memcached runs the SASL operations > in the > > _front end > > > threads_, it would block all the clients bound to that thread > every time someone tries to authenticate. If there is clients > connecting all > > the time one > > > could end up with all worker threads running PBKDF2 hashing > and all other operations timing out ;) > > > > > > In order to add support for SCRAM you would have to move the > hashing over to a separate thread, and there is not an infrastructure > for such > > thing in the > > > current memcached implementation so it would be a lot of work > ;) > > > > > > > There are actually mechanisms for passing connections to other > threads in > > the code now :) It's used in a few places. It's not incredibly > fast but > > connection rates typically aren't high enough to bother it. > You'd still > > burn out your CPU though... > > > > but, it's moot. if you don't trust your network you can't just > use SASL. > > :/ > > > > > Dormandos suggestion with stunnel (or ipsec) sounds like the > least amount of work, but if you _really_ don't want that (or you for > some > > reason really > > > want to im
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
Hey, Just to be clear: I'm completely positive you can make this work with just the libsasl2 that comes with openwrt, you don't need to rebuild it. the problem is you can't use sasl over an untrusted network: SASL is supposed to be used underneath TLS or a trusted network. Either way, try stunnel. that might just make your life easier in both directions, it's fairly simple. On Mon, 7 May 2018, Om Kale wrote: > Hi Dormando and Trond,I think I will first try Dormando's suggestion of > stunnel before delving into changing the memcached code itself. I haven't read > much about stunnel, so will need to look into it in some detail. > Again, thanks a lot for the support. It would have been very good if I could > have used sasl (using libsasl2) directly but because of the GPLV3 license > requirements that is a problem. > I will keep you updated with my progress. > > > Thanks and Regards,Om Kale > > On Sat, May 5, 2018 at 4:53 PM, dormando <dorma...@rydia.net> wrote: > > On Fri, May 4, 2018 at 10:46 PM dormando <dorma...@rydia.net> wrote: > > > > The closest would be SCRAM-SHA-256/512 mechanism, but the RFC > for that states "in combination with TLS" up front, and I'd be wary of > using it > > over the internet as well. > > > > > > If we ignore TLS for a second and just look at SCRAM it is fairly > easy to implement a minimalistic support for those mechanisms within > SASL. There is > > however one huge problem by using them in memcached without doing > major refactoring in the SASL support in memcached. By design SCRAM use a > hashing > > function with an iteration count, which should be set high enough to > burn enough CPU on both the client and the server to make brute force > attacks > > "impossible" (the RFC states that for SCRAM-SHA1 it should be _at > least 4096_). Given that the memcached runs the SASL operations in the > _front end > > threads_, it would block all the clients bound to that thread every > time someone tries to authenticate. If there is clients connecting all > the time one > > could end up with all worker threads running PBKDF2 hashing and all > other operations timing out ;) > > > > In order to add support for SCRAM you would have to move the hashing > over to a separate thread, and there is not an infrastructure for such > thing in the > > current memcached implementation so it would be a lot of work ;) > > > > There are actually mechanisms for passing connections to other threads > in > the code now :) It's used in a few places. It's not incredibly fast but > connection rates typically aren't high enough to bother it. You'd still > burn out your CPU though... > > but, it's moot. if you don't trust your network you can't just use SASL. > :/ > > > Dormandos suggestion with stunnel (or ipsec) sounds like the least > amount of work, but if you _really_ don't want that (or you for some > reason really > > want to implement something yourself) you could look into changing > memcached to use libevents bufferevents instead of the "basic" form it > use today, and > > then add support for using the SSL level on top of bufferevents. I > haven't tested this so I have no idea of the overhead of this and how it > would affect > > the overall performance. Unless all your clients want to use SSL you > probably want a dedicated port and thread pool serving these > connections. It all > > depends on the performance requirements you've got... > > I'm more concerned about the poor person ending up stuck with a fork > after > weeks of work.. it's not exactly a straightforward change. I do intend > to > add TLS support this year. Would help if someone sponsored the work > though > :P > > -- > > --- > You received this message because you are subscribed to the Google > Groups "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Benchmark projects for memcached
Hey, Sorry, missed this earlier. https://github.com/memcached/mc-crusher is the benchmark utility I use to validate releases. There's a complete README but it might be a little confusing to get going. There're a number of examples in the conf/ directory. Let me know if any concept is confusing and I can explain further. With it I can easily saturate memcached on a 48+ core machine, pulling 50m keys/sec. I use an external sampler to measure latency during the tests. On Wed, 2 May 2018, Arie Aharon wrote: > Hi there, > I am trying to do some CPU and memory utilization analysis for memcached. > Is there any good benchmark to stress memcached with a pre defined memory > limit and a number of threads? > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
> On Fri, May 4, 2018 at 10:46 PM dormando <dorma...@rydia.net> wrote: > > The closest would be SCRAM-SHA-256/512 mechanism, but the RFC for that > states "in combination with TLS" up front, and I'd be wary of using it > over the internet as well. > > > If we ignore TLS for a second and just look at SCRAM it is fairly easy to > implement a minimalistic support for those mechanisms within SASL. There is > however one huge problem by using them in memcached without doing major > refactoring in the SASL support in memcached. By design SCRAM use a hashing > function with an iteration count, which should be set high enough to burn > enough CPU on both the client and the server to make brute force attacks > "impossible" (the RFC states that for SCRAM-SHA1 it should be _at least > 4096_). Given that the memcached runs the SASL operations in the _front end > threads_, it would block all the clients bound to that thread every time > someone tries to authenticate. If there is clients connecting all the time one > could end up with all worker threads running PBKDF2 hashing and all other > operations timing out ;) > > In order to add support for SCRAM you would have to move the hashing over to > a separate thread, and there is not an infrastructure for such thing in the > current memcached implementation so it would be a lot of work ;) > There are actually mechanisms for passing connections to other threads in the code now :) It's used in a few places. It's not incredibly fast but connection rates typically aren't high enough to bother it. You'd still burn out your CPU though... but, it's moot. if you don't trust your network you can't just use SASL. :/ > Dormandos suggestion with stunnel (or ipsec) sounds like the least amount of > work, but if you _really_ don't want that (or you for some reason really > want to implement something yourself) you could look into changing memcached > to use libevents bufferevents instead of the "basic" form it use today, and > then add support for using the SSL level on top of bufferevents. I haven't > tested this so I have no idea of the overhead of this and how it would affect > the overall performance. Unless all your clients want to use SSL you probably > want a dedicated port and thread pool serving these connections. It all > depends on the performance requirements you've got... I'm more concerned about the poor person ending up stuck with a fork after weeks of work.. it's not exactly a straightforward change. I do intend to add TLS support this year. Would help if someone sponsored the work though :P -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
Actually I take this back... SASL is not usable over the internet in any form. It didn't click in my head that you were going over the internet for some reason. The closest would be SCRAM-SHA-256/512 mechanism, but the RFC for that states "in combination with TLS" up front, and I'd be wary of using it over the internet as well. Have you considered just dropping stunnel in front of things? It trivially supports using a database for client certificate authentication, then you could just disable SASL or keep it on PLAIN which would be alright given the situation. This'll depend on the client a bit. You didn't describe what the clients are which are connecting to these servers running on openwrt? Please fill us in at least. You have two options: 1) Start an stunnel locally and have the client connect through that to the remote server. This is easiest/best if you can do it. 2) Use stunnel to authenticate clients and unwrap TLS on the server, and add trivial support for blocking SSL to libmemcached. This could possibly be simpler (or at least half the work) than your original plan. Getting this all going in the server is pretty painful, but on the client end you only have to deal with a single certificate and you can block while connecting/running commands. On Fri, 4 May 2018, dormando wrote: > Hey, > > On Fri, 4 May 2018, Om Kale wrote: > > > Hey Dormando and Trond,Thanks a lot for all of your inputs. > > Let me give you guys a quick summary of what I am planning to do and the > > issues I am facing: > > I need memcached server with encryption and authentication support on > > wireless devices. (Encyption, there is a way using set encoding key with > > libmemcached, I need to write a decoding key function. (this looks > > feasible)) > > Basically, when a libmemcached client connects, the memcached server only > > allows the authenticated users/clients and then once this is done, the > > communication is encrypted) > > Hence I needed a openwrt package for SASL that can directly be downloaded > > from the website and used without modification - therefore the decision to > > use > > Cyrus-SASL) > > Here are the problems: > > 1. Cyrus-SASL > > Cyrus-SASL (version 2.1.27-rc7) has two files with GPLv3 license, hence I > > cannot use it. Also, the openwrt Cyrus-SASL has only the libsasl2 library, > > not > > some of the other libraries mentioned that are needed for SASL support as > > per the Couchbase link. > > https://blog.couchbase.com/sasl-memcached-now-available/) > > Have you tried to just use libsasl2 and run it? I'm pretty sure the daemon > doesn't need more than that after it's been compiled. The other libraries > linked in the blog are the saslpasswd/etc stuff that you don't need. > > Just installed libsasl2 on my openwrt/lede router and it has digest/cram > modules: > # opkg files libsasl2 > Package libsasl2 (2.1.26-3) is installed on root and has the following > files: > /usr/lib/sasl2/libdigestmd5.so.3 > /usr/lib/libsasl2.so.3.0.0 > /usr/lib/libsasl2.so.3 > /usr/lib/sasl2/libscram.so > /usr/lib/sasl2/libdigestmd5.so > /usr/lib/sasl2/libplain.so.3 > /usr/lib/sasl2/libanonymous.so.3.0.0 > /usr/lib/sasl2/libscram.so.3.0.0 > /usr/lib/sasl2/libdigestmd5.so.3.0.0 > /usr/lib/sasl2/libanonymous.so.3 > /usr/lib/sasl2/libanonymous.so > /usr/lib/libsasl2.so > /usr/lib/sasl2/libplain.so.3.0.0 > /usr/lib/sasl2/libplain.so > /usr/lib/sasl2/libcrammd5.so.3.0.0 > /usr/lib/sasl2/libcrammd5.so.3 > /usr/lib/sasl2/libcrammd5.so > /usr/lib/sasl2/libscram.so.3 > > So if you build the passwd db and ship it over, or use the spec to write a > small tool to generate the file properly, that seems like your least > effort route. > > > FYI, I see the client here apparently appends the @localhostname with the > > username. So this same format needs to be stored in the passwd file. > > I'd recommend code diving to understand this. > > > 2. I cannot use 'PLAIN' as I would see the username/passwords on packet > > sniffs (wireshark or any other tool) Additionally, if the password file is > > accessed it would be a problem. > > ok > > > 3. I also see lot of blogs involving DDOS on memcached with UDP mode. (I am > > not sure if there is any future release where this is going to be planned) > > not relevant? Just disable UDP mode. it's not necessary and off by default > in latest versions. > > > 4. One of the options that I could do is change the memcached/libmemcached > > code itself to turn the sockets into SSL/TLS sockets. This would add some > > overheard and few extra message exchanges but could be a viable solution. > > Also, would need some help from the memcached c
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
Hey, On Fri, 4 May 2018, Om Kale wrote: > Hey Dormando and Trond,Thanks a lot for all of your inputs. > Let me give you guys a quick summary of what I am planning to do and the > issues I am facing: > I need memcached server with encryption and authentication support on > wireless devices. (Encyption, there is a way using set encoding key with > libmemcached, I need to write a decoding key function. (this looks feasible)) > Basically, when a libmemcached client connects, the memcached server only > allows the authenticated users/clients and then once this is done, the > communication is encrypted) > Hence I needed a openwrt package for SASL that can directly be downloaded > from the website and used without modification - therefore the decision to use > Cyrus-SASL) > Here are the problems: > 1. Cyrus-SASL > Cyrus-SASL (version 2.1.27-rc7) has two files with GPLv3 license, hence I > cannot use it. Also, the openwrt Cyrus-SASL has only the libsasl2 library, not > some of the other libraries mentioned that are needed for SASL support as per > the Couchbase link. > https://blog.couchbase.com/sasl-memcached-now-available/) Have you tried to just use libsasl2 and run it? I'm pretty sure the daemon doesn't need more than that after it's been compiled. The other libraries linked in the blog are the saslpasswd/etc stuff that you don't need. Just installed libsasl2 on my openwrt/lede router and it has digest/cram modules: # opkg files libsasl2 Package libsasl2 (2.1.26-3) is installed on root and has the following files: /usr/lib/sasl2/libdigestmd5.so.3 /usr/lib/libsasl2.so.3.0.0 /usr/lib/libsasl2.so.3 /usr/lib/sasl2/libscram.so /usr/lib/sasl2/libdigestmd5.so /usr/lib/sasl2/libplain.so.3 /usr/lib/sasl2/libanonymous.so.3.0.0 /usr/lib/sasl2/libscram.so.3.0.0 /usr/lib/sasl2/libdigestmd5.so.3.0.0 /usr/lib/sasl2/libanonymous.so.3 /usr/lib/sasl2/libanonymous.so /usr/lib/libsasl2.so /usr/lib/sasl2/libplain.so.3.0.0 /usr/lib/sasl2/libplain.so /usr/lib/sasl2/libcrammd5.so.3.0.0 /usr/lib/sasl2/libcrammd5.so.3 /usr/lib/sasl2/libcrammd5.so /usr/lib/sasl2/libscram.so.3 So if you build the passwd db and ship it over, or use the spec to write a small tool to generate the file properly, that seems like your least effort route. > FYI, I see the client here apparently appends the @localhostname with the > username. So this same format needs to be stored in the passwd file. I'd recommend code diving to understand this. > 2. I cannot use 'PLAIN' as I would see the username/passwords on packet > sniffs (wireshark or any other tool) Additionally, if the password file is > accessed it would be a problem. ok > 3. I also see lot of blogs involving DDOS on memcached with UDP mode. (I am > not sure if there is any future release where this is going to be planned) not relevant? Just disable UDP mode. it's not necessary and off by default in latest versions. > 4. One of the options that I could do is change the memcached/libmemcached > code itself to turn the sockets into SSL/TLS sockets. This would add some > overheard and few extra message exchanges but could be a viable solution. > Also, would need some help from the memcached community for this. This is much, much harder than you'd expect and I'm only assuming you think it's easier than doing what Trond suggested because you expect there to be one place in the code where "connections" are handled. It's a fairly large change that would end up touching much of the codebase. By comparison, just using libsasl2 from openwrt (please tell us why you can't, specifically?) and writing a tool to generate the digest file looks good. I haven't look at the cyrus-sasl API's though. Failing that, doing what trond suggested is probably fairly fast since the code is isolated behind a few specific calls. > > > > > > Thanks and Regards,Om Kale > > > On Thu, May 3, 2018 at 11:04 PM, Trond Norbye <trond.nor...@gmail.com> wrote: > If all you need is SASL authentication with DIGEST-MD5 and PLAIN > mechanisms you should be able to implement the few methods used by memcached > relatively quickly after you read the SASL spec (and get around any > licensing issues). > During startup memcached calls sasl_server_init where it sets up an array of > callback functions the SASL implementation may call to get more > information. In this function you may read your entire password database into > memory to avoid file IO at a later time. > > When a client connects memcached calls sasl_server_new, where it creates and > initializes a handle to your library (which is later released by > memcached calling sasl_dispose with the same pointer. After memcached called > sasl_dispose it won't use that handle. > > A client needs to know which mechanisms the server supports, and in order to > do tha
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
You need the server to be on any OS? I thought it was a router thing you
were embedding.
What exactly are you doing, if you can share?
On Wed, 2 May 2018, Om Kale wrote:
> Hey Dormando,Yes you are right. I agree with you. I have gotten everything
> working with Ubuntu since Day 1 using the libsasl2,sasl2-bin and the other
> installs mentioned. However, the problem is I need it working on all OS's.
> Also, I cannot use Cyrus-SASL (that has only libsasl2 bundled) in my project
> now as some of the files in its latest 2.1.27-rc7 version have a GPLv3
> license requirement. That's the reason I have thought of the change. Bit of a
> dire
> straits situation here.
>
>
> Thanks and Regards,Om Kale
>
>
> On Wed, May 2, 2018 at 7:25 PM, dormando <dorma...@rydia.net> wrote:
> Hey,
>
> Please interpret this with kindness: if you're struggling getting sasl
> to
> work, getting asynchronous TLS to work, be performant enough, and not
> buggy, while also forking the project, is going to be a very very bad
> idea
> for you.
>
> If you're willing to put the effort into figuring out TLS into
> memcached,
> you're better off reading the cyrus source code to figure out how
> password
> databases work. Read the SASL protocol spec (it's not too bad).
>
> I see you spending a huge amount of time trying to work around the bugs
> you encounter; instead of going around, go through them. Get the
> password
> file to work the way you want it to.
>
> On Wed, 2 May 2018, Om Kale wrote:
>
> > Hi Dormando,Thanks for your reply. Yes, that works. Also, one more
> thing that I was
> > curious to know or rather want to add to memcached.
> > Is there anyway I can go ahead and modify memcached itself to support
> SSL/TLS (using
> > certificates) without using this third-party cyrus plugin/libsasl2?
> > If yes, where the memcached code need to be added for this.
> Basically, I want to know
> > where exactly in the memcached code does the client connect to server
> and do the SASL
> > protocol negotiation/exchanges)
> > I was thinking of adding a way in which memcached would be able to
> support
> > authentication depending on whatever ssl library the user wants to
> use. (not restrict it
> > to cyrus-sasl or libsasl2)
> >
> >
> > Thanks and Regards,Om Kale
> >
> >
> > On Mon, Apr 30, 2018 at 1:50 PM, dormando <dorma...@rydia.net> wrote:
> > Hey,
> >
> > The passwd needs to be created with saslpasswd for most of the
> other auth
> > types to work, otherwise you'll have to do it manually and I
> have no idea
> > how to do that. IE; the saslpasswd files I created when trying
> to
> > reproduce your method worked fine with DIGEST-MD5 as well.
> >
> > On Mon, 30 Apr 2018, Om Kale wrote:
> >
> > > Hi All,I am trying to get my head around making memcached
> work with SASL
> > support. The PLAIN auth is working but still running into
> issues for
> > > DIGEST-MD5.
> > > I have changed my memcached client side code to enable MD5 as
> follows. I
> > have enabled the behavior to support MD5 and then passed the MD5
> > >
> > > /*
> > > * Test that libmemcached is built with SASL support.
> > > */
> > > #include
> > > #include
> > > #include
> > > #include
> > > #include
> > >
> > > const char* key = "abc";
> > > const char* value = "value";
> > >
> > > // test basic get/set operation works.
> > > void test_getset(memcached_st* cache)
> > > {
> > > char* r_value;
> > > uint32_t flags = 0;
> > > uint32_t r_flags = 0;
> > > size_t val_length;
> > > memcached_return_t rc;
> > >
> > >
> > > rc = memcached_set(cache, key, strlen(key), value,
> strlen(value),
> > (time_t)0, flags);
> > > if (rc == MEMCACHED_TIMEOUT) {
> > > fprintf(stderr, "Set timeout\n");
> >
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
Hey,
Please interpret this with kindness: if you're struggling getting sasl to
work, getting asynchronous TLS to work, be performant enough, and not
buggy, while also forking the project, is going to be a very very bad idea
for you.
If you're willing to put the effort into figuring out TLS into memcached,
you're better off reading the cyrus source code to figure out how password
databases work. Read the SASL protocol spec (it's not too bad).
I see you spending a huge amount of time trying to work around the bugs
you encounter; instead of going around, go through them. Get the password
file to work the way you want it to.
On Wed, 2 May 2018, Om Kale wrote:
> Hi Dormando,Thanks for your reply. Yes, that works. Also, one more thing that
> I was
> curious to know or rather want to add to memcached.
> Is there anyway I can go ahead and modify memcached itself to support SSL/TLS
> (using
> certificates) without using this third-party cyrus plugin/libsasl2?
> If yes, where the memcached code need to be added for this. Basically, I want
> to know
> where exactly in the memcached code does the client connect to server and do
> the SASL
> protocol negotiation/exchanges)
> I was thinking of adding a way in which memcached would be able to support
> authentication depending on whatever ssl library the user wants to use. (not
> restrict it
> to cyrus-sasl or libsasl2)
>
>
> Thanks and Regards,Om Kale
>
>
> On Mon, Apr 30, 2018 at 1:50 PM, dormando <dorma...@rydia.net> wrote:
> Hey,
>
> The passwd needs to be created with saslpasswd for most of the other
> auth
> types to work, otherwise you'll have to do it manually and I have no
> idea
> how to do that. IE; the saslpasswd files I created when trying to
> reproduce your method worked fine with DIGEST-MD5 as well.
>
> On Mon, 30 Apr 2018, Om Kale wrote:
>
> > Hi All,I am trying to get my head around making memcached work with
> SASL
> support. The PLAIN auth is working but still running into issues for
> > DIGEST-MD5.
> > I have changed my memcached client side code to enable MD5 as
> follows. I
> have enabled the behavior to support MD5 and then passed the MD5
> >
> > /*
> > * Test that libmemcached is built with SASL support.
> > */
> > #include
> > #include
> > #include
> > #include
> > #include
> >
> > const char* key = "abc";
> > const char* value = "value";
> >
> > // test basic get/set operation works.
> > void test_getset(memcached_st* cache)
> > {
> > char* r_value;
> > uint32_t flags = 0;
> > uint32_t r_flags = 0;
> > size_t val_length;
> > memcached_return_t rc;
> >
> >
> > rc = memcached_set(cache, key, strlen(key), value, strlen(value),
> (time_t)0, flags);
> > if (rc == MEMCACHED_TIMEOUT) {
> > fprintf(stderr, "Set timeout\n");
> > return;
> > } else if (rc != MEMCACHED_SUCCESS) {
> > fprintf(stderr, "Set failed: %s\n", memcached_strerror(cache,
> rc));
> > return;
> > }
> >
> > r_value = memcached_get(cache, key, strlen(key), _length,
> _flags,
> );
> > if (rc == MEMCACHED_TIMEOUT) {
> > fprintf(stderr, "Get timeout\n");
> > return;
> > } else if (rc != MEMCACHED_SUCCESS) {
> > fprintf(stderr, "Get failed: %s\n", memcached_strerror(cache,
> rc));
> > return;
> > }
> >
> > if (strcmp(value, r_value) != 0) {
> > fprintf(stderr, "Get returned bad value! (%s != %s)!\n", value,
> r_value);
> > }
> >
> > if (r_flags != flags) {
> > fprintf(stderr, "Get returned bad flags! (%u != %u)!\n", flags,
> r_flags);
> > }
> >
> > fprintf(stdout, "Get/Set success!\n");
> > }
> >
> > // connect with SASL.
> > void authTest(const char* user, const char* pass, const char* server)
> > {
> > memcached_server_st *servers = NULL;
> > memcached_return_t rc;
> > memcached_st *cache;
> > uint32_t hashVal;
> > uint32_t hashPass;
> > uint32_t hash;
> > uint64_t behavior = 0;
> >
> >
&g
Re: Why memcached doesn't give an option or have the capability to ignore older values with same key based on some version.
Thanks for reaching out to figure things out; I do need to improve the documentation still. Good luck! On Wed, 25 Apr 2018, sachin shetty wrote: > Sorry I might have misunderstood. > I'm clear now. Thanks for clearing all my doubts and providing extensive info > on the > stampeding problem. It did help a lot. > In fact, the precedence did help to find the better solution. Thanks once > again. > > Cheers > > On Thursday, 26 April 2018 11:43:21 UTC+5:30, Dormando wrote: > You're way way over thinking this. > > I originally gave two examples, one was literally just the ADD and SET > commands, and the other is a wiki page that solves further issues, which > you should ignore for now. > > Again, memcached server is *atomic* for key manipulations. the server > works exactly as I described, no matter what you do or how you access > it. > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > Thank you. > > Does this memcached 'add' lock or 'set' lock defined in memecached > client > or > > memcached server? The reason I asked because, thread 1 from one > process > and thread 2 > > from another process tries to add and set simultaneously, will this > lock > happens at the > > memcached server or at the individual clients? > > > > On Thursday, 26 April 2018 09:26:30 UTC+5:30, Dormando wrote: > > Memcached is internally atomic on key operations. If you add > and set > at > > the same time, the set will effectively always win since they > are > > serialized. > > > > 1) add goes first. set overwrites it. > > 2) set goes first. add will fail. > > > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > > > Cool. > > > So let me assume the below scenario and correct me if I'm > wrong > here. > > > > > > Say thread 1 always does add and thread 2 always does set. > Will > there be > > any race conditions when both these threads do add and set > simultaneously? > > What > > > I mean is say thread1 does add and holds 'add' lock and if at > the > same > > time thread 2 comes for the set operation, how 'set' lock and > 'add' > lock is > > > handled here? > > > > > > > > > On Thursday, 26 April 2018 06:58:27 UTC+5:30, Dormando wrote: > > > Hey, > > > > > > ADD sets an item *only if it doesn't currently exist*. > > > > > > If you want thread 2 to be authoritative after updating > the > DB, you > > need > > > to use a SET. If you don't care and only ever want the > first > thread > > to > > > win, you can always use ADD. > > > > > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > > > > > Thank you for the reply. > > > > Can this add be used always, I mean during an update > as > well? > > > > What could be the potential disadvantage of this? > > > > So if two thread does an update using add, still lock > hold > well in > > this sceanrio? > > > > > > > > Thanks, > > > > Sachin > > > > > > > > > > > > On Wednesday, 25 April 2018 14:13:40 UTC+5:30, > Dormando > wrote: > > > > Hey, > > > > > > > > Two short answers: > > > > > > > > 1) thread 1 uses 'add' instead of 'set' > > > > 2) thread 2 uses 'set'. > > > > > > > > via add, a thread recaching an object can't > overwrite one > > already there. > > > > > > > > > > > > https://github.com/memcached/memcached/wiki/ProgrammingT
Re: Why memcached doesn't give an option or have the capability to ignore older values with same key based on some version.
You're way way over thinking this. I originally gave two examples, one was literally just the ADD and SET commands, and the other is a wiki page that solves further issues, which you should ignore for now. Again, memcached server is *atomic* for key manipulations. the server works exactly as I described, no matter what you do or how you access it. On Wed, 25 Apr 2018, sachin shetty wrote: > Thank you. > Does this memcached 'add' lock or 'set' lock defined in memecached client or > memcached server? The reason I asked because, thread 1 from one process and > thread 2 > from another process tries to add and set simultaneously, will this lock > happens at the > memcached server or at the individual clients? > > On Thursday, 26 April 2018 09:26:30 UTC+5:30, Dormando wrote: > Memcached is internally atomic on key operations. If you add and set at > the same time, the set will effectively always win since they are > serialized. > > 1) add goes first. set overwrites it. > 2) set goes first. add will fail. > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > Cool. > > So let me assume the below scenario and correct me if I'm wrong here. > > > > Say thread 1 always does add and thread 2 always does set. Will there > be > any race conditions when both these threads do add and set > simultaneously? > What > > I mean is say thread1 does add and holds 'add' lock and if at the same > time thread 2 comes for the set operation, how 'set' lock and 'add' > lock is > > handled here? > > > > > > On Thursday, 26 April 2018 06:58:27 UTC+5:30, Dormando wrote: > > Hey, > > > > ADD sets an item *only if it doesn't currently exist*. > > > > If you want thread 2 to be authoritative after updating the DB, > you > need > > to use a SET. If you don't care and only ever want the first > thread > to > > win, you can always use ADD. > > > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > > > Thank you for the reply. > > > Can this add be used always, I mean during an update as well? > > > What could be the potential disadvantage of this? > > > So if two thread does an update using add, still lock hold > well in > this sceanrio? > > > > > > Thanks, > > > Sachin > > > > > > > > > On Wednesday, 25 April 2018 14:13:40 UTC+5:30, Dormando wrote: > > > Hey, > > > > > > Two short answers: > > > > > > 1) thread 1 uses 'add' instead of 'set' > > > 2) thread 2 uses 'set'. > > > > > > via add, a thread recaching an object can't overwrite > one > already there. > > > > > > > > https://github.com/memcached/memcached/wiki/ProgrammingTricks#avoiding-stampeding-herd > > > > > > for related issues. using an advisory lock would change > the > flow: > > > > > > a. thread 1 gets a miss. > > > b. thread 1 runs 'add lock:key' > > > c. thread 1 wins, goes to db > > > d. thread 2 updates db. tries to grab key lock > > > e. thread 2 fails to grab key lock, waits and retries > > > > > > etc. bit more chatter but with added benefit of reducing > stampeding herd > > > if that's an issue. > > > > > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > > > > > There is a scenario where a cache gets updated by two > threads like the instance > > > > mentioned below > > > > > > > > a. thread 1 looks at the memcache key and gets a miss > > > > b. thread 1 falls back to the database > > > > c. thread 2 changes the database value > > > > d. thread 2 updates the memcache key with the new > value > > > > e. thread 1 sets the old database value into > m
Re: Why memcached doesn't give an option or have the capability to ignore older values with same key based on some version.
Memcached is internally atomic on key operations. If you add and set at the same time, the set will effectively always win since they are serialized. 1) add goes first. set overwrites it. 2) set goes first. add will fail. On Wed, 25 Apr 2018, sachin shetty wrote: > Cool. > So let me assume the below scenario and correct me if I'm wrong here. > > Say thread 1 always does add and thread 2 always does set. Will there be any > race conditions when both these threads do add and set simultaneously? What > I mean is say thread1 does add and holds 'add' lock and if at the same time > thread 2 comes for the set operation, how 'set' lock and 'add' lock is > handled here? > > > On Thursday, 26 April 2018 06:58:27 UTC+5:30, Dormando wrote: > Hey, > > ADD sets an item *only if it doesn't currently exist*. > > If you want thread 2 to be authoritative after updating the DB, you need > to use a SET. If you don't care and only ever want the first thread to > win, you can always use ADD. > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > Thank you for the reply. > > Can this add be used always, I mean during an update as well? > > What could be the potential disadvantage of this? > > So if two thread does an update using add, still lock hold well in > this sceanrio? > > > > Thanks, > > Sachin > > > > > > On Wednesday, 25 April 2018 14:13:40 UTC+5:30, Dormando wrote: > > Hey, > > > > Two short answers: > > > > 1) thread 1 uses 'add' instead of 'set' > > 2) thread 2 uses 'set'. > > > > via add, a thread recaching an object can't overwrite one > already there. > > > > > https://github.com/memcached/memcached/wiki/ProgrammingTricks#avoiding-stampeding-herd > > > > for related issues. using an advisory lock would change the > flow: > > > > a. thread 1 gets a miss. > > b. thread 1 runs 'add lock:key' > > c. thread 1 wins, goes to db > > d. thread 2 updates db. tries to grab key lock > > e. thread 2 fails to grab key lock, waits and retries > > > > etc. bit more chatter but with added benefit of reducing > stampeding herd > > if that's an issue. > > > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > > > There is a scenario where a cache gets updated by two threads > like the instance > > > mentioned below > > > > > > a. thread 1 looks at the memcache key and gets a miss > > > b. thread 1 falls back to the database > > > c. thread 2 changes the database value > > > d. thread 2 updates the memcache key with the new value > > > e. thread 1 sets the old database value into memcache > > > > > > I know this scenario is application specific. But the > question I have is if possible > > > there is an option to say the current value's timestamp is > older than the one already in > > > cache, then memcached should ignore the new entry. This could > solve race condition as > > > mentioned above. Suppose I say take the timestamp as the > version then memcached server > > > could make use of this to verify whether new entry coming is > older than the already > > > current one present. > > > > > > Handling at the client would be performance intensive because > of every time fetching an > > > existing value from the cache to check the timestamp. > > > > > > Are there any handlers for this to solve. Would be very > helpful if you could provide any > > > inputs on this. > > > > > > > > > Thanks, > > > Sachin > > > > > > > > > -- > > > > > > --- > > > You received this message because you are subscribed to the > Google Groups "memcached" > > > group. > > > To unsubscribe from this group and stop receiving emails from > it, send an email to > > > memcached+...@googlegro
Re: Why memcached doesn't give an option or have the capability to ignore older values with same key based on some version.
Hey, ADD sets an item *only if it doesn't currently exist*. If you want thread 2 to be authoritative after updating the DB, you need to use a SET. If you don't care and only ever want the first thread to win, you can always use ADD. On Wed, 25 Apr 2018, sachin shetty wrote: > Thank you for the reply. > Can this add be used always, I mean during an update as well? > What could be the potential disadvantage of this? > So if two thread does an update using add, still lock hold well in this > sceanrio? > > Thanks, > Sachin > > > On Wednesday, 25 April 2018 14:13:40 UTC+5:30, Dormando wrote: > Hey, > > Two short answers: > > 1) thread 1 uses 'add' instead of 'set' > 2) thread 2 uses 'set'. > > via add, a thread recaching an object can't overwrite one already there. > > > https://github.com/memcached/memcached/wiki/ProgrammingTricks#avoiding-stampeding-herd > > for related issues. using an advisory lock would change the flow: > > a. thread 1 gets a miss. > b. thread 1 runs 'add lock:key' > c. thread 1 wins, goes to db > d. thread 2 updates db. tries to grab key lock > e. thread 2 fails to grab key lock, waits and retries > > etc. bit more chatter but with added benefit of reducing stampeding herd > if that's an issue. > > On Wed, 25 Apr 2018, sachin shetty wrote: > > > There is a scenario where a cache gets updated by two threads like > the instance > > mentioned below > > > > a. thread 1 looks at the memcache key and gets a miss > > b. thread 1 falls back to the database > > c. thread 2 changes the database value > > d. thread 2 updates the memcache key with the new value > > e. thread 1 sets the old database value into memcache > > > > I know this scenario is application specific. But the question I have > is if possible > > there is an option to say the current value's timestamp is older than > the one already in > > cache, then memcached should ignore the new entry. This could solve > race condition as > > mentioned above. Suppose I say take the timestamp as the version then > memcached server > > could make use of this to verify whether new entry coming is older > than the already > > current one present. > > > > Handling at the client would be performance intensive because of > every time fetching an > > existing value from the cache to check the timestamp. > > > > Are there any handlers for this to solve. Would be very helpful if > you could provide any > > inputs on this. > > > > > > Thanks, > > Sachin > > > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "memcached" > > group. > > To unsubscribe from this group and stop receiving emails from it, > send an email to > > memcached+...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Why memcached doesn't give an option or have the capability to ignore older values with same key based on some version.
Hey, Two short answers: 1) thread 1 uses 'add' instead of 'set' 2) thread 2 uses 'set'. via add, a thread recaching an object can't overwrite one already there. https://github.com/memcached/memcached/wiki/ProgrammingTricks#avoiding-stampeding-herd for related issues. using an advisory lock would change the flow: a. thread 1 gets a miss. b. thread 1 runs 'add lock:key' c. thread 1 wins, goes to db d. thread 2 updates db. tries to grab key lock e. thread 2 fails to grab key lock, waits and retries etc. bit more chatter but with added benefit of reducing stampeding herd if that's an issue. On Wed, 25 Apr 2018, sachin shetty wrote: > There is a scenario where a cache gets updated by two threads like the > instance > mentioned below > > a. thread 1 looks at the memcache key and gets a miss > b. thread 1 falls back to the database > c. thread 2 changes the database value > d. thread 2 updates the memcache key with the new value > e. thread 1 sets the old database value into memcache > > I know this scenario is application specific. But the question I have is if > possible > there is an option to say the current value's timestamp is older than the one > already in > cache, then memcached should ignore the new entry. This could solve race > condition as > mentioned above. Suppose I say take the timestamp as the version then > memcached server > could make use of this to verify whether new entry coming is older than the > already > current one present. > > Handling at the client would be performance intensive because of every time > fetching an > existing value from the cache to check the timestamp. > > Are there any handlers for this to solve. Would be very helpful if you could > provide any > inputs on this. > > > Thanks, > Sachin > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" > group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: basic auth token support
Well, being extremely thorough; read from file on startup and then use online commands for modifications is perfectly doable too (and how some things work) but I feel like that will be hard for people to use. On Tue, 17 Apr 2018, dormando wrote: > Right; I'm saying there needs to be a mechanism of loading what the > current tokens are :) Which are probably going to be from a file on disk. > A rotation would be staged modifications to this file + reload commands or > auto reloading. > > Pulling from file would be necessary to avoid losing tokens during > transition periods if the process is restarted for any reason. > > This is unusual since memc doesn't have a habit of talking to the > filesystem post-startup (extstore nonwithstanding), but should be okay. > > On Tue, 17 Apr 2018, John Reilly wrote: > > > Reload would be handy to have but not absolutely necessary. > > For rotation, one would just set up their second token (the new one) at > > some point in time. Any time after that clients can transition to the new > > token. > > Once all clients are transitioned to the new token, the original token > > would then need to be removed on the memcached server. > > Thanks, > > John > > > > On Mon, Apr 16, 2018 at 10:57 AM dormando <dorma...@rydia.net> wrote: > > Hey, > > > > Thanks for the feedback! That should be doable. I'm used to this > > being a > > pain with TLS ticket rotation/etc anyway. This'll probably end up > > requiring a reload mechanism but shouldn't be too messy, I guess? > > > > On Mon, 16 Apr 2018, John Reilly wrote: > > > > > Hi dormando,I would love to see this change. One thing that would > > be great to have is support for multiple tokens for the purpose of key > > rotation. If > > > there are roles, one could just assign 2 equivalent roles with > > different tokens, but in the absence of roles as you mentioned just having > > the ability to > > > define multiple tokens on the server level would work nicely. This > > is an issue today with the redis password mechanism - once it is set, > > changing the > > > token across all clients and server at the same time is > > problematic. > > > > > > Of course, sasl already supports this so clients that want this > > capability can use sasl, but it would be nice to have it available in any > > new default > > > authentication mechanism. > > > > > > Thanks, > > > John > > > > > > On Wed, Apr 11, 2018 at 1:59 AM dormando <dorma...@rydia.net> wrote: > > > Hey, > > > > > > In the wake of all this exposed-internet fun, I want to do > > something I > > > should've years ago; add support for a basic authentication > > token. > > > > > > Currently, with binary protocol, you have the option of using > > SASL. This > > > requires compiling against sasl, a client that both speaks > > binprot and > > > sasl, and understand the sasl ecosystem enough to generate > > configurations, > > > password files, hook it up to kerberos, or what have you. > > This is useful; > > > I should also see if ascii can support it. > > > > > > However, it's not simple. It can never be a default. > > > > > > I propose to do more or less what redis does, except I'd call > > it a token > > > instead of a password. Both ascii and binprot would support > > it. > > > > > > There are two options I'm considering: > > > > > > 1) add a new command, "auth [token]", or "auth > > [length]\r\ntoken" > > > > > > or: > > > > > > 2) if a connection is in an unauthenticated state, it will > > only accept a > > > "set auth [etc]\r\ntoken" magic key. > > > > > > It should be possible to extend this down the line if we want > > roles for > > > tokens by just having multiple tokens on the server.. > > > > > > It would be passed by commandline (it would rewrite the > > string on start) > > > and/or passed as a file to open and read on start. A restart > >
Re: basic auth token support
Right; I'm saying there needs to be a mechanism of loading what the current tokens are :) Which are probably going to be from a file on disk. A rotation would be staged modifications to this file + reload commands or auto reloading. Pulling from file would be necessary to avoid losing tokens during transition periods if the process is restarted for any reason. This is unusual since memc doesn't have a habit of talking to the filesystem post-startup (extstore nonwithstanding), but should be okay. On Tue, 17 Apr 2018, John Reilly wrote: > Reload would be handy to have but not absolutely necessary. > For rotation, one would just set up their second token (the new one) at some > point in time. Any time after that clients can transition to the new token. > Once all clients are transitioned to the new token, the original token would > then need to be removed on the memcached server. > Thanks, > John > > On Mon, Apr 16, 2018 at 10:57 AM dormando <dorma...@rydia.net> wrote: > Hey, > > Thanks for the feedback! That should be doable. I'm used to this being a > pain with TLS ticket rotation/etc anyway. This'll probably end up > requiring a reload mechanism but shouldn't be too messy, I guess? > > On Mon, 16 Apr 2018, John Reilly wrote: > > > Hi dormando,I would love to see this change. One thing that would be > great to have is support for multiple tokens for the purpose of key > rotation. If > > there are roles, one could just assign 2 equivalent roles with > different tokens, but in the absence of roles as you mentioned just having > the ability to > > define multiple tokens on the server level would work nicely. This > is an issue today with the redis password mechanism - once it is set, > changing the > > token across all clients and server at the same time is problematic. > > > > Of course, sasl already supports this so clients that want this > capability can use sasl, but it would be nice to have it available in any > new default > > authentication mechanism. > > > > Thanks, > > John > > > > On Wed, Apr 11, 2018 at 1:59 AM dormando <dorma...@rydia.net> wrote: > > Hey, > > > > In the wake of all this exposed-internet fun, I want to do > something I > > should've years ago; add support for a basic authentication > token. > > > > Currently, with binary protocol, you have the option of using > SASL. This > > requires compiling against sasl, a client that both speaks > binprot and > > sasl, and understand the sasl ecosystem enough to generate > configurations, > > password files, hook it up to kerberos, or what have you. This > is useful; > > I should also see if ascii can support it. > > > > However, it's not simple. It can never be a default. > > > > I propose to do more or less what redis does, except I'd call > it a token > > instead of a password. Both ascii and binprot would support it. > > > > There are two options I'm considering: > > > > 1) add a new command, "auth [token]", or "auth > [length]\r\ntoken" > > > > or: > > > > 2) if a connection is in an unauthenticated state, it will only > accept a > > "set auth [etc]\r\ntoken" magic key. > > > > It should be possible to extend this down the line if we want > roles for > > tokens by just having multiple tokens on the server.. > > > > It would be passed by commandline (it would rewrite the string > on start) > > and/or passed as a file to open and read on start. A restart > would be > > required to change the token. > > > > Plaintext only on both ends, no hashing. It should exist to > help prevent > > accidents more than anything else. I will probably add a delay > on failure > > to mitigate brute-force, but no other features. > > > > The really hard part is adding support to clients, and perhaps > in a few > > years distro's can start shipping with default or randomized > auth tokens. > > > > Open to feedback. Thanks! > > -Dormando > > > > -- > > > > --- >
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
oh; you might need to `sudo ldconfig` before that works, too On Tue, 17 Apr 2018, Om Kale wrote: > Hey Dormando, > I was trying to play around with memcached sasl a bit more on Ubuntu. > I tried to use the cyrus sasl libraries. > However, when I try to run the memcached server it gives the following error: > ~/Downloads/memcached-1.5.7$ memcached -S -vv > memcached: error while loading shared libraries: libsasl2.so.2: cannot open > shared object file: No such file or directory > > > > I checked in usr/local/lib and I see libsasl2.so.3 present. > cisco@dd17-ubuntu-namsoo:/usr/local/lib$ ls -lrt > drwxrwsr-x 3 root staff 4096 Feb 28 10:25 python3.5 > drwxrwsr-x 4 root staff 4096 Feb 28 10:35 python2.7 > drwxr-xr-x 2 root root 4096 Apr 16 08:47 sasl2 > -rwxr-xr-x 1 root root 163912 Apr 17 03:09 libhashkit.so.2.0.0 > lrwxrwxrwx 1 root root 19 Apr 17 03:09 libhashkit.so.2 -> > libhashkit.so.2.0.0 > lrwxrwxrwx 1 root root 19 Apr 17 03:09 libhashkit.so -> > libhashkit.so.2.0.0 > -rwxr-xr-x 1 root root 938 Apr 17 03:09 libhashkit.la > -rwxr-xr-x 1 root root 1373952 Apr 17 03:09 libmemcached.so.11.0.0 > lrwxrwxrwx 1 root root 22 Apr 17 03:09 libmemcached.so.11 -> > libmemcached.so.11.0.0 > lrwxrwxrwx 1 root root 22 Apr 17 03:09 libmemcached.so -> > libmemcached.so.11.0.0 > -rwxr-xr-x 1 root root 978 Apr 17 03:09 libmemcached.la > -rwxr-xr-x 1 root root 114792 Apr 17 03:09 libmemcachedutil.so.2.0.0 > lrwxrwxrwx 1 root root 25 Apr 17 03:09 libmemcachedutil.so.2 -> > libmemcachedutil.so.2.0.0 > lrwxrwxrwx 1 root root 25 Apr 17 03:09 libmemcachedutil.so -> > libmemcachedutil.so.2.0.0 > -rwxr-xr-x 1 root root 1033 Apr 17 03:09 libmemcachedutil.la > -rw-r--r-- 1 root root 329582 Apr 17 03:09 libhashkit.a > -rw-r--r-- 1 root root 3175600 Apr 17 03:09 libmemcached.a > -rw-r--r-- 1 root root 220608 Apr 17 03:09 libmemcachedutil.a > drwxr-xr-x 2 root root 4096 Apr 17 03:09 pkgconfig > -rwxr-xr-x 1 root root 485528 Apr 17 03:43 libsasl2.so.3.0.0 > lrwxrwxrwx 1 root root 17 Apr 17 03:43 libsasl2.so.3 -> > libsasl2.so.3.0.0 > lrwxrwxrwx 1 root root 17 Apr 17 03:43 libsasl2.so -> libsasl2.so.3.0.0 > -rwxr-xr-x 1 root root 652 Apr 17 03:43 libsasl2.la > cisco@dd17-ubuntu-namsoo:/usr/local/lib$ > > Has anyone else seen similar error while working with cyrus-sasl-2.1.27? > > > Thanks and Regards,Om Kale > > > On Wed, Apr 11, 2018 at 8:34 PM, dormando <dorma...@rydia.net> wrote: > Hey, > > Good to hear! good luck. > > SASL is the only method. I sent a proposal to this mailing list > yesterday > for authentication tokens. > > On Wed, 11 Apr 2018, Om Kale wrote: > > > Hey Dormando, > > Works like a charm with Ubuntu. So its a MAC problem then. > > I also had an additional question: > > In memcached, is there any way of doing authentication without > actually using the SASL library available. For example, using some other > underlying ssl > > libraries. > > > > > > Thanks and Regards,Om Kale > > > > > > On Wed, Apr 11, 2018 at 4:14 PM, dormando <dorma...@rydia.net> wrote: > > I don't see anything wrong with it. Since you ultimately need > this to run > > on ubuntu, why don't you start testing with a VM? It might not > matter at > > all if the problem is just with the mac. > > > > On Wed, 11 Apr 2018, Om Kale wrote: > > > > > Ah, I see. This person on the memcached group also observed > the same issue on Cent OS (I see it on Mac OS) some time back: > > > > > > https://groups.google.com/forum/#!topic/memcached/mtzcFVYahZo > > > > > > I have attached my client program testsasl2.c with this mail. > I don't see any errors in the code. Please do let me know if you find > > anything. > > > > > > Used following to compile and run: > > > OKALE-M-33H5:mycode okale$ gcc -o testsasl2 testsasl2.c > -lmemcached -lsasl2 -lssl > > > OKALE-M-33H5:mycode okale$ ./testsasl2 testuser testpass > localhost > > > Set failed: AUTHENTICATION FAILURE > > > OKALE-M-33H5:mycode okale$ > > > > > > > > > On memcached server side I see the same error when I use > testuser:testpass in the sasl database. > > > OKALE-M-33H5:tmp oka
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
Did you recompile memcached on there or copy the binary? On Tue, 17 Apr 2018, Om Kale wrote: > Hey Dormando, > I was trying to play around with memcached sasl a bit more on Ubuntu. > I tried to use the cyrus sasl libraries. > However, when I try to run the memcached server it gives the following error: > ~/Downloads/memcached-1.5.7$ memcached -S -vv > memcached: error while loading shared libraries: libsasl2.so.2: cannot open > shared object file: No such file or directory > > > > I checked in usr/local/lib and I see libsasl2.so.3 present. > cisco@dd17-ubuntu-namsoo:/usr/local/lib$ ls -lrt > drwxrwsr-x 3 root staff 4096 Feb 28 10:25 python3.5 > drwxrwsr-x 4 root staff 4096 Feb 28 10:35 python2.7 > drwxr-xr-x 2 root root 4096 Apr 16 08:47 sasl2 > -rwxr-xr-x 1 root root 163912 Apr 17 03:09 libhashkit.so.2.0.0 > lrwxrwxrwx 1 root root 19 Apr 17 03:09 libhashkit.so.2 -> > libhashkit.so.2.0.0 > lrwxrwxrwx 1 root root 19 Apr 17 03:09 libhashkit.so -> > libhashkit.so.2.0.0 > -rwxr-xr-x 1 root root 938 Apr 17 03:09 libhashkit.la > -rwxr-xr-x 1 root root 1373952 Apr 17 03:09 libmemcached.so.11.0.0 > lrwxrwxrwx 1 root root 22 Apr 17 03:09 libmemcached.so.11 -> > libmemcached.so.11.0.0 > lrwxrwxrwx 1 root root 22 Apr 17 03:09 libmemcached.so -> > libmemcached.so.11.0.0 > -rwxr-xr-x 1 root root 978 Apr 17 03:09 libmemcached.la > -rwxr-xr-x 1 root root 114792 Apr 17 03:09 libmemcachedutil.so.2.0.0 > lrwxrwxrwx 1 root root 25 Apr 17 03:09 libmemcachedutil.so.2 -> > libmemcachedutil.so.2.0.0 > lrwxrwxrwx 1 root root 25 Apr 17 03:09 libmemcachedutil.so -> > libmemcachedutil.so.2.0.0 > -rwxr-xr-x 1 root root 1033 Apr 17 03:09 libmemcachedutil.la > -rw-r--r-- 1 root root 329582 Apr 17 03:09 libhashkit.a > -rw-r--r-- 1 root root 3175600 Apr 17 03:09 libmemcached.a > -rw-r--r-- 1 root root 220608 Apr 17 03:09 libmemcachedutil.a > drwxr-xr-x 2 root root 4096 Apr 17 03:09 pkgconfig > -rwxr-xr-x 1 root root 485528 Apr 17 03:43 libsasl2.so.3.0.0 > lrwxrwxrwx 1 root root 17 Apr 17 03:43 libsasl2.so.3 -> > libsasl2.so.3.0.0 > lrwxrwxrwx 1 root root 17 Apr 17 03:43 libsasl2.so -> libsasl2.so.3.0.0 > -rwxr-xr-x 1 root root 652 Apr 17 03:43 libsasl2.la > cisco@dd17-ubuntu-namsoo:/usr/local/lib$ > > Has anyone else seen similar error while working with cyrus-sasl-2.1.27? > > > Thanks and Regards,Om Kale > > > On Wed, Apr 11, 2018 at 8:34 PM, dormando <dorma...@rydia.net> wrote: > Hey, > > Good to hear! good luck. > > SASL is the only method. I sent a proposal to this mailing list > yesterday > for authentication tokens. > > On Wed, 11 Apr 2018, Om Kale wrote: > > > Hey Dormando, > > Works like a charm with Ubuntu. So its a MAC problem then. > > I also had an additional question: > > In memcached, is there any way of doing authentication without > actually using the SASL library available. For example, using some other > underlying ssl > > libraries. > > > > > > Thanks and Regards,Om Kale > > > > > > On Wed, Apr 11, 2018 at 4:14 PM, dormando <dorma...@rydia.net> wrote: > > I don't see anything wrong with it. Since you ultimately need > this to run > > on ubuntu, why don't you start testing with a VM? It might not > matter at > > all if the problem is just with the mac. > > > > On Wed, 11 Apr 2018, Om Kale wrote: > > > > > Ah, I see. This person on the memcached group also observed > the same issue on Cent OS (I see it on Mac OS) some time back: > > > > > > https://groups.google.com/forum/#!topic/memcached/mtzcFVYahZo > > > > > > I have attached my client program testsasl2.c with this mail. > I don't see any errors in the code. Please do let me know if you find > > anything. > > > > > > Used following to compile and run: > > > OKALE-M-33H5:mycode okale$ gcc -o testsasl2 testsasl2.c > -lmemcached -lsasl2 -lssl > > > OKALE-M-33H5:mycode okale$ ./testsasl2 testuser testpass > localhost > > > Set failed: AUTHENTICATION FAILURE > > > OKALE-M-33H5:mycode okale$ > > > > > > > > > On memcached server side I see the same error when I use > testuser:testpass in the sasl database. > > > OKALE-M-33H5:tmp oka
Re: basic auth token support
Hey, Thanks for the feedback! That should be doable. I'm used to this being a pain with TLS ticket rotation/etc anyway. This'll probably end up requiring a reload mechanism but shouldn't be too messy, I guess? On Mon, 16 Apr 2018, John Reilly wrote: > Hi dormando,I would love to see this change. One thing that would be great > to have is support for multiple tokens for the purpose of key rotation. If > there are roles, one could just assign 2 equivalent roles with different > tokens, but in the absence of roles as you mentioned just having the ability > to > define multiple tokens on the server level would work nicely. This is an > issue today with the redis password mechanism - once it is set, changing the > token across all clients and server at the same time is problematic. > > Of course, sasl already supports this so clients that want this capability > can use sasl, but it would be nice to have it available in any new default > authentication mechanism. > > Thanks, > John > > On Wed, Apr 11, 2018 at 1:59 AM dormando <dorma...@rydia.net> wrote: > Hey, > > In the wake of all this exposed-internet fun, I want to do something I > should've years ago; add support for a basic authentication token. > > Currently, with binary protocol, you have the option of using SASL. This > requires compiling against sasl, a client that both speaks binprot and > sasl, and understand the sasl ecosystem enough to generate > configurations, > password files, hook it up to kerberos, or what have you. This is > useful; > I should also see if ascii can support it. > > However, it's not simple. It can never be a default. > > I propose to do more or less what redis does, except I'd call it a token > instead of a password. Both ascii and binprot would support it. > > There are two options I'm considering: > > 1) add a new command, "auth [token]", or "auth [length]\r\ntoken" > > or: > > 2) if a connection is in an unauthenticated state, it will only accept a > "set auth [etc]\r\ntoken" magic key. > > It should be possible to extend this down the line if we want roles for > tokens by just having multiple tokens on the server.. > > It would be passed by commandline (it would rewrite the string on start) > and/or passed as a file to open and read on start. A restart would be > required to change the token. > > Plaintext only on both ends, no hashing. It should exist to help prevent > accidents more than anything else. I will probably add a delay on > failure > to mitigate brute-force, but no other features. > > The really hard part is adding support to clients, and perhaps in a few > years distro's can start shipping with default or randomized auth > tokens. > > Open to feedback. Thanks! > -Dormando > > -- > > --- > You received this message because you are subscribed to the Google > Groups "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "memcached" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to memcached+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
I don't see anything wrong with it. Since you ultimately need this to run on ubuntu, why don't you start testing with a VM? It might not matter at all if the problem is just with the mac. On Wed, 11 Apr 2018, Om Kale wrote: > Ah, I see. This person on the memcached group also observed the same issue on > Cent OS (I see it on Mac OS) some time back: > > https://groups.google.com/forum/#!topic/memcached/mtzcFVYahZo > > I have attached my client program testsasl2.c with this mail. I don't see any > errors in the code. Please do let me know if you find anything. > > Used following to compile and run: > OKALE-M-33H5:mycode okale$ gcc -o testsasl2 testsasl2.c -lmemcached -lsasl2 > -lssl > OKALE-M-33H5:mycode okale$ ./testsasl2 testuser testpass localhost > Set failed: AUTHENTICATION FAILURE > OKALE-M-33H5:mycode okale$ > > > On memcached server side I see the same error when I use testuser:testpass in > the sasl database. > OKALE-M-33H5:tmp okale$ pwd > /tmp > OKALE-M-33H5:tmp okale$ cat memcached-sasl-db > testuser:testpass > OKALE-M-33H5:tmp okale$ > > Memcached server: > OKALE-M-33H5:memcached-1.5.7 okale$ SASL_CONF_PATH="/Users/okale/sasl" > memcached -S -vv > Reading configuration from: > Initialized SASL. > slab class 1: chunk size 96 perslab 10922 > slab class 2: chunk size 120 perslab 8738 > slab class 3: chunk size 152 perslab 6898 > slab class 4: chunk size 192 perslab 5461 > slab class 5: chunk size 240 perslab 4369 > slab class 6: chunk size 304 perslab 3449 > slab class 7: chunk size 384 perslab 2730 > slab class 8: chunk size 480 perslab 2184 > slab class 9: chunk size 600 perslab 1747 > slab class 10: chunk size 752 perslab 1394 > slab class 11: chunk size 944 perslab 1110 > slab class 12: chunk size 1184 perslab 885 > slab class 13: chunk size 1480 perslab 708 > slab class 14: chunk size 1856 perslab 564 > slab class 15: chunk size 2320 perslab 451 > slab class 16: chunk size 2904 perslab 361 > slab class 17: chunk size 3632 perslab 288 > slab class 18: chunk size 4544 perslab 230 > slab class 19: chunk size 5680 perslab 184 > slab class 20: chunk size 7104 perslab 147 > slab class 21: chunk size 8880 perslab 118 > slab class 22: chunk size 11104 perslab 94 > slab class 23: chunk size 13880 perslab 75 > slab class 24: chunk size 17352 perslab 60 > slab class 25: chunk size 21696 perslab 48 > slab class 26: chunk size 27120 perslab 38 > slab class 27: chunk size 33904 perslab 30 > slab class 28: chunk size 42384 perslab 24 > slab class 29: chunk size 52984 perslab 19 > slab class 30: chunk size 66232 perslab 15 > slab class 31: chunk size 82792 perslab 12 > slab class 32: chunk size 103496 perslab 10 > slab class 33: chunk size 129376 perslab 8 > slab class 34: chunk size 161720 perslab 6 > slab class 35: chunk size 202152 perslab 5 > slab class 36: chunk size 252696 perslab 4 > slab class 37: chunk size 315872 perslab 3 > slab class 38: chunk size 394840 perslab 2 > slab class 39: chunk size 524288 perslab 2 > <17 server listening (binary) > <18 server listening (binary) > <19 new binary client connection. > <19 Read binary protocol data: > <19 0x80 0x20 0x00 0x00 > <19 0x00 0x00 0x00 0x00 > <19 0x00 0x00 0x00 0x00 > <19 0x00 0x02 0x00 0x00 > <19 0x00 0x00 0x00 0x00 > <19 0x00 0x00 0x00 0x00 > authenticated() in cmd 0x20 is true > >19 Writing bin response: > >19 0x81 0x20 0x00 0x00 > >19 0x00 0x00 0x00 0x00 > >19 0x00 0x00 0x00 0x05 > >19 0x00 0x02 0x00 0x00 > >19 0x00 0x00 0x00 0x00 > >19 0x00 0x00 0x00 0x00 > <19 Read binary protocol data: > <19 0x80 0x21 0x00 0x05 > <19 0x00 0x00 0x00 0x00 > <19 0x00 0x00 0x00 0x1f > <19 0x00 0x02 0x00 0x00 > <19 0x00 0x00 0x00 0x00 > <19 0x00 0x00 0x00 0x00 > authenticated() in cmd 0x21 is true > mech: ``PLAIN'' with 26 bytes of data > INFO: User <testuser@OKALE-M-33H5> failed to authenticate > SASL (severity 2): Password verification failed > sasl result code: -20 > Unknown sasl response: -20 > >19 Writing an error: Auth failure. > >19 Writing bin response: > >19 0x81 0x21 0x00 0x00 > >19 0x00 0x00 0x00 0x20 > >19 0x00 0x00 0x00 0x0d > >19 0x00 0x02 0x00 0x00 > >19 0x00 0x00 0x0
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
I'm on ubuntu.. I didn't do anything special or change anything, I gave a list of all the commands I ran to make it work verbatim. I didn't have the username@ETC issue happen at all. If I had to guess, that would need to be fixed on the client side. On Wed, 11 Apr 2018, Om Kale wrote: > Hey Dormando, > No the saslpasswd2 command didn't give me any output. I will use strace to > check for errors. > Additionally, are you using an Ubuntu machine (If yes, how did you install > sasl on your machine and did you make any changes to it inorder to make it > work). I am asking this as I will also be running this on Ubuntu later. > On my MAC, for SASL, I just installed the sasl2bin library and some other > dependencies. (I tried with cyrus-sasl-plain as well, but did't seem to work) > Also, for my other question about memcached client appending mylocalhost-mac > name as 'testuser@OKALE-M-33H5'? I saw some posts reporting this same issue > on the group and stackoverflow. > Will I be able to perform the authentication without saving the username in > this format in my sasl db file? > > Thanks and Regards,Om Kale > > > On Tue, Apr 10, 2018 at 11:40 PM, dormando <dorma...@rydia.net> wrote: > I don't really know. I don't have a mac so I don't know why saslpasswd2 > doesn't work. > > If it gives you any output when it doesn't work (with the -f argument), > please share it. You can also strace the command to see if there are any > obvious errors before it exits. There must be some reason why it's not > writing the file; it worked fine for me immediately. > > how did you install sasl on your machine? or did it come with it? > > On Tue, 10 Apr 2018, Om Kale wrote: > > > Hi Dormando, > > I finally figured it out the issue from the above thread itself. > > The small change in steps as shown below work on my MAC machine: > > > > OKALE-M-33H5:memcached-1.5.7 okale$ echo > "testuser@OKALE-M-33H5:testpass" > /tmp/memcached-sasl-db > > OKALE-M-33H5:memcached-1.5.7 okale$ > SASL_CONF_PATH="/Users/okale/sasl" memcached -v -S > > Reading configuration from: > > Initialized SASL. > > mech: ``PLAIN'' with 26 bytes of data > > sasl result code: 0 > > > > > > Client Side: > > OKALE-M-33H5:mycode okale$ ./testsasl testuser testpass 127.0.0.1 > > Get/Set success! > > > > I observe two things here: > > 1. The saslpasswd2 doesn't create the memcached-sasl-db file for me, > I instead used the echo command listed above. > > 2. Now memcached appends mylocalhost-mac name i.e. @OKALE-M-33H5 to > the username when I run the client. (Not sure why this is the case) > > It would be great if you could guide me as to whether there is a > specific reason to it and will I be able to perform the authentication > without saving > > the username in this format in my sasl db file. > > > > > > Appreciate all the help! > > > > Thanks and Regards,Om Kale > > > > > > > > On Tue, Apr 10, 2018 at 10:07 PM, Om Kale <omkal...@gmail.com> wrote: > > Hi Dormando, > > Thanks for your guidance. Meanwhile, reading through the > memcached email chain, I see someone else also observed something similar > but there > > was no solution. The question is how do I create the > memcahed-sasl-pwdb file and populate it with username and password as the > saslpasswd2 > > doesnt seem to be doing it in this case. > > > > > https://groups.google.com/d/msg/memcached/mtzcFVYahZo/ZGrX6i5FWsUJ > > > > > > > > Regards, > > Om Kale > > > > > > > > On Tuesday, April 10, 2018 at 7:07:58 PM UTC-7, Om Kale wrote: > > Yes, that is correct. But if this change is done, then the > entry in memcached.conf for sasldb_path should also change to sasldb_path: > > /tmp/memcached-sasl-pwdb. > > However, if this change is made still it fails. > > > > OKALE-M-33H5:sasl okale$ cat memcached.conf > > mech_list: plain > > log_level: 5 > > sasldb_path: /tmp/memcached-sasl-pwdb > > OKALE-M-33H5:sasl okale$ echo testpass | saslpasswd2 -f > /tmp/memcached-sasl-pwdb -a memcached -c -p testuser > > OKALE-M-33H5:sasl okale$ SASL_CONF_PATH="/Users/okale/sa
basic auth token support
Hey, In the wake of all this exposed-internet fun, I want to do something I should've years ago; add support for a basic authentication token. Currently, with binary protocol, you have the option of using SASL. This requires compiling against sasl, a client that both speaks binprot and sasl, and understand the sasl ecosystem enough to generate configurations, password files, hook it up to kerberos, or what have you. This is useful; I should also see if ascii can support it. However, it's not simple. It can never be a default. I propose to do more or less what redis does, except I'd call it a token instead of a password. Both ascii and binprot would support it. There are two options I'm considering: 1) add a new command, "auth [token]", or "auth [length]\r\ntoken" or: 2) if a connection is in an unauthenticated state, it will only accept a "set auth [etc]\r\ntoken" magic key. It should be possible to extend this down the line if we want roles for tokens by just having multiple tokens on the server.. It would be passed by commandline (it would rewrite the string on start) and/or passed as a file to open and read on start. A restart would be required to change the token. Plaintext only on both ends, no hashing. It should exist to help prevent accidents more than anything else. I will probably add a delay on failure to mitigate brute-force, but no other features. The really hard part is adding support to clients, and perhaps in a few years distro's can start shipping with default or randomized auth tokens. Open to feedback. Thanks! -Dormando -- --- You received this message because you are subscribed to the Google Groups "memcached" group. To unsubscribe from this group and stop receiving emails from it, send an email to memcached+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
I don't really know. I don't have a mac so I don't know why saslpasswd2 doesn't work. If it gives you any output when it doesn't work (with the -f argument), please share it. You can also strace the command to see if there are any obvious errors before it exits. There must be some reason why it's not writing the file; it worked fine for me immediately. how did you install sasl on your machine? or did it come with it? On Tue, 10 Apr 2018, Om Kale wrote: > Hi Dormando, > I finally figured it out the issue from the above thread itself. > The small change in steps as shown below work on my MAC machine: > > OKALE-M-33H5:memcached-1.5.7 okale$ echo "testuser@OKALE-M-33H5:testpass" > > /tmp/memcached-sasl-db > OKALE-M-33H5:memcached-1.5.7 okale$ SASL_CONF_PATH="/Users/okale/sasl" > memcached -v -S > Reading configuration from: > Initialized SASL. > mech: ``PLAIN'' with 26 bytes of data > sasl result code: 0 > > > Client Side: > OKALE-M-33H5:mycode okale$ ./testsasl testuser testpass 127.0.0.1 > Get/Set success! > > I observe two things here: > 1. The saslpasswd2 doesn't create the memcached-sasl-db file for me, I > instead used the echo command listed above. > 2. Now memcached appends mylocalhost-mac name i.e. @OKALE-M-33H5 to the > username when I run the client. (Not sure why this is the case) > It would be great if you could guide me as to whether there is a specific > reason to it and will I be able to perform the authentication without saving > the username in this format in my sasl db file. > > > Appreciate all the help! > > Thanks and Regards,Om Kale > > > > On Tue, Apr 10, 2018 at 10:07 PM, Om Kale <omkal...@gmail.com> wrote: > Hi Dormando, > Thanks for your guidance. Meanwhile, reading through the memcached > email chain, I see someone else also observed something similar but there > was no solution. The question is how do I create the memcahed-sasl-pwdb > file and populate it with username and password as the saslpasswd2 > doesnt seem to be doing it in this case. > > https://groups.google.com/d/msg/memcached/mtzcFVYahZo/ZGrX6i5FWsUJ > > > > Regards, > Om Kale > > > > On Tuesday, April 10, 2018 at 7:07:58 PM UTC-7, Om Kale wrote: > Yes, that is correct. But if this change is done, then the entry in > memcached.conf for sasldb_path should also change to sasldb_path: > /tmp/memcached-sasl-pwdb. > However, if this change is made still it fails. > > OKALE-M-33H5:sasl okale$ cat memcached.conf > mech_list: plain > log_level: 5 > sasldb_path: /tmp/memcached-sasl-pwdb > OKALE-M-33H5:sasl okale$ echo testpass | saslpasswd2 -f > /tmp/memcached-sasl-pwdb -a memcached -c -p testuser > OKALE-M-33H5:sasl okale$ SASL_CONF_PATH="/Users/okale/sasl" memcached -S -v > Reading configuration from: > Initialized SASL. > mech: ``PLAIN'' with 26 bytes of data > WARNING: Failed to open sasl database : No such file > or directory > SASL (severity 2): Password verification failed > sasl result code: -20 > Unknown sasl response: -20 > > > > The problem is the saslpasswd2 command is not creating the file in the > desired location. I am not able to see memcached-sasl-pwdb under /tmp > folder > OKALE-M-33H5:tmp okale$ ls -lrth > total 2920 > drwxrwxrwx 3 root wheel 96B Apr 4 14:42 boost_interprocess > drwx-- 3 okale wheel 96B Apr 4 14:43 com.apple.launchd.PJzhBv7YpC > drwx-- 3 okale wheel 96B Apr 4 14:43 com.apple.launchd.KfTcHnvIT3 > drwx-- 3 okale wheel 96B Apr 4 14:43 com.apple.launchd.ha1KS1S42u > drwx-- 4 okale wheel 128B Apr 6 15:44 com.apple.installermg8f7zLr > -rw-r--r-- 1 root wheel 111B Apr 9 13:01 progress.log > drwx-- 2 okale wheel 64B Apr 9 13:53 KSDownloadAction.uWLwKCAAOF > drwx-- 2 okale wheel 64B Apr 9 13:53 > KSOutOfProcessFetcher.3Esze3adI3 > -rw-r--r-- 1 root wheel 510B Apr 10 11:36 top.out > -rw-r--r-- 1 root wheel 0B Apr 10 12:02 adobesmuoutpk1EMzc > -rw-r--r-- 1 root wheel 0B Apr 10 12:02 adobesmuoutpmWzWeW > -rw-r--r-- 1 root wheel 0B Apr 10 12:02 adobesmuoutpzm2q3Y > -rw-r--r-- 1 root wheel 0B Apr 10 12:02 adobesmuoutpBo8m4d > -rw-rw-rw-@ 1 okale staff 1.0M Apr 10 16:12 libevent-2.1.8-stable.tar.gz > drwxr-xr-x@ 165 okale wheel 5.2K Apr 10 16:19 libevent-2.1.8-stable > -rw-rw-rw-@ 1 okale staff 447K Apr 10 16:21 memcached-1.5.7.tar.gz > srwxr-xr-x 1 okale wheel 0B Apr 10 17:25 SIP-Main > srwxr-xr-x 1 okale wheel 0B Apr 10 17:25 SIP-MsgQ > drwxr-xr-x@ 148 okale wheel 4.6K Apr 10 18:33 memcached-1.5.7 > OKALE-M-33H5:tmp okale$ > &
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
Hey, What is the exact output from saslpasswd2 when you run it? On Tue, 10 Apr 2018, Om Kale wrote: > Hi Dormando, > Thanks for your guidance. Meanwhile, reading through the memcached email > chain, I see someone else also observed something similar but there was no > solution. The question is how do I create the memcahed-sasl-pwdb file and > populate it with username and password as the saslpasswd2 doesnt seem to be > doing it in this case. > > https://groups.google.com/d/msg/memcached/mtzcFVYahZo/ZGrX6i5FWsUJ > > > > Regards, > Om Kale > > > > On Tuesday, April 10, 2018 at 7:07:58 PM UTC-7, Om Kale wrote: > Yes, that is correct. But if this change is done, then the entry in > memcached.conf for sasldb_path should also change to sasldb_path: > /tmp/memcached-sasl-pwdb. > However, if this change is made still it fails. > > OKALE-M-33H5:sasl okale$ cat memcached.conf > mech_list: plain > log_level: 5 > sasldb_path: /tmp/memcached-sasl-pwdb > OKALE-M-33H5:sasl okale$ echo testpass | saslpasswd2 -f > /tmp/memcached-sasl-pwdb -a memcached -c -p testuser > OKALE-M-33H5:sasl okale$ SASL_CONF_PATH="/Users/okale/sasl" memcached -S -v > Reading configuration from: > Initialized SASL. > mech: ``PLAIN'' with 26 bytes of data > WARNING: Failed to open sasl database : No such file > or directory > SASL (severity 2): Password verification failed > sasl result code: -20 > Unknown sasl response: -20 > > > > The problem is the saslpasswd2 command is not creating the file in the > desired location. I am not able to see memcached-sasl-pwdb under /tmp folder > OKALE-M-33H5:tmp okale$ ls -lrth > total 2920 > drwxrwxrwx 3 root wheel 96B Apr 4 14:42 boost_interprocess > drwx-- 3 okale wheel 96B Apr 4 14:43 com.apple.launchd.PJzhBv7YpC > drwx-- 3 okale wheel 96B Apr 4 14:43 com.apple.launchd.KfTcHnvIT3 > drwx-- 3 okale wheel 96B Apr 4 14:43 com.apple.launchd.ha1KS1S42u > drwx-- 4 okale wheel 128B Apr 6 15:44 com.apple.installermg8f7zLr > -rw-r--r-- 1 root wheel 111B Apr 9 13:01 progress.log > drwx-- 2 okale wheel 64B Apr 9 13:53 KSDownloadAction.uWLwKCAAOF > drwx-- 2 okale wheel 64B Apr 9 13:53 > KSOutOfProcessFetcher.3Esze3adI3 > -rw-r--r-- 1 root wheel 510B Apr 10 11:36 top.out > -rw-r--r-- 1 root wheel 0B Apr 10 12:02 adobesmuoutpk1EMzc > -rw-r--r-- 1 root wheel 0B Apr 10 12:02 adobesmuoutpmWzWeW > -rw-r--r-- 1 root wheel 0B Apr 10 12:02 adobesmuoutpzm2q3Y > -rw-r--r-- 1 root wheel 0B Apr 10 12:02 adobesmuoutpBo8m4d > -rw-rw-rw-@ 1 okale staff 1.0M Apr 10 16:12 libevent-2.1.8-stable.tar.gz > drwxr-xr-x@ 165 okale wheel 5.2K Apr 10 16:19 libevent-2.1.8-stable > -rw-rw-rw-@ 1 okale staff 447K Apr 10 16:21 memcached-1.5.7.tar.gz > srwxr-xr-x 1 okale wheel 0B Apr 10 17:25 SIP-Main > srwxr-xr-x 1 okale wheel 0B Apr 10 17:25 SIP-MsgQ > drwxr-xr-x@ 148 okale wheel 4.6K Apr 10 18:33 memcached-1.5.7 > OKALE-M-33H5:tmp okale$ > > Thanks and Regards,Om Kale > > > On Tue, Apr 10, 2018 at 6:41 PM, dormando <dorma...@rydia.net> wrote: > Change: > > $ echo testpass | saslpasswd2 -f /Users/okale/sasl/memcached-sasl-pwdb > -a > memcached -c -p testuser > > To: > > $ echo testpass | saslpasswd2 -f /tmp/memcached-sasl-pwdb -a > memcached -c -p testuser > > SASL_CONF_PATH points to where memcached.conf is. memcached.conf points > to > memcached-sasl-pwdb via the sasldb_path: line. > > On Tue, 10 Apr 2018, Om Kale wrote: > > > Hi Dormando, > > Thanks for the help. I tried the steps you mentioned but end up > getting similar error. However, the error is slightly different this > time. > > Why is it still pointing to '/tmp/memcached-sasl-db' when the > SASL_CONF_PATH specifies the location of the db file. > > > > OKALE-M-33H5:sasl okale$ pwd > > /Users/okale/sasl > > OKALE-M-33H5:sasl okale$ cat memcached.conf > > mech_list: plain > > log_level: 5 > > sasldb_path: /Users/okale/sasl/memcached-sasl-pwdb > > OKALE-M-33H5:sasl okale$ echo testpass | saslpasswd2 -f > /Users/okale/sasl/memcached-sasl-pwdb -a memcached -c -p testuser > > OKALE-M-33H5:sasl okale$ SASL_CONF_PATH="/Users/okale/sasl" memcached > -S -v > > Reading configuration from: > > Initialized SASL. > > mech: ``PLAIN'' with 26 bytes of data > > WARNING: Failed to open sasl database : No > such file or directory > > SASL (severity 2)
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
Change: $ echo testpass | saslpasswd2 -f /Users/okale/sasl/memcached-sasl-pwdb -a memcached -c -p testuser To: $ echo testpass | saslpasswd2 -f /tmp/memcached-sasl-pwdb -a memcached -c -p testuser SASL_CONF_PATH points to where memcached.conf is. memcached.conf points to memcached-sasl-pwdb via the sasldb_path: line. On Tue, 10 Apr 2018, Om Kale wrote: > Hi Dormando, > Thanks for the help. I tried the steps you mentioned but end up getting > similar error. However, the error is slightly different this time. > Why is it still pointing to '/tmp/memcached-sasl-db' when the SASL_CONF_PATH > specifies the location of the db file. > > OKALE-M-33H5:sasl okale$ pwd > /Users/okale/sasl > OKALE-M-33H5:sasl okale$ cat memcached.conf > mech_list: plain > log_level: 5 > sasldb_path: /Users/okale/sasl/memcached-sasl-pwdb > OKALE-M-33H5:sasl okale$ echo testpass | saslpasswd2 -f > /Users/okale/sasl/memcached-sasl-pwdb -a memcached -c -p testuser > OKALE-M-33H5:sasl okale$ SASL_CONF_PATH="/Users/okale/sasl" memcached -S -v > Reading configuration from: > Initialized SASL. > mech: ``PLAIN'' with 26 bytes of data > WARNING: Failed to open sasl database : No such file > or directory > SASL (severity 2): Password verification failed > sasl result code: -20 > Unknown sasl response: -20 > ^CSignal handled: Interrupt: 2. > OKALE-M-33H5:sasl okale$ > OKALE-M-33H5:sasl okale$ > OKALE-M-33H5:sasl okale$ > > > On client side: > OKALE-M-33H5:mycode okale$ ./testsasl testuser testpass 127.0.0.1 > Set failed: FAILED TO SEND AUTHENTICATION TO SERVER > OKALE-M-33H5:mycode okale$ ./testsasl testuser testpass localhost > Set failed: AUTHENTICATION FAILURE > OKALE-M-33H5:mycode okale$ > > > Any idea why? > > > > Thanks and Regards,Om Kale > > On Tue, Apr 10, 2018 at 4:38 PM, dormando <dorma...@rydia.net> wrote: > yes and yes. > > mkdir sasl > cd sasl > then created memcached.conf > I did not create memcached-sasl-pwdb manually. saslpasswd2 made that for > me after I passed the -f argument. > > On Tue, 10 Apr 2018, Om Kale wrote: > > > Hi Dormando, > > Thanks for the update. I will try this out now. But before this I had > one more quick question. > > Did you create the sasl folder and memcached.conf manually inside > /home/dormando/ ? > > > > > > Thanks and Regards,Om Kale > > > > > > On Tue, Apr 10, 2018 at 3:38 PM, dormando <dorma...@rydia.net> wrote: > > Hey, > > > > Was able to authenticate with your tool: > > > > $ pwd > > /home/dormando/sasl > > $ cat memcached.conf > > mech_list: plain > > log_level: 5 > > sasldb_path: /home/dormando/sasl/memcached-sasl-pwdb > > $ echo testpass | saslpasswd2 -f > > /home/dormando/sasl/memcached-sasl-pwdb -a memcached -c -p > testuser > > $ SASL_CONF_PATH="/home/dormando/sasl" memcached -S -v > > INFO: MEMCACHED_SASL_PWDB not specified. Internal passwd > database disabled > > Initialized SASL. > > $ ./testsasl testuser testpass 127.0.0.1 > > Get/Set success! > > > > Just add the "-f /path/to/sasl-pwdb" to saslpasswd2 and let it > create the > > entry for you. Your manual passwd DB isn't valid. > > > > Without the -f the tool was exiting with "Generic failure" > (should've > > asked you what the exit code was earlier, sorry). Strace'ing it > showed it > > was trying to open /etc/sasl and write a new file, but I wasn't > running as > > root. > > > > On Tue, 10 Apr 2018, Om Kale wrote: > > > > > Hey Dormando, > > > Today I tried reinstalling memcached from scratch and > followed the procedure in the wiki > > > and the points you mentiibed however same issue of 'no secret > in database' is still > > > observed. > > > > > > In addition, did the following steps but still no success. > > > > https://stackoverflow.com/questions/12919032/can-i-set-username-and-password-on-memcach > > > ed-like-mysql > > > > > > > > > Thanks and Regards,Om Kale > > > > > > On Mon, Apr
Re: Regarding setting up SASL with memcached server and getting a memcached client to associate with server.
yes and yes. mkdir sasl cd sasl then created memcached.conf I did not create memcached-sasl-pwdb manually. saslpasswd2 made that for me after I passed the -f argument. On Tue, 10 Apr 2018, Om Kale wrote: > Hi Dormando, > Thanks for the update. I will try this out now. But before this I had one > more quick question. > Did you create the sasl folder and memcached.conf manually inside > /home/dormando/ ? > > > Thanks and Regards,Om Kale > > > On Tue, Apr 10, 2018 at 3:38 PM, dormando <dorma...@rydia.net> wrote: > Hey, > > Was able to authenticate with your tool: > > $ pwd > /home/dormando/sasl > $ cat memcached.conf > mech_list: plain > log_level: 5 > sasldb_path: /home/dormando/sasl/memcached-sasl-pwdb > $ echo testpass | saslpasswd2 -f > /home/dormando/sasl/memcached-sasl-pwdb -a memcached -c -p testuser > $ SASL_CONF_PATH="/home/dormando/sasl" memcached -S -v > INFO: MEMCACHED_SASL_PWDB not specified. Internal passwd database > disabled > Initialized SASL. > $ ./testsasl testuser testpass 127.0.0.1 > Get/Set success! > > Just add the "-f /path/to/sasl-pwdb" to saslpasswd2 and let it create > the > entry for you. Your manual passwd DB isn't valid. > > Without the -f the tool was exiting with "Generic failure" (should've > asked you what the exit code was earlier, sorry). Strace'ing it showed > it > was trying to open /etc/sasl and write a new file, but I wasn't running > as > root. > > On Tue, 10 Apr 2018, Om Kale wrote: > > > Hey Dormando, > > Today I tried reinstalling memcached from scratch and followed the > procedure in the wiki > > and the points you mentiibed however same issue of 'no secret in > database' is still > > observed. > > > > In addition, did the following steps but still no success. > > > https://stackoverflow.com/questions/12919032/can-i-set-username-and-password-on-memcach > > ed-like-mysql > > > > > > Thanks and Regards,Om Kale > > > > On Mon, Apr 9, 2018 at 11:58 PM, dormando <dorma...@rydia.net> wrote: > > Sorry, ran out of time today. will try for earlier tomorrow > > > > On Mon, 9 Apr 2018, Om Kale wrote: > > > > > Hi Dormando,I was just curious to know whether you were able > to reproduce > > the above > > > mentioned issue? > > > > > > Thanks and Regards,Om Kale > > > > > > On Mon, Apr 9, 2018 at 12:53 PM, Om Kale <omkal...@gmail.com> > wrote: > > > Yes, that will be very helpful Dormando. I agree, might > be missing > > > something. > > > The points where I think I might be going wrong are as > follows: > > > > > > 1. The exact location and contents of memcached.conf and the > sasl db file > > - > > > memcached-sasl-pwdb (and the interaction between the two). > > > As per my understanding, SASL_CONF_PATH, tells the memcached > server where > > to read > > > the file from and then the line sasldb_path in the conf file > tells the > > server > > > where to get the sasl db file for username:password > authentication. I feel > > this > > > linkage is not happening correctly in my case. > > > > > > 2. Is the sasl db file generated on its own when I run the > server? If yes, > > do we > > > need to add command line parameters while running the > memcached server for > > this to > > > happen. (FYI: I have configured --enable-sasl-pwdb while > running > > configure) > > > > > > The main problem I am facing right now is memcached-sasl-pwdb > is not > > getting > > > created and populated on its own. > > > Please do let me know the outcome once you try to reproduce > it. I am > > cuurently > > > using MAC-OS high Sierra. > > > > > > Thanks and Regards,Om Kale > > > > > > > > > On Mon, Apr 9, 20
