Re: [Sks-devel] SKS scaling configuration
On 2/25/19 6:37 PM, Todd Fleisher wrote:
>> On Feb 23, 2019, at 8:35 PM, Jeremy T. Bouse
>> mailto:jeremy.bo...@undergrid.net>> wrote:
>>
>> I didn't have as many locations configured as you show in your example
>> but it looked like you were defining the map but I didn't see it being
>> used in any of your location blocks unless I'm missing something.
>> Shouldn't you be using $upstream_server in your proxy_pass configuration?
>>
> I think you’re on to something here. I just tried commenting out the
> other servers from the upstream sks_servers_primary block and am still
> seeing stats queries hitting the commented out servers.
>
> Kristian - could you please double check the configuration snippets you
> provided to me last year and see if something was missing related to this?
don't recall the specifics of the snippets I sent over, but the above is
correct, you do map definition at
map $arg_op $upstream_server {
"stats" sks_servers_primary;
default sks_servers;
}
which is used for
location /pks/lookup {
proxy_cache backcache;
proxy_ignore_headers Cache-Control "Expires";
#proxy_cache_bypass $http_cache_control;
add_header X-Proxy-Cache $upstream_cache_status;
proxy_cache_valid any 10m;
proxy_pass http://$upstream_server;
proxy_pass_header Server;
add_header Via "1.1 keys2.kfwebs.net";
proxy_ignore_client_abort on;
}
}
--
Kristian Fiskerstrand
Blog: https://blog.sumptuouscapital.com
Twitter: @krifisk
Public OpenPGP keyblock at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
Corruptissima re publica plurimæ leges
The greater the degeneration of the republic, the more of its laws
signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel
[Sks-devel] sks.domainmail.net (new!) is seeking peers.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hello! I'm seeking peers for a new SKS keyserver in Atlanta, GA, USA at sks.domainmail.net. DomainMail, LLC provides Mailman mailing list services. The server is running sks v1.1.6-4 (Debian/Stretch), it was "primed" from keyserver.mattrude.com on 4-Mar-2019 with a total of 5443075 keys. My membership line: sks.domainmail.net 11370 # Jim Popovitch 0x3F1C1EF2E6019EAC646CE45227155EB4C45A2705 Please contact me for any operational issues. Thanks!! - -Jim P. -BEGIN PGP SIGNATURE- iQIzBAEBCgAdFiEECPbAhaBWEfiXj/kxdRlcPb+1fkUFAlx98x4ACgkQdRlcPb+1 fkWIbA/8D+rZz+xL9gIEjxBMl0hcoKU2bJ0haJCYy0zw4UDxlbO2CZ7DVtKymKSG aFgaLakBJQfBVZdy6c+fNiu1/tOUav4F9Ttuc8ZJ45r6ZoHuT+uSibzEmoJfpswq L0mvvHwcQI3iFZTrQPdQrokRGZL4hJdb66lqPeiB5/NUKqPM1vFDm4oSBGsI4c2r JmVCfbE1dW/J2IjV5fd0zfMrVnk6k5is69IUPvoUaXY8nGQ5O/FkjDl30WohhGZM MBB51kZPhGSNRZQ0lr7/t/bspH0TDCFrBAaXEEJRzx7EZWXeTInkh5SReCaCbm3h BYehauWwi1fat+vTLi1BAim5ANitUrteNyhBq1zGU20H7VehNcNMak9HFg27MpW7 Wj26DKRnrxHa+4vt7tWZ8zBOBSP1iLzd/OhyktSphvojsgs2aVXu+16/+xnUMqsd omCFmZstnZc4tQCTEcBx9CH0usCokkbJSzVfxD1m5rN34ocCr0Dc2XoOvMIp5bli fj61FjDepDKb4gDZnUpWgHHKuCh9fbP72pcMmKMyI638mQjBYcTDOwm6DMVob/Oi yqzoqQ1oRiKIHS5fSjKTHnSP3s+vZtQk69UhnQvZBWRq3SMBnvKPWDjNItuWjAR+ 8Tg3rNKBIsWO4tiGKOhl7X+nhwMZ1GWOd1lJf1Z8pF1t4N33Wvc= =9HvN -END PGP SIGNATURE- ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
Re: [Sks-devel] SKS Memory pattern anomaly
So I have all my nodes synced with around 5445343 keys after disabling all external peering and letting the 4 nodes get in sync. I then added a single external peer and then my SKS DB process goes into a funky state once it begins peering externally. I installed strace and when I ran a 'strace -q -c' against the 'sks db' process on the 3 secondary nodes it comes out with something along the lines of: % time seconds usecs/call calls errors syscall -- --- --- - - 100.00 0.004000 500 8 select 0.00 0.00 0 5 read 0.00 0.00 0 7 write 0.00 0.00 0 5 close 0.00 0.00 0 2 stat 0.00 0.00 0 10 10 lseek 0.00 0.00 0 3 brk 0.00 0.00 0 16 alarm 0.00 0.00 0 5 accept -- --- --- - - 100.00 0.004000 61 10 total These are running what appears to be normal with under 10% CPU and 10% MEM according to ps.. My primary node on the other hand is another story entirely... For almost a half hour now it has shown using 80% MEM and CPU % varies but strace shows a totally different pattern. % time seconds usecs/call calls errors syscall -- --- --- - - 99.77 0.026483 9 2865 pread64 0.23 0.60 0 2563 pwrite64 0.00 0.00 0 141 ftruncate -- --- --- - - 100.00 0.026543 5569 total I don't know enough about the internal operations to know exactly what's going on but given the high memory usage and the fact my node is running 1 CPU core at 100% idle while the other is in nearly 100% io wait state that it's caught in some sore of loop unable to get out of it. If I look under /var/lib/sks/DB I've got multiple 100MB log files starting to build up but no other files appears to have their timestamps updating showing any sign of modification/update except the __db.00[123] files. Anyone have any thoughts for next steps? On 3/5/2019 1:09 AM, Jeremy T. Bouse wrote: > Has anyone else been monitoring the memory pattern for SKS and > noticed an exceedingly high memory usage pattern? My secondary nodes are > generally showing < 11% of the instance memory used but for some reason > I'm seeing my primary node using nearly 100% of memory, and CPU for that > matter. My primary node is the only one peering outside my network and > has a limited number of peers while the secondary nodes only peer with > themselves and the primary. I've placed the short-circuit hack to NGINX > for the bad keys that have been mentioned which has shown to lower CPU > usage overall but nothing has seemed to improve the primary node. I see > my primary spend much of the time at 100% CPU and 50-90% Memory while > it's in recon mode and it only appears to dip down when it recalculates > it's stats. > > > ___ > Sks-devel mailing list > Sks-devel@nongnu.org > https://lists.nongnu.org/mailman/listinfo/sks-devel ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
