Re: Last day for FOSDEM submission

[Wesley Peng via perl6-users]

Replacing Bash scripts with Raku? That’s an interesting thing. Can Linux be 
shipped with Raku by default?




hello people,

I just discovered this this morning:

https://www.reddit.com/r/rakulang/comments/rrcp4c/steal_these_ideas_for_raku_fosdem_talks/

I don't remember if there was a previous annoucement in this list but
it's still possible to jump in.

I just submitted one on "Replacing Bash scripts with Raku"

* features and way of doing things i love from zsh
* the raku way [1]
* getting started with raku ecosystem (raku.land, zef, Ddt, fez, rakudoc, 
prove6, ...)

gather/take=while read, Seq vs Pipes, ...) and ideas of modules

1: the raku way are things like

*sh        │ raku
───┼─
|          │ Seq.map
xargs -P  │ .race
while read │ gather/take

At some point it could become an Intro section on the Raku documentation
site.

If a native/fluent english speaker want to take over it: i'll be more
than happy to co-author the talk as well (i really feel like i'm a disaster as
speaker).

Regards
marc

Re: Last day for FOSDEM submission

[Wesley Peng via perl6-users]

Replacing Bash scripts with Raku? That’s an interesting thing. Can Linux be 
shipped with Raku by default?




hello people,

I just discovered this this morning:

https://www.reddit.com/r/rakulang/comments/rrcp4c/steal_these_ideas_for_raku_fosdem_talks/

I don't remember if there was a previous annoucement in this list but
it's still possible to jump in.

I just submitted one on "Replacing Bash scripts with Raku"

* features and way of doing things i love from zsh
* the raku way [1]
* getting started with raku ecosystem (raku.land, zef, Ddt, fez, rakudoc, 
prove6, ...)

gather/take=while read, Seq vs Pipes, ...) and ideas of modules

1: the raku way are things like

*sh        │ raku
───┼─
|          │ Seq.map
xargs -P  │ .race
while read │ gather/take

At some point it could become an Intro section on the Raku documentation
site.

If a native/fluent english speaker want to take over it: i'll be more
than happy to co-author the talk as well (i really feel like i'm a disaster as
speaker).

Regards
marc

Re: [ANNOUNCE] Apache BookKeeper 4.13.0 released

[Wesley Peng]

great to hear that. thank you.


On 26.02.2021 04:00, Andrey Yegorov wrote:

The Apache BookKeeper team is proud to announce Apache BookKeeper
version
4.13.0.

Apache BookKeeper is a scalable, fault-tolerant, and low-latency
storage service optimized for
real-time workloads. It has been used as a fundamental service to
build reliable services.
It is also the log segment store for Apache DistributedLog and the
message store for Apache Pulsar.

This is the 25th release of the Apache BookKeeper.

Release 4.13.0 improves the reliability of the Stream Storage, brings
additional configuration options for the Stream Storage and Prometheus
HTTP Server, fixes multiple bugs, and brings critical dependencies
up-to-date.

For BookKeeper release details and downloads, visit:

https://bookkeeper.apache.org/releases/

BookKeeper 4.13.0 Release Notes are at:

https://bookkeeper.apache.org/docs/4.13.0/overview/releaseNotes/

We would like to thank the contributors that made the release
possible.

Regards,
The BookKeeper Team

modperl vs fastcgi

[Wesley Peng]

Do you anybody have a comparison for modperl vs fastcgi?
For both the performance and convenience of development/deployment.

Thank you.

Re: [CFP] ApacheCon Asia about API/mrico service

[Wesley Peng]

nice to know it. thanks.

On 23.02.2021 09:02, Ming Wen wrote:

hello, dev,
The ApacheCon Asia started CFP[1],
and everyone is welcome to submit topics related to Apache APISIX.
Both Mandarin and English are fine. Looking forward to it.

[1] acasia2021.jamhosted.net 

Thanks,
Ming Wen, Apache APISIX PMC Chair
Twitter: _WenMing

Re: master deps error【20210219】

[Wesley Peng]

QQ MIME sucks. please use a standard email product such as Gmail.


On Fri, Feb 19, 2021, at 9:37 AM, 15717900 wrote:
> hello,
>I use Tencent virtual machine to compile the master 
> version.
>the error message is
> 
> 
> ---this is err 
> begin---
>inflating: luasocket-3.0-rc1/test/udpconnectsrvr.lua
>  inflating: luasocket-3.0-rc1/test/unixclnt.lua
>  inflating: luasocket-3.0-rc1/test/unixsrvr.lua
>  inflating: luasocket-3.0-rc1/test/upload.html
>  inflating: luasocket-3.0-rc1/test/urltest.lua
>  inflating: luasocket-3.0-rc1/test/utestclnt.lua
>  inflating: luasocket-3.0-rc1/test/utestsrvr.lua
>  inflating: luasocket-3.0-rc1/win32.cmd
> gcc -O2 -fPIC -I/usr/include -c src/mime.c -o src/mime.o 
> -DLUA_COMPAT_APIINTCASTS -DLUASOCKET_DEBUG 
> -DLUASOCKET_API=__attribute__((visibility("default"))) -D   
>
>
> 
> UNIX_API=__attribute__((visibility("default"))) 
> -DMIME_API=__attribute__((visibility("default")))
> src/mime.c:7:17: fatal error: lua.h: No such file or directory
> #include "lua.h"
> ^
> compilation terminated.
> 
> 
> Error: Failed installing dependency: 
> https://luarocks.org/luasec-0.9-1.src.rock - Failed installing dependency: 
> https://luarocks.org/luasocket-3.0rc1-2.src.r;
>
>
>ock - Build error: Failed 
> compiling object src/mime.o
> make: *** [deps] Error 1
> ---this is err end---
> （1）openresty -v（Both versions have the same error message）
> nginx version: openresty/1.19.3.1
> nginx version: openresty/1.17.8.2
> openssl-1.1.1g
> 
> （2）lua -v
> Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio
> 
> 
> 
> （3）luarocks
> LuaRocks 2.3.0, a module deployment system for Lua
> 
> 
> 
> （4）uname -a
> Linux VM-0-14-centos 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 
> UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
> 
> 
> 
> Looking forward to answer，thanks

Re: Moving ExecCGI to mod_perl - performance and custom 'modules' [EXT]

[Wesley Peng]

what's DBI sharing? do you mean Apache::DBI?
Does perl has Java similar DB connection pool?

Thanks.

On Mon, Feb 8, 2021, at 4:21 AM, James Smith wrote:
> DBI sharing doesn't really gain you much - and can actually lead you into a 
> whole world of pain. It isn't actually worth turning it on at all.
> 
> We use dedicated DB caching in the cases where we benefit from it as and when 
> you need it (low level caching database)...
> 
> Although milage may vary - our problem was DB servers with 30 or 40 databases 
> on them being connected from a number of approximately 50-100 child apaches, 
> meant we ended up blowing up the connections to the database server really 
> quickly...
> 
> 
> -Original Message-
> From: Vincent Veyron  
> Sent: 07 February 2021 19:06
> To: Steven Haigh 
> Cc: James Smith ; modperl@perl.apache.org
> Subject: Re: Moving ExecCGI to mod_perl - performance and custom 'modules' 
> [EXT]
> 
> On Sun, 07 Feb 2021 23:58:17 +1100
> Steven Haigh  wrote:
> > 
> > I haven't gotten into the preload or DBI sharing yet - as that'll end 
> > up needing a bit of a rewrite of code to take advantage of. I'd be 
> > open to suggestions here from those who have done it in the past to 
> > save me going down some dead ends :D
> 
> I use mod_perl handlers, so not sure how it mixes with PerlRegistry, but you 
> probably want to have a look at connect_cached
> 
> 
> -- 
> 
> Bien à vous, Vincent Veyron 
> 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__compta.libremen.com=DwIFAw=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo=oH2yp0ge1ecj4oDX0XM7vQ=C0OcuGbNbfxaSa8ASgV3uFXzejn7MpjIUH1aP1RbiyU=GPr8VuKQ3rZCzCPwggyAHdCOojK6ZThmShKk0Jb3maI=
> Logiciel libre de comptabilité générale en partie double
> 
> 
> 
> --
> The Wellcome Sanger Institute is operated by Genome Research
> Limited, a charity registered in England with number 1021457 and a
> company registered in England with number 2742969, whose registered
> office is 215 Euston Road, London, NW1 2BE.
> 

Re: Moving ExecCGI to mod_perl - performance and custom'modules'[EXT]

[Wesley Peng]

If you can take time to rewrite all codes with modPerl handlers, that will 
improve performance a lot.

On Sun, Feb 7, 2021, at 9:14 PM, Steven Haigh wrote:
> In fact, I just realised that 'ab' test is rather restrictive So here's a 
> bit more of an extended test:
> 
> # ab -k -n 1000 -c 32
> 
> Apache + ExecCGI:
> Requests per second:14.26 [#/sec] (mean)
> Time per request:   2244.181 [ms] (mean)
> Time per request:   70.131 [ms] (mean, across all concurrent requests)
> 
> Apache + mod_perl (ModPerl::PerlRegistry): 
> Requests per second: 132.14 [#/sec] (mean)
> Time per request:   242.175 [ms] (mean)
> Time per request:   7.568 [ms] (mean, across all concurrent requests)
> 
> Interestingly, without Keepalives, the story is much the same:
> 
> # ab -n 1000 -c 32
> 
> Apache + ExecCGI:
> Requests per second:14.15 [#/sec] (mean)
> Time per request:   2260.875 [ms] (mean)
> Time per request:   70.652 [ms] (mean, across all concurrent requests)
> 
> Apache + mod_perl (ModPerl::PerlRegistry): 
> Requests per second:154.48 [#/sec] (mean)
> Time per request:   207.140 [ms] (mean)
> Time per request:   6.473 [ms] (mean, across all concurrent requests)
> 
> Running some benchmarks across various parts of my site made me realise I 
> also had some RewriteRules in the apache config that still had H=cgi-script - 
> changed those to H=perl-script and saw similar improvements:
> 
> ExecCGI - Requests per second:11.84 [#/sec] (mean)
> mod_perl - Requests per second:130.97 [#/sec] (mean)
> 
> That's quite some gains for a days work.
> 
> --
> Steven Haigh  net...@crc.id.au  https://www.crc.id.au
> 
> On Sun, Feb 7, 2021 at 23:58, Steven Haigh  wrote:
>> Interestingly, I did get things working with ModPerl::PerlRegistry.
>> 
>> What I couldn't find *anywhere* is that the data I was loading in Template 
>> Toolkit was included in the file in the __DATA__ area - which causes 
>> mod_perl to fall over!
>> 
>> The only way I managed to find this was the following error in the *system* 
>> /var/log/httpd/error_log (didn't show up in the vhost error_log!):
>> readline() on unopened filehandle DATA at 
>> /usr/lib64/perl5/vendor_perl/Template/Provider.pm line 638.
>> 
>> Took me a LONG time to find a vague post that reading in lines from  
>> kills mod_perl. Not sure why - but I stripped all the templates out and put 
>> them in a file instead and re-wrote that bit of code, and things started 
>> working.
>> 
>> I had to fix a few lib path issues, but after getting my head around that, 
>> most things seem to work as before - however I don't notice much of an 
>> improvement in execution times, I do see this improvement using 'ab -n 100 
>> -c32':
>> 
>> Apache + ExecCGI: Requests per second:13.50 [#/sec] (mean)
>> Apache + mod_perl: Requests per second:59.81 [#/sec] (mean)
>> 
>> This is obviously a good thing.
>> 
>> I haven't gotten into the preload or DBI sharing yet - as that'll end up 
>> needing a bit of a rewrite of code to take advantage of. I'd be open to 
>> suggestions here from those who have done it in the past to save me going 
>> down some dead ends :D
>> 
>> --
>> Steven Haigh  net...@crc.id.au  https://www.crc.id.au
>> 
>> On Sun, Feb 7, 2021 at 12:49, James Smith  wrote:
>>> As welsey said – try Registry, that was the standard way of using mod_perl 
>>> to cache perl in the server  – but your problem might be due to the note in 
>>> PerlRun…
>>> 
>>> https://perl.apache.org/docs/2.0/api/ModPerl/PerlRun.html#Description
>>> META: document that for now we don't chdir() into the script's dir, because 
>>> it affects the whole process under threads. `ModPerl::PerlRunPrefork 
>>> ` should 
>>> be used by those who run only under prefork MPM.
>>> {tbh most people don’t use mod perl under threads anyway as there isn’t 
>>> really a gain from using them}
>>> 
>>> It suggests you use ModPerl/PerlRunPrefork – as this does an additional 
>>> step to cd to the script directory – which might be your issue….

>>>  

>>> *From:* Steven Haigh  
>>> *Sent:* 07 February 2021 01:00
>>> *To:* modperl@perl.apache.org
>>> *Subject:* Moving ExecCGI to mod_perl - performance and custom 'modules' 
>>> [EXT]

>>>  

>>> Hi all,

>>>  

>>> So for many years I've been slack and writing perl scripts to do various 
>>> things - but never needed more than the normal apache +ExecCGI and Template 
>>> Toolkit.

>>>  

>>> One of my sites has become a bit more popular, so I'd like to spend a bit 
>>> of time on performance. Currently, I'm seeing ~300-400ms of what I believe 
>>> to be execution time of the script loading, running, and then blatting its 
>>> output to STDOUT and the browser can go do its thing. 

>>>  

>>> I believe most of the delay would be to do with loading perl, its modules 
>>> etc etc

>>>  

>>> I know that the current trend would be to re-write the entire site in a 
>>> more modern, 

Re: Moving ExecCGI to mod_perl - performance and custom 'modules'

[Wesley Peng]

Is the url in your signature the website? It looks a beautiful UI design.
And you can try this method:
https://metacpan.org/pod/ModPerl::Registry

Thanks.

On Sun, Feb 7, 2021, at 8:59 AM, Steven Haigh wrote:
> Hi all,
> 
> So for many years I've been slack and writing perl scripts to do various 
> things - but never needed more than the normal apache +ExecCGI and Template 
> Toolkit.
> 
> One of my sites has become a bit more popular, so I'd like to spend a bit of 
> time on performance. Currently, I'm seeing ~300-400ms of what I believe to be 
> execution time of the script loading, running, and then blatting its output 
> to STDOUT and the browser can go do its thing. 
> 
> I believe most of the delay would be to do with loading perl, its modules etc 
> etc
> 
> I know that the current trend would be to re-write the entire site in a more 
> modern, daemon based solution - and I started down the Mojolicious path - but 
> the amount of re-writing to save 1/3rd of a second seems to be excessive
> 
> Would I be correct in thinking that mod_perl would help in this case?
> 
> I did try a basic test, but I have a 'use functions' in all my scripts that 
> loads a .pm with some global vars and a lot of common subs - and for whatever 
> reason (can't find anything on Google as to why), none of the subs are 
> recognised in the main script when loaded via ModPerl::PerlRun.
> 
> So throwing it out to the list - am I on the right track? wasting my time? or 
> just a simple mistake?
> 
> --
> Steven Haigh  net...@crc.id.au  https://www.crc.id.au

Re: dimensions in a multidimensional array

[Wesley Peng]

When will we have p6’s Numpy?


On Sat, Feb 6, 2021, at 2:04 AM, Brad Gilbert wrote:
> There is a reason that you can't just ask for the dimensions of an 
> unspecified multidimensional array.
> It may be multiple dimensions.
> 
> [[1,2,3],
> [4,5,6,7,8,9,10]].shape
> 
> If it gave a result it would be something like:
> 
> (2,3|7)
> 
> On Fri, Feb 5, 2021 at 8:50 AM Theo van den Heuvel  
> wrote:
>> Hi gurus,
>> 
>> I cannot seem to find an idiomatic way to get the dimensions of a 
>> multidimensional array,
>> other than by looking at the size of the first row and column, with 
>> @m[0;*].elems and @m[*;0].elems.
>> Am I missing something in the docs?
>> 
>> Thanks,
>> 
>> -- 
>> Theo van den Heuvel
>> 

Re: dimensions in a multidimensional array

[Wesley Peng]

When will we have p6’s Numpy?


On Sat, Feb 6, 2021, at 2:04 AM, Brad Gilbert wrote:
> There is a reason that you can't just ask for the dimensions of an 
> unspecified multidimensional array.
> It may be multiple dimensions.
> 
> [[1,2,3],
> [4,5,6,7,8,9,10]].shape
> 
> If it gave a result it would be something like:
> 
> (2,3|7)
> 
> On Fri, Feb 5, 2021 at 8:50 AM Theo van den Heuvel  
> wrote:
>> Hi gurus,
>> 
>> I cannot seem to find an idiomatic way to get the dimensions of a 
>> multidimensional array,
>> other than by looking at the size of the first row and column, with 
>> @m[0;*].elems and @m[*;0].elems.
>> Am I missing something in the docs?
>> 
>> Thanks,
>> 
>> -- 
>> Theo van den Heuvel
>> 

Re: Please can you opt me out from receiving mails from you.

[Wesley Peng]

You can send an email to beginners-unsubscr...@perl.org to unsubscribe yourself 
from it.


On Fri, Feb 5, 2021, at 4:20 PM, ONYEDIKACHI NNADI wrote:
> Please can you opt my email address dikachi4...@gmail.com from receiving 
> emails from you. Thanks in anticipation for your prompt action. 

any blocked websocket client for perl?

[Wesley Peng]

I am trying to find a websocket client for perl.
But most modules on cpan is asynchronous, such as AnyEvent::WebSocket::Client.
Do you know if there is a synchronous websocket client library available?

Thanks in advance.


BTW, I tried to use the anyevent module above to access my WS service, it 
doesn't work.
When program run, it just hangs without any output.
Can you see why this happens?
The code as below:

use strict;

use JSON;

use Data::Dumper;

use AnyEvent::WebSocket::Client 0.12;

 

my $client = AnyEvent::WebSocket::Client->new;

my $url = 
'ws://localhost:8080/ws/v2/producer/persistent/public/default/my-topic';



 

$client->connect($url)->cb(sub {

 

  # make $connection an our variable rather than

  # my so that it will stick around.  Once the

  # connection falls out of scope any callbacks

  # tied to it will be destroyed.

  our $connection = eval { shift->recv };

  if($@) {

# handle error...

warn $@;

return;

  }

   

  # send a message through the websocket...

  my $payload = encode_json({'payload' => 'hello world'});

  $connection->send($payload);

   

  # recieve message from the websocket...

  $connection->on(each_message => sub {

# $connection is the same connection object

# $message isa AnyEvent::WebSocket::Message

my($connection, $message) = @_;

print Dumper decode_json($message);

  });

   

  # handle a closed connection...

#  $connection->on(finish => sub {

## $connection is the same connection object

#my($connection) = @_;

#...

#  });

 

  # close the connection (either inside or

  # outside another callback)

#  $connection->close;

 

});

 

## uncomment to enter the event loop before exiting.

## Note that calling recv on a condition variable before

## it has been triggered does not work on all event loops

AnyEvent->condvar->recv;

Re: [VOTE] Relase Apache APISIX 2.3

[Wesley Peng]

Does it have an apt repo for installation on ubuntu? thanks.

On Wed, Feb 3, 2021, at 3:00 PM, dabue wrote:
> +1 binding
>  
> I checked:
>  
> 1. Download links are valid;
> 2. Checksums and PGP signatures are valid;
> 3. License and NOTICE files are good;
> 4. All files have the necessary License header;
> 5. Build successfully on Ubuntu 16  with Openresty 1.19
> 
> update the result of  run steps
> 
> ***
> 6. 1st Run
>It reports "apisix/core/utils.lua:22: module 'resty.dns.client' not found:"
> 
> 7. check the deps, find it added apisix of master-0 to the deps  when execute 
> 'make deps'
> check the luarocks version, find the version is too older,
> update the latest version  and redo 'make deps',
> get the correct deps.
> 8. 2nd Run,   sudo make run,   it works well.
> *** 
> 
> 
> 
> 
> 
>  
> From: dabue
> Date: 2021-02-03 11:37
> To: dev
> Subject: Re: [VOTE] Relase Apache APISIX 2.3
> +1 binding
>  
> I checked:
>  
> 1. Download links are valid;
> 2. Checksums and PGP signatures are valid;
> 3. License and NOTICE files are good;
> 4. All files have the necessary License header;
> 5. Build successfully on Ubuntu 16  with Openresty 1.19
> 
> ***
> 6. 1st Run
>It reports "apisix/core/utils.lua:22: module 'resty.dns.client' not found:"
> 
> 7. append  "lua-resty-dns-client = 5.2.1-1"  to 
> rockspec/apisix-master-0.rockspec
> 
> 8. redo make deps
> 
> 9. 2nd Run,   sudo make run,   it works well.
> *** 
> 
> 
> 
> 
>  
> From: Zexuan Luo
> Date: 2021-02-01 16:50
> To: dev
> Subject: [VOTE] Relase Apache APISIX 2.3
> Hello, Community,
> This is a call for the vote to release Apache APISIX version 2.3.
>  
> Release notes:
>  
> https://github.com/apache/apisix/blob/v2.3/CHANGELOG.md#230
>  
> The release candidates:
>  
> https://dist.apache.org/repos/dist/dev/apisix/2.3/
>  
> Release Commit ID:
>  
> https://github.com/apache/apisix/commit/529433bb94d3a6a8ae67f735c25d4cafacc887a5
>  
> Keys to verify the Release Candidate:
>  
> https://dist.apache.org/repos/dist/release/apisix/KEYS
>  
> Steps to validating the release:
>  
> 1. Download the release
>  
> wget
> https://dist.apache.org/repos/dist/dev/apisix/2.3/apache-apisix-2.3-src.tgz
>  
> 2. Checksums and signatures
>  
> wget https://dist.apache.org/repos/dist/release/apisix/KEYS
>  
> wget
> https://dist.apache.org/repos/dist/dev/apisix/2.3/apache-apisix-2.3-src.tgz.asc
>  
> wget
> https://dist.apache.org/repos/dist/dev/apisix/2.3/apache-apisix-2.3-src.tgz.sha512
>  
> shasum -c apache-apisix-2.3-src.tgz.sha512
>  
> gpg --verify apache-apisix-2.3-src.tgz.asc apache-apisix-2.3-src.tgz
>  
> 3. Unzip and Check files
>  
> tar zxvf apache-apisix-2.3-src.tgz
>  
> 4. Build Apache APISIX:
>  
> https://github.com/apache/apisix/blob/master/doc/how-to-build.md#installation-via-source-
> release
>  
> The vote will be open for at least 72 hours or until necessary number of
> votes are reached.
>  
> Please vote accordingly:
>  
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
> 

A simple publish takes 9s to finish

[Wesley Peng]

Can you please see below why a simple publish take 9s to finish?
Pulsar is deployed on a two cores VPS, and I run it with `pulsar-daemon start 
standalone` mode.

Thanks.

$ time bin/pulsar-client produce my-topic --messages "hello-pullsar fifth"

15:06:12.815 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x90192576, 
L:/127.0.0.1:45148 - R:localhost/127.0.0.1:6650]] Connected to server

15:06:13.385 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar 
producer perf with config: {

  "topicName" : "my-topic",

  "producerName" : null,

  "sendTimeoutMs" : 3,

  "blockIfQueueFull" : false,

  "maxPendingMessages" : 1000,

  "maxPendingMessagesAcrossPartitions" : 5,

  "messageRoutingMode" : "RoundRobinPartition",

  "hashingScheme" : "JavaStringHash",

  "cryptoFailureAction" : "FAIL",

  "batchingMaxPublishDelayMicros" : 1000,

  "batchingPartitionSwitchFrequencyByPublishDelay" : 10,

  "batchingMaxMessages" : 1000,

  "batchingMaxBytes" : 131072,

  "batchingEnabled" : true,

  "chunkingEnabled" : false,

  "compressionType" : "NONE",

  "initialSequenceId" : null,

  "autoUpdatePartitions" : true,

  "autoUpdatePartitionsIntervalSeconds" : 60,

  "multiSchema" : true,

  "properties" : { }

}

15:06:13.411 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: 
{

  "serviceUrl" : "pulsar://localhost:6650/",

  "authPluginClassName" : null,

  "operationTimeoutMs" : 3,

  "statsIntervalSeconds" : 60,

  "numIoThreads" : 1,

  "numListenerThreads" : 1,

  "connectionsPerBroker" : 1,

  "useTcpNoDelay" : true,

  "useTls" : false,

  "tlsTrustCertsFilePath" : "",

  "tlsAllowInsecureConnection" : false,

  "tlsHostnameVerificationEnable" : false,

  "concurrentLookupRequest" : 5000,

  "maxLookupRequest" : 5,

  "maxLookupRedirects" : 20,

  "maxNumberOfRejectedRequestPerConnection" : 50,

  "keepAliveIntervalSeconds" : 30,

  "connectionTimeoutMs" : 1,

  "requestTimeoutMs" : 6,

  "initialBackoffIntervalNanos" : 1,

  "maxBackoffIntervalNanos" : 600,

  "listenerName" : null,

  "useKeyStoreTls" : false,

  "sslProvider" : null,

  "tlsTrustStoreType" : "JKS",

  "tlsTrustStorePath" : "",

  "tlsTrustStorePassword" : "",

  "tlsCiphers" : [ ],

  "tlsProtocols" : [ ],

  "proxyServiceUrl" : null,

  "proxyProtocol" : null,

  "enableTransaction" : false

}

15:06:13.485 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [null] Creating 
producer on cnx [id: 0x90192576, L:/127.0.0.1:45148 - 
R:localhost/127.0.0.1:6650]

15:06:13.576 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [standalone-1-4] 
Created producer on cnx [id: 0x90192576, L:/127.0.0.1:45148 - 
R:localhost/127.0.0.1:6650]

15:06:13.919 [pulsar-client-io-1-1] INFO  
com.scurrilous.circe.checksum.Crc32cIntChecksum - SSE4.2 CRC32C provider 
initialized

15:06:14.002 [main] INFO  org.apache.pulsar.client.impl.PulsarClientImpl - 
Client closing. URL: pulsar://localhost:6650/

15:06:14.054 [main] INFO  org.apache.pulsar.client.impl.ProducerImpl - 
[my-topic] [standalone-1-4] Closed Producer

15:06:14.090 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ClientCnx - [id: 0x90192576, L:/127.0.0.1:45148 ! 
R:localhost/127.0.0.1:6650] Disconnected

15:06:14.164 [main] INFO  org.apache.pulsar.client.cli.PulsarClientTool - 1 
messages successfully produced



real 0m9.011s

user 0m10.884s

sys 0m1.172s

Re: CPU consuming question

[Wesley Peng]

Can you help check with log output?

$ tail -f pulsar-standalone-PresentSpicy-VM.log 

14:45:14.281 [main-EventThread] INFO  
org.apache.distributedlog.logsegment.PerStreamLogSegmentCache - 
01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76/CURRENT
 added log segment (logrecs_01 : [LogSegmentId:279, 
firstTxId:16, lastTxId:16, version:VERSION_V5_SEQUENCE_ID, 
completionTime:1612161014650, recordCount:1, regionId:0, status:0, 
logSegmentSequenceNumber:1, lastEntryId:0, lastSlotId:0, inprogress:false, 
minActiveDLSN:DLSN{logSegmentSequenceNo=1, entryId=0, slotId=0}, 
startSequenceId:0]) to cache.

14:45:14.297 [DLM-/stream/storage-OrderedScheduler-1-0] INFO  
org.apache.distributedlog.BKLogWriteHandler - Deleting ledger 
[LogSegmentId:279, firstTxId:16, lastTxId:16, version:VERSION_V5_SEQUENCE_ID, 
completionTime:1612161014650, recordCount:1, regionId:0, status:0, 
logSegmentSequenceNumber:1, lastEntryId:0, lastSlotId:0, inprogress:false, 
minActiveDLSN:DLSN{logSegmentSequenceNo=1, entryId=0, slotId=0}, 
startSequenceId:0] for 
01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76/CURRENT:

14:45:14.301 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  
org.apache.distributedlog.impl.ZKLogSegmentMetadataStore - Delete segment : 
[LogSegmentId:279, firstTxId:16, lastTxId:16, version:VERSION_V5_SEQUENCE_ID, 
completionTime:1612161014650, recordCount:1, regionId:0, status:0, 
logSegmentSequenceNumber:1, lastEntryId:0, lastSlotId:0, inprogress:false, 
minActiveDLSN:DLSN{logSegmentSequenceNo=1, entryId=0, slotId=0}, 
startSequenceId:0]

14:45:14.331 [main-EventThread] INFO  
org.apache.distributedlog.bk.SimpleLedgerAllocator - Ledger allocator for 
/stream/storage/01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76//allocation
 moved version from -1 to 0.

14:45:14.349 [io-checkpoint-scheduler-OrderedScheduler-1-0] INFO  
org.apache.bookkeeper.statelib.impl.rocksdb.checkpoint.RocksdbCheckpointTask - 
Delete remote checkpoint 0352d14e-8a42-4453-89ed-1fbf34051b76 from checkpoint 
store at 
01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76

14:45:33.435 [pulsar-web-68-5] INFO  org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [01/Feb/2021:14:45:33 +0800] "GET 
/admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false 
HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 10

14:46:03.429 [pulsar-web-68-1] INFO  org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [01/Feb/2021:14:46:03 +0800] "GET 
/admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false 
HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 6

14:46:33.441 [pulsar-web-68-6] INFO  org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [01/Feb/2021:14:46:33 +0800] "GET 
/admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false 
HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 10

14:47:03.431 [pulsar-web-68-8] INFO  org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [01/Feb/2021:14:47:03 +0800] "GET 
/admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false 
HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 7



Thank you
Regards.


On Mon, Feb 1, 2021, at 2:30 PM, Sijie Guo wrote:
> Hi Wesley,
> 
> The CPU usage might be coming from some background tasks. Do you happen to 
> have the log of the standalone server? This might give some insights to the 
> problem. 
> 
> - Sijie
> 
> On Sun, Jan 31, 2021 at 9:58 PM Wesley Peng  wrote:
>> __
>> I deployed a standalone pulsar server on a two cores VPS, which was new 
>> installed.
>> Even there is not any jobs running on pulsar, it still consumes CPU about 
>> 15+%.
>> 'top' shows the java process takes it.
>> 
>> Can you help explain this? Thank you.
>> 
>> Regards.

CPU consuming question

[Wesley Peng]

I deployed a standalone pulsar server on a two cores VPS, which was new 
installed.
Even there is not any jobs running on pulsar, it still consumes CPU about 15+%.
'top' shows the java process takes it.

Can you help explain this? Thank you.

Regards.

Re: perl.com's registration appears to have been hijacked

[Wesley Peng]

Nice sender domain. You would better sell it to ruby language developer.

On Sun, Jan 31, 2021, at 5:20 AM, Earl Ruby wrote:
> Re-sending using the "from" address that's subscribed to the list... 
> 
> On Fri, Jan 29, 2021 at 3:07 PM Earl Ruby  wrote:
>> A glance at the domain records show the contact information is now "REDACTED 
>> FOR PRIVACY". Prior to the change Tom Christiansen was listed as the domain 
>> administrative contact.
>> 
>> https://www.theregister.com/2021/01/28/perl_hijacking/
>> 
>> https://nakedsecurity.sophos.com/2021/01/29/the-mystery-of-the-missing-perl-webite/
>> 
> 
> -- 
> Earl Ruby
> http://earlruby.org/
> http://www.linkedin.com/in/earlruby
> https://twitter.com/earlruby

Re: perl.com's registration appears to have been hijacked

[Wesley Peng]

Nice sender domain. You would better sell it to ruby language developer.

On Sun, Jan 31, 2021, at 5:20 AM, Earl Ruby wrote:
> Re-sending using the "from" address that's subscribed to the list... 
> 
> On Fri, Jan 29, 2021 at 3:07 PM Earl Ruby  wrote:
>> A glance at the domain records show the contact information is now "REDACTED 
>> FOR PRIVACY". Prior to the change Tom Christiansen was listed as the domain 
>> administrative contact.
>> 
>> https://www.theregister.com/2021/01/28/perl_hijacking/
>> 
>> https://nakedsecurity.sophos.com/2021/01/29/the-mystery-of-the-missing-perl-webite/
>> 
> 
> -- 
> Earl Ruby
> http://earlruby.org/
> http://www.linkedin.com/in/earlruby
> https://twitter.com/earlruby

pulsar with perl from rest API

[Wesley Peng]

Hello

I need to write some codes with perl for accessing to pulsar cluster.
Can we use the rest API for this purpose since there is not the native client 
library for perl yet?

Thanks.

Re: perl's pulsar client

[Wesley Peng]

I am looking for a native client library since there is the functions more than 
publish/consume I want.

Thanks.

On Wed, Jan 27, 2021, at 9:19 AM, Mithun Bhattacharya wrote:
> This seems to demonstrate Pulsar with mojo.
> 
> https://www.perlmonks.org/?node_id=11123636
> 
> On Tue, Jan 26, 2021, 7:09 PM Wesley Peng  wrote:
>> __
>> I saw apache pulsar has many clients in other languages such as python's:
>> http://pulsar.apache.org/docs/en/client-libraries-python/
>> 
>> But it lacks a perl client library.
>> Would anyone have the plan to make one?
>> 
>> Thanks.

perl's pulsar client

[Wesley Peng]

I saw apache pulsar has many clients in other languages such as python's:
http://pulsar.apache.org/docs/en/client-libraries-python/

But it lacks a perl client library.
Would anyone have the plan to make one?

Thanks.

perl's pulsar client

[Wesley Peng]

I saw apache pulsar has many clients in other languages such as python's:
http://pulsar.apache.org/docs/en/client-libraries-python/

But it lacks a perl client library.
Would anyone have the plan to make one?

Thanks.

Re: Looking for feedback on the Ignite 3.0.0 Alpha

[Wesley Peng]

When will the stable version of 3.0 get released? thanks.

Valentin Kulichenko wrote:
I'm excited to announce that the first alpha build of the Ignite 3 is 
out and available for download!


Ignite 3 is the new project that was initiated by the Ignite community 
last year. Please refer to this page if you want to learn more: 
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0 



The just-released alpha build is a sneak peek into the future of Ignite. 
It doesn't represent a fully-functional product (no discovery, caches, 
compute, etc.), but demonstrates major mechanics of how you will 
interact with Ignite going forward.

Re: This is a test

[Wesley Peng]

Please don't send a test message to the public list which was read by
thousands of people. thanks.

On Fri, Jan 8, 2021 at 5:26 AM Craig Hatch  wrote:

> I have added you to the EMAIL  list, so when I have questions.
>
> Just learn for fun.
>
>
> Craig Hatch
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: graceful shutdown for C++ applications

[Wesley Peng]

Issuing a command like "kill process_id"  doesn't work?

regards.

On Thu, Jan 7, 2021 at 4:14 PM rakshita04 
wrote:

> Hi Team,
>
> We are using apache-ignite for our applications running on 2 machines and
> connected over network.
> We are facing some issue where if kill is performed on running application,
> it somehow corrupts the node and then node never comes up and keep on
> rebooting.
> Is there a way to handle this shutdown gracefully? so that there is no data
> loss and node corruption.
>
> regards,
> Rakshita Chaudhary
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: [ANNOUNCE] New Pulsar Committer: LinLin

[Wesley Peng]

Congrats Linlin~

Regards.

On Wed, Jan 6, 2021 at 5:25 PM Jinfeng Huang  wrote:

> Congratulations to Linlin~
>
> Best Regards,
> Jennifer
>
>
> On Wed, Jan 6, 2021 at 3:30 PM Matthias Peng 
> wrote:
>
>> congratulations linlin!
>>
>> On Wed, Jan 6, 2021 at 3:05 PM PengHui Li  wrote:
>>
>>> The Apache Pulsar PMC recently extended committer karma to LinLin and he
>>> has accepted.
>>> LinLin has done a lot of contributions to Pulsar core features and bug
>>> fixes. It is great to have LinLin onboard as Pulsar committers.
>>>
>>> We are looking forward to more contributions from him.
>>>
>>> Congratulations and welcome onboard LinLin!
>>>
>>> -
>>>
>>> Penghui on behave of the Pulsar PMC
>>>
>>

Re: Unsubscribe

[Wesley Peng]

Bhavya Jain wrote:

Unsubscribe


please send an email to: user-unsubscr...@spark.apache.org to 
unsubscribe yourself from the list. thanks.


-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: [prometheus-users] Prometheus disk I/O metrics

[Wesley Peng]

Brian,

Do you know if we can implement a Lua exporter within nginx who take 
application's APM and report to prometheus?


Thank you.


Brian Candler wrote:
Just to add, the data collected by node_exporter maps closely to the raw 
stats exposed by the kernel, so the kernel documentation is helpful:


--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/2e0ca03a-4747-f214-d693-2e54996f01c8%40pobox.com.

Re: [prometheus-users] Prometheus disk I/O metrics

[Wesley Peng]

You can calculate it from the basic IO metrics Prometheus provided.

Regards.


rsch...@gmail.com wrote:
In datadog I used metrics "system.io.await" to create alert on my 
linux instances. What is the equivalent metrics in prometheus? 


--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/500d9891-b3cb-9a0e-1424-fc45cd614978%40pobox.com.

Re: [prometheus-users] Prometheus.service status failed

[Wesley Peng]

are you running 64bit program on the 32bit OS, or reverse?

regards.

Suryaprakash Kancharlapalli wrote:
As i have setup the Prometheus configuration using from the binary. But 
in the end when i start the service it shows like Prometheus startup 
service failed. Please find the attached snap for reference. Can some 
suggest what could be the reason and how can i resolve it.


--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/57ce9c9b-e7c3-5089-d3f2-2004c047c839%40pobox.com.

Re: [prometheus-users] Alert Manager source URL to point to TLS URL of Prometheus

[Wesley Peng]

You can setup Nginx to proxy both Alertmanager and Prometheus itself on 
different http port.


regards.


sunils...@gmail.com wrote:
Now the challenge is , AlertManager is directly associated with 
PRometheus and when I access AlertManager , All the source links ate 
pointing to Prometheus directly .

Can we configure alertmanager source URL to TLS URL of Prometheus  ?


--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/427ffa62-06f1-9e59-d613-310ddcf1a32b%40pobox.com.

Re: cache a object in modperl

[Wesley Peng]

Mithun Bhattacharya wrote:
Does IANA have an easy way of determining whether there is an update 
since a certain date ? I was thinking it might make sense to just run a 
scheduled job to monitor for update and then restart your service or 
refresh your local cache depending upon how you solve it.


Yes I agree with this.
I may monitor IANA's database via their version changes, and run a 
crontab to restart my apache server during the non-active user time 
(i.e, 3:00 AM).


Or do you have better solution?
Thanks.

Re: cache a object in modperl

[Wesley Peng]

Hello

Mithun Bhattacharya wrote:
How frequently do you wish to refresh the cache ? if you do in startup 
then your cache refresh is tied to the service restart which might not 
be ideal or feasible.


I saw recent days IANA has updated their database on date of:

2020.09.09
2020.09.13

So I assume they will update the DB file in few days.

Regards.

Re: cache a object in modperl

[Wesley Peng]

That's great. Thank you Adam.

Adam Prime wrote:
If the database doesn't change very often, and you don't mind only 
getting updates to your database when you restart apache, and you're 
using prefork mod_perl, then you could use a startup.pl to load your 
database before apache forks, and get a shared copy globally in all your 
apache children.


https://perl.apache.org/docs/1.0/guide/config.html#The_Startup_File

This thread from 13 years ago seems to have a clear-ish example of how 
to use startup.pl to do what i'm talking about.


If you need it to update more frequently than when you restart apache, 
you could potentially use a PerlChildInitHandler to load the data when 
apache creates children.  This will use more memory, as each child will 
have it's own copy, and can also result in situation where children can 
have different versions of the database loaded and be serving requests 
at the same time.  If you want to go this way you might want to also add 
a MaxRequestsPerChild directive to your apache config to make sure that 
you're children die and get refreshed on the regular, if you don't 
already have one.


Adam


On 9/13/2020 10:51 PM, Wesley Peng wrote:

Hello

I am not so familiar with modperl.

For work requirement, I need to access IANA TLD database.

So I wrote this perl module:
https://metacpan.org/pod/Net::IANA::TLD

But, for each new() in the module, the database file will be 
downloaded from IANA's website.


I know this is pretty Inefficient.

My question is, can I cache the new'ed object by modperl?

If so, how to do?

Thanks.

cache a object in modperl

[Wesley Peng]

Hello

I am not so familiar with modperl.

For work requirement, I need to access IANA TLD database.

So I wrote this perl module:
https://metacpan.org/pod/Net::IANA::TLD

But, for each new() in the module, the database file will be downloaded 
from IANA's website.


I know this is pretty Inefficient.

My question is, can I cache the new'ed object by modperl?

If so, how to do?

Thanks.

Re: [prometheus-users] Can prometheurs scrape :/deamon/metrics ?

[Wesley Peng]

I am sure it can. b/c I just finished a monitoring like this form. :)


Rodolphe Ghio wrote:

I was wondering if prometheus can scrape a target like this ?
:/deamon/metrics, cause I've tried many times and prometheus
doesn't want to start anymore.


--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/65e5ab8b-793e-3e56-0ca5-085adf5b8d74%40gmx.ie.

Re: [prometheus-users] Using alert manager with external receiver

[Wesley Peng]

Hi

Nina Sc wrote:

Is there a way to use the alert manager to send an alert to external API,
I mean instead using excage server or slack etc, I will provide an endpoint
like `https://mypullendpoint.host.com` and the alert manager will send the
alert to this URL ?


Yes you need to define receivers and webhook.

such as:

receivers:
- name: "alerta"
  webhook_configs:
  - url: 'http://localhost:8080/webhooks/prometheus'
send_resolved: true


please refer these:
https://alerta.io/
https://github.com/alerta/prometheus-config


Regards.

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/375f41af-8bfb-e0c6-c4ec-38a0c6ca0847%40gmx.ie.

Re: unsubscribe

[Wesley Peng]

Please send an empty message to: user-unsubscr...@ignite.apache.org to 
unsubscribe yourself from the list.



Sijo Mathew wrote:

Re: run ignitevisorcmd in k8s and docker？

[Wesley Peng]

Hi

bbweb wrote:
we are meeting problem when we run ignitevisorcmd in K8S and docker 
environment. After we start a cluster in K8S and run ignitevisorcmd in 
the node, it just can't find any node when running "top" in it, it 
just show empty topology.


Do you have any error log?
Are you sure ignite cluster is started up correctly?

regards.

Re: Call for presentations for ApacheCon North America 2020 now open

[Wesley Peng]

Congrats. We could prepare a talking for "machine learning application 
with Ignite", as we did store feature engineering data into ignite for 
large-scale and fast access.


regards.


Saikat Maitra wrote:

Congrats!!!

It looks like both of our talks are on same day, Tuesday, September 29th

https://apachecon.com/acah2020/tracks/ignite.html

Re: Question about deployment of math computing [EXT]

[Wesley Peng]

James,

James Smith wrote:

The services which use apache/mod_perl work reliably and return data for these 
- the dancer/starman sometimes fail/hang as there are no backends to serve the 
requests or those backends timeout requests to the nginx/proxy (but still 
continue using resources). The team running the backends fail to notice this - 
because there is no easy to see reporting etc on these boxes.


Thanks for letting me know this.
We have been using starman for restful api service, they are light 
weight http request/response.
But for (machine learning)/(deep learning) serving stuff, we may 
consider to use modperl for more stability.


regards.

Re: Question about deployment of math computing

[Wesley Peng]

Thank you David. That makes thing clear. I have made mistake to think 
starman was event driven, who is really preforked.


I think any preforked server could serve our depolyment better.

Regards.


dc...@prosentient.com.au wrote:

Hi Wesley,

I don't know all the ins and outs of Starman. I do know that Starman is a 
preforking web server, which uses Net::Server::PreFork under the hood. You 
configure the number of preforked workers to correspond with your CPU and 
memory limits for that server.

As per the Starman documentation 
(https://metacpan.org/pod/release/MIYAGAWA/Starman-0.4015/lib/Starman.pm), you 
should put a frontend server/reverse proxy (like Nginx) in front of Starman. 
Nginx is often recommended because it's event-driven. The idea being that a few 
Nginx workers (rather than those thousands of Apache processes you mentioned) 
can handle a very large volume of HTTP requests, and then Nginx intelligently 
passes those requests to the backend server (e.g. Starman).

Of course, no matter what, you can still get timeouts if the backend server 
isn't responding fast enough, but typically the backend process is going as 
fast as it can. At that point, your only option is to scale up. You can do that 
by using Nginx as a load balancer and horizontally scaling your Starman 
instances, or you can put more CPUs on that machine, and configure Starman to 
prefork more workers.

Let's say you use Mod_Perl/Apache instead of Starman/Nginx. At the end of the 
day, you still need to think about how many concurrent requests you're needing 
to serve and how many CPUs you have available. If you've configured Apache to 
have too many processes, you're going to overwhelm your server with tasks. You 
need to use reasonable constraints.

But remember that this isn't specific to Perl/Starman/Nginx/Apache/mod_perl. 
These are concepts that translate to any stack regardless of programming 
language and web server. (Of course, languages like Node.js and Golang have 
some very cool features for dealing with blocking I/O, so that you can make the 
most of the resources you have. That being said, Perl has Mojo/Mojolicious, 
which claims to do non-blocking I/O in Perl. I have yet to try it though. I'm 
skeptical, but could give it a try.)

At the end of the day, it depends on the workload that you're trying to cater 
to.

David Cook

-Original Message-
From: Wesley Peng 
Sent: Wednesday, 5 August 2020 1:31 PM
To: dc...@prosentient.com.au; modperl@perl.apache.org
Subject: Re: Question about deployment of math computing

Hi

dc...@prosentient.com.au wrote:

That's interesting. After re-reading your earlier email, I think that I 
misunderstood what you were saying.

Since this is a mod_perl listserv, I imagine that the advice will always be to 
use mod_perl rather than starman?

Personally, I'd say either option would be fine. In my experience, the key 
advantage of mod_perl or starman (say over CGI) is that you can pre-load 
libraries into memory at web server startup time, and that processes are 
persistent (although they do have limited lifetimes of course).

You could use a framework like Catalyst or Mojolicious (note Dancer is another 
framework, but I haven't worked with it) which can support different web 
servers, and then try the different options to see what suits you best.

One thing to note would be that usually people put a reverse proxy in front of 
starman like Apache or Nginx (partially for serving static assets but other 
reasons as well). Your stack could be less complicated if you just went the 
mod_perl/Apache route.

That said, what OS are you planning to use? It's worth checking if mod_perl is 
easily available in your target OS's package repositories. I think Red Hat 
dropped mod_perl starting with RHEL 8, although EPEL 8 now has mod_perl in it. 
Something to think about.


We use ubuntu 16.04 and 18.04.

We do use dancer/starman in product env, but the service only handle light 
weight API requests, for example, a restful api for data validation.

While our math computing is heavy weight service, each request will take a lot 
time to finish, so I think should it be deployed in dancer?

Since the webserver behind dancer is starman by default, starman is event 
driven, it uses very few processes ,and the process can't scale up/down 
automatically.

We deploy starman with 5 processes by default. when 5 requests coming, all 5 
starman processes are so busy to compute them, so the next request will be 
blocked. is it?

But apache mp is working as prefork way, generally it can have as many as 
thousands of processes if the resource is permitted. And the process management 
can scale up/down the children automatically.

So my real question is, for a CPU consuming service, the event driven service 
like starman, has no advantage than preforked service like Apache.

Am I right?

Thanks.

Re: Question about deployment of math computing

[Wesley Peng]

Hi

dc...@prosentient.com.au wrote:

That's interesting. After re-reading your earlier email, I think that I 
misunderstood what you were saying.

Since this is a mod_perl listserv, I imagine that the advice will always be to 
use mod_perl rather than starman?

Personally, I'd say either option would be fine. In my experience, the key 
advantage of mod_perl or starman (say over CGI) is that you can pre-load 
libraries into memory at web server startup time, and that processes are 
persistent (although they do have limited lifetimes of course).

You could use a framework like Catalyst or Mojolicious (note Dancer is another 
framework, but I haven't worked with it) which can support different web 
servers, and then try the different options to see what suits you best.

One thing to note would be that usually people put a reverse proxy in front of 
starman like Apache or Nginx (partially for serving static assets but other 
reasons as well). Your stack could be less complicated if you just went the 
mod_perl/Apache route.

That said, what OS are you planning to use? It's worth checking if mod_perl is 
easily available in your target OS's package repositories. I think Red Hat 
dropped mod_perl starting with RHEL 8, although EPEL 8 now has mod_perl in it. 
Something to think about.


We use ubuntu 16.04 and 18.04.

We do use dancer/starman in product env, but the service only handle 
light weight API requests, for example, a restful api for data validation.


While our math computing is heavy weight service, each request will take 
a lot time to finish, so I think should it be deployed in dancer?


Since the webserver behind dancer is starman by default, starman is 
event driven, it uses very few processes ,and the process can't scale 
up/down automatically.


We deploy starman with 5 processes by default. when 5 requests coming, 
all 5 starman processes are so busy to compute them, so the next request 
will be blocked. is it?


But apache mp is working as prefork way, generally it can have as many 
as thousands of processes if the resource is permitted. And the process 
management can scale up/down the children automatically.


So my real question is, for a CPU consuming service, the event driven 
service like starman, has no advantage than preforked service like Apache.


Am I right?

Thanks.

Re: Question about deployment of math computing

[Wesley Peng]

Hi

dc...@prosentient.com.au wrote:

If your app isn't human-facing, then I don't see why a little delay would be a 
problem?


Our app is not human facing. The application by other department will 
request the result from our app via HTTP.


The company has huge big-data stack deployed, such as 
Hadoop/Flink/Storm/Spark etc, all these solutions have been existing 
there. The data traffic each day is as huge as xx PB.


But, those stacks have complicated privileges control layer, they are 
most time running as backend service, for example, offline analysis, 
feature engineering, and some real time streaming.


We train the modes in backend, and use the stacks mentioned above.

But, once the mode finished training, they will be pushed to online as 
prediction service and serve as HTTP API, b/c third party apps will only 
like to request the interface via HTTP way.


Thanks.

Re: Question about deployment of math computing

[Wesley Peng]

Hi

Mithun Bhattacharya wrote:

Do you really need a webserver which is providing a blocking service ?


yes, this is a prediction server, which would be deployed in PROD 
environment, the client application would request the prediction server 
for results as scores. You can think it as online recommendation systems.


regards.

Question about deployment of math computing

[Wesley Peng]

Hi

We do math programming (so called machine learning today) in webserver.
The response would be slow, generally it will take 100ms~500ms to finish 
a request.
For this use case, shall we deploy the code within preforked modperl ,or 
event-driven server like dancer/starman?
(we don't use DB like mysql or other slow IO storage server, all 
arguments were passed to webserver by HTTP POST from client).


Thank you.

Re: suggestions for perl as web development language [EXT]

[Wesley Peng]

jbiskofski wrote:

Excelent, stable, FAST, production ready HTTP server: Starman


yes starman is good. we use it for rest-api service, the app code is 
dancer, whose backend server is starman.


if we need more concurrent handlers, a simple front-proxy like nginx 
would be deployed.


regards.

Re: suggestions for perl as web development language [EXT]

[Wesley Peng]

Hi

Mark Blackman wrote:

mod_perl’s relative efficiency can be achieved by other well-known means.


for example?

thank you.

Re: suggestions for perl as web development language [EXT]

[Wesley Peng]

Joseph He wrote:
My company uses Perl for web development. It handles real time payment 
transactions without any problem. Good software is made by the people 
not by the language.


Maybe I am weak on this point, but how perl handle types more strictly?
for example,

123 + '456'

this is permitted in perl, and this is dangerous. It is not possible in 
other strong types language.


Thanks.

suggestions for perl as web development language

[Wesley Peng]

greetings,

My team use all of perl, ruby, python for scripting stuff.
perl is stronger for system admin tasks, and data analysis etc.
But for web development, it seems to be not as popular as others.
It has less selective frameworks, and even we can't get the right people 
to do the webdev job with perl.
Do you think in today we will give up perl/modperl as web development 
language, and choose the alternatives instead?


Thanks & Regards

Re: Apache Beam a Complete Guide - Review?

[Wesley Peng]

Rion Williams wrote:

The three authors of Streaming Systems are folks that work on Google’s Dataflow 
Project, which for all intents and purposes is essentially an implementation of 
the Beam Model. Two of them are members of the Beam PMC (essentially a steering 
committee for the project) and you’ll frequently see them if you are active 
within the mailing lists here.

As mentioned earlier in the thread, I can’t recommend Streaming Systems enough. 
If you are curious about Beam or want to dig into it, it’s the best book you’ll 
find on the topic.


Thanks for this info Rion.

Re: Apache Beam a Complete Guide - Review?

[Wesley Peng]

Hi Luke

Luke Cwik wrote:
The author for Apache Beam A Complete Guide does not have good reviews 
on Amazon for their other books and as you mentioned no reviews for this 
one.


I would second the Streaming Systems book as the authors directly worked 
on Apache Beam.



So, can Apache beam team in google write a book directly for the project 
users?


Thanks.

Re: Apache Beam a Complete Guide - Review?

[Wesley Peng]

Hi Rion

Rion Williams wrote:
I considered that one as well but was in the same boat in terms of not 
pulling the trigger (lack of reviews, price point, etc.). I eventually 
landed on Streaming Systems, which I highly, highly recommend if you 
want to learn more about the Beam model:


- http://streamingsystems.net/

I don’t think there’s a better book that I’ve come across that focuses 
on it more (and if there is one, I’d love to know about it). I wrote a 
blog post that includes a short-review of it if you want a slightly 
longer summary (http://rion.io/2020/05/09/an-education-in-streaming/), 
but again - I highly recommend checking it out if you hadn’t already.


Thanks for the answer. I will check the resource you gave above.

Regards.

Apache Beam a Complete Guide - Review?

[Wesley Peng]

Hello

Has anyone bought this book? Can you give a simple review, good or not?

https://www.thriftbooks.com/w/apache-beam-a-complete-guide---2020-edition/26243355/item/36997139/

I want to buy a book for beam the specific topic, but this book seems 
too new to have enough reviews.


Thanks.

Re: [prometheus-users] Re: not able to see metrics from query browser even though end point is up and showing metrics through curl

[Wesley Peng]

Hi

Brian Candler wrote:

"up" missing - scraping not configured.
"up" has value 0 - unable to communicate or data is bad.
"up" has value 1 - maybe partial scrape

Check logs at prometheus and exporter; check traffic between prometheus 
and exporter with tcpdump.


Can you show a sample tcpdump syntax for this purpose?

Thanks.

--
You received this message because you are subscribed to the Google Groups 
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-users/130d9158-d6d7-f49f-1884-b2dd1f66e314%40pobox.com.

Re: unsubscribe

[Wesley Peng]

please send an empty email to: user-unsubscr...@spark.apache.org to 
unsubscribe yourself from the list.



Sri Kris wrote:
Sent from Mail  for 
Windows 10




-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: I need to study Python

[Wesley Peng]

Pick a book like Core Python Programming, read it and do coding follow it.

regards.

sinanp...@gmail.com wrote:

Hey, I'm a completely noob.
I want to learn python, how can i or where can i study python?


--
https://mail.python.org/mailman/listinfo/python-list

Re: [go-nuts] SSL socket listener

['Wesley Peng' via golang-nuts]

Thanks. how about the sample of general socket listener with SSL rather 
than net/http implementation?


Regards

Dimas Prawira wrote:

Here is an example running server with TLS

package  main

import  (
 "net/http"
 "log"
)

func  HelloServer(w  http.ResponseWriter,req  *http.Request) {
 w.Header().Set("Content-Type","text/plain")
 w.Write([]byte("This is an example server.\n"))
}

func  main() {
 http.HandleFunc("/hello",HelloServer)
 err  :=  http.ListenAndServeTLS(":443","server.crt","server.key",nil)
 if  err  !=  nil  {
 log.Fatal("ListenAndServe: ",err)
 }
}



--
Wesley Peng
wesleyp...@aol.com

--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ffdb0882-3e1a-173d-6c3c-91f63d88bbfe%40aol.com.

Re: [VOTE] Apache CloudStack 4.14.0.0 RC3

[Wesley Peng]

Andrija Panic wrote:

http://docs.cloudstack.apache.org/en/latest/upgrading/upgrade/upgrade-4.13.html#time-zone-requirements


handled in the Upgrade docs - exactly the same..



[OT] is there a cloudstack private IAAS deployment solution? not that 
large scale, most for storage. if there is the commercial solution, we 
can pay for it.


Thanks.

[go-nuts] SSL socket listener

['Wesley Peng' via golang-nuts]

Hello,
How do I program with SSL to make a server listen on specific port which 
accepts SSL transfer only?
Is there any guide for this since I have no experience on SSL socket 
programming.
Thanks.

Wesley Peng
wesleyp...@aol.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1690752345.1320667.1591168756241%40mail.yahoo.com.

Re: [grpc-io] gRPC hostname

[Wesley Peng]

XCompiler wrote:
     server.Listen(IPAddress.Loopback, 5001, 
listenOptions =>


You have server listened on lookback interface, please change it to a 
regular IP address.


regards.

--
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/dd0615c0-f27f-1675-6825-1fe40cfb8076%40gmail.com.

Re: connection refused

[Wesley Peng]

Does either client or server have any logs?

Thanks

On Tue, Jun 2, 2020 at 6:37 PM Clay Teahouse  wrote:

> I'd appreciate your help with this issue.
> I have a server and a client node, the latter running in tomcat. I get
> connection refused if I try to connect to the server node from the client,
> although the server shows the client has joined the cluster. I don't have a
> problem running a service on the server's service grid from the client node.
> I don't have any issue connecting from any client anywhere, if the client
> is not running in tomcat.
> what could be the problem?
>
>

Re: Zookeeper discovery in mix environments.

[Wesley Peng]

Maybe you would ask the question on VM/container related mailing list?

Regards

On Mon, Jun 1, 2020 at 9:32 PM John Smith  wrote:

> Any news on this? Thanks
>
>
> On Thu., May 28, 2020, 1:10 p.m. John Smith, 
> wrote:
>
>> Hi, I'm running an ignite cluster on VMs running on Openstack and using
>> regular network stack nothing special here.
>>
>> My CLIENT (client=true) applications are running in DC/OS using docker
>> container in bridged network mode.
>>
>> When using TCP discovery everything works nice and dandy. But I recently
>> tried to switch to Zookeeper discovery but the docker application can't
>> seem to connect or get discovered?
>>
>> I'm assuming when running in bridge mode I need to open a port on the
>> bridge to allow other node to connect back. Is there a way to the ignite
>> client/zookeeper SPI that the CLIENT node is available at a specific
>> ip/port?
>>
>

[go-nuts] Is this AMQP 0.9.1 library updated?

['Wesley Peng' via golang-nuts]

Hello,
I tried to access RabbitMQ using this AMQP 
library:https://github.com/streadway/amqp

I am not sure if this library get updated following the recent RMQ 
version.Please suggest, thanks.

Wesley Peng
wesleyp...@aol.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1077505155.1304617.1588125421267%40mail.yahoo.com.

[dns-operations] DNS over Wikipedia

[Wesley Peng]

Wikipedia keeps track of official URLs for popular websites. With DNS 
over Wikipedia installed, domains ending with .idk are redirected by 
searching Wikipedia and extracting the relevant URL from the infobox.


Example:

1. Type scihub.idk/ in browser address bar
2. Observe redirect to https://sci-hub.tw (at the time of writing)

https://github.com/aaronjanse/dns-over-wikipedia

My question is about their implementation.
Does it mean any domain ending with .idk went to an interface, this 
interface queried the domain part before .idk from wikipedia, then 
returned a URL redirection to some other website?


I thought querying from wikipedia's API directly (or google API) is even 
more efficient than this way.


Thanks.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

[OT] looking for a good mobile MUA

[Wesley Peng]

Hello

On desktop I always use Thunderbird for MUA.

Thunderbird is clean and lite, handling protocol well.

But for mobile (I primarily use iOS), I couldn't find a good MUA to use.

I tried almost every release on appStore (outlook, edison, bluemail, 
spark, mymail...), they are either too complicated to use, or not 
following email standard (for example, can't choose plaintext composer, 
or can't setup sender auth username different from the email address, or 
never handle SSL correctly), or showing the message body ugly.


Can you suggest a suitable mobile MUA? it should like thunderbird, be 
fast, clean, standard, and easy to use.


Thanks.
(sorry for this OT message)

should we use plaintext for message?

[Wesley Peng]

Hello

Following this guide:
https://useplaintext.email/

Shall we use plaintext message in regular email communication?

Thanks

Re: Postfix stable release 3.5.0

[Wesley Peng]

Nice work on the release. Thanks Wietse.

Mar 17, 2020, 21:39 by emaw...@gmail.com:

> Hello! excellent work, the option to bounce emails manually is very useful 
> for me.
>
> Never perform the update through the source code. I would like to know if I 
> can get the src.rpm for Centos?
>
>
>
> El lun., 16 de mar. de 2020 a la(s) 11:18, Wietse Venema (> 
> wie...@porcupine.org> ) escribió:
>
>> [An on-line version of this announcement will be available at
>>  >> http://www.postfix.org/announcements/postfix-3.5.0.html>> ]
>>  
>>  Postfix stable release 3.5.0 is available. Support has ended for
>>  legacy release Postfix 3.1.
>>  
>>  The main changes are below. See the RELEASE_NOTES file for further details.
>>  
>>    * Support for the haproxy v2 protocol. The Postfix implementation
>>      supports TCP over IPv4 and IPv6, as well as non-proxied
>>      connections; the latter are typically used for heartbeat tests.
>>  
>>    * Support to force-expire email messages. This introduces new
>>      postsuper(1) command-line options to request expiration, and
>>      additional information in mailq(1) or postqueue(1) output.
>>  
>>    * The Postfix SMTP and LMTP client support a list of nexthop
>>      destinations separated by comma or whitespace. These destinations
>>      will be tried in the specified order. Examples:
>>  
>>      /etc/postfix/>> main.cf >> :
>>          relayhost = foo.example, bar.example
>>          default_transport = smtp:foo.example, bar.example
>>  
>>  Incompatible changes:
>>  
>>    * Logging: Postfix daemon processes now log the from= and to=
>>      addresses in external (quoted) form in non-debug logging (info,
>>      warning, etc.). This means that when an address localpart
>>      contains spaces or other special characters, the localpart will
>>      be quoted, for example:
>>  
>>          from=<"name with spaces"@>> example.com >> >
>>  
>>      Specify "info_log_address_format = internal" for backwards 
>> compatibility.
>>  
>>    * Postfix now normalizes IP addresses received with XCLIENT,
>>      XFORWARD, or with the HaProxy protocol, for consistency with
>>      direct connections to Postfix. This may change the appearance
>>      of logging, and the way that check_client_access will match
>>      subnets of an IPv6 address.
>>  
>>  You can find the updated Postfix source code at the mirrors listed
>>  at >> http://www.postfix.org/>> .
>>  
>>          Wietse
>>

Re: postfix for IoT

[Wesley Peng]

Hello

Thanks all answers for kind info.
My requirement is, for example, when refrigerator found there were no food
in itself, it will send an email to remind the people.
But refrigerator may not connect to internet directly, only home router
have direct access to internet.
So how to implement this mail sending feature?

Thanks.


On Mon, Jan 20, 2020 at 8:06 PM Mauricio Tavares 
wrote:

> On Sun, Jan 19, 2020 at 7:40 PM Wesley Peng  wrote:
> >
> > How to compile postfix into the  Embedded operating system (such as the
> home router) and make it as a mail gateway for  Smart home appliances?
> >
>   Main issue I have here is your router has a very important job
> to do (routing traffic) and you now are asking it to also do a rather
> demanding job (mail). If I were in your shoes, I would invest in
> either creating a container/virtual host or grabbing something on the
> size/cost/power of a raspberry pi and make that the mail server. Now,
> even if for some reason the internal mail server gets bogged down,
> your router (and connection to the outside and your other internal
> devices) will not be affected (in principle, because there is always
> DOS). Also, router updates will then be decoupled with the mail server
> ones.
>
> Divide and Conquer.
> >
> > Thank you.
>

postfix for IoT

[Wesley Peng]

How to compile postfix into the  Embedded operating system (such as the
home router) and make it as a mail gateway for  Smart home appliances?


Thank you.

ISP open relay

[Wesley Peng]

Hello

My ISP email even doesn’t require SMTP AUTH. Will they be acting as open
relay? How to stop abuse of outgoing mail?

Regards

Re: how to run regex calculation

[Wesley Peng]

what does (??{$1*$2}) means?

Thanks.

on 2020/1/9 12:49, Joseph He wrote:

I think $str =~ /(\d+)\s(\d+)\s(??{$1*$2})/   should do it
My Perl version is  v5.26.1

Re: how to run regex calculation

[Wesley Peng]

Hallo

on 2020/1/9 16:35, demerphq wrote:

$str=~/(\d+)\s(\d+)\s(\1*\2)/

$1 refers to the capture buffers from the last completed match, \1
inside of the pattern part of a regex refers to the capture buffer of
the currently matching regex.


This doesn't work too.

perl -Mstrict -le 'my $str = "2 3 6"; print "true" if 
$str=~/(\d+)\s(\d+)\s(\1*\2)/'


nothing printed.


Regards.

Re: how to improve throughput for long time running sessions

[Wesley Peng]

Any more details?

regards

on 2020/1/9 14:11, Mithun Bhattacharya wrote:
You need to split the request handler and the request processor. Receive 
the request in mod_perl and then queue it into a separate application 
which does the actual heavy lifting.

Re: what's difference between HTTP_HOST and SERVER_NAME

[Wesley Peng]

Hi

on 2020/1/9 14:14, Mithun Bhattacharya wrote:

This is for PHP but it still applies.

https://www.geeksforgeeks.org/what-is-the-difference-between-http_host-and-server_name-in-php/


This is good reference. Thanks.

Re: AW: Question on a rewrite rule

[Wesley Peng]

Hi

on 2020/1/9 14:27, Andreas Mock wrote:

You have to see in which virtualhost section this rule was put into.
I'm pretty sure it's only in the http section.


You are so right. they are put in plain HTTP section.
So requets were coming with HTTP, they will be rewriten to HTTPS, which 
is in another section, which doesn't have that rewrite config.


Regards.

gRPC implement question

[Wesley Peng]

Hello

is there a full gRPC implementation in perl rather than this protocal 
buffer library?


https://metacpan.org/pod/Google::ProtocolBuffers::Dynamic

Thanks.

DB connection pool management

[Wesley Peng]

Hallo

Is there DB connection pool management library like JDBC for mod_perl?
I know there is Apache::DBI, but that seems not the one who have pool 
capability.


Thanks & happy new year.

Regards.

how to improve throughput for long time running sessions

[Wesley Peng]

Hallo

We are running LR[1] and GBDT[2] and similar algorithm in MP2 handles.
For each request, there were about 1000 features as arguments passed 
into the handles, via HTTP POST.
The request will wait for about 100ms to get responses, coz the 
calculation is not cheap.

My question is, how can we improve the throughput by architecture
optimization?
Yes we know there are TFS[3] and RT[4] for prediction frameworks, but we 
didn't use Tensorflow yet.



[1] https://en.wikipedia.org/wiki/LR_parser
[2] https://en.wikipedia.org/wiki/Gradient_boosting
[3] https://www.tensorflow.org/tfx/guide/serving
[4] https://developer.nvidia.com/tensorrt


Thanks.

Question on a rewrite rule

[Wesley Peng]

Hello

After installation of Letsencrypt free SSL, it adds a rule into apache's 
config file:


RewriteCond %{SERVER_NAME} =sample.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]


I know it means when SERVER_NAME is sample.com, it rewrites the plain 
HTTP request to HTTPS request.


My question is, is this not a dead loop?

For HTTPS request, the SERVER_NAME is also matched, so it rewrites again 
and again...


My guess it's a dead rewrite loop, but it actually works fine.

I am not sure about this, please help.

Thanks.

what's difference between HTTP_HOST and SERVER_NAME

[Wesley Peng]

Hello

For HTTP headers, what's difference between HTTP_HOST and SERVER_NAME?

I was long time confused about them.

Thanks.

how to run regex calculation

[Wesley Peng]

Hello

Give the case I have a string,

$str = "2 3 6";

I want to match with:

true if $str =~ /(\d+)\s(\d+)\s($1*$2)/;

that's to say, the thrid column would be (firstCol * SecondCol).

How to write regex for this?

Thank you.

Question on catch all domains

[Wesley Peng]

Hi

I have a question needing your help.

I have a few domains, every domain has a contact email. 

How to configure postfix to accept every domain and every email on domain, and 
deliver them to a specific mailbox?

I meant, I don’t want to add the virtual domain one by one in postfix, but like 
to have a wildcard configuration for all domains. Even if yahoo points its MX 
to me, I will receive the messages normally.

Surely this won’t be a open relay, but all messages will be delivered into a 
specific local mailbox.

Thank you.
Regards 

Re: [dns-operations] Questions on private nameservers registration

[Wesley Peng]

Another more question, can I put DE glues into other domain registry's 
zone? For example, the COM's.


Regards.

on 2019/11/26 9:41, Wesley Peng wrote:

John,

on 2019/11/26 9:35, John W. O'Brien wrote:

Are ns{1,2}.wsly.de authoritative for wsly.de? Then glue is required in
DE. Otherwise probably not [0].


Yes I plan to setup ns{1,2}.wsly.de to be wsly.de's auth-nameservers.
Thank you for pointing out that.

Regards.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: Postfix stable release 3.4.8

[Wesley Peng]

Hi Wietse

on 2019/11/27 23:38, Wietse Venema wrote:

[An on-line version of this announcement will be available at
http://www.postfix.org/announcements/postfix-3.4.8.html]

Fixed in Postfix 3.4:


It's really nice to see postfix gets continuous development for these 
many years. Wish it becomes more and more stronger.


Thanks for your team's work.

regards.

Re: question on a SPF setting

[Wesley Peng]

Thanks.
While I am still not clear about what the description in mxtoolbox.
Can you give more details?

regards.

on 2019/11/27 16:08, patpro wrote:

On 2019-11-27 08:15, Wesley Peng wrote:

Hello

I saw myrambler.ru has a special setting for SPF:

myrambler.ru.   3599    IN  TXT "v=spf1
ip4:81.19.78.96/27 ip4:81.19.78.0/27 ip4:81.19.88.0/24
-exists:%{ir}.spf.rambler.ru ~all"

what does it mean for this part:

-exists:%{ir}.spf.rambler.ru



You'll find an explanation here:

https://mxtoolbox.com/SuperTool.aspx?action=spf%3amyrambler.ru=toolpage

patpro

question on a SPF setting

[Wesley Peng]

Hello

I saw myrambler.ru has a special setting for SPF:

myrambler.ru.   3599IN  TXT "v=spf1 
ip4:81.19.78.96/27 ip4:81.19.78.0/27 ip4:81.19.88.0/24 
-exists:%{ir}.spf.rambler.ru ~all"


what does it mean for this part:

-exists:%{ir}.spf.rambler.ru

Thank you.

Re: Forwarding mail without breaking SPF?

[Wesley Peng]

Hello

on 2019/11/27 12:20, Richard Damon wrote:

DMARC/SPF, which only validates to the From: header will break.


If the sender domain set up SPF to:

v=spf1 ip4:0.0.0.0/0 ~all

Will this pass through any SPF check?

regards.

Re: Reject Chinese mail

[Wesley Peng]

on 2019/11/27 10:00, 황병희 wrote:

Personally i read public mailing  lists' messages by Gmane. There is
example screenshot [0]. So i have no problem about that.


For customized spam policy, I found Tuffmail has a flexible interface.

Regards.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

Hi

on 2019/11/26 20:53, Jaroslaw Rafa wrote:

Sending mail out of a MTA is always on port 25. STARTTLS is used if
possible.


If using plain port 25, the messages are not secure enough for traffic.
From what I know there is a technology calling Traffic hijacking.

Regards.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

Hi

on 2019/11/26 19:27, Matus UHLAR - fantomas wrote:

...and there's no "starttls" on 465, that's what I meant "implicit".

while port 465 was assigned for SMTPS in January 2018, it's been used this
way on many sites/services for years (even decades)


How the traffic between big one's MTAs get through?
For example, gmail send messages to web.de via port 465 by SSL, or just 
plain port 25?


regards.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

on 2019/11/26 17:02, Matus UHLAR - fantomas wrote:
I would set up port 465 also. Note that TLS on 465 is implicit, while on 
587 is

explicit, so it's easier to allow unencrypted connections by a mistake on
587.


587 is also used for StartTLS, am I right?

regards.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

That look interesting. Do you provide a hosting plan Andre?

regards

on 2019/11/26 14:31, André Rodier wrote:

Hello, Bill.

I had the same concern a few years ago.

I have been self-hosting for more than a decade, and more recently, I
built this:

https://github.com/progmaticltd/homebox

This is oriented towards security and privacy, and include defence
mechanisms against remote and physical intrusion.

- All daemons are protected by AppArmor.
- The main drive is fully encrypted using LUKS, unlock with a Yubikey
locally or remotely using SSH.
- Implementation of latest standards, like DNSSEC, SSHFP, MTA-STS,
etc...
- Encrypted remote or local backups with borg, with jabber alerts.
- Everything coming from Debian repositories.
- Some bonus features, like Jabber, RoundCube, Zabbix, SOGo, gogs,
transmission, etc.

One feature you may find particularly useful, is a monthly report with
all the accesses, by country, ISP, hours:

https://homebox.readthedocs.io/en/dev/access-reports/


Real time alerts and/or blocking if you connect from a blacklisted IP
and various parameters.

Everything is tested using continuous integration with a Jenkins
server.

It is on Debian Stretch for now, but we will provide a buster version
next year.

I am currently working on a way to provide static IP address if you do
not have one...

Enjoy!

Kind regards,
André

On Tue, 2019-11-26 at 00:48 -0500, Bill Cole wrote:

On 25 Nov 2019, at 22:53, lists wrote:

> Security is privacy.

More precisely: Security includes privacy. Privacy is an essential
*PART
OF* security.

The remit requested by the OP is really too broad to answer on a
public
mailing list intended for discussion of a specific MTA (even though
Postfix would be a likely component...) because it could have very
different answers depending on the specific needs of a site and
issues
like scale, threat model, risk tolerances, and available resources.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

Hi

on 2019/11/26 10:22, lists wrote:

At a minimum, I would set it up to use port 587. Then block via firewall all 
the email ports other than port 25 all countries from which you will not be 
using the server.

Keep the attack surface small. For example don't provide for web based email.



Sorry I didn't talk about security. I pay attention to privacy, such as 
these ones, but run by myself.


https://restoreprivacy.com/secure-email/

Regards.

Re: [dns-operations] Questions on private nameservers registration

[Wesley Peng]

John,

on 2019/11/26 9:35, John W. O'Brien wrote:

Are ns{1,2}.wsly.de authoritative for wsly.de? Then glue is required in
DE. Otherwise probably not [0].


Yes I plan to setup ns{1,2}.wsly.de to be wsly.de's auth-nameservers.
Thank you for pointing out that.

Regards.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

how to setup a privacy oriented mailserver

[Wesley Peng]

Hi community,

I finally got a domain from registrar, if I want to run a privacy 
oriented mail server, what steps should I take?


For example, setup SSL over all, SPF, DKIM, DMARC, DNSSec, DoH, 
encrypted storage, app special pasword, secondary authentication?


Is there any guide for it?

Thanks in advance.

regards.

[dns-operations] Questions on private nameservers registration

[Wesley Peng]

Hello

If I want to run my own nameservers, saying they are:

ns1.wsly.de.   1.2.3.4
ns2.wsly.de.   5.6.7.8


Would I put the glues into DE's registry, or shall I put glues into all 
registries, including  COM, NET, INFO, ccTLD etc?


Thanks.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Questions about my domain's DNS

[Wesley Peng]

Thank you for instant support Peter. I love DENIC.

on 2019/11/25 22:38, Peter Koch wrote:

without knowing details about the registrar/reseller chain that you might be
using, informing the registrar of such a change is a prerequisite for the
delegation to change at the TLD level.  That means, the registrar will
change the respective entries in the TLD registry.  In the case of DE,
the current (sic!) cadence of zone publication is once per hour, which makes you
incur a delay of up to two ours in the worst case.  Meanwhile, your changes
have made it into the DE zone (as published trough the DE TLD nameservers).


Regards.
___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations

Re: [dns-operations] Questions about my domain's DNS

[Wesley Peng]

 
 

 Thanks for updating the info Victor.
 

 
 

 
 
>  
> On Nov 25, 2019 at 10:10 PM,   (mailto:ietf-d...@dukhovni.org)>  wrote:
>  
>  
>  
>  On Mon, Nov 25, 2019 at 02:56:51PM +0100, Elmar K. Bins wrote: 
>
> >   >  ;; AUTHORITY SECTION: 
> >   >  wsly.de. 86400 IN NS art.ns.cloudflare.com. 
> >   >  wsly.de. 86400 IN NS roxy.ns.cloudflare.com. 
> >  
> >  In order to update the records in "de" you need your domain provider to 
> > send 
> >  them an update of the nameservers. 
>
> More precisely, the registrar rather than the DNS operator when these 
> are different. But in this case no need, the .de glue has already been 
> updated: 
>
>  wsly.de. IN NS art.ns.cloudflare.com. 
>  wsly.de. IN NS roxy.ns.cloudflare.com. 
>
> and WHOIS reports: 
>
>  Domain: wsly.de 
>  Nserver: art.ns.cloudflare.com 
>  Nserver: roxy.ns.cloudflare.com 
>  Changed: 2019-11-25T13:20:29+01:00 
>
> -- 
>  Viktor. 
> ___ 
> dns-operations mailing list 
> dns-operations@lists.dns-oarc.net 
> https://lists.dns-oarc.net/mailman/listinfo/dns-operations 
>
>  ___
dns-operations mailing list
dns-operations@lists.dns-oarc.net
https://lists.dns-oarc.net/mailman/listinfo/dns-operations