Question about DMARC

[Wesley Peng]

Greetings,

When mail is relayed through mailing list, why the DMARC policy is 
possible to reject?


For example, I sent mail from x...@mail.ru to y...@googlegroups.com

Since mail.ru has the strictest DMARC policy, the recepients may choose 
to reject this mail which is relayed by googlegroups, the reason is due 
to DKIM or SPF fails.


So mailing list makes DKIM or SPF failed?

Thank you for your helps.

Regards.

Re: Question about DMARC

[Wesley Peng]

Richard Damon wrote:

The typical options for the mailing list are

1) Just not allow people from such domains to post to the list (the
reject option you mention)

2) Rewrite the from address from people from such a domain to be from
the domain of the list (often the list address). This is arguably
discouraged by the email RFCs, as the from address should indicate the
AUTHOR of the message, which is the original sender. It also can cause
problems with identifying who sent the message, and can corrupt peoples
address books if their program records that address as being associated
with the sender. It can also make it harder to reply just to the sender.

3) Rewrite the message by wrapping it as an attachment, with the outer
message being from the list. This has the problem that many clients
won't handle the message in a useful manner.


Thank you Richard.

The email I am using is with domain of mail.ru, which has the strictest 
DMARC policy setting.


So mailing list like postfix-users doesn't deliver my message to myself 
on this domain. And google groups rewrite the sender address to their 
own address.


I don't know why mail.ru has this setup, this seems unfriendly.

Thanks.

Re: Question about DMARC

[Wesley Peng]

Richard Damon wrote:

That is a question to ask them. Basically the strict DMARC policy is
designed for transactional email, where spoofing is a real danger. The
side effect of it is that addresses on such a domain really shouldn't be
used on mailing lists, or any other 3rd party senders not specifically
set up for that by the domain owner. For the proper usages of this, it
really isn't much of a problem, as the sorts of institutions that deal
with this sort of transactional mail, probably shouldn't be using that
same domain for less formal usages that tends to go with a mailing list.

The problems arise when a domain that doesn't really need that level of
protection adopts it for some reason, especially if they don't inform
their users of the implications of that decision.


Hello Richard,

If I am wrong, please forgive me.

Many ISP/Registrars provide email forwarding, I even had a pobox.com 
account which I used for 10+ years with just forwarding feature.


When a mail like mail.ru was relayed by those providers, it sounds easy 
to break SPF/DKIM, so the recepients may reject the message. This is not 
good practice for the sender, even for mail.ru itself.


Am I right?

regards.

Re: Question about DMARC

[Wesley Peng]

Richard Damon wrote:

  The
side effect of it is that addresses on such a domain really shouldn't be
used on mailing lists,


Thanks for pointing out this. I never knew it.
Now I changed my mail to fastmail account, which I owned it for many 
years. I just don't like its mobile app, it's just a web wrapper, not as 
good as gmail/mail.ru etc.

After I checked I think fastmail will do well on receiving mailing list.

Thanks.

Re: Question about DMARC

[Wesley Peng]

Hi

the mail I sent from mail.ru to this list got dropped, I didn’t get the message 
I sent.


On Fri, Nov 22, 2019, at 4:41 PM, Nick wrote:
> On 2019-11-22 04:21 GMT, Wesley Peng wrote:
> > The email I am using is with domain of mail.ru, which has the
> > strictest DMARC policy setting.
> > 
> > So mailing list like postfix-users doesn't deliver my message to
> > myself on this domain. And google groups rewrite the sender address
> > to their own address.
> > 
> > I don't know why mail.ru has this setup, this seems unfriendly.
> 
> All of your posts from mail.ru pass DMARC according to my instance of
> OpenDMARC. If mail.ru isn't returning your posts, it's probably
> nothing to do with DMARC. Perhaps you can ask them. I also have
> strict DMARC policy and no difficulty with this list.
> -- 
> Nick
> 

Re: Question about DMARC

[Wesley Peng]

I meant I didn’t get it in my mail.ru inbox. The other providers may or may not 
reject it. Thanks.

On Fri, Nov 22, 2019, at 5:52 PM, Wesley Peng wrote:
> Hi
> 
> the mail I sent from mail.ru to this list got dropped, I didn’t get the 
> message I sent.
> 
> 
> On Fri, Nov 22, 2019, at 4:41 PM, Nick wrote:
>> On 2019-11-22 04:21 GMT, Wesley Peng wrote:
>> > The email I am using is with domain of mail.ru, which has the
>> > strictest DMARC policy setting.
>> > 
>> > So mailing list like postfix-users doesn't deliver my message to
>> > myself on this domain. And google groups rewrite the sender address
>> > to their own address.
>> > 
>> > I don't know why mail.ru has this setup, this seems unfriendly.
>> 
>> All of your posts from mail.ru pass DMARC according to my instance of
>> OpenDMARC. If mail.ru isn't returning your posts, it's probably
>> nothing to do with DMARC. Perhaps you can ask them. I also have
>> strict DMARC policy and no difficulty with this list.
>> -- 
>> Nick
>> 
> 

Re: Question about DMARC

[Wesley Peng]

Would this list break SPF then? Thanks 

On Fri, Nov 22, 2019, at 7:15 PM, Richard Damon wrote:
> On 11/21/19 11:47 PM, Wesley Peng wrote:
> > Richard Damon wrote:
> >> That is a question to ask them. Basically the strict DMARC policy is
> >> designed for transactional email, where spoofing is a real danger. The
> >> side effect of it is that addresses on such a domain really shouldn't be
> >> used on mailing lists, or any other 3rd party senders not specifically
> >> set up for that by the domain owner. For the proper usages of this, it
> >> really isn't much of a problem, as the sorts of institutions that deal
> >> with this sort of transactional mail, probably shouldn't be using that
> >> same domain for less formal usages that tends to go with a mailing list.
> >>
> >> The problems arise when a domain that doesn't really need that level of
> >> protection adopts it for some reason, especially if they don't inform
> >> their users of the implications of that decision.
> >
> > Hello Richard,
> >
> > If I am wrong, please forgive me.
> >
> > Many ISP/Registrars provide email forwarding, I even had a pobox.com
> > account which I used for 10+ years with just forwarding feature.
> >
> > When a mail like mail.ru was relayed by those providers, it sounds
> > easy to break SPF/DKIM, so the recepients may reject the message. This
> > is not good practice for the sender, even for mail.ru itself.
> >
> > Am I right?
> >
> > regards.
> >
> Normal forwarding will break SPF, but not DKIM (one reason DMARC uses
> both). A mail provider that uses strict settings but doesn't DKIM sign
> the messages would be considered seriously broken in my experience. The
> issue is that many mailing list will break DKIM by slightly modifing the
> message, like adding a signal word to the subject or a footer with
> information like unsubscribing instructions (this can be a legal
> requirement in some jurisdictions). Note, this list does NOT do this
> sort of modification, so doesn't cause that sort of problem.
> 
> -- 
> Richard Damon
> 
> 

Re: Reject Chinese mail

[Wesley Peng]

SA (Spamassassin) is good idea, I saw most people running their own mail 
servers are using it.


On Sat, Nov 23, 2019, at 4:35 AM, Ralph Seichter wrote:
> * merr...@fn.de:
> 
> > We did get a lot of spam messages from Chinese providers. We speak not
> > Chinese, do you think if it is possible to reject all mails from
> > China?
> 
> SpamAssassin, which is often used in combination with Postfix, has a
> plugin called "RelayCountry" that allows you to change the spam score of
> email. It uses GeoIP and is therefore not always accurate, but overall
> it can help.
> 
> There is also the "TextCat" plugin that attempts to determine the
> language of email bodies, and it allows you to adjust spam scores based
> on wanted/unwanted languages.
> 
> Personally, I think that hard blocks based on these plugins are not a
> good idea, but if your business is not set up to handle communication
> written in language X (e.g. because none of your employees speak X),
> adjusting the spam score seems reasonable.
> 
> -Ralph
> 

Validation DMARC

[Wesley Peng]

Hi

when validating DMARC, it use the envelop address, or use from address from the 
header? Thanks 

Re: Question about DMARC

[Wesley Peng]

Thanks for helps.

On Sat, Nov 23, 2019, at 11:07 AM, Richard Damon wrote:
> On 11/22/19 6:25 AM, Wesley Peng wrote:
> > Would this list break SPF then? Thanks 
> >
> This list sends with an envelope sender in the lists domain, so it
> doesn't break general SPF, it will break DMARC SPF, since that check SPF
> only to the From: domain.
> 
> This list doesn't modify messages in a way to break DKIM, so messages
> that were DKIM signed to the From: Domain will still pass DMARC DKIM, so
> will pass DMARC (unless the domain doesn't DKIM sign messages, which
> would be very unusual for highly restricted DMARC).
> 
> -- 
> Richard Damon
> 
> 

Re: Validation DMARC

[Wesley Peng]

Google groups replace the from: with their group address. What I know the big 
providers having strict DMARC setting are:

mail.ru
laposte.net

I am glad the more large providers like gmail, outlook don’t have this stupid 
setting.

Regards

On Sat, Nov 23, 2019, at 5:13 PM, Roland Köbler wrote:
> Hi,
> 
> > when validating DMARC, it use the envelop address, or use from address from 
> > the header?
> it unfortunately uses the from-header.
> (If it would use the envelope address, it would not cause that much
> problems.)
> 
> Or in short: DMARC intentionally breaks every mailinglist and every 
> mail-forwarding.
> So, if a mail-provider uses a strict DMARC-policy, it effectively
> says: "Our mail-addresses may not be used for mailinglists."
> 
> The cleanest solution for mailinglists would be to reject mails from
> such adresses. (Spoofing the From-header by removing the authors address
> and replacing it by the lists address, and so hiding the original author,
> could of course also be done, but is not really a good solution.)
> 
> Roland
> 
> 

Re: Validation DMARC

[Wesley Peng]

Hello

if you have used a mail.ru email for google groups, when you posted message to 
group, it will replace From header with the list address.


On Sat, Nov 23, 2019, at 6:43 PM, Jaroslaw Rafa wrote:
> Dnia 23.11.2019 o godz. 17:19:53 Wesley Peng pisze:
> > Google groups replace the from: with their group address.
> 
> I have never seen it and I'm subscribed to many Google-based mailing lists.
> They replace the envelope from address (like almost every mailing list
> server does), but keep the original From: header.
> 
> Replacing the From: header would be a very bad idea, as - at it was already
> written - this header indicates the author of the message, and the author is
> a particular sender writing to the list, and not the list itself.
> 
> Would you really like to see in your mail client a whole thread of messages
> from a mailing list, every each of them having "From:" address as the list's
> address? What would you do if you need to quickly find a message written by
> a particular person (for example, you) in this conversation?
> -- 
> Regards,
>  Jaroslaw Rafa
>  r...@rafa.eu.org
> --
> "In a million years, when kids go to school, they're gonna know: once there
> was a Hushpuppy, and she lived with her daddy in the Bathtub."
> 

Re: Reject Chinese mail

[Wesley Peng]

I totally agreed with you @Rafa.

btw, is there any good reputation, strict standard email hosting for 
suggestions? I currently use fastmail, it is good for personal usage, but I 
heard some privacy problems for commercial use. 

thanks.

On Sat, Nov 23, 2019, at 6:35 PM, Jaroslaw Rafa wrote:
> Dnia 23.11.2019 o godz. 10:59:24 황병희 pisze:
> > 
> > How about moving to Gmail(Google Apps)?
> 
> If someone is running their own mail server, do not ask them to move to
> Gmail. That's what the big players like Google want - that everyone uses
> their service and there are no more small, independent servers on the
> Internet. We should defend the de-centralized Internet, not help big players
> to make it more centralized.
> -- 
> Regards,
>  Jaroslaw Rafa
>  r...@rafa.eu.org
> --
> "In a million years, when kids go to school, they're gonna know: once there
> was a Hushpuppy, and she lived with her daddy in the Bathtub."
> 

Re: Validation DMARC

[Wesley Peng]

I’m not sure , you may refer this discussion,

https://serverfault.com/questions/779730/why-dont-my-domains-messages-to-a-google-group-get-their-headers-rewritten-so


On Sat, Nov 23, 2019, at 7:23 PM, Jaroslaw Rafa wrote:
> Dnia 23.11.2019 o godz. 19:10:51 Wesley Peng pisze:
> > 
> > if you have used a mail.ru email for google groups, when you posted
> > message to group, it will replace From header with the list address.
> 
> Does it re-sign the message then? Because replacing the From: header would
> break DKIM, as this header is always signed...
> -- 
> Regards,
>  Jaroslaw Rafa
>  r...@rafa.eu.org
> --
> "In a million years, when kids go to school, they're gonna know: once there
> was a Hushpuppy, and she lived with her daddy in the Bathtub."
> 

Re: Reject Chinese mail

[Wesley Peng]

Or maybe block them by ESP? I saw there is a Perl module listing those big 
providers in China.

https://metacpan.org/pod/Data::ChineseESP

regards 

On Sat, Nov 23, 2019, at 7:33 PM, Rafael Azevedo wrote:
> I've blocked the entire ASIA netblocks in my ASN.
> We don't exchange any information with that part of the world, neither any of 
> our customers.
> All we get from that part of the world is DDoS attacks, brute force attacks 
> and spam.
> Sorry for those who don't agree with me, its ok, but I got tired of being 
> attacked and having this old type of thought "blocking China is blocking 
> money" or maybe "There are good people there as well".
> Over 20 years working with internet and the only thing that came to me from 
> China is my Macbook.
> Good people always end up paying for the bad ones. That's how world works.
> By the way, you can find ASIA NETBLOCK on the internet and block them all 
> easily.
> Good luck.
> BR,
> 
> Em sex., 22 de nov. de 2019 às 23:00, 황병희  escreveu:
>> merr...@fn.de writes:
>> 
>>  > [...] do you think if it is possible to reject all mails from China? 
>> Thanks 
>> 
>>  How about moving to Gmail(Google Apps)? Gmail's spam defense is not bad, i
>>  think. Plus don't block China. Blocking China is blocking money.
>> 
>>  Sincerely,
>> 
>>  -- 
>>  ^고맙습니다 _地平天成_ 감사합니다_^))//

Re: Validation DMARC

[Wesley Peng]

Why it doesn’t break From: header SPF? Just curious 

On Mon, Nov 25, 2019, at 4:12 AM, Chris Wedgwood wrote:
> > Or in short: DMARC intentionally breaks every mailinglist and every
> > mail-forwarding. So, if a mail-provider uses a strict DMARC-policy,
> > it effectively says: "Our mail-addresses may not be used for
> > mailinglists."
> 
> this message (i am replying to) from you on this mailing list is not
> broken
> 

Re: Validation DMARC

[Wesley Peng]

That's great explation. Thanks Richard.

On Mon, Nov 25, 2019, at 7:33 AM, Richard Damon wrote:
> On 11/24/19 6:21 PM, Wesley Peng wrote:
> > Why it doesn’t break From: header SPF? Just curious 
> >
> > On Mon, Nov 25, 2019, at 4:12 AM, Chris Wedgwood wrote:
> >> > Or in short: DMARC intentionally breaks every mailinglist and every
> >> > mail-forwarding. So, if a mail-provider uses a strict DMARC-policy,
> >> > it effectively says: "Our mail-addresses may not be used for
> >> > mailinglists."
> >>
> >> this message (i am replying to) from you on this mailing list is not
> >> broken
> >>
> It DOES break DMARC/SPF, as the IP address the message comes from
> doesn't match the From of the message, but with DMARC if EITHER SPF or
> DKIM pass, the message is to be considered to pass.
> 
> A Domain with strict DMARC, and which doesn't DKIM sign messages, will
> fail with any form of remailer, so would fail for this application.
> 
> -- 
> Richard Damon
> 
> 

how to setup a privacy oriented mailserver

[Wesley Peng]

Hi community,

I finally got a domain from registrar, if I want to run a privacy 
oriented mail server, what steps should I take?


For example, setup SSL over all, SPF, DKIM, DMARC, DNSSec, DoH, 
encrypted storage, app special pasword, secondary authentication?


Is there any guide for it?

Thanks in advance.

regards.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

Hi

on 2019/11/26 10:22, lists wrote:

At a minimum, I would set it up to use port 587. Then block via firewall all 
the email ports other than port 25 all countries from which you will not be 
using the server.

Keep the attack surface small. For example don't provide for web based email.



Sorry I didn't talk about security. I pay attention to privacy, such as 
these ones, but run by myself.


https://restoreprivacy.com/secure-email/

Regards.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

That look interesting. Do you provide a hosting plan Andre?

regards

on 2019/11/26 14:31, André Rodier wrote:

Hello, Bill.

I had the same concern a few years ago.

I have been self-hosting for more than a decade, and more recently, I
built this:

https://github.com/progmaticltd/homebox

This is oriented towards security and privacy, and include defence
mechanisms against remote and physical intrusion.

- All daemons are protected by AppArmor.
- The main drive is fully encrypted using LUKS, unlock with a Yubikey
locally or remotely using SSH.
- Implementation of latest standards, like DNSSEC, SSHFP, MTA-STS,
etc...
- Encrypted remote or local backups with borg, with jabber alerts.
- Everything coming from Debian repositories.
- Some bonus features, like Jabber, RoundCube, Zabbix, SOGo, gogs,
transmission, etc.

One feature you may find particularly useful, is a monthly report with
all the accesses, by country, ISP, hours:

https://homebox.readthedocs.io/en/dev/access-reports/


Real time alerts and/or blocking if you connect from a blacklisted IP
and various parameters.

Everything is tested using continuous integration with a Jenkins
server.

It is on Debian Stretch for now, but we will provide a buster version
next year.

I am currently working on a way to provide static IP address if you do
not have one...

Enjoy!

Kind regards,
André

On Tue, 2019-11-26 at 00:48 -0500, Bill Cole wrote:

On 25 Nov 2019, at 22:53, lists wrote:

> Security is privacy.

More precisely: Security includes privacy. Privacy is an essential
*PART
OF* security.

The remit requested by the OP is really too broad to answer on a
public
mailing list intended for discussion of a specific MTA (even though
Postfix would be a likely component...) because it could have very
different answers depending on the specific needs of a site and
issues
like scale, threat model, risk tolerances, and available resources.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

on 2019/11/26 17:02, Matus UHLAR - fantomas wrote:
I would set up port 465 also. Note that TLS on 465 is implicit, while on 
587 is

explicit, so it's easier to allow unencrypted connections by a mistake on
587.


587 is also used for StartTLS, am I right?

regards.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

Hi

on 2019/11/26 19:27, Matus UHLAR - fantomas wrote:

...and there's no "starttls" on 465, that's what I meant "implicit".

while port 465 was assigned for SMTPS in January 2018, it's been used this
way on many sites/services for years (even decades)


How the traffic between big one's MTAs get through?
For example, gmail send messages to web.de via port 465 by SSL, or just 
plain port 25?


regards.

Re: how to setup a privacy oriented mailserver

[Wesley Peng]

Hi

on 2019/11/26 20:53, Jaroslaw Rafa wrote:

Sending mail out of a MTA is always on port 25. STARTTLS is used if
possible.


If using plain port 25, the messages are not secure enough for traffic.
From what I know there is a technology calling Traffic hijacking.

Regards.

Re: Reject Chinese mail

[Wesley Peng]

on 2019/11/27 10:00, 황병희 wrote:

Personally i read public mailing  lists' messages by Gmane. There is
example screenshot [0]. So i have no problem about that.


For customized spam policy, I found Tuffmail has a flexible interface.

Regards.

Re: Forwarding mail without breaking SPF?

[Wesley Peng]

Hello

on 2019/11/27 12:20, Richard Damon wrote:

DMARC/SPF, which only validates to the From: header will break.


If the sender domain set up SPF to:

v=spf1 ip4:0.0.0.0/0 ~all

Will this pass through any SPF check?

regards.

question on a SPF setting

[Wesley Peng]

Hello

I saw myrambler.ru has a special setting for SPF:

myrambler.ru.   3599IN  TXT "v=spf1 
ip4:81.19.78.96/27 ip4:81.19.78.0/27 ip4:81.19.88.0/24 
-exists:%{ir}.spf.rambler.ru ~all"


what does it mean for this part:

-exists:%{ir}.spf.rambler.ru

Thank you.

Re: question on a SPF setting

[Wesley Peng]

Thanks.
While I am still not clear about what the description in mxtoolbox.
Can you give more details?

regards.

on 2019/11/27 16:08, patpro wrote:

On 2019-11-27 08:15, Wesley Peng wrote:

Hello

I saw myrambler.ru has a special setting for SPF:

myrambler.ru.   3599    IN  TXT "v=spf1
ip4:81.19.78.96/27 ip4:81.19.78.0/27 ip4:81.19.88.0/24
-exists:%{ir}.spf.rambler.ru ~all"

what does it mean for this part:

-exists:%{ir}.spf.rambler.ru



You'll find an explanation here:

https://mxtoolbox.com/SuperTool.aspx?action=spf%3amyrambler.ru&run=toolpage

patpro

Re: Postfix stable release 3.4.8

[Wesley Peng]

Hi Wietse

on 2019/11/27 23:38, Wietse Venema wrote:

[An on-line version of this announcement will be available at
http://www.postfix.org/announcements/postfix-3.4.8.html]

Fixed in Postfix 3.4:


It's really nice to see postfix gets continuous development for these 
many years. Wish it becomes more and more stronger.


Thanks for your team's work.

regards.

Question on catch all domains

[Wesley Peng]

Hi

I have a question needing your help.

I have a few domains, every domain has a contact email. 

How to configure postfix to accept every domain and every email on domain, and 
deliver them to a specific mailbox?

I meant, I don’t want to add the virtual domain one by one in postfix, but like 
to have a wildcard configuration for all domains. Even if yahoo points its MX 
to me, I will receive the messages normally.

Surely this won’t be a open relay, but all messages will be delivered into a 
specific local mailbox.

Thank you.
Regards 

ISP open relay

[Wesley Peng]

Hello

My ISP email even doesn’t require SMTP AUTH. Will they be acting as open
relay? How to stop abuse of outgoing mail?

Regards

postfix for IoT

[Wesley Peng]

How to compile postfix into the  Embedded operating system (such as the
home router) and make it as a mail gateway for  Smart home appliances?


Thank you.

Re: postfix for IoT

[Wesley Peng]

Hello

Thanks all answers for kind info.
My requirement is, for example, when refrigerator found there were no food
in itself, it will send an email to remind the people.
But refrigerator may not connect to internet directly, only home router
have direct access to internet.
So how to implement this mail sending feature?

Thanks.


On Mon, Jan 20, 2020 at 8:06 PM Mauricio Tavares 
wrote:

> On Sun, Jan 19, 2020 at 7:40 PM Wesley Peng  wrote:
> >
> > How to compile postfix into the  Embedded operating system (such as the
> home router) and make it as a mail gateway for  Smart home appliances?
> >
>   Main issue I have here is your router has a very important job
> to do (routing traffic) and you now are asking it to also do a rather
> demanding job (mail). If I were in your shoes, I would invest in
> either creating a container/virtual host or grabbing something on the
> size/cost/power of a raspberry pi and make that the mail server. Now,
> even if for some reason the internal mail server gets bogged down,
> your router (and connection to the outside and your other internal
> devices) will not be affected (in principle, because there is always
> DOS). Also, router updates will then be decoupled with the mail server
> ones.
>
> Divide and Conquer.
> >
> > Thank you.
>

Re: Postfix stable release 3.5.0

[Wesley Peng]

Nice work on the release. Thanks Wietse.

Mar 17, 2020, 21:39 by emaw...@gmail.com:

> Hello! excellent work, the option to bounce emails manually is very useful 
> for me.
>
> Never perform the update through the source code. I would like to know if I 
> can get the src.rpm for Centos?
>
>
>
> El lun., 16 de mar. de 2020 a la(s) 11:18, Wietse Venema (> 
> wie...@porcupine.org> ) escribió:
>
>> [An on-line version of this announcement will be available at
>>  >> http://www.postfix.org/announcements/postfix-3.5.0.html>> ]
>>  
>>  Postfix stable release 3.5.0 is available. Support has ended for
>>  legacy release Postfix 3.1.
>>  
>>  The main changes are below. See the RELEASE_NOTES file for further details.
>>  
>>    * Support for the haproxy v2 protocol. The Postfix implementation
>>      supports TCP over IPv4 and IPv6, as well as non-proxied
>>      connections; the latter are typically used for heartbeat tests.
>>  
>>    * Support to force-expire email messages. This introduces new
>>      postsuper(1) command-line options to request expiration, and
>>      additional information in mailq(1) or postqueue(1) output.
>>  
>>    * The Postfix SMTP and LMTP client support a list of nexthop
>>      destinations separated by comma or whitespace. These destinations
>>      will be tried in the specified order. Examples:
>>  
>>      /etc/postfix/>> main.cf >> :
>>          relayhost = foo.example, bar.example
>>          default_transport = smtp:foo.example, bar.example
>>  
>>  Incompatible changes:
>>  
>>    * Logging: Postfix daemon processes now log the from= and to=
>>      addresses in external (quoted) form in non-debug logging (info,
>>      warning, etc.). This means that when an address localpart
>>      contains spaces or other special characters, the localpart will
>>      be quoted, for example:
>>  
>>          from=<"name with spaces"@>> example.com >> >
>>  
>>      Specify "info_log_address_format = internal" for backwards 
>> compatibility.
>>  
>>    * Postfix now normalizes IP addresses received with XCLIENT,
>>      XFORWARD, or with the HaProxy protocol, for consistency with
>>      direct connections to Postfix. This may change the appearance
>>      of logging, and the way that check_client_access will match
>>      subnets of an IPv6 address.
>>  
>>  You can find the updated Postfix source code at the mirrors listed
>>  at >> http://www.postfix.org/>> .
>>  
>>          Wietse
>>

should we use plaintext for message?

[Wesley Peng]

Hello

Following this guide:
https://useplaintext.email/

Shall we use plaintext message in regular email communication?

Thanks

[OT] looking for a good mobile MUA

[Wesley Peng]

Hello

On desktop I always use Thunderbird for MUA.

Thunderbird is clean and lite, handling protocol well.

But for mobile (I primarily use iOS), I couldn't find a good MUA to use.

I tried almost every release on appStore (outlook, edison, bluemail, 
spark, mymail...), they are either too complicated to use, or not 
following email standard (for example, can't choose plaintext composer, 
or can't setup sender auth username different from the email address, or 
never handle SSL correctly), or showing the message body ugly.


Can you suggest a suitable mobile MUA? it should like thunderbird, be 
fast, clean, standard, and easy to use.


Thanks.
(sorry for this OT message)

learning resources for perl6 beginner

[Wesley Peng]

Hello,

I am a programmer most time writing code with c/perl 5/ruby.

I know perl6 is coming to the world, have the interest to take a glance on
it.

For beginner of perl6, can you suggest some resources including online
documentation or books to get start?

thanks & regards
Wesley

Re: learning resources for perl6 beginner

[Wesley Peng]

Thank you William. That's awesome!

On Sun, Aug 25, 2019 at 12:07 AM William Michels 
wrote:

> Hi Wesley,
>
> Andy's being modest. In addition to the official Perl6 docs at:
> http://docs.perl6.org (as Andy suggests), Check out his many Perl6
> resources (including videos) at:
>
> https://perl6.online/
> https://perl6.online/contents/
> https://perl6.online/category/talks/
>
> I'm using Andy Shitov's "Perl6 One-Liners" slides to teach myself
> Perl6 on the command-line, as a modern replacement for sed/awk:
>
> https://perl6.online/2019/03/08/perl-6-one-liners-slides/
>
> https://perl6.online/2018/12/20/using-command-line-options-in-perl-6-one-liners/
>
> https://perl6.online/2018/12/24/overview-of-the-perl-6-one-liner-advent-calendar-2018/
>
> Also check out David Farrell's "Perl6-One-Liners" Github repo:
>
> https://github.com/dnmfarrell/Perl6-One-Liners#tutorial
>
> Finally, take a look Naoum Hankache's "Perl6Intro" site, and
> associated Github repo. As an added bonus, Naoum's site is available
> in thirteen (13) languages:
>
> https://perl6intro.com
> https://github.com/hankache/perl6intro
>
> This just scratches the surface (apologies to anyone I've overlooked),
>
> HTH, Bill.
>
>
>
> On Sat, Aug 24, 2019 at 4:15 AM Andrew Shitov  wrote:
> >
> > https://perl6.online/perl6-at-a-glance/
> >
> > http://docs.perl6.org
> >
> >
> > On Sat, 24 Aug 2019 at 13:13, Wesley Peng  wrote:
> >>
> >> Hello,
> >>
> >> I am a programmer most time writing code with c/perl 5/ruby.
> >>
> >> I know perl6 is coming to the world, have the interest to take a glance
> on it.
> >>
> >> For beginner of perl6, can you suggest some resources including online
> documentation or books to get start?
> >>
> >> thanks & regards
> >> Wesley
> >
> > --
> > Andrew Shitov
> > __
> > a...@shitov.ru | http://shitov.ru
>

Re: learning resources for perl6 beginner

[Wesley Peng]

Thanks for all great books and their authors.
I downloaded these two and have been reading them.

https://deeptext.media/i/perl6-at-a-glance/perl6-at-a-glance-downloadable.pdf
https://github.com/LaurentRosenfeld/thinkperl6/raw/master/PDF/thinkperl6.pdf

regards.

On Sun, Aug 25, 2019 at 2:50 AM Laurent Rosenfeld <
laurent.rosenf...@googlemail.com> wrote:

> Just one additional point:
>
> *Think Perl 6* is available in printed form (e.g. from Amazon), but is
> also available for free download in PDF format:
> https://greenteapress.com/wp/think-perl-6/
>
> Le sam. 24 août 2019 à 20:32, Parrot Raiser <1parr...@gmail.com> a écrit :
>
>> Some books:
>>
>> "Think Perl 6"
>> http://shop.oreilly.com/product/0636920065883.do
>> "Learning Perl 6"http://shop.oreilly.com/product/0636920062776.do
>> "Perl 6 At a Glance"   https://perl6.online/perl6-at-a-glance/ (which
>> Andrew did mention)
>>
>> On 8/24/19, William Michels via perl6-users  wrote:
>> > Hi Wesley,
>> >
>> > Andy's being modest. In addition to the official Perl6 docs at:
>> > http://docs.perl6.org (as Andy suggests), Check out his many Perl6
>> > resources (including videos) at:
>> >
>> > https://perl6.online/
>> > https://perl6.online/contents/
>> > https://perl6.online/category/talks/
>> >
>> > I'm using Andy Shitov's "Perl6 One-Liners" slides to teach myself
>> > Perl6 on the command-line, as a modern replacement for sed/awk:
>> >
>> > https://perl6.online/2019/03/08/perl-6-one-liners-slides/
>> >
>> https://perl6.online/2018/12/20/using-command-line-options-in-perl-6-one-liners/
>> >
>> https://perl6.online/2018/12/24/overview-of-the-perl-6-one-liner-advent-calendar-2018/
>> >
>> > Also check out David Farrell's "Perl6-One-Liners" Github repo:
>> >
>> > https://github.com/dnmfarrell/Perl6-One-Liners#tutorial
>> >
>> > Finally, take a look Naoum Hankache's "Perl6Intro" site, and
>> > associated Github repo. As an added bonus, Naoum's site is available
>> > in thirteen (13) languages:
>> >
>> > https://perl6intro.com
>> > https://github.com/hankache/perl6intro
>> >
>> > This just scratches the surface (apologies to anyone I've overlooked),
>> >
>> > HTH, Bill.
>> >
>> >
>> >
>> > On Sat, Aug 24, 2019 at 4:15 AM Andrew Shitov  wrote:
>> >>
>> >> https://perl6.online/perl6-at-a-glance/
>> >>
>> >> http://docs.perl6.org
>> >>
>> >>
>> >> On Sat, 24 Aug 2019 at 13:13, Wesley Peng  wrote:
>> >>>
>> >>> Hello,
>> >>>
>> >>> I am a programmer most time writing code with c/perl 5/ruby.
>> >>>
>> >>> I know perl6 is coming to the world, have the interest to take a
>> glance
>> >>> on it.
>> >>>
>> >>> For beginner of perl6, can you suggest some resources including online
>> >>> documentation or books to get start?
>> >>>
>> >>> thanks & regards
>> >>> Wesley
>> >>
>> >> --
>> >> Andrew Shitov
>> >> __
>> >> a...@shitov.ru | http://shitov.ru
>> >
>>
>

Re: [ please ] request for perl machine learning utils development

[Wesley Peng]

Hi

on 2019/8/29 15:56, Fernando Santagata wrote:
I started working on Tensorflow, but I've been sidetracked by another 
more urgent project, libxml2.
I don't know when I will be able to work on Tensorflow again, since 
libxml2 is quite a large library (and Tensorflow C API, besides being 
large, is not adequately documented).


Nice to hear that.
 Perl even have the ability to make its own ML framework based on the 
great PDL.


regards,
Wesley

Re: perl.com's registration appears to have been hijacked

[Wesley Peng]

Nice sender domain. You would better sell it to ruby language developer.

On Sun, Jan 31, 2021, at 5:20 AM, Earl Ruby wrote:
> Re-sending using the "from" address that's subscribed to the list... 
> 
> On Fri, Jan 29, 2021 at 3:07 PM Earl Ruby  wrote:
>> A glance at the domain records show the contact information is now "REDACTED 
>> FOR PRIVACY". Prior to the change Tom Christiansen was listed as the domain 
>> administrative contact.
>> 
>> https://www.theregister.com/2021/01/28/perl_hijacking/
>> 
>> https://nakedsecurity.sophos.com/2021/01/29/the-mystery-of-the-missing-perl-webite/
>> 
> 
> -- 
> Earl Ruby
> http://earlruby.org/
> http://www.linkedin.com/in/earlruby
> https://twitter.com/earlruby

Re: dimensions in a multidimensional array

[Wesley Peng]

When will we have p6’s Numpy?


On Sat, Feb 6, 2021, at 2:04 AM, Brad Gilbert wrote:
> There is a reason that you can't just ask for the dimensions of an 
> unspecified multidimensional array.
> It may be multiple dimensions.
> 
> [[1,2,3],
> [4,5,6,7,8,9,10]].shape
> 
> If it gave a result it would be something like:
> 
> (2,3|7)
> 
> On Fri, Feb 5, 2021 at 8:50 AM Theo van den Heuvel  
> wrote:
>> Hi gurus,
>> 
>> I cannot seem to find an idiomatic way to get the dimensions of a 
>> multidimensional array,
>> other than by looking at the size of the first row and column, with 
>> @m[0;*].elems and @m[*;0].elems.
>> Am I missing something in the docs?
>> 
>> Thanks,
>> 
>> -- 
>> Theo van den Heuvel
>> 

Re: This is a test

[Wesley Peng]

Please don't send a test message to the public list which was read by
thousands of people. thanks.

On Fri, Jan 8, 2021 at 5:26 AM Craig Hatch  wrote:

> I have added you to the EMAIL  list, so when I have questions.
>
> Just learn for fun.
>
>
> Craig Hatch
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.6 on Windows - does a python3 alias get created by installation?

[Wesley Peng]

smart.thanks.

On 2019/10/12 12:19 下午, Gisle Vanem wrote:


An "alias" could also simply be created using:
   doskey python3=f:\ProgramFiles\Python36\python.exe

--
https://mail.python.org/mailman/listinfo/python-list

Re: [RELEASE] Python 3.8.0 is now available

[Wesley Peng]

awesome news. thanks for all the hard work.

on 2019/10/15 4:23, Łukasz Langa wrote:

We hope you enjoy Python 3.8!

Thanks to all of the many volunteers who help make Python Development and these 
releases possible! Please consider supporting our efforts by volunteering 
yourself or through organization contributions to the Python Software 
Foundation.

--
https://mail.python.org/mailman/listinfo/python-list

Re: A news aggregator for the Python community

[Wesley Peng]

Sebastian Steins wrote:

Over the last few weeks I've build a hacker news clone for the Python community:

https://news.python.sc

The source is at github.com/sebst/pythonic-news


that looks interesting. thanks.
--
https://mail.python.org/mailman/listinfo/python-list

Re: I need to study Python

[Wesley Peng]

Pick a book like Core Python Programming, read it and do coding follow it.

regards.

sinanp...@gmail.com wrote:

Hey, I'm a completely noob.
I want to learn python, how can i or where can i study python?


--
https://mail.python.org/mailman/listinfo/python-list

Re: [squid-users] HEAD requests: pass through?

[Wesley Peng]

on 2019/9/4 17:54, fansari wrote:

Is it possible to configure the squid in a way that such requests are not
answered by the squid itself but passed through to the internet? Because it
may happen that the content on the internet has changed - in this case the
client would compare against the older content from the suqid cache (this
should be avoided).


Can you setup squid to no-cache such HEAD requests?

regards.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [VOTE] Apache CloudStack 4.14.0.0 RC3

[Wesley Peng]

Andrija Panic wrote:

http://docs.cloudstack.apache.org/en/latest/upgrading/upgrade/upgrade-4.13.html#time-zone-requirements


handled in the Upgrade docs - exactly the same..



[OT] is there a cloudstack private IAAS deployment solution? not that 
large scale, most for storage. if there is the commercial solution, we 
can pay for it.


Thanks.

Re: flink sql row_number() over () OOM

[Wesley Peng]

Hi

on 2019/9/4 19:30, liu ze wrote:
I use the row_number() over() function to do topN, the total amount of 
data is 60,000, and the state is 12G .

Finally, oom, is there any way to optimize it?


ref: 
https://stackoverflow.com/questions/50812837/flink-taskmanager-out-of-memory-and-memory-configuration


The total amount of required physical and heap memory is quite difficult 
to compute since it strongly depends on your user code, your job's 
topology and which state backend you use.


As a rule of thumb, if you experience OOM and are still using the 
FileSystemStateBackend or the MemoryStateBackend, then you should switch 
to RocksDBStateBackend, because it can gracefully spill to disk if the 
state grows too big.


If you are still experiencing OOM exceptions as you have described, then 
you should check your user code whether it keeps references to state 
objects or generates in some other way large objects which cannot be 
garbage collected. If this is the case, then you should try to refactor 
your code to rely on Flink's state abstraction, because with RocksDB it 
can go out of core.


RocksDB itself needs native memory which adds to Flink's memory 
footprint. This depends on the block cache size, indexes, bloom filters 
and memtables. You can find out more about these things and how to 
configure them here.


Last but not least, you should not activate 
taskmanager.memory.preallocate when running streaming jobs, because 
streaming jobs currently don't use managed memory. Thus, by activating 
preallocation, you would allocate memory for Flink's managed memory 
which is reduces the available heap space.

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

[Wesley Peng]

On 2019/9/6 8:55 下午, Fabian Hueske wrote:

I'm very happy to announce that Kostas Kloudas is joining the Flink PMC.
Kostas is contributing to Flink for many years and puts lots of effort 
in helping our users and growing the Flink community.


Please join me in congratulating Kostas!


congratulation Kostas!

regards.

Re: [flink-1.9] how to read local json file through Flink SQL

[Wesley Peng]

On 2019/9/8 5:40 下午, Anyang Hu wrote:
In flink1.9, is there a way to read local json file in Flink SQL like 
the reading of csv file?


hi,

might this thread help you?
http://mail-archives.apache.org/mod_mbox/flink-dev/201604.mbox/%3cCAK+0a_o5=c1_p3sylrhtznqbhplexpb7jg_oq-sptre2neo...@mail.gmail.com%3e

regards.

Re: [DISCUSS] Drop older versions of Kafka Connectors (0.9, 0.10) for Flink 1.10

[Wesley Peng]

on 2019/9/11 16:17, Stephan Ewen wrote:

We still maintain connectors for Kafka 0.8 and 0.9 in Flink.
I would suggest to drop those with Flink 1.10 and start supporting only 
Kafka 0.10 onwards.


Are there any concerns about this, or still a significant number of 
users of these versions?


But we still have a large scale of deployment kafka 0.9 in production. 
Do you mean the new coming flink won't support kafka 0.9?

Though I understand for it, but just sounds sorry. :)

regards.

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Wesley Peng]

Hi

on 2019/9/11 17:22, Till Rohrmann wrote:
I'm very happy to announce that Zili Chen (some of you might also know 
him as Tison Kun) accepted the offer of the Flink PMC to become a 
committer of the Flink project.


Congratulations Zili Chen.

regards.

Apache Beam a Complete Guide - Review?

[Wesley Peng]

Hello

Has anyone bought this book? Can you give a simple review, good or not?

https://www.thriftbooks.com/w/apache-beam-a-complete-guide---2020-edition/26243355/item/36997139/

I want to buy a book for beam the specific topic, but this book seems 
too new to have enough reviews.


Thanks.

Re: Apache Beam a Complete Guide - Review?

[Wesley Peng]

Hi Rion

Rion Williams wrote:
I considered that one as well but was in the same boat in terms of not 
pulling the trigger (lack of reviews, price point, etc.). I eventually 
landed on Streaming Systems, which I highly, highly recommend if you 
want to learn more about the Beam model:


- http://streamingsystems.net/

I don’t think there’s a better book that I’ve come across that focuses 
on it more (and if there is one, I’d love to know about it). I wrote a 
blog post that includes a short-review of it if you want a slightly 
longer summary (http://rion.io/2020/05/09/an-education-in-streaming/), 
but again - I highly recommend checking it out if you hadn’t already.


Thanks for the answer. I will check the resource you gave above.

Regards.

Re: Apache Beam a Complete Guide - Review?

[Wesley Peng]

Hi Luke

Luke Cwik wrote:
The author for Apache Beam A Complete Guide does not have good reviews 
on Amazon for their other books and as you mentioned no reviews for this 
one.


I would second the Streaming Systems book as the authors directly worked 
on Apache Beam.



So, can Apache beam team in google write a book directly for the project 
users?


Thanks.

Re: Apache Beam a Complete Guide - Review?

[Wesley Peng]

Rion Williams wrote:

The three authors of Streaming Systems are folks that work on Google’s Dataflow 
Project, which for all intents and purposes is essentially an implementation of 
the Beam Model. Two of them are members of the Beam PMC (essentially a steering 
committee for the project) and you’ll frequently see them if you are active 
within the mailing lists here.

As mentioned earlier in the thread, I can’t recommend Streaming Systems enough. 
If you are curious about Beam or want to dig into it, it’s the best book you’ll 
find on the topic.


Thanks for this info Rion.

Re: [ANNOUNCE] Apache BookKeeper 4.13.0 released

[Wesley Peng]

great to hear that. thank you.


On 26.02.2021 04:00, Andrey Yegorov wrote:

The Apache BookKeeper team is proud to announce Apache BookKeeper
version
4.13.0.

Apache BookKeeper is a scalable, fault-tolerant, and low-latency
storage service optimized for
real-time workloads. It has been used as a fundamental service to
build reliable services.
It is also the log segment store for Apache DistributedLog and the
message store for Apache Pulsar.

This is the 25th release of the Apache BookKeeper.

Release 4.13.0 improves the reliability of the Stream Storage, brings
additional configuration options for the Stream Storage and Prometheus
HTTP Server, fixes multiple bugs, and brings critical dependencies
up-to-date.

For BookKeeper release details and downloads, visit:

https://bookkeeper.apache.org/releases/

BookKeeper 4.13.0 Release Notes are at:

https://bookkeeper.apache.org/docs/4.13.0/overview/releaseNotes/

We would like to thank the contributors that made the release
possible.

Regards,
The BookKeeper Team

Re: 关于flink 写于kafka时的transactionId 生成问题

[Wesley Peng]

Hi

on 2019/8/29 17:13, ddwcg wrote:

作业我已经停止了，但是看kafka的日志还是不断的在刷Initialized transactionalId………. ，而且该程序再此启动就会报：
Caused by: org.apache.kafka.common.errors.ProducerFencedException: Producer 
attempted an operation with an old epoch. Either there is a newer producer with 
the same transactionalId, or the producer's transaction has been expired by the 
broker.
错误，请问有什么办法可以避免这个问题？


Maybe you want to check all the broker and producers have the same 
timezone setup, and all time are synchronous.


regards.

Re: 关于flink 写于kafka时的transactionId 生成问题

[Wesley Peng]

Hi

on 2019/8/29 17:50, ddwcg wrote:

broker就一个，flink集群的时钟确实和broker的不一样，是不同的机房，能自己指定transactionalId吗，两个机房的调整成一样怕影响其他的应用


AFAIK the transID is generated by systems.

regards.

Re: FlinkKafkaProducer 开启Excatly Once之后 初始化事务状态超时的问题

[Wesley Peng]

Hi

on 2019/9/2 11:49, 陈赋赟 wrote:

2019-09-02 10:24:28,599 INFO  org.apache.flink.runtime.taskmanager.Task
 - Interval Join -> Sink: Unnamed (1/4) 
(e8b85b6f144879efbb0b4209f226c69b) switched from RUNNING to FAILED.
org.apache.kafka.common.errors.TimeoutException: Timeout expired while 
initializing transactional state in 6ms.


You may reference this:

https://stackoverflow.com/questions/54295588/kafka-streams-failed-to-rebalance-error

Possible options:

As this answer says, switch off Exactly Once for your streamer. It then 
doesn't use transactions and all seems to work ok. Not helpful if you 
require EOS or some other client code requires transactions.
restart any brokers that are reporting warnings to force them to 
re-resolve the IP address. They would need to be restarted in a way that 
they don't change IP address themselves. Not usually possible in kubernetes.
Defect raised Issue KAFKA-7958 - Transactions are broken with kubernetes 
hosted brokers


Update 2017-02-20 This may have been resolved in Kafka 2.1.1 (Confluent 
5.1.2) released today. See the linked issue.

Re: flink使用StateBackend问题

[Wesley Peng]

Hi

on 2019/9/3 12:14, 々守护々 wrote:

现象：我是在代码中配置StateBackend的，代码是这样写的env.setStateBackend(new 
RocksDBStateBackend("hdfs://host51:9000/flink/checkpoints",true))，但是我发现在下面的SQLwindow中始终在checkpoint那就过不去，总是报n/a，

下面我就把代码中的这句给注释掉了，在flink-conf.xml文件中配置
state.checkpoints.dir: hdfs://host51:9000/flink/flink-checkpoints这样就能正常执行，
请问SQLwindow和RocksDBStateBackend有什么冲突吗，上面的原因是什么？



It seems like a filesystem issue, though I have no experience on it too.

regards.

Re: 回复： flink使用StateBackend问题

[Wesley Peng]

on 2019/9/3 15:38, 守护 wrote:

org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Received late 
message for now expired checkpoint attempt 3 from 
24674987621178ed1a363901acc5b128 of job fd5010cbf20501339f1136600f0709c3.
请问这个是什么问题呢？


可以根据这些失败的task的id去查询这些任务落在哪一个taskmanager上，经过排查发现，是同一台机器，通过ui看到该机器流入的数据明显比别的流入量大 
因此是因为数据倾斜导致了这个问题，追根溯源还是下游消费能力不足的问题


also reference:
https://juejin.im/post/5c374fe3e51d451bd1663756

Re: flink集群模式下如何动态添加taskmanager

[Wesley Peng]

Hi

on 2019/9/4 10:27, 如影随形 wrote:

  在flink集群部署时，有1个jobmanager,有2个taskmanager(slave1,slave2)，
现在想添加slave3作为 taskmnanager。如何在不停止集群的情况下，可以像spark一样动态添加吗？


AFAIK the answer is NO for now. However, community tells that this has 
been under consideration from the FLIP-6 Flink Development and Process 
Model.


https://docs.google.com/document/d/1zwBP3LKnJ0LI1R4-9hLXBVjOXAkQS5jKrGYYoPz-xRk/edit#

Re: FLink WEB 怎么加登录验证？

[Wesley Peng]

Hi

on 2019/9/5 10:46, wanghongquan.sh wrote:

FLink WEB 控制台中，没有找到登录验证的配置，请问这个WEB怎么加登录验证？


Flink does not directly support authenticating access to the web UI, but 
you can always put something like Apache's basic_auth in front of it.

Re: flink1.9.0对DDL的支持

[Wesley Peng]

Hi

on 2019/9/5 11:23, pengcheng...@bonc.com.cn wrote:

请教一下， 1.flink1.9.0的table API/sql是不是还没有支持Create view？


from the official documentation of flink 1.9:

Views can also be created within a CLI session using the CREATE VIEW 
statement:


CREATE VIEW MyNewView AS SELECT MyField2 FROM MyTableSource;

Views created within a CLI session can also be removed again using the 
DROP VIEW statement:


DROP VIEW MyNewView;

Attention The definition of views in the CLI is limited to the mentioned 
syntax above. Defining a schema for views or escaping whitespaces in 
table names will be supported in future versions.


So create view is supported but has the limits.

regards.

Re: Flink Savepoint 超时

[Wesley Peng]

SJMSTER wrote:

Checkpoints一直都是成功的。
今天重新尝试了一下cancle job with savepoint又成功了..
不知道之前为什么试了几次都是超时的..


are there any log items for diagnosis?

regards.

Re: 希望获得一份编译后的1.6版本以上flink源码

[Wesley Peng]

Helo

guaishushu1...@163.com wrote:

因windows系统搭载虚拟机环境，总是不能成功编译flink源码，所以希望能得到一份编译后的1.6版本以上的flink源码


there is docker image for flink 1.9. since you are using a virtual 
machine, then docker might be used.


regards

Re: FLINK WEEKLY 2019/36

[Wesley Peng]

on 2019/9/9 11:23, Zili Chen wrote:

FLINK WEEKLY 2019/36

很高兴和大家分享上周 FLINK 社区的发展。在过去的一周里，更多 FLINK 1.10 的特性被提出和讨论，包括新的 FLIP，来自 Apache
Pulsar 社区的 Connector 贡献等等。专门讨论 FLINK 1.10 将要实现什么特性的议程也在进行。
用户问题


A nice work. thank you.

regards.

Re: Flink大state读取磁盘，磁盘IO打满，任务相互影响的问题探讨

[Wesley Peng]

on 2019/9/10 13:47, 蒋涛涛 wrote:

尝试手段：

1. 手动迁移IO比较高的任务到其他机器，但是yarn任务提交比较随机，只能偶尔为之

2. 目前没有SSD，只能用普通STATA盘，目前加了两块盘提示磁盘IO能力，但是单盘对单任务的磁盘IO瓶颈还在

还有哪些策略可以解决或者缓解么？


It seems the tricks to improve RocksDB's throughput might be helpfu.

With writes and reads accessing mostly the recent data, our goal is to 
let them stay in memory as much as possible without using up all the 
memory on the server. The following parameters are worth tuning:


Block cache size: When uncompressed blocks are read from SSTables, they 
are cached in memory. The amount of data that can be stored before 
eviction policies apply is determined by the block cache size. The 
bigger the better.


Write buffer size: How big can Memtable get before it is frozen. 
Generally, the bigger the better. The tradeoff is that big write buffer 
takes more memory and longer to flush to disk and to recover.


Write buffer number: How many Memtables to keep before flushing to 
SSTable. Generally, the bigger the better. Similarly, the tradeoff is 
that too many write buffers take up more memory and longer to flush to disk.


Minimum write buffers to merge: If most recently written keys are 
frequently changed, it is better to only flush the latest version to 
SSTable. This parameter controls how many Memtables it will try to merge 
before flushing to SSTable. It should be less than the write buffer 
number. A suggested value is 2. If the number is too big, it takes 
longer to merge buffers and there is less chance of duplicate keys in 
that many buffers.


The list above is far from being exhaustive, but tuning them correctly 
can have a big impact on performance. Please refer to RocksDB’s Tuning 
Guide for more details on these parameters. Figuring out the optimal 
combination of values for all of them is an art in itself.


please ref: https://klaviyo.tech/flinkperf-c7bd28acc67

regards.

Re: Split a stream into any number of streams

[Wesley Peng]

on 2019/9/17 9:55, 王佩 wrote:

I want to split a stream into any number of streams according to a field,
and then process the split stream one by one.


I think that should be easy done. refer to:
https://stackoverflow.com/questions/53588554/apache-flink-using-filter-or-split-to-split-a-stream

regards.

Re: Split a stream into any number of streams

[Wesley Peng]

Hi

on 2019/9/17 10:28, 王佩 wrote:

*  //  How should I do it?*
splitStream.select("productID1").print();


If I understand for that correctly, you want somewhat the dynamic number 
of Sinks?


regards

Re: 订阅邮件

[Wesley Peng]

Hello

You should not pose across multiple groups.
One thread to one group is more graceful.


杨利君 于2019年9月26日 周四下午3:42写道：

> 订阅flink社区邮件

Re: Could not forward element to next operator

[Wesley Peng]

Hello,

May this article match your issue?
https://blog.csdn.net/qq_41910230/article/details/90411237

regards.

On Sun, Sep 29, 2019 at 10:33 AM allan <18612537...@163.com> wrote:

> Hi，
>
> 最近发现作业一直在报错，我的窗口是一分钟的窗口。这是什么原因，谁能帮助一下？flink版本1.6 ,错误如下：
>
>
>
> TimerException{org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException:
> Could not forward element to next operator}
>
>at
> org.apache.flink.streaming.runtime.tasks.SystemProcessingTimeService$TriggerTask.run(SystemProcessingTimeService.java:288)
>
>at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>
>at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
>at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>
>at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>
>at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
>at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
>at java.lang.Thread.run(Thread.java:748)
>
> Caused by:
> org.apache.flink.streaming.runtime.tasks.ExceptionInChainedOperatorException:
> Could not forward element to next operator
>
>at org.apache.flink.streaming.
>
>
> runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:596)
>
>at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:554)
>
>at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:534)
>
>at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:689)

Re: 文件重命名

[Wesley Peng]

May you want to rename them in HDFS with FileSystem.rename method?


on 2019/10/8 17:39, yanggang_it_job wrote:

写入hdfs的文件都是
part-{parallel-task}-{count}
这种格式


如何重命名啊？

Re: flink权限问题

[Wesley Peng]

hi

hdfs有严格的权限管理系统，这个问题似乎绕不过去吧，flink该有的权限就应该给它开了。

On Thu, Oct 17, 2019 at 7:58 PM jinhai wang  wrote:

> 背景：
>
> 实时flink任务从kafka同步数据到hdfs。
>
>
>
> 问题：
>
> 比如flink的job使用A用户提交，但写hdfs的文件用户只能是B，那这个除了给hdfs添加A的操作权限外，flink
> 这块儿可以优雅解决这类问题吗？checkpoint的写权限应该是A用户吧
>

Re: unsubscribe

[Wesley Peng]

please send an empty email to: user-unsubscr...@spark.apache.org to 
unsubscribe yourself from the list.



Sri Kris wrote:
Sent from Mail  for 
Windows 10




-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Unsubscribe

[Wesley Peng]

Bhavya Jain wrote:

Unsubscribe


please send an email to: user-unsubscr...@spark.apache.org to 
unsubscribe yourself from the list. thanks.


-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: [ANNOUNCE] Announcing Apache Spark 2.4.4

[Wesley Peng]

on 2019/9/2 5:54, Dongjoon Hyun wrote:

We are happy to announce the availability of Spark 2.4.4!

Spark 2.4.4 is a maintenance release containing stability fixes. This
release is based on the branch-2.4 maintenance branch of Spark. We strongly
recommend all 2.4 users to upgrade to this stable release.


That's awesome. thanks for the work.

regards.

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Zookeeper discovery in mix environments.

[Wesley Peng]

Maybe you would ask the question on VM/container related mailing list?

Regards

On Mon, Jun 1, 2020 at 9:32 PM John Smith  wrote:

> Any news on this? Thanks
>
>
> On Thu., May 28, 2020, 1:10 p.m. John Smith, 
> wrote:
>
>> Hi, I'm running an ignite cluster on VMs running on Openstack and using
>> regular network stack nothing special here.
>>
>> My CLIENT (client=true) applications are running in DC/OS using docker
>> container in bridged network mode.
>>
>> When using TCP discovery everything works nice and dandy. But I recently
>> tried to switch to Zookeeper discovery but the docker application can't
>> seem to connect or get discovered?
>>
>> I'm assuming when running in bridge mode I need to open a port on the
>> bridge to allow other node to connect back. Is there a way to the ignite
>> client/zookeeper SPI that the CLIENT node is available at a specific
>> ip/port?
>>
>

Re: connection refused

[Wesley Peng]

Does either client or server have any logs?

Thanks

On Tue, Jun 2, 2020 at 6:37 PM Clay Teahouse  wrote:

> I'd appreciate your help with this issue.
> I have a server and a client node, the latter running in tomcat. I get
> connection refused if I try to connect to the server node from the client,
> although the server shows the client has joined the cluster. I don't have a
> problem running a service on the server's service grid from the client node.
> I don't have any issue connecting from any client anywhere, if the client
> is not running in tomcat.
> what could be the problem?
>
>

Re: Call for presentations for ApacheCon North America 2020 now open

[Wesley Peng]

Congrats. We could prepare a talking for "machine learning application 
with Ignite", as we did store feature engineering data into ignite for 
large-scale and fast access.


regards.


Saikat Maitra wrote:

Congrats!!!

It looks like both of our talks are on same day, Tuesday, September 29th

https://apachecon.com/acah2020/tracks/ignite.html

Re: run ignitevisorcmd in k8s and docker？

[Wesley Peng]

Hi

bbweb wrote:
we are meeting problem when we run ignitevisorcmd in K8S and docker 
environment. After we start a cluster in K8S and run ignitevisorcmd in 
the node, it just can't find any node when running "top" in it, it 
just show empty topology.


Do you have any error log?
Are you sure ignite cluster is started up correctly?

regards.

Re: unsubscribe

[Wesley Peng]

Please send an empty message to: user-unsubscr...@ignite.apache.org to 
unsubscribe yourself from the list.



Sijo Mathew wrote:

Re: graceful shutdown for C++ applications

[Wesley Peng]

Issuing a command like "kill process_id"  doesn't work?

regards.

On Thu, Jan 7, 2021 at 4:14 PM rakshita04 
wrote:

> Hi Team,
>
> We are using apache-ignite for our applications running on 2 machines and
> connected over network.
> We are facing some issue where if kill is performed on running application,
> it somehow corrupts the node and then node never comes up and keep on
> rebooting.
> Is there a way to handle this shutdown gracefully? so that there is no data
> loss and node corruption.
>
> regards,
> Rakshita Chaudhary
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Looking for feedback on the Ignite 3.0.0 Alpha

[Wesley Peng]

When will the stable version of 3.0 get released? thanks.

Valentin Kulichenko wrote:
I'm excited to announce that the first alpha build of the Ignite 3 is 
out and available for download!


Ignite 3 is the new project that was initiated by the Ignite community 
last year. Please refer to this page if you want to learn more: 
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3.0 



The just-released alpha build is a sneak peek into the future of Ignite. 
It doesn't represent a fully-functional product (no discovery, caches, 
compute, etc.), but demonstrates major mechanics of how you will 
interact with Ignite going forward.

compiler plugin

[Wesley Peng]

Does vim have a compiler plugin for compiling script languages like python?

Thanks.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--- 
You received this message because you are subscribed to the Google Groups "vim_use" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to vim_use+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/vim_use/2b00f6ed-4b64-3bdd-e0fb-ba29c4c27773%40gmail.com.

Re: [ANNOUNCE] New Pulsar Committer: LinLin

[Wesley Peng]

Congrats Linlin~

Regards.

On Wed, Jan 6, 2021 at 5:25 PM Jinfeng Huang  wrote:

> Congratulations to Linlin~
>
> Best Regards,
> Jennifer
>
>
> On Wed, Jan 6, 2021 at 3:30 PM Matthias Peng 
> wrote:
>
>> congratulations linlin!
>>
>> On Wed, Jan 6, 2021 at 3:05 PM PengHui Li  wrote:
>>
>>> The Apache Pulsar PMC recently extended committer karma to LinLin and he
>>> has accepted.
>>> LinLin has done a lot of contributions to Pulsar core features and bug
>>> fixes. It is great to have LinLin onboard as Pulsar committers.
>>>
>>> We are looking forward to more contributions from him.
>>>
>>> Congratulations and welcome onboard LinLin!
>>>
>>> -
>>>
>>> Penghui on behave of the Pulsar PMC
>>>
>>

pulsar with perl from rest API

[Wesley Peng]

Hello

I need to write some codes with perl for accessing to pulsar cluster.
Can we use the rest API for this purpose since there is not the native client 
library for perl yet?

Thanks.

CPU consuming question

[Wesley Peng]

I deployed a standalone pulsar server on a two cores VPS, which was new 
installed.
Even there is not any jobs running on pulsar, it still consumes CPU about 15+%.
'top' shows the java process takes it.

Can you help explain this? Thank you.

Regards.

Re: CPU consuming question

[Wesley Peng]

Can you help check with log output?

$ tail -f pulsar-standalone-PresentSpicy-VM.log 

14:45:14.281 [main-EventThread] INFO  
org.apache.distributedlog.logsegment.PerStreamLogSegmentCache - 
01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76/CURRENT
 added log segment (logrecs_01 : [LogSegmentId:279, 
firstTxId:16, lastTxId:16, version:VERSION_V5_SEQUENCE_ID, 
completionTime:1612161014650, recordCount:1, regionId:0, status:0, 
logSegmentSequenceNumber:1, lastEntryId:0, lastSlotId:0, inprogress:false, 
minActiveDLSN:DLSN{logSegmentSequenceNo=1, entryId=0, slotId=0}, 
startSequenceId:0]) to cache.

14:45:14.297 [DLM-/stream/storage-OrderedScheduler-1-0] INFO  
org.apache.distributedlog.BKLogWriteHandler - Deleting ledger 
[LogSegmentId:279, firstTxId:16, lastTxId:16, version:VERSION_V5_SEQUENCE_ID, 
completionTime:1612161014650, recordCount:1, regionId:0, status:0, 
logSegmentSequenceNumber:1, lastEntryId:0, lastSlotId:0, inprogress:false, 
minActiveDLSN:DLSN{logSegmentSequenceNo=1, entryId=0, slotId=0}, 
startSequenceId:0] for 
01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76/CURRENT:

14:45:14.301 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  
org.apache.distributedlog.impl.ZKLogSegmentMetadataStore - Delete segment : 
[LogSegmentId:279, firstTxId:16, lastTxId:16, version:VERSION_V5_SEQUENCE_ID, 
completionTime:1612161014650, recordCount:1, regionId:0, status:0, 
logSegmentSequenceNumber:1, lastEntryId:0, lastSlotId:0, inprogress:false, 
minActiveDLSN:DLSN{logSegmentSequenceNo=1, entryId=0, slotId=0}, 
startSequenceId:0]

14:45:14.331 [main-EventThread] INFO  
org.apache.distributedlog.bk.SimpleLedgerAllocator - Ledger allocator for 
/stream/storage/01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76//allocation
 moved version from -1 to 0.

14:45:14.349 [io-checkpoint-scheduler-OrderedScheduler-1-0] INFO  
org.apache.bookkeeper.statelib.impl.rocksdb.checkpoint.RocksdbCheckpointTask - 
Delete remote checkpoint 0352d14e-8a42-4453-89ed-1fbf34051b76 from checkpoint 
store at 
01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76

14:45:33.435 [pulsar-web-68-5] INFO  org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [01/Feb/2021:14:45:33 +0800] "GET 
/admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false 
HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 10

14:46:03.429 [pulsar-web-68-1] INFO  org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [01/Feb/2021:14:46:03 +0800] "GET 
/admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false 
HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 6

14:46:33.441 [pulsar-web-68-6] INFO  org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [01/Feb/2021:14:46:33 +0800] "GET 
/admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false 
HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 10

14:47:03.431 [pulsar-web-68-8] INFO  org.eclipse.jetty.server.RequestLog - 
127.0.0.1 - - [01/Feb/2021:14:47:03 +0800] "GET 
/admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false 
HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 7



Thank you
Regards.


On Mon, Feb 1, 2021, at 2:30 PM, Sijie Guo wrote:
> Hi Wesley,
> 
> The CPU usage might be coming from some background tasks. Do you happen to 
> have the log of the standalone server? This might give some insights to the 
> problem. 
> 
> - Sijie
> 
> On Sun, Jan 31, 2021 at 9:58 PM Wesley Peng  wrote:
>> __
>> I deployed a standalone pulsar server on a two cores VPS, which was new 
>> installed.
>> Even there is not any jobs running on pulsar, it still consumes CPU about 
>> 15+%.
>> 'top' shows the java process takes it.
>> 
>> Can you help explain this? Thank you.
>> 
>> Regards.

A simple publish takes 9s to finish

[Wesley Peng]

Can you please see below why a simple publish take 9s to finish?
Pulsar is deployed on a two cores VPS, and I run it with `pulsar-daemon start 
standalone` mode.

Thanks.

$ time bin/pulsar-client produce my-topic --messages "hello-pullsar fifth"

15:06:12.815 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x90192576, 
L:/127.0.0.1:45148 - R:localhost/127.0.0.1:6650]] Connected to server

15:06:13.385 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar 
producer perf with config: {

  "topicName" : "my-topic",

  "producerName" : null,

  "sendTimeoutMs" : 3,

  "blockIfQueueFull" : false,

  "maxPendingMessages" : 1000,

  "maxPendingMessagesAcrossPartitions" : 5,

  "messageRoutingMode" : "RoundRobinPartition",

  "hashingScheme" : "JavaStringHash",

  "cryptoFailureAction" : "FAIL",

  "batchingMaxPublishDelayMicros" : 1000,

  "batchingPartitionSwitchFrequencyByPublishDelay" : 10,

  "batchingMaxMessages" : 1000,

  "batchingMaxBytes" : 131072,

  "batchingEnabled" : true,

  "chunkingEnabled" : false,

  "compressionType" : "NONE",

  "initialSequenceId" : null,

  "autoUpdatePartitions" : true,

  "autoUpdatePartitionsIntervalSeconds" : 60,

  "multiSchema" : true,

  "properties" : { }

}

15:06:13.411 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: 
{

  "serviceUrl" : "pulsar://localhost:6650/",

  "authPluginClassName" : null,

  "operationTimeoutMs" : 3,

  "statsIntervalSeconds" : 60,

  "numIoThreads" : 1,

  "numListenerThreads" : 1,

  "connectionsPerBroker" : 1,

  "useTcpNoDelay" : true,

  "useTls" : false,

  "tlsTrustCertsFilePath" : "",

  "tlsAllowInsecureConnection" : false,

  "tlsHostnameVerificationEnable" : false,

  "concurrentLookupRequest" : 5000,

  "maxLookupRequest" : 5,

  "maxLookupRedirects" : 20,

  "maxNumberOfRejectedRequestPerConnection" : 50,

  "keepAliveIntervalSeconds" : 30,

  "connectionTimeoutMs" : 1,

  "requestTimeoutMs" : 6,

  "initialBackoffIntervalNanos" : 1,

  "maxBackoffIntervalNanos" : 600,

  "listenerName" : null,

  "useKeyStoreTls" : false,

  "sslProvider" : null,

  "tlsTrustStoreType" : "JKS",

  "tlsTrustStorePath" : "",

  "tlsTrustStorePassword" : "",

  "tlsCiphers" : [ ],

  "tlsProtocols" : [ ],

  "proxyServiceUrl" : null,

  "proxyProtocol" : null,

  "enableTransaction" : false

}

15:06:13.485 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [null] Creating 
producer on cnx [id: 0x90192576, L:/127.0.0.1:45148 - 
R:localhost/127.0.0.1:6650]

15:06:13.576 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ProducerImpl - [my-topic] [standalone-1-4] 
Created producer on cnx [id: 0x90192576, L:/127.0.0.1:45148 - 
R:localhost/127.0.0.1:6650]

15:06:13.919 [pulsar-client-io-1-1] INFO  
com.scurrilous.circe.checksum.Crc32cIntChecksum - SSE4.2 CRC32C provider 
initialized

15:06:14.002 [main] INFO  org.apache.pulsar.client.impl.PulsarClientImpl - 
Client closing. URL: pulsar://localhost:6650/

15:06:14.054 [main] INFO  org.apache.pulsar.client.impl.ProducerImpl - 
[my-topic] [standalone-1-4] Closed Producer

15:06:14.090 [pulsar-client-io-1-1] INFO  
org.apache.pulsar.client.impl.ClientCnx - [id: 0x90192576, L:/127.0.0.1:45148 ! 
R:localhost/127.0.0.1:6650] Disconnected

15:06:14.164 [main] INFO  org.apache.pulsar.client.cli.PulsarClientTool - 1 
messages successfully produced



real 0m9.011s

user 0m10.884s

sys 0m1.172s

Re: CPU consuming question

[Wesley Peng]

OK let me try.
Thank you @Sijie.

On Wed, Feb 3, 2021, at 1:11 PM, Sijie Guo wrote:
> Ah, I see. I am guessing most of the CPU cycles are burned by the table 
> service. 
> 
> You can try to disable that feature as it is still under developer preview 
> mode. You can run standalone as `bin/pulsar standalone --no-stream-storage`.
> 
> - Sijie
> 
> 
> On Sun, Jan 31, 2021 at 10:52 PM Wesley Peng  wrote:
>> __
>> Can you help check with log output?
>> 
>> $ tail -f pulsar-standalone-PresentSpicy-VM.log 

>> 14:45:14.281 [main-EventThread] INFO  
>> org.apache.distributedlog.logsegment.PerStreamLogSegmentCache - 
>> 01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76/CURRENT
>>  added log segment (logrecs_01 : [LogSegmentId:279, 
>> firstTxId:16, lastTxId:16, version:VERSION_V5_SEQUENCE_ID, 
>> completionTime:1612161014650, recordCount:1, regionId:0, status:0, 
>> logSegmentSequenceNumber:1, lastEntryId:0, lastSlotId:0, inprogress:false, 
>> minActiveDLSN:DLSN{logSegmentSequenceNo=1, entryId=0, slotId=0}, 
>> startSequenceId:0]) to cache.

>> 14:45:14.297 [DLM-/stream/storage-OrderedScheduler-1-0] INFO  
>> org.apache.distributedlog.BKLogWriteHandler - Deleting ledger 
>> [LogSegmentId:279, firstTxId:16, lastTxId:16, 
>> version:VERSION_V5_SEQUENCE_ID, completionTime:1612161014650, recordCount:1, 
>> regionId:0, status:0, logSegmentSequenceNumber:1, lastEntryId:0, 
>> lastSlotId:0, inprogress:false, minActiveDLSN:DLSN{logSegmentSequenceNo=1, 
>> entryId=0, slotId=0}, startSequenceId:0] for 
>> 01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76/CURRENT:

>> 14:45:14.301 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO  
>> org.apache.distributedlog.impl.ZKLogSegmentMetadataStore - Delete segment : 
>> [LogSegmentId:279, firstTxId:16, lastTxId:16, 
>> version:VERSION_V5_SEQUENCE_ID, completionTime:1612161014650, recordCount:1, 
>> regionId:0, status:0, logSegmentSequenceNumber:1, lastEntryId:0, 
>> lastSlotId:0, inprogress:false, minActiveDLSN:DLSN{logSegmentSequenceNo=1, 
>> entryId=0, slotId=0}, startSequenceId:0]

>> 14:45:14.331 [main-EventThread] INFO  
>> org.apache.distributedlog.bk.SimpleLedgerAllocator - Ledger allocator for 
>> /stream/storage/01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76//allocation
>>  moved version from -1 to 0.

>> 14:45:14.349 [io-checkpoint-scheduler-OrderedScheduler-1-0] INFO  
>> org.apache.bookkeeper.statelib.impl.rocksdb.checkpoint.RocksdbCheckpointTask 
>> - Delete remote checkpoint 0352d14e-8a42-4453-89ed-1fbf34051b76 from 
>> checkpoint store at 
>> 01/01/00/checkpoints/0352d14e-8a42-4453-89ed-1fbf34051b76

>> 14:45:33.435 [pulsar-web-68-5] INFO  org.eclipse.jetty.server.RequestLog - 
>> 127.0.0.1 - - [01/Feb/2021:14:45:33 +0800] "GET 
>> /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false
>>  HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 10

>> 14:46:03.429 [pulsar-web-68-1] INFO  org.eclipse.jetty.server.RequestLog - 
>> 127.0.0.1 - - [01/Feb/2021:14:46:03 +0800] "GET 
>> /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false
>>  HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 6

>> 14:46:33.441 [pulsar-web-68-6] INFO  org.eclipse.jetty.server.RequestLog - 
>> 127.0.0.1 - - [01/Feb/2021:14:46:33 +0800] "GET 
>> /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false
>>  HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 10

>> 14:47:03.431 [pulsar-web-68-8] INFO  org.eclipse.jetty.server.RequestLog - 
>> 127.0.0.1 - - [01/Feb/2021:14:47:03 +0800] "GET 
>> /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false
>>  HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 7

>> 
>> 
>> Thank you
>> Regards.
>> 
>> 
>> On Mon, Feb 1, 2021, at 2:30 PM, Sijie Guo wrote:
>>> Hi Wesley,
>>> 
>>> The CPU usage might be coming from some background tasks. Do you happen to 
>>> have the log of the standalone server? This might give some insights to the 
>>> problem. 
>>> 
>>> - Sijie
>>> 
>>> On Sun, Jan 31, 2021 at 9:58 PM Wesley Peng  wrote:
>>>> __
>>>> I deployed a standalone pulsar server on a two cores VPS, which was new 
>>>> installed.
>>>> Even there is not any jobs running on pulsar, it still consumes CPU about 
>>>> 15+%.
>>>> 'top' shows the java process takes it.
>>>> 
>>>> Can you help explain this? Thank you.
>>>> 
>>>> Regards.
>> 

Re: [ANNOUNCE] New committer: Mickael Maison

[Wesley Peng]

Matthias J. Sax wrote:

Congrats Mickeal!


-Matthias


Welcome Mickeal!

regards

function alias

[Wesley Peng via beginners]

Hello,

How to make a function alias in perl? for example, says() is alias to 
print().


thanks.

--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/

[go-nuts] Is this AMQP 0.9.1 library updated?

['Wesley Peng' via golang-nuts]

Hello,
I tried to access RabbitMQ using this AMQP 
library:https://github.com/streadway/amqp

I am not sure if this library get updated following the recent RMQ 
version.Please suggest, thanks.

Wesley Peng
wesleyp...@aol.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1077505155.1304617.1588125421267%40mail.yahoo.com.

[go-nuts] SSL socket listener

['Wesley Peng' via golang-nuts]

Hello,
How do I program with SSL to make a server listen on specific port which 
accepts SSL transfer only?
Is there any guide for this since I have no experience on SSL socket 
programming.
Thanks.

Wesley Peng
wesleyp...@aol.com

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1690752345.1320667.1591168756241%40mail.yahoo.com.

Re: [go-nuts] SSL socket listener

['Wesley Peng' via golang-nuts]

Thanks. how about the sample of general socket listener with SSL rather 
than net/http implementation?


Regards

Dimas Prawira wrote:

Here is an example running server with TLS

package  main

import  (
 "net/http"
 "log"
)

func  HelloServer(w  http.ResponseWriter,req  *http.Request) {
 w.Header().Set("Content-Type","text/plain")
 w.Write([]byte("This is an example server.\n"))
}

func  main() {
 http.HandleFunc("/hello",HelloServer)
 err  :=  http.ListenAndServeTLS(":443","server.crt","server.key",nil)
 if  err  !=  nil  {
 log.Fatal("ListenAndServe: ",err)
 }
}



--
Wesley Peng
wesleyp...@aol.com

--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ffdb0882-3e1a-173d-6c3c-91f63d88bbfe%40aol.com.

Re: Last day for FOSDEM submission

[Wesley Peng via perl6-users]

Replacing Bash scripts with Raku? That’s an interesting thing. Can Linux be 
shipped with Raku by default?




hello people,

I just discovered this this morning:

https://www.reddit.com/r/rakulang/comments/rrcp4c/steal_these_ideas_for_raku_fosdem_talks/

I don't remember if there was a previous annoucement in this list but
it's still possible to jump in.

I just submitted one on "Replacing Bash scripts with Raku"

* features and way of doing things i love from zsh
* the raku way [1]
* getting started with raku ecosystem (raku.land, zef, Ddt, fez, rakudoc, 
prove6, ...)

gather/take=while read, Seq vs Pipes, ...) and ideas of modules

1: the raku way are things like

*sh        │ raku
───┼─
|          │ Seq.map
xargs -P  │ .race
while read │ gather/take

At some point it could become an Intro section on the Raku documentation
site.

If a native/fluent english speaker want to take over it: i'll be more
than happy to co-author the talk as well (i really feel like i'm a disaster as
speaker).

Regards
marc

Re: Last day for FOSDEM submission

[Wesley Peng via perl6-users]

Replacing Bash scripts with Raku? That’s an interesting thing. Can Linux be 
shipped with Raku by default?




hello people,

I just discovered this this morning:

https://www.reddit.com/r/rakulang/comments/rrcp4c/steal_these_ideas_for_raku_fosdem_talks/

I don't remember if there was a previous annoucement in this list but
it's still possible to jump in.

I just submitted one on "Replacing Bash scripts with Raku"

* features and way of doing things i love from zsh
* the raku way [1]
* getting started with raku ecosystem (raku.land, zef, Ddt, fez, rakudoc, 
prove6, ...)

gather/take=while read, Seq vs Pipes, ...) and ideas of modules

1: the raku way are things like

*sh        │ raku
───┼─
|          │ Seq.map
xargs -P  │ .race
while read │ gather/take

At some point it could become an Intro section on the Raku documentation
site.

If a native/fluent english speaker want to take over it: i'll be more
than happy to co-author the talk as well (i really feel like i'm a disaster as
speaker).

Regards
marc