Typical timescales for publishing binary packages in -backports?

2021-08-19 Thread Andy Smith
Hi,

I notice that yesterday there's been an acceptance email for source
package linux-signed-amd64 version 5.10.46+4~bpo10+1 in
buster-backports:

https://lists.debian.org/debian-kernel/2021/08/msg00139.html

Previously there had also been one for version 5.10.46+3~bpo10+1.

Yet as of today there seems to only be binary packages built from
version 5.10.46+2~bpo10+1:

$ apt show linux-image-amd64/buster-backports
Package: linux-image-amd64
Version: 5.10.46-2~bpo10+1
Built-Using: linux (= 5.10.46-2~bpo10+1)
Priority: optional
Section: kernel
Source: linux-signed-amd64 (5.10.46+2~bpo10+1)

This is from back in July sometime.

So I was wondering what is the typical timescale for binary packages
from the kernel source upload to appear in buster-backports?

I am aware that I can build the binary packages myself if I want
them sooner. It looks like I could also install the package from sid
or bookworm:

https://packages.debian.org/sid/linux-image-amd64

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: docker image of bullseye is outdated

2021-08-15 Thread Andy Smith
Hi,

On Sun, Aug 15, 2021 at 10:15:45PM +0300, илья пащук wrote:
> docker image of debian bullseye should be updated to reflect it's release.

There is a message on debian-boot (and -devel) here about it:

https://lists.debian.org/debian-boot/2021/08/msg00096.html

I'm not sure that the Debian docker images are officially part of
Debian, so may not be possible to submit a Debian bug about them. I
don't use them though so I might be mistaken.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: what's wrong with my "/etc/apt/sources.list"? Updating from such a repository can't be done securely, and is therefore disabled by default

2021-08-15 Thread Andy Smith
Hi Evelyn,

On Sun, Aug 15, 2021 at 06:00:08PM +0200, Evelyn Pereira Souza wrote:
> E: The repository 'http://security.debian.org/debian-security
> bullseye/updates Release' does not have a Release file.

A few people have by now pointed out the specific cause of your
problem, but the root cause is that you've done an update to the
next version of Debian without reading the release notes. You always
need to read the release notes, every time, not just for bullseye.


https://www.debian.org/releases/bullseye/amd64/release-notes/ch-information.en.html

As you can see there are a number of other issues there. There
usually are. So save yourself some frustration by reading the
release notes every time.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting


signature.asc
Description: PGP signature


Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-11 Thread Andy Smith
Hello,

On Tue, Aug 10, 2021 at 09:12:08PM -0400, Polyna-Maude Racicot-Summerside wrote:
> On 2021-08-09 9:37 p.m., Andy Smith wrote:
> > I'm sorry you feel that way. It has felt quite frustrating having to
> > repeat myself when I thought I was being clear about what I meant.
> I think it would be hard to express yourself better than what you always
> do in all the messages you sent, the "moderation" you do and the fire
> you extinguish.

I think you've possibly mistaken me for Andy Cater who posts the
monthly conduct reminder thing.

I thought I better correct this as it would be wrong for me to take
credit for moderating and extinguishing that I don't think I've
done, as well as to possibly associate Andy Cater with my suggestion
of not doing support stuff on debian-user.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

> The optimum programming team size is 1.
Has Jurassic Park taught us nothing? — pfilandr



Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-09 Thread Andy Smith
Hello,

On Tue, Aug 10, 2021 at 02:51:02AM +0200, Linux-Fan wrote:
> I think that switching support over to a different medium i.e. from e-
> mail to Q will see a different sort of user participating.
> Hence, the "community" one would find on the Q site is not there yet. This
> explains why it would not be used much (initially) even if there was a lot
> of advertisement.

If we are talking about the pool of people asking the questions and
answering them, I'm not sure that so many would be left behind. I do
get the impression that those new to Linux do feel pretty
comfortable on web sites and less so on email lists. I actually know
a lot of people in their 20s and 30s now who don't really have an
email address, only for use as credentials on web sites!

I don't want to harp on about it but Ask Ubuntu does some good
numbers on daily answers marked solved. Also I've spotted quite a
few names from here on some of those Stack sites. :-)

If the new people don't actually know the thing exists it won't get
used though.

The other kind of participant, who's here mainly for debate, will I
guess still be here debating. Which I don't see as a problem.

> As far as I can tell, Debian's development communication mostly uses e-mail
> (for bugs, mailing lists, announcements) and IRC (for real-time
> communication e.g. release testing). Hence it seems only natural that e-mail
> and IRC would be the primary means to ask for help, too. The idea behind
> this is (in theory?) that the developers use the same means of communication
> as the users.

The first thing I would say to that though is that all those places
have much more rigidly defined topics than here. I don't know about
IRC, but I'm in many of those other places and off-topicness isn't
much of a problem there. Neither do they tend to see an influx of
low experience new users.

For discussion, email is king, I'm with you there, never do get on
well with web forums. I really like to see topics enforced though.

It's interesting that even in the extremely technical community of
Linux kernel development, there is increasing call for patch
submission and management by web interface rather than email:

https://lwn.net/Articles/811528/
https://lwn.net/Articles/860607/

(I mention this only as an aside and don't see it as really relevant
when talking about user support.)

> Where does the notion that the mailing list is the primary support channel
> stem from?

Personally I see people being pointed to it all the time when they
ask user-level questions in some other Debian email space.

I've been on IRC since 1995 but I don't hang around in any of the
mainstream Debian channels so I'm not aware of what they're like.

I can well imagine that someone looking for assistance does a search
and finds https://www.debian.org/support where the first part talks
about a thing that seems obscure and requires software they probably
never used before. The next one down is email, and everyone's heard
of email even if they only use it to log in to Netflix.

> I even tried out Reddit for a few weeks but noticing how much data they
> collect just by my clicks on up/down and choice of topics to read is quite a
> revelation. Both, mailing-lists and IRC are in a way more public that
> everything one sends is published for all to read but also more private in
> that what one does not intend to send (which messages I read and how long I
> take for it for instance) stays private.

I do think it is important for Debian to self-host where it can. The
privacy aspect is critical as so many third party web services are
primarily about selling the users' personal data and their activity
than the actual service they purport to provide.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-09 Thread Andy Smith
Hello,

On Mon, Aug 09, 2021 at 06:05:58PM -0700, Weaver wrote:
> > So that's where I think the problems are and why I think it would be
> > good to try separating the user support from the debate club.
> 
> I'm afraid this conversation is a waste of time.

I'm sorry you feel that way. It has felt quite frustrating having to
repeat myself when I thought I was being clear about what I meant.
But, I do believe that at least you have got across how you see the
purpose and nature of debian-user, so we have at least seen some
more perspectives - thanks!

My last reply was sent before I saw this one, and is not an attempt
to prolong a conversation that you don't want to have.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-09 Thread Andy Smith
Hello,

On Mon, Aug 09, 2021 at 04:22:46PM -0700, Weaver wrote:
> On 10-08-2021 09:01, Andy Smith wrote:
> > On Mon, Aug 09, 2021 at 10:19:19PM +0000, Andy Smith wrote:
> >> I am asking the Debian Project to can this list in favour of an
> >> alternate solution or else to make it strictly for Debian-related
> >> posts only.
> > 
> > I want to walk this one back a bit as there's no need to destroy the
> > community that exists here already by forcing it to move elsewhere.
> 
> Oh, I don't think you'll achieve that.

Just wanted to clarify that it isn't something that I want to see
happen.

> > What I meant to say is that I think it'd be best if debian-user
> > continue to exist as it is while having its support element covered
> > by some other thing, and that be consistently documented throughout
> > Debian's documentation and websites wherever debian-user currently
> > is pointed to.
> 
> By all means, if you see that as desirable, go ahead and initiate it.
> If things are a little quiet, you'll find us here.

I did mention that the Stack site idea, although it has been partially
tried out at least two times by various Debian Developers, had no
success and would not see success without a concerted effort by the
project to direct users there.

Clearly I'm not in a position to go editing Debian's web sites. I'm
guessing any such wiki edit would get reverted if it tried to
suggest something as being preferable to debian-user, so I'm not in
a position to do more than suggest.

I have tried to help out those last two times there was an Stack
site thing launched though, so if any DD does so again I will try
again with you.

> I'm sorry, but I find it extremely difficult to equate the term
> `community' with that of `rigid, militant conformity'.

If you take discussion lists like debian-devel or debian-project as
examples, these are places that have a much more rigid definition of
topic than debian-user, and yet are also community spaces.

Occasionally posters get very off-topic on debian-project for
example and are told, on-list, to restrict their posting to being
about the Debian project. No one seems to find this to be militant
conformity. The posters tend to obey.

I find debian-user to be quite rare amongst Debian spaces in the
lack of its topic focus. I know that it has always been like that, I
just don't think that works well for the support aspect.

Again going back to Ask Ubuntu — mainly because it's both a Stack
site and a Debian derivative so being a support venue answering
questions much like we do here on debian-user — you couldn't really
say it is an community in and of itself. I expect there are
forum-like sections of it for those so inclined but I've never
wanted to look into that. While it is a part of the Ubuntu community
and covered by the same code of conduct etc., community is not
really evident or emphasised there and I don't think that it needs
to be, for that narrow purpose.

I think I've made a mistake in not being clear that I would suggest
taking the support element out of debian-user and then apply the
rigid rules to wherever that went, leaving debian-user as it is.
Instead I've talked like debian-user should be shut down and
re-opened in a completely different manner. I can see how that could
ruffle feathers and it was unintentional.

> I should respectfully suggest that this list be employed after all
> of them have been exhausted for an answer. Post your question,
> after checking the archives, and follow only that thread in order
> to avoid all the other aspects you see as undesirable.
> In this way, I believe any injury to your sensibilities can be easily
> avoided.

Do you see this as advice that belongs on the Debian web site next
to any directions towards debian-user? Just so the users are clear
what sort of experience awaits them?

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-09 Thread Andy Smith
Hello,

On Mon, Aug 09, 2021 at 04:06:43PM -0700, Weaver wrote:
> On 10-08-2021 07:54, Andy Smith wrote:
> > I really don't want to get into calling out specific sub-threads
> > that have been ridiculously off-topic recently, They are not hard to
> > find; there's just so many of them.
> 
> That wasn't the point. I pointed out an over dramatisation of the
> situation is not conducive to the sort of accurate pre-analytical stage
> required in order to specify a problem in order to deal with it
> accurately.
> Your answer to that is to specify `ridiculously off-topic threads'.

I'm saying that the number of off-topics posts here is often well in
excess of the number of on-topic ones, and that I think it isn't
conducive to user support.

And by "off-topic" I'm not talking about just replying in a
conversational tone, or asking for clarifications, or suggesting
other solutions or anything like that. I mean posts that become
totally unrelated to Debian and its use.

For the purposes of this conversation I do think these are easy to
spot. I do understand that you feel these aren't an issue. I'm just
saying why I don't think there's a need to specifically call these
out right now.

> Is there to be a rigorous deletion of anything  not Debian related
> within each email?

I think it would be best if such things were not posted here, yes,
not while this is the support venue.

> Is there to be no polite, courteous format, simply because it's not
> `Debian-related'?

I would really have hoped that it would be obvious that I'm not
asking for people to be impolite or discourteous; that I'm not
talking about normal conversational responses to support queries
being banned.

I'm talking about things that have drifted completely away from
being about Debian.

> Things can get just a little too rigid, on the way to creating a total
> lack of community that nobody wants to be a part of.
> 
> > I understand that there's plenty of people who think the current
> > situation is not a problem, but I think there's also people who do
> > think there is some issue here. I'm one of them and I'm giving my
> > opinion in a thread where it was specifically asked for.
> 
> And who isn't?

Well, this bit was in response to you saying, "if there's a problem
that requires resolution…" so was just me reiterating that I do
think there is, but that I do understand that plenty of people don't
think there is. i.e. this has not just come out of nowhere.

> > Absolutely, but it's discouraged by the format and what gets through
> > tends to be moderated away so it's less prominent. This results in a
> > better experience both for the question asker and later researchers
> > who come across it.
> 
> No, the rudeness is jumped on by members of the community more than
> `moderators'.
> The format changes nothing.

I'm afraid I don't understand what you mean. When I said
"moderated" here, I meant by the people doing the moderation,
which on a Stack site is mostly the community.

I do find that quite effective in making poor answers and disruptive
comments less visible on such sites, so I can't agree that the
format changes nothing.

> You might see one interjection as rude and unnecessary, while I might
> see it as a required ingredient in placing a clown in their place.

The idea that it would be necessary to put a clown in their place
publicly and with the same visibility as other posts in the thread
is something that feels to me most toxic in a support environment.

We've all been there - someone posts a silly, inadvisable,
ill-thought out or downright incorrect response to a support
question. One feels compelled to post a correction. Hopefully one
manages to do so without being overly offensive or cruel, but
putting a clown in their place can go that way sometimes. It's good
that the correction was delivered, less so if it ended up being
delivered in an offensive way, but even after that, the correction
just has the weight of one email in a thread.

Often times, the worst clowns are convinced they aren't clowns at
all. They will double down on their wrongness, and they can post
just as often as you can.

A lot of the time it needs experienced users to spot what is a
good answer (or good advice) and what is bad. It doesn't work so
well to go by who delivers the most devastating come-back or who
hammers their point home most forcefully or most often.

The Stack sites I frequent do seem to benefit from poor quality
answers and comments being moderated away. People can still engage
in back and forth conflict but what most people consider to be the
best answers float to the top. There isn't much need to place clowns
in their place for all to see and I think the support experience is
better for it.

> > There are good reasons why most times when I have a problem, a
> > search 

Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-09 Thread Andy Smith
On Mon, Aug 09, 2021 at 10:19:19PM +, Andy Smith wrote:
> I am asking the Debian Project to can this list in favour of an
> alternate solution or else to make it strictly for Debian-related
> posts only.

I want to walk this one back a bit as there's no need to destroy the
community that exists here already by forcing it to move elsewhere.

What I meant to say is that I think it'd be best if debian-user
continue to exist as it is while having its support element covered
by some other thing, and that be consistently documented throughout
Debian's documentation and websites wherever debian-user currently
is pointed to.

As mentioned, ideally I would see that support function served by a
Stack Overflow-like web site as I think those are easier to keep
focused for question-answer purposes. However if a mailing list
absolutely had to be used then it should be understood that posts
not related to Debian would not be welcome, and for that to be
enforced.

Thanks,
Andy



Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-09 Thread Andy Smith
Hello,

On Mon, Aug 09, 2021 at 07:26:50AM +0200, to...@tuxteam.de wrote:
> On Sun, Aug 08, 2021 at 09:33:02PM +0000, Andy Smith wrote:
> > My only suggestion was a Stack Overflow-style question-answer site.
> > Those aren't discussion forums.
> 
> Two things: (a) SO is a commercial site. It has paid staff.

In my first post in this thread I did point out that it would be
better done in project-hosted DFSG-free service and that it has
already been tried at least twice by Debian and failed due to lack
of use.

If your point here is "who's going to do the work?" Well, most of
the moderation work on such sites is done by the people using the
sites.

Personally speaking I'd much rather have the option to -1 a totally
off-topic response than have to just ctrl-r to mark read a whole
sub-thread of nonsense when it only benefits me to do so.

> in such an unstructured medium, some self-imposed rules
> may help.

If people were able to impose them, yes. But here we are.

> See, for example Andrew's regular posts. This isn't anything
> "imposed from above" -- he went around here asking for ideas, etc.

Andrew does have perceived authority as a Debian Developer.

I agree that Andrew's posts serve as a good reminder of how things
should be.

> You could do that.

I am doing that: I am asking the Debian Project to can this list in
favour of an alternate solution or else to make it strictly for
Debian-related posts only.

I've already said that I think there will be too much opposition to
that suggestion, so I don't expect to see it happen. But you did
ask.

Self-restraint isn't working.

> > Have a look at https://askubuntu.com/ to get some sort of idea,
> > since that is at least a Debian derivative. But as I said, it has
> > been tried before and I think won't/can't succeed without buy-in
> > from the project.
> 
> That's not my kind of discussion forum. I come across them through
> web searches, that's all the interaction I have there.

Yes, that is exactly my point in showing it to you. You have
correctly deduced that it is not a discussion site, mostly just
provide answers to your questions, and you have benefited from
finding those answers yourself.

You have also correctly seen my suggestion that Debian's primary
support venue should not have much interaction outside of problem
solving.

Those were my exact reasons for bringing up Ask Ubuntu.

I'm not saying there shouldn't be discussion and debate around
Debian (although I struggle to see why anyone would want to bring in
non-Debian topics even there). I'm just saying that the main place
for users to get support shouldn't be where that is expected to
happen.

> > > I feel we aren't doing that bad, considering the volume.
> > 
> > It is perhaps not so bad for a general Debian community discussion
> > group,
> 
> which is what it is

Yes, that's what it is right now, but you asked what could be
changed and I have suggested what I think should be changed about
Debian's primary user support venue. That it should not also be a
general Debian community discussion group.

> > where you would go into it thinking that pretty much anything
> > goes, but the fact is that this is Debian's primary support venue
> > for users new and old.
> 
> TBH, I've seen many people here finding answers to their questions.

I've not argued that zero people are helped. You created this
sub-thread in response to someone [else] who said that there was too
much off-topic posting here.

> > I don't think that both audiences can be catered for in the same
> > place and I really think that we could and should do better.
> 
> Feel free :)

Self-restraint isn't working.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-09 Thread Andy Smith
Hello,

On Sun, Aug 08, 2021 at 05:01:50PM -0700, Weaver wrote:
> On 09-08-2021 07:33, Andy Smith wrote:
> > A lack of politeness isn't really debian-user's biggest problem. I
> > think debian-user's biggest problem is the lack of restraint
> > prolific posters have on posting every thought that comes into their
> > heads and debating such into the ground.
> 
> We all have our perceptions.
> This would appear to be an overly dramatic one.
> `posting every thought that comes into their
> > heads and debating such into the ground' - really?
> If there's a problem requiring resolution, I think it might pay to be
> more concise than that.

I really don't want to get into calling out specific sub-threads
that have been ridiculously off-topic recently, They are not hard to
find; there's just so many of them.

If you can't see this then I just have to assume that you don't find
the current situation to be a problem, in which case I don't know
how to convince you that there's a problem. It doesn't seem like me
listing out sub-threads that have departed far from anything
Debian-related would convince you, and would probably only serve to
feel like an attack on individual posters.

I understand that there's plenty of people who think the current
situation is not a problem, but I think there's also people who do
think there is some issue here. I'm one of them and I'm giving my
opinion in a thread where it was specifically asked for.

> > That sort of thing is not really possible on a question-answer site
> > as conciseness is rewarded in both question and answer.
> 
> Not in the reality I inhabit.
> I'm a member of a couple of stack sires, and I have witnessed many a
> humorous aside and any number of examples of downright rudeness.

Absolutely, but it's discouraged by the format and what gets through
tends to be moderated away so it's less prominent. This results in a
better experience both for the question asker and later researchers
who come across it.

By contrast on a mailing list like this it's about who shouts
loudest and most often, and that's even before the posts start
appearing that are not even about Debian at all.

There are good reasons why most times when I have a problem, a
search engine expedition will usually lead me to answers on Stack
Overflow-like sites before the archives of discussion lists.

> There is also an extremely efficient means of weeding out those
> conversations an individual sees as not necessary for their immediate
> notice, or downright unnecessary, and ones they see as beng answerable -
> within ther capability - and of interest. Personally, that takes me all
> of 5 seconds.

New users can't do this. Of course they can be taught but that is a
huge impediment to getting their problems solved.

People coming by later to find answers also still have to sift
through it all.

It seems really odd to take the position that the primary venue for
user support must be drowned in content that is not about use of
Debian, because anyone who isn't interested in that can just filter
it away.

> > Off-topic discussion is specifically something which I suggest there
> > is too much of here.
> 
> It depends on what you see as `off-topic'. Your view is yours, and not
> necessarily everybody's.
> Do you see the value in discussion, yet?

I get that everyone has different opinions about what this list is
for. I think I'm being pretty clear in expressing the opinion that
it is for user support, not general debate. It's not a problem when
there's a slight amount of debate around the problems and solutions.
It is a problem when threads shift entirely away from the use of
Debian. Again, I really don't want to have to call out specific
recent incidents as I think they're easy to recognise.

I'm not saying that I think such conversations shouldn't be had,
anywhere, just that the primary place for support shouldn't be the
place that they happen, and that I think this could best be achieved
by not using a discussion list for it in the first place.

So no, I don't see the value of such wide-ranging discussion in the
support venue, even having given a fair amount of it over a
reasonably long period of time. There is nothing that has convinced
me that diverging off into some topic not at all related to use of
Debian has value here. I've done it myself, regretted it later, it's
usually been a product of frustration and I wish it wasn't tolerated
here from me or anyone else.

> > It is perhaps not so bad for a general Debian community discussion
> > group, where you would go into it thinking that pretty much anything
> > goes, but the fact is that this is Debian's primary support venue
> > for users new and old.
> 
> Something it has been doing very well at for some considerable time now.

This sub-thread asking for suggestions on how to improve the list
could ha

Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-08 Thread Andy Smith
Hello,

On Sun, Aug 08, 2021 at 11:00:55PM +0200, to...@tuxteam.de wrote:
> On Sun, Aug 08, 2021 at 03:26:25PM +0000, Andy Smith wrote:
> > To be honest I don't think that mailing lists are a very good venue
> > for user support and I would these days prefer to direct people to a
> > Stack Overflow-like site [...]
> 
> I stringly disagree on that one. There's tooling and there's
> politeness, and they are, IMO, uncorrelated variables.

A lack of politeness isn't really debian-user's biggest problem. I
think debian-user's biggest problem is the lack of restraint
prolific posters have on posting every thought that comes into their
heads and debating such into the ground.

That sort of thing is not really possible on a question-answer site
as conciseness is rewarded in both question and answer.

> Some people are rather wired towards "forum style", others more
> towards "mail style" -- and I think that's why this kind of
> discussion tends to come up time and again.

I haven't advocated for a forum. What I've suggested is that a
discussion list tends to promote discussion, not user support.

> > The main reason why I see mailing lists as inappropriate for user
> > support is that there is a severe signal to noise ratio problem.
> 
> I think you'll get the same on unmoderated fora.

…which, again, I haven't suggested. I don't know why you keep going
back to the idea of web forums. It's obvious that a web discussion
forum would have the same problems as an email discussion list, if
it were unmoderated.

My only suggestion was a Stack Overflow-style question-answer site.
Those aren't discussion forums.

Off-topic discussion is specifically something which I suggest there
is too much of here.

Have a look at https://askubuntu.com/ to get some sort of idea,
since that is at least a Debian derivative. But as I said, it has
been tried before and I think won't/can't succeed without buy-in
from the project.

> I feel we aren't doing that bad, considering the volume.

It is perhaps not so bad for a general Debian community discussion
group, where you would go into it thinking that pretty much anything
goes, but the fact is that this is Debian's primary support venue
for users new and old.

I don't think that both audiences can be catered for in the same
place and I really think that we could and should do better.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: On improving mailing list [was: How to Boot Linux ISO Images Directly From Your Hard Drive Debian]

2021-08-08 Thread Andy Smith
Hello,

On Sun, Aug 08, 2021 at 11:35:15AM +0200, to...@tuxteam.de wrote:
> any ideas on how to make the situation better?

To be honest I don't think that mailing lists are a very good venue
for user support and I would these days prefer to direct people to a
Stack Overflow-like site. The chief advantages of such sites are
that posted problems are narrowed down to contain the required
information, and answers are ranked so as to make poor answers (and
ultimately, disruptive posters) disappear. Ask Ubuntu. I think,
works well.

There have been a few attempts to set up such sites for Debian, so
that people could be directed to a site running on DFSG-free
software instead of proprietary platforms like Stack Overflow. Sadly
each of these efforts have foundered through lack of use.

I don't see the lack of use as an indictment of their effectiveness;
rather I think it's just because it's too hard to change the status
quo without significant work.

The previous attempts have sort of started as an announcement that
such a site is available, but not followed up by any level of
advertising on Debian's web site. The announcement threads on the
mailing lists then got dominated by arguments from the same small
group of people loudly and repeatedly arguing how they would never
use or support such a thing. That's fine, but without a way to
continually advertise a site as a support venue, it will not get
used.

The main reason why I see mailing lists as inappropriate for user
support is that there is a severe signal to noise ratio problem.

In debian-user there's a relatively small group of people who value
getting their opinions on a vast variety of topics across more than
they value actually answering on-topic questions. So we endure
mega-threads of opinion-based off-topic content that regularly
descend into personal attacks. It is hard to sift through all of
this for nuggets of on-topic wisdom and even when a post is
on-topic, a newcomer often doesn't have the base knowledge to
distinguish good answers from bad. I find it hard to justify
subjecting someone to that.

Naturally the voluminous opinion-posters are mostly against anything
that would reduce their ability to treat debian-user like a debating
society, so effecting change is going to be hard if the only metric
one would use to justify the change would be a simple noise-based
sentiment analysis of the response to any proposal.

If we have to continue using a mailing list for user support then my
best suggestion would be to severely tighten up the on-topic
requirements so that every post must be about use of Debian, and
giving time-outs to posters who repeatedly can't stick to this.

This would be a difficult and dramatic change since debian-user has
practically no oversight; currently even severe breaches of Debian's
Code of Conduct need to be reported to the Community team directly
and at most result in a post weeks later saying, "please don't do
that" directed at no individual. So the idea that there would be
people actively dealing with off-topic content and taking action
against individuals would be quite a departure from today's reality.

So in summary, I don't think any of the things that would be
necessary to improve the way this list works are going to be popular
with the regular posters, while starting over with a different
solution requires consensus and support from the Debian project that
has up until now not been there.

We can try to self-moderate by asking ourselves, "does my reply help
the poster? Does it belong on debian-user?" Unfortunately for some
the mind set is, "I'm a user of Debian so any opinion I wish to post
is on-topic on debian-user". I appreciate I have also failed at this
from time to time and I include myself in the list of those who
should do better. Ways of making us do better are needed.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

> I'd be interested to hear any (even two word) reviews of their sofas…
Provides seating.— Andy Davidson



Re: why 1G memory is missing?

2021-08-03 Thread Andy Smith
Hello,

On Tue, Aug 03, 2021 at 07:14:32AM +0800, loushanguan2...@sina.com wrote:
> Linux debian 4.9.0-13-686-pae #1 SMP Debian 4.9.228-1 (2020-07-05) i686 
> GNU/Linux

So you are running 32-bit kernel. Will the hardware do 64-bit? What
does

$ cat /proc/cpuinfo

say?

You may be able to install an amd64 kernel and see all memory even
though userland is still 32-bit.

Thanks,
Andy



Re: Updating kernels impossible when /boot is getting full

2021-08-01 Thread Andy Smith
Hello,

OP: You are pretty safe deleting (rm) vmlinuz* and initrd* things
from /boot that are related to any kernels you aren't actually
booted into at the time. That can give you back enough space to let
apt finish what it wants to do. Just remember to do:

# update-initramfs -u -k all

afterwards to regenerate the initrd for any installed kernel that
you do want to boot into next time.

I would suggest not deleting the initrd* for the current kernel
because if you find yourself unable to regenerate it for any reason
then you have a system that can't be rebooted. If you leave the
current kernel's files alone then at least you know you have a
known-good setup.

On Sun, Aug 01, 2021 at 10:51:59AM -0400, Gene Heskett wrote:
> So If you wind up reinstalling, make /boot a minimum of 2G so you do not 
> hit this situation in the lifetime of the hardware ever again, make 2x 
> you memory as swap, and split the rest into / and /home. It just works 
> for me, your storage will have room to keep itself clean and functional. 
> But YMMV. :)

There is very little advantage these days to separating out /boot
and /home etc on such devices. You are far better off just putting
it all in / and making sure you have backups a quick way to re-image
the thing.

If you absolutely must do it, I advise making a fairly small / (5G
or so counts as small these days) that has /boot in it (not separate
fs) and then do your other splits with a volume manager like LVM,
btrfs or ZFS.

Splitting things into multiple filesystems fundamentally invites
problems such as the one encountered in this thread - you guess
wrong and make something too small. Nobody is perfect or omniscient
so this happens quite often. Meanwhile a lot of the reasons for
splitting things up have been obsoleted back in the mists of time.
Just strongly consider not doing it any more and see if your life
improves.

At this point there will probably be some people who consider
themselves veterans saying that one must absolutely split things off
because of various reasons like differing mount options being
desirable, ability to re-use contents of /home after reinstall,
having multiple devices and wanting to suit filesystem contents to
drive characteristics, … or whatever. Most of that will not apply to
any given person, and if it does then I believe it's better done
with volume management.

So really think hard before splitting off a filesystem outside of
volume management. I believe it is more likely to cause problems
than it is to avoid problems.

I think I've heard all the arguments for doing it and I also agree
with some of them in some situations - but since the '90s we've had
volume management to help with this. If someone has come up with
some obscure reason why they must split their storage into multiple
mount points with no volume management then I don't need to hear
about it - I'm happy to believe them that it may be necessary for
their specific circumstances while also not agreeing that it's a
good idea in the general case!

Gene's recommendation is to not spare the drive capacity and be
generous, but then Gene recommends doing something that severely
restricts drive capacity: making hard-to-change decisions about
carving it up. I agree with Gene's suggestion to be generous with
capacity, and I suggest that is achieved by just giving it all to /
unless you have very very good reason not to (and then use volume
management if you must).

Cheers,
Andy



Re: merged /usr considered harmful (was Re: Bits from the Technical Committee)

2021-07-18 Thread Andy Smith
Hello,

On Sun, Jul 18, 2021 at 04:31:11PM -0400, Polyna-Maude Racicot-Summerside wrote:
> My personal opinion is that Debian is going into a mostly "we got the
> best idea in the world but forgot that not everyone implement things the
> same way".

I recommend understanding the issue before putting forth an opinion.

> I currently have a different partition for my /usr and this has been the
> case since the end of 1990s when I started on Linux. Maybe I'm wrong but
> I like it this way.
> 
> Will the merge-usr cause myself problem ?

No. Not as long as you use an initramfs created by any of the
supported Debian tools like initramfs-tools or dracut, which you
will do unless you have gone out of your way to do something
different.

And regardless of merged-/usr, /usr on separate partition has not
been supported in Debian without an initramfs since the stretch
release in 2017.

I think all of this is quite clearly explained in:

https://salsa.debian.org/md/usrmerge/raw/master/debian/README.Debian

which is linked from:

https://wiki.debian.org/UsrMerge

If you think it's not then you should probably raise a bug against
the usrmerge package with your suggested edits/clarifications.

Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: RAID-1 and disk I/O

2021-07-17 Thread Andy Smith
Hi Urs,

Your plan to change the SATA cable seems wise - your various error
rates are higher than I have normally seen.

Also worth bearing in mind that Linux MD RAID 1 will satisfy all
read IO for a given operation from one device in the mirror. If
you have processes that do occasional big reads then by chance those
can end up being served by the same device leading to a big
disparity in per-device LBAs read.

You can do RAID-10 (even on 2 or 3 devices) which will stripe data
at the chunk size resulting in even a single read operation being
striped across multiple devices, though overall this may not be more
performant than RAID-1, especially if your devices were
non-rotational. You would have to measure.

I don't know about the write overhead you are seeing.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Memory allocation failed during fsck of large EXT4 filesystem

2021-07-06 Thread Andy Smith
Hello,

On Tue, Jul 06, 2021 at 02:34:30PM +0300, IL Ka wrote:
> > I use a 32bit OS

Is the hardware capable of 64-bit? If so then it should be possible
to install an amd64 kernel and e2fsprogs without completely
converting your system to amd64.

https://wiki.debian.org/CrossGrading

(Stop after booting in to the new kernel and then install
e2fsprogs:amd64)

You should then be able to fsck larger ext* filesystems.

There is also the "crossgrader":

https://packages.debian.org/bullseye/crossgrader

(though it is only in testing and unstable it is intended to work on
stable as well)

You may or may not find that simpler.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Server setup

2021-06-14 Thread Andy Smith
Hello,

On Mon, Jun 14, 2021 at 04:39:11PM -0400, Polyna-Maude Racicot-Summerside wrote:
> I can understand the idea of cutting out part of the messages when I
> answer. But this is now forcing me to repeat many times...

You're being asked direct questions because your rambling style has
no real information about what exactly you're trying to achieve.

Instead of answering the direct questions you provide rude responses
berating the people that are trying to help you. You are what is
known as a support vampire and as a rule I do not participate in
that. Good luck.

Andy



Re: Server setup

2021-06-14 Thread Andy Smith
Hi Polyna-Maude,

On Mon, Jun 14, 2021 at 12:31:08PM -0400, Polyna-Maude Racicot-Summerside wrote:
> Now what I did was to install the machine using the "helper" given by
> the provider (OVH/OneProvider). This way I can dissect the working
> system and see how the configuration is done.

So what does it look like after that, and what do you want to
change?

Maybe you can make the desired changes without reinstalling by
debootstrap. But if that's necessary, at least we'll understand what
it is that you want to achieve with that.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: How to manage a firewall script with minor tweaks for different machines?

2021-06-12 Thread Andy Smith
Hello,

On Sat, Jun 12, 2021 at 07:02:50PM +0300, Anssi Saari wrote:
> But then... One machine has a radius server that needs UDP port 1812
> open. And another is a print server with CUPS and SMB which apparently
> need at least TCP ports 631 and 137 open.

It sounds like you need configuration management software. You
already have several machines by the sounds of it, so it's a good
time to look in to it.

Ansible can be very simple and quick to learn and everything you've
mentioned in your post can easily be done with it.

I found Puppet a bit of a nicer thing to develop in, but a lot more
complicated and a lot more work to keep up to date, so I switched to
Ansible.

Other configuration management software is available and I don't
think it matters that much which one you go for; you will no doubt
discover your preferences.

All configuration management solutions will cover the use case of
different config for different hosts or groups of hosts, templating
of configuration files, and pushing files and assets out to where
they need to be.

You can invent your own with a big shell script and an ssh loop but
to be honest, Ansible is really very simple, may as well use
something that has solved all these problems.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: ifconfig stats ??? Wrong behavior OR BUG ?

2021-06-11 Thread Andy Smith
Hi Kanto,

On Fri, Jun 11, 2021 at 02:01:02PM +, Kanto Andria wrote:
> dada@Jradebian:~$ sudo ifconfig enp0s31f6 stats    

You just resolved "stats" in DNS and set the IP address of interface
enp0s31f6 to that IP.

>     inet 54.36..162.17  netmask 255.0.0.0  broadcast 54.255.255.255

I'm guessing that due to your DNS configuration, partiocularly
search list, or any default answer your ISP provides, 54.36.162.17
came back as the answer to looking up "stats".

> So the main question: Why the stats did not generate error like no such 
> options ?

There is no way to tell that you didn't mean to look up a name in
DNS.

> Why the IP is reset to a public IP here belonging to OVH?

That's what it resolved to.

This is not a bug.

The bigger issue here is that ifconfig has been deprecated for 10+
years and whatever you are trying to do with it you should be doing
with more modern commands, usually something out of the "iproute"
package, e.g. "ip".

What stats are you trying to view? Maybe "ip -s link"?


https://access.redhat.com/sites/default/files/attachments/rh_ip_command_cheatsheet_1214_jcs_print.pdf

may help.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Slightly off-topic: anybody know of a way to keep one's Debian User List posts from failing DMARC?

2021-06-09 Thread Andy Smith
Hello,

On Wed, Jun 09, 2021 at 09:58:13AM -0700, James H. H. Lampert wrote:
> I'm told that the Debian List Server doesn't rewrite "From"
> headers for DMARC-enabled senders, and neither does it do anything
> else to handle DMARC-enabled senders.

Correct, so the SPF test will always fail as the list pretends to be
your domain when it sends out mail from you. But the DKIM test can
still pass, because the Debian list software does not alter the body
of your mail or any of the headers you are likely to sign with DKIM.

And indeed, your email that I am replying to was a DKIM pass and
thus would be a DMARC pass as only one of the two is needed.

Many mailing lists modify the body, e.g. to prepend tags to the
subject and/or to append a footer with list information. These
mailing lists can never pass DKIM because the mail content is
signed. Their only option if they care about a DMARC pass is to
rewrite the sender address so that their mail comes from their own
domain, then they can make SPF and DKIM pass and alter the content
as they like.

You will see many DMARC failures from such mailing lists.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: ipv6 stable privacy (rfc 7217) as default ipv6 address for outgoing traffic

2021-05-25 Thread Andy Smith
Hi Serge,

On Mon, May 24, 2021 at 11:51:12AM +0200, Serge Pouliquen wrote:
> I'm getting 2 addresses : one from slaac with stable privacy and one from
> dhcpv6.
> It looks like the one from dhcpv6 is used as the default for outgoing
> traffic.
> 
> How can I indicate that I want to use the one related to stable privacy as
> the default outgoing address ?

I'm not sure if you can do it with DHCPv6.

If you can find a way to alter your routing table, you would be
wanting to set the source address on your v6 default route.

> I can also disable dhcpv6, but I would like to keep dhcpv6.
>   iface enp6s0 inet6 auto

Could you maybe use a post-up directive to change your default
route?

$ ip -6 route show default
default via fe80::200:24ff:fec4:36dd dev enp0s31f6 proto ra metric 100 pref 
medium
$ my_route=$(ip -6 route show default); ip -6 route change $my_route src 
2001:8b0:ca07:c57a:127b:44ff:fe93:fac4
$ ip -6 route show default
default via fe80::200:24ff:fec4:36dd dev enp0s31f6 proto ra src 
2001:8b0:ca07:c57a:127b:44ff:fe93:fac4 metric 100 pref medium

(might possibly want to check that your v6 default route doesn't
already have a src ip. Also there can be multiple default routes…)

The other common way to influence source address selection is to set
preferred_lft to 0 for every IPv6 address that you DON'T want used
as a source address. Such addresses will then be marked as
"deprecated"; they will still accept traffic but will not be
selected as the source address unless forced.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: grub-set-default

2021-05-19 Thread Andy Smith
Hi Richmond,

On Wed, May 19, 2021 at 04:34:34PM +0100, Richmond wrote:
> grub-set-default 5
> 
> I cannot see any changes in /etc/grub.d/ or /etc/default/grub

grub-set-default makes changes to /boot/grub/grubenv, which is read
by the grub binary at boot time. You can examine its contents with:

$ sudo grub-editenv list
saved_entry=Advanced options for Debian GNU/Linux (with Xen hypervisor)>Xen 
hypervisor, version 4.14.2-amd64-xsa368>Debian GNU/Linux, with Xen 
4.14.2-amd64-xsa368 and Linux 4.19.0-16-amd64

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Restoring sticky bits after accidentally moving /usr directory

2021-05-19 Thread Andy Smith
Hello,

On Tue, May 18, 2021 at 11:40:38PM -0400, Greg Wooledge wrote:
> On Tue, May 18, 2021 at 11:46:38PM +0000, Andy Smith wrote:
> > I can't think of an easy way if you don't have backups. If you have
> > another system you could get a list of all its permissions like so:
> > 
> > # find /usr -xdev -printf '%p %m\0' | sort -z > good-perms
> > 
> > Then on your suspect machine:
> > 
> > # find /usr -xdev -printf '%p %m\0' | sort -z > suspect-perms
> > 
> > And then run this perl script:
> > 
> > https://gist.github.com/grifferz/1c478ea5eb789b2a1d1a3e49d2a9345c
> 
> The serialization format you're using (pathname followed by mode) is
> not ideal for parsing.  I'd suggest putting the mode *first*, because it's
> in a known, fixed format, and then the pathname second.

I was wondering about that at the time but it seemed like the end of
each item could only ever be a space, a sequence of digits and then
\0, so I didn't think it could be a problem. But yeah, not too many
changes needed if you want to switch them around.

I also realised later that there isn't too much point in sorting the
both the file lists. I did that because I was initially going to use
the "comm" command, which needs the files to be sorted, but later
decided against it.

If you are happy with seeing the results in the order that the
"suspect-perms" find command outputs them, then just don;t bother
sorting either of them.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Restoring sticky bits after accidentally moving /usr directory

2021-05-18 Thread Andy Smith
Hello,

On Tue, May 18, 2021 at 06:26:18PM -0400, Steve Dondley wrote:
> I goofed up and accidentally moved my /usr directory while trying to make
> room on a full drive. I was able to recover, but I'm finding that services
> are not working because the sticky bits for many files /usr/bin/* were lost.
> For example, I can't send email with exim because of this error:
> 
> Failed to create spool file /var/spool/exim4//input//1lj87g-0002tS-5J-D:
> Permission denied

I'm guessing you actually mean setuid/setgid bit, not sticky bit.

> Is there an easy way to ensure I set all the permissions back to where they
> were before I move /usr?

I can't think of an easy way if you don't have backups. If you have
another system you could get a list of all its permissions like so:

# find /usr -xdev -printf '%p %m\0' | sort -z > good-perms

Then on your suspect machine:

# find /usr -xdev -printf '%p %m\0' | sort -z > suspect-perms

And then run this perl script:

https://gist.github.com/grifferz/1c478ea5eb789b2a1d1a3e49d2a9345c

The "find" and the "sort" are using NULL-separated strings so that
your filenames can contain newlines. Although I don't expect you
have any such paths under /usr.

The perl script will print out a chmod for any differences, it will
tell you about paths you have which your "good" host does not, and
it will say nothing about paths that match permissions both sides.
It doesn't actually do anything, it just prints suggested chmod
actions. You maybe want to capture the output to a file.

If you don't have another working system, well, perhaps you can tell
us which Debian release this is and someone can provide a list of
paths and permissions from their machine.

Good luck!

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: AppleWEBkit

2021-05-11 Thread Andy Smith
Hello,

On Tue, May 11, 2021 at 10:22:27PM -0400, Gene Heskett wrote:
> Is AppleWebKit a bot?

"AppleWebKit" is found in the user agent of web browsers on macOS
and iOS, including Google Chrome.

https://developers.whatismybrowser.com/useragents/explore/layout_engine_name/webkit/

Since anyone can set any user agent, however, malicious bots often
pick the user agent of a regular web browser. So while you may see
bots using a user agent setting that contains that string, blocking
clients with that string in their user agent is not typically
helpful.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: command to start sshfs at bootup?

2021-04-18 Thread Andy Smith
Hi Gene,

On Sun, Apr 18, 2021 at 12:41:05PM -0400, Gene Heskett wrote:
> My best guess, and admittedly a WAG, certainly not a SWAG, is that ssh 
> and its ilk have been around for yonks. It has not been updated to allow 
> what the lastest rfc now does. Based on that, perhaps a low priority bug 
> should be entered?

It's trivial to test this without wasting anyone's time:

$ getent hosts specialbrew
2001:8b0:ca07:c57a::5 specialbrew.localnet
$ ssh specialbrew hostname
specialbrew.localnet
$ echo "2001:8b0:ca07:c57a::5 specialbrew.localnet 3gene" | sudo tee -a 
/etc/hosts
2001:8b0:ca07:c57a::5 specialbrew.localnet 3gene
$ ssh 3gene hostname
The authenticity of host '[3gene]: ([2001:8b0:ca07:c57a::5]:)' can't be 
established.
ECDSA key fingerprint is SHA256:fdRpfj3HfBaY3K3lCyrBIU2+/bUPX1e5EtC2/bInB5w.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[3gene]:' (ECDSA) to the list of known hosts.
specialbrew.localnet

So no, you are not correct. If you somehow still think you are
please work out a reproducible test that confirms your theory,
before reporting a bug that forces someone else to spend time
debunking it.

Since you have a habit of not issuing commands people ask you to,
and issuing other commands without reporting that you have done so nor
recording what the effect was, my best guess is that you simply
changed something else that you have no recollection of. Another
string possibility is that you made a human error in the first
place. Now it works but you don't know what changed.

The "it starts with a number" theory is not it.

Regards,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm and whole disk array members

2021-03-26 Thread Andy Smith
Hello,

On Fri, Mar 26, 2021 at 02:31:21PM -0400, Gary Dale wrote:
> I'm not convinced that the problem is the BIOS writing a partition
> table. In your link the last post talks about zapping the
> partition table to stop the behaviour. This suggests the BIOS/UEFI
> was restoring the backup partition table. However I'd already
> zapped the partition table on my disks so that wouldn't be my
> case.

It should be pretty easy to check if this is what's happening,
without md being in the picture, if you have a will.

Though if you now have a working system I appreciate that you
probably don't want to tear it apart to test a theory.

I was only mentioning it as this is the only time I have ever seen
anything remotely like what you describe.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm and whole disk array members

2021-03-26 Thread Andy Smith
Hello,

On Mon, Mar 22, 2021 at 06:20:56PM -0400, Gary Dale wrote:
> I've found many other people complaining about similar issues when using
> whole disks to create mdadm RAID arrays. Some of these complaints go back
> many years, so this isn't new.

Any time I've seen this problem pursued (as opposed to throwing up
hands and just using partitions) it was found to be the motherboard
(well, the UEFI bit) deciding that a storage device with no GPT or
MBR needs an empty GPT adding to it. This corrupts md arrays on whole
devices.

Example:
http://forum.asrock.com/forum_posts.asp?TID=10174=asrock-motherboard-destroys-linux-software-raid

Is it possible that this is happening to you?

If not, once again I urge you to go on over to linux-raid list and
describe what's happening.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm and whole disk array members

2021-03-22 Thread Andy Smith
Hi Gary,

On Mon, Mar 22, 2021 at 06:20:56PM -0400, Gary Dale wrote:
> I suggest that, since it appears the developers can't get this work
> reliably, that the option to use the whole disk be removed and mdadm insist
> on using partitions. At the very least, mdadm --create should issue a
> warning that using a whole device instead of a partition may create
> problems.

I've been using whole disks in mdadm arrays for more than 15 years
across many many servers on Debian stable and have never experienced
what you describe. There must be something else at play here.

I suggest you post a detailed description of your problem to the
linux-raid mailing list and hopefully someone can help debug it.

https://raid.wiki.kernel.org/index.php/Linux_Raid#Mailing_list

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Hardware failure?: Now what?

2021-03-20 Thread Andy Smith
Hi,

On Sat, Mar 20, 2021 at 02:29:25PM -0600, Charles Curley wrote:
> MCE events:
> 1 2021-03-20 13:58:30 -0600 error: Internal parity error, mcg mcgstatus=0, 
> mci Corrected_error Error_enabled, mcgcap=0x0c09, 
> status=0x904f0005, tsc=0xf442c87fda, walltime=0x605653e5, 
> cpu=0x0003, cpuid=0x000306c3, apicid=0x0006

This could be a RAM error, but it could also be a memory error for
the cache inside the CPU, so a CPU error. But it could also be a
spurious CPU bug:

https://trick77.com/qemu-on-haswell-causes-spurious-mce-events/

Are you running qemu or KVM or some other kind of virtualisation? If
yes and if there doesn't appear to be any actual instability then it
may be spurious.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: fsck error on boot: /dev/sda1: UNEXPECTED INCONSISTENCY and Partition 1 does not start on physical sector boundary

2021-03-20 Thread Andy Smith
Hello,

On Fri, Mar 19, 2021 at 10:36:37PM +0500, Alexander V. Makartsev wrote:
> Personally, I don't think it is wise to throw away any HDD as soon as it
> gets a few pending bad blocks for whatever reason.

It really depends upon your risk stance.

At home, on my home fileserver, it has RAID, it has backups, so if a
HDD sees a few remapped sectors I'm not going to throw the HDD out.
When it starts seeing many many increasing numbers of remapped
sectors then yes it's being replaced. But indeed it can be many
years between picking up a few remapped sectors and complete
meltdown.

https://gist.github.com/grifferz/64808f61079fe610c6f21f03ac7fd1aa

$ sudo ./blkleaderboard.sh 
 sdd 100418 hours (11.45 years) 0.29TiB ST3320620AS
 sdb  95783 hours (10.92 years) 0.29TiB ST3320620AS
 sda  94252 hours (10.75 years) 0.29TiB ST3320620AS
 sdi  66276 hours ( 7.56 years) 0.45TiB ST500DM002-1BD14
 sdk  55418 hours ( 6.32 years) 2.73TiB WDC WD30EZRX-00D
 sdh  44511 hours ( 5.07 years) 0.91TiB Hitachi HUA72201
 sde  24239 hours ( 2.76 years) 0.91TiB SanDisk SDSSDH31
 sdc  17672 hours ( 2.01 years) 0.29TiB ST3320418AS
 sdf   7252 hours ( 0.82 years) 1.82TiB Samsung SSD 860
 sdj   7130 hours ( 0.81 years) 1.75TiB KINGSTON SUV5001
 sdg   1560 hours ( 0.17 years) 1.75TiB KINGSTON SUV5001

I've replaced some drives in the last 2 years and those ones, once
they started gaining reallocated sectors they didn't survive long
even though I gave them the chance. Hence the three replacements in
the last 2 years. sdc and sdd are hanging on:

$ for d in /dev/sd?; do echo -n "$d: "; sudo smartctl -A $d | grep '^  5'; done
/dev/sda:   5 Reallocated_Sector_Ct   0x0033   100   100   036Pre-fail  
Always   -   0
/dev/sdb:   5 Reallocated_Sector_Ct   0x0033   100   100   036Pre-fail  
Always   -   0
/dev/sdc:   5 Reallocated_Sector_Ct   0x0033   097   097   036Pre-fail  
Always   -   151
/dev/sdd:   5 Reallocated_Sector_Ct   0x0033   100   100   036Pre-fail  
Always   -   5
/dev/sde:   5 Reallocated_Sector_Ct   0x0032   100   100   ---Old_age   
Always   -   0
/dev/sdf:   5 Reallocated_Sector_Ct   0x0033   100   100   010Pre-fail  
Always   -   0
/dev/sdg:   5 Reallocated_Sector_Ct   0x0033   100   100   010Pre-fail  
Always   -   0
/dev/sdh:   5 Reallocated_Sector_Ct   0x0033   100   100   005Pre-fail  
Always   -   0
/dev/sdi:   5 Reallocated_Sector_Ct   0x0033   100   100   036Pre-fail  
Always   -   0
/dev/sdj:   5 Reallocated_Sector_Ct   0x0033   100   100   010Pre-fail  
Always   -   0
/dev/sdk:   5 Reallocated_Sector_Ct   0x0033   200   200   140Pre-fail  
Always   -   0

At work, where's it's other people's data on the line, drives get
replaced soon as they show any defect like that, as when it does
escalate it tends to do so very quickly.

My own risk stance doesn't even permit running without redundancy
(unless inherently impossible due to the machine in question not
supporting that), because once you encounter Offline_Uncorrectable
in normal daily use it means that without redundancy, data loss has
occurred.

The drive couldn't read one or more of its sectors. If it's just
file data you can get it from backup but if, like OP here, it's
filesystem metadata then your actual filesystem is damaged and needs
fsck. And if unluckier still, whole filesystem can be broken. I'd
really rather not have to spend time on fixing that sort of thing.

> Even brand new drives are shipped with information about factory remapped
> sectors in special section inside their firmware, to cover up platter
> imperfections.

That's true, and to some extent with the densities in use today all
reading from drive is probabilistic and corrected by checksums. But
when they arrive like that they are supposed to be in a stable
state, without such errors increasing, so when they do start to
appear it is a cause for serious concern.

> This is why performing regular backups and validating them is better, I mean
> you do it all anyway, than replacing drives as soon as they get a few bad
> sectors.

I would say the two strategies are orthogonal because backups and
self-tests are advisable for everyone. Once a drive gets some
Offline_Uncorrectable the data is gone from it; backups and
self-tests didn't stop that from happening, they just helped you
recover from it (backups) or spot it early by testing even unused
areas of the drive (self-tests).

Anyway in OP's position, they have lost data which they need to
restore and while they could wait and see if the errors are
increasing in number they probably just want to get it replaced
ASAP.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Package release number.

2021-03-20 Thread Andy Smith
Hi Peter,

On Sat, Mar 20, 2021 at 08:16:22AM -0700, pe...@easthope.ca wrote:
> peter@joule:/home/peter$ dpkg -l | grep qemu-system-x86
> ii  qemu-system-x86   1:3.1+dfsg-8+deb10u8
>   i386 QEMU full system emulation binaries (x86)
> 
> How is the "version number" interpreted?

It's ony important within Debian so aside from some conventions it
does *have* to correspond to anything.

> "1:" ?

This is referred to within Debian and derivatives as "an epoch". It
is typically used when the format of the rest of the version needs
to change in ways that would not otherwise guarantee that subsequent
versions are considered newer than previous versions.

Something with an epoch of 1: will be considered newer than
something without an epoch, and if an epoch happened again then it
would be 2: and that would be newer than anything that starts with
no epoch or with 1: as an epoch.

> "3.1" qemu release number?

Yes, it is desirable to match the Debian package version with the
upstream version that it's based upon. Sometimes this is done
incorrectly and it has to be fixed and then a typical convention is
to use another suffix of "-reallyx.y.z".

> "dfsg-8" ?

A convention indicating that the package includes some number of
Debian-specific changes to make the package comply with Debian Free
Software Guidelines. For example, some documentation contains
invariant sections that no one has permission to change and those
don't fit what Debian considers to be "free", so they can get
stripped out.

https://wiki.debian.org/GFDLPositionStatement

> "deb10u8" Debian 10, update 8?

Yes, a convention saying that the basis for this package is the
version that first appeared in release 10 and this is the 8th update
to it since then.

> Additional ideas?

There are a lot of other conventions in use (the "-really" one being
one example) and I'm not sure if they are all listed out somewhere.

> I checked /https://www.debian.org/distrib/packages for an explanation.
> Nothing relevant.

If anywhere, I would expect it to be in documentation aimed at
Debian developers and contributors.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Where can I change spamd logging?

2021-03-14 Thread Andy Smith
Hello,

On Sat, Mar 13, 2021 at 05:10:42PM -0500, Gene Heskett wrote:
> OPTIONS="--create-prefs --max-children 5 --helper-home-dir -s ~/log/mail.log"

spamd is a system service and it normally (initially) runs as root,
so using a ~ there probably isn't what you want. Storing logs from
such a daemon inside your home directory also doesn't seem
appropriate.

> Its not hitting the named file, but its not spamming syslog any more.
> So I've no clue where all that is going

I wouldn't be surprised if it had ended up somewhere inside /root
(user root's home directory), or nowhere.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Where can I change spamd logging?

2021-03-13 Thread Andy Smith
Hi Gene,

On Fri, Mar 12, 2021 at 11:25:20PM -0500, Gene Heskett wrote:
> What file, and where, do I edit to put that log someplace else?

What's unclear or not working about the --syslog= option in "man
spamd"?

https://manpages.debian.org/stretch/spamassassin/spamd.8p.en.html

You can change the options that spamd runs with by editing
/etc/default/spamassassin.

Don't forget to arrange for log rotation of whatever file you do
redirect this to.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Deb10 installer can't install grub

2021-03-05 Thread Andy Smith
Hello,

On Fri, Mar 05, 2021 at 10:04:34AM +, Darac Marjal wrote:
> So, if your file is small, then yes, you won't see any performance
> benefit. But if your file is larger than a block, or if you want
> to access more than one file at once, then RAID can read the
> second block from a different drive.

However, Linux MD RAID-1 will only ever achieve the read performance
of a single device per thread, so in order to get the read
performance benefit of multiple copies you either need to use
multiple reading threads / processes or else use RAID-10 instead.

https://www.spinics.net/lists/raid/msg47240.html

ZFS does not suffer from this.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Custom mariadb installation

2021-02-27 Thread Andy Smith
Hello,

On Fri, Feb 26, 2021 at 10:44:27AM -0800, Bill wrote:
> So I'd like to install mariadb on Debian 10 with the
> --basedir=/usr/local/mariadb and --datadir=/data/mariadb. I've tried to add
> these options to the "apt install mariadb-server" command line but I get
> error messages saying the options aren't recognized.

Those are options for the mariadb program itself. You can put them
in configuration files, overriding what is already set, e.g.:

$ sudo ack datadir /etc/mysql/
/etc/mysql/mariadb.conf.d/50-server.cnf
21:datadir = /var/lib/mysql

So I would suggest making your own custom config file in
/etc/mysql/mariadb.conf.d/ numbered after 50 and override it there:

[mysqld]
basedir = /usr/local/mariadb
datadir = /data/mariadb

Do bear in mind that the installation of mariadb will create some
databases in the default location first as part of its setup. You
may want to pre-create your config files (which the install won't
overwrite) so that it gets the correct settings at first startup and
during the initial configuration.

Also mariadb doesn't have an AppArmor profile at the moment but it
could grow one, so you'd have to watch out for that as it's very
likley to restrict the daemon to only being able to access
/var/lib/mysql.

Cheers,
Andy



Re: shadowy, sort of fly by night debian mirrors? ...

2021-02-24 Thread Andy Smith
Albrecht,

On Wed, Feb 24, 2021 at 11:27:31AM -0500, Albretch Mueller wrote:
> I take pride at being from very prejudiced to cautiously racist
> towards those not only "un-Amerikan", but,  even "communist"
> Chinese before they spread the Corona Virus…

Your racist conspiracy theories are not only abhorrent but also a
violation of Debian's Code of Conduct. Please do not post this kind
of thing to any part of Debian's infrastructure again (or
preferably, anywhere, ever, but it is specifically not tolerated
at Debian).

https://lists.debian.org/debian-user/2021/02/msg00010.html
https://www.debian.org/MailingLists/#codeofconduct
https://www.debian.org/code_of_conduct

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: shadowy, sort of fly by night debian mirrors? ...

2021-02-22 Thread Andy Smith
Hi Albrecht,

On Mon, Feb 22, 2021 at 03:50:01AM -0500, Albretch Mueller wrote:
> Andy Smith wrote:
> > Those SHA1 hashes do appear here on another mirror:
> >
> > http://mirrorservice.org/sites/cdimage.debian.org/debian-cd/10.8.0/amd64/iso-dvd/SHA1SUMS

[…]

>  I would expect for that string to appear on a few mirrors at least.

I just showed you exactly where the hashes for the ISO files are on
one mirror, I assume they are in the same place on every other
mirror.

You have not yet explained how come you show hashes with mismatched
file names - whether that was a simple error on your side while
composing the email or something you actually downloaded from the
Debian mirror.

> Also, hy ere their servers not producing any server side logs?

I am unable to parse the question as my understanding of what
"server side logs" means can't possibly line up with yours. Please
elaborate.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: shadowy, sort of fly by night debian mirrors? ...

2021-02-21 Thread Andy Smith
Hello,

On Sun, Feb 21, 2021 at 08:45:08AM -0500, Albretch Mueller wrote:
>  1) as I used a known public hotspot connection, there was a new
> hotspot advertising itself as "Wifi4EU" (of course, I didn't bite that
> bait)

Does not really seem relevant to a remote Debian mirror, unless you
are suggesting that someone has set up a rogue wifi hotspot in that
particular location and used it to distribute compromised Debian
images, which seems rather far-fetched.

>  2) getting a connection through (apparently) the right hotspot took
> way more time than expected

I'm not saying it's aliens
but it's aliens.

>  3) downloads were being redirected real time

OK? Web servers are allowed to issue redirects, and you're being
redirected to another hostname at the same org, so doesn't seem very
suspicious.

>  4) the usual server side responses were not being produced, just:
> 
> WARNING: certificate common name `ftp.acc.umu.se' doesn't match
> requested host name `chuangtzu.ftp.acc.umu.se'.
> 2021-02-17 11:14:47
> URL:https://chuangtzu.ftp.acc.umu.se/debian-cd/current/amd64/iso-dvd/debian-10.8.0-amd64-DVD-2.iso
> [4697370624/4697370624] -> "debian-10.8.0-amd64-DVD-2.iso" [1]

Right, so it's just saying you requested something at ftp.acc.umu.se
but it's HTTP redirecting you to chuangtzu.ftp.acc.umu.se which
doesn't have a TLS certificate with the name "ftp.acc.umu.se".

Many Debian mirrors don't support HTTPS enough to have a TLS cert in
the correct name and/or a debian.org name. I think you can use host
deb.debian.org in your sources.list to hit a Fastly CDN node that is
network-wise reasonably close to you and will work with TLS without
complaint, though you don't know what transports it uses between
itself and the origin servers in the background.

>  5) the mirror debian site (ftp.acc.umu.se) had smelly prefixes as
> subdomains (apparently Chinese transliterations) {chuangtzu, laotzu}

Why do Chinese names seem "smelly" to you?

>  6) whois registry for umu.se

Unclear why the domain registry info for a Swedish university is of
any bearing…

>  7) the md5 and sha1 hashes that I computed could not be found online
> 
> 0296cfbeaf3823055901d7ad2077a077
> 0b742d83d23207db9a24553100d4155eb8c701bf debian
> 10.8.0-amd64-DVD-2.iso
> 37baf26293b8132fe95b4bd19262ca6b
> 122a2612ed63ff89db56eec0765e87268bf72318 debian
> 10.8.0-amd64-DVD-3.iso

Those SHA1 hashes do appear here on another mirror:


http://mirrorservice.org/sites/cdimage.debian.org/debian-cd/10.8.0/amd64/iso-dvd/SHA1SUMS

though they seem to be associated with different files in the
sequence:

122a2612ed63ff89db56eec0765e87268bf72318 debian-10.8.0-amd64-DVD-2.iso
0b742d83d23207db9a24553100d4155eb8c701bf debian-10.8.0-amd64-DVD-3.iso

Was it a copy/paste error on your side that switched these around or
is that really what you downloaded?

> I later downloaded what seem to be the right files, anyway. They
> would make for some easy and nice forensic analysis (just
> extracting the content of those iso files, using find and diff)
> whenever I find the time to do so.

Knock yourself out but I don't see any indication that anything
nefarious has happened nor that you have downloaded tampered files,
so it just sounds like a huge waste of time.

If that's not the case and you did manage to download something that
claims to be a Debian ISO but isn't, please do tell us more.

I mean, worst case, they've somehow got the names of some genuine
files mixed up - because the SHA1 hashes match real Debian files but
with different names. That's assuming no mix up on your side. Unless
you are experiencing a SHA1 collision as well on top of everything
else.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: "Run fsck manually"..?

2021-02-02 Thread Andy Smith
On Wed, Feb 03, 2021 at 01:41:54AM +, Andy Smith wrote:
> On Tue, Feb 02, 2021 at 07:13:16PM -0500, hobie of RMN wrote:
> > He enters "fsck" or "fsck /dev/sda1", and in a short while gets fsck
> > identifying it's version, and nothing else.
> 
> There can be issues trying to run fsck on a mounted filesystem. What
> happens if you do:
> 
> # touch /forcefsck

Oh, sorry, I missed your mention of (initramfs) prompt. So your
filesystem is too damaged to allow boot to complete and you won't be
able to do that "touch /forcefsck" thing.

If fsck is just printing its version it may think it doesn't need to
be run. You can force it to do a check/repair with "-f", so:

(initramfs) fsck.ext4 -vf /dev/sda1

If it find things that it wants to fix it will ask yuo and you'll
have to press 'y' each time. If you're certain that you always want
to answer 'y' then you can ctrl-c that and try again with -y:

(initramfs) fsck.ext4 -yvf /dev/sda1

If you want to see what it would do without it actually doing it you
can use -n instead of -y.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: "Run fsck manually"..?

2021-02-02 Thread Andy Smith
Hi,

On Tue, Feb 02, 2021 at 07:13:16PM -0500, hobie of RMN wrote:
> My brother's Debian system suddenly says on attempt to boot, "/dev/sda1:
> UNEXPECTED INCONSISTENCY:Runfsck manually", and, "inodes that were part of
> a corrupted orphan linked list found."
> 
> He enters "fsck" or "fsck /dev/sda1", and in a short while gets fsck
> identifying it's version, and nothing else.

There can be issues trying to run fsck on a mounted filesystem. What
happens if you do:

# touch /forcefsck
# reboot

?

That will force the system to do a fsck on boot, before the
filesystem is mounted for use. If that doesn't help I think you will
indeed have to try this from a live or rescue environment. The
Debian install media can boot into a rescue mode for tasks like
this.

The only time I've had something like this was when I created an
ext4 filesystem in Debian buster and then used it as a root
filesystem for CentOS 7.

The ext code in buster used a new filesystem feature that isn't
present in the ext4 driver in CentOS 7, so when CentOS 7 tried to
mount its root filesystem it said there were "inconsistencies" every
time. Yet doing a fsck in CentOS or trying the /forcefsck strategy I
mentioned above just said the filesystem was fine, and the
filesystem seemed fine in everyday use.

This was because the fsck in CentOS also could not understand the
new filesystem feature.

In the end I had to fsck it from Debian buster and then remove the
feature with tune2fs. CentOS 7 was happy with it then.

I am not saying this is what has happened to you. I'm just giving an
example of one weird set of circumstances that can lead to something
like this.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: debian stable kernel not updating on one machine

2021-02-02 Thread Andy Smith
Hi,

On Tue, Feb 02, 2021 at 04:13:36PM -0700, D. R. Evans wrote:
> I see that synaptic lists 4.19.0-14-amd64 as being available in
> the repository; and, indeed, on another machine I updated earlier
> in the day the kernel was updated from -13 to -14.
> 
> How might I be able to diagnose why the files relating to the -14 kernel are
> not selected when I hit synaptic's "Mark All Upgrades" button?

I don't know about synaptic as I don't use it. What does:

$ dpkg -l | grep linux-image

say?

Perhaps you do not have the virtual package "linux-image-amd64" for
some reason. That package depends upon the latest actual kernel
package, so causes you to see upgrades.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: debian-user list info and guidelines: spam

2021-01-25 Thread Andy Smith
Hi John,

On Mon, Jan 25, 2021 at 08:16:38AM -0500, John Kaufmann wrote:
> nb.net no longer run their own MTA (maybe for just this reason?),
> farming it out to userservices.net. As a result they increasingly
> unresponsive to complaints. I have not figured out my next step.

As mentioned, I do self-host my own email but when friends and
family ask for a solution I like to point them at fastmail.com.

I've no association with fastmail.com, I just find them pleasant to
deal with when helping people.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: debian-user list info and guidelines: spam

2021-01-24 Thread Andy Smith
Hi John,

On Mon, Jan 25, 2021 at 01:03:07AM -0500, John Kaufmann wrote:
> Your post provides a hook to ask about a question that arises sporadically: 
> Probably less than one a month I receive from Debian Listmaster Team 
>  a message "lists.debian.org has received 
> bounces from you". Invariably it reads:
> >In the last seven days we've seen bounces for the following list:
> >* debian-user
> > 1 bounce out of  mails in one day (%, kick-score is 80%)
> 
> First: How common is this occurrence for others?

Probably around the same frequency as you until I set my mail server
to never reject spammy mails from Debian lists but instead silently
discard them.

The thing is, this is about spam. The best practice when dealing
with a piece of mail that has been identified as so spammy that you
don't want to receive it is not to file it away in a spam folder,
but to reject it at SMTP time.

If it were directed to a spam folder then the sender believe it has
been delivered but the user may not look at it ever. At least with
an SMTP-time reject, the sending system knows that the mail has not
been safely sent. If it is a human sender than they can take
appropriate action if a mistake has been made.

Unfortunately Debian's listserv doesn't like it when you do this and
will eventually unsubscribe you because it sees so many errors with
the (spam) emails it tries to send you. At least as you say the
threshold is quite high and it would be very unlikely to send 80%
rejectable spam in one day, but I still found it annoying enough to
abandon the SMTP reject best practice policy for mails from Debian
lists.

You don't have that option, because you don't run your MTA. But the
people who run your MTA are doing the right thing by rejecting email
that they consider too spammy. So although it may be worth asking if
there's anything they can do, it's likely they won't want to do
anything differently there.

>   I cannot currently post to the SOGo list because their spam
>   filter (UCEPROTECT) claims that my ISP currently hosts 6
>   spammers. In response, they block all mail from that ISP (called
>   UCEPROTECT Level2 protection). Again, getting my ISP to care
>   about this is an ongoing challenge.

Now this one is different. UCEPROTECT doesn't have a good reputation
amongst DNSBLs. They routinely list entire providers for a small
number of incidents and they require a payment to be removed.

I would call it extremely unwise to outright block email for
UCEPROTECT level 2 listing. I personally would not even score on
that. They will be rejecting a lot of legitimate email, not just
yours.

All you can do is try to persuade them to stop using UCEPROTECT
though.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Raid 1

2021-01-24 Thread Andy Smith
Hi Pankaj,

Not wishing to put words in Linux-Fan's mouth, but my own views
are…

On Mon, Jan 25, 2021 at 11:04:09AM +0530, Pankaj Jangid wrote:
> Linux-Fan  writes:
> 
> > * OS data bitrot is not covered, but OS single HDD failure is.
> >   I achieve this by having OS and Swap on MDADM RAID 1
> >   i.e. mirrored but without ZFS.
> 
> I am still learning.
> 
> 1. By "by having OS and Swap on MDADM", did you mean the /boot partition
>and swap.

When people say, "I put OS and Swap on MDADM" they typically mean
the entire installed system before user/service data is put on it.
So that's / and all its usual sub-directories, and swap, possibly
with things later split off after install.

> 2. Why did you put Swap on RAID? What is the advantage?

If you have swap used, and the device behind it goes away, your
system will likely crash.

The point of RAID is to increase availability. If you have the OS
itself in RAID and you have swap, the swap should be in RAID too.

There are use cases where the software itself provides the
availability. For example, there is Ceph, which typically uses
simple block devices from multiple hosts and distributes the data
around.

A valid setup for Ceph is to have the OS in a small RAID just so
that a device failure doesn't take down a machine entirely, but then
have the data devices stand alone as Ceph itself will handle a
failure of those. Small boot+OS devices are cheap and it's so simple
to RAID them.

Normally Ceph is set up so that an entire host can be lost. If host
reinstallation is automatic and quick and there's so many hosts that
losing any one of them is a fairly minor occurrence then it could be
valid to not even put the OS+swap in RAID. Though for me it still
sounds like a lot more hassle than just replacing a dead drive in a
running machine, so I wouldn't do it personally.

>- I understood that RAID is used to detect disk failures early.

Not really. Although with RAID or ZFS or the like it is typical to
have a periodic (weekly, monthly, etc) scrub that reads all data and
may uncover drive problems like unreadable sectors, usually failures
happen when they will happen. The difference is that a copy of the
data still exists somewhere else, so that can be used and the
failure does not have to propagate to the application.

> How do you decide which partition to cover and which not?

For each of the storage devices in your system, ask yourself:

- Would your system still run if that device suddenly went away?

- Would your application(s) still run if that device suddenly went
  away?

- Could finding a replacement device and restoring your data from
  backups be done in a time span that you consider reasonable?

If the answer to those questions are not what you could tolerate,
add some redundancy in order to reduce unavailability. If you decide
you can tolerate the possible unavailability then so be it.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Raid 1

2021-01-24 Thread Andy Smith
Hi Mick,

On Sun, Jan 24, 2021 at 11:36:09AM +, mick crane wrote:
> I know I'm a bit thick about these things, what I'm blocked about is where
> is the OS.

Wherever you installed it.

> Let's say I have one PC and 2 unpartitioned disks.
> Put one disk in PC and install Debian on it.

I think you are fundamentally going about this the wrong way.

There are several concerns and I think you are mixing them up. If I
understand you correctly, you concerns are:

1. Your data and OS should be backed up.
2. Your data and OS should be available even if a disk dies

Concern #1 is totally separate from concern #2 and is achieved by
setting up a backup system, has very little to do with whether you
use RAID or ZFS or whatever. It is worth a separate thread because
it's separate project.

For concern #2, that being *availability* of data and OS, there's
many ways to do it. You seem to have settled upon ZFS for your data,
and OS separately by some other means. That's fine.

A ZFS mirror vdev is going to need two identically-sized devices.
And you want to keep your OS separate. This suggests that each of
your disks should have two partitions. The first one would be for
the OS, and the second one would be for ZFS.

If you are going to keep your OS separate, I don't see any reason
not to use mdadm RAID-1 for the OS even if you're going to use zfs
for your data. Yes you could just install the OS onto a single
partition of a single disk, but you have two disks so why not use
RAID-1? If a disk breaks, your computer carries on working, what's
not to like?

So personally I would just do the install of Debian with both disks
inside the machine, manual partitioning, create a single partition
big enough for your OS on the first disk and then another one the
same on the second disk. Mark them as RAID members, set them to
RAID-1, install on that.

Once it's up and running you can then go and create a second
partition that spans the rest of each disk, and then when you are
ready to create your zfs pool:

> "zpool create tank mirror disk1 disk2"

# zpool create tank mirror /dev/disk/by-id/ata-DISK1MODEL-SERIAL-part2 
/dev/disk/by-id/ata-DISK2MODEL-SERIAL-part2

The DISK1MODEL-SERIAL bits will be different for you based on what
the model and serial numbers are of your disks. Point is it's a pair
of devices that are partition 2 of each disk.

> Can I then remove disk1 and PC will boot Debian from disk2 ?

This is only going to work if you have gone to the effort of
installing your OS on RAID. The easiest way to achieve that is to
have both disks in the machine when you install it and to properly
tell it that the first partition of each is a RAID member, create
them as a RAID-1 and tell the installer to install onto that.

As other mentioned, after it's installed you do have to manually
install the grub bootloader to the second device as well as by
default it only gets installed on the first one.

A word of warning: RAID is quite a big topic for the uninitiated and
so is ZFS. You are proposing to take on both at once. You have some
learning to do. You may make mistakes, and this data seems precious
to you. I advise you to sort out the backups first. You might need
them sooner than you'd hoped.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Backup debconf state

2021-01-19 Thread Andy Smith
Hi Erwan,

On Tue, Jan 19, 2021 at 04:09:47PM +0100, Erwan David wrote:
> It would be handy to have the state of debconf (with all the
> answers I already gave).

I do:

dpkg --get-selections \* > /var/lib/dpkg_selections
debconf-get-selections > /var/lib/debconf_selections

(and then back up those files)

You can restore them with the corresponding --set-selections.

> Is a backup of /var/cache debconf sufficient for this ?

I think that stuff lives in /var/lib, but it's better to export it
in a format where it can be re-imported.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: One network card many rj45 sockets

2021-01-19 Thread Andy Smith
Hi Mick,

On Tue, Jan 19, 2021 at 12:47:34PM +, mick crane wrote:
> I see that you can get a single network card with 2, 3, 4 connections.
> Can you happily make each one on a separate private address block ?
> 10.0.0.0, 172.16.0.0, 192.168.0.0

There is no strong connection between the concept of a physical port
and the logical addresses you put on an interface associated with.
You can create a virtual network interface in a machine with no
network hardware at all, and put a billion different IPv6 networks
on it if you like!

Cheers,
Andy



Re: list package version if installed (scriptable)

2021-01-07 Thread Andy Smith
Hi Jim,

On Thu, Jan 07, 2021 at 08:12:52AM -0500, Jim Popovitch wrote:
> What is a script'able way to list a pkg version (or nothing if it is not
> installed)?

$ dpkg-query --showformat '${Version}\t${Status}\n' --show coreutils 
8.23-4  install ok installed
$ dpkg-query --showformat '${Version}\t${Status}\n' --show coreutils | awk 
'/installed/ { print $1 }'
8.23-4
$ dpkg-query --showformat '${Version}\t${Status}\n' --show wowbagger
dpkg-query: no packages found matching wowbagger
$ echo $?
1

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm usage

2020-12-31 Thread Andy Smith
Hello,

On Thu, Dec 31, 2020 at 06:18:29PM +0300, Reco wrote:
> On Thu, Dec 31, 2020 at 03:06:34PM +0000, Andy Smith wrote:
> > Datasheet says:
> > 
> > * Enhanced Power-Loss Data Protection with Tantal capacitors
> 
> It does not have a battery = it does not have a BBU.
> Samsung can dance around that fact all they want, but it won't change.

That is a really strange comment to me. No SSDs have batteries.
Almost no RAID cards have batteries anymore. Supercapacitors have
obsoleted the battery for such purposes. All SSD power loss
protection is supercaps. And if you try to buy a modern RAID card
with a BBU you will mostly just find cards with supercaps instead.
But OP was asking about SSDs not RAID cards so even that nuance
doesn't make much sense.

But okay, your SSD doesn't have pink elephants either. Samsung can
dance around that fact all they want.

Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm usage

2020-12-31 Thread Andy Smith
Hello,

On Thu, Dec 31, 2020 at 05:42:35PM +0300, Reco wrote:
> It's a cheap model (relatively), and lack all those fancy BBU features,
> but it works for my employer:
> 
> # smartctl -a /dev/sdj
> ...
> Vendor:   SAMSUNG
> Product:  MZILT1T9HAJQ/007

Datasheet says:

* Enhanced Power-Loss Data Protection with Tantal capacitors

https://www.hyperscalers.com/image/catalog/!Petar/PM1643%202.5%20SAS%20SSD%20Datasheet_v1.0_for%20General%20(002).pdf

Also corroborated at:


https://www.anandtech.com/show/12448/samsung-begins-mass-production-of-pm1643-sas-ssds-with-3072-tb-capacity

"Some of the features of the PM1643 that Samsung is willing to
discuss right now (metadata protection, power loss protection,
data recovery, end-to-end data protection, encryption, etc.)
confirm that the drives are indeed aimed at servers that require
advanced reliability."

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm usage

2020-12-31 Thread Andy Smith
Hello,

On Thu, Dec 31, 2020 at 09:17:03AM -0500, Michael Stone wrote:
> On Thu, Dec 31, 2020 at 07:25:54AM -0500, rhkra...@gmail.com wrote:
> >What do you mean by power loss protection -- do you mean, for example, that
> >the host computer is on a UPS, or is that a feature of some SSDs?
> 
> It's a feature of server SSDs. I wouldn't worry about it on a consumer
> device, especially if you have a UPS.

Though OP did ask about NAS-quality SSDs for RAID use.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm usage

2020-12-30 Thread Andy Smith
Hello,

On Wed, Dec 30, 2020 at 11:14:37PM +0100, deloptes wrote:
> Can someone recommend server or NAS grade (SATA) SSD - a reliable one for
> RAID use?

I suggest checking smartctl values on your existing device to see
how much data you write in a week or so, then convert that to the
matching units (Drive Writes Per Day or TeraBytes Written) shown in
SSD specs so you can work out what you require for however many
years of operation you expect.

Then make sure it has power loss protection.

I would expect any major brand that passes those requirements to be
fine for your use case.

Personally for SATA interface I like Samsung's SM883 or PM883 (3
DWPD vs 1.3 DWPD assuming no over provisioning), but certainly there
are much cheaper options that are still good.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm usage

2020-12-30 Thread Andy Smith
Hello,

On Wed, Dec 30, 2020 at 02:38:46PM +0100, Jesper Dybdal wrote:
> I would hope that the most recently modified half of the array would be the
> one to overwrite the least recently modified one, so that a temporary
> absence of one disk which later comes back unmodified, will not destroy
> data.

Consider what happens if you take an mdadm RAID-1 member and put it
in another machine, mount it and then start writing to it. The
events count of the device will increase with each write.

If you then take that device and put it back in the original
machine, and it has a higher event count than the device in the
machine already, on next assemble mdadm will overwrite the device
that has the lower event count.

You can see the event count with:

# mdadm --examine /dev/sda1 # or whatever the member device is

So yes in one way your idea that the most recently modified half is
the one chosen could be said to be correct, if by "most recently
modified" you actually mean "most number of events".

As you were thinking, it is pretty safe to do if you never write to
the device you take out.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm usage

2020-12-30 Thread Andy Smith
On Wed, Dec 30, 2020 at 11:42:37AM +0100, Thomas A. Anderson wrote:
> When i enter mdadm --examine /dev/sdb
> 
> I get:
> 
> /dev/sdb:
> 
>     MBR Magic: aa55
> 
> Partition[0] : 3907026944 sectors at         2048 (type 83)

It would say more than that if sdb had ever been an md RAID member.

Are you sure it was sdb? Could it have been a partition on sdb?
"fdisk -l /dev/sdb" to list partitions. Also be really careful that
sdb really is the device you think it is!

> If hardware raid (like if I bought a controller), would it be any
> different, if I removed the drives and just put on one another machine
> -- would I be able to see the data on it like a normal drive? Or would I
> run into the same issue??

You would run into the same issue but it would be worse because the
other computer would have to have the same brand (possibly even the
exact same model) of hardware RAID. Every RAID system has to put
metadata onto the devices.

With mdadm, the structure on disk is public information. If you run
into difficulty you can get help from a wide pool of people. I have
seen data brought back from some truly disastrous situations in
threads on the linux-raid mailing list (where mostly md-related
things are discussed).

Try the same thing with hardware RAID and your only port of call is
the manufacturer's support desk because the layout of your data is
now proprietary information. For most of us the support desks of
such vendors don't work out well.

In many cases hardware RAID performs better, especially if you get
one with a supercap-backed write cache, but the trend these days is
to do Just a Bunch of Disks (JBOD) with software RAID, btrfs or
zfs.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: mdadm usage

2020-12-30 Thread Andy Smith
Hi Mick,

On Tue, Dec 29, 2020 at 03:32:07PM +, mick crane wrote:
> On 2020-12-29 13:10, Andy Smith wrote:
> >The default metadata format (v1.2) for mdadm is at the beginning of
> >the device. If you've put a filesystem directly on the md device
> >then the presence of the metadata will prevent it being recognised
> >as a simple filesystem. What you can do is force mdadm to import it
> >as a degraded RAID-1.
> <..>
> I've puzzled about this. Are you supposed to have 3 disks ?
> One for the OS and the other 2 for the raid1 ?

It's unclear to me how what you have quoted relates to your
subsequent question. Can you elaborate on what the location of mdadm
metadata has to do with whether one puts the operating system on a
RAID-1 device or not?

I expect that most people using RAID put their OS into the RAID as
well. I certainly do. I don't understand your mental separation of
"OS" and "RAID-1" or why it might mandate 3 devices. It is perfectly
straightforward to put a single partition¹ on each of two devices,
RAID-1 them together and use it as root filesystem and that's it.

Probably we are just misunderstanding each other and there is a
question here that I haven't understood.

Cheers,
Andy

¹ You can just use the devices as well, no partitions, but sometimes
  weird things can happen with BIOSes that think this is an invalid
  configuration, so I wouldn't recommend it for devices you will
  boot from.



Re: mdadm usage

2020-12-29 Thread Andy Smith
Hello,

On Tue, Dec 29, 2020 at 07:58:50AM +0100, Thomas A. Anderson wrote:
> I have been "using" mdadm to run software raid1 (stripping) on a file
> server i have been running.

As others have noted, RAID-1 is not striping but mirroring. I'll
assume you have used RAID-1. Showing us the content of your
/proc/mdstat file would help.

> now that I try to access data on these two original drives on
> another system, I am unable to.

The default metadata format (v1.2) for mdadm is at the beginning of
the device. If you've put a filesystem directly on the md device
then the presence of the metadata will prevent it being recognised
as a simple filesystem. What you can do is force mdadm to import it
as a degraded RAID-1.

Back in the days before GRUB understood md RAID-1 people used to
have to specifically use metadata format v1.0 or v0.9 for the device
containing /boot, in order to get metadata placed at end of device
so that GRUB was tricked into thinking it was a simple filesystem.

If you need help importing your single RAID-1 device, get it plugged
in to a system and recognised as a block device (see
/proc/partitions for a list of block devices), then show us the
output of:

# mdadm --examine /dev/whatever

where /dev/whatever is the block device for this single RAID-1
member device.

Then we'll help you get mdadm to assemble it.

Alternatively, assuming filesystem directly on md device it is also
possible to use the loop driver to create a new device that is an
offset into the md member device and then mount that as the
filesystem, but in my opinion that is more complicated and dangerous
than just getting mdadm to assemble a degraded RAID-1.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: transfer speed data

2020-12-22 Thread Andy Smith
Hi Mick,

On Wed, Dec 23, 2020 at 12:55:58AM +, mick crane wrote:
> I have a buster PC and a bullseye PC which are both supposed to have
> gigabyte network cards connected via a little Gigabyte switch box.

"gigabyte" is not a network speed. You probably mean gigabit; that
is 10⁹ bits per second, i.e. 1000 * 1000 * 1000 bits per
second.

GNU units can be useful to indicate what you can expect:

$ units gigabit
Definition: 1e9 bit = 1e+09 bit
$ units megabyte
Definition: 1e6 byte = 800 bit
$ units 1gigabit/sec megabyte/sec
* 125
/ 0.008

So on a gigabit network the absolute maximum you could expect is
125MByte/sec. Note that's SI prefix mega- meaning million bytes, not IEC
binary prefix MiB, meaning 1024 * 1024 bytes.

> Transferring files between them, I forget which shows the transfer speed per
> file, either scp or rsync the maximum is 50 Mbs per file.

I shall assume that "50Mbs" means "50 megabytes per second" and not
what it literally means which is "50 Megabits per second", a
quantity one eighth as much.

scp and rsync add a lot of overhead, especially when operating on
relatively small files. On a gigabit network I find myself lucky to
get more than about 90MB/sec through ssh or rsync-over-ssh.

So I find 50MB/s plausible.

> Would you expect that to be quicker ?

Not really.

To get a more realistic idea of your network's performance use
something like Iperf. You still won't see the full 125MB/s but I'd
expect it to go over 100.

If you are trying to transfer files as fast as possible and don't
need encryption, consider netcat. If you do need the encryption of
ssh, but don't need the features of rsync, then "tar | ssh" will be
a little faster.

On a low latency network (like your local network) at gigabit+
speeds, compression won't make things faster.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: adding a disk to a Volume group

2020-12-22 Thread Andy Smith
Hello,

On Tue, Dec 22, 2020 at 11:10:20PM +, mick crane wrote:
> On 2020-12-22 22:19, Michael wrote:
> >- pvcreate  to add a partition to a pool of physical volumes

[…]

> I have only skimmed reading the fine manual and other things but I'm
> guessing that if the partition is full I'll either need to make it bigger or
> make another one before invoking LVM?

The partition can't be full because LVM works with block devices,
completely owning them. So if you type:

# pvcreate /dev/foo

then /dev/foo better be already empty because whatever is on it
already is going to be trashed¹.

So the expectation is that the block device ("partition") is empty
and unused when you start.

Seeing as you were talking about installing a new drive and then
putting that in LVM, that doesn't sound like it would pose a problem
to you.

Since it is easy to grow LVM things but often trickier to shrink
them, I advise you to start small.

e.g. if you install a drive and it shows up in your OS as /dev/foo
of size 1TB, then:

# pvcreate /dev/foo
# vgcreate myvg /dev/foo

Now you have a volume group called "myvg" with ~1TB (some space
reserved for metadata) available for allocation.

Your temptation at this point may be to create a logical volume that
uses the whole ~1TB, and you can, but unless you actually
immediately need that size of block device it would be best to hold
off.

Say you need 50GiB of scratch space right away. You'd be better off:

# lvcreate -L50g /dev/myvg/scratch
# mkdir /srv/scratch
# mkfs.ext4 /dev/myvg/scratch
# mount /dev/myvg/scratch /srv/scratch

(names and filesystem types to taste)

…because that leaves you plenty of capacity in the myvg volume group
to later create other logical volumes. If you do end up needing to
grow the scratch space you can do it online:

# lvresize -r -L+50g -n /dev/myvg/scratch

If instead you had immediately made your first LV use the entire
capacity and then later found you wanted other LVs, there'd be no
available capacity and you'd have to shrink the scratch one. Some
filesystems can't be shrunk online (need umount). Some filesystems
can't be shrunk at all!

You also should think about what happens with drive failure,
especially if you are thinking of putting multiple drives into a
volume group with no redundancy.

Cheers,
Andy

¹ Yes I am aware that there are various tricks to move an existing
  filesystem directly on a block device into an LVM LV on the same
  block device, but discussion of this seems way above OP's level at
  this time.

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Debian 10 64bit

2020-12-20 Thread Andy Smith
Hello,

On Sun, Dec 20, 2020 at 11:32:33AM +, mick crane wrote:
> I noticed that if you add yourself to sudo group in /etc/group you
> have to logout and log back in for it to be noticed.

If you don't want to log out of the shell you can do this:

$ exec sg sudo "newgrp $(id -ng)"

The "exec" causes what follows to replace the current shell. Without
this you'd end up with two extra shells running and would have to
"exit" three times to close them.

"sg sudo" causes the following command to be executed with primary
group of "sudo".

"newgrp $(id -ng)" puts you back in your original primary group,
leaving you with group "sudo" as an additional group.

You can just do "newgroup sudo" but this:

- starts an extra shell so you'd have to "exit" it twice

- leaves you with "sudo" as your primary group

You can just do "su - $USER" but this will ask you for your
password.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Migrating LVM volumes to a new machine

2020-12-12 Thread Andy Smith
Hello,

On Sat, Dec 12, 2020 at 11:51:17AM +, Mark Fletcher wrote:
> That generates a followup question, out of curiosity. Presumably for 
> that to work, all the info needed for the computer to learn about the VG 
> at boot must be stored on the PV. What happens when there is more than 
> one PV for a VG? Is the info stored on all of them, or just one?

All of them.

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/logical_volume_manager_administration/lvm_metadata

"The configuration details of a volume group are referred to as
the metadata. By default, an identical copy of the metadata is
maintained in every metadata area in every physical volume
within the volume group."

Have a read of it with:

# vgcfgbackup -f config.txt vgname

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: i386 debian to 64bit intel

2020-12-04 Thread Andy Smith
Hello,

On Fri, Dec 04, 2020 at 04:05:18PM +, Darac Marjal wrote:
> I am somewhat boggling, though, at the idea in the Instructions[1]
> of crossgrading from arm64 to amd64. What manner of machine can
> interpret both of those instruction sets?!

A virtual machine, hence qemu! :)

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: swamp rat bots Q

2020-12-03 Thread Andy Smith
On Fri, Dec 04, 2020 at 01:40:53AM -0500, Gene Heskett wrote:
> On Friday 04 December 2020 01:03:34 Andy Smith wrote:
> > Again we have been down this avenue before, but I will try one last
> > time.

[…]

> > So, can you show us a few lines of logs from your
> > /var/log/httpd/other_vhosts_access.log of the accesses from the
> > offending bot(s)?
> >
> No.  Bad idea.
> 
> By publishing the name of the bot, it probably will be changed before the 
> day is done.

It is beyond ridiculous to believe that a bot that is scanning the
entire Internet every day — and probably already published on
literally thousands of web sites that put up log analyses — is going
to find your single posting on debian-user and change its ways.

As predicted, you have zero interest in finding a practical solution
to your issue and so as promised, I am out.

I look forward to seeing your next try at this exact same thread.

Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: swamp rat bots Q

2020-12-03 Thread Andy Smith
Hello,

On Fri, Dec 04, 2020 at 12:03:57AM -0500, Gene Heskett wrote:
> what file do I edit to add todays
> /var/log/httpd/other_vhosts_access.log to its list of logs to
> watch.  That's the log file with the real data in it today.  And
> does it need enabled in another, different file.

Again we have been down this avenue before, but I will try one last
time.

It seems quite likely that the bot you have a problem with has the
same user agent string, or a very small variation on the same
string. If so then you can block it just with Apache.

So, can you show us a few lines of logs from your
/var/log/httpd/other_vhosts_access.log of the accesses from the
offending bot(s)?

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Cron Jobs and Time Zones Has Anything Changed?

2020-12-03 Thread Andy Smith
Hi David,

On Fri, Dec 04, 2020 at 01:32:35PM +1100, David wrote:
> On Fri, 4 Dec 2020 at 13:10, Andy Smith  wrote:
> > So much text written without clear statement of problem!
> 
> I understand why you wrote that, but you might be unaware that
> Martin has previously mentioned on this list that he is blind.

Does that stop him writing exactly what he wants to achieve, what
he's tried and where that failed? He can clearly write a lot, just
not the things that move the situation forward, it seems.

Let's get a problem, and help Martin fix it - what this list is for.

Otherwise, personal blogs are a thing.

Cheers,
Andy



Re: swamp rat bots Q

2020-12-03 Thread Andy Smith
Hello,

On Thu, Dec 03, 2020 at 07:35:27AM -0500, Gene Heskett wrote:
> I've had it with a certain bot that that ignore my robots.txt and 
> proceeds to mirror my site, several times a day, burning up my upload 
> bandwidth. They've moved it to 5 different addresses since midnight.

This must be the third or fourth time we have been here with this
exact question from you. Every time the answers have been "Fail2Ban
and block by user agent". I don't know why you expect the answers to
change.

Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Cron Jobs and Time Zones Has Anything Changed?

2020-12-03 Thread Andy Smith
Hello,

On Thu, Dec 03, 2020 at 07:39:14AM -0600, Martin McCormick wrote:
> I am guilty as charged but haven't yet found the relevant information
> as to how systemd helps solve this issue.

You can put a time zone in a systemd timer. I can't see how it can
be stated any simpler than that.

If you want to ask the question, "How do I make a systemd timer that
runs a command at a given time in a given time zone?" just ask that
question! Though I am surprised that just looking up documentation
on systemd timers doesn't answer that question for you.

If that is not your question, well, what is your question? So much
text written without clear statement of problem! Sort of ironic that
this started with asking why there are 3.5MiB of files in
/usr/share/zoneinfo/ - has more than 3.5N of data been created yet
between these couple of threads?

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Cron Jobs and Time Zones Has Anything Changed?

2020-12-02 Thread Andy Smith
Hello,

On Wed, Dec 02, 2020 at 01:58:45PM +, James B wrote:
> This might be wrong, but as far as I understand doesn't systemd
> now have the ability to manage cron jobs (as well as mount points,
> home folders and other things)?. Is there anything in this newer
> functionality that might make such a thing (re the request at the
> beginning of this thread) possible?

Yes, I pointed this out to OP last time OP asked this exact question
just a few days ago, so I don't know why they are asking again.
Nothing has changed in the last couple of days to give crond TZ
support!

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Problem with /var/mail file > 2GB with pop3

2020-11-22 Thread Andy Smith
Hi Flo,

On Sun, Nov 22, 2020 at 06:57:46PM +0100, Flo wrote:
> Let's assume an average message size of 20MB. Then 100 messages are
> enough to make it INBOX file that big. This doesn't necessarily mean
> that this is disorganized.

The average size of mail I have received in the last 2 months is
16.2KiB. It seems you receive mails one thousand times as large as I
do.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: 780 files in /usr/share/zoneinfo/

2020-11-21 Thread Andy Smith
Hi Martin,

On Sat, Nov 21, 2020 at 08:48:51PM -0600, Martin McCormick wrote:
> find . -name "*" -exec ls -l {} \;  \
> |grep -F / \
> | awk ' { total += $5 } END { print total }'
> 
>   That usually just adds the sizes of all the files it can
> find all the way through the tree.
> 
>   If this is not an accurate way to determine how many
> bytes there are in a directory then that would be the reason for
> the discrepancy.

The same file can be reached by multiple names. So by doing this you
end up, in this case, with a ~256x amplification.

A simple "du -sh" does a better job here!

> cron only works in the time zone for wherever the TZ for the
> system is set.

Ah, I see. I've never tried it but I believe that systemd timers can
have a time spec that includes time zone, so you can set timers that
fire on a different time zone to that used by the rest of the system.

$ systemd-analyze calendar '11:00 Europe/London'
  Original form: 11:00 Europe/London
Normalized form: *-*-* 11:00:00 Europe/London
Next elapse: Sun 2020-11-22 11:00:00 UTC
   From now: 6h left

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: adding a second HDD in debian

2020-11-21 Thread Andy Smith
Hello,

On Sat, Nov 21, 2020 at 06:31:53PM -0400, Antonio Barragan wrote:
> I have a PC with Debian 10 installed (on dev/sda), and working properly.
> Now I would like to add to it a second, 150 GB HDD (SATA), taken from
> another machine.

[…]

> How could that be done?

If you don't have hot swap drive bays:

- Power off your computer
- Fit the new drive in the chassis and plug it into a free SATA port
- Boot your computer again

If you do have hot swap drive bays:

- Insert the new drive into a spare drive slot

Then:

- cat /proc/partitions to see what new partitions have appeared
- check /var/log/syslog and dmesg to see what new drives have been
  seen

Most likely the partitions from the new drive just appear and can be
mounted somewhere, put in /etc/fstab and so on.

A possible failure mode is that your computer reorders its drives
and the new one becomes sda. This may interfere with your boot
process. Hopefully you referred to everything by UUID or label so
that this doesn't happen. If it does cause problems just take the
new drive out again and things should return to normal. Then you may
have to consider swapping cables between ports, or adjusting BIOS
boot order, or similar.

Most of the time though, just plugging the new drive in goes fine.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Problems PXE booting a UEFI debian-installer

2020-11-21 Thread Andy Smith
This was very occasionally working but mostly not. I updated the
boot firmware of the Intel I350 NICs and now it seems to work every
time.

Previously it was stopping after downloading
debian-installer/amd64/grubx64.efi by TFTP, and just showing me a
grub> prompt. Now it goes on to request all the other bits of the
installer, and works.

I have no idea what was wrong before but there are a few other
reports of EFI PXE boot problems being fixed by NIC firmware
upgrade.

Cheers,
Andy



Re: Size of initrd

2020-11-21 Thread Andy Smith
Hi Stefan,

On Sat, Nov 21, 2020 at 02:52:49PM -0500, Stefan Monnier wrote:
> One of the has /boot/initrd.img files that take about 15MB while the
> other has /boot/initrd.img files that take about 30MB (in both cases,
> they are compressed with `lzma`).
> 
> Any idea what this difference could come from (or how I could try and
> track it down) and how I could fix the size to be more like 15MB?

A common question during install is whether to build an initramfs
that has all drivers as modules, or one that only has the "targeted"
drivers as modules. That could be the difference you are seeing.

$ grep MODULES /etc/initramfs-tools/initramfs.conf

will tell you what it is currently set to, and if you change it and
then do

# update-initramfs -u -k 

or

# update-initramfs -u -k all

then you can compare the differences.

If that doesn't show it, you may need to take the initramfs apart to
compare contents.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: 780 files in /usr/share/zoneinfo/

2020-11-21 Thread Andy Smith
Hi Martin,

On Sat, Nov 21, 2020 at 01:20:39PM -0600, Martin McCormick wrote:
>   I just cd'd to that directory and it looks like there's
> about 1 GB there.

Are you sure about this? There is no Debian or Ubuntu host I have
access to that has a /usr/share/zoneinfo/ that contains more than
4MiB of data. For yours to have 256 times this much is quite an
aberration. What did you type to determine that your
/usr/share/zoneinfo/ has 1GiB of data in it?

>   I've wished one could just set certain parts of the
> computer to other times but I can also understand why this could
> be a problem.

You can set any process to have a different time zone by use of
environment variables.

$ date
Sat 21 Nov 21:34:55 UTC 2020
$ TZ=America/Los_Angeles date
Sat 21 Nov 13:35:04 PST 2020

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Redundancy for EFI System Partition: what do people do in 2020?

2020-11-21 Thread Andy Smith
Hello,

More of my adventures in EFI land.

Machines that boot by EFI need an EFI System Partition. I'm used
to using software RAID everywhere and providing redundancy for
everything. It seems that the designers of EFI didn't think about
that one.

https://www.tinkerfairy.net/efi-raid.txt

https://www.claudiokuenzler.com/blog/696/uefi-efi-boot-does-not-like-software-raid-system-partition-grub-error-17

https://unix.stackexchange.com/questions/265368/why-is-uefi-firmware-unable-to-access-a-software-raid-1-boot-efi-partition

So, those of you who boot by EFI and use software RAID, how do you
choose to provide redundancy for your ESP any why did you make that
choice?

I understand the main choices are:

a) Don't provide redundancy.

   There's only one ESP. If the device it's on dies you can recreate
   it with a live environment such as the rescue mode of the
   installer.

b) Put the ESP in a v1.0 mdraid level 1.

   As the RAID metadata is at the end, it appears to the firmware
   like a normal filesystem for read purposes. Updating it from
   within the OS writes to both copies as it's a RAID-1.

   Has the risk that if the firmware writes to it (which apparently
   it sometimes does), it will corrupt the RAID.

c) Manually sync the ESP to another partition which can be used if
   the first device dies.

   An identical partition can be created on the second device and an
   arrangement made to copy the real ESP to the secondary partition
   every time grub-install would be run.

   You would have to be sure that this is as automated and foolproof
   as possible, to avoid being lulled into a false sense of security
   and then have a problem at the worst time.

d) Something else?

Cheers,
Andy



Re: 780 files in /usr/share/zoneinfo/

2020-11-20 Thread Andy Smith
Hi Mike,

On Sat, Nov 21, 2020 at 02:30:08AM +, mike.junk...@att.net wrote:
> Of the 780 files in /usr/share/zoneinfo/ America/Chicago and
> CST6CDT are the only two that might apply to me. Are the rest of
> any use to me at all? If so how? And, yes, I understand that they
> need to be supplied for every zone in the world initially but am
> curious if there is any use for them after one's own time zone is
> set.

It's the Olson database, needed for any application that will want
to know the time in another time zone, or parse such.

https://en.wikipedia.org/wiki/Tz_database

The files are tiny. It's not worth removing them IMHO. It's 3½MiB of
space on my system.

Cheers,
Andy



Re: Problems PXE booting a UEFI debian-installer

2020-11-20 Thread Andy Smith
Hi Andrei,

On Fri, Nov 20, 2020 at 12:30:49PM +0200, Andrei POPESCU wrote:
> On Vi, 20 nov 20, 00:56:19, Andy Smith wrote:
> > 
> > I have tried both the buster netboot.tar.gz and the daily d-i build
> > and get the same behaviour with both.
> > 
> > I've also read the relevant part of the release notes:
> > 
> > https://www.debian.org/releases/buster/amd64/ch04s05.en.html
> > 
> > and they don't tell me anything different than what I've already
> > done, either.
> > 
> > Any ideas?
> 
> Maybe the Installation Guide has more information:
> 
> https://www.debian.org/releases/stable/installmanual

That links to the same place:
https://www.debian.org/releases/buster/amd64/ch04s05.en.html

Cheers,
Andy



Problems PXE booting a UEFI debian-installer

2020-11-19 Thread Andy Smith
Hi,

I'm new to machines that absolutely require EFI booting. I have a
machine that only has NVMe devices, and I understand it needs to be
installed in EFI mode to boot.

I'm trying to install by PXE. I've done this hundreds of times with
a legacy BIOS, but it's not working for me in EFI mode and I feel
like the problem is with my tftp setup or the netboot files within.

Can anyone see what I'm doing wrong here?

I've prepped my netboot server as covered here:

https://wiki.debian.org/PXEBootInstall

Specifically where it mentions EFI I've made the changes it
suggests:

If you are booting with UEFI, you should link grub and
grubx64.efi into the root of your tftp directory:

# cd /srv/tftp
# ln -s debian-installer/amd64/grubx64.efi .
# ln -s debian-installer/amd64/grub .

You may also have to edit grub/grub.cfg in order to set your
serial console, if needed (I replaced the section about the
graphical terminal)

For that bit I just shoved:

serial --speed=115200 --unit=1 --word=8 --parity=no --stop=1
terminal_input console serial
terminal_output console serial

into the grub/grub.cfg file just after the section at the top about
setting gfxmode.

Also each of the "linux" lines in the menuentry sections I changed
them from:

linux/debian-installer/amd64/linux vga=788 --- quiet

to:

linux/debian-installer/amd64/linux --- console=ttyS1,115200n8r 
console=tty1

because I am doing this on a serial console.

The full content of this grub/grub.cfg file is available here:

https://paste.debian.net/1173533/

When I trigger a PXE boot by telling the machine to give me a boot
menu, then selecting its eth0, I see it get an address from DHCP and
my tftpd logs show it request the correct files:

Nov 20 00:29:48 admin in.tftpd[19654]: RRQ from 192.168.80.30 filename 
debian-installer/amd64/bootnetx64.efi
Nov 20 00:29:49 admin in.tftpd[19654]: tftp: client does not accept options
Nov 20 00:29:49 admin in.tftpd[19655]: RRQ from 192.168.80.30 filename 
debian-installer/amd64/bootnetx64.efi
Nov 20 00:29:50 admin in.tftpd[19656]: RRQ from 192.168.80.30 filename 
debian-installer/amd64/grubx64.efi

On the console of the machine I see it say that it's downloaded an
"NBP" then the screen shows:

Welcome to GRUB!

then after a pause it clears and I just have a grub> prompt where I
would normally have expected the installer menu. Therefore I believe
it has correctly downloaded the files it needs, I'm looking at the
running grub from the tftp server, yet something has gone wrong with
the grub configuration.

I have tried both the buster netboot.tar.gz and the daily d-i build
and get the same behaviour with both.

I've also read the relevant part of the release notes:

https://www.debian.org/releases/buster/amd64/ch04s05.en.html

and they don't tell me anything different than what I've already
done, either.

Any ideas?

Cheers,
Andy



Re: Problem with /var/mail file > 2GB with pop3

2020-11-19 Thread Andy Smith
Hello,

On Thu, Nov 19, 2020 at 10:42:53PM +0100, Flo wrote:
> The mail files for each account are stored at /var/mail. No it has come to
> that point that such a file exceeded 2GB. And 'Get Messages' doesn't work
> anymore.
> 
> Does anyone know about this issue? Any hints to solve it? I could try a
> different pop3 server?

I have no experience with popa3d but if it supports Maildir instead
of mbox format then you could start storing users' mails in Maildir
format. Maildir's one file per message approach usually provides
better performance, though you do end up with a lot of metadata
activity on the filesystem.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Most maintainable way to install perl modules on Debian sysetms

2020-11-15 Thread Andy Smith
Hello,

On Sun, Nov 15, 2020 at 08:14:05PM -0500, Michael Grant wrote:
> > Well, that would do the job thoughtlessly. It might backfire
> > spectacularly.

[…]

> apt has an excellent reputation, I'm not sure I see why mechanizing
> such a process as apt does should be necessarily be bad.  I'm not
> talking about blind nightly updates.

The issue is that somewhere behind the scenes is a human being, the
Debian package maintainer, who is backporting security fixes (but
not new features!) from newer releases of the module into a Debian
package for a given Debian release.

That's not a mechanical process.

So when proposing to adopt a mechanical process, it's not comparable
to what an actually maintained Debian package is like; it's just a
CPAN module in a convenient format.

Most of the time the upstream author is mindful of backward
compatibility issues and isn't going to release something onto CPAN
that breaks things, but sometimes it's unavoidable and sometimes the
author isn't mindful of this. There is a big variance in quality of
CPAN modules.

So probably the best thing you can do is cpan2deb and stick with
those versions that you have tested, unless an update comes along
with security fixes or features you need.

You can install the package "cpanoutdated" which will tell you about
newer versions on CPAN compared to on your system, though it will
report quite a lot of packaged stuff as being outdated, which is
only to be expected.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: rsync --delete

2020-10-17 Thread Andy Smith
Hi Mike,

On Fri, Oct 16, 2020 at 05:09:42PM -0500, Mike McClain wrote:
> A section of the backup script is so:
> Params=(-a --inplace --delete);
> Flash=/sda/rpi4b
> cd /home/mike
> [ ! -d $Flash/mike ] && mkdir $Flash/mike;
> 
> #   exclude compressed files and the contents of most of the .* directories
> /mc/bin/mk_rsync_exclude.sh
> echo /usr/bin/rsync $Params --exclude-from=/home/mike/.rsync_exclude . 
> $Flash/mike

shellcheck has this to say:

$ shellcheck ./foo.sh 

In ./foo.sh line 6:
echo /usr/bin/rsync $Params --exclude-from=/home/mike/.rsync_exclude . 
$Flash/mike
^-- SC2128: Expanding an array without an index only gives 
the first element.

It's worth using shellcheck.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Stretch => Buster: Entropy during boot

2020-10-16 Thread Andy Smith
Hi Jesper,

On Fri, Oct 16, 2020 at 12:28:13PM +0200, Jesper Dybdal wrote:
> I run a few Stretch systems on old processors that do not support the RDRAND
> instruction.
> 
> Can I simply install "haveged" on the Stretch systems *before* the upgrade
> to Buster to avoid problems during the upgrade?

In July last year I experimented with boot times on a virtual
machine while:

- running normally

- disallowing RDRAND for early entropy

- disallowing RDRAND entirely

The normal boot (RDRAND) took ~1 second; the "no RDRAND at all" boot
took ~49 seconds. Given that a virtual machine has no real hardware
to provide sources of entropy I would consider this to be near to a
worst case for SSH. If you have other boot-time services that
require entropy then they may take significantly longer.

So if it's mainly SSH you're worried about, I don't think this will
be the end of the world for you to just do it and see what happens.


https://strugglers.net/~andy/blog/2019/07/11/experiments-with-rdrand-and-entropykey/

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Ownerships in /var/log/.

2020-10-11 Thread Andy Smith
Hello,

On Sun, Oct 11, 2020 at 02:41:37PM -0700, pe...@easthope.ca wrote:
> All of the files and most of the directories in /var/log/ are owned 
> by root.  These are the exceptions.
> 
> root@joule:/var/log#   ls -ld {c*,ex*,s*}/
> drwxr-xr-x 2 _chrony _chrony  4096 Jul 22  2017 chrony/
> drwxr-s--- 2 Debian-exim adm  4096 Oct 11 00:00 exim4/
> drwxr-xr-x 2 stunnel4stunnel4 4096 Jul 16  2019 stunnel4/
> 
> Comments about the unusual ownerships?

These daemons don't run as root and so to allow them to still log
it's easier to just let their log files be owned by their non-root
user.

There's a few other ways this could have been achieved, but that's
probably the simplest.

> Most of the group ownerships are root or adm.  
>
> What determines adm or root?

adm isn't widely used, and is kind of historical, but is sometimes
used for log files that may contain private information. The
files/directories are left readable by group adm so that users and
tools in that group can read them. Other less sensitive log files
are often left group root but world readable.

See https://wiki.debian.org/SystemGroups for more info.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: SSD and HDD

2020-10-11 Thread Andy Smith
Hello,

On Sun, Oct 11, 2020 at 01:48:48PM -0400, Jim Popovitch wrote:
> On Sun, 2020-10-11 at 19:47 +0200, Sven Joachim wrote:
> > "Percentage Used Endurance Indicator"
> 
> Where do you see that?

Usually a SMART attribute like "233 Media Wearout Indicator" or if
that isn't available devices often have "241 Total_LBAs_Written"
which can be compared¹ against published write endurance
specifications.

Sometimes the devices also have a proprietary tool for getting this
information, though in the majority of cases all this is doing is
parsing SMART attributes.

Cheers,
Andy

¹ Though with some care: devices vary on what they consider a
  logical block size to be; also due to internal merging and cell
  erasure, a logical write doesn't necessarily match to a flash
  write.

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: SSD and HDD

2020-10-11 Thread Andy Smith
Hi Mick,

On Sun, Oct 11, 2020 at 05:45:45PM +0100, mick crane wrote:
> Got a PC that has SSD and a HDD. I see that you are supposed to avoid writes
> to SSD for longevity.

Flash write endurance has come on leaps and bounds over the last
decade to the point where most people don't have to worry about
this.

You can look at "tune2fs -l" output or at SMART attributes to see
how much has been written to your current filesystems / devices over
their life times, to see how your use case matches up against the
write endurance advertised for your SSD.

I wouldn't recommend taking any special measures unless you have
some doubt that the SSD endurance is up to it.

With only a single SSD and a single HDD I'd rate device failure from
other problems as a higher risk than wearing out the SSD.

> Is it a matter of putting entries in fstab for /swap /var /home to suitably
> formatted partitions on HDD ?

If you still think you will have a problem then yes, that is one way
to go. Another is to leave some percentage of the SSD unpartitioned
and never used. That will increase its write endurance.

[ Leaving aside the fact that if I were doing this I'd have an extra
storage device for redundancy… ]

If I were in your position and still had concerns about write
endurance I'd probably put everything in LVM with a volume group on
the SSD and a volume group on the HDD. I'd then use separate logical
volumes for the filesystems that got a lot of writes. The use of LVM
like this would allow me to change my mind later and move LVs
between the SSD and HDD while the machine is online.

Plus any time you are thinking of doing multiple filesystems, LVM is
a good bet.

Plus you might be using LVM anyway for encryption.

But again I can't emphasise enough how you are probably over thinking
write endurance.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Question: SSD speed

2020-10-07 Thread Andy Smith
Hello,

On Wed, Oct 07, 2020 at 06:40:01PM +0200, Hans wrote:
> And second: If the real transferrate is only 1,5Gbyte/sec, does this mean, 
> that the sata controller is not capable to higher transferrates or does it 
> possibly mean, that my configuration is wrong?

This is all meant to be automatic. If you have some reason to
believe that your SATA/SAS controller is 6Gbit/s and your storage
device is 6Gbit/s but it comes up as less than that, then that's a
problem/bug/fault and not something that you generally just set.

And even with a 6Gbit/s negotiated link, you aren't going to achieve
6Gbit/sec of data transferred.

Start by working out what hardware you have to see what it's
actually meant to be capable of. Then you'll see if you have a
problem or if the behaviour is expected.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Debian 10 auto upgraded to Kali rolling

2020-10-05 Thread Andy Smith
Hello,

On Tue, Oct 06, 2020 at 05:14:38AM +0530, Saurabh Kannaujia wrote:
> I run 'apt upgrade' and it
> changed to Kali linux rolling. I don't know why this happend

This could only have happened by putting Kali's repositories into
/etc/apt/sources.list or in a file under /etc/apt/sources.list.d/
and then doing an "apt update" or equivalent, before you did "apt
upgrade".

> I want to comeback to Debian. Is there any way to comeback or do I
> have to reinstall again(ugh!).

No, you have changed to a completely different Linux distribution
and there is no supported way to go back without reinstall.

You could possibly, MAYBE, just put in the repositories for Debian
unstable and HOPE that the package versions of everything are high
enough that a dist-upgrade gets you back to Debian, but:

1. It probably wont get you back to Debian but some Frankendebian
   that still has traces of Kali in it. It's not a supported
   operation.

2. The chances of doing this without breaking the whole system are
   slim, and then you end up reinstalling anyway after spending many
   hours.

3. You'd still end up on debian unstable (sid), not Debian stable
   (buster).

Also, I am not sure that Kali Linux supports being installed as an
upgrade from Debian stable. If not then what you have now is very
likely already some sort of Frankendebian because it'll still have
some Debian packages on it. As Kali is a derivative of Debian this
might not be disastrous though. It's still not ideal even if you did
want Kali.

Basically this was a very bad thing to do. Sorry. Reinstall from
backups is my advice.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Mounting /dev/shm noexec

2020-10-02 Thread Andy Smith
Hello,

On Fri, Oct 02, 2020 at 10:35:51PM +0300, Valter Jaakkola wrote:
> So where can I change the mounting parameters of /dev/shm, or otherwise 
> arrange
> it so that /dev/shm is noexec already at/after boot?
> 
> (Out of curiosity, where is /dev/shm mounted from?)

I think from systemd:


https://github.com/systemd/systemd/blob/c7828862b39883cf1f55235a937d29588d5a806b/src/core/mount-setup.c#L79

and I think if you wish to alter the mount options you should put it
in /etc/fstab and then systemd will do the equivalent of:

# mount -oremount /dev/shm

to get your options set, though there would be a small window where
it had the default options.

Though note that it seems systemd once did use "noexec" for /dev/shm
but stopped 10 years ago because it broke some uses of mmap:


https://github.com/systemd/systemd/commit/501c875bffaef3263ad42c32485c7fde41027175

On SysV init systems I think this is part of the initscripts
package.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: [SOLVED] Re: Riddling activity on encrypted and mounted partition

2020-09-29 Thread Andy Smith
On Tue, Sep 29, 2020 at 01:02:35PM +0200, Thomas Schmitt wrote:
> Andy Smith wrote:
> > Create with:
> >mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0
> 
> This lasts significantly longer than my first mkfs run.
> The drive makes ~ 1950 write operations per second. So i estimate that
> the job would have lasted hours with ~ 16 writes per second.
> In the end mkfs.ext4 caused 733702 write ops on the 3.6 TB partition.
> 
> Ok. New UUID into fstab ... mount ... mkdir ... touch ... Yay !
> 
> The i/o is still lazy (no wonder with 32 GB RAM), but after about a minute
> i see no newly counted writes.
> 
> Thanks a lot !

No problem, but your reply doesn't make it clear to me whether the
lazy init was the cause of your writes or not. Maybe I just lack the
reading comprehension.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Riddling activity on encrypted and mounted partition

2020-09-29 Thread Andy Smith
Hello,

On Tue, Sep 29, 2020 at 10:24:44AM +0200, Thomas Schmitt wrote:
> i have encrypted my HDD's (*) data partition. Now the disk access LED is
> blinking rapidly as soon as i mount it.

Could it possibly be the lazy init feature of ext4, which is enabled
by default and can sometimes result in several minutes of background
writes to a newly-created fs?


https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bfff68738f1cb5c93dab1114634cea02aae9e7ba
https://www.thomas-krenn.com/en/wiki/Ext4_Filesystem#Lazy_Initialization

Create with:

mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 …

to avoid this sort of thing.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Deterministic delays in POSIX shell scripts (Was: Re: notify via virtual terminal available packages)

2020-09-25 Thread Andy Smith
Hello,

On Fri, Sep 25, 2020 at 07:49:19AM -0400, Greg Wooledge wrote:
> On Fri, Sep 25, 2020 at 07:44:25AM +0000, Andy Smith wrote:
> > "hostid" tends to return a hexadecimal representation of the first
> > IPv4 address (but isn't guaranteed to).
> 
> unicorn:~$ hostid
> 007f0101
> 
> Doesn't look very useful.  That's just 127.0.1.1 in a 16-bit little
> endian format.

Oh, none of mine do that, it seems to pick the other IP address for
me. But if it's a problem there are other sources of "machine" ID as
I mentioned. There's some more here:

http://0pointer.de/blog/projects/ids.html

> You know what else works really well?  Just putting a different start
> time in each system's crontab.

If that works for you, great, but I have quite a few machines, VMs
and containers provisioned identically and would rather not have to
change the scripts or configuration on a per-host basis.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Deterministic delays in POSIX shell scripts (Was: Re: notify via virtual terminal available packages)

2020-09-25 Thread Andy Smith
Hello,

On Thu, Sep 24, 2020 at 08:49:07AM -0600, Charles Curley wrote:
> On Thu, 24 Sep 2020 10:38:55 -0400
> Greg Wooledge  wrote:
> > So you're just doing "sleep 1" every time.
> 
> Ah, thank you. Yup. Which is weird, because it worked when I first
> wrote that many years ago.

In cron scripts where I want a "random" delay, I actually tend to
not really want it to be random, but just different for that host as
opposed to other hosts, otherwise deterministic. I like it if the
delay is the same every time on that host as long as it is a
different delay on different hosts.

So what I tend to do is something like:

sleep $(( $(printf %d "0x$(hostid)") % 60 ))m; /some/command

which will sleep for some amount of time between 0 and 59 minutes,
the same amount every time, but different on different hosts.

(Obviously change the "60" and the "m" to different values for
different things, like you might want "1440" and "m" for minutes in
a day.)

Note that in a file parsed by cron you do need to escape both the
'%' (like '\%').

The printf is needed to turn the hexadecimal value from the "hostid"
command into a decimal number. Is there a way to do that with pure
shell internals that isn't very verbose?

"hostid" tends to return a hexadecimal representation of the first
IPv4 address (but isn't guaranteed to). On a systemd system one
could instead use /etc/machine-id. On Linux there is also
/proc/sys/kernel/random/boot_id (but needs dashes removed).

Systemd timers can do this sort of thing themselves, so no need
there for this sort of scripting.

> But I will move toward more use of unattended-upgrades, which
> handles the original problem differently.

Yup, I use apticron and unattended-upgrades for solving these
problems these days.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

Please consider the environment before reading this e-mail.
 — John Levine



Re: bullseye LTS?

2020-09-14 Thread Andy Smith
Hi Byung-Hee,

On Tue, Sep 15, 2020 at 10:29:46AM +0900, 황병희 wrote:
> Hi i just curious about Bullseye [0]. The Bullseye will be LTS?

Debian does not designate certain releases as LTS releases, so
bullseye will be no more "an LTS release" than buster is now, or
stretch, jessie etc were before them. You are taking an Ubuntu
concept and trying to apply it to Debian.

Debian *does* have an LTS team, who continue to provide security
support on a best-effort basis after a release has become so old
that it ceases to receive security support by the Debian security
team. They've been doing this since the squeeze (6) release.

For example, from 6 July 2020 stretch ceased receiving security
support by the Debian security team and only receives such support
from the LTS team. This will be for a limited selection of packages
and may not be as speedy as regular security support. The LTS
support for stretch will end on 30 June 2022.

Ubuntu's LTS releases do tend to look tempting from a support
lifetime point of view, but it is important to realise that quite a
lot of useful software is excluded from their LTS support. I have
also found updates of the things that are theoretically supported to
be very varied.

I run Ubuntu on some of my desktops and laptops and it is rare that
I manage to reach the end of the theoretical LTS support schedule
before needing updated software has forced me to upgrade release.

For more information please see:

https://wiki.debian.org/LTS

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Can one install packages from Parrot or Kali on Debian testing? (Was: Re: Hi :))

2020-09-10 Thread Andy Smith
Hi Richard,

Your question is one of user support but you've sent it to the
debian-project list, which is about the Debian project itself and
not for asking user questions. So, I have directed replies to the
correct place which is debian-user.

On Thu, Sep 10, 2020 at 04:14:35PM -0400, richard loomis wrote:
> I have a question using debian 10 i noticed ive upgraded till theres no
> more using testing,

Use "testing" is probably for advanced users, but the question you
ask below about mixing in things that aren't Debian suggests you are
maybe not that familiar with Debian. Be careful!

> when i add parrot os repos and kali linux repos theres tons of
> upgrades knowing there using testing also, Is it safe to upgrade
> debian 10 with there repos?

No. You should not mix in things that aren't Debian into Debian
without knowing exactly what you are doing. None of those things
(Parrot, Kali) are designed to be installed on a Debian system. You
will very likely break your entire system doing this. It may even
appear to work for a while, but will break later in mysterious ways.

See https://wiki.debian.org/DontBreakDebian for more details.

In general, upgrading to newer versions of packages for no reason
other than that they exist is not a good practice. You should have a
reason for wanting a newer package than what exists in Debian
testing. I recommend that if you do have such a need for specific
newer packages, you install them individually from upstream
following upstream's instructions.

Cheers,
Andy



Re: 32 versus 64 bit reading list suggestions

2020-09-07 Thread Andy Smith
On Mon, Sep 07, 2020 at 09:37:47PM +, Andy Smith wrote:
> Basically there are already fewer upstream kernel developers that
> care about and understand 32-bit x86, and bug and even security
> fixes specific to 32-bit x86 lag behind those for amd64. KPTI fixes
> to address Meltdown and Spectre took an extra 6 months to reach
> 32-bit x86.
> 
> https://lwn.net/Articles/743265/
> 
> https://www.phoronix.com/scan.php?page=news_item=Linux-32-Bit-KPTI-Bug-Fix

Oh, and:


https://lwn.net/ml/oss-security/calcetrw1z0gclfjz-1jwj_wct3+axxkp_wocxy8jkbslzv8...@mail.gmail.com/

"To those of you who actually support x86_32: please either
consider stopping supporting it or finding and paying someone to
give it serious upstream attention.  We need real CI resources
and we need developers to test things for real, fix what's
broken, and generally keep it up to date. And the developers in
question should have an appropriate degree of nostalgic
adoration of segments, gates, and other delights from the i386
era."

Kind of suggests to me that changes specific to x86_32 aren't being
made, and when they are being made they aren't being tested except
by users in the wild. If you never upgrade your kernel and it's in a
more secure environment (e.g. device with only one user, not exposed
to public Internet, etc.) then it's obviously less of a worry, but…

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: 32 versus 64 bit reading list suggestions

2020-09-07 Thread Andy Smith
Hello,

On Mon, Sep 07, 2020 at 07:57:18PM +0100, Joe wrote:
> One practical point: it isn't possible to upgrade from a 32-bit
> installation to a 64-bit one, it's a reinstall job. I did actually have
> a go once, but quickly got bogged down with 'do A before B, and do B
> before A'.

I've done it successfully more than 10 times, but:

- It isn't officially supported

- I did it on virtual machines which by their nature had fewer
  packages installed and are much simpler than the typical desktop
  or "server that does everything" in a small office.

It is not for the faint of heart and on any moderately complex
install it is faster and certainly easier to just reinstall.

Basic process:

https://wiki.debian.org/CrossGrading

Do not do this if you aren't willing to possibly render your system
unbootable requiring a lengthy repair session before you get
frustrated and just reinstall anyway.

So far in this thread people have pointed out that upstream software
vendors are increasingly moving away from 32-bit x86 for various
reasons. What hasn't been mentioned yet is that one of the upstream
projects where 32-bit support is sub-optimal and getting worse is
the Linux kernel itself.

Basically there are already fewer upstream kernel developers that
care about and understand 32-bit x86, and bug and even security
fixes specific to 32-bit x86 lag behind those for amd64. KPTI fixes
to address Meltdown and Spectre took an extra 6 months to reach
32-bit x86.

https://lwn.net/Articles/743265/

https://www.phoronix.com/scan.php?page=news_item=Linux-32-Bit-KPTI-Bug-Fix

If your hardware supports it then you are best off planning to move
to it sooner rather than later. X86_32 is already in the critical
care ward.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



Re: Can't log in after Stretch to Buster upgrade

2020-09-02 Thread Andy Smith
Hello,

On Wed, Sep 02, 2020 at 11:13:30AM -0700, cgi...@surfnaked.ca wrote:
> The Buster upgrade seemed to work OK.  I re-booted and got to my
> xfce login screen.  But when I entered my user ID and password,
> the screen blanked for a second or so, then came back to a blank
> login screen.

Use ctrl-alt-F3 to switch to a virtual console and attempt to log in
there, then examine your logs to see if there are any hints as to
anything that has failed to start?

I am not an xfce user so hopefully one of them will be along shortly
to tell you which log files to look in for likely answers, but you
could start with /var/log/syslog and /var/log/Xorg.0.log.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting



<    2   3   4   5   6   7   8   9   10   11   >