Re: [Development] Archiving is working

[Paul Wicking]

On 2/1/19 1:38 PM, Shawn Rutledge wrote:
> It appears the archive has been restored, as far as I can tell.  Thanks to 
> whoever got that done!
> 
It is indeed much better! As some of you may have noticed, there are 
still a few things to iron out. Please report any further findings on 
the Jira bug issue for this, which you'll find here:

https://bugreports.qt.io/browse/QTQAINFRA-2553

Hope ya'll have a wonderful weekend :)

Paul
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Giuseppe D'Angelo via Development]

Il 01/02/19 13:38, Shawn Rutledge ha scritto:

It appears the archive has been restored, as far as I can tell.  Thanks to 
whoever got that done!


Development looks ok-ish, but other mailing lists (e.g. interest, 
creator) are still broken!


Ok-ish: the archives using EET/EEST instead of CET/CEST (or UTC). That 
again will shuffle indexing (not break permalinks, just aggregate things 
differently because of the timezone). Please bring it back to CET/CEST.


Cheers,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Shawn Rutledge]

It appears the archive has been restored, as far as I can tell.  Thanks to 
whoever got that done!

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Paul Wicking]

On 1/28/19 1:26 PM, Shawn Rutledge wrote:
> 
> 
>> On 28 Jan 2019, at 09:43, Mitch Curtis  wrote:
>>
>>> -Original Message-
>>> From: Development  On Behalf Of
>>> Giuseppe D'Angelo via Development
>>> Sent: Friday, 25 January 2019 7:23 PM
>>> To: development@qt-project.org
>>> Subject: Re: [Development] Archiving is working
>>>
>>> Il 25/01/19 18:41, Edward Welbourne ha scritto:
>>>> The pages have all been updated and had their URLs changed.
>>>> Or are you trying to tell me something else ?
>>>> I'm not sure I fully understand.
>>>
>>> Yes. And that's the problem. It *MUST NOT* happen. Links to mails in the
>>> mailing list are found in commit messages, on the bugtracker, and so on.
>>> Breaking them is a big no-no-no.
>>
>> Another +1. I don't want my commit messages to become useless because of 
>> broken mailing list archive links. I'm pretty sure I also linked to a 
>> mailing list thread when I did a bulk Qt 4 jira task cleanup, so if that 
>> link is now broken it's broken in hundreds of comments.
> 
> +1 from me too
> 
> If the archive can’t be regenerated with the same numbers, aren’t there 
> backups somewhere that can be restored?
> 
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development
> 

Another +1. Not being able to handle this properly reflects poorly on 
the Qt project, imho. There is (nor was) no need to regenerate the 
archive, that shouldn't have happened. As it stands, restoring a backup 
to a new VM and populating that with the entries since the situation 
arose (ca mid-November, if my memory serves me right) appears as a 
solution that will keep the integrity of our archives until that date, 
at the cost of possibly upsetting the integrity of the entries since 
(could probably also be avoidable but requires more effort). In other 
words, the longer we're in the current state, the worse the problem becomes.

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Shawn Rutledge]

> On 28 Jan 2019, at 09:43, Mitch Curtis  wrote:
> 
>> -Original Message-
>> From: Development  On Behalf Of
>> Giuseppe D'Angelo via Development
>> Sent: Friday, 25 January 2019 7:23 PM
>> To: development@qt-project.org
>> Subject: Re: [Development] Archiving is working
>> 
>> Il 25/01/19 18:41, Edward Welbourne ha scritto:
>>> The pages have all been updated and had their URLs changed.
>>> Or are you trying to tell me something else ?
>>> I'm not sure I fully understand.
>> 
>> Yes. And that's the problem. It *MUST NOT* happen. Links to mails in the
>> mailing list are found in commit messages, on the bugtracker, and so on.
>> Breaking them is a big no-no-no.
> 
> Another +1. I don't want my commit messages to become useless because of 
> broken mailing list archive links. I'm pretty sure I also linked to a mailing 
> list thread when I did a bulk Qt 4 jira task cleanup, so if that link is now 
> broken it's broken in hundreds of comments.

+1 from me too

If the archive can’t be regenerated with the same numbers, aren’t there backups 
somewhere that can be restored?

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Mitch Curtis]

> -Original Message-
> From: Development  On Behalf Of
> Giuseppe D'Angelo via Development
> Sent: Friday, 25 January 2019 7:23 PM
> To: development@qt-project.org
> Subject: Re: [Development] Archiving is working
> 
> Il 25/01/19 18:41, Edward Welbourne ha scritto:
> > The pages have all been updated and had their URLs changed.
> > Or are you trying to tell me something else ?
> > I'm not sure I fully understand.
> 
> Yes. And that's the problem. It *MUST NOT* happen. Links to mails in the
> mailing list are found in commit messages, on the bugtracker, and so on.
> Breaking them is a big no-no-no.

Another +1. I don't want my commit messages to become useless because of broken 
mailing list archive links. I'm pretty sure I also linked to a mailing list 
thread when I did a bulk Qt 4 jira task cleanup, so if that link is now broken 
it's broken in hundreds of comments.

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Elvis Stansvik]

Den fre 25 jan. 2019 kl 19:23 skrev Giuseppe D'Angelo via Development
:
>
> Il 25/01/19 18:41, Edward Welbourne ha scritto:
> > The pages have all been updated and had their URLs changed.
> > Or are you trying to tell me something else ?
> > I'm not sure I fully understand.
>
> Yes. And that's the problem. It *MUST NOT* happen. Links to mails in the
> mailing list are found in commit messages, on the bugtracker, and so on.
> Breaking them is a big no-no-no.
>
>
> As I said before, the new archive is corrupted, and thus must be fixed.
> Look at the numbering of emails from the new archives and note at how it
> "jumps":
>
> > https://lists.qt-project.org/pipermail/development/2011-October/date.html
>
> Compare it to the old numbering (progressive):
>
> > https://web.archive.org/web/20180408194615/http://lists.qt-project.org/pipermail/development/
>
>
>
> Then take a look at the current archive:
>
> > https://lists.qt-project.org/pipermail/development/2011-October.txt.gz
>
> Compare it with the old archive:
>
> > https://web.archive.org/web/20180408194615/http://lists.qt-project.org/pipermail/development/2011-October.txt.gz
>
> The new archive is corrupted; there are duplicated emails in there.
> What's needed now is to make pipermail produce a result identical to the
> old archive.

+1 https://www.w3.org/Provider/Style/URI ("Cool URIs don't change")

If you're going accept this as "solved", be aware that you're breaking
links in a *lot* of places, not just on Qt infra like QUIPs and
whatnot, but lots of other public forums, mailing lists, code bases,
commit messages, issue trackers etc.

I really think you need to find a way to make existing links work again.

Elvis

>
>
> My 2 c,
> --
> Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
> KDAB (France) S.A.S., a KDAB Group company
> Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
> KDAB - The Qt, C++ and OpenGL Experts
>
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Giuseppe D'Angelo via Development]

Il 25/01/19 18:41, Edward Welbourne ha scritto:

The pages have all been updated and had their URLs changed.
Or are you trying to tell me something else ?
I'm not sure I fully understand.


Yes. And that's the problem. It *MUST NOT* happen. Links to mails in the 
mailing list are found in commit messages, on the bugtracker, and so on. 
Breaking them is a big no-no-no.



As I said before, the new archive is corrupted, and thus must be fixed. 
Look at the numbering of emails from the new archives and note at how it 
"jumps":



https://lists.qt-project.org/pipermail/development/2011-October/date.html


Compare it to the old numbering (progressive):


https://web.archive.org/web/20180408194615/http://lists.qt-project.org/pipermail/development/




Then take a look at the current archive:


https://lists.qt-project.org/pipermail/development/2011-October.txt.gz


Compare it with the old archive:


https://web.archive.org/web/20180408194615/http://lists.qt-project.org/pipermail/development/2011-October.txt.gz


The new archive is corrupted; there are duplicated emails in there. 
What's needed now is to make pipermail produce a result identical to the 
old archive.



My 2 c,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Edward Welbourne]

Liang Qi (25 January 2019 14:17)
> Looks like the index was also regenerated for October 2016,
>
> [Development] Coin news - Jedrzej Nowacki
> Tue Oct 25 10:22:41 EEST 2016
>
> Current link: 
> https://lists.qt-project.org/pipermail/development/2016-October/055084.html
>
> It was: 
> http://lists.qt-project.org/pipermail/development/2016-October/027542.html
>
> via 
> https://web.archive.org/web/20161108191742/http://lists.qt-project.org/pipermail/development/2016-October/thread.html

The pages have all been updated and had their URLs changed.
Or are you trying to tell me something else ?
I'm not sure I fully understand.

I've just pushed a change for review, that updates all existing QUIPs to
use URLs in the new archive; and commented on all pending QUIPs in
review on my best guesses at which URLs in the new archive match the
ones they refer to in the old one.  See various reviews at:
* https://codereview.qt-project.org/#/q/status:open+project:meta/quips,n,z

I note, in passing, that there are several outstanding reviews that seem
not to have moved in a long time.  Please think about what you're doing
with those QUIPs, authors and reviewers.

As part of that, I've also updated the heroku instance that displays the
current state of everything; it uses the most recent patch set from each
outstanding review, plus some clean-up of mine (renaming quip-*.txt to
quip-*.rst and fixing the URLs to those I suggested in reviews).  If you
find any remaining broken links in those, please let me know.
* https://quips-qt-io.herokuapp.com

Now I shall go home and have a week-end that feels over-due,

Eddy.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Liang Qi]

Looks like the index was also regenerated for October 2016,

[Development] Coin news - Jedrzej Nowacki
Tue Oct 25 10:22:41 EEST 2016

Current link: 
https://lists.qt-project.org/pipermail/development/2016-October/055084.html

It was: 
http://lists.qt-project.org/pipermail/development/2016-October/027542.html

via 
https://web.archive.org/web/20161108191742/http://lists.qt-project.org/pipermail/development/2016-October/thread.html

--Liang

On Fri, 25 Jan 2019 at 14:00, Edward Welbourne  wrote:
>
> Hi all,
>
> Again, our sysadmins claim they've fixed what was broken in the
> archives.  Please check up on any archives in which you've seen holes
> lately and let us know if there are still any problems.
>
> If all's well, I'll hope our archives are stable again and begin going
> through all our QUIPs working out where the mails they've linked have
> ended up after all of this - please do likewise for other links you know
> about.
>
> Eddy.
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Edward Welbourne]

Hi all,

Again, our sysadmins claim they've fixed what was broken in the
archives.  Please check up on any archives in which you've seen holes
lately and let us know if there are still any problems.

If all's well, I'll hope our archives are stable again and begin going
through all our QUIPs working out where the mails they've linked have
ended up after all of this - please do likewise for other links you know
about.

Eddy.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Giuseppe D'Angelo via Development]

Il 15/01/19 16:19, Sampo Heikkinen ha scritto:

Yes it reindexes everytime whenever I want to merge something. Then I have to 
run the
re-archiving command which basically generates all the html-files again. Please 
go through
the needed mailing lists 
viahttps://lists.qt-project.org/pipermail//
 From there you see the archive and all the mails.


The email archives appear to be corrupted: the same message is present 
multiple times, I guess screwing up the numbering and thus the links.


For instance, take a look at


https://lists.qt-project.org/pipermail/development/2011-October.txt


And compare it with


https://web.archive.org/web/20150919192156/http://lists.qt-project.org/pipermail/development/2011-October.txt.gz


(Yes, they also used to be gzipped).


Cheers,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Shawn Rutledge]

> On 15 Jan 2019, at 16:19, Sampo Heikkinen via Development 
>  wrote:
> 
> Hello
> 
> Yes it reindexes everytime whenever I want to merge something. Then I have to 
> run the
> re-archiving command which basically generates all the html-files again. 
> Please go through
> the needed mailing lists via 
> https://lists.qt-project.org/pipermail//  
> From there you see the archive and all the mails.
> 
> Unfortunately old links seems to be gone. But this was needed operation to 
> merge
> stuff from the old mailing list server.

Then maybe we should use different archiving software such that links don't 
change even when the archive is regenerated.  For example, it could hash the 
body of the message and generate the link from that.  Maybe have part of the 
subject line, sender etc. encoded in the URL so that it’s somewhat readable 
even if links get broken, so that people can manually rediscover where it 
moved.  Otherwise, it’s quite bad when we have cases like QTBUG-35476 which 
refers to 
http://lists.qt-project.org/pipermail/development/2013-December/014491.html and 
now that’s gone, and it’s hard to guess which email (out of the whole month) it 
could have been referring to.

> 
> br, Sampo
> 
> 
> Sampo Heikkinen
> Senior System Specialist
> Digital Business
>   
> Telia Inmics-Nebula
> +358 9 6818 3848
> sampo.heikki...@inmicsnebula.fi
> www.inmicsnebula.fi
> Valimotie 21, 00380 Helsinki, Finland
>  
> 
> -Original Message-
> From: Development  On Behalf Of Giuseppe 
> D'Angelo via Development
> Sent: tiistai 15. tammikuuta 2019 17.10
> To: development@qt-project.org
> Subject: Re: [Development] Archiving is working
> 
> Il 15/01/19 15:15, Julius Bullinger ha scritto:
>> It's also mails that have been in the archive already that are now 
>> missing. For example, search for "site:lists.qt-project.org creator" 
>> on Google. Most of the links there that have been archived do not work 
>> anymore. This affects all lists, development, interest, qt-creator, 
>> qbs, and so on.
> 
> Looks like the lists have been somehow reindexed. Google will eventually 
> catch up, but it's a PITA for all the existing links that are now broken.
> 
> 
> Compare the email numbering of
> 
>> https://lists.qt-project.org/pipermail/development/2011-October/date.html#start
> 
> (current), where the email numbers "jump", with the original numbering, 
> which has sequential numbers:
> 
>> https://web.archive.org/web/20170331145821/http://lists.qt-project.org/pipermail/development/2011-October/date.html
> 
> 
> My 2 c,
> -- 
> Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
> KDAB (France) S.A.S., a KDAB Group company
> Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
> KDAB - The Qt, C++ and OpenGL Experts
> 
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Sampo Heikkinen via Development]

Hello

Yes it reindexes everytime whenever I want to merge something. Then I have to 
run the
re-archiving command which basically generates all the html-files again. Please 
go through
the needed mailing lists via 
https://lists.qt-project.org/pipermail//  
From there you see the archive and all the mails.

Unfortunately old links seems to be gone. But this was needed operation to merge
stuff from the old mailing list server.

br, Sampo


Sampo Heikkinen
Senior System Specialist
Digital Business
  
Telia Inmics-Nebula
+358 9 6818 3848
sampo.heikki...@inmicsnebula.fi
www.inmicsnebula.fi
Valimotie 21, 00380 Helsinki, Finland
 

-Original Message-
From: Development  On Behalf Of Giuseppe 
D'Angelo via Development
Sent: tiistai 15. tammikuuta 2019 17.10
To: development@qt-project.org
Subject: Re: [Development] Archiving is working

Il 15/01/19 15:15, Julius Bullinger ha scritto:
> It's also mails that have been in the archive already that are now 
> missing. For example, search for "site:lists.qt-project.org creator" 
> on Google. Most of the links there that have been archived do not work 
> anymore. This affects all lists, development, interest, qt-creator, 
> qbs, and so on.

Looks like the lists have been somehow reindexed. Google will eventually catch 
up, but it's a PITA for all the existing links that are now broken.


Compare the email numbering of

> https://lists.qt-project.org/pipermail/development/2011-October/date.html#start

(current), where the email numbers "jump", with the original numbering, 
which has sequential numbers:

> https://web.archive.org/web/20170331145821/http://lists.qt-project.org/pipermail/development/2011-October/date.html


My 2 c,
-- 
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Giuseppe D'Angelo via Development]

Il 15/01/19 15:15, Julius Bullinger ha scritto:

It's also mails that have been in the archive already that are now
missing. For example, search for "site:lists.qt-project.org creator" on
Google. Most of the links there that have been archived do not work
anymore. This affects all lists, development, interest, qt-creator, qbs,
and so on.


Looks like the lists have been somehow reindexed. Google will eventually 
catch up, but it's a PITA for all the existing links that are now broken.



Compare the email numbering of


https://lists.qt-project.org/pipermail/development/2011-October/date.html#start


(current), where the email numbers "jump", with the original numbering, 
which has sequential numbers:



https://web.archive.org/web/20170331145821/http://lists.qt-project.org/pipermail/development/2011-October/date.html



My 2 c,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Giuseppe D'Angelo via Development]

Il 15/01/19 15:10, Sampo Heikkinen via Development ha scritto:

Quicker look to development archives shows that it should be

working now. At least my recent email can be found from archives.

It’s most likely that every email sent during these two weeks are gone

forever because file permissions. I will investigate this still though.



It's not. Your "test" email does not appear, and


https://lists.qt-project.org/pipermail/development/


has an *extremely suspicious* "15 January 2019" entry.


Cheers,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Archiving is working

[Julius Bullinger]

On 15.01.2019 15:10, Sampo Heikkinen via Development wrote:

Quicker look to development archives shows that it should be
working now. At least my recent email can be found from archives.
It’s most likely that every email sent during these two weeks are gone
forever because file permissions. I will investigate this still though.


It's also mails that have been in the archive already that are now 
missing. For example, search for "site:lists.qt-project.org creator" on 
Google. Most of the links there that have been archived do not work 
anymore. This affects all lists, development, interest, qt-creator, qbs, 
and so on.

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

[Development] Archiving is working

[Sampo Heikkinen via Development]

Quicker look to development archives shows that it should be
working now. At least my recent email can be found from archives.
It's most likely that every email sent during these two weeks are gone
forever because file permissions. I will investigate this still though.

br, Sampo

Sampo Heikkinen
Senior System Specialist
Digital Business

Telia Inmics-Nebula
+358 9 6818 3848
sampo.heikki...@inmicsnebula.fi
www.inmicsnebula.fi
Valimotie 21, 00380 Helsinki, Finland


___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development