Re: [Scons-dev] **JUNK** Re: SCons, Mercurial, BitBucket, Git, and GitHub

[Dirk Baechle]

Bill, 

sorry that was really a misunderstanding. So far I've been working on the 
migration Tigris->Roundup. So the "backend" definitely needs to be rewritten 
for Github. 

Regards, 

Dirk


Am 31. August 2017 22:47:15 MESZ schrieb Bill Deegan 
:
>Dirk,
>
>I guess I mis-understood.
>I thought there was a version for pushing to github already?
>
>-Bill
>
>On Thu, Aug 31, 2017 at 1:20 PM, Dirk Bächle  wrote:
>
>> Hi all,
>>
>> On 31.08.2017 08:27, Dirk Bächle wrote:
>>
>> Bill,
>>
>> [...]
>>
>>
>>
>> True. This can also happen after the initial migration.
>> Dirk - is it simple to grab the various attachments from tigris?
>>
>>
>>
>> I'd say so, yes. Let me dig up the sources for that, which might take
>a
>> day or two.
>>
>>
>> please find attached a recent version of my migration script
>> "Tigris->Roundup". Follow the comments inside to download all current
>> issues (including attachments). Then you'll be mostly interested in
>> "create_file", it's currently using xmlrpclib to write to a Roundup
>tracker
>> instance directly. But hooking your own code in for pushing to Github
>> shouldn't take that much work. ;)
>>
>> Best of luck, and have fun!
>>
>> Dirk
>>
>>
>>
>> ___
>> Scons-dev mailing list
>> Scons-dev@scons.org
>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>
>>

-- 
Sent from my Android with K-9 Mail.___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] **JUNK** Re: SCons, Mercurial, BitBucket, Git, and GitHub

[Bill Deegan]

Dirk,

I guess I mis-understood.
I thought there was a version for pushing to github already?

-Bill

On Thu, Aug 31, 2017 at 1:20 PM, Dirk Bächle  wrote:

> Hi all,
>
> On 31.08.2017 08:27, Dirk Bächle wrote:
>
> Bill,
>
> [...]
>
>
>
> True. This can also happen after the initial migration.
> Dirk - is it simple to grab the various attachments from tigris?
>
>
>
> I'd say so, yes. Let me dig up the sources for that, which might take a
> day or two.
>
>
> please find attached a recent version of my migration script
> "Tigris->Roundup". Follow the comments inside to download all current
> issues (including attachments). Then you'll be mostly interested in
> "create_file", it's currently using xmlrpclib to write to a Roundup tracker
> instance directly. But hooking your own code in for pushing to Github
> shouldn't take that much work. ;)
>
> Best of luck, and have fun!
>
> Dirk
>
>
>
> ___
> Scons-dev mailing list
> Scons-dev@scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] **JUNK** Re: SCons, Mercurial, BitBucket, Git, and GitHub

[Dirk Bächle]

Hi all,


On 31.08.2017 08:27, Dirk Bächle wrote:


Bill,


[...]



True. This can also happen after the initial migration.
Dirk - is it simple to grab the various attachments from tigris?


I'd say so, yes. Let me dig up the sources for that, which might take 
a day or two.




please find attached a recent version of my migration script 
"Tigris->Roundup". Follow the comments inside to download all current 
issues (including attachments). Then you'll be mostly interested in 
"create_file", it's currently using xmlrpclib to write to a Roundup 
tracker instance directly. But hooking your own code in for pushing to 
Github shouldn't take that much work. ;)


Best of luck, and have fun!

Dirk


""" Import tracker data from Tigris.org

This script needs the following steps to work:

1. Extract the issues as XML files via the project's xml.cgi:

import_tigris.py files  

   this will place all the downloaded XML files in the files dir.
   An example:

import_tigris.py files scons import

2. Import the data via xmlrpc:

import_tigris.py push  

   Example:

import_tigris.py push http://admin:admin@localhost:8917/demo/xmlrpc import

And you're done!
"""

import sys
import os
import glob
import lxml
import lxml.etree
from urllib2 import urlopen
import base64
import xmlrpclib
import csv

# -
# natsort: Natural string sorting.
# -

# By Seo Sanghyeon.  Some changes by Connelly Barnes.

def try_int(s):
"Convert to integer if possible."
try: return int(s)
except: return s


def natsort_key(s):
"Used internally to get a tuple by which s is sorted."
import re
return map(try_int, re.findall(r'(\d+|\D+)', s))


def natcmp(a, b):
"Natural string comparison, case sensitive."
return cmp(natsort_key(a), natsort_key(b))


def natcasecmp(a, b):
"Natural string comparison, ignores case."
return natcmp(a.lower(), b.lower())


def natsort(seq, cmp=natcmp):
"In-place natural string sort."
seq.sort(cmp)


def natsorted(seq, cmp=natcmp):
"Returns a copy of seq, sorted by natural string sort."
import copy
temp = copy.copy(seq)
natsort(temp, cmp)
return temp

# -
# Download issues from Tigris
# -


def issue_exists(id, url):
""" Return whether the issue page with the given
index (1-based!) exists, or not.
@param id Index (1-based) of the issue to test
@param url Base URL to the project's xml.cgi (no params attached!)
@return `True` if the issue exists, `False` if not
"""
query_url = url + '?include_attachments=false=%d' % id
try:
issues_xml = lxml.etree.XML(urlopen(query_url).read())
for issue in issues_xml.xpath('issue'):
error = issue.attrib.get('status_code', None)
if error and error == "404":
return False
else:
return True
except:
pass

return False


def binprobe(left, right, index_exists):
""" Searches the last existing entry in a
"sequence of indices" from left to right (including).
Assumes that "left" starts on an existing entry,
and left <= right, and left >= 0, and right >= 0.
The index "right" may either be the last existing entry,
or points to an entry that doesn't exist.
@param left Start index
@param right End index
@param index_exists Function that checks whether a 1-based index
 is in or out of the sequence (exists or not).
@return 1-based index of the last existing entry, in
 the given interval
"""
while ((right - left) > 1):
middle = left + (right - left) // 2
if not index_exists(middle):
right = middle - 1
else:
left = middle

# Special handling for when only the two
# last IDs are left...or a single one (left=right).
if index_exists(right):
return right
return left


def get_number_of_issues(url, start_id=1, BSEARCH_STEP_SIZE=1024):
""" Return the 1-based index of the highest available (=existing)
issue for the given base URL, when starting to
probe at start_id.
@param url Base URL to the project's xml.cgi (no params attached!)
@param start_id Index (1-based) from where to probe upwards
@return 1-based index of the last existing issue
"""
# Start at the given index
id = start_id
# Loop in large steps, until id doesn't exist
steps = 0
while issue_exists(id, url):
id += BSEARCH_STEP_SIZE
steps += 1

if steps:
# Start the binary search
left = id - BSEARCH_STEP_SIZE
right = id - 1
return binprobe(left, right,
lambda x: 

Re: [Scons-dev] **JUNK** Re: SCons, Mercurial, BitBucket, Git, and GitHub

[Dirk Bächle]

Bill,


On 31.08.2017 02:46, Bill Deegan wrote:



On Wed, Aug 30, 2017 at 9:55 AM, Gaurav Juvekar 
> wrote:


Hi,

[...]

Couldn't the patches be re-uploaded as gists and then linked from
within the
GitHub issue?


True. This can also happen after the initial migration.
Dirk - is it simple to grab the various attachments from tigris?


I'd say so, yes. Let me dig up the sources for that, which might take a 
day or two.


What I also have is a complete download of the old "devel" and "user" 
mailing lists from Tigris. My goal for these is to write some data 
classes that enable you to search for certain keywords over the single 
entries. Like this it could serve as some kind of "knowledge base".
Together with the snapshot of the "issues", we could add them to a 
separate "scons-archive" such that no information gets lost.
The remaining task would then be to make this "info heap" searchable in 
some way.


Regarding size, we are talking about approximately:

issues : 53 MB (without attachments)
devel : 164 MB
user : 244 MB

Best regards,

Dirk

___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev

Re: [Scons-dev] **JUNK** Re: SCons, Mercurial, BitBucket, Git, and GitHub

[Bill Deegan]

On Wed, Aug 30, 2017 at 9:55 AM, Gaurav Juvekar 
wrote:

> Hi,
>
> > 4) Retaining a link to the tigris issue in the migrated issue to retain
> > access to patches,etc.  If possible a notation or perhaps a tag that
> there
> > are files on tigris, but if it's painful to do so, then it's not a
> > requirement (the tagging).
>
> Couldn't the patches be re-uploaded as gists and then linked from within
> the
> GitHub issue?
>

True. This can also happen after the initial migration.
Dirk - is it simple to grab the various attachments from tigris?


>
> > 6) A change to github for source control doesn't need to wait for the
> > issues and the wiki to be migrated.  Code first, the rest follow
> (hopefully
> > fairly quickly but immediate is not necessary).
>
> If we want to retain the issue numbers (the numbers could be mentioned as
> in tigris comments), we will have to migrate them at the same time so
> that someone else doesn't create a issue or send a pull request in the
> meantime.
>

Good point.
___
Scons-dev mailing list
Scons-dev@scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev