> > Is there anybody who kindly provides a github account to make sure that > "any notifications are never triggered" when executing migration? I > confirmed this with my account, just wanted to test it with other one or > two accounts. >
I've subscribed to all notifications for that sandbox-lucene-10557 repo. Let me know if you are going to create a different repo and I'll subscribe to that as well. - Houston On Fri, Jun 24, 2022 at 12:47 PM Tomoko Uchida <tomoko.uchida.1...@gmail.com> wrote: > I started to play around with the (unofficial) Issue Import API > <https://gist.github.com/jonmagic/5282384165e0f86ef105> and so far so > good. > - Original timestamps (created, updated) are preserved > - An issue and its comments can be imported with one post request; > importing is done asynchronously but the background job looks sufficiently > fast. > - Notifications are not triggered with mentions in the issue comments > (e.g. https://github.com/mocobeta/sandbox-lucene-10557/issues/249) > > I will do more tests and rewrite the scripts with this API and if it works > as expected, try to migrate whole Jira issues into a test repo next time; > it still needs two-pass migration but could be done in a few hours with the > default rate limit. > Is there anybody who kindly provides a github account to make sure that > "any notifications are never triggered" when executing migration? I > confirmed this with my account, just wanted to test it with other one or > two accounts. > > Tomoko > > 2022年6月24日(金) 21:12 Adrien Grand <jpou...@gmail.com>: > >> FWIW I don't mind receiving these one-time notifications for the purpose >> of the migration. >> >> On Fri, Jun 24, 2022 at 12:58 PM Tomoko Uchida < >> tomoko.uchida.1...@gmail.com> wrote: >> >>> > If you could give me some "example mail", I can add a "feed to trash" >>> sieve rule before you start it. >>> >>> If we decide to migrate all Jira issues to GItHub with reporter/author >>> account re-mapping (hyperlinks), and we still cannot find a workaround >>> not to trigger notifications, I'll let you all know the example mail >>> text. >>> I can also trigger some real notifications by simulating a few >>> migrations for people who wish to know what will happen in their >>> mailbox/github UI beforehand. I'm really not sure what will happen to >>> everyone by triggering so many notifications. >>> >>> Precisely speaking, you will receive the same number of notifications >>> to your all activities in Jira so far: reporting an issue, assigning >>> yourself to an issue, and leaving a comment. >>> GitHub UI and mailers will aggregate them into "issues", I think. >>> >>> > Another question: With the alternative API is it possible to keep >>> original Reporters and Authors of comments, too? >>> >>> I haven't tried the unofficial API yet, but as far as I read the >>> examples, it is not possible to keep original reporters or authors. >>> You cannot change/tweak GitHub issue reporter or author by any means - >>> it looks a sane decision to me. >>> >>> >>> 2022年6月24日(金) 16:18 Uwe Schindler <u...@thetaphi.de>: >>> > >>> > Hi, >>> > >>> > Am 23.06.2022 um 19:09 schrieb Tomoko Uchida: >>> > >>> > uschindler,Uwe Schindler,2474 >>> > >>> > If you could give me some "example mail", I can add a "feed to trash" >>> sieve rule before you start it. >>> > >>> > We have 109158 resources (10608 issues + 98550 comments) in total. >>> With the default rate limit of 5000 reqs per hour, it will take 22 hours if >>> there are no failures/retries. With an enterprise account, it will take 7-8 >>> hours I think. >>> > >>> > So a sieve rule would be good to not get constant mail all the time, >>> because my 2474 issues (or notifications?) will be distrubuted over the >>> whole day. >>> > >>> > Another question: With the alternative API is it possible to keep >>> original Reporters and Authors of comments, too? >>> > >>> > Uwe >>> > >>> > Tomoko >>> > >>> > >>> > 2022年6月24日(金) 0:39 Tomoko Uchida <tomoko.uchida.1...@gmail.com>: >>> >> >>> >> It seems not new - I don't figure out why this is not published as a >>> public API yet but according to the comments there, it could be >>> buggy/unstable (still worth a try to me). >>> >> >>> >> >>> >> >>> >> 2022年6月24日(金) 0:26 Tomoko Uchida <tomoko.uchida.1...@gmail.com>: >>> >>> >>> >>> I just browsed through this article about the "import issues" API >>> (looks pretty new and on technical preview status?) >>> >>> https://gist.github.com/jonmagic/5282384165e0f86ef105 >>> >>> >>> >>> Seems it solves many of our considerations - preserving original >>> timestamp, bulk importing issue comments, and not triggering notifications. >>> >>> >>> >>> I'll try it later; thank you Rob for providing the information. >>> >>> >>> >>> >>> >>> 2022年6月23日(木) 23:18 Michael Sokolov <msoko...@gmail.com>: >>> >>>> >>> >>>> oh phew! glad to hear this was expected >>> >>>> >>> >>>> On Thu, Jun 23, 2022 at 10:17 AM Tomoko Uchida >>> >>>> <tomoko.uchida.1...@gmail.com> wrote: >>> >>>> > >>> >>>> > > Many comments were lost in the transfer. The last one in the >>> copy is >>> >>>> > > only about 1/4 of the way through this gigantic issue. This >>> really is >>> >>>> > > a blocker I think. >>> >>>> > >>> >>>> > I limited the number of comments per issue up to ten for testing. >>> We can migrate literally all comments - one by one. >>> >>>> > >>> >>>> > Tomoko >>> >>>> > >>> >>>> > >>> >>>> > 2022年6月23日(木) 23:09 Tomoko Uchida <tomoko.uchida.1...@gmail.com>: >>> >>>> >> >>> >>>> >> Hi, >>> >>>> >> I have little now to carefully read/respond to all replies right >>> now, but just wanted to answer this. >>> >>>> >> >>> >>>> >> > Will it be possible to preserve links from issues -> pull >>> requests? >>> >>>> >> >>> >>>> >> Yes it's a bit cumbersome (and it could be difficult to make >>> sure that all links to PRs are covered - it's not solid metadata, you need >>> to parse github bot's comments) but surely possible. >>> >>>> >> See https://github.com/mocobeta/sandbox-lucene-10557/issues/188. >>> >>>> >> >>> >>>> >> As for notifications and attachment files, if there are ways to >>> manage these it'd be great. >>> >>>> >> >>> >>>> >> >>> >>>> >> 2022年6月23日(木) 22:59 Dawid Weiss <dawid.we...@gmail.com>: >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> Interesting, thanks Rob. I see the attachments have been ported >>> in that article as well - something the official API is not able to do. >>> >>>> >>> >>> >>>> >>> https://jira.spring.io/browse/DATACMNS-617?redirect=false >>> >>>> >>> >>> https://github.com/spring-projects/spring-data-commons/issues/1080 >>> >>>> >>> >>> >>>> >>> Dawid >>> >>>> >>> >>> >>>> >>> On Thu, Jun 23, 2022 at 3:26 PM Rob Audenaerde < >>> rob.audenae...@gmail.com> wrote: >>> >>>> >>>> >>> >>>> >>>> I didn't read the entire thread, so apologies if this is a >>> duplicate: >>> >>>> >>>> >>> >>>> >>>> Did you check >>> https://spring.io/blog/2021/01/07/spring-data-s-migration-from-jira-to-github-issues >>> >>>> >>>> >>> >>>> >>>> They especially write there is an api that doesn't trigger >>> notifications. >>> >>>> >>>> >>> >>>> >>>> It is documented here: >>> https://gist.github.com/jonmagic/5282384165e0f86ef105 >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> On Thu, Jun 23, 2022, 15:14 Michael Sokolov < >>> msoko...@gmail.com> wrote: >>> >>>> >>>>> >>> >>>> >>>>> Yes thank you! You say this is not difficult, but it looks >>> like a big >>> >>>> >>>>> job to me! Here are a bunch of things I noticed that we would >>> ideally >>> >>>> >>>>> address (from looking at one long and complex issue, >>> LUCENE-9004). I >>> >>>> >>>>> wouldn't be so bold as to say these should block us from >>> proceeding if >>> >>>> >>>>> they're not addressed, just want to point out there is >>> potentially a >>> >>>> >>>>> lot to do: >>> >>>> >>>>> >>> >>>> >>>>> Will it be possible to preserve links from issues -> pull >>> requests? >>> >>>> >>>>> That seems like one of the most important pieces of >>> "metadata". >>> >>>> >>>>> >>> >>>> >>>>> As far as attached files go, I see you seem to have made an >>> attempt? >>> >>>> >>>>> There is a link in >>> https://issues.apache.org/jira/browse/LUCENE-9004 >>> >>>> >>>>> where you had posted a picture of a graph, for example; in >>> >>>> >>>>> https://github.com/mocobeta/sandbox-lucene-10557/issues/171 >>> it is >>> >>>> >>>>> represented as a link. When you click on the link you get an >>> error >>> >>>> >>>>> though. I wonder if it would be possible to link back to the >>> images >>> >>>> >>>>> hosted in JIRA? (Ideally as an <IMG> tag; otherwise a link >>> would be >>> >>>> >>>>> good). >>> >>>> >>>>> >>> >>>> >>>>> I agree with Ryan - I'd be willing to bulk-delete 1000 >>> notifications >>> >>>> >>>>> if it means preserving hyperlinks to people >>> >>>> >>>>> >>> >>>> >>>>> Numbered list formatting became giant bold text (see the >>> comment >>> >>>> >>>>> containing "Here's a strategy") >>> >>>> >>>>> >>> >>>> >>>>> Many comments were lost in the transfer. The last one in the >>> copy is >>> >>>> >>>>> only about 1/4 of the way through this gigantic issue. This >>> really is >>> >>>> >>>>> a blocker I think. I wonder what happened? Maybe some API >>> calls failed >>> >>>> >>>>> and we need to retry??? >>> >>>> >>>>> >>> >>>> >>>>> I wanted to check other fancy formatting (tables, block >>> comments, code >>> >>>> >>>>> blocks, etc) but haven't looked at these yet... >>> >>>> >>>>> >>> >>>> >>>>> On Wed, Jun 22, 2022 at 8:34 AM Ryan Ernst <r...@iernst.net> >>> wrote: >>> >>>> >>>>> > >>> >>>> >>>>> > This is great work Tomoko! A couple minor thoughts: >>> >>>> >>>>> > >>> >>>> >>>>> > * I don’t think a flood of notifications from the import is >>> a problem. It’s a one time hassle, and having the actual user links is nice >>> for GitHub’s cross linking system. >>> >>>> >>>>> > >>> >>>> >>>>> > * Do you have an estimate for how many api calls are >>> needed? How many total issues+comments exist in jira? I assume the limits >>> you dealt with were the default 5k requests per hour. If that will take too >>> long, we could consider using a user from an enterprise account which has >>> 3x the limit. >>> >>>> >>>>> > >>> >>>> >>>>> > On Tue, Jun 21, 2022 at 15:56 Tomoko Uchida < >>> tomoko.uchida.1...@gmail.com> wrote: >>> >>>> >>>>> >> >>> >>>> >>>>> >> Hi all, >>> >>>> >>>>> >> again - this is about GitHub migration. >>> >>>> >>>>> >> >>> >>>> >>>>> >> We have a large disagreement on whether we should migrate >>> existing Jira issues (including all closed issues) to GitHub or not. >>> >>>> >>>>> >> >>> >>>> >>>>> >> I drafted a tiny migration tool [1] to see how it looks if >>> we move Jira issues to GitHub, and tried to migrate a small portion of Jira >>> issues/comments to a test repo. You can see it here: >>> >>>> >>>>> >> - Closed issues list >>> https://github.com/mocobeta/sandbox-lucene-10557/issues?q=is%3Aissue+is%3Aclosed >>> >>>> >>>>> >> - Unresolved issues list: >>> https://github.com/mocobeta/sandbox-lucene-10557/issues >>> >>>> >>>>> >> >>> >>>> >>>>> >> I don't deserve to have a strong opinion on how we should >>> treat 20+ years of history so I would reserve my opinion - would the >>> prototype be of some help to have a conversation? >>> >>>> >>>>> >> I have to leave for a while, I'd be glad if you have a >>> talk on it while I'm away and hopefully reach an agreement. >>> >>>> >>>>> >> >>> >>>> >>>>> >> Here's a summary of what can be done. >>> >>>> >>>>> >> >>> >>>> >>>>> >> You can: >>> >>>> >>>>> >> * migrate all texts in issue descriptions and comments to >>> GitHub; browsing/searching old issues should work fine. >>> >>>> >>>>> >> * extract every issue metadata from Jira and port it to >>> labels or issue descriptions (as plain text). >>> >>>> >>>>> >> * map Jira cross-issue link "LUCENE-xxx" to GitHub issue >>> mention "#yyy". >>> >>>> >>>>> >> * see this example: >>> https://github.com/mocobeta/sandbox-lucene-10557/issues/218 >>> >>>> >>>>> >> * map Jira user ids to GitHub accounts if the mapping is >>> given. >>> >>>> >>>>> >> * convert Jira markups to Markdown with parser library. >>> >>>> >>>>> >> * not perfect - there can be many conversion errors >>> >>>> >>>>> >> >>> >>>> >>>>> >> And here are the limitations. (Please correct me if I'm >>> missing something.) >>> >>>> >>>>> >> >>> >>>> >>>>> >> You cannot: >>> >>>> >>>>> >> * simulate original authors and timestamps; they have to >>> be preserved in free-text forms. >>> >>>> >>>>> >> * migrate attached files (patches, images, etc.) to >>> GitHub; these have to remain in Jira. >>> >>>> >>>>> >> * it's not allowed to programmatically upload files and >>> attach them to issues. >>> >>>> >>>>> >> * create hyperlinks from issues to GitHub accounts >>> (reporters, comment authors, etc.) by mentions; otherwise everyone will >>> receive a huge volume of notifications. >>> >>>> >>>>> >> * still accounts can be noted with a markup `@xxxx` >>> (without mentioning) in their right place >>> >>>> >>>>> >> * "bulk" import issues/comments. Each resource has to be >>> posted one by one. Migration would take many hours (perhaps days?) due to >>> the severe API call rate limit. >>> >>>> >>>>> >> >>> >>>> >>>>> >> It's not a particularly difficult task, however, there >>> will be other uncontrollable things I haven't noticed yet. >>> >>>> >>>>> >> >>> >>>> >>>>> >> [1] >>> https://github.com/mocobeta/sandbox-lucene-10557/tree/main/migration >>> >>>> >>>>> >> >>> >>>> >>>>> >> Tomoko >>> >>>> >>>>> >>> >>>> >>>>> >>> --------------------------------------------------------------------- >>> >>>> >>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> >>>> >>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>>> >>>>> >>> >>>> >>> >>>> >>> --------------------------------------------------------------------- >>> >>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> >>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>>> >>> > -- >>> > Uwe Schindler >>> > Achterdiek 19, D-28357 Bremen >>> > https://www.thetaphi.de >>> > eMail: u...@thetaphi.de >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> >> >> -- >> Adrien >> >
