Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+---
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  closed
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:  fixed
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+---
Changes (by gaba):

 * status:  needs_review => closed
 * resolution:   => fixed


Comment:

 Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--

Comment (by gaba):

 I didn't close it before because we had
 https://trac.torproject.org/projects/tor/ticket/28339 as a child but I
 will "unchild" it and then close this one.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--

Comment (by phw):

 I hear that the refactored code is already deployed. Is there anything
 left before we can close this ticket?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--

Comment (by arma):

 Replying to [comment:8 hiro]:
 > Current branch:
 https://gitweb.torproject.org/user/hiro/gettor.git/tree/?h=refactoring
 > Also we should review the list of people currently able to contribute to
 gettor. At the moment this list includes:
 > arma ilv sukhbir

 I am happy to be removed from this list if it makes roles clearer.

 I also support some new people being added to the list.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--

Comment (by phw):

 Thanks, hiro! The commit history looks much better now.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--

Comment (by hiro):

 Hi phw,
 I have condensed the git history into fewer comments and cleaned up some
 of the files generated by the coverage script (like the HUGE TAGS file).
 Also I think this would help you to visualise it:
 https://dip.torproject.org/hiro/gettor/merge_requests/1/commits

 Also fixed the print bug.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--

Comment (by phw):

 I think it's a great idea to document the code, but it's orthogonal to my
 point that we should strive for clean git histories. Also, I'm happy to
 help, and polish the git history to the best of my abilities! Do you want
 me to make a pass over it?

 Finally, we should take care of the following remark, which seems to be an
 actual bug:
 > The last line in ​61973769 should probably be print_footer() and not
 print_footer.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--

Comment (by hiro):

 I think it would make much more sense to document the code in a wiki
 possibly on dip.tp.o than going through the commits again. A big part of
 what I did was fixing what was in the repo from Isra, what he had testing
 on prod and what could work with our current repo.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--
Changes (by phw):

 * cc: phw (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--

Comment (by phw):

 I had a look at the branch and here are my thoughts:

 * I find it a bit difficult to understand what some of the commits are
 trying to accomplish.  For example,
 
[https://gitweb.torproject.org/user/hiro/gettor.git/commit/?h=refactoring&id=9f5394e7b32c502f1a0e4d294605996ace50ceaa
 9f5394e7] says "Start with main task" but I'm not quite sure what that
 means.  Another example: commit
 
[https://gitweb.torproject.org/user/hiro/gettor.git/commit/?h=refactoring&id=5271b12c105ae394a870936dab9ba634e1af4014
 5271b12c] says "Update script".  A more descriptive commit message would
 be "Use Python 3-style format message." Here's a page that provides an
 excellent overview of crisp and useful commit messages: https://git-
 scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project

 * Several commits seem to group unrelated changes together, in a single
 commit.  For example,
 
[https://gitweb.torproject.org/user/hiro/gettor.git/commit/?h=refactoring&id=309e4e384385d45a1db5d0da336837a56472acc8
 309e4e38] says "Create script to add links to db" but it also adds and
 modifies code comments.  The modification of comments should be a separate
 commit.

 * The last line in
 
[https://gitweb.torproject.org/user/hiro/gettor.git/commit/?h=refactoring&id=61973769729691082d197caa8e07a0a0cd55fa27
 61973769] should probably be `print_footer()` and not `print_footer`.

 * Nitpick: Do we really need spaces in between the `\n` in
 
[https://gitweb.torproject.org/user/hiro/gettor.git/commit/?h=refactoring&id=d4777d66c7cd528bdc4e903de4bea5482eff5d29
 d4777d66]?

 * For substantial changes, it's helpful to provide more than just a one-
 line summary in the git commit message.  For example, I was curious what
 the purpose of the restructuring in
 
[https://gitweb.torproject.org/user/hiro/gettor.git/commit/?h=refactoring&id=ba19003400fe11a68444bbb585fcdb12a93c100f
 ba190034] was. Does it make maintenance easier? A short paragraph under
 the one-line summary wouldn't leave me guessing :)  Commit
 
[https://gitweb.torproject.org/user/hiro/gettor.git/commit/?h=refactoring&id=02ae46d0ea3224c041b343832c5f30e8c072c3db
 02ae46d0] is another example.  It changes several hundred lines of code,
 yet the commit message only says "Update structure and code".

 * What's the purpose of the giant TAGS file that was added in commit
 
[https://gitweb.torproject.org/user/hiro/gettor.git/commit/?h=refactoring&id=c953136225ea9f959f94d9257b606973bbad421b
 c9531362]?

 Hiro, do you mind cleaning up the commit messages to make them more
 descriptive? Please let me know if I can help. Also, let me know if
 there's any code that you want me to pay particular attention to.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--
Changes (by gaba):

 * cc: gaba (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:  phw  |Sponsor:  Sponsor19
-+--
Changes (by gaba):

 * reviewer:   => phw


Comment:

 Adding phw to review this code.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:   |Sponsor:  Sponsor19
-+--

Comment (by hiro):

 Current branch:
 https://gitweb.torproject.org/user/hiro/gettor.git/tree/?h=refactoring
 Also we should review the list of people currently able to contribute to
 gettor. At the moment this list includes:
 arma ilv sukhbir

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+--
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  needs_review
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:   |Sponsor:  Sponsor19
-+--
Changes (by gaba):

 * status:  assigned => needs_review


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+---
 Reporter:  ilv  |  Owner:  hiro
 Type:  enhancement  | Status:  assigned
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:   |Sponsor:  Sponsor19
-+---
Changes (by gaba):

 * status:  new => assigned
 * cc: hiro (removed)
 * owner:  ilv => hiro


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+---
 Reporter:  ilv  |  Owner:  ilv
 Type:  enhancement  | Status:  new
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:  gettor-roadmap   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:   |Sponsor:  Sponsor19
-+---
Changes (by gaba):

 * keywords:   => gettor-roadmap
 * sponsor:   => Sponsor19


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+-
 Reporter:  ilv  |  Owner:  ilv
 Type:  enhancement  | Status:  new
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by pili):

 * cc: hiro (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+-
 Reporter:  ilv  |  Owner:  ilv
 Type:  enhancement  | Status:  new
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:   |  Actual Points:
Parent ID:  #28232   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by hiro):

 Hi I have been looking at the code of gettor and bridge db. I am thinking
 many of the functionalities implemented in twistd in bridgedb could be
 easily ported.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+-
 Reporter:  ilv  |  Owner:  ilv
 Type:  enhancement  | Status:  new
 Priority:  Very High|  Milestone:
Component:  Applications/GetTor  |Version:
 Severity:  Major| Resolution:
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by traumschule):

 See also #28091

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #28152 [Applications/GetTor]: Gettor code refactor with Python Twisted

[Tor Bug Tracker & Wiki]

#28152: Gettor code refactor with Python Twisted
-+-
 Reporter:  ilv  |  Owner:  ilv
 Type:  enhancement  | Status:  new
 Priority:  Very High|  Component:  Applications/GetTor
  Version:   |   Severity:  Major
 Keywords:   |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
 Code refactor

 Gettor needs some love. It should be more robust to make it: easier to
 maintain (by me or somebody else), to know when it is working or not, and
 to allow more developers to contribute to it.

 For the above, I propose to refactor the current code and turn it into a
 twisted daemon [1, 2]. This would preserve the main logic of the current
 system and add all the benefits of having a daemonized application. This
 service approach considers two main parts:


1. Distribution channels. Whenever gettor receives a request or sends a
 reply it uses a channel (e.g. e-mail). Each channel could be handled by
 one or more services. These services would be constantly fetching and
 updating information in a SQLite database to know how to proceed.

In the case of e-mail, there should be a script that receives messages
 forwarded by the MTA, process them, and add a request with a given status
 to the SQLite database. On the other hand, a service running on background
 will be fetching ready-to-be-sent requests from the database and send
 e-mails with the requested information.

For a twitter bot, a single service that receives DMs, process them and
 send replies would be enough.

2. Tor Browser sync. A service constantly checking new Tor Browser
 releases, downloading the new packages and updating the SQLite database
 with the new links.

 The logging system provided by twistd is easy to use and works very well.
 This will solve one of the problems with the current code and the use of
 logging, also providing useful information for debugging and statistics.
 Log rotation is automatic.

 I have developed a similar service using twistd. Adapting it to gettor
 would be fairly easy and it would take me a few weeks of spare time.

 Twisted is not installed on getulum, so I will collect all the needed
 packages and ask for them to be installed.

 1:
 https://twistedmatrix.com/documents/current/core/howto/application.html.
 2:
 https://twistedmatrix.com/documents/current/core/howto/basics.html#twistd

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs