Re: [Twisted-Python] Release questions

[Tristan Seligmann]

On Wed, Apr 3, 2013 at 11:58 PM, Laurens Van Houtven <_...@lvh.cc> wrote:

> Is the accidental corruption thing a real risk? I thought that was the
> point of, say, TCP checksums :) Perhaps I'm just mistaken as to how often
> his happens in the wild...
>

TCP checksums don't protect you against corruption occurring on the server
or client, only corruption by intermediary routers (and if they recalculate
the checksum, not even that). In practice, the most common form of
accidental corruption I've encountered is truncation; many HTTP user agents
don't report any kind of error when fewer bytes are received from the
server (due to prematurely terminated HTTP connection, bad proxy server,
etc.) than expected.
-- 
mithrandi, i Ainil en-Balandor, a faer Ambar
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release questions

[Glyph]

On Apr 3, 2013, at 3:23 PM, Laurens Van Houtven <_...@lvh.cc> wrote:

> On Thu, Apr 4, 2013 at 12:04 AM, Glyph  wrote:
> The release manager already _does_ sign something.  Since PyCon, we do have 
> much better trust web integration, which is great, but that's not really 
> relevant to this discussion, which is just about changing what we sign and 
> how it gets signed.
> 
> Yes, sorry; I thought there were just a bunch of hashes in a file somewhere, 
> and forgot they were signed.
>  
> I think we should carry on with signing the list of signatures for now, and 
> just upgrade the hash algorithm.  Baby steps.  Perhaps there are some 
> theoretical benefits that come from signing the whole binary blob, but that's 
> a much bigger change for a much smaller benefit.
> 
> If anyone does have an interest in us doing this, I think the first step 
> would be to write up a clear explanation of how it should be done.
> 
> I agree. Can't we just replace "md5sum" in the command line with "shasum -a 
> 512"? Do we need a grace period where we deliver both the MD5 and SHA512 
> sums? (Perhaps there's an automated system out there that relies on the MD5 
> version being available, since that's all we have now.)

Signing both certainly doesn't create any problems.  I just have no idea what 
automation stuff parses this file as part of our _own_ release process.  I was 
hoping someone who knew how it worked could examine it and tell me.

-glyph



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release questions

[Laurens Van Houtven]

On Thu, Apr 4, 2013 at 12:04 AM, Glyph  wrote:

> The release manager already _does_ sign something.  Since PyCon, we do
> have much better trust web integration, which is great, but that's not
> really relevant to this discussion, which is just about changing what we
> sign and how it gets signed.
>

Yes, sorry; I thought there were just a bunch of hashes in a file
somewhere, and forgot they were signed.


> I think we should carry on with signing the list of signatures for now,
> and just upgrade the hash algorithm.  Baby steps.  Perhaps there are some
> theoretical benefits that come from signing the whole binary blob, but
> that's a much bigger change for a much smaller benefit.
>
> If anyone does have an interest in us doing this, I think the first step
> would be to write up a clear explanation of how it should be done.
>

I agree. Can't we just replace "md5sum" in the command line with "shasum -a
512"? Do we need a grace period where we deliver both the MD5 and SHA512
sums? (Perhaps there's an automated system out there that relies on the MD5
version being available, since that's all we have now.)


> -glyph
>

-- 
cheers
lvh
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release questions

[Glyph]

On Apr 3, 2013, at 1:51 PM, exar...@twistedmatrix.com wrote:

> On 04:36 pm, _...@lvh.cc wrote:
>> On Wed, Apr 3, 2013 at 6:14 PM, Thomas Hervé  wrote:
>>> * Glyph mumbled something about sha sums of the release files, 
>>> instead
>>> of md5. Should we pursue that? We may need to update some trac
>>> integration code.
>> 
>> Depends, what's the goal of the checksums? If it's "we want people to 
>> be
>> able to check that the tarball they have is in fact the release and not
>> something tainted by patches or malware", perhaps we either should have 
>> a
>> Twisted signing key, or have the release manager sign the release 
>> instead
>> (especially since we have a lot of signatures since PyCon :)).

The release manager already _does_ sign something.  Since PyCon, we do have 
much better trust web integration, which is great, but that's not really 
relevant to this discussion, which is just about changing what we sign and how 
it gets signed.

> The question relates to step 4 beneath "Cut the tarballs & installers":
> 
> http://twistedmatrix.com/trac/wiki/ReleaseProcess#Cutthetarballsinstallers
> 
> The checksums are intended to let people verify their download was 
> neither accidentally corrupted nor intentionally tampered with.
> 
> I think the original motivation for signing some checksums instead of 
> signing the release artifacts was something like:
> 
>  * gpg is a pain to use, signing one thing is nicer than signing 30 
> things
>  * lots of people do not care about cryptographic concerns here, and the 
> checksum is good enough for them
> 
> Generating and signing a single document containing checksums of all the 
> files is less work for the release manager and offers both possible 
> audiences some value.
> 
> Perhaps it's a round-about way to achieve those goals, though.  Is there 
> something simpler that we could do that wouldn't make releases harder or 
> kick sand in the eyes of people just trying to make sure their ethernet 
> card didn't hiccup?

Security-wise, signing an actually-secure hash is not that much different than 
signing the tarballs themselves.  Signing MD5 hashes, on the other hand, is 
useless as a security measure.

I think we should carry on with signing the list of signatures for now, and 
just upgrade the hash algorithm.  Baby steps.  Perhaps there are some 
theoretical benefits that come from signing the whole binary blob, but that's a 
much bigger change for a much smaller benefit.

If anyone does have an interest in us doing this, I think the first step would 
be to write up a clear explanation of how it should be done.

-glyph


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 13.0.0 released

[Laurens Van Houtven]

Yay! Thanks, Thomas and everyone who contributed to this release :)


On Wed, Apr 3, 2013 at 1:11 PM, Thomas Hervé  wrote:

> On behalf of Twisted Matrix Laboratories, I am pleased to announce the
> release of Twisted 13.0.
>
> Among the 70 tickets closed, we can see:
>
>  * A new "Introduction to Deferreds" document that you can find here:
> http://twistedmatrix.com/documents/13.0.0/core/howto/defer-intro.html
>
>  * A fix in twisted.web.template where attributes were not quoted
> properly, risking HTML injection.
>
>  * Support for unicode domain names in twisted.names SRVConnector and
> Name classes, after a 12.3 regression.
>
>  * A workaround for platform limitations when trying to schedule events
> far in the future.
>
>
> For more information, see the NEWS file here:
>
>  http://twistedmatrix.com/Releases/Twisted/13.0/NEWS.txt
>
>
> Download it now from:
>
>  http://pypi.python.org/packages/source/T/Twisted/Twisted-13.0.0.tar.bz2or
>
>
> http://pypi.python.org/packages/2.7/T/Twisted/Twisted-13.0.0.win32-py2.7.msi
>
>
> Thanks to the supporters of Twisted via the Software Freedom Conservancy
> and to the many contributors for this release.
>
> --
> Thomas
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>



-- 
cheers
lvh
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release questions

[Laurens Van Houtven]

On Wed, Apr 3, 2013 at 10:51 PM,  wrote:

> The question relates to step 4 beneath "Cut the tarballs & installers":
>
> http://twistedmatrix.com/trac/wiki/ReleaseProcess#Cutthetarballsinstallers
>
> The checksums are intended to let people verify their download was
> neither accidentally corrupted nor intentionally tampered with.
>

Is the accidental corruption thing a real risk? I thought that was the
point of, say, TCP checksums :) Perhaps I'm just mistaken as to how often
his happens in the wild...


> I think the original motivation for signing some checksums instead of
> signing the release artifacts was something like:
>
>   * gpg is a pain to use, signing one thing is nicer than signing 30
> things
>   * lots of people do not care about cryptographic concerns here, and the
> checksum is good enough for them
>

Okay, fair enough. I'm a little worried about the "I don't care about the
cryptography" part, if a user is consciously choosing that, fine; but what
if they think they're doing something (verifying the integrity of the
Twisted release) when in fact not doing that at all? Perhaps that's even
rarer than the accidental corruption thing I so quickly dismissed just now,
though ;-)

As for gpg being a pain to use, `ls | xargs -n 1 gpg --sign` seems to work
for me provided you have gpg-agent (and have it configured to not need a
signature every time). Is gpg-agent something we don't want to require from
release managers?

Generating and signing a single document containing checksums of all the
> files is less work for the release manager and offers both possible
> audiences some value.
>
> Perhaps it's a round-about way to achieve those goals, though.  Is there
> something simpler that we could do that wouldn't make releases harder or
> kick sand in the eyes of people just trying to make sure their ethernet
> card didn't hiccup?
>

Probably not, the current thing seems pretty easy, right? If I understand
correctly, the only complaint is that "MD5 sucks". So if we upgrade that to
SHA-256/512 (SHA-3 would be nice, but plenty of people don't have access to
it yet on the command line...), that'd do it.

I don't think there is anything wrong with a hash sum file, I'm just
concerned that the reasons for *not* having or verifying signatures might
not be that great.


> Jean-Paul
>

-- 
cheers
lvh
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release questions

[Glyph]

On Apr 3, 2013, at 9:14 AM, Thomas Hervé  wrote:

> Hey everyone,
> 
> During the latest release process, I was left with several things to
> clarify, so now that it's done I think it's time:
> 
> * We started building wheels for Windows. What do we do with me, should
> we upload it to pypi? What happens when users try them?

We should definitely upload them to PyPI.  Right now, users won't even get them 
unless they explicitly say "pip --use-wheel", so there's not much risk 
associated with it.  When users try them, they should just get a working 
installation instead of what happens now ;).

> * Glyph mumbled something about sha sums of the release files, instead
> of md5. Should we pursue that? We may need to update some trac
> integration code.

We should.  If we're signing them, it's for security.  If it's for security, we 
should use a secure hash.  Let's skip sha1 and go straight to sha512 or 
something.

> * There are several issues with SVN and the release branch. First the
> Quotes file conflicts almost all the time. But this time we also
> reverted a branch from trunk during the release, and then merged it
> again, which created all sorts of troubles. Maybe we (I) just need to
> write some documentation about it. But if anyone has an idea on how to
> improve our tools, they are welcome.

I think that as long as we're still using an old version of SVN, documentation 
is the only solution.  Also we should have fewer bugs that need reverting :).

-glyph


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Current callback best practices

[Glyph]

On Apr 3, 2013, at 4:25 AM, Hynek Schlawack  wrote:

> So you’re cool with calling callbacks after the event that triggered? That 
> would’ve been my second question but I didn’t want it to divert the 
> discussion. I kind of like `onData` et al too but the Twisted examples rather 
> shun it.

The examples are not always the best examples of every aspect of style :).  
Perhaps people don't like 'on' because it reminds them of JavaScript too much.  
(Not me, though, it reminds *me* of SuperCard…)

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 13.0.0 released

[Glyph]

On Apr 3, 2013, at 4:11 AM, Thomas Hervé  wrote:

> On behalf of Twisted Matrix Laboratories, I am pleased to announce the
> release of Twisted 13.0.

Thank you Thomas for managing yet another excellent release!

> Among the 70 tickets closed, we can see:
> 
> * A new "Introduction to Deferreds" document that you can find here:
> http://twistedmatrix.com/documents/13.0.0/core/howto/defer-intro.html
> 
> * A fix in twisted.web.template where attributes were not quoted
> properly, risking HTML injection.
> 
> * Support for unicode domain names in twisted.names SRVConnector and
> Name classes, after a 12.3 regression.
> 
> * A workaround for platform limitations when trying to schedule events
> far in the future.

This is a great list.  And we have even more great stuff in the pipeline.  
Everybody get to work on 13.1 ;-).

-glyph



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release questions

[exarkun]

On 04:36 pm, _...@lvh.cc wrote:
>On Wed, Apr 3, 2013 at 6:14 PM, Thomas Hervé  wrote:
>>  * Glyph mumbled something about sha sums of the release files, 
>>instead
>>of md5. Should we pursue that? We may need to update some trac
>>integration code.
>
>Depends, what's the goal of the checksums? If it's "we want people to 
>be
>able to check that the tarball they have is in fact the release and not
>something tainted by patches or malware", perhaps we either should have 
>a
>Twisted signing key, or have the release manager sign the release 
>instead
>(especially since we have a lot of signatures since PyCon :)).

The question relates to step 4 beneath "Cut the tarballs & installers":

http://twistedmatrix.com/trac/wiki/ReleaseProcess#Cutthetarballsinstallers

The checksums are intended to let people verify their download was 
neither accidentally corrupted nor intentionally tampered with.

I think the original motivation for signing some checksums instead of 
signing the release artifacts was something like:

  * gpg is a pain to use, signing one thing is nicer than signing 30 
things
  * lots of people do not care about cryptographic concerns here, and the 
checksum is good enough for them

Generating and signing a single document containing checksums of all the 
files is less work for the release manager and offers both possible 
audiences some value.

Perhaps it's a round-about way to achieve those goals, though.  Is there 
something simpler that we could do that wouldn't make releases harder or 
kick sand in the eyes of people just trying to make sure their ethernet 
card didn't hiccup?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release questions

[byr sa]

On Wed, Apr 3, 2013 at 9:14 AM, Thomas Hervé  wrote:

>  * Glyph mumbled something about sha sums of the release files, instead
> of md5. Should we pursue that? We may need to update some trac
> integration code.
>
>
+1 for SHA-256 or SHA-512. High profile collision attacks against MD5 have
happened (google: flame md5 collision). People are working to crack SHA1
right now, get ahead of the curve!

byrsa
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release questions

[Laurens Van Houtven]

On Wed, Apr 3, 2013 at 6:14 PM, Thomas Hervé  wrote:

>  * Glyph mumbled something about sha sums of the release files, instead
> of md5. Should we pursue that? We may need to update some trac
> integration code.
>

Depends, what's the goal of the checksums? If it's "we want people to be
able to check that the tarball they have is in fact the release and not
something tainted by patches or malware", perhaps we either should have a
Twisted signing key, or have the release manager sign the release instead
(especially since we have a lot of signatures since PyCon :)).

-- 
cheers
lvh
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Release questions

[Thomas Hervé]

Hey everyone,

During the latest release process, I was left with several things to
clarify, so now that it's done I think it's time:

 * We started building wheels for Windows. What do we do with me, should
we upload it to pypi? What happens when users try them?

 * Glyph mumbled something about sha sums of the release files, instead
of md5. Should we pursue that? We may need to update some trac
integration code.

 * There are several issues with SVN and the release branch. First the
Quotes file conflicts almost all the time. But this time we also
reverted a branch from trunk during the release, and then merged it
again, which created all sorts of troubles. Maybe we (I) just need to
write some documentation about it. But if anyone has an idea on how to
improve our tools, they are welcome.

Thanks!

-- 
Thomas

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.cred interface deficiences

[Kevin Horn]

On Tue, Apr 2, 2013 at 6:04 PM, Glyph  wrote:

>
> On Apr 1, 2013, at 2:34 PM, Shell  wrote:
>
> I propose that IUsernamePassword should be split into at least two
> interfaces:
>
> * IUsernamePassword, with only username and password, no methods,
> which allows password to be used in any way
> * Another interface, which only defines username and checkPassword() -
> possibly just a rename of IUsernameHashedPassword, which declares a
> similar interface
>
> However, this has the issue that any credential checker which can use
> the second interface would also be able to use an IUsernamePassword
> here if there were an adapter between the two, but support for this
> would have to go into every credential checker which supports the
> second interface at present. Maybe the Portal could automatically
> search for adapters if it can't find a direct match?
>
>
> These don't sound like bad ideas, but I think that if you're going to try
> to fix cred's built-in credentials, this is a pretty labor-intensive and
> not particularly rewarding path to go down.  Further refining the sad
> username+password credential interface will provide only some internal
> factoring improvements to existing types of authentication, at the cost of
> retrofitting all of them; not to mention dealing with broad-spectrum
> deprecations.
>
> A better use of energy would be directed at getting a generic SASL
> credentials implementation; in other words, fixing this fairly ancient
> ticket: .
>
>
For what it's worth, I worked on this back when, and while I don't have
time to work on it right now (and probably won't for at least a couple of
weeks), I'm happy to answer questions, if anybody has them.

Mostly what needed to happen was to merge the code in the branch with the
code in trunk which had diverged somewhat, and create some documentation on
how to use them.

My recollection was that integrating the results of that with specific
protocols (and possibly with Cred using a "higher level" interface) would
happen on a subsequent ticket, which is probably a good idea.


> A well-implemented SASL credentials interface will address these issues,
> as well as allowing for proper challenge-response authentication, digest
> auth, external auth mechanisms like TLS, et cetera.
>
> Your idea about adapters is well taken though; having the portal do that
> when SASL-ified checkers are available seems reasonable, provided that it
> won't break anything.
>
> -glyph
>
>

--
Kevin Horn
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Current callback best practices

[Hynek Schlawack]

Am 02.04.2013 um 16:32 schrieb Glyph :

>> My question can be simplified to: Closures yes or no?
> "As appropriate".

I had that coming. :)

[…]

Thanks for your verbose explanations!

>> If “yes, closures”: Still using cb/eb prefixes? I don’t see them very often 
>> in recent examples. What about addBoth handlers?
> If you're using a closure, use a descriptive name.  The 'cb/eb' prefix 
> notation is not particularly illustrative; I often use an english word like 
> 'when'.  whenDisconnected, whenResponseReceived, etc.

So you’re cool with calling callbacks after the event that triggered? That 
would’ve been my second question but I didn’t want it to divert the discussion. 
I kind of like `onData` et al too but the Twisted examples rather shun it.

Thanks,
Hynek

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Twisted 13.0.0 released

[Thomas Hervé]

On behalf of Twisted Matrix Laboratories, I am pleased to announce the
release of Twisted 13.0.

Among the 70 tickets closed, we can see:

 * A new "Introduction to Deferreds" document that you can find here:
http://twistedmatrix.com/documents/13.0.0/core/howto/defer-intro.html

 * A fix in twisted.web.template where attributes were not quoted
properly, risking HTML injection.

 * Support for unicode domain names in twisted.names SRVConnector and
Name classes, after a 12.3 regression.

 * A workaround for platform limitations when trying to schedule events
far in the future.


For more information, see the NEWS file here:

 http://twistedmatrix.com/Releases/Twisted/13.0/NEWS.txt


Download it now from:

 http://pypi.python.org/packages/source/T/Twisted/Twisted-13.0.0.tar.bz2 or

 http://pypi.python.org/packages/2.7/T/Twisted/Twisted-13.0.0.win32-py2.7.msi


Thanks to the supporters of Twisted via the Software Freedom Conservancy
and to the many contributors for this release.

-- 
Thomas

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] How to determine whether transport.write() succeed?

[Todong Ma]

Ok, will try AMP

thank you very much!


On Mon, Apr 1, 2013 at 3:49 PM, Glyph  wrote:

>
> On Mar 31, 2013, at 6:09 PM, Todong Ma  wrote:
>
> I wrote a simple client and server to get system information (e.g. cpu,
> memory, etc.)
> Client send 'cpu' to the server, then server will return cpu's information
> to client.
>
> Now client send multiple commands to server one time, e.g. cpu, memory,
> database, hard disk. But sometimes server couldn't receive some type
> information (e.g. cpu, database), and the missed information type is not
> fixed every time.
>
> So I want client to check whether ITransport.write() sends message to
> server successfully. If failed, client will resend the message to server.
>
>
> It's a good thing that write() doesn't convey information about send()
> succeeding so you can't trust it :).  You can't rely on send() succeeding
> to convey this information: send() can succeed but all that means is that
> data moved from your application into your kernel.
>
> What you want to do is to have an application-level acknowledgement of the
> data that was sent.
>
> AMP will provide this for you automatically; when the Deferred returned by
> callRemote fires, the other end has received and acknowledged the
> information.
>
> -glyph
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python