RE: General question about documentation

2009-12-02 Thread Mark
 

 The problem is that the documentation may not be correct, 
 sending your coders on a wild goose chase.

Anything may contain errors.  I don't think this is a valid reason for
not
doing it.

 Think of the source code as a safe but boring investment 
 (with little barrier to entry), and the documentation as one 
 with high potential yield (with a high barrier to entry) but 
 significant risk of errors.

Reverse engineering from source code is extremely time consuming and
prone
to errors too.  Given the choice I would always prefer documentation to
reverse engineering. BTDTGTTS.

 Now, go balance your software portfolio for whatever 
 risk/reward ratio you find optimal.
  
 Remember, many code for their own benefit, but documentation 
 benefits others (mostly), so it's hard to encourage high 
 quality documentation without paying for it.

I'd say documentation benefits everyone.  How to the developers know
what
to implement if there is no documentation?  What do they test the code 
against?  etc

Mark.

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-12-02 Thread Mark H. Wood
On Tue, Dec 01, 2009 at 03:23:15PM -0800, Rene Hollan wrote:
 The problem is that the documentation may not be correct, sending your coders 
 on a wild goose chase.

Bah, if the code does not do what the documentation describes then the
*code* is incorrect.  Documentation can only be incorrect if it does
not model the problem that the code is intended to solve.

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Friends don't let friends publish revisable-form documents.


pgpKR3QEobidk.pgp
Description: PGP signature


Re: General question about documentation

2009-12-02 Thread Mark H. Wood
On Tue, Dec 01, 2009 at 02:08:08PM -0800, Randy Turner wrote:
 As an investor, I would rather have my coders use a product with
 documentation to make progress on the actual goals of the product,
 rather than reverse-engineer the information they're trying to look
 for.

 With the former method, my cost is (n), with the latter method, my
 cost could be unbounded, depending upon how complex the source code
 is (i.e., explicit code, or 14 levels of indirection and C macros
 that have to be understood).

 It sounds like you're making the case for documentation to meand
 I agree.

Hear, hear!  I've done such reverse-engineering.  I once disassembled
the compiled code (all there was on the Unsupported tape) for the
TOPS-20 Programmable Command Language; edited it (through many
iterations) into clean, idiomatic, well-commented MACRO-10; studied
how it hooked into the EXEC; and from the coments wrote a user's
manual, so I could figure out what the heck it did.  I'm quite proud
of my work.  I never, EVER want to do anything like that again.  It
was lengthy and exhausting and ultimately unsatisfactory.

Along the way I learned all the way down to my toes how little
information is conveyed by code about what the designer was thinking
or how he expected his design to be used.  That's why, in a commercial
OS, right next to each Reference Manual there is a Programmer's Guide
or a User's Guide.

I wish I *could* write some of the Programmer's Guides I have wanted
over the years but, obviously, the person who needs one is the person
least able to write one.  My PCL manual, proud as I am of it, was a
botch, much too short and incomplete.  I simply wasn't able to glean
enough information from the source to write properly.  The designer
knows things the rest of us do not, and it is precisely that knowledge
which gives documentation much of its value.

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Friends don't let friends publish revisable-form documents.


pgpwKJpF5MXBS.pgp
Description: PGP signature


Re: General question about documentation

2009-12-02 Thread Randy Turner

Great to hear from another former TOPS-20 userI worked on TOPS back in the 
early 80s, then VMS of course.
Also reverse-engineered (to some degree, more like reverse-compiled) PDP-8 
paper tape.  All in all, I'll take the docs. :)

Randy

On Dec 2, 2009, at 6:42 AM, Mark H. Wood wrote:

 On Tue, Dec 01, 2009 at 02:08:08PM -0800, Randy Turner wrote:
 As an investor, I would rather have my coders use a product with
 documentation to make progress on the actual goals of the product,
 rather than reverse-engineer the information they're trying to look
 for.
 
 With the former method, my cost is (n), with the latter method, my
 cost could be unbounded, depending upon how complex the source code
 is (i.e., explicit code, or 14 levels of indirection and C macros
 that have to be understood).
 
 It sounds like you're making the case for documentation to meand
 I agree.
 
 Hear, hear!  I've done such reverse-engineering.  I once disassembled
 the compiled code (all there was on the Unsupported tape) for the
 TOPS-20 Programmable Command Language; edited it (through many
 iterations) into clean, idiomatic, well-commented MACRO-10; studied
 how it hooked into the EXEC; and from the coments wrote a user's
 manual, so I could figure out what the heck it did.  I'm quite proud
 of my work.  I never, EVER want to do anything like that again.  It
 was lengthy and exhausting and ultimately unsatisfactory.
 
 Along the way I learned all the way down to my toes how little
 information is conveyed by code about what the designer was thinking
 or how he expected his design to be used.  That's why, in a commercial
 OS, right next to each Reference Manual there is a Programmer's Guide
 or a User's Guide.
 
 I wish I *could* write some of the Programmer's Guides I have wanted
 over the years but, obviously, the person who needs one is the person
 least able to write one.  My PCL manual, proud as I am of it, was a
 botch, much too short and incomplete.  I simply wasn't able to glean
 enough information from the source to write properly.  The designer
 knows things the rest of us do not, and it is precisely that knowledge
 which gives documentation much of its value.
 
 -- 
 Mark H. Wood, Lead System Programmer   mw...@iupui.edu
 Friends don't let friends publish revisable-form documents.

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-12-02 Thread Rene Hollan
Not quite. The docs may not indicate whether a returned reference is 
dynamically allocated, statically allocated, reference counted, how to 
deallocate, etc. The API will still be correct as far as a reference being 
returned, but the documentation will be incomplete.

Often the library implementer has to make a design decision to fill a gap that 
the documentation leaves.

It is therefore possible for the documentation to correctly describe the API, 
but not to a sufficient degree of completeness for other code to use it 
correctly and effectively.

Having the code lets one determine WHAT IT DOES, unambiguously. It may be 
possible to use it in that state to do what YOU WANT, or it may have an 
implementation error. But it can't have an implementation error if the 
documentation does not cover that detail.

Correct documentation is not enough. Complete documentation is necessary, but 
often an unrealistic goal, especially when code is still in flux (and library 
implementation details that affect library use have not been firmed up).

Do you prefer incomplete but correct documentation, or complete but possibly 
incorrect documentation? Most open source projects strive for correctness over 
completeness -- likely because the missing bits can be determined through 
review of the code as to those aspects without having to review it completely 
to reverse engineer any docs.

Then there is the issue of the code following an RFC but the docs not matching 
it. Should the code follow the standard or the docs? Which is wrong? (My bet is 
on the docs).

The BIGGEST issue I have with openssl is when I get a pointer to some 
subordinate piece of information managed by, say, an X509_foo. I call some 
function taking an X509_foo* and get an X509_bar*. O.K. What do I do with that 
X509_bar* when I no longer need it? X509_bar_free it? Dunno -- maybe it isn't 
reference counted and it's lifetime is bounded by the lifetime of X509_foo, and 
all I need to do is lose my pointer to it (let it go out of function scope when 
the function returns). If I X509_bar_free it, and it isn't reference counted, 
or actually part of the internals of the X509_foo object, I've done a bad 
thing. Here I see the docs as lacking a description of the GLOBAL approach to 
dynamically allocated objects, while I do see local hints about some of them.

Reading code is not has hard as some think -- you eventually get the idea of 
what s/he was thinking. That's not to say it's easy for everyone. But it sure 
beats following docs and finding things don't work because the docs don't cover 
some aspect one was forced to assume and was implemented differently.




-Original Message-
From: owner-openssl-us...@openssl.org on behalf of Mark H. Wood
Sent: Wed 12/2/2009 6:47 AM
To: openssl-users@openssl.org
Subject: Re: General question about documentation
 
On Tue, Dec 01, 2009 at 03:23:15PM -0800, Rene Hollan wrote:
 The problem is that the documentation may not be correct, sending your coders 
 on a wild goose chase.

Bah, if the code does not do what the documentation describes then the
*code* is incorrect.  Documentation can only be incorrect if it does
not model the problem that the code is intended to solve.

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Friends don't let friends publish revisable-form documents.



RE: General question about documentation

2009-12-02 Thread Rene Hollan

To someone who uses code, it doesn't matter a fig what the designer was 
thinking. It matter what the code does. Then you can decide if it does 
something correctly enough to be usable in the state it's in.

Now, if what you're trying to do is distill some overall global design details 
from patterns you find in the code, you may or may not get lucky -- there may 
simply not be any, and the code was written in an ad hoc manner to solve a 
particular limited problem and not intended for more general use.

The fallacy here is thinking that if the code does not follow the docs (if 
any), our invester can simply blame the code for being wrong. Doesn't get 
your program working, does it? Might give you an idea who to sue for 
damanges, but unless you have pockets deep enough to mount a suit and they 
have assets enough to satisfy you, you are out of luck.

Often the designer, programmer, and documentation writer are one and the same 
person, and the proper global design (and thus documentation) goals are not 
clear until some attempt at implementation. Agile development trumps waterfall 
design because you can never be perfectly right at each stage. Unfortunately, 
it makes documentation somewhat of a catch up activity. (Ideally, the docs 
are always correct but not necessarily complete, and completeness is achieved 
through global patterns rather than local exceptions thus reflecting the 
refactoring effort). More likely, it results in a series of design, 
implementation and REFACTOR. Do you keep your docs in sync at every step of the 
way, or do you wait until some semblance of stability has appeared in the 
refactor cycle?

-Original Message-
From: owner-openssl-us...@openssl.org on behalf of Mark H. Wood
Sent: Wed 12/2/2009 6:42 AM
To: openssl-users@openssl.org
Subject: Re: General question about documentation
 
On Tue, Dec 01, 2009 at 02:08:08PM -0800, Randy Turner wrote:
 As an investor, I would rather have my coders use a product with
 documentation to make progress on the actual goals of the product,
 rather than reverse-engineer the information they're trying to look
 for.

 With the former method, my cost is (n), with the latter method, my
 cost could be unbounded, depending upon how complex the source code
 is (i.e., explicit code, or 14 levels of indirection and C macros
 that have to be understood).

 It sounds like you're making the case for documentation to meand
 I agree.

Hear, hear!  I've done such reverse-engineering.  I once disassembled
the compiled code (all there was on the Unsupported tape) for the
TOPS-20 Programmable Command Language; edited it (through many
iterations) into clean, idiomatic, well-commented MACRO-10; studied
how it hooked into the EXEC; and from the coments wrote a user's
manual, so I could figure out what the heck it did.  I'm quite proud
of my work.  I never, EVER want to do anything like that again.  It
was lengthy and exhausting and ultimately unsatisfactory.

Along the way I learned all the way down to my toes how little
information is conveyed by code about what the designer was thinking
or how he expected his design to be used.  That's why, in a commercial
OS, right next to each Reference Manual there is a Programmer's Guide
or a User's Guide.

I wish I *could* write some of the Programmer's Guides I have wanted
over the years but, obviously, the person who needs one is the person
least able to write one.  My PCL manual, proud as I am of it, was a
botch, much too short and incomplete.  I simply wasn't able to glean
enough information from the source to write properly.  The designer
knows things the rest of us do not, and it is precisely that knowledge
which gives documentation much of its value.

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Friends don't let friends publish revisable-form documents.



Re: General question about documentation

2009-12-02 Thread Victor Duchovni
On Wed, Dec 02, 2009 at 11:17:44AM -0800, Rene Hollan wrote:

 
 To someone who uses code, it doesn't matter a fig what the designer was 
 thinking. It matter what the code does. Then you can decide if it does 
 something correctly enough to be usable in the state it's in.
 

My sense is that this thread is no longer productive. Perhaps we can stop.

FWIW, the Postfix project strives to avoid documentation gaps, by not
accepting code contributions that are not *fully* documented at the time
the code is contributed. The first step towards improving OpenSSL docs,
would be to raise the bar for contributed code, and on-going development.
If code is not documented, it should not move from dev branches to the
trunk, and definitely not from the trunk to release branches.

Once on-going documentation stops being optional, one can start thinking
about past sins.

I don't want to add fuel to the fire, this is my final comment in this
thread. Sorry to prolong it any further...

-- 
Viktor.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-12-02 Thread Rene Hollan
That works for *future* documentation for *new* code, but it doesn't address 
the existing gaps.

Perhaps examining where the existing gaps are biggest would be productive.

For me, two areas have always been confusing:

1) What objects are dynamically allocated, appropriately reference counted, and 
have associated pointer destructors intended to be use on every reference no 
longer needed? IOW, If I get an X509_FOO*, should I ALWAYS call X509_FOO_free 
on it, for ALL FOOs?

2) Things like OCSP, CRLs, and other SSL extensions have always stumped me. 
Is it something the user of the library is responsible for, when validating a 
cert, or can the library do it itself when I try to establish an SSL 
connection, and to what degree can I control this? It always struck me that the 
cert validation callback is the place to do this, but I take heed of the fact 
that openssl is supposed to have a reasonable default cert validation routine 
and not to replace it without good reason.

3) Better documentation regarding BIO chaining and BIOs in general. BIOs are 
cool, they remind me a bit of System V Streams.

In general, I've seen good docs regarding the trees of individual functions, 
fair docs regarding type name and function patterns (e.g. regarding ASN1 to 
internal conversion), and poor docs of the various forests of types of 
functions and their interaction (CRL support, OCSP support, TLS1 Hello 
extensions, etc.)



-Original Message-
From: owner-openssl-us...@openssl.org on behalf of Victor Duchovni
Sent: Wed 12/2/2009 11:29 AM
To: openssl-users@openssl.org
Subject: Re: General question about documentation
 
On Wed, Dec 02, 2009 at 11:17:44AM -0800, Rene Hollan wrote:

 
 To someone who uses code, it doesn't matter a fig what the designer was 
 thinking. It matter what the code does. Then you can decide if it does 
 something correctly enough to be usable in the state it's in.
 

My sense is that this thread is no longer productive. Perhaps we can stop.

FWIW, the Postfix project strives to avoid documentation gaps, by not
accepting code contributions that are not *fully* documented at the time
the code is contributed. The first step towards improving OpenSSL docs,
would be to raise the bar for contributed code, and on-going development.
If code is not documented, it should not move from dev branches to the
trunk, and definitely not from the trunk to release branches.

Once on-going documentation stops being optional, one can start thinking
about past sins.

I don't want to add fuel to the fire, this is my final comment in this
thread. Sorry to prolong it any further...

-- 
Viktor.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org

winmail.dat

Re: General question about documentation

2009-12-02 Thread Patrick Patterson
Hi Rene:

Rene Hollan wrote:
 
 2) Things like OCSP, CRLs, and other SSL extensions have always
 stumped me. Is it something the user of the library is responsible
 for, when validating a cert, or can the library do it itself when I
 try to establish an SSL connection, and to what degree can I control
 this? It always struck me that the cert validation callback is the
 place to do this, but I take heed of the fact that openssl is
 supposed to have a reasonable default cert validation routine and not
 to replace it without good reason.
 
This really has nothing to do with OpenSSL - or at least, only
tangentially - If you don't understand the basics of how the IETF and
ITU have defined X.509 and the Internet profiles, then no amount of
OpenSSL documentation is going to save you.

If you read the specifications, you can very easily see that not only
does OpenSSL not do every kind of validation under the sun, you probably
wouldn't want it to (do you really want OpenSSL implementing a full HTTP
stack in the crypto library just so that it can chase AIA, OCSP and
CRLDP?) - So until one has a fuller understanding of exactly how the
X.509 path discovery and validation process works, then it is very
difficult to see where OpenSSL should fit in this. The current
documentation, I think, for X509_verify and friends is, while not
perfect, certainly adequate for anyone that knows exactly what they are
doing to do the right thing (it was certainly fine for us when writing
Pathfinder and WvStreams).

Please don't confuse OpenSSL needs more documentation, with The IETF
and others should work harder to ensure that there are  better examples
and tutorials out there so that people implement PKI (and by extension
TLS/SSL) properly.

And I agree with Victor - unless someone from the core OpenSSL team
steps up, or someone volunteers to set up some sort of Wiki and pay for
someone to spend a lot of time writing documentation, then I think this
thread has gotten about as far as it can.

Patrick.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-12-02 Thread Rene Hollan
Ah, but this hits at the crux of the openssl docs.

Of course, it is silly to presume a fixed library will handle open-ended 
verification. And, indeed, I handle CRLs and OCSP in my own verify function.

The problem is when I read a vague admonishment to not supply my own 
verification function because the default one probably does the right thing. 
Well, OK, but WHAT exactly is that? So that I can decide if I NEED to do more.

This is compounded when I see openssl functions to manage CRL and OCSP objects 
in ASN1 and internal coded forms.

So, I put two and two together and ask myself, Self, to what degree is there 
support for some kind of 'default' handling of CRL retrieval or OCSP lookup in 
the 'built in' verification function?

I can see arguments for none at all -- openssl does not make TCP connections 
behind your back to well, something arbitrary that we thought was a good 
idea.

What I want to see from the docs is WHICH of these is true.

It looks like the openssl verification function is provided for convenience 
to do the most basic of verification (validation against internally loaded 
trusted certs) and serves as a sample of what to do. In that case, it should be 
in a separate section of code, at a different level, if you will, provided as 
an application convenience or starting point, almost part of an apps 
library rather than openssl proper code.

Now, given that many openssl users will WANT to do CRL checking, or OCSP 
retrieval, (and, heck, if there is a stapled OCSP response, lets have at it in 
a callback), it stands to reason that a convenience library above openssl is 
the next level of abstraction and sophistication.

But, the current docs do not hint at these kind of strata within the existing 
library, or that the user is expected to provide them.

I see a need to break functions up according to

1) object manipulation

2) state manipulation

3) protocol handling

4) higher level validation and verification.

Some of these may not belong in openssl, but if there is a decision of what the 
scope of the project is, it should be made clear.

To provide some perspective, I am using openssl in a proxying configuration, 
generally carrying HTTP traffic. (We decrypt, and reencrypt traffic with certs 
resigned by CAs the downstream client trusts). So, I have to mimic all the 
stuff a browser does: CRL checking, OCSP lookup, etc. It is confusing when, at 
a glance, I see that openssl has CRL support and OCSP support. The question 
that screams in my mind is, O.K. To WHAT degree? What is the reasoning behind 
the SCOPE of support to provide for these higher level security aspects in the 
openssl project? Object manipulation? Callback functions? (TLS extensions in 
Hello messages are nice, but I'd like to see functions to let me set encoded 
versions on Hellos I want to send -- is that in scope or not? Implemented yet, 
or not? If you give me something that is received, I'd like to be able to send 
that same something)

I am not suggesting that this has to be implemented to satisfy my need. But, it 
is not clear from the documentation in it present form what the scope of 
supporting things like this is, and confusing when I see that version such and 
such now supports foo.


-Original Message-
From: owner-openssl-us...@openssl.org on behalf of Patrick Patterson
Sent: Wed 12/2/2009 12:33 PM
To: openssl-users@openssl.org
Subject: Re: General question about documentation
 
Hi Rene:

Rene Hollan wrote:
 
 2) Things like OCSP, CRLs, and other SSL extensions have always
 stumped me. Is it something the user of the library is responsible
 for, when validating a cert, or can the library do it itself when I
 try to establish an SSL connection, and to what degree can I control
 this? It always struck me that the cert validation callback is the
 place to do this, but I take heed of the fact that openssl is
 supposed to have a reasonable default cert validation routine and not
 to replace it without good reason.
 
This really has nothing to do with OpenSSL - or at least, only
tangentially - If you don't understand the basics of how the IETF and
ITU have defined X.509 and the Internet profiles, then no amount of
OpenSSL documentation is going to save you.

If you read the specifications, you can very easily see that not only
does OpenSSL not do every kind of validation under the sun, you probably
wouldn't want it to (do you really want OpenSSL implementing a full HTTP
stack in the crypto library just so that it can chase AIA, OCSP and
CRLDP?) - So until one has a fuller understanding of exactly how the
X.509 path discovery and validation process works, then it is very
difficult to see where OpenSSL should fit in this. The current
documentation, I think, for X509_verify and friends is, while not
perfect, certainly adequate for anyone that knows exactly what they are
doing to do the right thing (it was certainly fine for us when writing
Pathfinder and WvStreams).

Please don't confuse OpenSSL

Re: General question about documentation

2009-12-01 Thread Kenneth Goldman
owner-openssl-us...@openssl.org wrote on 11/26/2009 06:35:42 PM:

  Finally, the source code IS the only reliable source of documentation
  (assuming you can trust your compiler, OS, and hardware to do the
  right thing). It isn't the most CONVENIENT, which is why we desire
  other forms.

Two problems:

1 - Reading the source is only as reliable as the skill of the reader and
the comments in the code.  I'd rather have the answers than a research
project.

2 - If I read the source, I can't determine which functions are stable
and intended to be used by applications and which are internal and
subject to change or deletion with every release.

 the implementation details of the 250-odd API entry points in libssl.so
 would tell me very little about how to properly USE those APIs, and in
 fact, designing an application around my interpretation of the library
 developers intent would likely lead me down some rabbit holes I'd rather
 not explore.

User manual type documentation would be nice, but I'd be content with
a more complete implementation of what's there now.  E.g., the AES
functions are not documented yet.

I find the current documentation quite clear and easy to use.  My
only issue is that some functions are just not documented.


Re: General question about documentation

2009-12-01 Thread Graham Leggett
Kenneth Goldman wrote:

 1 - Reading the source is only as reliable as the skill of the reader and
 the comments in the code.  I'd rather have the answers than a research
 project.

So would I. But far too often, in code of all kinds, this documentation
doesn't exist. As an investor I would far rather have my coders reverse
engineer the code and make it work, than be faced with no information,
and have code doomed to be thrown away.

 2 - If I read the source, I can't determine which functions are stable
 and intended to be used by applications and which are internal and
 subject to change or deletion with every release.

This is only a problem if the developers of the library haven't packaged
their library properly, something that one doesn't expect to still see
in 2009.

Regards,
Graham
--

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-12-01 Thread Randy Turner

As an investor, I would rather have my coders use a product with documentation 
to make progress
on the actual goals of the product, rather than reverse-engineer the 
information they're trying to look for.

With the former method, my cost is (n), with the latter method, my cost could 
be unbounded, depending upon
how complex the source code is (i.e., explicit code, or 14 levels of 
indirection and C macros that have to be understood).

It sounds like you're making the case for documentation to meand I agree.

Randy


On Dec 1, 2009, at 2:01 PM, Graham Leggett wrote:

 Kenneth Goldman wrote:
 
 1 - Reading the source is only as reliable as the skill of the reader and
 the comments in the code.  I'd rather have the answers than a research
 project.
 
 So would I. But far too often, in code of all kinds, this documentation
 doesn't exist. As an investor I would far rather have my coders reverse
 engineer the code and make it work, than be faced with no information,
 and have code doomed to be thrown away.
 
 2 - If I read the source, I can't determine which functions are stable
 and intended to be used by applications and which are internal and
 subject to change or deletion with every release.
 
 This is only a problem if the developers of the library haven't packaged
 their library properly, something that one doesn't expect to still see
 in 2009.
 
 Regards,
 Graham
 --
 
 __
 OpenSSL Project http://www.openssl.org
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org
 

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-12-01 Thread Graham Leggett
Randy Turner wrote:

 As an investor, I would rather have my coders use a product with 
 documentation to make progress
 on the actual goals of the product, rather than reverse-engineer the 
 information they're trying to look for.

Obviously, as I already stated below:

 So would I.

Regards,
Graham
--
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-12-01 Thread Rene Hollan
The problem is that the documentation may not be correct, sending your coders 
on a wild goose chase.
 
Think of the source code as a safe but boring investment (with little barrier 
to entry), and the documentation as one with high potential yield (with a high 
barrier to entry) but significant risk of errors.
 
Now, go balance your software portfolio for whatever risk/reward ratio you find 
optimal.
 
Remember, many code for their own benefit, but documentation benefits others 
(mostly), so it's hard to encourage high quality documentation without paying 
for it.



From: owner-openssl-us...@openssl.org on behalf of Randy Turner
Sent: Tue 12/1/2009 2:08 PM
To: openssl-users@openssl.org
Subject: Re: General question about documentation




As an investor, I would rather have my coders use a product with documentation 
to make progress
on the actual goals of the product, rather than reverse-engineer the 
information they're trying to look for.

With the former method, my cost is (n), with the latter method, my cost could 
be unbounded, depending upon
how complex the source code is (i.e., explicit code, or 14 levels of 
indirection and C macros that have to be understood).

It sounds like you're making the case for documentation to meand I agree.

Randy


On Dec 1, 2009, at 2:01 PM, Graham Leggett wrote:

 Kenneth Goldman wrote:

 1 - Reading the source is only as reliable as the skill of the reader and
 the comments in the code.  I'd rather have the answers than a research
 project.

 So would I. But far too often, in code of all kinds, this documentation
 doesn't exist. As an investor I would far rather have my coders reverse
 engineer the code and make it work, than be faced with no information,
 and have code doomed to be thrown away.

 2 - If I read the source, I can't determine which functions are stable
 and intended to be used by applications and which are internal and
 subject to change or deletion with every release.

 This is only a problem if the developers of the library haven't packaged
 their library properly, something that one doesn't expect to still see
 in 2009.

 Regards,
 Graham
 --

 __
 OpenSSL Project http://www.openssl.org 
 http://www.openssl.org/ 
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org


__
OpenSSL Project http://www.openssl.org 
http://www.openssl.org/ 
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


winmail.dat

RE: General question about documentation

2009-11-27 Thread Mark
Hi All, 

 Rene Hollan wrote:
 
  Oh, you need to dig deeper, to understand the semantics and 
 not just 
  the syntax of those APIs.
 
  I didn't say using the source as documentation was 
 convenient, but it 
  is possible, to any degree of detail you want.
 
  To wit: given the source code, it is possible to create 
 documentation 
  to any degree of convenience. But, given some instance of external 
  documentation and no source, it is not possible to improve the 
  convenience factor of that documentation to an arbitrary degree.
 
  Suggesting what people who donate their time DO is rather 
 like herding 
  cats. Some like coding and others like documenting and some 
 like both. 
  Perhaps instead of an admonition that the project needs better 
  documentation, a question regarding who is willing to contribute to 
  said better documentation is more in order.
 
 
 unluckily, those of us who most need the docs are least able to 
 contribute, as I haven't the foggiest notion how to properly 
 use any of 
 the APIs at present.   I suppose I need to get the ORA book and start 
 reading, as eventually Im going to be helping another 
 development team 
 at work with getting an SSL connection going that needs to 
 use a client 
 certificate stored on a PKCS#11 PKI token, so I'll be sorting 
 out how to 
 use libssl w/ opensc's engine-pkcs11 module, who's 
 documentation is just 
 about as non-existant as that of openssl.  this task was very easy in 
 Java, as Java's SecureSocket hides all the complexity, up to and 
 including full support for PKCS#11 plugins.

The O'Reilly book is essential reading IMHO but it is far from a
complete
guide to OpenSSL.  There are many APIs that it does not mention at all.
It is also quite old now (2002).

I realize that in Open source projects it is hard to find the time to
document
the software but I believe that documentation is an essential part of
any
project, especially something as complex as OpenSSL.

Even if a wiki turnout out to be a better FAQ then it would still be
very useful IMHO.

Is it possible to gain sponsorship for this project or charge for the
documentation,
or make a commercial variant of OpenSSL which can support the open
source
version?

Mark.

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-11-27 Thread Rene Hollan

Yes, what ever attempts are made to improve the documentation, they should 
probably start with errors and omissions in this book, rather than from scratch.

Perhaps O'Rielly might want to publish a followup?

-Original Message-
From: owner-openssl-us...@openssl.org on behalf of Mark
Sent: Fri 11/27/2009 2:46 AM
To: openssl-users@openssl.org
Subject: RE: General question about documentation
 
Hi All, 

 Rene Hollan wrote:
 
  Oh, you need to dig deeper, to understand the semantics and 
 not just 
  the syntax of those APIs.
 
  I didn't say using the source as documentation was 
 convenient, but it 
  is possible, to any degree of detail you want.
 
  To wit: given the source code, it is possible to create 
 documentation 
  to any degree of convenience. But, given some instance of external 
  documentation and no source, it is not possible to improve the 
  convenience factor of that documentation to an arbitrary degree.
 
  Suggesting what people who donate their time DO is rather 
 like herding 
  cats. Some like coding and others like documenting and some 
 like both. 
  Perhaps instead of an admonition that the project needs better 
  documentation, a question regarding who is willing to contribute to 
  said better documentation is more in order.
 
 
 unluckily, those of us who most need the docs are least able to 
 contribute, as I haven't the foggiest notion how to properly 
 use any of 
 the APIs at present.   I suppose I need to get the ORA book and start 
 reading, as eventually Im going to be helping another 
 development team 
 at work with getting an SSL connection going that needs to 
 use a client 
 certificate stored on a PKCS#11 PKI token, so I'll be sorting 
 out how to 
 use libssl w/ opensc's engine-pkcs11 module, who's 
 documentation is just 
 about as non-existant as that of openssl.  this task was very easy in 
 Java, as Java's SecureSocket hides all the complexity, up to and 
 including full support for PKCS#11 plugins.

The O'Reilly book is essential reading IMHO but it is far from a
complete
guide to OpenSSL.  There are many APIs that it does not mention at all.
It is also quite old now (2002).

I realize that in Open source projects it is hard to find the time to
document
the software but I believe that documentation is an essential part of
any
project, especially something as complex as OpenSSL.

Even if a wiki turnout out to be a better FAQ then it would still be
very useful IMHO.

Is it possible to gain sponsorship for this project or charge for the
documentation,
or make a commercial variant of OpenSSL which can support the open
source
version?

Mark.

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org



RE: General question about documentation

2009-11-26 Thread Will Bickford
IMO a wiki would be a great resource for both developers and users of
OpenSSL.

Something along the lines of the Subversion Book - an online reference
book for OpenSSL.

--Will

 -Original Message-
 From: owner-openssl-us...@openssl.org 
 [mailto:owner-openssl-us...@openssl.org] On Behalf Of Mark
 Sent: Wednesday, November 25, 2009 3:27 AM
 To: openssl-users@openssl.org
 Subject: RE: General question about documentation
 
  I would like to post a general observation regarding users of the 
  OpenSSL toolkit.
 
 [snip stuff about documentation]]
 
 A long time ago it was suggested to use a wiki for this 
 purpose. Can this idea be resurrected?
 
 Mark.
 
 __
 OpenSSL Project http://www.openssl.org
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org
 
 
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-26 Thread Randy Turner

That's a great idea Mark and Will,   I would be happy to contribute anything 
that I learn about the toolkit.

There have been a wide range of comments from people saying look at the code 
all the way to basically suggesting an attempt
at a new version of the O'Reilly book.

I can't imagine anyone with any experience at all in software development 
suggesting that the only source of documentation be the source code.

And I don't think I was suggesting that the OpenSSL team necessarily write a 
new version of the O'Reilly book.

Someone also said that if we wanted documentation we should pay for it - which 
seems counter to the whole open source effort.  I'm assuming that the OpenSSL 
developers
are not spending all this time working on the toolkit for the hell of it - I 
would think they would like as many people to use it as possible, and with that 
goal in mind, I think the 1.0 release (when it comes out of beta) would be a 
good stopping point to re-visit the documentation set and providing examples 
that reflect the most common use-cases. The mailing list is always here for 
unusual use-cases.

That being said, I think a Wiki is also a great idea, but would not obviate the 
need for the developers of the toolkit to complete the documentation set. I've 
spent a quite a bit of time with OpenSSL and would be happy to contribute to a 
Wiki.

Thanks!!
Randy


On Nov 25, 2009, at 3:13 PM, Will Bickford wrote:

 IMO a wiki would be a great resource for both developers and users of
 OpenSSL.
 
 Something along the lines of the Subversion Book - an online reference
 book for OpenSSL.
 
 --Will
 
 -Original Message-
 From: owner-openssl-us...@openssl.org 
 [mailto:owner-openssl-us...@openssl.org] On Behalf Of Mark
 Sent: Wednesday, November 25, 2009 3:27 AM
 To: openssl-users@openssl.org
 Subject: RE: General question about documentation
 
 I would like to post a general observation regarding users of the 
 OpenSSL toolkit.
 
 [snip stuff about documentation]]
 
 A long time ago it was suggested to use a wiki for this 
 purpose. Can this idea be resurrected?
 
 Mark.
 
 __
 OpenSSL Project http://www.openssl.org
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org
 
 
 __
 OpenSSL Project http://www.openssl.org
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org
 

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-11-26 Thread Rene Hollan
You are confusing goals and desires.

Someone who wants documentation beyond what they have can either (a) write it 
themselves, (b) wait, (c) offer a bounty. (c) is the only practical choice, if 
they have money but neither time, nor expertise.

Someone who wants to produce open source code to solve a problem and share the 
solution will do the least to get the job done. Someone who wants to produce 
open source software to INFLUENCE others to use it will try to produce the most 
usable software they can. I submit most open source projects fall somewhere in 
the middle.

Finally, the source code IS the only reliable source of documentation (assuming 
you can trust your compiler, OS, and hardware to do the right thing). It 
isn't the most CONVENIENT, which is why we desire other forms.

Should the O'Reilly book be rewritten? Not unless it's truly awful. But, a wiki 
shouldn't start from scratch. It should draw upon existing documentation and 
fill in the gaps. Sucks if you don't have those docs (or can't afford the 
book), but it is the fastest way to get there from here because you don't 
have to start from nothing.

The problem with wikis is that they tend to be a large forest of information 
with little organization. Unless there is some editing effort, it will be 
little more useful than a FAQ and mailing list archive.



-Original Message-
From: owner-openssl-us...@openssl.org on behalf of Randy Turner
Sent: Thu 11/26/2009 11:38 AM
To: openssl-users@openssl.org
Subject: Re: General question about documentation
 

That's a great idea Mark and Will,   I would be happy to contribute anything 
that I learn about the toolkit.

There have been a wide range of comments from people saying look at the code 
all the way to basically suggesting an attempt
at a new version of the O'Reilly book.

I can't imagine anyone with any experience at all in software development 
suggesting that the only source of documentation be the source code.

And I don't think I was suggesting that the OpenSSL team necessarily write a 
new version of the O'Reilly book.

Someone also said that if we wanted documentation we should pay for it - which 
seems counter to the whole open source effort.  I'm assuming that the OpenSSL 
developers
are not spending all this time working on the toolkit for the hell of it - I 
would think they would like as many people to use it as possible, and with that 
goal in mind, I think the 1.0 release (when it comes out of beta) would be a 
good stopping point to re-visit the documentation set and providing examples 
that reflect the most common use-cases. The mailing list is always here for 
unusual use-cases.

That being said, I think a Wiki is also a great idea, but would not obviate the 
need for the developers of the toolkit to complete the documentation set. I've 
spent a quite a bit of time with OpenSSL and would be happy to contribute to a 
Wiki.

Thanks!!
Randy


On Nov 25, 2009, at 3:13 PM, Will Bickford wrote:

 IMO a wiki would be a great resource for both developers and users of
 OpenSSL.
 
 Something along the lines of the Subversion Book - an online reference
 book for OpenSSL.
 
 --Will
 
 -Original Message-
 From: owner-openssl-us...@openssl.org 
 [mailto:owner-openssl-us...@openssl.org] On Behalf Of Mark
 Sent: Wednesday, November 25, 2009 3:27 AM
 To: openssl-users@openssl.org
 Subject: RE: General question about documentation
 
 I would like to post a general observation regarding users of the 
 OpenSSL toolkit.
 
 [snip stuff about documentation]]
 
 A long time ago it was suggested to use a wiki for this 
 purpose. Can this idea be resurrected?
 
 Mark.
 
 __
 OpenSSL Project http://www.openssl.org
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org
 
 
 __
 OpenSSL Project http://www.openssl.org
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org
 

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org



Re: General question about documentation

2009-11-26 Thread Randy Turner

 Finally, the source code IS the only reliable source of documentation 
 (assuming you can trust your compiler, OS, and hardware to do the right 
 thing). It isn't the most CONVENIENT, which is why we desire other forms.
 
Just to clarify...

There isn't a debate about whether source code is documentation - 
documentation is something altogether different from source code, and it this 
altogether different form that is open to debate.

I think the layout, and organization of the existing OpenSSL website is 
evidence enough that the group intended to provide an organized documentation 
tree - that much is certain, so
we can defer the goals/desires semantic discussion.  My only point is, I think 
documentation has taken a back seat priority-wise due to other apparently 
higher-priority development tasks (including possibly paid consulting time).  
My suggestion is, that we try to find some way to update the documentation on 
the website to reflect the features and functionality of the 1.0 release. The 
1.0 release seems like a natural point at which to revisit how we (the 
community) or the core developers, or both, can re-sync the documentation 
sufficient to cover the common use-cases
envisioned for the feature-set. Including the basic API docs which make up the 
bulk of the existing documentation at the site today.

I wouldn't necessarily damn the wiki out of the gate, but I agree it will need 
some organization and possibly editorial support from the core development 
team, to be sufficiently usable to keep
users of the toolkit productive. And I reiterate, as a user of the toolkit, I 
would be happy to contribute to such a Wiki.

Randy


On Nov 26, 2009, at 1:15 PM, Rene Hollan wrote:

 You are confusing goals and desires.
 
 Someone who wants documentation beyond what they have can either (a) write it 
 themselves, (b) wait, (c) offer a bounty. (c) is the only practical choice, 
 if they have money but neither time, nor expertise.
 
 Someone who wants to produce open source code to solve a problem and share 
 the solution will do the least to get the job done. Someone who wants to 
 produce open source software to INFLUENCE others to use it will try to 
 produce the most usable software they can. I submit most open source projects 
 fall somewhere in the middle.
 
 Finally, the source code IS the only reliable source of documentation 
 (assuming you can trust your compiler, OS, and hardware to do the right 
 thing). It isn't the most CONVENIENT, which is why we desire other forms.
 
 Should the O'Reilly book be rewritten? Not unless it's truly awful. But, a 
 wiki shouldn't start from scratch. It should draw upon existing documentation 
 and fill in the gaps. Sucks if you don't have those docs (or can't afford the 
 book), but it is the fastest way to get there from here because you don't 
 have to start from nothing.
 
 The problem with wikis is that they tend to be a large forest of information 
 with little organization. Unless there is some editing effort, it will be 
 little more useful than a FAQ and mailing list archive.
 
 
 
 -Original Message-
 From: owner-openssl-us...@openssl.org on behalf of Randy Turner
 Sent: Thu 11/26/2009 11:38 AM
 To: openssl-users@openssl.org
 Subject: Re: General question about documentation
 
 
 That's a great idea Mark and Will,   I would be happy to contribute anything 
 that I learn about the toolkit.
 
 There have been a wide range of comments from people saying look at the 
 code all the way to basically suggesting an attempt
 at a new version of the O'Reilly book.
 
 I can't imagine anyone with any experience at all in software development 
 suggesting that the only source of documentation be the source code.
 
 And I don't think I was suggesting that the OpenSSL team necessarily write a 
 new version of the O'Reilly book.
 
 Someone also said that if we wanted documentation we should pay for it - 
 which seems counter to the whole open source effort.  I'm assuming that the 
 OpenSSL developers
 are not spending all this time working on the toolkit for the hell of it - I 
 would think they would like as many people to use it as possible, and with 
 that goal in mind, I think the 1.0 release (when it comes out of beta) would 
 be a good stopping point to re-visit the documentation set and providing 
 examples that reflect the most common use-cases. The mailing list is always 
 here for unusual use-cases.
 
 That being said, I think a Wiki is also a great idea, but would not obviate 
 the need for the developers of the toolkit to complete the documentation set. 
 I've spent a quite a bit of time with OpenSSL and would be happy to 
 contribute to a Wiki.
 
 Thanks!!
 Randy
 
 
 On Nov 25, 2009, at 3:13 PM, Will Bickford wrote:
 
  IMO a wiki would be a great resource for both developers and users of
  OpenSSL.
 
  Something along the lines of the Subversion Book - an online reference
  book for OpenSSL.
 
  --Will
 
  -Original Message-
  From

Re: General question about documentation

2009-11-26 Thread John R Pierce


Finally, the source code IS the only reliable source of documentation 
(assuming you can trust your compiler, OS, and hardware to do the 
right thing). It isn't the most CONVENIENT, which is why we desire 
other forms.




the implementation details of the 250-odd API entry points in libssl.so 
would tell me very little about how to properly USE those APIs, and in 
fact, designing an application around my interpretation of the library 
developers intent would likely lead me down some rabbit holes I'd rather 
not explore.


This is my idea of how open source documentation should be organized and 
written.

http://www.postgresql.org/docs/current/static/index.html


__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-11-26 Thread Rene Hollan
Oh, you need to dig deeper, to understand the semantics and not just the syntax 
of those APIs.

I didn't say using the source as documentation was convenient, but it is 
possible, to any degree of detail you want.

To wit: given the source code, it is possible to create documentation to any 
degree of convenience. But, given some instance of external documentation and 
no source, it is not possible to improve the convenience factor of that 
documentation to an arbitrary degree.

Suggesting what people who donate their time DO is rather like herding cats. 
Some like coding and others like documenting and some like both. Perhaps 
instead of an admonition that the project needs better documentation, a 
question regarding who is willing to contribute to said better documentation is 
more in order.



-Original Message-
From: owner-openssl-us...@openssl.org on behalf of John R Pierce
Sent: Thu 11/26/2009 3:35 PM
To: openssl-users@openssl.org
Subject: Re: General question about documentation
 

 Finally, the source code IS the only reliable source of documentation 
 (assuming you can trust your compiler, OS, and hardware to do the 
 right thing). It isn't the most CONVENIENT, which is why we desire 
 other forms.


the implementation details of the 250-odd API entry points in libssl.so 
would tell me very little about how to properly USE those APIs, and in 
fact, designing an application around my interpretation of the library 
developers intent would likely lead me down some rabbit holes I'd rather 
not explore.

This is my idea of how open source documentation should be organized and 
written.
http://www.postgresql.org/docs/current/static/index.html


__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org



Re: General question about documentation

2009-11-26 Thread John R Pierce

Rene Hollan wrote:


Oh, you need to dig deeper, to understand the semantics and not just 
the syntax of those APIs.


I didn't say using the source as documentation was convenient, but it 
is possible, to any degree of detail you want.


To wit: given the source code, it is possible to create documentation 
to any degree of convenience. But, given some instance of external 
documentation and no source, it is not possible to improve the 
convenience factor of that documentation to an arbitrary degree.


Suggesting what people who donate their time DO is rather like herding 
cats. Some like coding and others like documenting and some like both. 
Perhaps instead of an admonition that the project needs better 
documentation, a question regarding who is willing to contribute to 
said better documentation is more in order.




unluckily, those of us who most need the docs are least able to 
contribute, as I haven't the foggiest notion how to properly use any of 
the APIs at present.   I suppose I need to get the ORA book and start 
reading, as eventually Im going to be helping another development team 
at work with getting an SSL connection going that needs to use a client 
certificate stored on a PKCS#11 PKI token, so I'll be sorting out how to 
use libssl w/ opensc's engine-pkcs11 module, who's documentation is just 
about as non-existant as that of openssl.  this task was very easy in 
Java, as Java's SecureSocket hides all the complexity, up to and 
including full support for PKCS#11 plugins.




.




__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-26 Thread Tim Ward

From: John R Pierce pie...@hogranch.com


this task was very easy in Java, as Java's SecureSocket hides all the 
complexity, up to and including full support for PKCS#11 plugins.


Weren't you lucky.

I gave up trying to do that sort of thing in Java when I ran across its 
habit of doing reverse DNS lookups on every IP address it came across, just 
in case it needed the FQDN for anything (I think it was stuck in the mindset 
of people only use SSL to talk to web servers, don't they, and the only way 
of validating a web server is a certificate containing an FDQN, isn't it, so 
whenever I see an IP address I'd better get the FQDN, hadn't I, because I'm 
bound to need it soon, aren't I).


Well, no, actually. Guess what: sometimes people use SSL for purposes other 
than talking to web severs, and in the cases of embedded devices with no DNS 
records talking to each other by explicitly configured IP address the DNS 
lookup took minutes to time out before Java would deign to get on with doing 
what it was told. With no way of switching this nonsense off. Hence I used a 
C++ DLL to do the crypto stuff.


Tim Ward - Brett Ward Limited - 07801 703 600
www.brettward.co.uk 


__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-26 Thread Randy Turner

Yes, I noted that usage of the APIs in combination with common use-cases is 
more appropriate, but this doesn't obviate the need for per-API documentation, 
as has occurred so far on the openssl website.

And I agree with the previous point that we should be trying to collectively 
figure out how to update the documentation in sync with the available features 
and functionality of the 1.0 release.

Randy



On Nov 26, 2009, at 3:35 PM, John R Pierce wrote:

 
 Finally, the source code IS the only reliable source of documentation 
 (assuming you can trust your compiler, OS, and hardware to do the right 
 thing). It isn't the most CONVENIENT, which is why we desire other forms.
 
 
 the implementation details of the 250-odd API entry points in libssl.so would 
 tell me very little about how to properly USE those APIs, and in fact, 
 designing an application around my interpretation of the library developers 
 intent would likely lead me down some rabbit holes I'd rather not explore.
 
 This is my idea of how open source documentation should be organized and 
 written.
 http://www.postgresql.org/docs/current/static/index.html
 
 
 __
 OpenSSL Project http://www.openssl.org
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org
 

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-26 Thread Randy Turner

This is an example of a relatively common use-case that I was alluding to in a 
previous email...it would be nice to not have to figure this out either by 
guessing, reverse-engineering something, or other sub-optimal form of 
development strategy

Randy


On Nov 26, 2009, at 4:03 PM, John R Pierce wrote:

 Rene Hollan wrote:
 
 Oh, you need to dig deeper, to understand the semantics and not just the 
 syntax of those APIs.
 
 I didn't say using the source as documentation was convenient, but it is 
 possible, to any degree of detail you want.
 
 To wit: given the source code, it is possible to create documentation to any 
 degree of convenience. But, given some instance of external documentation 
 and no source, it is not possible to improve the convenience factor of that 
 documentation to an arbitrary degree.
 
 Suggesting what people who donate their time DO is rather like herding cats. 
 Some like coding and others like documenting and some like both. Perhaps 
 instead of an admonition that the project needs better documentation, a 
 question regarding who is willing to contribute to said better documentation 
 is more in order.
 
 
 unluckily, those of us who most need the docs are least able to contribute, 
 as I haven't the foggiest notion how to properly use any of the APIs at 
 present.   I suppose I need to get the ORA book and start reading, as 
 eventually Im going to be helping another development team at work with 
 getting an SSL connection going that needs to use a client certificate stored 
 on a PKCS#11 PKI token, so I'll be sorting out how to use libssl w/ opensc's 
 engine-pkcs11 module, who's documentation is just about as non-existant as 
 that of openssl.  this task was very easy in Java, as Java's SecureSocket 
 hides all the complexity, up to and including full support for PKCS#11 
 plugins.
 
 
 
 .
 
 
 
 
 __
 OpenSSL Project http://www.openssl.org
 User Support Mailing Listopenssl-users@openssl.org
 Automated List Manager   majord...@openssl.org
 

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-25 Thread John R Pierce

Rene Hollan wrote:

Crypto is hard... mostly because X509 is a dog's breakfast of committee 
compromisitis.

That said, openssl docs should AT LEAST address one who is familiar with X509.
  


openssl docs should go way beyond that, and include tutorials of the 
'right way' to do a wide range of the sorts of things that SSL/TLS 
programs need to do.   as it stands, as a crypto novice,I find them 
completely unusable, and wouldn't know how to begin to write an SSL 
application, short of looking at others and blindly copying what I find 
without any real understanding of why.  And I really really hate 
programming that way, it feels like arcane magic ('any sufficiently 
advanced technology is indistinguishable from magic, yada yada)



__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-25 Thread Tim Ward

From: Graham Leggett minf...@sharp.fm


Use the source: while not the easiest to read it is the most accurate
documentation available at any given time. (No, this is not a
justification for a lack of or bad documentation).


The objection, and it's a major one, to reverse engineering the API from the 
source is that the source doesn't tell you what the contract is - it doesn't 
tell you what this API guarantees to deliver, it only tells you what the 
current version's implementation actually does. So if you rely on the 
implementation of the current version your code will break when the next 
version does something different, which is not the case if you're writing to 
a published and documented API as you won't be accidentally relying on 
non-guaranteed implementation details.


Tim Ward - Brett Ward Limited - 07801 703 600
www.brettward.co.uk 


__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-25 Thread Victor Duchovni
On Tue, Nov 24, 2009 at 06:27:19PM -0800, John R Pierce wrote:

 openssl docs should go way beyond that, and include tutorials of the 'right 
 way' to do a wide range of the sorts of things that SSL/TLS programs need 
 to do.

That's what books are for. Don't confuse reference documentation with
tutorials.

The reference documentation for OpenSSL can and should be better, but
tutorials are are typically material for books, or community maintained
Wikis, ...

-- 
Viktor.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-25 Thread Michael S. Zick
On Tue November 24 2009, Graham Leggett wrote:
 Tim Ward wrote:
 
 In the really big corporates I have been involved with, they have all
 demanded either source code with the product or the source in escrow
 before they will consider using it. 


My experience also.  
One of the best arguments for using OSS in commercial products.

They want a single point of contact that is responsible (and liable)
for the sources.  The single point they sent their money too.  ;)

The response:
My code's vendor *claims* to have the source in escrow.
Rarely works in real life.

Mike
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-11-25 Thread Rene Hollan
This is true, but often the code does not match the docs. Or, the docs
leave something out (is this dynamically allocated? Who is responsible
for deallocating it, etc).

Reconciling what the code does vs. what it should do vs. what is
ambiguous is always a matter for debate (is that a feature or a bug?),
but when you don't have the code and you do have docs only, there will
always be something missing in the docs, whether a bug or ambiguity.

So, pick your poison: docs that do not completely describe the code you
don't have, or code that doesn't do what you think it does.

The first you pay for, the second you don't, though you can pay someone
who might understand it better.

The bottom line is this: if you want better docs for free code, PAY to
have then written, or shut up.

Maybe someday someone will, altruistically, document the combined wisdom
here and in the code. But no one owes it to anyone.




-Original Message-
From: owner-openssl-us...@openssl.org
[mailto:owner-openssl-us...@openssl.org] On Behalf Of Tim Ward
Sent: Wednesday, November 25, 2009 12:59 AM
To: openssl-users@openssl.org
Subject: Re: General question about documentation

From: Graham Leggett minf...@sharp.fm

 Use the source: while not the easiest to read it is the most accurate
 documentation available at any given time. (No, this is not a
 justification for a lack of or bad documentation).

The objection, and it's a major one, to reverse engineering the API from
the 
source is that the source doesn't tell you what the contract is - it
doesn't 
tell you what this API guarantees to deliver, it only tells you what the

current version's implementation actually does. So if you rely on the 
implementation of the current version your code will break when the next

version does something different, which is not the case if you're
writing to 
a published and documented API as you won't be accidentally relying on 
non-guaranteed implementation details.

Tim Ward - Brett Ward Limited - 07801 703 600
www.brettward.co.uk 

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-11-25 Thread Mark
 I would like to post a general observation regarding users of 
 the OpenSSL toolkit.

[snip stuff about documentation]]

A long time ago it was suggested to use a wiki for this purpose. Can
this
idea be resurrected?

Mark.

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-24 Thread Tim Ward

From: Randy Turner rtur...@amalfisystems.com


From the length of some of the threads I've read in the past, a number
of developers seem to be burning a lot of development hours guessing
at how functions are supposed to work


Yes indeed. This is why I often go for commercial software in preference fo 
free - it took me a day and a half to get a working Visual Studio 2005 
debug DLL built, at a cost to my client of ... er ... well ... none of 
anyone else's business really, but lots more than any software package I 
remember buying. If instead of the free OpenSSL there'd been available a 
non-free product that cost £500 with normal commercial standards of 
installation and documentation and support we'd be laughing all the way to 
the bank.


Having said which, trying to use Windows' native Schannel isn't a lot 
better, to be honest: sure, there's more documentation than there is of 
OpenSSL, but much of it is incomprehensible, and sure, there's free (ie 
included in the MSDN subscription) support from Microsoft engineers, but 
that takes time.


Bottom line seems to me to be that this crypto/security business is in a 
class of its own; it's essentially difficult, and no product has received 
the productisation effort that is common in other fields.


---

One thing I still haven't found anywhere, it's not even in the O'Reilly 
book, is a description of the object system - I've sort-of picked up that 
if you XXX_new() something you maybe ought to XXX_free() it sometime later, 
and there's some sort of use counting going on, but I've not found any 
documentation for any of this yet.


Tim Ward - Brett Ward Limited - 07801 703 600
www.brettward.co.uk 


__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


Re: General question about documentation

2009-11-24 Thread Graham Leggett
Tim Ward wrote:

 Yes indeed. This is why I often go for commercial software in preference
 fo free - it took me a day and a half to get a working Visual Studio
 2005 debug DLL built, at a cost to my client of ... er ... well ... none
 of anyone else's business really, but lots more than any software
 package I remember buying. If instead of the free OpenSSL there'd been
 available a non-free product that cost £500 with normal commercial
 standards of installation and documentation and support we'd be laughing
 all the way to the bank.

I go the opposite way - the risk of encountering an un-solveable problem
due to lack of access to source code means the potential to be forced to
throw away and reimplement code is always present. This factor increases
with time, as with age it becomes more and more certain that the code
will either be updated outside of your control or no longer supported. A
greater cost with a lower risk is always preferable to me in the long run.

In the really big corporates I have been involved with, they have all
demanded either source code with the product or the source in escrow
before they will consider using it. Some of the in house systems I
encountered started out life as commercial software whose vendors had
ceased supporting the software.

 One thing I still haven't found anywhere, it's not even in the O'Reilly
 book, is a description of the object system - I've sort-of picked up
 that if you XXX_new() something you maybe ought to XXX_free() it
 sometime later, and there's some sort of use counting going on, but I've
 not found any documentation for any of this yet.

Use the source: while not the easiest to read it is the most accurate
documentation available at any given time. (No, this is not a
justification for a lack of or bad documentation).

Regards,
Graham
--
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: General question about documentation

2009-11-24 Thread Rene Hollan
Crypto is hard... mostly because X509 is a dog's breakfast of committee 
compromisitis.

That said, openssl docs should AT LEAST address one who is familiar with X509.

-Original Message-
From: owner-openssl-us...@openssl.org [mailto:owner-openssl-us...@openssl.org] 
On Behalf Of Tim Ward
Sent: Tuesday, November 24, 2009 1:37 PM
To: openssl-users@openssl.org
Subject: Re: General question about documentation

From: Randy Turner rtur...@amalfisystems.com

 From the length of some of the threads I've read in the past, a number
 of developers seem to be burning a lot of development hours guessing
 at how functions are supposed to work

Yes indeed. This is why I often go for commercial software in preference fo 
free - it took me a day and a half to get a working Visual Studio 2005 
debug DLL built, at a cost to my client of ... er ... well ... none of 
anyone else's business really, but lots more than any software package I 
remember buying. If instead of the free OpenSSL there'd been available a 
non-free product that cost £500 with normal commercial standards of 
installation and documentation and support we'd be laughing all the way to 
the bank.

Having said which, trying to use Windows' native Schannel isn't a lot 
better, to be honest: sure, there's more documentation than there is of 
OpenSSL, but much of it is incomprehensible, and sure, there's free (ie 
included in the MSDN subscription) support from Microsoft engineers, but 
that takes time.

Bottom line seems to me to be that this crypto/security business is in a 
class of its own; it's essentially difficult, and no product has received 
the productisation effort that is common in other fields.

---

One thing I still haven't found anywhere, it's not even in the O'Reilly 
book, is a description of the object system - I've sort-of picked up that 
if you XXX_new() something you maybe ought to XXX_free() it sometime later, 
and there's some sort of use counting going on, but I've not found any 
documentation for any of this yet.

Tim Ward - Brett Ward Limited - 07801 703 600
www.brettward.co.uk 

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org