Re: [fossil-users] minor milestone: Java client
On 1 Oct 2011, at 15:28 , Stephan Beal wrote:
> On Sat, Oct 1, 2011 at 3:07 PM, Paul Ruizendaal wrote:
> > Maybe you could do some http-over-stdin/stdout, and speak json there. :)
>
> Isn't "fossil cgi" already doing that? A front-end could build the
> appropriate environment variables and fork/exec "fossil cgi", feeding the
> post body to fossil's stdin.
>
> That is basically what server mode does - prepares the cgi environment
> (including stdin/stdout) and runs it through CGI mode...
Yes, and this is what any wrapper program can do. For example, there are folks
that would like to have Tcl/Jim with Fossil, I would prefer Javascript and I'm
sure there are those who would like Python or PHP. In each of those
environments it would be easy to add a command/library function that calls
Fossil's json interface through exec'ing Fossil in cgi mode. Tcl etc. would be
wrapped around Fossil, rather than Fossil wrapped around the language and that
way everybody can have their preferred language without further bloat in the
Fossil binary.
Doing fork/exec sounds expensive, but on a posix box there is not much
difference between that and spawning a thread:
http://bulk.fefe.de/scalable-networking.pdf
> (where output is very controlled (entirely buffered) because no body text may
> be sent before the headers are sent). The majority of the parameter handling
> in all of the fossil commands relies on the environment having a fresh state
> for each request, which means the whole environment, GET, POST, and
> everything, needs to be reconstructed before each commands. e.g. the way a
> given command is invoked is based on the request path (an environment
> variable), which can only be set to one value for any given invocation of
> server/CGI mode. There is no mechanism (in the current code base) with which
> we can simply re-set the whole environment and run a different command.
Yes, and why would that be difficult to handle for a wrapper (see below)?
>
> > after every command fossil runs, exit() is called somewhere, which makes it
> > difficult or impossible to chain commands together in the same app session.
>
> Why?
>
> The same reason attempts to adding a shell mode for fossil have failed so
> far. Almost every error condition in fossil passes through fossil_fatal() or
> similar, and that ends up calling exit(). So if you add a command and use any
> of the db functions, and those hit any errors at all, your command will end
> up exit()ing. If you mean "why does it exit() so much?" That's just the way
> it was originally designed - historic momentum. For fossil's original purpose
> (and its current primary uses), the current model actually works quite well,
> and simplifies some of the trickier handling needed when producing HTML
> output (e.g. the buffering of the body and headers, and sending them out at
> the proper times). There is of course lots of interest in refactoring fossil
> into a library, but the amount of effort required is huge. It could easily
> take a few full-time weeks of development effort, and would probably require
> the assistance of at least one person who's "exceedingly familiar" with fos
sil's "more intricate bits" (e.g. how artifacts are
maintained/linked/created/etc.). While i'm relatively familiar with the code in
general, there are huge blocks of voodoo in there i won't touch for fear of
breaking them.
My questions was indeed the former: why do the calls to exit make it difficult
or impossible to chain commands -- there would simply be a chain of CGI calls
and if one step fails the wrapper has to decide what to do next, just as it
would have to do when a classic API call fails.
My question was not the latter ("why does it exit on exceptions"). Richard
calls it vintage unix thinking, but I think it is equally valid in a RESTfull
design -- perhaps any design.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
On Sat, Oct 1, 2011 at 9:07 AM, Paul Ruizendaal wrote: > > > after every command fossil runs, exit() is called somewhere, which makes > it difficult or impossible to chain commands together in the same app > session. > > Why? > > I built Fossil using the old-school unix design paradigm of a light-weight process that does one thing quickly, then exits, leaving the OS clean up the mess. As long as the process is reasonably light-weight, this pattern works very, very well. No worries with memory leaks or threads or unwinding the stack following an error, or any of the other management issues that plague long-running processes. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Wildcards not working on windows
On 30 Sep 2011, at 16:28 , Jan Danielsson wrote: > On 09/30/11 15:00, Tomek Kott wrote: >>> On a semi-related note: Do the Windows binaries include SSL support? >>> >> I don't think so, because by default there's no openssl library installed >> on win machines. > > If it is the case that it is built without SSL for that reason, would > it be possible to include SSL through static linking for the Windows > binary? I realize the file will grow with a not insignificant amount, > but I think client certificate support is important enough to warrant it. Last time I looked (about 18 months ago), CyaSSL was only ~250kb and API compatible with OpenSSL. Not sure about license compatibility, but if that works out it could make fossil depend on the C std lib only. It would push the fossil binary towards 1.5 MB though. When I first came to fossil it was only ~750 kB, now it is ~1.2 MB. I very much appreciate all the usability enhancements that came to fossil over the past 2..3 years, but sometimes I wonder if it is not time to carefully review overlaps and dead ends in the functionality, and trim the code base a bit -- which could also help with the security and performance work that was discussed on this list recently. Paul ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] delete a branch
On Sat, Oct 1, 2011 at 3:19 PM, Paul Ruizendaal wrote: > There is something unsatisfying about the "mistake" solution, at least that > is my feeling. Conceptually, there is a difference between not deleting > history (which I think is a very valuable asset in fossil) and always > showing full history, rubbing mistakes in your face, especially when still > learning fossil. > LOL! If i'm not mistaken, some people have removed data by exporting their repo to git format, deleting it there, and then re-importing it. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
On Sat, Oct 1, 2011 at 3:07 PM, Paul Ruizendaal wrote: > > Maybe you could do some http-over-stdin/stdout, and speak json there. :) > > Isn't "fossil cgi" already doing that? A front-end could build the > appropriate environment variables and fork/exec "fossil cgi", feeding the > post body to fossil's stdin. > That is basically what server mode does - prepares the cgi environment (including stdin/stdout) and runs it through CGI mode (where output is very controlled (entirely buffered) because no body text may be sent before the headers are sent). The majority of the parameter handling in all of the fossil commands relies on the environment having a fresh state for each request, which means the whole environment, GET, POST, and everything, needs to be reconstructed before each commands. e.g. the way a given command is invoked is based on the request path (an environment variable), which can only be set to one value for any given invocation of server/CGI mode. There is no mechanism (in the current code base) with which we can simply re-set the whole environment and run a different command. > Actually, I was thinking that http/json was a more modern and universal API > for doing a fossil lib than an API based on a long list of C functions and > structs. > In a way it is, but at the lowest level it's of course all C bits. But once we have a usable JSON API in place, the effect will basically be a "library used over the net" instead of one you link to directly. Not that i'm discounting the idea - it's fundamentally a good one (and could even allow json-over-ssh/rsh/whatever, as Alaric mentioned). But it won't work out of the box on the current code base. It is, however, something i will certainly keep in mind as the library gradually mutates in a direction where that would be feasible. > after every command fossil runs, exit() is called somewhere, which makes > it difficult or impossible to chain commands together in the same app > session. > > Why? > The same reason attempts to adding a shell mode for fossil have failed so far. Almost every error condition in fossil passes through fossil_fatal() or similar, and that ends up calling exit(). So if you add a command and use any of the db functions, and those hit any errors at all, your command will end up exit()ing. If you mean "why does it exit() so much?" That's just the way it was originally designed - historic momentum. For fossil's original purpose (and its current primary uses), the current model actually works quite well, and simplifies some of the trickier handling needed when producing HTML output (e.g. the buffering of the body and headers, and sending them out at the proper times). There is of course lots of interest in refactoring fossil into a library, but the amount of effort required is huge. It could easily take a few full-time weeks of development effort, and would probably require the assistance of at least one person who's "exceedingly familiar" with fossil's "more intricate bits" (e.g. how artifacts are maintained/linked/created/etc.). While i'm relatively familiar with the code in general, there are huge blocks of voodoo in there i won't touch for fear of breaking them. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] delete a branch
There is something unsatisfying about the "mistake" solution, at least that is my feeling. Conceptually, there is a difference between not deleting history (which I think is a very valuable asset in fossil) and always showing full history, rubbing mistakes in your face, especially when still learning fossil. Perhaps it is bloat, but would it be useful if branches with the tag 'mistake' or perhaps 'hidden' were not shown on the default timeline in the web interface, and having an option for showing the full timeline? It would be a bit like the full file view and the current file view when showing files. Paul On 30 Sep 2011, at 22:30 , Richard Hipp wrote: > > > On Fri, Sep 30, 2011 at 4:14 PM, Tomek Kott wrote: > nope*. What many people do, including the SQLite & Fossil repos, is simply > rename the branch as "mistake," close the leaf, update back to trunk, and > start again. > > See, for example, http://www.sqlite.org/src/timeline?r=mistake > > Tomek > > * - I think it would be somehow possible to get the artifact # and shun it, > so technically, it is possible. But there is no command for it. > > The intent of Fossil is to preserve history, not delete it. Hence, there is > no easy way to delete a branch. > > Shunning is intended as an emergency mechanism to remove content that should > have never gotten into the repository in the first place - things such as > wiki or ticket spam. Shunning is not intended to remove content just because > you think it is obsolete. Mark it as a mistake, as Tomek suggests above. > But don't destroy history. > > > > On Fri, Sep 30, 2011 at 4:01 PM, Erlis Vidal wrote: > I'm sorry for this so n00b question but I'm not able to find this anywhere. > > Is it possible to delete a branch? I've created a branch by mistake, in this > case I don't want to merge the branch, I'm looking for the git equivalent -f > (force delete) > > Thanks, > Erlis > > ___ > fossil-users mailing list > fossil-users@lists.fossil-scm.org > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users > > > > ___ > fossil-users mailing list > fossil-users@lists.fossil-scm.org > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users > > > > > -- > D. Richard Hipp > d...@sqlite.org > ___ > fossil-users mailing list > fossil-users@lists.fossil-scm.org > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
> Maybe you could do some http-over-stdin/stdout, and speak json there. :) Isn't "fossil cgi" already doing that? A front-end could build the appropriate environment variables and fork/exec "fossil cgi", feeding the post body to fossil's stdin. Actually, I was thinking that http/json was a more modern and universal API for doing a fossil lib than an API based on a long list of C functions and structs. > after every command fossil runs, exit() is called somewhere, which makes it > difficult or impossible to chain commands together in the same app session. Why? Paul On 1 Oct 2011, at 13:42 , Stephan Beal wrote: > 2011/10/1 Lluís Batlle i Rossell > Maybe you could do some http-over-stdin/stdout, and speak json there. :) > > I think of people wanting to write frontends to fossil. > > i was just thinking about that while i was out shopping. It is, in principal, > possible, but fossil's heavy use of exit() as an error recovery strategy > would interfere with that. After every command fossil runs, exit() is called > somewhere, which makes it difficult or impossible to chain commands together > in the same app session. > > -- > - stephan beal > http://wanderinghorse.net/home/stephan/ > ___ > fossil-users mailing list > fossil-users@lists.fossil-scm.org > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
2011/10/1 Lluís Batlle i Rossell > Maybe you could do some http-over-stdin/stdout, and speak json there. :) > > I think of people wanting to write frontends to fossil. > i was just thinking about that while i was out shopping. It is, in principal, possible, but fossil's heavy use of exit() as an error recovery strategy would interfere with that. After every command fossil runs, exit() is called somewhere, which makes it difficult or impossible to chain commands together in the same app session. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Protection against timing attacks
On Oct 1, 2011, at 9:24 , Ron Wilson wrote: > On Fri, Sep 30, 2011 at 2:27 PM, Dmitry Chestnykh > wrote: >> The attacker cannot supply hash, he supplies password. To do timing attack, >> the >> attacker have to find a such string, for which the hash has a few bytes >> changed. > > You and I seem to be talking about different use cases, There are > scenarios where both the client and server generate hashes. The client > sends its has to the server and the server compares the hashes. This is correct, yes. -- Dmitry Chestnykh ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
I thought that was how fossil over ssh works already; on the far end fossil is started with some undocumented private command that talks http over stdin/stdout. you may well find your JSON work works transparently with that mechanism too due to it being baked into the fossil http stack! Sent from my BlackBerry® wireless device -Original Message- From: Lluís Batlle i Rossell Sender: fossil-users-boun...@lists.fossil-scm.org Date: Sat, 1 Oct 2011 11:28:39 To: Reply-To: fossil-users@lists.fossil-scm.org Subject: Re: [fossil-users] minor milestone: Java client On Sat, Oct 01, 2011 at 11:23:32AM +0200, Stephan Beal wrote: > 2011/10/1 Lluís Batlle i Rossell > > > But that's only a hack. The spawner will not know when the server will be > > listening. And killing it, will not know if anyone else uses it and it is > > working. > > > i don't see any other way to do it unless/until fossil is split up into a > lib. The only entry points clients currently have into fossil are via CLI or > HTTP modes. Maybe you could do some http-over-stdin/stdout, and speak json there. :) I think of people wanting to write frontends to fossil. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
On Sat, Oct 01, 2011 at 11:23:32AM +0200, Stephan Beal wrote: > 2011/10/1 Lluís Batlle i Rossell > > > But that's only a hack. The spawner will not know when the server will be > > listening. And killing it, will not know if anyone else uses it and it is > > working. > > > i don't see any other way to do it unless/until fossil is split up into a > lib. The only entry points clients currently have into fossil are via CLI or > HTTP modes. Maybe you could do some http-over-stdin/stdout, and speak json there. :) I think of people wanting to write frontends to fossil. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
2011/10/1 Lluís Batlle i Rossell > But that's only a hack. The spawner will not know when the server will be > listening. And killing it, will not know if anyone else uses it and it is > working. i don't see any other way to do it unless/until fossil is split up into a lib. The only entry points clients currently have into fossil are via CLI or HTTP modes. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
On Sat, Oct 01, 2011 at 11:12:51AM +0200, Stephan Beal wrote: > 2011/10/1 Lluís Batlle i Rossell > > > Sorry, not that I care much about JSON, but can json be used in a way that > > a > > program *spawns* fossil and talks to it using json, until it decides it's > > enough and fossil stops? > > > > In theory, yes. It just has to start fossil server --port XYZ, send its > requests to localhost:XYZ/json/..., and kill of the spawned process when > it's done (because the server mode forks before json mode is entered, i > don't think i could add a json command to shut down the server). But that's only a hack. The spawner will not know when the server will be listening. And killing it, will not know if anyone else uses it and it is working. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
2011/10/1 Lluís Batlle i Rossell > Sorry, not that I care much about JSON, but can json be used in a way that > a > program *spawns* fossil and talks to it using json, until it decides it's > enough and fossil stops? > In theory, yes. It just has to start fossil server --port XYZ, send its requests to localhost:XYZ/json/..., and kill of the spawned process when it's done (because the server mode forks before json mode is entered, i don't think i could add a json command to shut down the server). The little I read about the json implementation in fossil, it sounds to me > simply as RPC over http, so requiring a server ready listening at a known > address. > That's basically the case, yes. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] minor milestone: Java client
On Sat, Oct 01, 2011 at 02:14:38AM +0200, Stephan Beal wrote: > Hi, all! > > Another minor milestone: the 2nd proof-of-concept non-HTML client, this time > in Java. It's fairly basic, and only supports synchronous operation, but > it's a start. Sorry, not that I care much about JSON, but can json be used in a way that a program *spawns* fossil and talks to it using json, until it decides it's enough and fossil stops? The little I read about the json implementation in fossil, it sounds to me simply as RPC over http, so requiring a server ready listening at a known address. Regards, Lluís ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] looking for feed back on /json/artifact output format
Hi, all,
i've started to implement /json/artifact, which is analogous to the /info,
/vinfo, etc. family of functions (but consolidated into one interface). It
is called like this:
/json/artifact/ID
or
/json/artifact?uuid=ID
where ID may be any artifact ID or tag (in which case it resolves to the
newest artifact with that tag). It currently only handles "commit" artifacts
- tickets, wikis, etc. are TODO.
The response looks like:
[stephan@cheyenne:~/cvs/fossil/fossil-json]$ ./fossil json artifact json -I
2 # 2nd "json"==tag name
{
"fossil":"22fc0ab81bf965cad657992932d559850a79bd8a",
"timestamp":1317453248,
"procTimeMs":1,
"payload":{
"type":"checkin",
/* name = uuid arg provided (may resolve to something different - see
artifact.uuid) */
"name":"json",
"rid":14023,
/* artifact property has artifact-type-dependent structure ... this one is
for "checkin" artifacts...*/
"artifact":{
"isLeaf":true,
"uuid":"22fc0ab81bf965cad657992932d559850a79bd8a",
"user":"stephan",
"comment":"more timeline/artifact refactoring.",
"mtime":1317452671,
/* if the artifact has been edited, the original comment/mtime/user are also
included */
"parentUuid":"b1f9257213c4447b0a072e73f7845b43e85074cd",
"tags":[
"json",
"json-multitag-test"
],
"files":[
{
"name":"src/json_artifact.c",
"uuid":"9a3cded924b5fb1aaffb6f09f220cdef4369107f",
"prevUuid":"59606a8986e68cac5bf0fe13971004759c535345",
"state":"modified"
},
{
"name":"src/json_timeline.c",
"uuid":"9b9bf636eb7b5bfd77d4e538fb7ee5c06b306e44",
"prevUuid":"8552eb9bdb9f64d9bc01fbd5b92f9f251110fb18",
"state":"modified"
}
]
}
}
}
My questions include (but are not necessarily limited to):
a) Is that structure acceptable (or close) for checkin artifacts?
b) are there any strong opinions on what the structures for other artifact
types should look like? If not, i will be modelling them off of the
analogous /Xinfo page.
You can play around with this at:
http://fossil.wanderinghorse.net/repos/fossil-sgb/json/index.html
Look for a button labeled "artifact/XYZ". To try different branches/IDs,
edit the "Request" field after clicking the artifact/XYZ button once, then
click Send.
:-?
--
- stephan beal
http://wanderinghorse.net/home/stephan/
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Protection against timing attacks
On Fri, Sep 30, 2011 at 2:27 PM, Dmitry Chestnykh wrote: > The attacker cannot supply hash, he supplies password. To do timing attack, > the > attacker have to find a such string, for which the hash has a few bytes > changed. You and I seem to be talking about different use cases, There are scenarios where both the client and server generate hashes. The client sends its has to the server and the server compares the hashes. Of course, another protection against attacks is to limit the time window before a whole new set of hashes needs to be computed. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
