[fossil-users] Current status on hooks?
Hi, I'd like to know where we are in the topic of hooks. Their lack is one of the major setbacks of using Fossil. The last time I've noticed a discussion on their topic it was said, that it's blocked by inability to start background processes on Windows. Now there is a way to run Fossil itself in background on Windows. Has this helped towards getting hooks into main line? Kind regards, Remigiusz Modrzejewski ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Current status on hooks?
On Sat, Sep 10, 2011 at 10:32 AM, Remigiusz Modrzejewski l...@maxnet.org.pl wrote: Hi, I'd like to know where we are in the topic of hooks. Their lack is one of the major setbacks of using Fossil. The last time I've noticed a discussion on their topic it was said, that it's blocked by inability to start background processes on Windows. Now there is a way to run Fossil itself in background on Windows. Has this helped towards getting hooks into main line? The current limiting factor is DRH bandwidth. I have way too much else going on and this is a low priority issue for me. But you can help by contributing code! The hook mechanism should include a delay. This is because client-to-server push operations can occur in multiple stateless steps, and we really want to wait and run the hooks after all steps of the push operation are complete. So, for example, after the first push step, the server says to itself I'm going to run the hooks if I don't get any new information within 15 seconds. Then as additional rounds of push arrive, the counter is reset. The 15-second wait interval should be configurable. As part of the hook mechanism, I think we also need auto-sync between server instances. For example, Fossil itself is stored on three separate servers: http://www.fossil-scm.org/ in Dallas, http://www2.fossil-scm.org/ in New York, and http://www3.fossil-scm.org/ in Fremont. I currently keep these all in sync by setting a cron job to run sync every couple of hours. But it would be much better if one could enter, on each server, the names of other servers with which it should automatically synchronize whenever new information arrives. This auto-sync should be smart about avoiding sync-loops. Hence if server A gets new content and then auto-syncs to server B, server B should know that it got its new information from A and should avoid auto-syncing back to server A. Some other projects (besides hooks) that I think would be good to work on, for anybody who is interested: (1) Making Fossil more RESTful, and adding JSON replies. Stephan Beal is already working on this, but I'm sure he'd like to have collaborators. (2) Add the ability to push/pull/sync just a specific branch, as an alternative to the current behavior of push/pull/sync of everything. The current behavior should continue as the default. Single-branch push/pull/sync should be a command-line option. (3) Use the FTS4 full-text search engine of SQLite to implement full-text search over the entire repository history. Kind regards, Remigiusz Modrzejewski ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Draft doc for JSON/REST Fossil interface
On Fri, Sep 9, 2011 at 2:46 PM, Martin S. Weber martin.we...@nist.gov wrote: ... and c) a separation of concerns (you, fossil library, take care of the fossil stuff, I take care of invoking hooks, presenting ... I think that invoking hooks would be an intermediary between the client and a pure Fossil server. Keep in mind that a Fossil instance is a client of another Fossil instance during a sync/pull/push operation and both instances could have their own hooks to invoke. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Current status on hooks?
On Sep 10, 2011, at 7:53 PM, Richard Hipp wrote: (3) Use the FTS4 full-text search engine of SQLite to implement full-text search over the entire repository history. Ha, that was the next thing I was going to ask ;) But unfortunately I'm unable to contribute any programming time this month :( Kind regards, Remigiusz Modrzejewski ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Current status on hooks?
On Sat, Sep 10, 2011 at 1:53 PM, Richard Hipp d...@sqlite.org wrote: The hook mechanism should include a delay. This is because client-to-server push operations can occur in multiple stateless steps, and we really want to wait and run the hooks after all steps of the push operation are complete. So, for example, after the first push step, the server says to itself I'm going to run the hooks if I don't get any new information within 15 seconds. Then as additional rounds of push arrive, the counter is reset. The 15-second wait interval should be configurable. I assume that similar logic would apply for running pre-hooks - ie, the pre hook is run only for the initial step of an operation. As part of the hook mechanism, I think we also need auto-sync between server instances. I have actually been looking into this, though not yet written any code. But now, I think I can put more time into this. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Getting a check-in's info without an open checkout
(Via the command line interface...) The 'timeline' command can take a -R argument to specify a repository. It reports a set of information (time, user, check-in id, comment, etc.), but I do not see it reporting the parent(s) of a check-in The 'info' command can take an object (for example, a check-in's sha1) and report various information about it. If you give it a check-in's id, you can see the list of parents to that check-in. However, the 'info' command does not allow for a -R argument to specify a repo, it will only accept a check-in sha1 if you are from within an open checkout. It should be a legal operation to get the info of a check-in within a repository, without having to be in an open checkout (as is my situation), but I can not find the command line voodoo to get it. Am I missing something? In the end, I just need to get the list of parent check-ins for a check-in, without being in an open checkout folder, so by specifying the repository path. Does anyone know any way I can just get this information? Thanks, Jeff ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Presentation slides for Fossil?
On Thu, Sep 8, 2011 at 4:08 PM, Ingo Koch ingo.k...@gmx.de wrote: Take a look at http://arnebachmann.pytalhost.de/pdf/integrated-version-control-with-fossil-scm.pdf Good presentation. I noted it said that configuration is not shared. Actually, it is possible to share configuration between Fossil instances. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Current status on hooks?
On Sat, 10 Sep 2011, Richard Hipp wrote: The hook mechanism should include a delay. This is because client-to-server push operations can occur in multiple stateless steps, and we really want to wait and run the hooks after all steps of the push operation are complete. So, for example, after the first push step, the server says to itself I'm going to run the hooks if I don't get any new information within 15 seconds. Then as additional rounds of push arrive, the counter is reset. The 15-second wait interval should be configurable. I don't like timers like that, a single server may have clients on fast connections 50 feet away, and at the end of a slow intercontinental link, so setting the timer correctly is about impossible. On top of that: they'll be hard to implement for cgi or inetd based servers, where nothing is running unless a client has an open connection. It's been a while since I looked at the protocol specs, but the client usually maintains (implicit) state in this type of protocol, and could very well indicate to the server that a push operation step is the first one (for a pre-commit hook), last one (for a post-commit hook), or both, which could then trigger the hooks immediately. Downsides: - needs a protocol change - what do you do if the last step never arrives? Gé___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Current status on hooks?
On Sat, Sep 10, 2011 at 7:53 PM, Richard Hipp d...@sqlite.org wrote: (1) Making Fossil more RESTful, and adding JSON replies. Stephan Beal is already working on this, but I'm sure he'd like to have collaborators. +1 on that second part :). i made fairly good progress on this on Thursday/Friday. Today i'm taking a break to ease my aching elbow. A live demo of what currently works can be found here: http://fossil.wanderinghorse.net/repos/fossil-sgb/json/ -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Current status on hooks?
On Sat, Sep 10, 2011 at 4:49 PM, Gé Weijers g...@weijers.org wrote: I don't like timers like that, a single server may have clients on fast... It's been a while since I looked at the protocol specs, but the client usually maintains (implicit) state in this type of protocol, and could very well indicate to the server that a push operation step is the first one (for a pre-commit hook), last one (for a post-commit hook), or both, which could then trigger the hooks immediately. Downsides: - needs a protocol change - what do you do if the last step never arrives? In term so f invoking the post hook, I think a timer would be needed. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Current status on hooks?
On Sun, Sep 11, 2011 at 12:26 AM, Ron Wilson ronw.m...@gmail.com wrote: In term so f invoking the post hook, I think a timer would be needed. That can't work in CGI mode, though, only in server mode and only if you don't kill the server after committing. :/ One of the big hurdles in implementing triggers, i think, has been platform portability. Now that jimtcl is part of the source tree, if it were integrated into fossil (e.g. as a replacement for the more limited th1), then triggers could be based off of jimtcl and run as part of the commit process both in CGI and server modes. In theory, anyway. -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Current status on hooks?
On Sun, Sep 11, 2011 at 12:54 AM, Stephan Beal sgb...@googlemail.comwrote: One of the big hurdles in implementing triggers, i think, has been platform portability. s/triggers/hooks/g -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] First Fossil/REST proto-demo
Hi Stephan,
I love JSON web services, and I really like the idea you've presented. I've
often wished I could display Fossil ticket reports in a more flexible way, and
the changes you propose would give me that. I look forward to seeing this
progress.
In my experience writing both JSON web services (in Perl), and the consumers of
those services, sometimes you can get output that contains not only a JSON
string, but stray newlines or other (debugging?) output. I'm sure you and
Fossil would never do such a thing, right? Then again, anything's possible
(useful debugging output, ISP-modified content, etc.)
This makes it difficult to parse the JSON string out of the response content.
I have found that surrounding the JSON string with a predefined tag (I've
standardized on TJSON/TJSON; because my company's name starts with T, but
Text-JSON works, too) makes it easy to extract the JSON string from whatever
else might be in the response content.
This also helps deal with either a raw JSON array ([...]) or the more common
object ({...}) without having to test for both patterns.
Also, as part of what I've come to call the TJSON format, the JSON always
includes a status object where a result code and message can be transmitted.
So, in TJSON, the minimum response would always look like:
TJSON{RC:{code:0,msg:Success}, fossil:{...}}/TJSON
This makes it easy to communicate app-level errors/messages because the
resulting object always has the RC property to hold a numeric code, and an
error description string. The consumer can always rely on being able to check
RC.code to see if there was an error (and capture the error string to do with
as he sees fit).
All of the other properties of the JSON object are arbitrary based on the
requirements of the requested service method.
-Clark
From: Stephan Beal sgb...@googlemail.com
To: fossil-users fossil-users@lists.fossil-scm.org
Sent: Thursday, September 8, 2011 7:58 PM
Subject: [fossil-users] First Fossil/REST proto-demo
It's not much, but it's a start:
Here's the code (not counting 10 or 12 lines of infrastructure-level code):
/*
** WEBPAGE: /json
**
*/
void json_page_top(void){
Blob buf = empty_blob;
cson_value * json = NULL;
cgi_set_content_type( cson_cgi_guess_content_type(g.csonCgi) );
json = cson_cgi_env_get_val( g.csonCgi, 'e', 0 ); /* the $ENV vars map */
blob_zero(buf);
cson_output_Blob( json, buf, NULL );
cgi_set_content(buf)/*takes ownership of the buf memory*/;
}
When visiting it with the path /json/a/b/c that generates:
Headers:
1. Cache-control:no-cache, no-store
2. Connection:Keep-Alive
3. Content-Encoding:gzip
4. Content-Length:650
5. Content-Type:application/json; charset=utf-8
6. Date:Fri, 09 Sep 2011 02:49:37 GMT
7. Keep-Alive:timeout=15, max=100
8. Server:Apache/2.2.16 (Ubuntu)
9. Vary:Accept-Encoding
10. X-Frame-Options:SAMEORIGIN
Content:
{ HTTP_HOST:whiki, HTTP_CONNECTION:keep-alive,
HTTP_USER_AGENT:Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.30 (KHTML,
like Gecko) Chrome/12.0.742.124 Safari/534.30,
HTTP_ACCEPT:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,
HTTP_ACCEPT_ENCODING:gzip,deflate,sdch,
HTTP_ACCEPT_LANGUAGE:en-US,en;q=0.8,
HTTP_ACCEPT_CHARSET:ISO-8859-1,utf-8;q=0.7,*;q=0.3,
PATH:/usr/local/bin:/usr/bin:/bin,
SERVER_SIGNATURE:addressApache/2.2.16 (Ubuntu) Server at whiki Port
80/address\n, SERVER_SOFTWARE:Apache/2.2.16 (Ubuntu),
SERVER_NAME:whiki, SERVER_ADDR:127.0.1.1, SERVER_PORT:80,
REMOTE_ADDR:127.0.1.1,
DOCUMENT_ROOT:/home/stephan/cvs/fossil/whiki/website, SERVER_ADMIN:[no
address given],
SCRIPT_FILENAME:/home/stephan/cvs/fossil/whiki/website/cgi-bin/fossil.cgi,
REMOTE_PORT:52331, GATEWAY_INTERFACE:CGI/1.1,
SERVER_PROTOCOL:HTTP/1.1, REQUEST_METHOD:GET,
QUERY_STRING:, REQUEST_URI:/cgi-bin/fossil.cgi/json/a/b/c,
SCRIPT_NAME:/cgi-bin/fossil.cgi, PATH_INFO:/json/a/b/c,
PATH_TRANSLATED:/home/stephan/cvs/fossil/whiki/website/json/a/b/c,
PATH_INFO_SPLIT:[ json, a, b, c ]
}
i might have to fight with fossil a bit over who gets to parse the POST data
from stdin, since fossil will, by default, consume stdin before the JSON CGI
bits get the chance to (and if they do it first then Fossil can't do it). But i
don't need to solve that until i write a request type which needs POST, so that
can wait.
-- - stephan beal
http://wanderinghorse.net/home/stephan/
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] First Fossil/REST proto-demo
On Sun, Sep 11, 2011 at 4:19 AM, Clark Christensen cdcmi...@yahoo.comwrote:
In my experience writing both JSON web services (in Perl), and the
consumers of those services, sometimes you can get output that contains not
only a JSON string, but stray newlines or other (debugging?) output. I'm
sure you and Fossil would never do such a thing, right? Then again,
anything's possible (useful debugging output, ISP-modified content, etc.)
At the moment it's very possible that some of Fossil's console-/html-mode
text slips free, but as i add/refactor the JSON code i'm making sure that no
non-JSON goes to stdout. There might still be places where stderr output
goes, but i _think_ that Apache sends those to the error log. My demo/test
app literally won't work if a given command produces invalid JSON, so it's
unlikely that any non-JSON output will slip through during testing.
This makes it difficult to parse the JSON string out of the response
content. I have
You don't need to parse it yourself (or i never do). Use
JSON.parse(theInput) or equivalent for the client side. e.g. using the C
JSON API it looks like:
cson_value * root = NULL;
int rc = cson_parse_FILE( root, stdin, NULL );
if( 0 == rc ) { /* success */ assert(NULL != root); ... }
The root JSON can be either an array or object, but the fossil API uses only
Objects for the root nodes.
As a client, if the server generates invalid JSON then the client is out of
luck. The best we can do here is make sure that fossil/json emits no
non-JSON.
found that surrounding the JSON string with a predefined tag (I've
standardized on TJSON/TJSON; because my company's name starts with T,
but Text-JSON works, too) makes it easy to extract the JSON string from
whatever else might be in the response content.
But it also makes it invalid JSON. :-?
Also, as part of what I've come to call the TJSON format, the JSON always
includes a status object where a result code and message can be
transmitted. So, in TJSON, the minimum response would always look like:
TJSON{RC:{code:0,msg:Success}, fossil:{...}}/TJSON
That's exactly what i've been doing in the majority of my JSON lib
(including this one) - an outer envelope with result code/text and an inner
payload object with response-specific data.
All of the other properties of the JSON object are arbitrary based on the
requirements of the requested service method.
Right. To avoid muddling up the envelope i've decided for this impl that the
request-specific args always go in the payload property of the envelope,
leaving us free to expand the envelope definition without colliding with
command-specific properties. Because of how the underlying API works, it's
also trivial to pass most request-specific options as GET parameters
(provided they're not too big).
--
- stephan beal
http://wanderinghorse.net/home/stephan/
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] authentication in JSON: anonymous vs. guest user
This question is primarily aimed at Richard, but anyone who's got some insight or opinions is of course free to chime in... As i understand it, the primary intention behind requiring the anonymous user login is to keep spiders from crawling the whole repo history, and the distinction between the two users is that anonymous gets hyperlinks and guest does not. In a JSON context, link-following is not an issue. There are no links, as such, in JSON docs - though individual JSON strings might incidentally contain HTML link strings, bots don't generically try to extract HTML text from JSON. Doing anything at all with the data requires writing an app-specific bot to do it. Given that, would be against fossil's nature if i reduce the JSON API's authentication to only 2 levels: read and write? Non-logged in users would be read-only and logged in would have write access only if their user profile allows it (and if it doesn't then logging in for JSON access doesn't have any benefit at all for the client). As far as i can see so far, the only ops which _need_ to be authenticated (for purposes of a JSON interface) are write-ops, and so far none of those are implemented. Commit, wiki-save, artifact-edit, etc., would be authenticated using the existing per-user permissions. :-? -- - stephan beal http://wanderinghorse.net/home/stephan/ ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
