Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On Mon, Mar 27, 2017 at 9:19 AM, Stefan Bellon wrote: > would have to take a backup of the fossil in version 1 state? Or is > there any "go back to version 1 format" emergency way? > fossil 1 has a strict "artifact" format which v2 breaks (in that the hash lengths differ), so v1 cannot read artifacts (checkins, tickets, etc.) which were created in v2, and trying to do so will cause a fatal error for most commands. Thus there is no way back for data generated with an sha3 hash. There "might" (i don't know) be a way to revert up to the point just before where the first sha3 hash was introduced to a repo. -- - stephan beal http://wanderinghorse.net/home/stephan/ "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of those who insist on a perfect world, freedom will have to do." -- Bigby Wolf ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On Mon, 13 Mar, Richard Hipp wrote: > > The upgrade is simple. Just replace your current fossil executable > with a new one. Everything should work exactly as it did before. > There are no incompatibilities. The only difference is that Fossil > 2.1 will read and write the latest repositories whereas Fossil 1.x > will not. Thanks for the encouragement, it turned out that it exactly worked like this. We did not yet find any breakage in command line usage or output. Nice job. :-) However one concern was raised: What, if for some reason, some fossil user has to go back to version 1.x from 2.x. I just did a short test with a "fossil-1 rebuild" on a fossil-2 repository. The only thing that in my quick test survived was the initial empty checkin, but not the following content checkins. Another "fossil-2 rebuild" however brought back the contents. So, do I assume that once you start working with fossil-2 on the repository, you cannot "rebuild back" to version 1, even if you do not (intent to) use new fossil 2 features? So, to be on the safe side one would have to take a backup of the fossil in version 1 state? Or is there any "go back to version 1 format" emergency way? Greetings, Stefan -- Stefan Bellon ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On Wed, Mar 15, 2017 at 1:25 AM, Stephan Beal wrote:
> Yes, it can accept JSON input (even from the CLI). For the full details
> see:
>
> https://docs.google.com/document/d/1fXViveNhDbiXgCuE7QDXQOKeFzf2q
> NUkBEgiUvoqFN4/view
>
> and:
>
> fossil help json
>
> doh - it seems that's missing a reference to the -json-input parameter,
> which expects a filename. i'll get that added.
>
Example:
[stephan@host:~/fossil/fossil]$ cat foo.json
{"command":"HAI"}
[stephan@host:~/fossil/fossil]$ ./fossil json -json-input foo.json
{
"fossil":"2e76b99f5ce5301c4cf82f06d069ec608da888cce543cf8d736d51b110eff6d2",
"timestamp":1489537970,
"command":"HAI",
"procTimeUs":0,
"procTimeMs":0,
"payload":{
"manifestUuid":"2e76b99f5ce5301c4cf82f06d069ec608da888cce543cf8d736d51b110eff6d2",
"manifestVersion":"[2e76b99f5c]",
"manifestDate":"2017-03-15 00:19:41",
"manifestYear":"2017",
"releaseVersion":"2.1",
"releaseVersionNumber":20100,
"resultCodeParanoiaLevel":0,
"jsonApiVersion":"20120713"
}
}
--
- stephan beal
http://wanderinghorse.net/home/stephan/
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On Tue, Mar 14, 2017 at 11:06 PM, Stefan Bellon wrote: > On Mon, 13 Mar, Ross Berteig wrote: > > > When fossil is configured to include JSON support it has a fossil > > json command that consumes a JSON object full of command and > > arguments and returns a JSON object of results. A similar feature is > > available over HTTP(S) through URLs beginning with /json. > > Ah, that's why I never got JSON to work. The input to the JSON support > is via JSON object as well?! I always assumed you can just use ordinary > command line and specify something like --json to tell fossil to output > the result of the command as a JSON object. > Yes, it can accept JSON input (even from the CLI). For the full details see: https://docs.google.com/document/d/1fXViveNhDbiXgCuE7QDXQOKeFzf2qNUkBEgiUvoqFN4/view and: fossil help json doh - it seems that's missing a reference to the -json-input parameter, which expects a filename. i'll get that added. -- - stephan beal http://wanderinghorse.net/home/stephan/ "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of those who insist on a perfect world, freedom will have to do." -- Bigby Wolf ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On 3/14/2017 3:06 PM, Stefan Bellon wrote:
On Mon, 13 Mar, Ross Berteig wrote:
When fossil is configured to include JSON support it has a fossil
json command that consumes a JSON object full of command and
arguments and returns a JSON object of results. A similar feature is
available over HTTP(S) through URLs beginning with /json.
Ah, that's why I never got JSON to work. The input to the JSON support
is via JSON object as well?! I always assumed you can just use ordinary
command line and specify something like --json to tell fossil to output
the result of the command as a JSON object.
The JSON API originated as a JSON over HTTP, but a mapping between URLs
and the command line made sense for debugging. So it can be used through
the fossil json command by a fairly direct mapping from /json... URLs
used for GET and POST. HTTP POST allows the submission of a command
packet rather than fitting everything into the URL.POST data can also be
simulated from the command line, from either a file or stdin.
The output is a JSON object that contains metadata about the fossil
instance and the command success or failure, as well as the result of
the command.
The documentation is not rolled into the fossil documentation as yet.
You can find it on Google Docs: https://goo.gl/Iv8OFL
The original use cases were all of the web app flavor, hence the
emphasis on HTTP in the docs, with the CLI support more of an
after-thought. I suspect that is why a --json global option was not
considered at all.
Here's a sample of output for the timeline, limited to the latest two
checkins:
C:...>fossil json timeline checkin --limit 2
{
"fossil":"38115a0bd96c3788e36506b965f938bd31f0d0b65cf26dd7866abefb80a9a15d",
"timestamp":1489530880,
"command":"timeline/checkin",
"procTimeUs":0,
"procTimeMs":0,
"payload":{
"limit":2,
"timeline":[
{
"type":"checkin",
"uuid":"38115a0bd96c3788e36506b965f938bd31f0d0b65cf26dd7866abefb80a9a15d",
"isLeaf":true,
"timestamp":1489526151,
"user":"rberteig",
"comment":"Update hash regexp to defend
against full length SHA3 hashes.",
"parents":["f92672bda33bf86752053272ef1f7aa9de088e823b9a864877042c2b8802800b"],
"tags":["trunk"]
},
{
"type":"checkin",
"uuid":"f92672bda33bf86752053272ef1f7aa9de088e823b9a864877042c2b8802800b",
"isLeaf":false,
"timestamp":1489522870,
"user":"rberteig",
"comment":"New test case for the
Markdown bug, fixed issues with hash matching, fixed tests for the
TH1 unversioned command so they no longer depend on unversioned files
from a past release in the real repository.",
"parents":["dd41f85acf57290aba64c1f42ddeafcdf80432df80a07cea699bbb7456ff1386"],
"tags":["trunk"]
}
]
}
}
You'll notice that the SHA3 hashes simply slot in place where SHA1 had
been without the format changing.
Is there a reason that the default builds as available for download
have JSON support not enabled?
We've discussed this in the past, and there hasn't been a strong
advocate for enabling it by default. It does increase the attack surface
for fossil as CGI to some degree. Until I started writing test cases for
it, it wasn't covered at all by the internal test suite. I know the lack
of testing was a big concern. I've increased its coverage from "none" to
"some", but I can't claim that the JSON support is well covered by the
tests yet. Most features only have tests that confirm the feature is
present and got a correct answer, with no attempt to probe at edges and
errors.
Maybe it is time to bring that discussion back up.
Also, not every command is available via JSON, and I suspect that
there is no JSON equivalent to fossil annotate. If we can figure out
how to represent the annotate/blame/praise data in JSON, then adding
it should be possible.
I can tell you how JSON output of IBM Rational Team Concert (which is a
complete mess, but has wonderful JSON output) looks like for the
annotate command for a specific file:
{
"annotations": [
{
"author": "author of commit changing line 1 last",
"comment": "commit message of commit changing line 1 last",
"line": "file content of line 1",
"line-no": "1",
"modified": "timestamp of commit changing line 1 last",
"uuid": "revision hash of commit changing line 1 last",
},
{
"author": "author of commit changing line 2 last",
"comment":
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On Mon, 13 Mar, Ross Berteig wrote:
> When fossil is configured to include JSON support it has a fossil
> json command that consumes a JSON object full of command and
> arguments and returns a JSON object of results. A similar feature is
> available over HTTP(S) through URLs beginning with /json.
Ah, that's why I never got JSON to work. The input to the JSON support
is via JSON object as well?! I always assumed you can just use ordinary
command line and specify something like --json to tell fossil to output
the result of the command as a JSON object.
> Unfortunately JSON support is not configured in the default builds of
> fossil. I often turn it on for my personal builds, because I have
> some scripts that draw from the ticket system for report writing, and
> the JSON support for reading and querying tickets works quite nicely.
> I'd personally like to see the JSON support trusted enough to be on
> by default.
Is there a reason that the default builds as available for download
have JSON support not enabled?
> Also, not every command is available via JSON, and I suspect that
> there is no JSON equivalent to fossil annotate. If we can figure out
> how to represent the annotate/blame/praise data in JSON, then adding
> it should be possible.
I can tell you how JSON output of IBM Rational Team Concert (which is a
complete mess, but has wonderful JSON output) looks like for the
annotate command for a specific file:
{
"annotations": [
{
"author": "author of commit changing line 1 last",
"comment": "commit message of commit changing line 1 last",
"line": "file content of line 1",
"line-no": "1",
"modified": "timestamp of commit changing line 1 last",
"uuid": "revision hash of commit changing line 1 last",
},
{
"author": "author of commit changing line 2 last",
"comment": "commit message of commit changing line 2 last",
"line": "file content of line 2",
"line-no": "2",
"modified": "timestamp of commit changing line 2 last",
"uuid": "revision hash of commit changing line 2 last",
},
{
"author": "author of commit changing line 3 last",
"comment": "commit message of commit changing line 3 last",
"line": "file content of line 3",
"line-no": "3",
"modified": "timestamp of commit changing line 3 last",
"uuid": "revision hash of commit changing line 3 last",
},
...
]
}
I hope this helps a bit.
Greetings,
Stefan
--
Stefan Bellon
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On Mon, 13 Mar, Richard Hipp wrote: > I'm a little confused. If you need to continue using Fossil 1.29 for > text output compatibility, then why is it a problem that you cannot > "fossil update" to the latest trunk? After all, you won't be using > the latest trunk, right? Ha, right, that's a little bit complex, too. ;-) There are two use-cases of fossil for us. 1) We use fossil as a SCM for generated files during software build processes, i.e. source code that is not stored in the primary version control system at customer site in order to retrieve generated files consistent to a certain state of source code. 2) We are using fossil trunk source code itself to run our static code analyses against - kind of regression test. For #1 we still use fossil 1.29 because of the command line compatibility issue. For #2 we used fossil trunk up until now, but this gave us the issue initially brought up with my posting. > Also: What can we do to help you move away from scripts that depend > on the details of command-line output and toward something that is > more likely to survive an update? Indeed something like fossil timeline --json ... fossil status --json ... fossil ls --json ... fossil blame --json ... where we can rely on the output in the future. > What are your scripts doing? We are using the following commands (for various scenarios): timeline, update and pull (in order to sync to a certain repository state), status and ls (in order to find out which file is in which revision in the working tree), cat (to retrieve a file in a specific version), blame (to find out which author changed which line in a file last), clone and remote (in order to initially set up a repository clone automatically), init, settings and open (in order to create a repository and put it in a specific usable state), user, addremove and commit (in order to store the latest batch of generated files in the repository). > Would it be better to run SQL queries directly against the repository > database? I fear that the SQL schema changes more often than the command line. :-/ > Are new fixed-output-format commands needed in Fossil to > extract information that is important to scripts? It indeed would be nice to have a fixed-output-format, defined once and used (in a compatible way) in the future. This could be very well JSON. Greetings, Stefan -- Stefan Bellon ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On 3/13/2017 8:57 AM, Richard Hipp wrote: On 3/13/17, Stefan Bellon wrote: When upgrading from 1.24 to 1.29 years ago, the output of "fossil annotate" changed and broke all our scripts (and even some customer databases). I expect even more incompatibilities when switching from 1.29 to 2.0 and therefore this is nothing that can be done in a few minutes. We have to check for all the output of all fossil commands we use in scripts whether they still produce the same output they did before. I'm a little confused. If you need to continue using Fossil 1.29 for text output compatibility, then why is it a problem that you cannot "fossil update" to the latest trunk? After all, you won't be using the latest trunk, right? Also: What can we do to help you move away from scripts that depend on the details of command-line output and toward something that is more likely to survive an update? What are your scripts doing? Would it be better to run SQL queries directly against the repository database? Are new fixed-output-format commands needed in Fossil to extract information that is important to scripts? When fossil is configured to include JSON support it has a fossil json command that consumes a JSON object full of command and arguments and returns a JSON object of results. A similar feature is available over HTTP(S) through URLs beginning with /json. Unfortunately JSON support is not configured in the default builds of fossil. I often turn it on for my personal builds, because I have some scripts that draw from the ticket system for report writing, and the JSON support for reading and querying tickets works quite nicely. I'd personally like to see the JSON support trusted enough to be on by default. Also, not every command is available via JSON, and I suspect that there is no JSON equivalent to fossil annotate. If we can figure out how to represent the annotate/blame/praise data in JSON, then adding it should be possible. -- Ross Berteig r...@cheshireeng.com Cheshire Engineering Corp. http://www.CheshireEng.com/ +1 626 303 1602 ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On 3/13/17, Stefan Bellon wrote: > > When upgrading from 1.24 to 1.29 years ago, the output of "fossil > annotate" changed and broke all our scripts (and even some customer > databases). > > I expect even more incompatibilities when switching from 1.29 to 2.0 > and therefore this is nothing that can be done in a few minutes. > > We have to check for all the output of all fossil commands we use in > scripts whether they still produce the same output they did before. > I'm a little confused. If you need to continue using Fossil 1.29 for text output compatibility, then why is it a problem that you cannot "fossil update" to the latest trunk? After all, you won't be using the latest trunk, right? Also: What can we do to help you move away from scripts that depend on the details of command-line output and toward something that is more likely to survive an update? What are your scripts doing? Would it be better to run SQL queries directly against the repository database? Are new fixed-output-format commands needed in Fossil to extract information that is important to scripts? -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
Sadly it is not that simple. When upgrading from 1.24 to 1.29 years ago, the output of "fossil annotate" changed and broke all our scripts (and even some customer databases). I expect even more incompatibilities when switching from 1.29 to 2.0 and therefore this is nothing that can be done in a few minutes. We have to check for all the output of all fossil commands we use in scripts whether they still produce the same output they did before. But ok, then we know why our automated tests do not work any more. Greetings, Stefan On Mon, 13 Mar, Richard Hipp wrote: > On 3/13/17, Stefan Bellon wrote: > > > > Is this (old) version not able to work with current trunk anymore? > > Do we have to upgrade to 2.0 or 2.1? > > > > Correct. As of check-in > https://www.fossil-scm.org/fossil/info/7815d015f3b5a663 (2017-03-10) > the self-hosting Fossil repository now contains artifacts named by > SHA3 hashes, and Fossil version 2.0 or later is necessary to > understand those now hashes. > > The upgrade is simple. Just replace your current fossil executable > with a new one. Everything should work exactly as it did before. > There are no incompatibilities. The only difference is that Fossil > 2.1 will read and write the latest repositories whereas Fossil 1.x > will not. > > -- > D. Richard Hipp > d...@sqlite.org -- Stefan Bellon ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Old fossil (1.29) accessing current fossil trunk?
On 3/13/17, Stefan Bellon wrote: > > Is this (old) version not able to work with current trunk anymore? Do > we have to upgrade to 2.0 or 2.1? > Correct. As of check-in https://www.fossil-scm.org/fossil/info/7815d015f3b5a663 (2017-03-10) the self-hosting Fossil repository now contains artifacts named by SHA3 hashes, and Fossil version 2.0 or later is necessary to understand those now hashes. The upgrade is simple. Just replace your current fossil executable with a new one. Everything should work exactly as it did before. There are no incompatibilities. The only difference is that Fossil 2.1 will read and write the latest repositories whereas Fossil 1.x will not. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Old fossil (1.29) accessing current fossil trunk?
Hi all, recently when updating our fossil trunk workspace with an old copy of fossil (1.29) we are getting error messages as follows: $ fossil version This is fossil version 1.29 [3e5ebe2b90] 2014-06-12 17:25:56 UTC $ fossil update Autosync: http://www.fossil-scm.org/ Round-trips: 1 Artifacts sent: 0 received: 0 unknown command: [igot] Round-trips: 1 Artifacts sent: 0 received: 0 Pull finished with 368 bytes sent, 1088 bytes received Autosync failed Cannot proceed with update Is this (old) version not able to work with current trunk anymore? Do we have to upgrade to 2.0 or 2.1? Greetings, Stefan -- Stefan Bellon ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
