Re: [fossil-users] Adding binary files to fossil

[Warren Young]

On Jan 3, 2017, at 1:31 PM, Scott Doctor  wrote:
> 
> I added my binary files. Did not get any warning. Should I get a warning if 
> fossil detects that a file is binary?

$ mkdir x
$ cd x
$ f init ../x.fossil
$ f open ../x.fossil 
$ dd if=/dev/random of=x count=1k
1024+0 records in
1024+0 records out
524288 bytes transferred in 0.033915 secs (15458863 bytes/sec)
$ f add x
ADDED  x
$ f ci -m 'a binary file'
./x contains binary data. Use --no-warnings or the "binary-glob" setting to 
disable this warning.
Commit anyhow (a=all/y/N)? 


What does “file my-supposed-binary-file” tell you?  (Presuming you’re not doing 
this on Windows, or you’re on a Windows box that has an implementation of 
file(1), such as via Cygwin or WSL.)

> The files are partial text with a bunch of embedded binary control codes.

Fossil only searches the first N bytes of the file to work out if it is text or 
binary.  It doesn’t scan the whole thing.  If you want the full details, see 
looks_like_utf8() in src/lookslike.c.

(or …utf16() if you’re on Windows.)

> Should I force fossil to recognize a file as being binary

No.  The only reason Fossil cares about this kind of thing at all is to do 
various things convenient to the developer.  (i.e. Show the file data in Fossil 
UI, offer CRLF conversions, etc.)

From a certain perspective, Fossil treats *all* files as binary, which is why 
you can have mixed CRLF line endings in a “text” file and Fossil doesn’t get 
confused.

> On a separate topic:

Next time, start a new thread, please.  They cost nothing, and they keep the 
conversations organized and searchable.

> How do I change a previously added file into an unversioned file?

As far as I know, you can’t.  You have to decide when you check the file in, 
using “fossil unversioned add” instead of “fossil add”.

I wouldn’t recommend using unversioned files in your case anyway, since it 
requires a special checkout command to retrieve them.  It sounds like you want 
them to be part of the default checkout, so...

> an unversioned file will be replaced by a newer version and the older version 
> disappears. Correct?

Yes.  That’s what “unversioned” means.  The alternative would be “versioned.”

If you think you will ever need to roll back to that prior version, such as for 
testing an old shipped version of the system to replicate a reported customer 
problem, then I recommend not using unversioned files.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Adding binary files to fossil

[Warren Young]

On Jan 3, 2017, at 11:08 AM, Scott Doctor  wrote:
> 
> There are a few binary files that are needed to compile and use the program.

If you’d care to share the details about those few binary files, we might be 
able to improve the way Fossil works for you.

Eduard’s point about saving OpenOffice XML docs with ZIP packaging and 
compression turned off is a good example of this.

There are many other cases where a change in the workflow can yield significant 
improvements in the way Fossil works with those binary files.

For example, it may be better to store bitmaps in a file format like Windows 
BMP than TIFF or PNG, because BMP has no compression and carries very little 
metadata compared to TIFF or PNG.  Thus, any changes to the bitmap result in 
only localized changes to the underlying file, which Fossil is better at 
handling than changes to a file format like PNG or JPEG where a single-pixel 
change could statistically change every byte in the file.

If you need a compressed bitmap format for the build, generating it from BMP 
can be part of the build process.

A nice side effect of doing things that way is that improvements in compression 
methodology don’t require that you re-compress and re-checkin all of your 
bitmaps: just update the build process to use the new compression method.

About once every 5 years or so, I find myself re-generating all of my PNGs.  
First there was the move from 8-bit PNGs to matted 24-bit once IE6 finally died 
off.  Then there was the move from matted PNGs to alpha-blended PNGs once IE7 
died off.  Then there was the move from Photoshop’s PNG compression to Zopfli 
compression.  If I’d stored them as RGBA BMPs from the start, I wouldn’t now 
have 3 different versions of substantially the same same bitmaps in my Fossil 
repository today.

Another similar example is storing vector art in Fossil as plain SVG rather 
than SVGZ or some binary format like Adobe Illustrator.

> the contents need to stay as-is binary.

Binary file formats intended for final delivery may be less desirable for 
storing in Fossil than formats intended for manipulation.  The process for 
getting from the one type of binary file format to the other sort can be part 
of the build process.

You don’t necessarily have to store the final delivery form of the data in 
Fossil.  Another form that encodes the same data may make more sense.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Adding binary files to fossil

[Scott Doctor]

I added my binary files. Did not get any warning. Should I get a warning 
if fossil detects that a file is binary? Alloptions are the default 
except for the skin (I like the Blitz no logo skin). The files are 
partial text with a bunch of embedded binary control codes. Should I 
force fossil to recognize a file as being binary (How to do that?)?


On a separate topic:

How do I change a previously added file into an unversioned file? I 
cannot seem to find a command that does that.


Regarding unversioned files. I am a bit unclear how they are handled. 
Usually those files do not change (they are specific to the hardware), 
but they might if the hardware is modified. From what I read, an 
unversioned file will be replaced by a newer version and the older 
version disappears. Correct?


As a side note, I am starting to get the hang of using fossil. The 
ticket/wiki/tech note system is very handy for tracking issues, to-do's, 
and documenting the how-to instructions during development. Much better 
than any other program I tried including Git, Subversion, and Mercurial. 
(I really hate Git).


-
Scott Doctor
sc...@scottdoctor.com
-


___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Adding binary files to fossil

[Eduard]

On 01/03/2017 01:25 PM, Richard Hipp wrote:
> I have entire projects consisting of *only* binary files - for example
> my collection of OpenOffice presentations.
Actually you can use the flat XML representation (simply by using the
.fodt/fodp/fods extension when saving), which turns them into (quite
large) text files, more amenable to merging.

Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Adding binary files to fossil

[Richard Hipp]

On 1/3/17, Martin S. Weber  wrote:
> On 2017-01-03 10:08:40, Scott Doctor wrote:
>> What is the proper way to add binary files to a repository?
>
> Follow the following steps:
>
> (1) (fossil) add the files.
> (2) you're done.
> (3) No, really, you're done.
> (4) Enjoy.

As just one of many examples,
https://www.fossil-scm.org/fossil/artifact/0fa38d60655faf3d in
Fossil's self-hosting repository is a binary file.  There are many
others.

I have entire projects consisting of *only* binary files - for example
my collection of OpenOffice presentations.

The only restriction with binary files is that if you make independent
changes in separate branches, those changes cannot be merged.

>
> fossil will warn you that these files look binary. Read the warning
> and answer accordingly.
>
> With fossil settings you can specify the binary-glob which will allow
> you to circumvent the warning.
>
> With versioned settings (create a .fossil-settings directory in your
> top-level checkout directory and create a, say, binary-glob file in
> it which contains a comma-or-newline separated list of glob patterns
> describing your binaries) you can make sure this setting propagates
> properly to other clones.
>
> cf. (4) above
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>


-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Adding binary files to fossil

[Martin S. Weber]

On 2017-01-03 10:08:40, Scott Doctor wrote:
> What is the proper way to add binary files to a repository?

Follow the following steps:

(1) (fossil) add the files.
(2) you're done. 
(3) No, really, you're done.
(4) Enjoy.

fossil will warn you that these files look binary. Read the warning
and answer accordingly.

With fossil settings you can specify the binary-glob which will allow
you to circumvent the warning.

With versioned settings (create a .fossil-settings directory in your
top-level checkout directory and create a, say, binary-glob file in
it which contains a comma-or-newline separated list of glob patterns
describing your binaries) you can make sure this setting propagates
properly to other clones.

cf. (4) above
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Adding binary files to fossil

[Martin Gagnon]

On Tue, Jan 03, 2017 at 10:08:40AM -0800, Scott Doctor wrote:
> I have a repository with typical C source text files (about 100
> files across several sub-folders). There are a few binary files that
> are needed to compile and use the program. The files are part of the
> program, but the contents need to stay as-is binary. What is the
> proper way to add binary files to a repository? I want the others to
> be able to simply unzip and have the full set of files without
> having to do a separate download and unzip for the binaries.
> 

There's nothing special to do, the binary files will always stay "as-is"
like regular text files. 

I often have some small binary "resources" files in my repositories and
I never had problem with them. If they are not too big and doesn't
change often, the size and speed of the repository should not be
affected.

Regards,

-- 
Martin G.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Adding binary files to fossil

[Scott Doctor]

I have a repository with typical C source text files (about 100 files 
across several sub-folders). There are a few binary files that are 
needed to compile and use the program. The files are part of the 
program, but the contents need to stay as-is binary. What is the proper 
way to add binary files to a repository? I want the others to be able to 
simply unzip and have the full set of files without having to do a 
separate download and unzip for the binaries.


--

-
Scott Doctor
sc...@scottdoctor.com
-

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users