Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Clemens Koller]

Hello!

On 30/06/2020 18.46, Jon Evans wrote:
> Give this issue a thumbs up: https://gitlab.com/kicad/code/kicad/-/issues/4763

Just some additions to this issue's desired features:

> 1. Capture the entire project state into a single backup archive (.zip file)

...including a timestamp in the .zip's or folder's name (outside of your VCS' 
visibility).

> 2. Configurable (turn the entire feature on or off, and control how many 
> backups to keep
> 3. Store backups in a configurable location (default: a subdirectory of the 
> project, like ./projectname-backups/)
> 4. Automatically save backups at some configurable interval (every N hours or 
> days, maybe) -- NOT every time you save a file

... every minute(s)!
In the case KiCad crashes I can hardly remember the last 10 minutes of work I 
did.
Additionally: It might useful to save on each CreateUndoEntry to be able to 
reconstruct the a crash for debug purposes.

> 5. A feature (in project manager, probably) to restore a project's state from 
> a given backup archive

Thanks,

Clemens



> 
> On Tue, Jun 30, 2020 at 12:21 PM Diego Herranz
>  wrote:
>>
>> This possibility to save the last N backups would be useful. It is what 
>> Altium does more or less and it has proven useful to me in the past. 
>> Especially if you change your mind after 2h working on something which is 
>> not going anywhere :)
>>
>> Thanks,
>> Diego
>>
>> On Tue, 30 Jun 2020 at 15:15, Wayne Stambaugh  wrote:
>>>
>>> Sounds like a great solution to me.
>>>
>>> On 6/30/20 10:12 AM, Jon Evans wrote:
 JP, I agree that true backups are useful.

 Maybe even it is a good idea for KiCad to have a built-in backup function.

 I just don't think the current backup function is actually useful
 because of my first point (backups are overwritten on each save).

 I would propose:

 1) Remove the current backup file generation

 2) Create a spec (in GitLab issue) for a better backup system that:

 - Can be turned on or off
 - Backs up the whole project in a zip file
 - Can keep the last N backups
 - Runs on a schedule, not necessarily every time you click Save.

 Anyone opposed to this?

 -Jon

 On Tue, Jun 30, 2020 at 3:32 AM jp charras  wrote:
>
> Le 30/06/2020 à 00:13, hauptmech a écrit :
>>
>> While I agree that it is not KiCad's job to do archival backups or 
>> version control, I do think that KiCad should preserve the integrity of 
>> users data through a crash. Even better if the work between the last 
>> save and the crash is also preserved and recovered on restart.
>>
>> I have had to use the backup files to recover data in the past. I have 
>> no idea if that recovery was related to something that is now no longer 
>> a possible issue.
>>
>>
>> -Hauptmech
>>
>>
>
> I am also thinking a backup can be useful when something unexpected 
> happens.
>
> Backups, like any security system, bothers you as long as you do not need 
> to use them.
> But you are happy to find them in case of trouble.
>
> I like the way some CAD tools manage backup:
> only one zip archive is created (for instance projectname_backup.zip) and 
> contains all saved files
> (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
> This is not a full project backup, just main files are saved.
>
> This is not invasive (only one file, or a few .zip if one want to keep 
> last n saved versions)
> and is a security against  unexpected cases.
>
> For me, backups are like a accident insurance: you need them and you hope 
> never use them.
>
> And about VCS use:
>
> Many good electronics guys do not even know what is it, and have never 
> compiled any source code.
> Electronics world and Software world are not exactly the same world.
>
> --
> Jean-Pierre CHARRAS
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp

 ___
 Mailing list: https://launchpad.net/~kicad-developers
 Post to : kicad-developers@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~kicad-developers
 More help   : https://help.launchpad.net/ListHelp

>>>
>>> ___
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to : kicad-developers@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>>
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : 

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Oleg Endo]

On Tue, 2020-06-30 at 09:28 +0200, jp charras wrote:
> 
> For me, backups are like a accident insurance: you need them and you hope 
> never use them.
> 
> And about VCS use:
> 
> Many good electronics guys do not even know what is it, and have never 
> compiled any source code.
> Electronics world and Software world are not exactly the same world.

This is unfortunately true.  It's also true that other commercial EDA
solutions are now moving towards providing built-in GIT support.  It's
inevitable -- the benefits are too obvious.  The VCS is the backup.  If
not, chances are it's being used wrongly.

There are also "software developers" out there who still use zip files
as a backup & VCS system.  And probably there are people still doing
dot & tape board layout... 

The world is changing.  Hopefully for the better :)

Cheers,
Oleg


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Jon Evans]

Give this issue a thumbs up: https://gitlab.com/kicad/code/kicad/-/issues/4763

On Tue, Jun 30, 2020 at 12:21 PM Diego Herranz
 wrote:
>
> This possibility to save the last N backups would be useful. It is what 
> Altium does more or less and it has proven useful to me in the past. 
> Especially if you change your mind after 2h working on something which is not 
> going anywhere :)
>
> Thanks,
> Diego
>
> On Tue, 30 Jun 2020 at 15:15, Wayne Stambaugh  wrote:
>>
>> Sounds like a great solution to me.
>>
>> On 6/30/20 10:12 AM, Jon Evans wrote:
>> > JP, I agree that true backups are useful.
>> >
>> > Maybe even it is a good idea for KiCad to have a built-in backup function.
>> >
>> > I just don't think the current backup function is actually useful
>> > because of my first point (backups are overwritten on each save).
>> >
>> > I would propose:
>> >
>> > 1) Remove the current backup file generation
>> >
>> > 2) Create a spec (in GitLab issue) for a better backup system that:
>> >
>> > - Can be turned on or off
>> > - Backs up the whole project in a zip file
>> > - Can keep the last N backups
>> > - Runs on a schedule, not necessarily every time you click Save.
>> >
>> > Anyone opposed to this?
>> >
>> > -Jon
>> >
>> > On Tue, Jun 30, 2020 at 3:32 AM jp charras  wrote:
>> >>
>> >> Le 30/06/2020 à 00:13, hauptmech a écrit :
>> >>>
>> >>> While I agree that it is not KiCad's job to do archival backups or 
>> >>> version control, I do think that KiCad should preserve the integrity of 
>> >>> users data through a crash. Even better if the work between the last 
>> >>> save and the crash is also preserved and recovered on restart.
>> >>>
>> >>> I have had to use the backup files to recover data in the past. I have 
>> >>> no idea if that recovery was related to something that is now no longer 
>> >>> a possible issue.
>> >>>
>> >>>
>> >>> -Hauptmech
>> >>>
>> >>>
>> >>
>> >> I am also thinking a backup can be useful when something unexpected 
>> >> happens.
>> >>
>> >> Backups, like any security system, bothers you as long as you do not need 
>> >> to use them.
>> >> But you are happy to find them in case of trouble.
>> >>
>> >> I like the way some CAD tools manage backup:
>> >> only one zip archive is created (for instance projectname_backup.zip) and 
>> >> contains all saved files
>> >> (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
>> >> This is not a full project backup, just main files are saved.
>> >>
>> >> This is not invasive (only one file, or a few .zip if one want to keep 
>> >> last n saved versions)
>> >> and is a security against  unexpected cases.
>> >>
>> >> For me, backups are like a accident insurance: you need them and you hope 
>> >> never use them.
>> >>
>> >> And about VCS use:
>> >>
>> >> Many good electronics guys do not even know what is it, and have never 
>> >> compiled any source code.
>> >> Electronics world and Software world are not exactly the same world.
>> >>
>> >> --
>> >> Jean-Pierre CHARRAS
>> >>
>> >> ___
>> >> Mailing list: https://launchpad.net/~kicad-developers
>> >> Post to : kicad-developers@lists.launchpad.net
>> >> Unsubscribe : https://launchpad.net/~kicad-developers
>> >> More help   : https://help.launchpad.net/ListHelp
>> >
>> > ___
>> > Mailing list: https://launchpad.net/~kicad-developers
>> > Post to : kicad-developers@lists.launchpad.net
>> > Unsubscribe : https://launchpad.net/~kicad-developers
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>>
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Ian McInerney]

Yes, it will use the system file rename command to rename the temp saved
file. If that system command fails, then wx will automatically attempt to
copy the temp saved file instead, and that will preserve the temp saved
file if it fails (so basically, the temp saved file will only be removed if
the operation succeeds).

-Ian

On Tue, Jun 30, 2020 at 5:07 PM Andrew Lutsenko 
wrote:

> Ah, I reread now what the change to save logic was and I agree, the main
> reason for the current backup system is now irrelevant, assuming the last
> step of copying over the file is done using system functions and not by
> opening and writing the file.
>
> On Tue, Jun 30, 2020 at 8:15 AM Jon Evans  wrote:
>
>> What is the scenario that they guard against today?  I claim that they
>> don't do this anymore.
>>
>> Scenario 1:
>>
>> 1) Save, which succeeds
>> 2) Do some work, but don't hit save
>> 3) KiCad crashes
>>
>> Result: data after last save is lost
>>
>> Scenario 2:
>>
>> 1) Save, which succeeds
>> 2) Do some work, hit save
>> 3) KiCad crashes while saving file
>>
>> Result: The same as Scenario 1, since my recent change to save behavior.
>>
>> -Jon
>>
>> On Tue, Jun 30, 2020 at 11:09 AM Andrew Lutsenko 
>> wrote:
>> >
>> > I don't think removing the current backup system before implementing a
>> new one is the right thing to do.
>> > As limited and simple as it is, the current system provides valuable
>> safeguard against data loss on crashes that may corrupt the main save file.
>> And no, VCS is not a replacement, most people hit ctrl-s a lot more often
>> than they do "git commit".
>> >
>> >
>> > On Tue, Jun 30, 2020 at 7:13 AM Jon Evans  wrote:
>> >>
>> >> JP, I agree that true backups are useful.
>> >>
>> >> Maybe even it is a good idea for KiCad to have a built-in backup
>> function.
>> >>
>> >> I just don't think the current backup function is actually useful
>> >> because of my first point (backups are overwritten on each save).
>> >>
>> >> I would propose:
>> >>
>> >> 1) Remove the current backup file generation
>> >>
>> >> 2) Create a spec (in GitLab issue) for a better backup system that:
>> >>
>> >> - Can be turned on or off
>> >> - Backs up the whole project in a zip file
>> >> - Can keep the last N backups
>> >> - Runs on a schedule, not necessarily every time you click Save.
>> >>
>> >> Anyone opposed to this?
>> >>
>> >> -Jon
>> >>
>> >> On Tue, Jun 30, 2020 at 3:32 AM jp charras 
>> wrote:
>> >> >
>> >> > Le 30/06/2020 à 00:13, hauptmech a écrit :
>> >> > >
>> >> > > While I agree that it is not KiCad's job to do archival backups or
>> version control, I do think that KiCad should preserve the integrity of
>> users data through a crash. Even better if the work between the last save
>> and the crash is also preserved and recovered on restart.
>> >> > >
>> >> > > I have had to use the backup files to recover data in the past. I
>> have no idea if that recovery was related to something that is now no
>> longer a possible issue.
>> >> > >
>> >> > >
>> >> > > -Hauptmech
>> >> > >
>> >> > >
>> >> >
>> >> > I am also thinking a backup can be useful when something unexpected
>> happens.
>> >> >
>> >> > Backups, like any security system, bothers you as long as you do not
>> need to use them.
>> >> > But you are happy to find them in case of trouble.
>> >> >
>> >> > I like the way some CAD tools manage backup:
>> >> > only one zip archive is created (for instance
>> projectname_backup.zip) and contains all saved files
>> >> > (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
>> >> > This is not a full project backup, just main files are saved.
>> >> >
>> >> > This is not invasive (only one file, or a few .zip if one want to
>> keep last n saved versions)
>> >> > and is a security against  unexpected cases.
>> >> >
>> >> > For me, backups are like a accident insurance: you need them and you
>> hope never use them.
>> >> >
>> >> > And about VCS use:
>> >> >
>> >> > Many good electronics guys do not even know what is it, and have
>> never compiled any source code.
>> >> > Electronics world and Software world are not exactly the same world.
>> >> >
>> >> > --
>> >> > Jean-Pierre CHARRAS
>> >> >
>> >> > ___
>> >> > Mailing list: https://launchpad.net/~kicad-developers
>> >> > Post to : kicad-developers@lists.launchpad.net
>> >> > Unsubscribe : https://launchpad.net/~kicad-developers
>> >> > More help   : https://help.launchpad.net/ListHelp
>> >>
>> >> ___
>> >> Mailing list: https://launchpad.net/~kicad-developers
>> >> Post to : kicad-developers@lists.launchpad.net
>> >> Unsubscribe : https://launchpad.net/~kicad-developers
>> >> More help   : https://help.launchpad.net/ListHelp
>>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : 

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Diego Herranz]

This possibility to save the last N backups would be useful. It is what
Altium does more or less and it has proven useful to me in the past.
Especially if you change your mind after 2h working on something which is
not going anywhere :)

Thanks,
Diego

On Tue, 30 Jun 2020 at 15:15, Wayne Stambaugh  wrote:

> Sounds like a great solution to me.
>
> On 6/30/20 10:12 AM, Jon Evans wrote:
> > JP, I agree that true backups are useful.
> >
> > Maybe even it is a good idea for KiCad to have a built-in backup
> function.
> >
> > I just don't think the current backup function is actually useful
> > because of my first point (backups are overwritten on each save).
> >
> > I would propose:
> >
> > 1) Remove the current backup file generation
> >
> > 2) Create a spec (in GitLab issue) for a better backup system that:
> >
> > - Can be turned on or off
> > - Backs up the whole project in a zip file
> > - Can keep the last N backups
> > - Runs on a schedule, not necessarily every time you click Save.
> >
> > Anyone opposed to this?
> >
> > -Jon
> >
> > On Tue, Jun 30, 2020 at 3:32 AM jp charras 
> wrote:
> >>
> >> Le 30/06/2020 à 00:13, hauptmech a écrit :
> >>>
> >>> While I agree that it is not KiCad's job to do archival backups or
> version control, I do think that KiCad should preserve the integrity of
> users data through a crash. Even better if the work between the last save
> and the crash is also preserved and recovered on restart.
> >>>
> >>> I have had to use the backup files to recover data in the past. I have
> no idea if that recovery was related to something that is now no longer a
> possible issue.
> >>>
> >>>
> >>> -Hauptmech
> >>>
> >>>
> >>
> >> I am also thinking a backup can be useful when something unexpected
> happens.
> >>
> >> Backups, like any security system, bothers you as long as you do not
> need to use them.
> >> But you are happy to find them in case of trouble.
> >>
> >> I like the way some CAD tools manage backup:
> >> only one zip archive is created (for instance projectname_backup.zip)
> and contains all saved files
> >> (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
> >> This is not a full project backup, just main files are saved.
> >>
> >> This is not invasive (only one file, or a few .zip if one want to keep
> last n saved versions)
> >> and is a security against  unexpected cases.
> >>
> >> For me, backups are like a accident insurance: you need them and you
> hope never use them.
> >>
> >> And about VCS use:
> >>
> >> Many good electronics guys do not even know what is it, and have never
> compiled any source code.
> >> Electronics world and Software world are not exactly the same world.
> >>
> >> --
> >> Jean-Pierre CHARRAS
> >>
> >> ___
> >> Mailing list: https://launchpad.net/~kicad-developers
> >> Post to : kicad-developers@lists.launchpad.net
> >> Unsubscribe : https://launchpad.net/~kicad-developers
> >> More help   : https://help.launchpad.net/ListHelp
> >
> > ___
> > Mailing list: https://launchpad.net/~kicad-developers
> > Post to : kicad-developers@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~kicad-developers
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Andrew Lutsenko]

Ah, I reread now what the change to save logic was and I agree, the main
reason for the current backup system is now irrelevant, assuming the last
step of copying over the file is done using system functions and not by
opening and writing the file.

On Tue, Jun 30, 2020 at 8:15 AM Jon Evans  wrote:

> What is the scenario that they guard against today?  I claim that they
> don't do this anymore.
>
> Scenario 1:
>
> 1) Save, which succeeds
> 2) Do some work, but don't hit save
> 3) KiCad crashes
>
> Result: data after last save is lost
>
> Scenario 2:
>
> 1) Save, which succeeds
> 2) Do some work, hit save
> 3) KiCad crashes while saving file
>
> Result: The same as Scenario 1, since my recent change to save behavior.
>
> -Jon
>
> On Tue, Jun 30, 2020 at 11:09 AM Andrew Lutsenko 
> wrote:
> >
> > I don't think removing the current backup system before implementing a
> new one is the right thing to do.
> > As limited and simple as it is, the current system provides valuable
> safeguard against data loss on crashes that may corrupt the main save file.
> And no, VCS is not a replacement, most people hit ctrl-s a lot more often
> than they do "git commit".
> >
> >
> > On Tue, Jun 30, 2020 at 7:13 AM Jon Evans  wrote:
> >>
> >> JP, I agree that true backups are useful.
> >>
> >> Maybe even it is a good idea for KiCad to have a built-in backup
> function.
> >>
> >> I just don't think the current backup function is actually useful
> >> because of my first point (backups are overwritten on each save).
> >>
> >> I would propose:
> >>
> >> 1) Remove the current backup file generation
> >>
> >> 2) Create a spec (in GitLab issue) for a better backup system that:
> >>
> >> - Can be turned on or off
> >> - Backs up the whole project in a zip file
> >> - Can keep the last N backups
> >> - Runs on a schedule, not necessarily every time you click Save.
> >>
> >> Anyone opposed to this?
> >>
> >> -Jon
> >>
> >> On Tue, Jun 30, 2020 at 3:32 AM jp charras 
> wrote:
> >> >
> >> > Le 30/06/2020 à 00:13, hauptmech a écrit :
> >> > >
> >> > > While I agree that it is not KiCad's job to do archival backups or
> version control, I do think that KiCad should preserve the integrity of
> users data through a crash. Even better if the work between the last save
> and the crash is also preserved and recovered on restart.
> >> > >
> >> > > I have had to use the backup files to recover data in the past. I
> have no idea if that recovery was related to something that is now no
> longer a possible issue.
> >> > >
> >> > >
> >> > > -Hauptmech
> >> > >
> >> > >
> >> >
> >> > I am also thinking a backup can be useful when something unexpected
> happens.
> >> >
> >> > Backups, like any security system, bothers you as long as you do not
> need to use them.
> >> > But you are happy to find them in case of trouble.
> >> >
> >> > I like the way some CAD tools manage backup:
> >> > only one zip archive is created (for instance projectname_backup.zip)
> and contains all saved files
> >> > (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
> >> > This is not a full project backup, just main files are saved.
> >> >
> >> > This is not invasive (only one file, or a few .zip if one want to
> keep last n saved versions)
> >> > and is a security against  unexpected cases.
> >> >
> >> > For me, backups are like a accident insurance: you need them and you
> hope never use them.
> >> >
> >> > And about VCS use:
> >> >
> >> > Many good electronics guys do not even know what is it, and have
> never compiled any source code.
> >> > Electronics world and Software world are not exactly the same world.
> >> >
> >> > --
> >> > Jean-Pierre CHARRAS
> >> >
> >> > ___
> >> > Mailing list: https://launchpad.net/~kicad-developers
> >> > Post to : kicad-developers@lists.launchpad.net
> >> > Unsubscribe : https://launchpad.net/~kicad-developers
> >> > More help   : https://help.launchpad.net/ListHelp
> >>
> >> ___
> >> Mailing list: https://launchpad.net/~kicad-developers
> >> Post to : kicad-developers@lists.launchpad.net
> >> Unsubscribe : https://launchpad.net/~kicad-developers
> >> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Jon Evans]

What is the scenario that they guard against today?  I claim that they
don't do this anymore.

Scenario 1:

1) Save, which succeeds
2) Do some work, but don't hit save
3) KiCad crashes

Result: data after last save is lost

Scenario 2:

1) Save, which succeeds
2) Do some work, hit save
3) KiCad crashes while saving file

Result: The same as Scenario 1, since my recent change to save behavior.

-Jon

On Tue, Jun 30, 2020 at 11:09 AM Andrew Lutsenko  wrote:
>
> I don't think removing the current backup system before implementing a new 
> one is the right thing to do.
> As limited and simple as it is, the current system provides valuable 
> safeguard against data loss on crashes that may corrupt the main save file. 
> And no, VCS is not a replacement, most people hit ctrl-s a lot more often 
> than they do "git commit".
>
>
> On Tue, Jun 30, 2020 at 7:13 AM Jon Evans  wrote:
>>
>> JP, I agree that true backups are useful.
>>
>> Maybe even it is a good idea for KiCad to have a built-in backup function.
>>
>> I just don't think the current backup function is actually useful
>> because of my first point (backups are overwritten on each save).
>>
>> I would propose:
>>
>> 1) Remove the current backup file generation
>>
>> 2) Create a spec (in GitLab issue) for a better backup system that:
>>
>> - Can be turned on or off
>> - Backs up the whole project in a zip file
>> - Can keep the last N backups
>> - Runs on a schedule, not necessarily every time you click Save.
>>
>> Anyone opposed to this?
>>
>> -Jon
>>
>> On Tue, Jun 30, 2020 at 3:32 AM jp charras  wrote:
>> >
>> > Le 30/06/2020 à 00:13, hauptmech a écrit :
>> > >
>> > > While I agree that it is not KiCad's job to do archival backups or 
>> > > version control, I do think that KiCad should preserve the integrity of 
>> > > users data through a crash. Even better if the work between the last 
>> > > save and the crash is also preserved and recovered on restart.
>> > >
>> > > I have had to use the backup files to recover data in the past. I have 
>> > > no idea if that recovery was related to something that is now no longer 
>> > > a possible issue.
>> > >
>> > >
>> > > -Hauptmech
>> > >
>> > >
>> >
>> > I am also thinking a backup can be useful when something unexpected 
>> > happens.
>> >
>> > Backups, like any security system, bothers you as long as you do not need 
>> > to use them.
>> > But you are happy to find them in case of trouble.
>> >
>> > I like the way some CAD tools manage backup:
>> > only one zip archive is created (for instance projectname_backup.zip) and 
>> > contains all saved files
>> > (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
>> > This is not a full project backup, just main files are saved.
>> >
>> > This is not invasive (only one file, or a few .zip if one want to keep 
>> > last n saved versions)
>> > and is a security against  unexpected cases.
>> >
>> > For me, backups are like a accident insurance: you need them and you hope 
>> > never use them.
>> >
>> > And about VCS use:
>> >
>> > Many good electronics guys do not even know what is it, and have never 
>> > compiled any source code.
>> > Electronics world and Software world are not exactly the same world.
>> >
>> > --
>> > Jean-Pierre CHARRAS
>> >
>> > ___
>> > Mailing list: https://launchpad.net/~kicad-developers
>> > Post to : kicad-developers@lists.launchpad.net
>> > Unsubscribe : https://launchpad.net/~kicad-developers
>> > More help   : https://help.launchpad.net/ListHelp
>>
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Andrew Lutsenko]

I don't think removing the current backup system before implementing a new
one is the right thing to do.
As limited and simple as it is, the current system provides valuable
safeguard against data loss on crashes that may corrupt the main save file.
And no, VCS is not a replacement, most people hit ctrl-s a lot more often
than they do "git commit".


On Tue, Jun 30, 2020 at 7:13 AM Jon Evans  wrote:

> JP, I agree that true backups are useful.
>
> Maybe even it is a good idea for KiCad to have a built-in backup function.
>
> I just don't think the current backup function is actually useful
> because of my first point (backups are overwritten on each save).
>
> I would propose:
>
> 1) Remove the current backup file generation
>
> 2) Create a spec (in GitLab issue) for a better backup system that:
>
> - Can be turned on or off
> - Backs up the whole project in a zip file
> - Can keep the last N backups
> - Runs on a schedule, not necessarily every time you click Save.
>
> Anyone opposed to this?
>
> -Jon
>
> On Tue, Jun 30, 2020 at 3:32 AM jp charras  wrote:
> >
> > Le 30/06/2020 à 00:13, hauptmech a écrit :
> > >
> > > While I agree that it is not KiCad's job to do archival backups or
> version control, I do think that KiCad should preserve the integrity of
> users data through a crash. Even better if the work between the last save
> and the crash is also preserved and recovered on restart.
> > >
> > > I have had to use the backup files to recover data in the past. I have
> no idea if that recovery was related to something that is now no longer a
> possible issue.
> > >
> > >
> > > -Hauptmech
> > >
> > >
> >
> > I am also thinking a backup can be useful when something unexpected
> happens.
> >
> > Backups, like any security system, bothers you as long as you do not
> need to use them.
> > But you are happy to find them in case of trouble.
> >
> > I like the way some CAD tools manage backup:
> > only one zip archive is created (for instance projectname_backup.zip)
> and contains all saved files
> > (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
> > This is not a full project backup, just main files are saved.
> >
> > This is not invasive (only one file, or a few .zip if one want to keep
> last n saved versions)
> > and is a security against  unexpected cases.
> >
> > For me, backups are like a accident insurance: you need them and you
> hope never use them.
> >
> > And about VCS use:
> >
> > Many good electronics guys do not even know what is it, and have never
> compiled any source code.
> > Electronics world and Software world are not exactly the same world.
> >
> > --
> > Jean-Pierre CHARRAS
> >
> > ___
> > Mailing list: https://launchpad.net/~kicad-developers
> > Post to : kicad-developers@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~kicad-developers
> > More help   : https://help.launchpad.net/ListHelp
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Wayne Stambaugh]

Sounds like a great solution to me.

On 6/30/20 10:12 AM, Jon Evans wrote:
> JP, I agree that true backups are useful.
> 
> Maybe even it is a good idea for KiCad to have a built-in backup function.
> 
> I just don't think the current backup function is actually useful
> because of my first point (backups are overwritten on each save).
> 
> I would propose:
> 
> 1) Remove the current backup file generation
> 
> 2) Create a spec (in GitLab issue) for a better backup system that:
> 
> - Can be turned on or off
> - Backs up the whole project in a zip file
> - Can keep the last N backups
> - Runs on a schedule, not necessarily every time you click Save.
> 
> Anyone opposed to this?
> 
> -Jon
> 
> On Tue, Jun 30, 2020 at 3:32 AM jp charras  wrote:
>>
>> Le 30/06/2020 à 00:13, hauptmech a écrit :
>>>
>>> While I agree that it is not KiCad's job to do archival backups or version 
>>> control, I do think that KiCad should preserve the integrity of users data 
>>> through a crash. Even better if the work between the last save and the 
>>> crash is also preserved and recovered on restart.
>>>
>>> I have had to use the backup files to recover data in the past. I have no 
>>> idea if that recovery was related to something that is now no longer a 
>>> possible issue.
>>>
>>>
>>> -Hauptmech
>>>
>>>
>>
>> I am also thinking a backup can be useful when something unexpected happens.
>>
>> Backups, like any security system, bothers you as long as you do not need to 
>> use them.
>> But you are happy to find them in case of trouble.
>>
>> I like the way some CAD tools manage backup:
>> only one zip archive is created (for instance projectname_backup.zip) and 
>> contains all saved files
>> (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
>> This is not a full project backup, just main files are saved.
>>
>> This is not invasive (only one file, or a few .zip if one want to keep last 
>> n saved versions)
>> and is a security against  unexpected cases.
>>
>> For me, backups are like a accident insurance: you need them and you hope 
>> never use them.
>>
>> And about VCS use:
>>
>> Many good electronics guys do not even know what is it, and have never 
>> compiled any source code.
>> Electronics world and Software world are not exactly the same world.
>>
>> --
>> Jean-Pierre CHARRAS
>>
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Jon Evans]

JP, I agree that true backups are useful.

Maybe even it is a good idea for KiCad to have a built-in backup function.

I just don't think the current backup function is actually useful
because of my first point (backups are overwritten on each save).

I would propose:

1) Remove the current backup file generation

2) Create a spec (in GitLab issue) for a better backup system that:

- Can be turned on or off
- Backs up the whole project in a zip file
- Can keep the last N backups
- Runs on a schedule, not necessarily every time you click Save.

Anyone opposed to this?

-Jon

On Tue, Jun 30, 2020 at 3:32 AM jp charras  wrote:
>
> Le 30/06/2020 à 00:13, hauptmech a écrit :
> >
> > While I agree that it is not KiCad's job to do archival backups or version 
> > control, I do think that KiCad should preserve the integrity of users data 
> > through a crash. Even better if the work between the last save and the 
> > crash is also preserved and recovered on restart.
> >
> > I have had to use the backup files to recover data in the past. I have no 
> > idea if that recovery was related to something that is now no longer a 
> > possible issue.
> >
> >
> > -Hauptmech
> >
> >
>
> I am also thinking a backup can be useful when something unexpected happens.
>
> Backups, like any security system, bothers you as long as you do not need to 
> use them.
> But you are happy to find them in case of trouble.
>
> I like the way some CAD tools manage backup:
> only one zip archive is created (for instance projectname_backup.zip) and 
> contains all saved files
> (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
> This is not a full project backup, just main files are saved.
>
> This is not invasive (only one file, or a few .zip if one want to keep last n 
> saved versions)
> and is a security against  unexpected cases.
>
> For me, backups are like a accident insurance: you need them and you hope 
> never use them.
>
> And about VCS use:
>
> Many good electronics guys do not even know what is it, and have never 
> compiled any source code.
> Electronics world and Software world are not exactly the same world.
>
> --
> Jean-Pierre CHARRAS
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Jeff Young]

I find the backup files in the directory pretty annoying.

While it’s not ideal I’m willing to completely lose the backup functionality to 
fix that.

I’m also fine with building a more sophisticated system to folder them (ie: one 
that can handle the external files), but only if it’s going to actually get 
done for 6.0.  If it’s going to result in the can getting kicked further down 
the road, them I’d rather remove the feature.

Cheers,
Jeff.

> On 30 Jun 2020, at 12:57, Nick Østergaard  wrote:
> 
> Last time I looked, there was rescue feature in the file menu of pcbnew, I 
> think. I never used it, I don't really know what it does, but I guess it just 
> reads the bak file.
> 
> tir. 30. jun. 2020 09.32 skrev jp charras  >:
> Le 30/06/2020 à 00:13, hauptmech a écrit :
> > 
> > While I agree that it is not KiCad's job to do archival backups or version 
> > control, I do think that KiCad should preserve the integrity of users data 
> > through a crash. Even better if the work between the last save and the 
> > crash is also preserved and recovered on restart.
> > 
> > I have had to use the backup files to recover data in the past. I have no 
> > idea if that recovery was related to something that is now no longer a 
> > possible issue.
> > 
> > 
> > -Hauptmech
> > 
> > 
> 
> I am also thinking a backup can be useful when something unexpected happens.
> 
> Backups, like any security system, bothers you as long as you do not need to 
> use them.
> But you are happy to find them in case of trouble.
> 
> I like the way some CAD tools manage backup:
> only one zip archive is created (for instance projectname_backup.zip) and 
> contains all saved files
> (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
> This is not a full project backup, just main files are saved.
> 
> This is not invasive (only one file, or a few .zip if one want to keep last n 
> saved versions)
> and is a security against  unexpected cases.
> 
> For me, backups are like a accident insurance: you need them and you hope 
> never use them.
> 
> And about VCS use:
> 
> Many good electronics guys do not even know what is it, and have never 
> compiled any source code.
> Electronics world and Software world are not exactly the same world.
> 
> -- 
> Jean-Pierre CHARRAS
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers 
> 
> Post to : kicad-developers@lists.launchpad.net 
> 
> Unsubscribe : https://launchpad.net/~kicad-developers 
> 
> More help   : https://help.launchpad.net/ListHelp 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Nick Østergaard]

Last time I looked, there was rescue feature in the file menu of pcbnew, I
think. I never used it, I don't really know what it does, but I guess it
just reads the bak file.

tir. 30. jun. 2020 09.32 skrev jp charras :

> Le 30/06/2020 à 00:13, hauptmech a écrit :
> >
> > While I agree that it is not KiCad's job to do archival backups or
> version control, I do think that KiCad should preserve the integrity of
> users data through a crash. Even better if the work between the last save
> and the crash is also preserved and recovered on restart.
> >
> > I have had to use the backup files to recover data in the past. I have
> no idea if that recovery was related to something that is now no longer a
> possible issue.
> >
> >
> > -Hauptmech
> >
> >
>
> I am also thinking a backup can be useful when something unexpected
> happens.
>
> Backups, like any security system, bothers you as long as you do not need
> to use them.
> But you are happy to find them in case of trouble.
>
> I like the way some CAD tools manage backup:
> only one zip archive is created (for instance projectname_backup.zip) and
> contains all saved files
> (in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
> This is not a full project backup, just main files are saved.
>
> This is not invasive (only one file, or a few .zip if one want to keep
> last n saved versions)
> and is a security against  unexpected cases.
>
> For me, backups are like a accident insurance: you need them and you hope
> never use them.
>
> And about VCS use:
>
> Many good electronics guys do not even know what is it, and have never
> compiled any source code.
> Electronics world and Software world are not exactly the same world.
>
> --
> Jean-Pierre CHARRAS
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[jp charras]

Le 30/06/2020 à 00:13, hauptmech a écrit :
> 
> While I agree that it is not KiCad's job to do archival backups or version 
> control, I do think that KiCad should preserve the integrity of users data 
> through a crash. Even better if the work between the last save and the crash 
> is also preserved and recovered on restart.
> 
> I have had to use the backup files to recover data in the past. I have no 
> idea if that recovery was related to something that is now no longer a 
> possible issue.
> 
> 
> -Hauptmech
> 
> 

I am also thinking a backup can be useful when something unexpected happens.

Backups, like any security system, bothers you as long as you do not need to 
use them.
But you are happy to find them in case of trouble.

I like the way some CAD tools manage backup:
only one zip archive is created (for instance projectname_backup.zip) and 
contains all saved files
(in our case: *.kicad_sch (and sub paths) and .kicad_pcb)
This is not a full project backup, just main files are saved.

This is not invasive (only one file, or a few .zip if one want to keep last n 
saved versions)
and is a security against  unexpected cases.

For me, backups are like a accident insurance: you need them and you hope never 
use them.

And about VCS use:

Many good electronics guys do not even know what is it, and have never compiled 
any source code.
Electronics world and Software world are not exactly the same world.

-- 
Jean-Pierre CHARRAS

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Adam Wolf]

I have found them more confusing than valuable, both when working on
projects and when teaching folks and they're curious about each of the new
files.

I guess we can be glad that the state of user backup/VCS is better now than
a decade ago!  I tend to have little hope for things like that, but hey!

On Mon, Jun 29, 2020, 2:33 PM Jon Evans  wrote:

> That save process change is what I was referring to in (2)
>
> The old save process (in 5.1 and earlier) is:
>
> 1) Move (not copy) filename.ext to filename.ext-bak
> 2) Write the save to filename.ext (a new file at this point)
>
> Our current save process (in nightlies) is:
>
> 1) Copy filename.ext to filename.ext-bak
> 2) Write to .filename.ext$ (a temporary file)
> 3) If everything worked, move .filename.ext$ to filename.ext
> (overwrites the old file)
>
> There is no need to delete anything: if the save works, the move
> option effectively deletes the temporary file.  If the save doesn't
> work, the old file is untouched.
>
> On Mon, Jun 29, 2020 at 3:27 PM Andrew Zonenberg
>  wrote:
> >
> > I find them annoying and have to add them to my gitignore's etc
> constantly.
> >
> > They're of no value to me because I tend to save every 30 seconds or so
> > while working.
> >
> > The *one* thing they might be useful for is guarding against kicad
> > segfaulting or other failures (disk full, NFS connection lost, etc)
> > during the save process itself. To do this, I'd suggest:
> >
> > * Move old file to file.bak
> > * Write new file
> > * Delete old file if everything worked. If something goes wrong before
> > reaching this point, you can restore from the bak.
> >
> > On 6/29/20 12:23 PM, Jon Evans wrote:
> > > Currently, KiCad automatically creates backups of schematic and PCB
> > > files when you save a file.
> > >
> > > The logic for these backups is basically: if a file already exists
> > > with the same name as what we are saving, copy that file to a new file
> > > and give it the "-bak" suffix on the file extension.
> > >
> > > These backups are stored next to the original file in the current
> > > KiCad codebase.  This understandably creates clutter that some people
> > > don't like (myself included) so in the project-settings branch that is
> > > about ready to be merged, I changed this behavior to place all these
> > > backups in a special backups folder for the project.
> > >
> > > This proved to have some complications around the handling of files
> > > outside the project path (which it's possible to have with
> > > hierarchical schematic sheets) so I need to do something else.
> > >
> > > After some thought, I am pretty convinced that the right thing to do
> > > is just *remove this backup feature entirely*.  Here's why:
> > >
> > > 1) It's not a very good backup:  It just stores the state from the
> > > last time you hit "save".  If you hit save again, your backup is blown
> > > up.  So, it's really like a "undo" function, but on disk, and with
> > > only one level of undo.
> > >
> > > 2) Recently I changed how we save schematic and board files to fix
> > > some unrelated issues people were seeing with cloud backup services.
> > > Before this change, if KiCad crashed or had some other serious error
> > > while saving a file, the file would be lost (because we used to delete
> > > the old file and then write a new one in its place).  After this
> > > change, we write the new file to a temporary location, and only if
> > > that write succeeds do we copy it on top of the old file.  I think
> > > this vulnerability to generating corrupt files if we crash was one of
> > > the reasons for this backup file system, and that reason is now gone.
> > >
> > > 3) Because it's not a very good backup, I worry that the fact that
> > > "bak" files exist might cause some users to have a false sense of
> > > security and not use a true backup system (like a version control
> > > system, or some other mechanism that actually backs up files
> > > periodically).  I want to remove this false sense of security so that
> > > it is more clear that users should back up their files in some way if
> > > they care about the work.
> > >
> > > In other words, I don't think this feature actually gives enough value
> > > to make it worth the clutter in the project folder and/or the
> > > development effort to make it work well with less clutter.
> > >
> > > So, I'd like to hear from others on this: anyone disagree?
> > >
> > > Thanks,
> > > Jon
> > >
> > > ___
> > > Mailing list: https://launchpad.net/~kicad-developers
> > > Post to : kicad-developers@lists.launchpad.net
> > > Unsubscribe : https://launchpad.net/~kicad-developers
> > > More help   : https://help.launchpad.net/ListHelp
> > >
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : 

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Wayne Stambaugh]

Autosave is independent of the back up file.

On 6/29/2020 6:22 PM, Jeff Young wrote:
> How does the current backup file strategy interact with the autosave feature? 
>  If autosave works without it, that might address a lot of these concerns.
> 
> 
> 
>> On 29 Jun 2020, at 23:13, hauptmech  wrote:
>>
>>
>> While I agree that it is not KiCad's job to do archival backups or version 
>> control, I do think that KiCad should preserve the integrity of users data 
>> through a crash. Even better if the work between the last save and the crash 
>> is also preserved and recovered on restart.
>>
>> I have had to use the backup files to recover data in the past. I have no 
>> idea if that recovery was related to something that is now no longer a 
>> possible issue.
>>
>>
>> -Hauptmech
>>
>>
>> On 6/30/2020 7:23 AM, Jon Evans wrote:
>>> Currently, KiCad automatically creates backups of schematic and PCB
>>> files when you save a file.
>>>
>>> The logic for these backups is basically: if a file already exists
>>> with the same name as what we are saving, copy that file to a new file
>>> and give it the "-bak" suffix on the file extension.
>>>
>>> These backups are stored next to the original file in the current
>>> KiCad codebase.  This understandably creates clutter that some people
>>> don't like (myself included) so in the project-settings branch that is
>>> about ready to be merged, I changed this behavior to place all these
>>> backups in a special backups folder for the project.
>>>
>>> This proved to have some complications around the handling of files
>>> outside the project path (which it's possible to have with
>>> hierarchical schematic sheets) so I need to do something else.
>>>
>>> After some thought, I am pretty convinced that the right thing to do
>>> is just *remove this backup feature entirely*.  Here's why:
>>>
>>> 1) It's not a very good backup:  It just stores the state from the
>>> last time you hit "save".  If you hit save again, your backup is blown
>>> up.  So, it's really like a "undo" function, but on disk, and with
>>> only one level of undo.
>>>
>>> 2) Recently I changed how we save schematic and board files to fix
>>> some unrelated issues people were seeing with cloud backup services.
>>> Before this change, if KiCad crashed or had some other serious error
>>> while saving a file, the file would be lost (because we used to delete
>>> the old file and then write a new one in its place).  After this
>>> change, we write the new file to a temporary location, and only if
>>> that write succeeds do we copy it on top of the old file.  I think
>>> this vulnerability to generating corrupt files if we crash was one of
>>> the reasons for this backup file system, and that reason is now gone.
>>>
>>> 3) Because it's not a very good backup, I worry that the fact that
>>> "bak" files exist might cause some users to have a false sense of
>>> security and not use a true backup system (like a version control
>>> system, or some other mechanism that actually backs up files
>>> periodically).  I want to remove this false sense of security so that
>>> it is more clear that users should back up their files in some way if
>>> they care about the work.
>>>
>>> In other words, I don't think this feature actually gives enough value
>>> to make it worth the clutter in the project folder and/or the
>>> development effort to make it work well with less clutter.
>>>
>>> So, I'd like to hear from others on this: anyone disagree?
>>>
>>> Thanks,
>>> Jon
>>>
>>> ___
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to : kicad-developers@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Jeff Young]

How does the current backup file strategy interact with the autosave feature?  
If autosave works without it, that might address a lot of these concerns.



> On 29 Jun 2020, at 23:13, hauptmech  wrote:
> 
> 
> While I agree that it is not KiCad's job to do archival backups or version 
> control, I do think that KiCad should preserve the integrity of users data 
> through a crash. Even better if the work between the last save and the crash 
> is also preserved and recovered on restart.
> 
> I have had to use the backup files to recover data in the past. I have no 
> idea if that recovery was related to something that is now no longer a 
> possible issue.
> 
> 
> -Hauptmech
> 
> 
> On 6/30/2020 7:23 AM, Jon Evans wrote:
>> Currently, KiCad automatically creates backups of schematic and PCB
>> files when you save a file.
>> 
>> The logic for these backups is basically: if a file already exists
>> with the same name as what we are saving, copy that file to a new file
>> and give it the "-bak" suffix on the file extension.
>> 
>> These backups are stored next to the original file in the current
>> KiCad codebase.  This understandably creates clutter that some people
>> don't like (myself included) so in the project-settings branch that is
>> about ready to be merged, I changed this behavior to place all these
>> backups in a special backups folder for the project.
>> 
>> This proved to have some complications around the handling of files
>> outside the project path (which it's possible to have with
>> hierarchical schematic sheets) so I need to do something else.
>> 
>> After some thought, I am pretty convinced that the right thing to do
>> is just *remove this backup feature entirely*.  Here's why:
>> 
>> 1) It's not a very good backup:  It just stores the state from the
>> last time you hit "save".  If you hit save again, your backup is blown
>> up.  So, it's really like a "undo" function, but on disk, and with
>> only one level of undo.
>> 
>> 2) Recently I changed how we save schematic and board files to fix
>> some unrelated issues people were seeing with cloud backup services.
>> Before this change, if KiCad crashed or had some other serious error
>> while saving a file, the file would be lost (because we used to delete
>> the old file and then write a new one in its place).  After this
>> change, we write the new file to a temporary location, and only if
>> that write succeeds do we copy it on top of the old file.  I think
>> this vulnerability to generating corrupt files if we crash was one of
>> the reasons for this backup file system, and that reason is now gone.
>> 
>> 3) Because it's not a very good backup, I worry that the fact that
>> "bak" files exist might cause some users to have a false sense of
>> security and not use a true backup system (like a version control
>> system, or some other mechanism that actually backs up files
>> periodically).  I want to remove this false sense of security so that
>> it is more clear that users should back up their files in some way if
>> they care about the work.
>> 
>> In other words, I don't think this feature actually gives enough value
>> to make it worth the clutter in the project folder and/or the
>> development effort to make it work well with less clutter.
>> 
>> So, I'd like to hear from others on this: anyone disagree?
>> 
>> Thanks,
>> Jon
>> 
>> ___
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to : kicad-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[hauptmech]

While I agree that it is not KiCad's job to do archival backups or 
version control, I do think that KiCad should preserve the integrity of 
users data through a crash. Even better if the work between the last 
save and the crash is also preserved and recovered on restart.


I have had to use the backup files to recover data in the past. I have 
no idea if that recovery was related to something that is now no longer 
a possible issue.



-Hauptmech


On 6/30/2020 7:23 AM, Jon Evans wrote:

Currently, KiCad automatically creates backups of schematic and PCB
files when you save a file.

The logic for these backups is basically: if a file already exists
with the same name as what we are saving, copy that file to a new file
and give it the "-bak" suffix on the file extension.

These backups are stored next to the original file in the current
KiCad codebase.  This understandably creates clutter that some people
don't like (myself included) so in the project-settings branch that is
about ready to be merged, I changed this behavior to place all these
backups in a special backups folder for the project.

This proved to have some complications around the handling of files
outside the project path (which it's possible to have with
hierarchical schematic sheets) so I need to do something else.

After some thought, I am pretty convinced that the right thing to do
is just *remove this backup feature entirely*.  Here's why:

1) It's not a very good backup:  It just stores the state from the
last time you hit "save".  If you hit save again, your backup is blown
up.  So, it's really like a "undo" function, but on disk, and with
only one level of undo.

2) Recently I changed how we save schematic and board files to fix
some unrelated issues people were seeing with cloud backup services.
Before this change, if KiCad crashed or had some other serious error
while saving a file, the file would be lost (because we used to delete
the old file and then write a new one in its place).  After this
change, we write the new file to a temporary location, and only if
that write succeeds do we copy it on top of the old file.  I think
this vulnerability to generating corrupt files if we crash was one of
the reasons for this backup file system, and that reason is now gone.

3) Because it's not a very good backup, I worry that the fact that
"bak" files exist might cause some users to have a false sense of
security and not use a true backup system (like a version control
system, or some other mechanism that actually backs up files
periodically).  I want to remove this false sense of security so that
it is more clear that users should back up their files in some way if
they care about the work.

In other words, I don't think this feature actually gives enough value
to make it worth the clutter in the project folder and/or the
development effort to make it work well with less clutter.

So, I'd like to hear from others on this: anyone disagree?

Thanks,
Jon

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Ian McInerney]

I'm +1 on removing the backup files completely. I can see how they might
have been useful in the past given the save behavior, but I think the
recent changes to that have made them obsolete. I am also someone who puts
all my projects into a git repo, so I have always found them annoying and
never even used them.

Also, did we ever even have anything like an auto-recover mechanism for
them (where if the main file failed loading it would fallback to the
backup)? I can't recall... (which probably means I never used it if we have
one).

-Ian

On Mon, Jun 29, 2020 at 8:24 PM Jon Evans  wrote:

> Currently, KiCad automatically creates backups of schematic and PCB
> files when you save a file.
>
> The logic for these backups is basically: if a file already exists
> with the same name as what we are saving, copy that file to a new file
> and give it the "-bak" suffix on the file extension.
>
> These backups are stored next to the original file in the current
> KiCad codebase.  This understandably creates clutter that some people
> don't like (myself included) so in the project-settings branch that is
> about ready to be merged, I changed this behavior to place all these
> backups in a special backups folder for the project.
>
> This proved to have some complications around the handling of files
> outside the project path (which it's possible to have with
> hierarchical schematic sheets) so I need to do something else.
>
> After some thought, I am pretty convinced that the right thing to do
> is just *remove this backup feature entirely*.  Here's why:
>
> 1) It's not a very good backup:  It just stores the state from the
> last time you hit "save".  If you hit save again, your backup is blown
> up.  So, it's really like a "undo" function, but on disk, and with
> only one level of undo.
>
> 2) Recently I changed how we save schematic and board files to fix
> some unrelated issues people were seeing with cloud backup services.
> Before this change, if KiCad crashed or had some other serious error
> while saving a file, the file would be lost (because we used to delete
> the old file and then write a new one in its place).  After this
> change, we write the new file to a temporary location, and only if
> that write succeeds do we copy it on top of the old file.  I think
> this vulnerability to generating corrupt files if we crash was one of
> the reasons for this backup file system, and that reason is now gone.
>
> 3) Because it's not a very good backup, I worry that the fact that
> "bak" files exist might cause some users to have a false sense of
> security and not use a true backup system (like a version control
> system, or some other mechanism that actually backs up files
> periodically).  I want to remove this false sense of security so that
> it is more clear that users should back up their files in some way if
> they care about the work.
>
> In other words, I don't think this feature actually gives enough value
> to make it worth the clutter in the project folder and/or the
> development effort to make it work well with less clutter.
>
> So, I'd like to hear from others on this: anyone disagree?
>
> Thanks,
> Jon
>
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Jon Evans]

That save process change is what I was referring to in (2)

The old save process (in 5.1 and earlier) is:

1) Move (not copy) filename.ext to filename.ext-bak
2) Write the save to filename.ext (a new file at this point)

Our current save process (in nightlies) is:

1) Copy filename.ext to filename.ext-bak
2) Write to .filename.ext$ (a temporary file)
3) If everything worked, move .filename.ext$ to filename.ext
(overwrites the old file)

There is no need to delete anything: if the save works, the move
option effectively deletes the temporary file.  If the save doesn't
work, the old file is untouched.

On Mon, Jun 29, 2020 at 3:27 PM Andrew Zonenberg
 wrote:
>
> I find them annoying and have to add them to my gitignore's etc constantly.
>
> They're of no value to me because I tend to save every 30 seconds or so
> while working.
>
> The *one* thing they might be useful for is guarding against kicad
> segfaulting or other failures (disk full, NFS connection lost, etc)
> during the save process itself. To do this, I'd suggest:
>
> * Move old file to file.bak
> * Write new file
> * Delete old file if everything worked. If something goes wrong before
> reaching this point, you can restore from the bak.
>
> On 6/29/20 12:23 PM, Jon Evans wrote:
> > Currently, KiCad automatically creates backups of schematic and PCB
> > files when you save a file.
> >
> > The logic for these backups is basically: if a file already exists
> > with the same name as what we are saving, copy that file to a new file
> > and give it the "-bak" suffix on the file extension.
> >
> > These backups are stored next to the original file in the current
> > KiCad codebase.  This understandably creates clutter that some people
> > don't like (myself included) so in the project-settings branch that is
> > about ready to be merged, I changed this behavior to place all these
> > backups in a special backups folder for the project.
> >
> > This proved to have some complications around the handling of files
> > outside the project path (which it's possible to have with
> > hierarchical schematic sheets) so I need to do something else.
> >
> > After some thought, I am pretty convinced that the right thing to do
> > is just *remove this backup feature entirely*.  Here's why:
> >
> > 1) It's not a very good backup:  It just stores the state from the
> > last time you hit "save".  If you hit save again, your backup is blown
> > up.  So, it's really like a "undo" function, but on disk, and with
> > only one level of undo.
> >
> > 2) Recently I changed how we save schematic and board files to fix
> > some unrelated issues people were seeing with cloud backup services.
> > Before this change, if KiCad crashed or had some other serious error
> > while saving a file, the file would be lost (because we used to delete
> > the old file and then write a new one in its place).  After this
> > change, we write the new file to a temporary location, and only if
> > that write succeeds do we copy it on top of the old file.  I think
> > this vulnerability to generating corrupt files if we crash was one of
> > the reasons for this backup file system, and that reason is now gone.
> >
> > 3) Because it's not a very good backup, I worry that the fact that
> > "bak" files exist might cause some users to have a false sense of
> > security and not use a true backup system (like a version control
> > system, or some other mechanism that actually backs up files
> > periodically).  I want to remove this false sense of security so that
> > it is more clear that users should back up their files in some way if
> > they care about the work.
> >
> > In other words, I don't think this feature actually gives enough value
> > to make it worth the clutter in the project folder and/or the
> > development effort to make it work well with less clutter.
> >
> > So, I'd like to hear from others on this: anyone disagree?
> >
> > Thanks,
> > Jon
> >
> > ___
> > Mailing list: https://launchpad.net/~kicad-developers
> > Post to : kicad-developers@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~kicad-developers
> > More help   : https://help.launchpad.net/ListHelp
> >

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Andy Peters]

> On Jun 29, 2020, at 12:23 PM, Jon Evans  wrote:
> 
> Currently, KiCad automatically creates backups of schematic and PCB
> files when you save a file.
> 
[snip]
> 
> In other words, I don't think this feature actually gives enough value
> to make it worth the clutter in the project folder and/or the
> development effort to make it work well with less clutter.
> 
> So, I'd like to hear from others on this: anyone disagree?

One user’s perspective (mine):

Keeping designs in a proper source-code control system eliminates the need for 
such backups.

What, you say, some people don’t use SCC? Oy.

-a
___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Kicad-developers] Auto-generated backup files: are they useful?

[Andrew Zonenberg]

I find them annoying and have to add them to my gitignore's etc constantly.

They're of no value to me because I tend to save every 30 seconds or so
while working.

The *one* thing they might be useful for is guarding against kicad
segfaulting or other failures (disk full, NFS connection lost, etc)
during the save process itself. To do this, I'd suggest:

* Move old file to file.bak
* Write new file
* Delete old file if everything worked. If something goes wrong before
reaching this point, you can restore from the bak.

On 6/29/20 12:23 PM, Jon Evans wrote:
> Currently, KiCad automatically creates backups of schematic and PCB
> files when you save a file.
> 
> The logic for these backups is basically: if a file already exists
> with the same name as what we are saving, copy that file to a new file
> and give it the "-bak" suffix on the file extension.
> 
> These backups are stored next to the original file in the current
> KiCad codebase.  This understandably creates clutter that some people
> don't like (myself included) so in the project-settings branch that is
> about ready to be merged, I changed this behavior to place all these
> backups in a special backups folder for the project.
> 
> This proved to have some complications around the handling of files
> outside the project path (which it's possible to have with
> hierarchical schematic sheets) so I need to do something else.
> 
> After some thought, I am pretty convinced that the right thing to do
> is just *remove this backup feature entirely*.  Here's why:
> 
> 1) It's not a very good backup:  It just stores the state from the
> last time you hit "save".  If you hit save again, your backup is blown
> up.  So, it's really like a "undo" function, but on disk, and with
> only one level of undo.
> 
> 2) Recently I changed how we save schematic and board files to fix
> some unrelated issues people were seeing with cloud backup services.
> Before this change, if KiCad crashed or had some other serious error
> while saving a file, the file would be lost (because we used to delete
> the old file and then write a new one in its place).  After this
> change, we write the new file to a temporary location, and only if
> that write succeeds do we copy it on top of the old file.  I think
> this vulnerability to generating corrupt files if we crash was one of
> the reasons for this backup file system, and that reason is now gone.
> 
> 3) Because it's not a very good backup, I worry that the fact that
> "bak" files exist might cause some users to have a false sense of
> security and not use a true backup system (like a version control
> system, or some other mechanism that actually backs up files
> periodically).  I want to remove this false sense of security so that
> it is more clear that users should back up their files in some way if
> they care about the work.
> 
> In other words, I don't think this feature actually gives enough value
> to make it worth the clutter in the project folder and/or the
> development effort to make it work well with less clutter.
> 
> So, I'd like to hear from others on this: anyone disagree?
> 
> Thanks,
> Jon
> 
> ___
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
> 

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp

[Kicad-developers] Auto-generated backup files: are they useful?

[Jon Evans]

Currently, KiCad automatically creates backups of schematic and PCB
files when you save a file.

The logic for these backups is basically: if a file already exists
with the same name as what we are saving, copy that file to a new file
and give it the "-bak" suffix on the file extension.

These backups are stored next to the original file in the current
KiCad codebase.  This understandably creates clutter that some people
don't like (myself included) so in the project-settings branch that is
about ready to be merged, I changed this behavior to place all these
backups in a special backups folder for the project.

This proved to have some complications around the handling of files
outside the project path (which it's possible to have with
hierarchical schematic sheets) so I need to do something else.

After some thought, I am pretty convinced that the right thing to do
is just *remove this backup feature entirely*.  Here's why:

1) It's not a very good backup:  It just stores the state from the
last time you hit "save".  If you hit save again, your backup is blown
up.  So, it's really like a "undo" function, but on disk, and with
only one level of undo.

2) Recently I changed how we save schematic and board files to fix
some unrelated issues people were seeing with cloud backup services.
Before this change, if KiCad crashed or had some other serious error
while saving a file, the file would be lost (because we used to delete
the old file and then write a new one in its place).  After this
change, we write the new file to a temporary location, and only if
that write succeeds do we copy it on top of the old file.  I think
this vulnerability to generating corrupt files if we crash was one of
the reasons for this backup file system, and that reason is now gone.

3) Because it's not a very good backup, I worry that the fact that
"bak" files exist might cause some users to have a false sense of
security and not use a true backup system (like a version control
system, or some other mechanism that actually backs up files
periodically).  I want to remove this false sense of security so that
it is more clear that users should back up their files in some way if
they care about the work.

In other words, I don't think this feature actually gives enough value
to make it worth the clutter in the project folder and/or the
development effort to make it work well with less clutter.

So, I'd like to hear from others on this: anyone disagree?

Thanks,
Jon

___
Mailing list: https://launchpad.net/~kicad-developers
Post to : kicad-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kicad-developers
More help   : https://help.launchpad.net/ListHelp