Re: [sqlite] Move to Github!!?

[Damien Sykes]

Hi,
This is a question I have asked myself many times (I.E. Git projects moving 
to Fossil).
GitHub is well known and boasts over 74 million repositories, yet Fossil, 
which both hosts and utilises one of the most well-known database systems in 
the world, I doubt can count a thousand. Even the ChiselApp hosting platform 
hosts a mere 360 public repositories, Hydra hosts 11, WanderingHorse hosts 
23, outside of which lie Fossil itself, the Fossil book, SQLite and friends 
(5 publicly accessible repositories in all), and TCL and friends (7 
repositories), making a total of 408. Add SQLite private repositories, and 
private repositories that I host, have access to or otherwise generally know 
exist, and I come up with an estimate of roughly 470 repositories. Of course 
this is not an accurate statistic since it may exclude more private 
repositories, and definitely excludes any local repositories (I for one have 
about a dozen Git repositories as Fossil repositories).
While I am making every attempt to try to persuade friends towards Fossil, 
they are also choosing Git. Looks to me like the only people who seem to use 
Fossil are those who are most associated with it, which is a real shame.
The only advantage I can see with GitHub is that it's the source code 
Twitter equivalent. Everybody's repository is in one place. As long as you 
know the username and repository name you know the full repository URL, and 
you don't have to worry about server administration. With Fossil, if I 
wanted to make it feel like github, I.E. address.tld/user/repo, I would have 
to script it and serve it via a webserver rather than Fossil's own server, 
two processes which I am not at all skilled enough, at least at the moment, 
to undertake. To give you an example, I am currently having to run two 
systems, one for my website and one for Fossil, so that they can both work 
on port 80, because I know nothing about networking in order to understand 
IP addresses, ports and connections in the way I'd need to get a server and 
Fossil to run on port 80 on the same machine, nor do I know enough about 
webservers to be able to get it to work with CGI. In fact I know so little 
that I follow installation guides to the letter and have to do a complete 
fresh server reset and reinstall from scratch when something goes wrong 
because I haven't a clue how to fix it. If I'm to be totally honest at the 
moment I'm even beginning to doubt my own software development skills.
If there were a Fossil-based github-like system, and both Fossil and the 
hosting system were well promoted, Fossil may or may not become the norm.
Having said that, the advantage of Fossil over Git is that, thanks to the 
webserver, you can easily look at your changes in a laid-out website, even 
on your own machine. I've many a time found myself importing Git 
repositories into Fossil just to look at the timeline. Plus, you don't have 
to worry about complicated concepts like pull requests, synchronising forks 
and submodules etc. Though it doesn't matter what VCS I use I always seem to 
come across, and struggle with, the concept of branching and merging!
In any case, my incompetencies aside. People seem to be slowly moving away 
from SourceForge in favour of GitHub. If only we could make the same 
revolution with Fossil! Ironically when I first came across a site called 
FossHub I actually thought that was an attempt to make a Fossil-based 
GitHub. Seems that isn't the case after all.

Cheers.
Damien.
-Original Message- 
From: Richard Hipp

Sent: Tuesday, December 26, 2017 1:10 PM
To: SQLite mailing list
Cc: shekharreddy.k...@gmail.com
Subject: Re: [sqlite] Move to Github!!?

On 12/25/17, Shekhar Reddy  wrote:


Is there any particular reason that the source is not moved to GitHub? I
think that would reach more number of people there.



There is a mirror of the SQLite repository on GitHub at
https://github.com/mackyle/sqlite (maintained by GitHub user "mackyle"
whom I do not know, but whose efforts I do appreciate).

SQLite uses a different version control system called Fossil.  See
https://www.fossil-scm.org/ for more information about Fossil.  Fossil
is superior to Git+GitHub in many respects.  You can easily see this
by doing a side-by-side comparison of the SQLite Fossil repository
against the GitHub mirror.

For example, here is the GitHub view of the "dbpage" branch of SQLite:

https://github.com/mackyle/sqlite/commits/dbpage

Compare the above against the equivalent Fossil view:

https://www.sqlite.org/src/timeline?p=dfdebd12bfc80b91

The Fossil view clearly shows that the head of "dbpage" is the merger
of two other branches, and Fossil shows clearly where the branch
diverged from trunk.  That information is very difficult to discern
from the GitHub view.

Fossil also has the ability to show the complete context of an
individual branch.  For the "dbpage" branch, the context is shown
here:

Re: [sqlite] Article about using sqlite3 in Python

[Damien Sykes-Lindley]

Hi David,
Very useful regarding the SQLite updating procedure. I was looking in lib or 
wherever the Python SQLite bindings are, but all I could find there was 
sqlite3.pyd.
As for executemany, I personally use executescript which then allows me to 
use begin/commit statements which also saves a lot of time.

Cheers.
Damien.
-Original Message- 
From: David Raymond

Sent: Monday, October 23, 2017 3:36 PM
To: SQLite mailing list
Subject: Re: [sqlite] Article about using sqlite3 in Python

Basic but good. execute will prepare the statement each time through, 
whereas executemany will prepare once and then just bind for each run 
through, which is where the time saving comes from. Depending on how complex 
what you're doing is though it can be hard, or more often awkward to create 
an iterator for executemany.


The other thing I'd add is that you can upgrade your SQLite version by 
replacing the sqlite3.dll file in your \pythonxx\DLLs folder and get the 
latest new SQLite features. Just remember that if you're then passing your 
script off to someone else to run that they might still have the originally 
installed .dll from 6 years ago and won't be able to handle new feature X.



-Original Message-
From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On 
Behalf Of Simon Slavin

Sent: Sunday, October 22, 2017 12:47 PM
To: SQLite mailing list
Subject: [sqlite] Article about using sqlite3 in Python

I don’t know enough about Python to evaluate this, but the sqlite3 side is 
sound, and some readers might find it useful.




Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users 


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Protecting databases

[Damien Sykes-Lindley]

Hi Ryan,
I think there may have been some misunderstanding here. The application is a 
local application and thus needs no server, at least on the data side. But 
multiple people may have access to the same machine. So while user control 
is needed elsewhere, that certainly isn't on the local side. Just a simple 
password would do.
The network/user control scenario is much more important in the 
collaboration features of the software. I am still trying to work out 
ultimately how best to do this (client/server, peer-to-peer possibly with 
relay server etc). However the specific data that is being collaborated on 
doesn't need to be stored remotely, unless of course I use a 
check-in/check-out style system. Again, collaboration is way into the future 
and so I'm not thinking along those lines just yet.
I have looked into the SEE extension again, just to make sure I am correct 
in my assumptions, and it appears I am. I doubt I even get $2000 disposable 
income in one year, let alone a month so that option would be impossible for 
me at this time. Also it appears that you are responsible for compiling SEE, 
again which I would find it very hard to do, never having compiled anything 
in C.
On the other hand, searching is a necessity so it appears that encrypting 
the data from the application itself is also not an option. So I think at 
this point I am stuck

Cheers.
Damien.
-Original Message- 
From: R Smith

Sent: Saturday, October 08, 2016 10:02 AM
To: sqlite-users@mailinglists.sqlite.org
Subject: Re: [sqlite] Protecting databases

Hi Damien,

We are obviously all fans of SQLite and keen to involve more people in
what is one of the best DB systems in existence and most widely used DB
in the World.

But...

SQLite answers a need that is quite specific (though widely used) in
that it is very small, very fast and, as a bonus, can be compiled right
into most software projects so distribution is not an issue - which
further means it is also server-less. This last feature makes it
specifically not well suited for large network-server and service type
applications and not well suited for user control. Encryption is easy
(as others explained) but your replies indicate you are more interested
in user control. While some extensions do exist to allow a form of user
control, I think what you really want is called PostGres or Maria-DB
(previously MySQL). They are equally free, open-source, they support
client-server architecture and both storage-encryption and user-control
are inherent to these systems, and they play very well with web-script
engines (such as PHP and its ilk).

Use SQLite for local storage that need not be protected from users with
access to the local file system. Store all your application settings in
SQLite, etc. It is not uncommon to use multiple database engines in a
single project. Right tool for the job and such.

Of the two alternate engines above, my preferred is PostGres, but there
are situations where Maria DB works a treat (such as the usual
LAMP-stack quick web-server solutions and the like) and it has some easy
and strong scripting capabilities.

Links:
https://mariadb.org/
https://www.postgresql.org/

Good luck,
Ryan


On 2016/10/08 9:18 AM, Damien Sykes-Lindley wrote:

Hi Darren,
You are correct in that genealogy is generally public. However more often 
than not the information you want to publish may very well differ from 
what is in your private database. You may have private notes telling you 
what you need to do. You may have anecdotes shared by many family members 
that may need to be kept private, at least until the involved parties are 
deceased or otherwise choose to divulge it publicly themselves.
Even more importantly I may choose to add an address-book style feature in 
there so you can easily group and contact appropriate family members for 
whatever reason (special occasions etc). Of course that will be private.
Password protecting it is also good on many levels - if the database is to 
be used online then it is needless to say that authentication would be 
required for various people to view it. Even if I decide to make it local 
only, there is the possibility that anyone sharing the computer or network 
may peruse the database when you don't want them to.

Kind regards,
Damien.
-Original Message- From: Darren Duncan
Sent: Saturday, October 08, 2016 6:54 AM
To: SQLite mailing list
Subject: Re: [sqlite] Protecting databases

On 2016-10-07 10:46 PM, Damien Sykes-Lindley wrote:

Hi there,
My name is Damien Lindley, and I am, among other things, an independent, 
hobbiest programmer. I have been blind since birth and thus all my 
computer work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled 
programming and so I guess I am still a beginner in many respects. 
However I don’t work in C or C++, so most of my programming, if using a 
library, relies on precompiled static or dynamic libraries

Re: [sqlite] Protecting databases

[Damien Sykes-Lindley]

Hi Darren,
You are correct in that genealogy is generally public. However more often 
than not the information you want to publish may very well differ from what 
is in your private database. You may have private notes telling you what you 
need to do. You may have anecdotes shared by many family members that may 
need to be kept private, at least until the involved parties are deceased or 
otherwise choose to divulge it publicly themselves.
Even more importantly I may choose to add an address-book style feature in 
there so you can easily group and contact appropriate family members for 
whatever reason (special occasions etc). Of course that will be private.
Password protecting it is also good on many levels - if the database is to 
be used online then it is needless to say that authentication would be 
required for various people to view it. Even if I decide to make it local 
only, there is the possibility that anyone sharing the computer or network 
may peruse the database when you don't want them to.

Kind regards,
Damien.
-Original Message- 
From: Darren Duncan

Sent: Saturday, October 08, 2016 6:54 AM
To: SQLite mailing list
Subject: Re: [sqlite] Protecting databases

On 2016-10-07 10:46 PM, Damien Sykes-Lindley wrote:

Hi there,
My name is Damien Lindley, and I am, among other things, an independent, 
hobbiest programmer. I have been blind since birth and thus all my 
computer work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled 
programming and so I guess I am still a beginner in many respects. However 
I don’t work in C or C++, so most of my programming, if using a library, 
relies on precompiled static or dynamic libraries. Or of course libraries 
that are written or converted specifically for the language I work in 
(FreeBASIC).
Recently, I decided I needed to create a piece of software that could 
manage family trees, since there seems to be a lack of screenreader 
accessible genealogy managers out there. I was advised the best way to do 
this is to use a database engine. I was also informed that SQLite is 
always a good choice for databases.
I must admit, I have never worked with databases before and so now I am in 
the process of learning SQL. However looking at the programming API for 
SQLite I cannot see any means of password protecting the database without 
either buying a commercial extension to do this, or recompiling SQLite 
with the authentication extension. Due to financial constraints and 
unfamiliarity with compiling in C both of these are not an option for me. 
Also I need a secure way to do this, as I think I read that the SQLite 
version simply uses a table to store the user data, which of course can be 
read and accessed elsewhere.

Are there any other options available for doing this?
Any help appreciated.
Thanks.
Damien.


Damien,

Why do you need to password protect the database?

Genealogy information is generally of the public record variety so there is
nothing sensitive to protect.  I am making genealogy software myself and so 
am

familiar with many of the relevant issues.

I would say please explain why you think you need password protection for 
this

project and then the real issue at hand can be addressed.

If yours is a network application and you don't want people on the open 
internet

from accessing the database, fair enough, but that's an application-level
solution; what you're asking for here is that people who have direct access 
to

the SQLite database file are blocked by a password, and this I question.

-- Darren Duncan

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users 


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Protecting databases

[Damien Sykes-Lindley]

Hi there,
My name is Damien Lindley, and I am, among other things, an independent, 
hobbiest programmer. I have been blind since birth and thus all my computer 
work relies on screenreader software and keyboard.
I have only just come through the brink of scripting into compiled programming 
and so I guess I am still a beginner in many respects. However I don’t work in 
C or C++, so most of my programming, if using a library, relies on precompiled 
static or dynamic libraries. Or of course libraries that are written or 
converted specifically for the language I work in (FreeBASIC).
Recently, I decided I needed to create a piece of software that could manage 
family trees, since there seems to be a lack of screenreader accessible 
genealogy managers out there. I was advised the best way to do this is to use a 
database engine. I was also informed that SQLite is always a good choice for 
databases.
I must admit, I have never worked with databases before and so now I am in the 
process of learning SQL. However looking at the programming API for SQLite I 
cannot see any means of password protecting the database without either buying 
a commercial extension to do this, or recompiling SQLite with the 
authentication extension. Due to financial constraints and unfamiliarity with 
compiling in C both of these are not an option for me. Also I need a secure way 
to do this, as I think I read that the SQLite version simply uses a table to 
store the user data, which of course can be read and accessed elsewhere.
Are there any other options available for doing this?
Any help appreciated.
Thanks.
Damien.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users