Author: svn-role
Date: Thu Mar 19 18:23:21 2015
New Revision: 1667828
URL: http://svn.apache.org/r1667828
Log:
Merge the r1664476 group from trunk:
* r1664476, r1664480, r1664481, r1664483, r1664507, r1664520, r1664521,
r1664523, r1664526, r1664527, r1665845, r1665850, r1665852
Automatic create sqlite_stat1 table in wc.db if it doesn't exist.
Justification:
Avoid performance regression on some old working copies.
Notes:
This stats table provides very valuable hints to the sqlite query
optimizer. Sqlite 3.8.x needs this for most of our queries.
.
See "Manual Control Of Query Plans Using SQLITE_STAT Tables"
on https://www.sqlite.org/optoverview.html#manctrl
I (philip) think we cannot release 1.9.0 without this fix.
Votes:
+1: philip, rhuijben, ivan, brane
Added:
subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests_data/wc-without-stat1.tar.bz2
- copied, changed from r1664520,
subversion/trunk/subversion/tests/cmdline/upgrade_tests_data/wc-without-stat1.tar.bz2
Modified:
subversion/branches/1.9.x/ (props changed)
subversion/branches/1.9.x/STATUS
subversion/branches/1.9.x/subversion/libsvn_wc/wc-queries.sql
subversion/branches/1.9.x/subversion/libsvn_wc/wc.h
subversion/branches/1.9.x/subversion/libsvn_wc/wc_db_wcroot.c
subversion/branches/1.9.x/subversion/tests/cmdline/svntest/wc.py
subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests.py
subversion/branches/1.9.x/subversion/tests/libsvn_wc/wc-queries-test.c
Propchange: subversion/branches/1.9.x/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Mar 19 18:23:21 2015
@@ -89,4 +89,4 @@
/subversion/branches/verify-at-commit:1462039-1462408
/subversion/branches/verify-keep-going:1439280-1546110
/subversion/branches/wc-collate-path:1402685-1480384
-/subversion/trunk:1660545-1660547,1660549-1662901,1663003,1663450,1663697,1663706,1663749,1664084-1664085,1664531-1664532,1665164,1665611-1665612
+/subversion/trunk:1660545-1660547,1660549-1662901,1663003,1663450,1663697,1663706,1663749,1664084-1664085,1664476,1664480-1664481,1664483,1664507,1664520-1664521,1664523,1664526-1664527,1664531-1664532,1665164,1665611-1665612,1665845,1665850,1665852
Modified: subversion/branches/1.9.x/STATUS
URL:
http://svn.apache.org/viewvc/subversion/branches/1.9.x/STATUS?rev=1667828&r1=1667827&r2=1667828&view=diff
==============================================================================
--- subversion/branches/1.9.x/STATUS (original)
+++ subversion/branches/1.9.x/STATUS Thu Mar 19 18:23:21 2015
@@ -178,20 +178,6 @@ Veto-blocked changes:
Approved changes:
=================
- * r1664476, r1664480, r1664481, r1664483, r1664507, r1664520, r1664521,
r1664523, r1664526, r1664527, r1665845, r1665850, r1665852
- Automatic create sqlite_stat1 table in wc.db if it doesn't exist.
- Justification:
- Avoid performance regression on some old working copies.
- Notes:
- This stats table provides very valuable hints to the sqlite query
- optimizer. Sqlite 3.8.x needs this for most of our queries.
- .
- See "Manual Control Of Query Plans Using SQLITE_STAT Tables"
- on https://www.sqlite.org/optoverview.html#manctrl
- I (philip) think we cannot release 1.9.0 without this fix.
- Votes:
- +1: philip, rhuijben, ivan, brane
-
*
r1664078,r1664080,r1664187,r1664191,r1664200,r1664344,r1664588,r1664927,r1665886
Instead of making more changes to the auth batons from ra sessions, reduce
the number of changes by introducing an internal slave auth baton feature.
Modified: subversion/branches/1.9.x/subversion/libsvn_wc/wc-queries.sql
URL:
http://svn.apache.org/viewvc/subversion/branches/1.9.x/subversion/libsvn_wc/wc-queries.sql?rev=1667828&r1=1667827&r2=1667828&view=diff
==============================================================================
--- subversion/branches/1.9.x/subversion/libsvn_wc/wc-queries.sql (original)
+++ subversion/branches/1.9.x/subversion/libsvn_wc/wc-queries.sql Thu Mar 19
18:23:21 2015
@@ -1780,6 +1780,10 @@ WHERE wc_id = ?1
AND op_depth = 0
AND (inherited_props not null)
+-- STMT_HAVE_STAT1_TABLE
+SELECT 1 FROM sqlite_master WHERE name='sqlite_stat1' AND type='table'
+LIMIT 1
+
/* ------------------------------------------------------------------------- */
/* Grab all the statements related to the schema. */
Modified: subversion/branches/1.9.x/subversion/libsvn_wc/wc.h
URL:
http://svn.apache.org/viewvc/subversion/branches/1.9.x/subversion/libsvn_wc/wc.h?rev=1667828&r1=1667827&r2=1667828&view=diff
==============================================================================
--- subversion/branches/1.9.x/subversion/libsvn_wc/wc.h (original)
+++ subversion/branches/1.9.x/subversion/libsvn_wc/wc.h Thu Mar 19 18:23:21 2015
@@ -190,6 +190,10 @@ extern "C" {
/* A version < this has no work queue (see workqueue.h). */
#define SVN_WC__HAS_WORK_QUEUE 13
+/* While we still have this DB version we should verify if there is
+ sqlite_stat1 table on opening */
+#define SVN_WC__ENSURE_STAT1_TABLE 31
+
/* Return a string indicating the released version (or versions) of
* Subversion that used WC format number WC_FORMAT, or some other
* suitable string if no released version used WC_FORMAT.
Modified: subversion/branches/1.9.x/subversion/libsvn_wc/wc_db_wcroot.c
URL:
http://svn.apache.org/viewvc/subversion/branches/1.9.x/subversion/libsvn_wc/wc_db_wcroot.c?rev=1667828&r1=1667827&r2=1667828&view=diff
==============================================================================
--- subversion/branches/1.9.x/subversion/libsvn_wc/wc_db_wcroot.c (original)
+++ subversion/branches/1.9.x/subversion/libsvn_wc/wc_db_wcroot.c Thu Mar 19
18:23:21 2015
@@ -304,7 +304,7 @@ svn_wc__db_pdh_create_wcroot(svn_wc__db_
apr_pool_t *result_pool,
apr_pool_t *scratch_pool)
{
- if (sdb != NULL)
+ if (sdb && format == FORMAT_FROM_SDB)
SVN_ERR(svn_sqlite__read_schema_version(&format, sdb, scratch_pool));
/* If we construct a wcroot, then we better have a format. */
@@ -456,6 +456,56 @@ read_link_target(const char **link_targe
return SVN_NO_ERROR;
}
+/* Verify if the sqlite_stat1 table exists and if not tries to add
+ this table (but ignores errors on adding the schema) */
+static svn_error_t *
+verify_stats_table(svn_sqlite__db_t *sdb,
+ int format,
+ apr_pool_t *scratch_pool)
+{
+ svn_sqlite__stmt_t *stmt;
+ svn_boolean_t have_row;
+
+ if (format != SVN_WC__ENSURE_STAT1_TABLE)
+ return SVN_NO_ERROR;
+
+ SVN_ERR(svn_sqlite__get_statement(&stmt, sdb,
+ STMT_HAVE_STAT1_TABLE));
+ SVN_ERR(svn_sqlite__step(&have_row, stmt));
+ SVN_ERR(svn_sqlite__reset(stmt));
+
+ if (!have_row)
+ {
+ svn_error_clear(
+ svn_wc__db_install_schema_statistics(sdb, scratch_pool));
+ }
+
+ return SVN_NO_ERROR;
+}
+
+/* Sqlite transaction helper for opening the db in
+ svn_wc__db_wcroot_parse_local_abspath() to avoid multiple
+ db operations that each obtain and release a lock */
+static svn_error_t *
+fetch_sdb_info(apr_int64_t *wc_id,
+ int *format,
+ svn_sqlite__db_t *sdb,
+ apr_pool_t *scratch_pool)
+{
+ *wc_id = -1;
+ *format = -1;
+
+ SVN_SQLITE__WITH_LOCK4(
+ svn_wc__db_util_fetch_wc_id(wc_id, sdb, scratch_pool),
+ svn_sqlite__read_schema_version(format, sdb, scratch_pool),
+ verify_stats_table(sdb, *format, scratch_pool),
+ SVN_NO_ERROR,
+ sdb);
+
+ return SVN_NO_ERROR;
+}
+
+
svn_error_t *
svn_wc__db_wcroot_parse_local_abspath(svn_wc__db_wcroot_t **wcroot,
const char **local_relpath,
@@ -696,9 +746,10 @@ try_symlink_as_dir:
/* We finally found the database. Construct a wcroot_t for it. */
apr_int64_t wc_id;
+ int format;
svn_error_t *err;
- err = svn_wc__db_util_fetch_wc_id(&wc_id, sdb, scratch_pool);
+ err = fetch_sdb_info(&wc_id, &format, sdb, scratch_pool);
if (err)
{
if (err->apr_err == SVN_ERR_WC_CORRUPT)
@@ -717,7 +768,7 @@ try_symlink_as_dir:
symlink_wcroot_abspath
? symlink_wcroot_abspath
: local_abspath),
- sdb, wc_id, FORMAT_FROM_SDB,
+ sdb, wc_id, format,
db->verify_format,
db->state_pool, scratch_pool);
if (err && (err->apr_err == SVN_ERR_WC_UNSUPPORTED_FORMAT ||
Modified: subversion/branches/1.9.x/subversion/tests/cmdline/svntest/wc.py
URL:
http://svn.apache.org/viewvc/subversion/branches/1.9.x/subversion/tests/cmdline/svntest/wc.py?rev=1667828&r1=1667827&r2=1667828&view=diff
==============================================================================
--- subversion/branches/1.9.x/subversion/tests/cmdline/svntest/wc.py (original)
+++ subversion/branches/1.9.x/subversion/tests/cmdline/svntest/wc.py Thu Mar 19
18:23:21 2015
@@ -1035,6 +1035,16 @@ def sqlite_stmt(wc_root_path, stmt):
c.execute(stmt)
return c.fetchall()
+def sqlite_exec(wc_root_path, stmt):
+ """Execute STMT on the SQLite wc.db in WC_ROOT_PATH and return the
+ results."""
+
+ db = open_wc_db(wc_root_path)[0]
+ c = db.cursor()
+ c.execute(stmt)
+ db.commit()
+
+
# ------------
### probably toss these at some point. or major rework. or something.
### just bootstrapping some changes for now.
Modified: subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests.py
URL:
http://svn.apache.org/viewvc/subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests.py?rev=1667828&r1=1667827&r2=1667828&view=diff
==============================================================================
--- subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests.py
(original)
+++ subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests.py Thu Mar
19 18:23:21 2015
@@ -1438,6 +1438,47 @@ def upgrade_1_7_dir_external(sbox):
# svn: warning: W200033: sqlite[S5]: database is locked
svntest.actions.run_and_verify_svn(None, [], 'upgrade', sbox.wc_dir)
+def auto_analyze(sbox):
+ """automatic SQLite ANALYZE"""
+
+ sbox.build(create_wc = False)
+
+ replace_sbox_with_tarfile(sbox, 'wc-without-stat1.tar.bz2')
+ svntest.main.run_svnadmin('setuuid', sbox.repo_dir,
+ '52ec7e4b-e5f0-451d-829f-f05d5571b4ab')
+
+ # Don't use svn to do relocate as that will add the table.
+ svntest.wc.sqlite_exec(sbox.wc_dir,
+ "update repository "
+ "set root ='" + sbox.repo_url + "'")
+ val = svntest.wc.sqlite_stmt(sbox.wc_dir,
+ "select 1 from sqlite_master "
+ "where name = 'sqlite_stat1'")
+ if val != []:
+ raise svntest.Failure("initial state failed")
+
+ # Make working copy read-only (but not wc_dir itself as
+ # svntest.main.chmod_tree will not reset it.)
+ for path, subdirs, files in os.walk(sbox.wc_dir):
+ for d in subdirs:
+ os.chmod(os.path.join(path, d), 0555)
+ for f in files:
+ os.chmod(os.path.join(path, f), 0444)
+
+ state = svntest.actions.get_virginal_state(sbox.wc_dir, 1)
+ svntest.actions.run_and_verify_status(sbox.wc_dir, state)
+
+ svntest.main.chmod_tree(sbox.wc_dir, 0666, 0022)
+
+ state = svntest.actions.get_virginal_state(sbox.wc_dir, 1)
+ svntest.actions.run_and_verify_status(sbox.wc_dir, state)
+
+ val = svntest.wc.sqlite_stmt(sbox.wc_dir,
+ "select 1 from sqlite_master "
+ "where name = 'sqlite_stat1'")
+ if val != [(1,)]:
+ raise svntest.Failure("analyze failed")
+
########################################################################
# Run the tests
@@ -1494,6 +1535,7 @@ test_list = [ None,
iprops_upgrade1_6,
changelist_upgrade_1_6,
upgrade_1_7_dir_external,
+ auto_analyze,
]
Copied:
subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests_data/wc-without-stat1.tar.bz2
(from r1664520,
subversion/trunk/subversion/tests/cmdline/upgrade_tests_data/wc-without-stat1.tar.bz2)
URL:
http://svn.apache.org/viewvc/subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests_data/wc-without-stat1.tar.bz2?p2=subversion/branches/1.9.x/subversion/tests/cmdline/upgrade_tests_data/wc-without-stat1.tar.bz2&p1=subversion/trunk/subversion/tests/cmdline/upgrade_tests_data/wc-without-stat1.tar.bz2&r1=1664520&r2=1667828&rev=1667828&view=diff
==============================================================================
Binary files - no diff available.
Modified: subversion/branches/1.9.x/subversion/tests/libsvn_wc/wc-queries-test.c
URL:
http://svn.apache.org/viewvc/subversion/branches/1.9.x/subversion/tests/libsvn_wc/wc-queries-test.c?rev=1667828&r1=1667827&r2=1667828&view=diff
==============================================================================
--- subversion/branches/1.9.x/subversion/tests/libsvn_wc/wc-queries-test.c
(original)
+++ subversion/branches/1.9.x/subversion/tests/libsvn_wc/wc-queries-test.c Thu
Mar 19 18:23:21 2015
@@ -108,6 +108,7 @@ static const int slow_statements[] =
/* Slow, but just if foreign keys are enabled:
* STMT_DELETE_PRISTINE_IF_UNREFERENCED,
*/
+ STMT_HAVE_STAT1_TABLE, /* Queries sqlite_master which has no index */
-1 /* final marker */
};
