Re: [Maria-developers] MWL#39 improving mysqlbinlog output and doing rename
Michael Widenius writes:
> Hi!
>
>> "Kristian" == Kristian Nielsen writes:
>
> Kristian> Sergey Petrunya writes:
>>> I've worked through the list and filed
>>>
>>> MWL#39 Replication tasks
>>>
>>> and its subordinate tasks:
>>>
>>> MWL#36 Add a mysqlbinlog option to change the used database
>>> MWL#37 Add an option to mysqlbinlog to produce SQL script with fewer
>>> roundtrips
>>> MWL#38 Make mysqlbinlog not to output unneeded statements when using
>>> --database
>>> MWL#40 Add a mysqlbinlog option to filter updates to certain tables
>>> MWL#41 Add a mysqlbinlog option to filter certain kinds of statements
> Kristian> One idea that occured to me and that I would like to bounce off of
> you others
> Kristian> (while I still continue thinking more about it and also working on
> the
> Kristian> original proposals):
>
> Kristian> In some sense the root of the problem is the magic "BINLOG"
> statement, which
> Kristian> really is not very nice. It is much much harder than it should be
> to see from
> Kristian> the binlog what data is actually changed from row-based events.
>
> Kristian> If the BINLOG statement was using proper syntax showing the actual
> data
> Kristian> changes, then the original customer problem would likely be solved
> as they
> Kristian> could treat it the same way as statement-based replication events.
> This is
> Kristian> especially nice when one considers that even row-based replication
> uses lots
> Kristian> of statement-based events, not to mention mixed-mode replication.
>
> Kristian> But readable BINLOG statements would be very nice in any case, and
> solve a
> Kristian> much more general problem than these worklogs, in an arguably nicer
> way.
>
> Note that it would also be very nice to have in the binlog the exact
> original statement:
Yes. There was actually a customer request for this.
I think for this we would actually need a new binlog event type
(Comment_log_event?). Unless we want to log an empty statement Query_log_event
containing only a comment (a bit of a hack).
> - Easier to understand why updates happened.
> - Would make it easier to find out where in application things went
> wrong (as you can search for exact strings)
> - Allow one to filter things based on comments in the statement.
>
> The cost would be to have an approximately 2 as big binlog.
> (Especially insert of big blob's would be a bit painful).
Yes. So should maybe be optional.
> Kristian> BINLOG
> Kristian> WITH TIMESTAMP xxx SERVER_ID 1 MASTER_POS 415 FLAGS 0x0
> Kristian> TABLE db1.table1 AS 1 COLUMNS (INT NOT NULL, BLOB, VARCHAR(100))
> FLAGS 0x0
> Kristian> TABLE db2.table2 AS 2 COLUMNS (CHAR(10)) FLAGS 0x0
> Kristian> WRITE_ROW INTO db1.table1(1,3) VALUES (42, 'foobar'), (10, NULL)
> FLAGS 0x2
> Kristian> UPDATE_ROW INTO db2.table2 (1) (1) VALUES FROM ('beforeval') TO
> ('toval'),
> Kristian>FROM ('a') TO ('b') FLAGS 0x0
> Kristian> DELETE_ROW INTO db2.table2 (1) VALUES ('row_to_delete') FLAGS 0x0;
>
> Kristian> This is basically a dump of what is stored in the events, and would
> be an
> Kristian> alternative to BINLOG 'gwWEShMBAA...'.
>
> Kristian> So what do people think?
>
> The above would be a much better option than using the current syntax.
> Kristian> The implementation of this is not necessarily all that much harder
> than the
> Kristian> suggested worklogs under MWL#39 I think. The server upon reading
> this would
> Kristian> just reconstruct the binary representation of the binlog and
> proceed as the
> Kristian> old BINLOG statement. The mysqlbinlog program would just print out
> the
> Kristian> fields. The main complications are the addition of syntax to the
> Bison grammer
> Kristian> plus the need to handle all the possible types.
>
> Kristian> If we choose the easiest option in all the MWL#39 subtasks that
> would probably
> Kristian> be somewhat easier. On the other hand this would be a much more
> generally
> Kristian> useful feature, and would make trivial a lot of the suggested
> modifications to
> Kristian> mysqlbinlog.
>
> Which of the original customer problems would the above solve ?
The original customer problem was that they have a working solution for SBR
using some custom awk post-processing of the mysqlbinlog output. But this
solution can not be adapted for RBR, as the BINLOG 'xxx' statements are
impossible (or very close) to do anything with.
Having a readable BINLOG statement would allow them to adapt their awk script
to work with RBR events as well. Since my understanding was that they have a
working solution for SBR now. So they might decide they do not need any new
mysqlbinlog options at all.
I agree it does not do anything solve the general problems with using regexp
post-processing to filter/modify mysqlbinlog output for SBR.
You could say that the general method of awk post-processing the mysqlbinlog
output becomes more usable when it works for RBR as well, reducing the need
for special filtering and rewr
[Maria-developers] MWL#39 improving mysqlbinlog output and doing rename
Hi!
> "Kristian" == Kristian Nielsen writes:
Kristian> Sergey Petrunya writes:
>> I've worked through the list and filed
>>
>> MWL#39 Replication tasks
>>
>> and its subordinate tasks:
>>
>> MWL#36 Add a mysqlbinlog option to change the used database
>> MWL#37 Add an option to mysqlbinlog to produce SQL script with fewer
>> roundtrips
>> MWL#38 Make mysqlbinlog not to output unneeded statements when using
>> --database
>> MWL#40 Add a mysqlbinlog option to filter updates to certain tables
>> MWL#41 Add a mysqlbinlog option to filter certain kinds of statements
Kristian> Thanks, Sergey! I've started to work my way through them and the many
Kristian> associated issues to understand for this.
Kristian> One idea that occured to me and that I would like to bounce off of
you others
Kristian> (while I still continue thinking more about it and also working on the
Kristian> original proposals):
Kristian> In some sense the root of the problem is the magic "BINLOG"
statement, which
Kristian> really is not very nice. It is much much harder than it should be to
see from
Kristian> the binlog what data is actually changed from row-based events.
Kristian> If the BINLOG statement was using proper syntax showing the actual
data
Kristian> changes, then the original customer problem would likely be solved as
they
Kristian> could treat it the same way as statement-based replication events.
This is
Kristian> especially nice when one considers that even row-based replication
uses lots
Kristian> of statement-based events, not to mention mixed-mode replication.
Kristian> But readable BINLOG statements would be very nice in any case, and
solve a
Kristian> much more general problem than these worklogs, in an arguably nicer
way.
Note that it would also be very nice to have in the binlog the exact
original statement:
- Easier to understand why updates happened.
- Would make it easier to find out where in application things went
wrong (as you can search for exact strings)
- Allow one to filter things based on comments in the statement.
The cost would be to have an approximately 2 as big binlog.
(Especially insert of big blob's would be a bit painful).
Kristian> There are also a number of potential problems that need careful
consideration
Kristian> of course. I haven't yet fully thought things through, just wanted to
throw up
Kristian> the idea.
Kristian> I went throught the code for RBR events, and came up with the
following
Kristian> possible syntax:
Kristian> BINLOG
Kristian> WITH TIMESTAMP xxx SERVER_ID 1 MASTER_POS 415 FLAGS 0x0
Kristian> TABLE db1.table1 AS 1 COLUMNS (INT NOT NULL, BLOB, VARCHAR(100))
FLAGS 0x0
Kristian> TABLE db2.table2 AS 2 COLUMNS (CHAR(10)) FLAGS 0x0
Kristian> WRITE_ROW INTO db1.table1(1,3) VALUES (42, 'foobar'), (10, NULL)
FLAGS 0x2
Kristian> UPDATE_ROW INTO db2.table2 (1) (1) VALUES FROM ('beforeval') TO
('toval'),
Kristian>FROM ('a') TO ('b') FLAGS 0x0
Kristian> DELETE_ROW INTO db2.table2 (1) VALUES ('row_to_delete') FLAGS 0x0;
Kristian> This is basically a dump of what is stored in the events, and would
be an
Kristian> alternative to BINLOG 'gwWEShMBAA...'.
Kristian> So what do people think?
The above would be a much better option than using the current syntax.
Kristian> The implementation of this is not necessarily all that much harder
than the
Kristian> suggested worklogs under MWL#39 I think. The server upon reading this
would
Kristian> just reconstruct the binary representation of the binlog and proceed
as the
Kristian> old BINLOG statement. The mysqlbinlog program would just print out the
Kristian> fields. The main complications are the addition of syntax to the
Bison grammer
Kristian> plus the need to handle all the possible types.
Kristian> If we choose the easiest option in all the MWL#39 subtasks that would
probably
Kristian> be somewhat easier. On the other hand this would be a much more
generally
Kristian> useful feature, and would make trivial a lot of the suggested
modifications to
Kristian> mysqlbinlog.
Which of the original customer problems would the above solve ?
For exampling, doing general rename of databases wouldn't be much
easier to do with the the above syntax (as we still need to handle SBR).
Kristian> I'm pretty sure this would be easier than some of the harder options
in the
Kristian> MWL#39 subtasks.
Which one are you thinking about ?
(I like the proposed syntax, but don't grasp why things would be
simple when doing this ).
Kristian> So I need to think a bit more about this to make up my mind if I like
it. But
Kristian> I think at least it is a very interesting idea.
Kristian> Comments?
Kristian> (Also, isn't it time to move the discussion to the public
Kristian> maria-developers@lists.launchpad.net list?)
Done.
Regards,
Monty
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://lau
[Maria-developers] Updated (by Guest): improving mysqlbinlog output and doing rename (39)
--- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...: improving mysqlbinlog output and doing rename CREATION DATE..: Sun, 09 Aug 2009, 12:24 SUPERVISOR.: Monty IMPLEMENTOR: COPIES TO..: CATEGORY...: Client-RawIdeaBin TASK ID: 39 (http://askmonty.org/worklog/?tid=39) VERSION: Server-9.x STATUS.: Un-Assigned PRIORITY...: 60 WORKED HOURS...: 17 ESTIMATE...: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Fri, 14 Aug 2009, 15:52)=-=- Title modified. --- /tmp/wklog.39.old.11123 2009-08-14 15:52:29.0 +0300 +++ /tmp/wklog.39.new.11123 2009-08-14 15:52:29.0 +0300 @@ -1 +1 @@ -Replication tasks +improving mysqlbinlog output and doing rename -=-=(Guest - Mon, 10 Aug 2009, 16:32)=-=- Adding 1 hour for Monty's initial work on starting the architecture review. Worked 1 hour and estimate 0 hours remain (original estimate increased by 1 hour). -=-=(Psergey - Mon, 10 Aug 2009, 15:59)=-=- Re-searched and added subtasks. Worked 16 hours and estimate 0 hours remain (original estimate increased by 16 hours). -=-=(Psergey - Mon, 10 Aug 2009, 15:31)=-=- Dependency created: 39 now depends on 41 -=-=(Guest - Mon, 10 Aug 2009, 14:52)=-=- Dependency created: 39 now depends on 40 -=-=(Psergey - Sun, 09 Aug 2009, 12:27)=-=- Dependency created: 39 now depends on 36 -=-=(Psergey - Sun, 09 Aug 2009, 12:24)=-=- Dependency created: 39 now depends on 38 -=-=(Psergey - Sun, 09 Aug 2009, 12:24)=-=- Dependency created: 39 now depends on 37 DESCRIPTION: A combine task for all replication tasks. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE --- WorkLog (v3.5.9) ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
[Maria-developers] Updated (by Guest): improving mysqlbinlog output and doing rename (39)
--- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...: improving mysqlbinlog output and doing rename CREATION DATE..: Sun, 09 Aug 2009, 12:24 SUPERVISOR.: Monty IMPLEMENTOR: COPIES TO..: CATEGORY...: Client-RawIdeaBin TASK ID: 39 (http://askmonty.org/worklog/?tid=39) VERSION: Server-9.x STATUS.: Un-Assigned PRIORITY...: 60 WORKED HOURS...: 17 ESTIMATE...: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Guest - Fri, 14 Aug 2009, 15:52)=-=- Title modified. --- /tmp/wklog.39.old.11123 2009-08-14 15:52:29.0 +0300 +++ /tmp/wklog.39.new.11123 2009-08-14 15:52:29.0 +0300 @@ -1 +1 @@ -Replication tasks +improving mysqlbinlog output and doing rename -=-=(Guest - Mon, 10 Aug 2009, 16:32)=-=- Adding 1 hour for Monty's initial work on starting the architecture review. Worked 1 hour and estimate 0 hours remain (original estimate increased by 1 hour). -=-=(Psergey - Mon, 10 Aug 2009, 15:59)=-=- Re-searched and added subtasks. Worked 16 hours and estimate 0 hours remain (original estimate increased by 16 hours). -=-=(Psergey - Mon, 10 Aug 2009, 15:31)=-=- Dependency created: 39 now depends on 41 -=-=(Guest - Mon, 10 Aug 2009, 14:52)=-=- Dependency created: 39 now depends on 40 -=-=(Psergey - Sun, 09 Aug 2009, 12:27)=-=- Dependency created: 39 now depends on 36 -=-=(Psergey - Sun, 09 Aug 2009, 12:24)=-=- Dependency created: 39 now depends on 38 -=-=(Psergey - Sun, 09 Aug 2009, 12:24)=-=- Dependency created: 39 now depends on 37 DESCRIPTION: A combine task for all replication tasks. ESTIMATED WORK TIME ESTIMATED COMPLETION DATE --- WorkLog (v3.5.9) ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
[Maria-developers] Updated (by Knielsen): Add a mysqlbinlog option to filter updates to certain tables (40)
---
WORKLOG TASK
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
TASK...: Add a mysqlbinlog option to filter updates to certain tables
CREATION DATE..: Mon, 10 Aug 2009, 13:25
SUPERVISOR.: Monty
IMPLEMENTOR:
COPIES TO..: Psergey
CATEGORY...: Server-RawIdeaBin
TASK ID: 40 (http://askmonty.org/worklog/?tid=40)
VERSION: Server-9.x
STATUS.: Un-Assigned
PRIORITY...: 60
WORKED HOURS...: 0
ESTIMATE...: 0 (hours remain)
ORIG. ESTIMATE.: 0
PROGRESS NOTES:
-=-=(Knielsen - Fri, 14 Aug 2009, 15:47)=-=-
High-Level Specification modified.
--- /tmp/wklog.40.old.10896 2009-08-14 15:47:39.0 +0300
+++ /tmp/wklog.40.new.10896 2009-08-14 15:47:39.0 +0300
@@ -72,3 +72,21 @@
/* !mysqlbinlog: updates t1,db3.t2 */ UPDATE t1 LEFT JOIN ...
and further processing in mysqlbinlog will be trivial.
+
+2.4 Implement server functionality to ignore certain tables
+---
+
+We could add a general facility in the server to ignore certain tables:
+
+ SET SESSION ignored_tables = "db1.t1,db2.t2";
+
+This would work similar to --replicate-ignore-table, but in a general way not
+restricted to the slave SQL thread.
+
+It would then be trivial for mysqlbinlog to add such statements at the start
+of the output, or probably the user could just do it manually with no need for
+additional options for mysqlbinlog.
+
+It might be useful to integrate this with the code that already handles
+--replicate-ignore-db and similar slave options.
+
-=-=(Psergey - Mon, 10 Aug 2009, 15:41)=-=-
High-Level Specification modified.
--- /tmp/wklog.40.old.12989 2009-08-10 15:41:23.0 +0300
+++ /tmp/wklog.40.new.12989 2009-08-10 15:41:23.0 +0300
@@ -1,6 +1,7 @@
-
1. Context
--
+(See http://askmonty.org/wiki/index.php/Scratch/ReplicationOptions for global
+overview)
At the moment, the server has these replication slave options:
--replicate-do-table=db.tbl
-=-=(Guest - Mon, 10 Aug 2009, 14:52)=-=-
Dependency created: 39 now depends on 40
-=-=(Guest - Mon, 10 Aug 2009, 14:51)=-=-
High Level Description modified.
--- /tmp/wklog.40.old.16985 2009-08-10 14:51:59.0 +0300
+++ /tmp/wklog.40.new.16985 2009-08-10 14:51:59.0 +0300
@@ -1,3 +1,4 @@
Replication slave can be set to filter updates to certain tables with
---replicate-[wild-]{do,ignore}-table options. This task is about adding similar
-functionality to mysqlbinlog.
+--replicate-[wild-]{do,ignore}-table options.
+
+This task is about adding similar functionality to mysqlbinlog.
-=-=(Guest - Mon, 10 Aug 2009, 14:51)=-=-
High-Level Specification modified.
--- /tmp/wklog.40.old.16949 2009-08-10 14:51:33.0 +0300
+++ /tmp/wklog.40.new.16949 2009-08-10 14:51:33.0 +0300
@@ -1 +1,73 @@
+1. Context
+--
+At the moment, the server has these replication slave options:
+
+ --replicate-do-table=db.tbl
+ --replicate-ignore-table=db.tbl
+ --replicate-wild-do-table=pattern.pattern
+ --replicate-wild-ignore-table=pattern.pattern
+
+They affect both RBR and SBR events. SBR events are checked after the
+statement has been parsed, the server iterates over list of used tables and
+checks them againist --replicate instructions.
+
+What is interesting is that this scheme still allows to update the ignored
+table through a VIEW.
+
+2. Table filtering in mysqlbinlog
+-
+
+Per-table filtering of RBR events is easy (as it is relatively easy to extract
+the name of the table that the event applies to).
+
+Per-table filtering of SBR events is hard, as generally it is not apparent
+which tables the statement refers to.
+
+This opens possible options:
+
+2.1 Put the parser into mysqlbinlog
+---
+Once we have a full parser in mysqlbinlog, we'll be able to check which tables
+are used by a statement, and will allow to show behaviour identical to those
+that one obtains when using --replicate-* slave options.
+
+(It is not clear how much effort is needed to put the parser into mysqlbinlog.
+Any guesses?)
+
+
+2.2 Use dumb regexp match
+-
+Use a really dumb approach. A query is considered to be modifying table X if
+it matches an expression
+
+CREATE TABLE $tablename
+DROP $tablename
+UPDATE ...$tablename ... SET// here '...' can't contain the word 'SET'
+DELETE ...$tablename ... WHERE // same as above
+ALTER TABLE $tablename
+.. etc (go get from the grammar) ..
+
+The advantage over doing the same in awk is that mysqlbinlog will also process
+RBR statements, and together with that will provide a working solution for
+those who are careful with their table names not mixing with string constants
+and such.
+
+(TODO: string constants are of particular concern as they come from
+[potentially hos
[Maria-developers] Updated (by Knielsen): Add a mysqlbinlog option to filter updates to certain tables (40)
---
WORKLOG TASK
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
TASK...: Add a mysqlbinlog option to filter updates to certain tables
CREATION DATE..: Mon, 10 Aug 2009, 13:25
SUPERVISOR.: Monty
IMPLEMENTOR:
COPIES TO..: Psergey
CATEGORY...: Server-RawIdeaBin
TASK ID: 40 (http://askmonty.org/worklog/?tid=40)
VERSION: Server-9.x
STATUS.: Un-Assigned
PRIORITY...: 60
WORKED HOURS...: 0
ESTIMATE...: 0 (hours remain)
ORIG. ESTIMATE.: 0
PROGRESS NOTES:
-=-=(Knielsen - Fri, 14 Aug 2009, 15:47)=-=-
High-Level Specification modified.
--- /tmp/wklog.40.old.10896 2009-08-14 15:47:39.0 +0300
+++ /tmp/wklog.40.new.10896 2009-08-14 15:47:39.0 +0300
@@ -72,3 +72,21 @@
/* !mysqlbinlog: updates t1,db3.t2 */ UPDATE t1 LEFT JOIN ...
and further processing in mysqlbinlog will be trivial.
+
+2.4 Implement server functionality to ignore certain tables
+---
+
+We could add a general facility in the server to ignore certain tables:
+
+ SET SESSION ignored_tables = "db1.t1,db2.t2";
+
+This would work similar to --replicate-ignore-table, but in a general way not
+restricted to the slave SQL thread.
+
+It would then be trivial for mysqlbinlog to add such statements at the start
+of the output, or probably the user could just do it manually with no need for
+additional options for mysqlbinlog.
+
+It might be useful to integrate this with the code that already handles
+--replicate-ignore-db and similar slave options.
+
-=-=(Psergey - Mon, 10 Aug 2009, 15:41)=-=-
High-Level Specification modified.
--- /tmp/wklog.40.old.12989 2009-08-10 15:41:23.0 +0300
+++ /tmp/wklog.40.new.12989 2009-08-10 15:41:23.0 +0300
@@ -1,6 +1,7 @@
-
1. Context
--
+(See http://askmonty.org/wiki/index.php/Scratch/ReplicationOptions for global
+overview)
At the moment, the server has these replication slave options:
--replicate-do-table=db.tbl
-=-=(Guest - Mon, 10 Aug 2009, 14:52)=-=-
Dependency created: 39 now depends on 40
-=-=(Guest - Mon, 10 Aug 2009, 14:51)=-=-
High Level Description modified.
--- /tmp/wklog.40.old.16985 2009-08-10 14:51:59.0 +0300
+++ /tmp/wklog.40.new.16985 2009-08-10 14:51:59.0 +0300
@@ -1,3 +1,4 @@
Replication slave can be set to filter updates to certain tables with
---replicate-[wild-]{do,ignore}-table options. This task is about adding similar
-functionality to mysqlbinlog.
+--replicate-[wild-]{do,ignore}-table options.
+
+This task is about adding similar functionality to mysqlbinlog.
-=-=(Guest - Mon, 10 Aug 2009, 14:51)=-=-
High-Level Specification modified.
--- /tmp/wklog.40.old.16949 2009-08-10 14:51:33.0 +0300
+++ /tmp/wklog.40.new.16949 2009-08-10 14:51:33.0 +0300
@@ -1 +1,73 @@
+1. Context
+--
+At the moment, the server has these replication slave options:
+
+ --replicate-do-table=db.tbl
+ --replicate-ignore-table=db.tbl
+ --replicate-wild-do-table=pattern.pattern
+ --replicate-wild-ignore-table=pattern.pattern
+
+They affect both RBR and SBR events. SBR events are checked after the
+statement has been parsed, the server iterates over list of used tables and
+checks them againist --replicate instructions.
+
+What is interesting is that this scheme still allows to update the ignored
+table through a VIEW.
+
+2. Table filtering in mysqlbinlog
+-
+
+Per-table filtering of RBR events is easy (as it is relatively easy to extract
+the name of the table that the event applies to).
+
+Per-table filtering of SBR events is hard, as generally it is not apparent
+which tables the statement refers to.
+
+This opens possible options:
+
+2.1 Put the parser into mysqlbinlog
+---
+Once we have a full parser in mysqlbinlog, we'll be able to check which tables
+are used by a statement, and will allow to show behaviour identical to those
+that one obtains when using --replicate-* slave options.
+
+(It is not clear how much effort is needed to put the parser into mysqlbinlog.
+Any guesses?)
+
+
+2.2 Use dumb regexp match
+-
+Use a really dumb approach. A query is considered to be modifying table X if
+it matches an expression
+
+CREATE TABLE $tablename
+DROP $tablename
+UPDATE ...$tablename ... SET// here '...' can't contain the word 'SET'
+DELETE ...$tablename ... WHERE // same as above
+ALTER TABLE $tablename
+.. etc (go get from the grammar) ..
+
+The advantage over doing the same in awk is that mysqlbinlog will also process
+RBR statements, and together with that will provide a working solution for
+those who are careful with their table names not mixing with string constants
+and such.
+
+(TODO: string constants are of particular concern as they come from
+[potentially hos
[Maria-developers] Updated (by Knielsen): Add a mysqlbinlog option to filter certain kinds of statements (41)
--- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...: Add a mysqlbinlog option to filter certain kinds of statements CREATION DATE..: Mon, 10 Aug 2009, 15:30 SUPERVISOR.: Monty IMPLEMENTOR: COPIES TO..: CATEGORY...: Client-BackLog TASK ID: 41 (http://askmonty.org/worklog/?tid=41) VERSION: Benchmarks-3.0 STATUS.: Un-Assigned PRIORITY...: 60 WORKED HOURS...: 0 ESTIMATE...: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 Aug 2009, 14:17)=-=- High-Level Specification modified. --- /tmp/wklog.41.old.6963 2009-08-14 14:17:32.0 +0300 +++ /tmp/wklog.41.new.6963 2009-08-14 14:17:32.0 +0300 @@ -1,6 +1,11 @@ The implementation will depend on design choices made in WL#40: -- If we decide to parse the statement, SQL-verb filtering will be trivial -- If we decide not to parse the statement, we still can reliably distinguish the + +Option 1: + +If we decide to parse the statement, SQL-verb filtering will be trivial + +Option 2: +If we decide not to parse the statement, we still can reliably distinguish the statement by matching the first characters against a set of patterns. If we chose the second, we'll have to perform certain normalization before -=-=(Psergey - Mon, 10 Aug 2009, 15:47)=-=- High-Level Specification modified. --- /tmp/wklog.41.old.13282 2009-08-10 15:47:13.0 +0300 +++ /tmp/wklog.41.new.13282 2009-08-10 15:47:13.0 +0300 @@ -2,3 +2,10 @@ - If we decide to parse the statement, SQL-verb filtering will be trivial - If we decide not to parse the statement, we still can reliably distinguish the statement by matching the first characters against a set of patterns. + +If we chose the second, we'll have to perform certain normalization before +matching the patterns: + - Remove all comments from the command + - Remove all pre-space + - Compare the string case-insensitively + - etc -=-=(Psergey - Mon, 10 Aug 2009, 15:35)=-=- High-Level Specification modified. --- /tmp/wklog.41.old.12689 2009-08-10 15:35:04.0 +0300 +++ /tmp/wklog.41.new.12689 2009-08-10 15:35:04.0 +0300 @@ -1 +1,4 @@ - +The implementation will depend on design choices made in WL#40: +- If we decide to parse the statement, SQL-verb filtering will be trivial +- If we decide not to parse the statement, we still can reliably distinguish the +statement by matching the first characters against a set of patterns. -=-=(Psergey - Mon, 10 Aug 2009, 15:31)=-=- Dependency created: 39 now depends on 41 DESCRIPTION: Add a mysqlbinlog option to filter certain kinds of statements, i.e. (syntax subject to discussion): mysqlbinlog --exclude='alter table,drop table,alter database,...' HIGH-LEVEL SPECIFICATION: The implementation will depend on design choices made in WL#40: Option 1: If we decide to parse the statement, SQL-verb filtering will be trivial Option 2: If we decide not to parse the statement, we still can reliably distinguish the statement by matching the first characters against a set of patterns. If we chose the second, we'll have to perform certain normalization before matching the patterns: - Remove all comments from the command - Remove all pre-space - Compare the string case-insensitively - etc ESTIMATED WORK TIME ESTIMATED COMPLETION DATE --- WorkLog (v3.5.9) ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
[Maria-developers] Updated (by Knielsen): Add a mysqlbinlog option to filter certain kinds of statements (41)
--- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...: Add a mysqlbinlog option to filter certain kinds of statements CREATION DATE..: Mon, 10 Aug 2009, 15:30 SUPERVISOR.: Monty IMPLEMENTOR: COPIES TO..: CATEGORY...: Client-BackLog TASK ID: 41 (http://askmonty.org/worklog/?tid=41) VERSION: Benchmarks-3.0 STATUS.: Un-Assigned PRIORITY...: 60 WORKED HOURS...: 0 ESTIMATE...: 0 (hours remain) ORIG. ESTIMATE.: 0 PROGRESS NOTES: -=-=(Knielsen - Fri, 14 Aug 2009, 14:17)=-=- High-Level Specification modified. --- /tmp/wklog.41.old.6963 2009-08-14 14:17:32.0 +0300 +++ /tmp/wklog.41.new.6963 2009-08-14 14:17:32.0 +0300 @@ -1,6 +1,11 @@ The implementation will depend on design choices made in WL#40: -- If we decide to parse the statement, SQL-verb filtering will be trivial -- If we decide not to parse the statement, we still can reliably distinguish the + +Option 1: + +If we decide to parse the statement, SQL-verb filtering will be trivial + +Option 2: +If we decide not to parse the statement, we still can reliably distinguish the statement by matching the first characters against a set of patterns. If we chose the second, we'll have to perform certain normalization before -=-=(Psergey - Mon, 10 Aug 2009, 15:47)=-=- High-Level Specification modified. --- /tmp/wklog.41.old.13282 2009-08-10 15:47:13.0 +0300 +++ /tmp/wklog.41.new.13282 2009-08-10 15:47:13.0 +0300 @@ -2,3 +2,10 @@ - If we decide to parse the statement, SQL-verb filtering will be trivial - If we decide not to parse the statement, we still can reliably distinguish the statement by matching the first characters against a set of patterns. + +If we chose the second, we'll have to perform certain normalization before +matching the patterns: + - Remove all comments from the command + - Remove all pre-space + - Compare the string case-insensitively + - etc -=-=(Psergey - Mon, 10 Aug 2009, 15:35)=-=- High-Level Specification modified. --- /tmp/wklog.41.old.12689 2009-08-10 15:35:04.0 +0300 +++ /tmp/wklog.41.new.12689 2009-08-10 15:35:04.0 +0300 @@ -1 +1,4 @@ - +The implementation will depend on design choices made in WL#40: +- If we decide to parse the statement, SQL-verb filtering will be trivial +- If we decide not to parse the statement, we still can reliably distinguish the +statement by matching the first characters against a set of patterns. -=-=(Psergey - Mon, 10 Aug 2009, 15:31)=-=- Dependency created: 39 now depends on 41 DESCRIPTION: Add a mysqlbinlog option to filter certain kinds of statements, i.e. (syntax subject to discussion): mysqlbinlog --exclude='alter table,drop table,alter database,...' HIGH-LEVEL SPECIFICATION: The implementation will depend on design choices made in WL#40: Option 1: If we decide to parse the statement, SQL-verb filtering will be trivial Option 2: If we decide not to parse the statement, we still can reliably distinguish the statement by matching the first characters against a set of patterns. If we chose the second, we'll have to perform certain normalization before matching the patterns: - Remove all comments from the command - Remove all pre-space - Compare the string case-insensitively - etc ESTIMATED WORK TIME ESTIMATED COMPLETION DATE --- WorkLog (v3.5.9) ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] Documentation for new features in MariaDB
Henrik Ingo writes: > Now that I think about it, maybe this part of the documentation effort > could for the time being be done as a diff against the MySQL manual. > (Which is DocBook?) Have you checked the implications of the copyright/license issues for this idea? - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] Documentation for new features in MariaDB
Henrik Ingo wrote: Now that I think about it, maybe this part of the documentation effort could for the time being be done as a diff against the MySQL manual. (Which is DocBook?) According to this, yes docbook. http://dev.mysql.com/doc/mysqldoc-guide/en/mysqldoc-guide-overview.html I guess I should do a presentation in Majorca about this. Very good idea. I could actually give you from 30 to 60 minutes on Saturday as a scheduled presentation (not unconference). 30 minutes should be more then enough, unless people have a lot of questions. -- Bryan Alsdorf, Lead Web Developer Monty Program, AB. http://askmonty.org ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
