Re: [Maria-developers] request for feedback on live schema changes patch

[Zardosht Kasheff]

Hello all,

I am hoping to resurrect this discussion. Has there been any
discussion on implementing the feature of allowing handlers to do fast
schema changes in MariaDB (on any version)?

I have put this patch on MySQL 5.1, and I think I have it working. I
started looking at MariaDB 5.2, but I do not understand some of the
virtual column stuff and field->option_struct and key->option_struct.
I am guessing they are related to the new create table syntax, but I
do not understand it well enough to be able to integrate this patch
in.

Any thoughts?

Thanks
-Zardosht

On Thu, Nov 18, 2010 at 2:53 PM, Zardosht Kasheff  wrote:
> Hello,
>
> I am wondering if we can resurrect this discussion. In summary, I have
> a patch that I think I can apply to 5.1, although I realize it is not
> the ideal solution. It is a solution that I could code that minimizes
> the risk of introducing a regression for storage engines that do not
> use this.
>
> The fix is basically to have a new alter table for storage engines
> that want this feature, and the old alter table for others.
>
> The two biggest reasons I did it this way was:
>  - I did not want to complicate the fix by handling storage engines
> that implement handler::add_index.
>  - I did not understand the partitioning fixes required.
>
> That being said, what do people think are the next steps that should
> be taken? I would love to see this feature in MariaDB 5.3, but I do
> not have the expertise to be able to complete the feature. I am just
> moving code from one MySQL branch to another.
>
> Thanks
> -Zardosht
>
> On Fri, Oct 1, 2010 at 8:59 AM, Zardosht Kasheff  wrote:
>> I did it this way for two reasons:
>>  - I did not want to try to port partitioning's alter table changes.
>> That was getting into a realm where I could easily mess up
>>  - I was concerned with implementing the backwards-compatible
>> emulation of the old API. I figured i would do it this way for 5.1,
>> and then future versions that do not need to be instantly GA-ready
>> could either deprecate some of the old interface or have a backward
>> compatible emulation.
>>
>> So, I agree with what you state below about having one alter table
>> path, but I figured the safe way to do that would be for future
>> versions that are not currently GA. Any thoughts on this?
>>
>> The cluster code does not support the following alter table flags
>> (they do not even exist):
>>
>> #define HA_ONLINE_ADD_INDEX_NO_WRITES           (1L << 0) /*add index 
>> w/lock*/
>> #define HA_ONLINE_DROP_INDEX_NO_WRITES          (1L << 1) /*drop index 
>> w/lock*/
>> #define HA_ONLINE_ADD_UNIQUE_INDEX_NO_WRITES    (1L << 2) /*add unique 
>> w/lock*/
>> #define HA_ONLINE_DROP_UNIQUE_INDEX_NO_WRITES   (1L << 3) /*drop uniq. 
>> w/lock*/
>> #define HA_ONLINE_ADD_PK_INDEX_NO_WRITES        (1L << 4) /*add prim. 
>> w/lock*/
>> #define HA_ONLINE_DROP_PK_INDEX_NO_WRITES       (1L << 5) /*drop prim. 
>> w/lock*/
>> #define HA_ONLINE_ADD_INDEX                     (1L << 6) /*add index 
>> online*/
>> #define HA_ONLINE_DROP_INDEX                    (1L << 7) /*drop index 
>> online*/
>> #define HA_ONLINE_ADD_UNIQUE_INDEX              (1L << 8) /*add unique 
>> online*/
>> #define HA_ONLINE_DROP_UNIQUE_INDEX             (1L << 9) /*drop uniq. 
>> online*/
>> #define HA_ONLINE_ADD_PK_INDEX                  (1L << 10)/*add prim. 
>> online*/
>> #define HA_ONLINE_DROP_PK_INDEX                 (1L << 11)/*drop prim. 
>> online*/
>>
>> So, what would need to happen is for the default implementations of
>> the new interface to check for and support these flags.
>>
>> -Zarodsht
>>
>> On Fri, Oct 1, 2010 at 3:44 AM, Kristian Nielsen
>>  wrote:
>>> Zardosht Kasheff  writes:
>>>
 That being said, I would love to hear feedback on this patch, and
 would also like to understand what (if anything) can be done to get
 this into MariaDB 5.1. I realize that may not be possible, but if it
 is, I would like to try.
>>>
>>> I didn't look at the patch yet. However, I would oppose putting this into
>>> MariaDB if it is a hodge-podge mixture of the old and the new alter table
>>> code.
>>>
>>> (Just imagine if the existing code you had to work in had already contained 
>>> 4
>>> versions of alter table code, from MySQL 3.x, MySQL 4.0, MySQL 4.1, and 
>>> MySQL
>>> 5.1, say. I'm sure you found the task sufficiently tricky with just one
>>> version to understand).
>>>
>>> On the other hand, it might make good sense to support the old interface in
>>> storage engines, for example by implementing a backward-compatible emulation
>>> of the old API on top of the new alter table code.
>>>
>>> How is it done in the original Cluster tree that you based your work on? 
>>> Does
>>> it convert all storage engines to the new interface? Does it implement
>>> backwards compatibility on top of the new interface, leaving (some) storage
>>> engines unmodified? Or does it contain both the old and the new code, like
>>> your patch?
>>>
>>>  - Kristian.
>>>
>>
>

___

Re: [Maria-developers] Windows installer MWL#55 finished.

[Vladislav Vaintroub]

 
 
From: Peter Laursen [mailto:peter_laur...@webyog.com] 
Sent: Mittwoch, 2. März 2011 11:30
To: Vladislav Vaintroub
Cc: Kazuhisa Ichikawa; maria-developers@lists.launchpad.net
Subject: Re: [Maria-developers] Windows installer MWL#55 finished.
 
It seems that   
http://downloads.askmonty.org/mariadb/ still has the old NSIS-based installer 
for 5.1.55? – Peter
Yes, 5.1 will have the old installer until the end of 5.1’s life.  Also, in 
5.2/5.3 the new installer is not pushed yet, because it is not yet reviewed. 
On Tue, Feb 8, 2011 at 00:46, Vladislav Vaintroub < 
 w...@montyprogram.com> wrote:
 
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] Rev 2934: Fix LP BUG#719198 in file:///home/tsk/mprog/src/5.3/

[Sergey Petrunya]

Hi Timour,

On Tue, Mar 08, 2011 at 04:35:24PM +0200, tim...@askmonty.org wrote:
> At file:///home/tsk/mprog/src/5.3/
> 
> 
> revno: 2934
> revision-id: tim...@askmonty.org-20110308143512-0m7uotvxp9rmegak
> parent: pser...@askmonty.org-20110304155430-l0ad4iag9gn5zeka
> committer: tim...@askmonty.org
> branch nick: 5.3
> timestamp: Tue 2011-03-08 16:35:12 +0200
> message:
>   Fix LP BUG#719198
>   
>   Analysis:
>   The assert failed because the execution code for
>   partial matching is designed with the assumption that
>   NULLs on the left side are detected as early as possible,
>   and a NULL result is returned before any lookups are
>   performed at all.
>   
>   However, in the case of an Item_cache object on the left
>   side, null was not detected properly, because detection
>   was done via Item::is_null(), which is not implemented at
>   all for Item_cache, and resolved to the default Item::is_null()
>   which always returns FALSE.
>   
>   Solution:
>   Use the property Item::null_value instead of is_null(), which
>   is properly updated for Item_cache objects as well.
>   
>   Alternatively one could implement Item_cache:is_null(), but
>   its not quite clear how that will interact with the cache
>   update methods, so the current fix seems simpler and cleaner.

Ok to push.

Since it is guaranteed that cache_value() have been called for the object,
it's ok to use null_value instead of is_null().

Btw, out of interest I've implemented Item_cache::is_null() and 
t/subselect*.test ran successfully, so it seems to be ok to have. 
I don't insist that it is added as part of this fix, though.


> === modified file 'mysql-test/r/subselect_mat.result'
> --- a/mysql-test/r/subselect_mat.result   2011-01-14 10:51:30 +
> +++ b/mysql-test/r/subselect_mat.result   2011-03-08 14:35:12 +
> @@ -1371,3 +1371,41 @@ SELECT pk FROM t1 WHERE (b,c,d) IN (SELE
>  pk
>  2
>  DROP TABLE t1, t2;
> +#
> +# LPBUG#719198 Ordered_key::cmp_key_with_search_key(rownum_t): Assertion 
> `!compare_pred[i]->null_value'
> +# failed with subquery on both sides of NOT IN and materialization
> +#
> +CREATE TABLE t1 (f1a int, f1b int) ;
> +INSERT IGNORE INTO t1 VALUES (1,1),(2,2);
> +CREATE TABLE t2 ( f2 int);
> +INSERT IGNORE INTO t2 VALUES (3),(4);
> +CREATE TABLE t3 (f3a int, f3b int);
> +set session 
> optimizer_switch='materialization=on,partial_match_rowid_merge=on,partial_match_table_scan=off';
> +EXPLAIN
> +SELECT * FROM t2 WHERE (SELECT f3a FROM t3) NOT IN (SELECT f1a FROM t1);
> +id  select_type table   typepossible_keys   key key_len ref  
>rowsExtra
> +1   PRIMARY t2  ALL NULLNULLNULLNULL2   
> Using where
> +3   SUBQUERYt1  ALL NULLNULLNULLNULL2
>
> +2   SUBQUERYt3  system  NULLNULLNULLNULL0
>const row not found
> +SELECT * FROM t2 WHERE (SELECT f3a FROM t3) NOT IN (SELECT f1a FROM t1);
> +f2
> +EXPLAIN
> +SELECT * FROM t2 WHERE (SELECT f3a, f3b FROM t3) NOT IN (SELECT f1a, f1b 
> FROM t1);
> +id  select_type table   typepossible_keys   key key_len ref  
>rowsExtra
> +1   PRIMARY t2  ALL NULLNULLNULLNULL2   
> Using where
> +3   SUBQUERYt1  ALL NULLNULLNULLNULL2
>
> +2   SUBQUERYt3  system  NULLNULLNULLNULL0
>const row not found
> +SELECT * FROM t2 WHERE (SELECT f3a, f3b FROM t3) NOT IN (SELECT f1a, f1b 
> FROM t1);
> +f2
> +insert into t3 values (1,1),(2,2);
> +EXPLAIN
> +SELECT * FROM t2 WHERE (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a 
> FROM t1);
> +id  select_type table   typepossible_keys   key key_len ref  
>rowsExtra
> +1   PRIMARY t2  ALL NULLNULLNULLNULL2   
> Using where
> +3   SUBQUERYt1  ALL NULLNULLNULLNULL2
>
> +2   SUBQUERYt3  ALL NULLNULLNULLNULL2
>Using where
> +SELECT * FROM t2 WHERE (SELECT f3a FROM t3 where f3a > 3) NOT IN (SELECT f1a 
> FROM t1);
> +f2
> +3
> +4
> +drop table t1, t2, t3;
> 
> === modified file 'mysql-test/t/subselect_mat.test'
> --- a/mysql-test/t/subselect_mat.test 2011-01-14 10:51:30 +
> +++ b/mysql-test/t/subselect_mat.test 2011-03-08 14:35:12 +
> @@ -1011,3 +1011,31 @@ SELECT pk FROM t1 WHERE (a) IN (SELECT a
>  SELECT pk FROM t1 WHERE (b,c,d) IN (SELECT b,c,d FROM t2 WHERE pk > 0);
>  DROP TABLE t1, t2;
>  
> +--echo #
> +--echo # LPBUG#719198 Ordered_key::cmp_key_with_search_key(rownum_t): 
> Assertion `!compare_pred[i]->null_value'
> +--echo # failed with subquery on both sides of NOT IN and materialization
> +--echo #
> +
> +CREATE TABLE t1 (f1a int, f1b int) ;
> +INSERT IGNORE INTO t1 VALUES (1,1),(2,2);
> +CREATE TABLE t2 ( f2 int);
> +INSERT IGNORE INTO t2 VALUES (3)

[Maria-developers] Review: Part 3 of 3 for MWL#90 combined patch for review

[Michael Widenius]

Hi!

Here is the third and last part of the review.

> "Sergey" == Sergey Petrunya  writes:



> @@ -7433,8 +7718,18 @@
>  
>  table_map used_tables2= (join->const_table_map |
>   OUTER_REF_TABLE_BIT | RAND_TABLE_BIT);
> - for (tab= join->join_tab+join->const_tables; tab <= last_tab ; tab++)
> +for (JOIN_TAB *tab= first_linear_tab(join, TRUE); 
> + tab; 
> + tab= next_linear_tab(join, tab, TRUE))
>  {
> +  if (!tab->table)
> +  {
> +/* 
> +  psergey-todo: this is probably incorrect, fix this when we get
> +  correct processing for outer joins + semi joins 
> +*/
> +continue;
> +  }

When is the above fix going to happen ? (Which worklog).
What is the effect if we execute the continue?
a) Wrong result
b) Crash
3) Slower code ?

Please add a comment about this if it's 3)  If it's 1 or 2 we should
fix it before pushing to trunk.

> @@ -7480,37 +7775,12 @@
>  (*sel_cond_ref)->update_used_tables();
>  if (cond_tab->select)
>cond_tab->select->cond= cond_tab->select_cond;
> -  }  
> +  }
> +  if (tab == last_tab)
> +break;

Wouldn't it be better to fix the 'for' loop, as otherwise you may get
a crash if tab->table is not true for the tab == last_tab.

How about this for a better for loop

for (JOIN_TAB *tab= first_linear_tab(join, TRUE), prev_tab= 0; 
 prev_tab != last_tab;
 prev_tab= tab, tab = next_linear_tab(join, tab, TRUE))

Note that we don't have to test for tab == 0 as we will always find
last_tab first.

> @@ -7968,8 +8240,8 @@
>  Don't use join buffering if we're dictated not to by no_jbuf_after (this
>  ...)
>*/
> -  if (!(i <= no_jbuf_after) || tab->loosescan_match_tab || 
> -  sj_is_materialize_strategy(join->best_positions[i].sj_strategy))
> +  if ((!tab->bush_root_tab? !(i <= no_jbuf_after) : FALSE) || 
> +   tab->loosescan_match_tab || tab->bush_children)
>  goto no_join_cache;

The above was a bit complex if. How about:

 if (((!tab->bush_root_tab && !(i <= no_jbuf_after)) || 
   tab->loosescan_match_tab || tab->bush_children)
goto no_join_cache;

Don't really understand how we could use join_cache with
SMJ_NEST child (ie, a tab with tab->bush_root_tab set).


>for (JOIN_TAB *first_inner= tab->first_inner; first_inner;
> @@ -8122,16 +8394,24 @@
>  void check_join_cache_usage_for_tables(JOIN *join, ulonglong options,
> uint no_jbuf_after)
>  {
> -  JOIN_TAB *first_sjm_table= NULL;
> -  JOIN_TAB *last_sjm_table= NULL;
> +  //JOIN_TAB *first_sjm_table= NULL;
> +  //JOIN_TAB *last_sjm_table= NULL;

Remove comments.  (Please remove all these before you request a review
as the deleted lines will show up nicely in the diff anyway).

> +  JOIN_TAB *tab;
>  
> -  for (uint i= join->const_tables; i < join->tables; i++)
> -join->join_tab[i].used_join_cache_level= 
> join->max_allowed_join_cache_level;  
> -   
> -  for (uint i= join->const_tables; i < join->tables; i++)
> -  {
> -JOIN_TAB *tab= join->join_tab+i;
> +  //for (uint i= join->const_tables; i < join->tables; i++)

Remove comment

> +  for (tab= first_linear_tab(join, TRUE); 
> +   tab; 
> +   tab= next_linear_tab(join, tab, TRUE))
> +  {
> +tab->used_join_cache_level= join->max_allowed_join_cache_level;  
> +  }
>  
> +  //for (uint i= join->const_tables; i < join->tables; i++)

Remove comment

> +  for (tab= first_linear_tab(join, TRUE); 
> +   tab; 
> +   tab= next_linear_tab(join, tab, TRUE))
> +  {
> +#if 0
>  if (sj_is_materialize_strategy(join->best_positions[i].sj_strategy))
>  {
>first_sjm_table= tab;
> @@ -8142,9 +8422,16 @@
>  } 
>  if (!(tab >= first_sjm_table && tab < last_sjm_table))
>tab->first_sjm_sibling= NULL;
> -
> +#endif 

Remove above ifdefed block


> +JOIN_TAB *prev_tab;
> +restart:
>  tab->icp_other_tables_ok= TRUE;
>  tab->idx_cond_fact_out= TRUE;
> +
> +prev_tab= tab - 1;

Isn't it more clear to set prev_tab as part of the for loop?
(This will make it independent of how next_linear_tab() is implemented)

> @@ -8154,12 +8441,22 @@

> +  /*
>if (join->return_tab)
>  i= join->return_tab-join->join_tab-1;   // always >= 0
> +  */

Remove commented code


> @@ -8204,10 +8501,17 @@
>setup_semijoin_dups_elimination(join, options, no_jbuf_after))
>  DBUG_RETURN(TRUE); /* purecov: inspected */
>  
> -  for (i= 0; i < join->const_tables; i++)
> -join->join_tab[i].partial_join_cardinality= 1; 
> +  for (tab= first_linear_tab(join, TRUE); 
> +   tab; 
> +   tab= next_linear_tab(join, tab, TRUE))
> +  {
> +tab->partial_join_cardinality= 1; 
> +  }

Isn't the above loop wrong ?
It was supposed to only loop over const tables, not skip const tables.

> -  for (i=join->cons

Re: [Maria-developers] Report from OpenSourceDays2011

[Adam M. Dutko]

> That's it :-)

Thanks for the update. It's good to read about what some of the core
members are doing to evangelize in the community.

___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Report from OpenSourceDays2011

[Kristian Nielsen]

Hi,

On Saturday, March 5, I gave a talk at the OpenSourceDays 2011 conference in
Copenhagen, Denmark:

http://opensourcedays.org/

My talk was titled "Latest news from the MariaDB (and MySQL) community". I put
the slides from the talk up on my web server here:

http://knielsen-hq.org/maria/osd2011.pdf

This year the OpenSourceDays conference was reduced somewhat from what it
usually is, only one day and I heard something like 400 tickes available
(which were sold out rather quickly). Still, I had a good bunch of people at
the talk, maybe 30 or so. There were many questions, people seemed interested,
so I think it went well.

I also talked to some people outside of the talk, again there seems to be a
lot of interest in what is happening around MySQL generally and in MariaDB
specifically.

I think generally people are pleasantly surprised to learn the depth and
dedication of the MariaDB effort, that we really are a major effort and a
viable alternative.

There was one good question at the talk about if there were reference
customer statements or so to show that companies are deploying MariaDB with
success at large scale; I guess this will be useful for people that want to
sell the idea of using MariaDB at their job or institution or whatever. I did
not have a very good answer to this, except that I knew there were some. I
guess we already have this on one of the web sites, or if not it would be
worth it to collect the different press releases and such as we get them, I
think.

The night before the conference there was a speaker's dinner, and I had a
great talk with Magnus from Postgresql. I got the impression that things are
also really moving in the Postgres space, and it seems to be picking up a lot
of interest and development effort. I think that is great, I am starting to be
hopeful that we are now seeing the open source databases really take off as a
community effort, as opposed to just a project of whatever "acquirer of the
year".

That's it :-)

 - Kristian.

___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Pbxt loses rows

[Michael Widenius]

Hi!

> "Paul" == Paul McCullagh  writes:

Paul> I suspect that this is not due to rows being lost, but rather to the  
Paul> rows not being found for some timing reason.

Paul> But, of course, only debugging will tell...

Paul> We have scheduled time to look at this error.

Any progress ?

One of the most common failures happens in preload.test:

@@ -40,10 +40,10 @@
 insert into t1(b) select b from t2;
 select count(*) from t1;
 count(*)
-33448
+33441
 select count(*) from t2;
 count(*)
-20672
+20667
 flush tables;
 flush status;
 show status like "key_read%"

The only thing this test does is a lot of repeated:

insert into t2(b) select b from t1;
insert into t1(b) select b from t2;  

so it should be relatively easy to repeat.

Regards,
Monty

___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp