Re: [Maria-developers] MDEV-9423: FTWRL and Binlog checkpoint
Nirbhay Choubey writes: [Cc: maria-developers@, please always keep these discussions on the mailing list] > In Galera cluster, the state transfer scripts perform FTWRL and > copy data along with the last of all available binlog files to the > joiner node. > > After MDEV-181, I understand that the binlog checkpoint can be > in any of the binary log files (and not necessarily the last one). > > This seemingly has caused MDEV-9423, in which the joiner node > complains of the missing binlog file. > > Now the question is : Is FTWRL not sufficient to ensure that the > checkpoint is always the last binlog file? So if I understand correctly, the issue is related to having binlog files available during XA crash recovery. When the binlog file is rotated, there is a small window where both the latest and the previous binlog files are needed for crash recovery. The binlog checkpoint is the earliest binlog file that is needed for crash recovery, and it can be seen from the binlog checkpoint event. So the problem here is that a copy is made just after binlog rotation, and Galera only copies the most recent, mostly-empty binlog file, leaving insufficient information for XA recovery, right? One option to solve this is to always copy the last two binlog files. While it is theoretically possible to have the binlog checkpoint more than two files back, I think it will not occur in practice. Another option is to wait for the binlog checkpoint to reach the current binlog file. You can see this done in the test suite: mysql-test/include/wait_for_binlog_checkpoint.inc The binlog checkpointing happens asynchroneously, I *think* it can complete even while FTWRL is active, but I am not 100% sure though. The checkpoint happens after InnoDB has made its commits durable with fsync() or similar - only after that is it safe to discard the old binlog data and still have correct crash recovery. - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] ad8b439: MDEV-9618 solaris sparc build fails on 10.1.
Hi, Alexey!
On May 02, Alexey Botchkov wrote:
> revision-id: ad8b4399f2fcd1d5e9dcdb8b2a8832cb3a6745ae
> (mariadb-10.1.13-23-gad8b439)
> parent(s): ad4239cc3dc7ad5f6f264e1fb3cf6d24084bda90
> committer: Alexey Botchkov
> timestamp: 2016-05-02 12:03:39 +0400
> message:
>
> MDEV-9618 solaris sparc build fails on 10.1.
>
> Compiler on Solaris is sensitive to C/C++ call models
> differences, so it fails if we try to mix these two in
> '?' operator.
> Fixed by trying to keep the call models uniformity with the
> proper amount of 'extern "C"' hints.
>
> ---
> include/my_crypt.h | 6 ++
> include/mysql/plugin_audit.h | 7 +++
> include/mysql/plugin_encryption.h | 9 +
> .../example_key_management_plugin.cc | 2 +-
> plugin/file_key_management/file_key_management_plugin.cc | 2 +-
> sql/encryption.cc | 14
> ++
> unittest/sql/mf_iocache-t.cc | 2 +-
> 7 files changed, 35 insertions(+), 7 deletions(-)
>
> diff --git a/include/my_crypt.h b/include/my_crypt.h
> index e1e94c9..db280ca 100644
> --- a/include/my_crypt.h
> +++ b/include/my_crypt.h
> @@ -82,6 +82,12 @@ static inline uint my_aes_ctx_size(enum my_aes_mode mode
> __attribute__((unused))
>return MY_AES_CTX_SIZE;
> }
>
> +static inline uint my_aes_ctx_size_for_handler(unsigned int a,
> + unsigned int b __attribute__((unused)))
> +{
> + return my_aes_ctx_size((enum my_aes_mode) a);
> +}
> +
This won't be inlined, because you store a pointer to this function in a
structure. Better use the cast, as before.
> int my_random_bytes(uchar* buf, int num);
>
> #ifdef __cplusplus
> diff --git a/include/mysql/plugin_audit.h b/include/mysql/plugin_audit.h
> index 31589f0..e96f743 100644
> --- a/include/mysql/plugin_audit.h
> +++ b/include/mysql/plugin_audit.h
> @@ -23,6 +23,10 @@
>
> #include "plugin.h"
>
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
I agree, in general. But why does your short patch have nothing
about audit api?
> #define MYSQL_AUDIT_CLASS_MASK_SIZE 1
>
> #define MYSQL_AUDIT_INTERFACE_VERSION 0x0302
> @@ -174,5 +178,8 @@ struct st_mysql_audit
>unsigned long class_mask[MYSQL_AUDIT_CLASS_MASK_SIZE];
> };
>
> +#ifdef __cplusplus
> +}
> +#endif
>
> #endif
> diff --git a/include/mysql/plugin_encryption.h
> b/include/mysql/plugin_encryption.h
> index 3f35c2b..d748c4f 100644
> --- a/include/mysql/plugin_encryption.h
> +++ b/include/mysql/plugin_encryption.h
> @@ -27,6 +27,10 @@
>
> #include
>
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
...etc
I agree that these extern "C" are the correct fix.
On the other hand, I don't see a point in moving casts from the
assignment to a new variable definition. So, I'd suggest to keep all
extern "C" hunks in the patch, but remove cast avoidance changes.
Then ok to push!
Regards,
Sergei
Chief Architect MariaDB
and secur...@mariadb.org
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] GSoC 2016 Community Bonding
Welcome to the MariaDB project Charles! I'm Colin Charles, and I will be your mentor. Let's say hi and connect on irc as well. Ping me, I'm bytee there On Monday, 2 May 2016, Charles Dirk wrote: > Hi everyone, > > I am Charles Dirk, also Razor on IRC and am glad to be selected as a > Google summer of Code student with MariaDB. I will be working on Providing > GTID support in MHA and I will be mentored by Byte. I look forward to > contributing to and learning much from the MariaDB developer community this > summer. I am currently reading some documentation on KB to get more > pointers on this task while waiting to hear from my Mentor. > > Cheers, > Charles. > > > ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] 9058071: MDEV-9362: InnoDB tables using DATA_DIRECTORY created using MySQL 5.6 do not work with MariaDB 10.1
Hi, Jan! On May 02, Jan Lindström wrote: > Hi Sergei, > > Firstly, thank you for your review. I agree that the proposed patch is > huge and intrusive. Let my try to reason why we should apply it to > 10.1: > > Problem: Oracle MySQL 5.6 database is not binary compatible to MariaDB 10.1 > IF AND ONLY IF DATA_DIRECTORY is used or UNIV_PAGE_SIZE != 16KB > > Actual bug on MDEV-9362 can be fixed by not touching these bitmaps at > all. Note that bug on using UNIV_PAGE_SIZE!= 16KB requires touching > these bitmaps. > > So alternatives are: > (1) Proposed intrusive fix (with some modifications based on review) > (2) Move MariaDB flags to end of bitmap > (3) Just fix reported problem on MDEV-9362 > > Opinions? > R: Jan I would suggest to try (3) in 10.1 and your big (1) patch in 10.2. And then I'd really like to see whether we can do table discovery in InnoDB 10.2 based on your SYS_TABLE_OPTIONS table. Regards, Sergei Chief Architect MariaDB and secur...@mariadb.org ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] InnoDB blob for primary key
Hi, Sachin!
On May 02, Sachin Setia wrote:
> Hi Sergei!
>
> As i told you i was prototyping for hash table
> It is done around 90% apart from one thing when hash is same
> how to get record from .myd file when i have offset of record
> so currently i am skipping it
> But it is very fast i do not know why this is so fast here are
> results of employee database
> salary table definition
> CREATE TABLE salaries (
> emp_no INT NOT NULL,
> salary blob NOT NULL,
> from_date DATENOT NULL,
> to_date DATENOT NULL,
> FOREIGN KEY (emp_no) REFERENCES employees (emp_no) ON DELETE CASCADE,
> PRIMARY KEY (emp_no, from_date)
> )
I presume, you had ENGINE=Aria here? Because your code patch is for Aria.
> ;
> And query is
> MariaDB [employees]> select distinct salary from salaries;
>
> Result with out using hash table
>
> ++
> 85814 rows in set (2 min 24.76 sec)
>
>
> Result with using hash table
>
> | 39420 |
> ++
> 85809 rows in set (6.24 sec)
>
> ( number of rows are not equal but this can be solved if i get record by
> offset)
>
> I am sure there is something wrong.The whole hash table is in memory
> like wise the b tree of hash is in memory but why there is so much
> improvement. Please sir check the prototype and tell if i am wrong
> .thanks
Sure.
But still, please put your code on github and commit often.
Let's use a proper development process instead of sending patches back
and forth. If you need help with that, feel free to ping me on irc.
And using // comments makes the code more difficult to review - you
change every line in a big block. Better use
#if 0
...
#endif
> diff --git a/storage/maria/ma_hash_table.h b/storage/maria/ma_hash_table.h
> new file mode 100644
> index 000..c8e4578
> --- /dev/null
> +++ b/storage/maria/ma_hash_table.h
why are you doing it in Aria? I thought we've agreed to do it on the
upper level, in sql/
> @@ -0,0 +1,45 @@
> +#include"../../mysys/my_malloc.c"
> +#include"../../include/my_global.h"
> +typedef struct ma_hash_table_element{
> + unsigned int hash_code;
> + unsigned int record_offset;
> + struct ma_hash_table * next; //we will use single link list because no
> delete operations
> +} ma_hash_table_element;
Did you look at reusing the HASH data structure? include/hash.h, mysys/hash.c?
> +
> +typedef struct ma_hash_table{
> + unsigned int size;
> + ma_hash_table_element * h_t_e;
> +}ma_hash_table;
Because of the above (on the wrong level, ma_hash_table instead of HASH,
using // comments to disable code) I've only quickly looked through the patch.
But I didn't notice anything obviously wrong.
Regards,
Sergei
Chief Architect MariaDB
and secur...@mariadb.org
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] InnoDB blob for primary key
Hi Sergei!
As i told you i was prototyping for hash table
It is done around 90% apart from one thing when hash is same
how to get record from .myd file when i have offset of record
so currently i am skipping it
But it is very fast i do not know why this is so fast here are
results of employee database
salary table definition
CREATE TABLE salaries (
emp_no INT NOT NULL,
salary blob NOT NULL,
from_date DATENOT NULL,
to_date DATENOT NULL,
FOREIGN KEY (emp_no) REFERENCES employees (emp_no) ON DELETE CASCADE,
PRIMARY KEY (emp_no, from_date)
)
;
And query is
MariaDB [employees]> select distinct salary from salaries;
Result with out using hash table
++
85814 rows in set (2 min 24.76 sec)
Result with using hash table
| 39420 |
++
85809 rows in set (6.24 sec)
( number of rows are not equal but this can be solved if i get record by
offset)
I am sure there is something wrong.The whole hash table is in memory like
wise the
b tree of hash is in memory but why there is so much improvement. Please
sir check the
prototype and tell if i am wrong .thanks
Regards
sachin
On Mon, May 2, 2016 at 11:43 AM, Sergei Golubchik wrote:
> Hi, Sachin!
>
> On May 02, Sachin Setia wrote:
> > I am sorry sir Currently my exam are going on
> > But i am working on prototype of second project. Will be done by tommorow
> > Regards
> > sachin
>
> Sure thing, that's totally fine!
>
> Still, in the future, if you plan to go silent for a while (exam or you
> just want to relax for a few days or something else :) - please drop me
> a short email and then I will know that you didn't disappear from the
> project. Thanks!
>
> Regards,
> Sergei
> Chief Architect MariaDB
> and secur...@mariadb.org
>
diff --git a/storage/maria/ma_hash_table.h b/storage/maria/ma_hash_table.h
new file mode 100644
index 000..c8e4578
--- /dev/null
+++ b/storage/maria/ma_hash_table.h
@@ -0,0 +1,45 @@
+#include"../../mysys/my_malloc.c"
+#include"../../include/my_global.h"
+typedef struct ma_hash_table_element{
+ unsigned int hash_code;
+ unsigned int record_offset;
+ struct ma_hash_table * next; //we will use single link list because no
delete operations
+} ma_hash_table_element;
+
+typedef struct ma_hash_table{
+ unsigned int size;
+ ma_hash_table_element * h_t_e;
+}ma_hash_table;
+ma_hash_table * ma_create_hash_table(int size){
+ ma_hash_table_element * _ht = (ma_hash_table
*)my_malloc(sizeof(ma_hash_table)*size,MYF(MY_WME));
+ ma_hash_table_element * temp=_ht;
+ for(int i=0;inext=NULL;
+ temp++;
+ }
+ ma_hash_table *temp_hash_table = (ma_hash_table
*)my_malloc(sizeof(ma_hash_table),MYF(MY_WME));
+ temp_hash_table->size =size;
+ temp_hash_table->h_t_e = _ht;
+ return temp_hash_table;
+}
+
+int add_key_to_hash_table(ma_hash_table *_ht,unsigned int key , unsigned int
offset){
+ int hash_position = key % _ht->size; //just a simple logic thinking of
upgrading it may we whatever
+ ma_hash_table_element *tmp= _ht->h_t_e;
+ tmp = tmp+hash_position;
+ ma_hash_table_element *parent;
+ //transverse the whole list
+ while(tmp!=NULL){
+ if(tmp->hash_code==key)
+ return 1;
+ parent=tmp;
+ tmp=tmp->next;
+ }
+ ma_hash_table_element *temp_hash = (ma_hash_table_element
*)my_malloc(sizeof(ma_hash_table_element),MYF(MY_WME));
+ temp_hash->hash_code=key;
+ temp_hash->record_offset=offset;
+ temp_hash->next=NULL;
+ parent->next=temp_hash;
+ return 0;
+}
+
diff --git a/storage/maria/ma_state.h b/storage/maria/ma_state.h
index 2903986..d89324f 100644
--- a/storage/maria/ma_state.h
+++ b/storage/maria/ma_state.h
@@ -14,7 +14,7 @@
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
/* Struct to store tables in use by one transaction */
-
+//#include"ma_hash_table.h"
typedef struct st_maria_status_info
{
ha_rows records; /* Rows in table */
@@ -24,6 +24,7 @@ typedef struct st_maria_status_info
my_off_t key_file_length;
my_off_t data_file_length;
ha_checksum checksum;
+ void * hash_table;
uint32 changed:1, /* Set if table was changed */
no_transid:1; /* Set if no transid was set on rows */
} MARIA_STATUS_INFO;
diff --git a/storage/maria/ma_write.c b/storage/maria/ma_write.c
index f57c462..b31f5a7 100644
--- a/storage/maria/ma_write.c
+++ b/storage/maria/ma_write.c
@@ -21,7 +21,7 @@
#include "trnman.h"
#include "ma_key_recover.h"
#include "ma_blockrec.h"
-
+#include "ma_hash_table.h"
/* Functions declared in this file */
static int w_search(MARIA_HA *info, uint32 comp_flag,
@@ -93,6 +93,7 @@ int maria_write(MARIA_HA *info, uchar *record)
my_bool lock_tree= share->lock_key_trees;
my_bool fatal_error;
MARIA_KE
Re: [Maria-developers] [GSOC Students] Allow to contribute
Hi Daniil, Congratulations! Nice to have you onboard! On 05/02/2016 10:09 AM, Sergei Golubchik wrote: Hi, Даниил! On May 02, Даниил Медведев wrote: Hello everyone! I was selected for GSoC to work with MariaDB, with task NO PAD Collatons. Congratulations! Me profile on github is medvdanil, and i'm planning to code in my own branch 10.2-MDEV-9711 or 10.1-MDEV-9711. Which version I should select 10.1 or 10.2? Please go ahead with your own repository, as Vicențiu suggested in an earlier email today. 10.1 is fine, I think. You'll need to rebase to 10.2 eventually, but currently this branch is still in the flux, it'd be easier to base on something stable. I propose to do it in 10.2 from the start. Earlier I made some preparatory changes in 10.2, related to MDEV-9711: removed the unused "my_bool diff_if_only_endspace_difference" parameter from my_strnncollsp_xxx() and all related unused code, to make your life easier :) Also, we're planning no other my_strnncollsp_xxx related changes for 10.2, so there should not be conflicts with other changes. Thanks. And I ask to give me access to create one of those branches and commit to it. Just fork the repository on github and work there. Regards, Sergei Chief Architect MariaDB and secur...@mariadb.org ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] [GSOC Students] Allow to contribute
Okay, thanks! 2016-05-02 9:09 GMT+03:00 Sergei Golubchik : > Hi, Даниил! > > On May 02, Даниил Медведев wrote: > > Hello everyone! > > I was selected for GSoC to work with MariaDB, with task NO PAD Collatons. > > Congratulations! > > > Me profile on github is medvdanil, and i'm planning to code in my own > > branch 10.2-MDEV-9711 or 10.1-MDEV-9711. > > Which version I should select 10.1 or 10.2? > > 10.1 is fine, I think. You'll need to rebase to 10.2 eventually, but > currently this branch is still in the flux, it'd be easier to base on > something stable. > > > And I ask to give me access to create one of those branches and commit > > to it. > > Just fork the repository on github and work there. > > Regards, > Sergei > Chief Architect MariaDB > and secur...@mariadb.org > ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] 95e1c46: MDEV-9898 SET ROLE NONE can crash mysqld.
Hi, Alexey! On May 02, Alexey Botchkov wrote: > Fine. > But can you please explain the logic here? When connected to the server, > the user is authenticated with the > find_user_or_anon(). But when we check permissions for the SET ROLE NONE, > we use find_user_wild(). > Why is that? Because 'user' is thd->security_ctx->priv_user in this case, not thd->security_ctx->user. find_user_or_anon() works with user/host pair, using wildcard matching for both user and host values. While find_user_exact() works with priv_user/priv_host pairs, using exact matching for both user and host values. The very weird find_user_wild() works with inconsistent priv_user/host pairs, using wildcards only for the host, not for the user. This pair make no sense to me, but this is the historical MySQL behavior that I didn't risk to break :( It's MDEV-5233 Regards, Sergei ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] 9058071: MDEV-9362: InnoDB tables using DATA_DIRECTORY created using MySQL 5.6 do not work with MariaDB 10.1
Hi, Jan! On May 02, Jan Lindström wrote: > Hi Sergei, ... > Actual bug on MDEV-9362 can be fixed by not touching these bitmaps at all. ... > So alternatives are: > (1) Proposed intrusive fix (with some modifications based on review) > (2) Move MariaDB flags to end of bitmap > (3) Just fix reported problem on MDEV-9362 How can you fix MDEV-9362 bug without touching the bitmaps? Regards, Sergei ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
