Re: [Maria-developers] Participation in MariaDB Community's Open Source Project

[Vicențiu Ciorbaru]

Hi Yingquan!

I recommend you start reading this article:
https://mariadb.org/get-involved/getting-started-for-developers/

That should guide you in the right direction.

Feel free to reach out on zulip https://mariadb.zulipchat.com/ if you have
any questions.
You can have a look at our beginner friendly tasks on JIRA and try to fix
any one of the open ones.
https://jira.mariadb.org/browse/MDEV-30879?jql=labels%20%3D%20beginner-friendly%20and%20status%20%3D%20Open%20

Vicentiu

On Sun, 9 Apr 2023 at 06:50, 和映泉  wrote:

> Dear MariaDB Development Team,
>
> I am writing to express my interest in participating in your open source
> project. I am an experienced programmer with knowledge of multiple
> programming languages and technologies, and I have some understanding of
> MariaDB products and source code. I believe I can make positive
> contributions to your project and gain more learning and growth
> opportunities from it.
>
> Here is a summary of my programming experience and skills:
>
> - I have 6 years of experience in software development and have
> participated in commercial projects.
> - I am proficient in multiple programming languages such as Java, Python,
> and Scala, and have knowledge of SQL and database technologies.
> - I am familiar with Linux systems and command-line tools, and can use
> version control tools such as Git.
> - I have some understanding and practical experience in software testing,
> code quality, and performance optimization.
>
> In addition, I have also used MySQL products and have some understanding
> of their functions and features. I believe this will help me better
> understand and contribute to your open source project.
>
> Thank you for taking the time to read my email. I look forward to your
> reply and hope to have the opportunity to contribute to the MariaDB
> community.
>
> Best regards,
>
> Yingquan He
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Google Summer of Code 2022

[Vicențiu Ciorbaru]

Hi Sergey!

I'm happy to see that you have experience in MariaDB code base already. The
MDEV-18827 is definitely up for grabs for GSoC 2022. The project is mostly
a part-time project (requiring a commitment of around 175 hours), however
depending on your availability we can increase the scope of the project and
bump it up to a full time project.

My suggestion for this task is to start looking at code parsing FRM files.
Search for the function TABLE_SHARE::init_from_binary_frm_image to get an
idea how an .frm is structured.

After that, think how you would like to implement the feature described in
MDEV-18827. Feel free to submit a draft proposal and we can iterate on it.
Not 100% sure how the GSoC interface works this year, but last time you
were able to share with potential mentors a Google Doc with the possibility
to comment.

Have a look at the proposal guidelines
https://mariadb.org/gsoc2022-instructions/ and start writing it. When the
program application opens you can submit it (4th of April).

In the meantime, feel free to ask any questions you might have here. I
encourage you to keep all discussions on the mailing list as much as
possible so others can chime in.

Vicențiu

On Sat, 19 Mar 2022 at 09:54, Сергей Ваниславский  wrote:

> Hello Vicentiu!
> My name is Vanislavsky Sergey. Your colleague Nikita Malyavin
> recommended contacting you about GSoC.I would like to do your task
> MDEV-18827. I am a 3rd year student at FEFU. Here is my last pull request:
> https://github.com/MariaDB/server/pull/2016
> Thank you for your consideration.
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 6f1628b: MDEV-25565 Crash on 2-nd execution of SP/PS for query calculating window functions

[Vicențiu Ciorbaru]

Hi Igor!

The patch looks good, however I suggest the following changes to make it
smaller and easier to read & maintain.

1. You do not need to have special if statements for save_partition_list
and save_order_list when you do the cleanup.
2. You do not need to have special code that saves the partition list and
the order list *only* when it is overwritten.

You can instead always set save_partition_list to be equal to
partition_list in the Window_spec's constructor. Same for save_order_list
and order_list.

Then, during cleanup, you can always execute
order_list= save_order_list;
partition_list= save_partition_list;

The advantage of this change is that it removes all the references to
save__list in sql_window.cc. You then only have a simple cleanup
function sql_union.cc.

3. I would prefer to name the variables "initial_order_list" and
"initial_partition_list".

The patch then becomes:
diff --git a/sql/sql_union.cc b/sql/sql_union.cc
index e5648e6989b..54a33bb7863 100644
--- a/sql/sql_union.cc
+++ b/sql/sql_union.cc
@@ -30,6 +30,7 @@
 #include "filesort.h"   // filesort_free_buffers
 #include "sql_view.h"
 #include "sql_cte.h"
+#include "item_windowfunc.h"

 bool mysql_union(THD *thd, LEX *lex, select_result *result,
  SELECT_LEX_UNIT *unit, ulong setup_tables_done_option)
@@ -1551,6 +1552,21 @@ static void cleanup_order(ORDER *order)
 }


+static void cleanup_window_funcs(List _funcs)
+{
+  List_iterator_fast it(win_funcs);
+  Item_window_func *win_func;
+  while ((win_func= it++))
+  {
+Window_spec *win_spec= win_func->window_spec;
+if (!win_spec)
+  continue;
+win_spec->partition_list= win_spec->initial_partition_list;
+win_spec->order_list= win_spec->initial_order_list;
+  }
+}
+
+
 bool st_select_lex::cleanup()
 {
   bool error= FALSE;
@@ -1559,6 +1575,8 @@ bool st_select_lex::cleanup()
   cleanup_order(order_list.first);
   cleanup_order(group_list.first);

+  cleanup_window_funcs(window_funcs);
+
   if (join)
   {
 List_iterator ti(leaf_tables);
diff --git a/sql/sql_window.h b/sql/sql_window.h
index e0c1563e5bb..cf84e26c822 100644
--- a/sql/sql_window.h
+++ b/sql/sql_window.h
@@ -99,8 +99,10 @@ class Window_spec : public Sql_alloc
   LEX_STRING *window_ref;

   SQL_I_List *partition_list;
+  SQL_I_List *initial_partition_list;

   SQL_I_List *order_list;
+  SQL_I_List *initial_order_list;

   Window_frame *window_frame;

@@ -111,7 +113,8 @@ class Window_spec : public Sql_alloc
   SQL_I_List *ord_list,
   Window_frame *win_frame)
 : window_names_are_checked(false), window_ref(win_ref),
-  partition_list(part_list), order_list(ord_list),
+  partition_list(part_list), initial_partition_list(part_list),
+  order_list(ord_list), initial_order_list(ord_list),
   window_frame(win_frame), referenced_win_spec(NULL) {}

   virtual char *name() { return NULL; }


On Sat, 10 Jul 2021 at 04:57, IgorBabaev  wrote:

> revision-id: 6f1628b917d365ecfc9c4c9951011613f4212592
> (mariadb-10.2.31-1048-g6f1628b)
> parent(s): fb0b28932ce82903f2fcfb690a71bff52355507f
> author: Igor Babaev
> committer: Igor Babaev
> timestamp: 2021-07-09 18:56:34 -0700
> message:
>
> MDEV-25565 Crash on 2-nd execution of SP/PS for query calculating window
> functions
>from view
>
> A crash of the server happened when executing a stored procedure whose the
> only query calculated window functions over a mergeable view specified
> as a select from non-mergeable view. The crash could be reproduced if
> the window specifications of the window functions were identical and both
> contained PARTITION lists and ORDER BY lists. A crash also happened on
> the second execution of the prepared statement created for such query.
> If to use derived tables or CTE instead of views the problem still
> manifests itself crashing the server.
>
> When optimizing the window specifications of a window function the
> server can substitute the partition lists and the order lists for
> the corresponding lists from another window specification in the case
> when the lists are identical. This substitution is not permanent and should
> be rolled back before the second execution. It was not done and this
> ultimately led to a crash when resolving the column names at the second
> execution of SP/PS.
>
> ---
>  mysql-test/r/win.result | 287
> 
>  mysql-test/t/win.test   | 147 +
>  sql/sql_union.cc|  26 +
>  sql/sql_window.cc   |  12 ++
>  sql/sql_window.h|   5 +-
>  5 files changed, 476 insertions(+), 1 deletion(-)
>
> diff --git a/mysql-test/r/win.result b/mysql-test/r/win.result
> index 8a31dcc..bc017ea 100644
> --- a/mysql-test/r/win.result
> +++ b/mysql-test/r/win.result
> @@ -3911,5 +3911,292 @@ sum(i) over () IN ( SELECT 1 FROM t1 a)
>  0
>  DROP TABLE t1;
>  #
> +# MDEV-25565: 2-nd call of SP with SELECT from view / derived table / CTE
> +#   

[Maria-developers] Initial prototype for MDEV-10825 Persist exact view definitions DDL

[Vicențiu Ciorbaru]

Hi Sergei!

This email is a followup to the brief discussion on Zulip here:

https://mariadb.zulipchat.com/#narrow/stream/118759-general/topic/preserve.20original.20formatting.20for.20views

You mentioned we store the view's definition (source) inside the FRM. I've
used that information to extend the I_S.views table with a source column.
The patch is very small, but I have 2 questions:
1. Is this how the feature should look like? I wonder if we should
prepend *create
view * to the SOURCE column, to make it behave very similar to
SHOW CREATE VIEW. Perhaps SOURCE as a column name is not the most well
chosen name.
2. I don't know if I should use:
  table->field[11]->store(tables->source.str, tables->source.length,
  tables->view_creation_ctx->get_client_cs());
or
  table->field[11]->store(tables->source.str, tables->source.length,
  cs);

when storing the source data.

Here is the patch:
https://github.com/MariaDB/server/compare/10.6-mdev-10825

As soon as we agree on the complete specs for the feature, I'll clean up
test failures in other tests, etc.

Vicențiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] GSOC21: MDEV-16375 & MDEV-23143

[Vicențiu Ciorbaru]

Hi Songlin!

It's great that you are excited about this project! Here are my thoughts on
your proposal and what I think you should focus on:

JSON_NORMALIZE seems simple at first, but I believe there are a lot of
corner cases. In order to get a proper specification for this function can
you have a look at other databases, to see if they implement something
similar? Have a look at the pandas python library, can you learn from their
experience?

Normalizing JSON can have some tricky cases such as:
a. How are arrays sorted if the values inside them are a mix of objects,
arrays, literals, numbers.
b. How do you define a sorting criteria between two JSON objects in this
case?
c. JSON is represented as text, however one can use it to store floating
point values. How do you plan to compare doubles and how would those values
be sorted? For example: 1 vs 1.0 vs 1.00 or 1000 vs 1e3?
d. What's the priority of null values, are they first, last?

The way we should handle this project is via TDD (Test Driven Development).
You would first write your test cases, covering as many corner cases as
possible, then implement the code such that it passes all the tests.

I suggest you add to your proposal some examples of how you define
JSON_NORMALIZE and JSON_EQUALS to behave, so that we can see you have
thought about points a, b, c, d from above.

As for JSON_EQUALS, assuming JSON_NORMALIZE is done correctly, it may work
as a simple strcmp between two normalized JSON objects, but I am not 100%
confident at this point, you would have to prove it :)

Vicențiu

On Tue, 30 Mar 2021 at 09:00, Hollow Man  wrote:

> Hi community!
>
> I've had my proposal shared with
> https://drive.google.com/file/d/1sv0qbqt9W-ob3GqxygWwRGurpRS1lCiv/view ,
> hope to get some feedback from the community.
>
> Songlin
>
> -- Original --
> *From: * "Hollow Man";
> *Date: * Thu, Mar 11, 2021 00:17 AM
> *To: * "maria-developers";
> *Subject: * GSOC21: MDEV-16375 & MDEV-23143
>
> Hi MariaDB community!
>
>Glad to be here! My github account is @HollowMan6. Though I'm new to
> MariaDB community, I'm interested in MDEV-16375 & MDEV-23143: Function to
> normalize a json value & missing a JSON_EQUALS function for this year's
> GSOC project. Here are my first thoughts on these issues:
>
>I have checked part of the codebase and I think the two issues can be
> merged into one. First we can create a function named JSON_NORMALIZE to
> normalize the json, which automatically parses the inputed json document,
> recursively sorts the keys (for objects) / sorts the numbers (for arrays),
> removes the spaces, and then return the json document string.
>
>Then we create a function named JSON_EQUALS, which can be used to
> compare 2 json documents for equality realized by first seperately
> normalize the two json documents using JSON_NORMALIZE, then the 2 can be
> compared exactly as binary strings.
>
>I have taken some inspirations from the Item_func_json_keys and
> json_scan_start for parsing json documents, and I think it's possible to
> sort the keys using std::map in STL for objects.
>
>That's all for my ideas so far. Please correct me if I made some
> mistakes, and I'm going to work on my ideas later.
>
> Cheers!
>
> Hollow Man
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] RFC: incompatible changes to mtr debugging flags

[Vicențiu Ciorbaru]

Hi Sergei!

Just tested this out. I like the changes and can live with what you
proposed. However, I don't see a reason to get rid of "--manual-debug". I
like being able to see what command line parameters are needed to start the
server with the exact same setup. Sometimes I use it to start the server
manually, with/without the debugger.

Vicențiu



On Wed, 27 Jan 2021 at 14:40, Vladislav Vaintroub 
wrote:

>
>
> Hi Serg,
>
>
>
> vc_express   is historical, does not exist eanymore. I’m not aware vc ever
> existed
>
>
>
> I’d  propose to only have a few options ( –debugger, --server-debug,
> --boot-debug, --client-debug, --manual-debug) , rather than 4-5 options for
> each debugger,  as per Occam’s razor.
>
>
>
> A reasonable choice of default debugger can be easily made per platform
> (e.g vsjitdebugger preferred to windbg on Windows , and gdb on anything
> else preferred to llvm debugger). Then  –debugger can be omitted in most
> cases.
>
>
>
> *From: *Sergei Golubchik 
> *Sent: *Tuesday, 26 January 2021 21:32
> *To: *maria-developers@lists.launchpad.net
> *Subject: *[Maria-developers] RFC: incompatible changes to mtr debugging
> flags
>
>
>
> Hi, all,
>
>
>
> Here's a feature I want to discuss, as it's rather incompatibly removes
>
> a bunch of mtr command-line options.
>
>
>
> It's a unified handling of all debuggers.
>
>
>
> I wanted to do it for quite a while, but the actual trigger was me
>
> trying to add support for --rr (https://rr-project.org/) in embedded.
>
> It was supported only for non-embedded server, and making it work for
>
> embedded needed way more copy-pasting than I was comfortable with.
>
>
>
> So, in my branch all debuggers are not haphazardly added using whatever
>
> syntax one thought of at the moment and copy-pasted all over.
>
> They've handled by a common module, all support the same set of options
>
> (using gdb as an example): --gdb, --boot-gdb, --client-gdb,
>
> --manual-gdb. Both --gdb and --client-gdb work for embedded.
>
> All four of them accept an optional argument that can be a semicolon
>
> separated list of gdb commands, like in
>
>
>
>   ./mtr 1st --gdb='b mysql_parse;r'
>
>
>
> but can start from command-line options too:
>
>
>
>   ./mtr 1st --boot-gdb='-nh -q;b mysql_parse;r'
>
>
>
> A "debugger" is anything that wraps mysqld or mysqltest execution,
>
> the current list of debuggers is: gdb, ddd, dbx, lldb, valgrind, strace,
>
> ktrace, rr, devenv, windbg, vc_express, vc, vsjitdebugger. There's also
>
> "valgdb" that does what '--valgrind --gdb' was doing before.
>
>
>
> This removes the following mtr options: --rr-args --rr-dir
>
> --manual-debug --debugger --strace-option --stracer --valgrind-all
>
> --valgrind-mysqltest --valgrind-mysqld --valgrind-options
>
> --valgrind-option --valgrind-path --callgrind, and --valgrind only
>
> enables it for the server, not for everything.
>
>
>
> If pushed, it'll definitely break backward compatibility. But I don't
>
> think mtr backward compatibility is all that important, it might only
>
> affect a handful of developers who use scripts to start mtr with specific
>
> hard-coded settings, those scripts could be easily adjusted.
>
> It's much more important to have same mtr features in all branches, so
>
> that when switching between branches, one wouldn't need to guess what
>
> command line options mtr supports now. That is, if pushed, this should
>
> go into 10.2.
>
>
>
> Opinions? Missing features? Push / not push?
>
> for the reference, it's in bb-10.2-serg branch at the moment.
>
>
>
> Regards,
>
> Sergei
>
>
>
> ___
>
> Mailing list: https://launchpad.net/~maria-developers
>
> Post to : maria-developers@lists.launchpad.net
>
> Unsubscribe : https://launchpad.net/~maria-developers
>
> More help   : https://help.launchpad.net/ListHelp
>
>
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Review of MDEV-4851

[Vicențiu Ciorbaru]

Hi Daniel!

The patch looks good, some comments about the test cases.
About the commit message, we have --write-binlog not just --binlog.
Also, check for typos in the last commit message and also, I would squash
the
two test cases commits into just one.

> diff --git a/mysql-test/suite/rpl/t/rpl_mysql_upgrade.test
b/mysql-test/suite/rpl/t/rpl_mysql_upgrade.test
> index 1931e2eab2a..b836e889429 100644
> --- a/mysql-test/suite/rpl/t/rpl_mysql_upgrade.test
> +++ b/mysql-test/suite/rpl/t/rpl_mysql_upgrade.test
> @@ -14,8 +14,8 @@ call mtr.add_suppression("table or database name
'mysqltest-1'");
>
>  connection master;
>  --disable_warnings
> -DROP DATABASE IF EXISTS `#mysql50#mysqltest-1`;
> -CREATE DATABASE `#mysql50#mysqltest-1`;
> +DROP DATABASE IF EXISTS `mysqltest-1`;
> +CREATE DATABASE `mysqltest-1`;
Since you are doing cleanup here anyway, let's get rid of the DROP
statements
a test case should not leave any dirty context behind and if we drop things
just to be sure, we are masking potential problems.
>  --enable_warnings
>  sync_slave_with_master;
>
> @@ -34,13 +34,20 @@ if ($before_position == $after_position)
>echo Master position is not changed;
>  }
>
> -#Some log events of the mysql_upgrade's will cause errors on slave.
> +# Some log events of the mysql_upgrade previously caused errors on slave,
> +# however with MDEV-4851 this should be ok, so we test it:
>  connection slave;
> -STOP SLAVE SQL_THREAD;
> -source include/wait_for_slave_sql_to_stop.inc;
> +SET @old_general_log_state = @@global.general_log;
> +SET @old_slow_log_state = @@global.slow_query_log;
> +SET @old_log_output = @@global.log_output;
> +SET GLOBAL general_log = 'ON';
> +SET GLOBAL slow_query_log = 'ON';
> +SET GLOBAL log_output = 'FILE';
>
>  connection master;
>  #With '--force' option, mysql_upgrade always executes all sql statements
for upgrading.
> +ALTER TABLE mysql.slow_log DROP COLUMN thread_id, DROP COLUMN
rows_affected;
> +DROP DATABASE IF EXISTS `mysqltest-1`;
Why IF EXISTS? It *should* be there already.
Can you add some syncs between master and slave and then show from the slave
connection that the log table table is missing those columns and also that
they get applied *after* mysql_upgrade is run?
>  --exec $MYSQL_UPGRADE --skip-verbose --write-binlog --force --user=root
> $MYSQLTEST_VARDIR/log/mysql_upgrade.log 2>&1
>
>  let $datadir= `select @@datadir`;
> @@ -55,8 +62,14 @@ if ($before_position != $after_position)
>echo Master position has been changed;
>  }
>
> -DROP DATABASE `mysqltest-1`;
Notice that the original code didn't use "IF EXISTS".
> +sync_slave_with_master;
>  connection slave;
> -DROP DATABASE `#mysql50#mysqltest-1`;
> ---let $rpl_only_running_threads= 1
> +SET GLOBAL general_log = 'OFF';
> +SET GLOBAL slow_query_log = 'OFF';
> +truncate mysql.slow_log;
> +truncate mysql.general_log;
Why do you need to clear the tables? I tested without clearing and it still
needs to pass.
> +SET GLOBAL general_log = @old_general_log_state;
> +SET GLOBAL slow_query_log = @old_slow_log_state;
> +SET GLOBAL log_output = @old_log_output;
> +
>  --source include/rpl_end.inc
> diff --git a/mysql-test/t/log_tables.test b/mysql-test/t/log_tables.test
> index f822ec8d758..0196e62f12f 100644
> --- a/mysql-test/t/log_tables.test
> +++ b/mysql-test/t/log_tables.test
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Welcome to Google Summer of Code 2020 with MariaDB

[Vicențiu Ciorbaru]

Hi!

GSoC 2020 accepted proposals have just been announced and you are part of
the successful applicants.

This year we had a lot of very good proposals and so we have a one of the
highest number of students (7) to mentor for GSoC. Congratulations on the
effort to learn about the program, get involved and submit successful
proposals!

The fun part begins now!

If you have not done so already, please reach out to your mentor and seek
further advice about your project. Each one of your projects can be
implemented in multiple ways, some more complex than others. I strongly
advise you to create a concrete plan based off of your proposal with your
mentor and aim to stick to it. The MariaDB Server codebase is a complex one
with lots of different parts, however your mentor can guide you to ensure
you have a successful GSoC.

Remember that you are here to learn about Open Source, what it means to be
part of an Open Source community, to interact with the community, all while
becoming better programmers. Please take advantage of this opportunity and
keep your technical discussions public, on Zulip
https://mariadb.zulipchat.com, and on our developer mailing list
https://launchpad.net/~maria-developers. By keeping your discussions
public, you get the benefit of not only having your mentor see your
questions but all of the MariaDB community. You never know when someone
might lend a hand in case your mentor is not yet online. :)

The mantra of our community should be that we are always welcoming to new
people. Do not be afraid to ask questions. There are no bad or stupid or
silly questions. The only key requirement is that you first make an effort
to find the answer yourself and only afterwards seek outside help. If you
can show a bit of involvement, the community will show you the same.

I recommend that you agree with your mentor on some form of weekly progress
reporting, via either a blog-post, an email, or something similar that can
be publicly shared. I can not stress this enough: we are working on an open
source project, our work must be done in the open and it is important to *show
the world the good we are doing*.

Hope all of you will have a great GSoC!

Vicențiu
MariaDB Foundation GSoC Admin
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] MDEV-14296: Moving wsrep_sst_common to /usr/libexec

[Vicențiu Ciorbaru]

Hi Otto!

I am doing a push towards cleaning up as much as possible from our
packaging issues. Starting with the older ones:

*Background:*
wsrep_sst_common.sh is a script that should never be run standalone, it is
rather a helper script used for other wsrep_* scripts. As such, it *does
not really belong in /usr/bin*. See MDEV-14296 [1] for more details.


It looks like rpm packagers - read Fedora - [1a] require us to ship
wsrep_sst_common in */usr/libexec* (or alternatively /usr/share, but
/usr/libexec makes much more sense semantically to me, after reading the
FHS [2]).

There is PR #603 [3] from Daniel Black, which has some missing changes,
which I've done myself, namely updating the debian/*.install files to put
the file in the correct location and a change in path for our apparmor
profile [4] [5].

*My question for you is*: Is this acceptable to do in 10.1 (merge PR 603 +
commits [4] [5] into 10.1) from a packaging point of view?

The debian packaging policy here [6], allows one to follow FSH version 3.0
and does not mention any exception for libexec.


I have tested this and I have not encountered any regressions while
upgrading and the script seems to function as intended (although I am not
an expert DBA to make use of wsrep_* scripts).

Thanks,
Vicențiu

[1] https://jira.mariadb.org/browse/MDEV-14296
[1a]
https://src.fedoraproject.org/rpms/mariadb/blob/ecb40d449c382ea7e4052c75d5d798734c9f7b68/f/mariadb.spec#_1303
[2] https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html
[3] https://github.com/MariaDB/server/pull/603
[4]
https://github.com/MariaDB/server/commit/8042265659656c312f39304a5dc60de70876d0a1
[5]
https://github.com/MariaDB/server/commit/dbed51d86e4aae9b8a54d6b5aa31c974fe5b3479
[6] https://www.debian.org/doc/debian-policy/ch-opersys.html
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [DISCUSS] Don't see ARM release information in MariaDB download page

[Vicențiu Ciorbaru]

Hi!

Yes, this is in the works, I will come with an update during this week!

Vicențiu

On Tue, 14 Jan 2020, 03:51 bo zhaobo,  wrote:

> Hi Team and  Sergei,
>
> I opened a issue about this  https://jira.mariadb.org/browse/MDEV-21432 ,
> and the status of it is confirmed for several days, is there any plan to
> fix it? Team?
>
> Thanks.
>
> BR
>
> ZhaoBo
>
>
> [image: Mailtrack]
> 
>  Sender
> notified by
> Mailtrack
> 
>  20/01/14
> 上午09:49:04
>
> bo zhaobo  于2020年1月7日周二 下午11:06写道：
>
>> Thanks very much, Sergei.
>>
>> That sounds pretty good. ;-). Users will know  the more information  once
>> the download pages [1][2] updated.
>>
>> And so much thanks that you help to figure out another problem in [2],
>> when user chooses centos aarch64 package. And I check it, the url is
>> accessible after replace 'amd64' with 'aarch64'. ;-)
>>
>> So both of them will be updated. Right?
>> If so, That's pretty great. Thanks
>>
>> Best Regards,
>>
>> ZhaoBo
>>
>> [1] https://downloads.mariadb.org/mariadb/10.5.0/
>> [2]https://downloads.mariadb.org/mariadb/repositories/
>>
>>
>>
>>
>> [image: Mailtrack]
>> 
>>  Sender
>> notified by
>> Mailtrack
>> 
>>  20/01/07
>> 下午11:02:21
>>
>> Sergei Golubchik  于2020年1月7日周二 下午10:49写道：
>>
>>> Hi, bo zhaobo!
>>>
>>> Yes. I've passed this information further and the download pages will be
>>> updated.
>>>
>>> Meanwhile, if you want CentOS 7 aarch64 packages, use the repository
>>> configurator for, say, amd64 and replace "amd64" with "aarch64" in the
>>> resulting snippet.
>>>
>>> https://downloads.mariadb.org/mariadb/repositories/
>>>
>>> On Jan 07, bo zhaobo wrote:
>>> > Hello, does anyone from team could give some help / advice about this?
>>> > Thanks
>>> >
>>> > > I want to make clear a thing about the Mariadb download page. Hope
>>> > > team could give some advices or help. As I had sent out an
>>> > > discussion to maria-docs, but didn't get some help. So I think I may
>>> > > get something in dev channel.
>>>
>>> Regards,
>>> Sergei
>>> VP of MariaDB Server Engineering
>>> and secur...@mariadb.org
>>>
>> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] MariaDB for CentOS 8

[Vicențiu Ciorbaru]

Should be fixed now!

Thanks for letting us know about the problem.

On 04.10.2019 00:31, Daniel Bartholomew wrote:

On Thu, 3 Oct 2019 22:00:16 +0530
Thomas Stephen Lee  wrote:


Hi,

Please release MariaDB 10.4 for CentOS 8.

https://downloads.mariadb.org/mariadb/repositories/

does not show CentOS 8.


Please file a ticket at https://jira.mariadb.org and it will get taken
care of right away. In the mean time, you should be able to use the
RHEL 8 instructions to get the repository set up.

Thanks!



___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] MariaDB for CentOS 8

[Vicențiu Ciorbaru]

Hi Thomas,

That is a bug in our code which we shall fix as soon as possible. In the 
meantime, you can replace the instructions on 
https://downloads.mariadb.org/mariadb/repositories/ with the correct URL 
from yum.mariadb.org.



# MariaDB 10.4 CentOS repository list - created 2019-10-04 06:20 UTC
# http://downloads.mariadb.org/mariadb/repositories/
[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.4/centos8-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1

Let me know if this works.

Thanks!
Vicențiu


On 03.10.2019 19:30, Thomas Stephen Lee wrote:

Hi,

Please release MariaDB 10.4 for CentOS 8.

https://downloads.mariadb.org/mariadb/repositories/

does not show CentOS 8.

thanks

--
Thomas Stephen Lee

___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Congratulations on a successful GSoC!

[Vicențiu Ciorbaru]

Hi!

By now, all of you have probably received the results of this year's Google
Summer of Code. I want to congratulate you all for the work that you have
done this year, students and mentors alike.

We have had 4 projects this year:
1. EXCEPT ALL  and INTERSECT ALL operations - Ruihang Xia
https://github.com/MariaDB/server/pull/1378
2. INSERT ... RETURNING - Rucha Deodar
https://medium.com/@ruchad1998/google-summer-of-code-2019-add-returning-to-insert-statement-b876a63fa7e5
3. UPDATE ... RETURNING - Miroslav Koberskii
https://gist.github.com/Mup0c/43c781e2135e55bf5126e0f8c60e2a24
4. Support for indexes on expressions - Alexey Mogilyovkin
https://gist.github.com/ZeroICQ/f0c35a20cae7065368dae31d7ad9b001

Implementing these projects certainly took a lot of effort. The first one
is already merged within MariaDB and the rest will probably follow, after a
bit more polishing. The MariaDB community would be grateful if you stuck
around to watch over your projects, extend them and improve them over time.

I am sure there was a lot to learn from this experience. Now that you have
gotten a feel to what it means to contribute significant features to
MariaDB you are in a great position to help others! In this sense, I
encourage you to keep being active in the community and share what you have
learned. :)

Thank you for a great summer!
Vicențiu Ciorbaru
GSoC MariaDB Foundation Admin
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Coding Style Review INSERT RETURNING

[Vicențiu Ciorbaru]

Hi Rucha!

This is going to be a very thorough review looking at coding style. I'll
nitpick on every line of code where I find something, just be patient and
let's clean up everything.

You might consider me picking on the coding style a bit pedantic, but it
really is important. It is important because it helps in searching, it helps
in understanding the code faster, it helps to look-up history faster.
The good part is that once you get used to writing things properly, you won't
even have to think about it actively. It will just come naturally.

I suspect you do not have your editor configured to show whitespaces and
show you tabs versus spaces. Please look into your editor's settings to
highlight both tabs and spaces.

There are scripts that remove trailing whitespaces for you, across whole files.
I don't recommend them for MariaDB code as we have remnants from MySQL times of
these and because we care more about preserving the immediate history for
running `git blame`, we did not fully fix all files. For any other project, I
highly recommend you use them.

Some of these comments might slightly contradict what I said before. That is
because I wanted us to get things working first and I didn't want to burden
you with too many details initially. I'll look to explain myself now clearly
so you understand why some things are bad ideas. I want you to take this
review also as a general guideline for any other code you write, anywhere.
With these hints, tricks, ideas, you'll produce much higher quality code in
the long run. It will take you longer to write sometimes, but it really makes
a difference. :)

Also, you may have noticed some parts of our code do not agree with my general
guidelines. You have to take into account that this code was written over the
course of many years, by different people, with different levels of experience
or available time to implement something. Try not to make use of it as an
excuse for new code that you write. It's up to you to always write the best
possible version *with the time allowed*. We have adequate time now so we
can afford to look for better solutions.

With all that said, this took me a while to write and I may still
have missed a few things. Feel free to point out anything that looks strange.
Now, let's get to the review.

> 
> diff --git a/sql/sql_class.h b/sql/sql_class.h
> index 18ccf992d1e..9e4bb1d4579 100644
> --- a/sql/sql_class.h
> +++ b/sql/sql_class.h
> @@ -5475,7 +5476,13 @@ class select_dump :public select_to_file {
>  
>  class select_insert :public select_result_interceptor {
>   public:
> +  select_result* sel_result;
The * should be next to the variable name. Not everybody does this, but the
code around it does, so let's be consistent.
>TABLE_LIST *table_list;
> +  /*
> +Points to the table in which we're inserting records. We need to save the
> +insert table before it is masked in the parsing phase.
> +  */
> +  TABLE_LIST *insert_table; 
This sort of code and comment is usually a bad idea:
1. We are adding an extra member, because of a hack in a different part of
   the code.
2. We are mentioning a specific flow for which this member is good for. This
   already begins to hint at a bad design decision. Sometimes there's no way
   around it, but this guarantees lack of re-usability. If, in the future
   we fix the "masking in the parsing phase" to no longer mask the first table,
   which we probably will do, because we have to to allow INSERT within other
   queries, the comment will become outdated. One needs to remember this too.

   Forgetting to update comments happens easily. Here is one example:
   One checks the class initially, finds the appropriate member to use,
   uses it in a different context than the original author thought of. The
   class then is no longer changed so it does not show up during review time,
   the reviewer doesn't think to check the class member's definition either.
   You now have outdated comments.

   To prevent this from happening, avoid as much as possible to explain where
   a member is used, rather explain what type of data it is supposed to store.
   The first sentence accomplishes that goal.
>TABLE *table;
>List *fields;
>ulonglong autoinc_value_of_last_inserted_row; // autogenerated or not
> @@ -5484,7 +5491,7 @@ class select_insert :public select_result_interceptor {
>select_insert(THD *thd_arg, TABLE_LIST *table_list_par,
>   TABLE *table_par, List *fields_par,
>   List *update_fields, List *update_values,
> - enum_duplicates duplic, bool ignore);
> + enum_duplicates duplic, bool ignore,select_result* 
> sel_ret_list,TABLE_LIST *save_first);
This is an artifact of our long-standing code base. TABS mixed with spaces. :(
Let's clean up all of this select_insert's constructor. Do not keep the tabs,
we are modifying it anyway. Turn all these tabs to proper spaces. Make sure
the function arguments match the beginning parenthesis like so:
  

[Maria-developers] INSERT ... RETURNING intermediate review

[Vicențiu Ciorbaru]

Hi Rucha!
I've spent quite a bit of time looking at the implementation. It is working and
passes almost all test cases, although you forgot to also commit the
insert_parser updated result file. Good job so far.
I want to give you something to work on and think about for a bit before we do
another overall overview of the whole project. Hopefully this will help you
across all your coding endeavours, not just now. What I want you to learn is
that it is important to think about how your code can and will evolve in the
future. Just getting things to work is ok up to a certain point, but not
implementing things in a decoupled fashion is what we call "hacks". These hacks
are to be avoided as much as possible as it leads to much more difficult work
down the line. Let's elaborate on the topic.
There are a number of problems with our current code (hacks). I am not referring
to yours in particular. All these hacks are a bit difficult to overcome in one
go. I do not expect you to fix them yourself, not during this GSoC at least, but
it is something to consider and if you are willing to work towards fixing some
of them, excellent!
Historically our codebase evolved from a simple parser to the "monster" that you
see now. Sometimes due to time constraints, things were not implemented in the
most extendable way, rather they made use of certain particular constructs, only
valid in the contexts they were written in. Keyword here is context, as you'll
soon see. Some of this contain things you have interacted with so far in your
project. Here is an example that you've had to deal with:
A SELECT statement has what we call a SELECT LIST, this list is traditionally
stored in SELECT_LEX::item_list. This item_list however is overloaded at times.
For example DELETE ... RETURNING uses it to store the expressions part of the
RETURNING clause. This overloading generally works ok, unless you run into
situations like you did with your INSERT ... RETURNING project.
INSERT ... RETURNING clauses already made use of the item_list to store
something else. Now you were forced to introduce another list in SELECT_LEX,
which we called returning_list. Everything ok so far. But then, you ran into
another problem, the bison grammar rules do not put Items in the list you want.
They always used item_list, so you could not use your returning_list. So what
did we do then? Well, we came up with another hack! We suggested this hack to
you because it is something that will work quickly and get you unstuck and that
is to use the same grammar rules like before, but swap the item_list with the
returning list temporarily, such that the grammar rules will put the right items
in the right list. This works, but it masks the underlying problem. The problem
is that we have a very rigid implementation for select_item_list grammar rule. 
What we should do is to make select_item_list grammar return a list using the
bison stack. That means no relying on SELECT_LEX::item_list to store our result
temporarily. You have done steps towards fixing this, which brings us to where
your code's current state. The problem with your implementation is you have not
lifted the restriction of the grammar rules making use of
SELECT_LIST::item_list. Instead, you have masked it by returning the address of
that member on the bison stack. The funny part is that this works, but it still
a hack. It is still a hack, because these grammar rules have a side effect of
modifying this member behind the scenes. A very, very, very (!) good rule is to
consider all side effects as bad, especially now that you are starting out. With
experience you might get away with them every now-and-then, but for now avoid
them like the plague and try to remove them whenever possible.
The current code makes use of a function I really don't like. The inline
function add_item_list(). It is one of those hacky functions that strictly
relies on context. The context is: take the thd->lex->current_select and put the
item passed as a parameter in the item_list. The diff I've attached shows an
implementation of how to use the bison stack properly. I took inspiration from
the "expr" rule. Unfortunately the hack chain goes deeper, but we need to do
baby steps and some things are best left outside your GSoC project, or we risk
going down a rabbit whole we might not get out of.
One other issue I've observed is with the REPLACE code here:
  insert_field_spec   {if
($6){  List
list;  list=*($6);  Lex->current_select-
>item_list.empty();  $6=}  }  opt_
select_expressions  {if ($8)  Lex-
>returning_list=*($8);if ($6)  Lex->current_select-
>item_list=*($6);Lex->pop_select(); //main selectif
(Lex->check_main_unit_semantics())  MYSQL_YYABORT;  }
This is really problematic and your probably wrote it because of

[Maria-developers] INSERT ... RETURNING Initial code review

[Vicențiu Ciorbaru]

Hi Rucha!
Here is the promised review. I have run the main test suite locally and
have noticedthat some tests fail if you run with a debug build. The
failure is an assertionfailure on the protocol level. This means you
forgot to call a my_ok or my_errorat an appropriate time. I suggest you
try and find which one of your commitsintroduces this bug. A handy tool
is `git bisect`. It's a great time to learnabout it, if you have not
used it before.
There are quite a few other things to fix, but first, let's tackle
theinitial review comments.
Now, on to the actual code.
Your text editor inserts tabs instead of spaces. Please configure it to
onlyinsert tabs and have the "tab" size be equal to 2 spaces.

> diff --git a/mysql-test/main/insert_parser.test b/mysql-
test/main/insert_parser.test> new file mode 100644> index
000..5c1f9408a96> --- /dev/null> +++ b/mysql-
test/main/insert_parser.test> @@ -0,0 +1,64 @@> +#> +# Syntax check for
INSERT...RETURNING> +#> +> +--echo # RETURNING * and RETURNING
 should work. Trailing whitespace here, please fix.> +
--echo # Other cases with RETURNING should output syntax error.> +> +
--disable_warnings> +drop table if exists t1,t2;> +
--enable_warningsTest cases in MariaDB are written such that they do
not leave behind anytables or other elements from running the test.
Thus, such statements arenot required (and actively discouraged) in new
test cases. If tables fromprevious tests are present and mysql-test-run 
(mtr) does not mark the test asfailed then inserting such statements
will hide the mysql-test-run bug.Please delete.> +> +CREATE TABLE
t1(id1 INT, val1 VARCHAR(1));> +--echo Table t1 created successfully.
Fields: id1 INT, val1 VARCHAR(1);This comment is not necessary, please
delete. The fact that mtr passed meansthe command was succesful. Also,
when adding comments, I recommend you prefixthem with #, like you did
for the start of the test.> +> +CREATE TABLE t2(id2 INT, val2
VARCHAR(1));> +--echo Table t2 created successfully. Fields: id2 INT,
val2 VARCHAR(1);Same as previous comment, not necessary, please
delete.> +> +#> +#Simple insert statement> +#> +--echo #Simple insert
statementOnly keep one, I suggest the --echo variant, but put a space
after #.I also like to add 2 extra --echo # like so:--echo #--echo #
Simple insert statement--echo #
This makes the result file easier to read.> +INSERT INTO t1 (id1, val1)
VALUES (1, 'a');> +INSERT INTO t1 (id1, val1) VALUES (2, 'b') RETURNING
*;> +INSERT INTO t1 (id1, val1) VALUES (3, 'c') RETURNING id1, val1;>
+INSERT INTO t1 (id1, val1) VALUES (3, 'c') RETURNING id1 AS id;>
+SELECT * FROM t1;> +TRUNCATE TABLE t1;> +SELECT * FROM t1;> +> +#>
+#multiple values in one insert statement> +#> +--echo #multiple values
in one insert statementSame as above.> +INSERT INTO t1 VALUES
(1,'a'),(2,'b');> +INSERT INTO t1 VALUES (3,'c'),(4,'d') RETURNING *;>
+INSERT INTO t1 VALUES (5,'e'),(6,'f') RETURNING id1, val1;> +INSERT
INTO t1 VALUES (5,'e'),(6,'f') RETURNING id1 AS id;> +> +#>
+#INSERT...ON DULPICATE KEY UPDATE> +#> +--echo # INSERT...ON DULPICATE
KEY UPDATESame as above. Typo too.> +CREATE TABLE ins_duplicate (id INT
PRIMARY KEY, val VARCHAR(1));> +INSERT INTO ins_duplicate VALUES
(1,'a');> +INSERT INTO ins_duplicate VALUES (2,'b') ON DUPLICATE KEY
UPDATE val='b' RETURNING id,val;> +INSERT INTO ins_duplicate VALUES
(3,'b') ON DUPLICATE KEY UPDATE val='c' RETURNING *;> +INSERT INTO
ins_duplicate VALUES (4,'b') ON DUPLICATE KEY UPDATE val='d' RETURNING
id AS id1;> +#> +# INSERT...SET> +#> +> +--echo # INSERT...SETSame as
above.> +INSERT INTO  t1 SET id1 = 1, val1 = 'a'; > +INSERT INTO  t1
SET id1 = 2, val1 = 'b' RETURNING *;> +INSERT INTO  t1 SET id1 = 3,
val1 = 'c' RETURNING id1,val1;> +INSERT INTO  t1 SET id1 = 3, val1 =
'c' RETURNING id1 AS id;> +> +DROP TABLE t1; > +DROP TABLE t2;> +DROP
TABLE ins_duplicate;> +> +#> +--echo #End of test case > +#This is
superfluous, please delete everything after the last DROP TABLE.> \ No
newline at end of file> diff --git a/mysql-
test/main/insert_parser_1.test b/mysql-test/main/insert_parser_1.test>
new file mode 100644> index 000..f83c288a800> --- /dev/null>
+++ b/mysql-test/main/insert_parser_1.test> @@ -0,0 +1,75 @@Hmm, this
file does not have a corresponding .result file. It's almost a copyof
the previous one and it has windows line endings. It's the only one
thathas those. I suggest you move the statements not present in
insert_parser.testthere and delete this file entirely.> +#> +# Syntax
check for INSERT...RETURNING> +#> +> +--disable_warnings> +drop table
if exists t1,t2;> +--enable_warnings> +> +CREATE TABLE t1(id1 INT, val1
VARCHAR(1));> +--echo Table t1 created successfully. Fields: id1 INT,
val1 VARCHAR(1);> +> +CREATE TABLE t2(id2 INT, val2 VARCHAR(1));> +
--echo Table t2 created successfully. Fields: id2 INT, val2
VARCHAR(1);> +> +#> +#Simple insert statement> +#> +--echo #Simple
insert statement> +INSERT INTO t1 (id1, val1) VALUES (1, 'a');> +INSERT
INTO t1 (id1, 

Re: [Maria-developers] GSoC Introduction

[Vicențiu Ciorbaru]

Hi Rucha!

Great start. Whenever you have some code ready for the parser part, please
share the repository and the branch onto which you are pushing your code.

We should discuss implementation details once you have managed to get the
grammar to work. Remember to include test cases to show that the grammar
works.

For details as to how you should write test cases, check out

https://mariadb.org/get-involved/getting-started-for-developers/writing-good-test-cases-mariadb-server/

Vicențiu

On Mon, 27 May 2019, 06:55 Rucha Deodhar,  wrote:

> Hello,
>
> I am Rucha Deodhar. I am a GSoC student and will implement
> INSERT...RETURNING.
> In the first week I plan to extend the parser. Approximately the next 2
> months, I plan to make changes the in the codes in insert.cc and other
> files related to the project, and remaining time for testing. As suggested,
> I am allotting 40% of the time for testing.This is my approximate plan,
>
> Regards,
> Rucha.
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Welcome to GSoC 2019

[Vicențiu Ciorbaru]

Hi!

As you may have noticed a few days ago, you have received an email from
Google that you have been accepted as part of Google Summer of Code.
Congratulations on the effort to learn about the program, get involved
and submit succesful proposals.

The fun part begins now!

If you have not done so already, please reach out to your mentor and
seek further advice about your project. Each one of your projects can
be implemented in multiple ways, some more complex than others. I
strongly advise you to create a concrete plan based off of your
proposal with your mentor and aim to stick to it. Your mentor can guide
you to ensure you have a succesful GSoC.

Remember that you are here to learn about Open Source, what it means to
be part of an Open Source community, to interact with the community,
all while becoming better programmers. Please take advantage of this
opportunity and keep your technical discussions public, on Zulip 
https://mariadb.zulipchat.com, and on our developer mailing list 
https://launchpad.net/~maria-developers. By keeping your discussions
public, you get the benefit of not only having your mentor see your
questions but all of the MariaDB community. You never know when someone
might lend a hand in case your mentor is not yet online. :)

The mantra of our community should be that we are always welcoming to
new people. Do not be afraid to ask questions. There are no bad or
stupid or silly questions. The only key requirement is that you first
make an effort to find the answer yourself and only afterwards seek
outside help. If you can show a bit of involvement, the community will
show you the same.

I recommend that you agree with your mentor on some form of weekly
progress reporting, via either a blog-post, an email, or something
similar that can be publicly shared.

Hope all of you will have a great GSoC!
Vicențiu

MariaDB Foundation GSoC 2019 Admin



___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Review for ANALYZE TABLE with Sampling

[Vicențiu Ciorbaru]

Hi Sergei!

Can you review that you are happy with the storage engine API changes?
I'veustructured the commit to be as small as possible to achieve the
desired outcome. In my tests, we are now twice as fast as MySQL for a
10 mil row table with 13 columns.

Vicențiu

 Forwarded Message From: vicentiu@mariadb.orgTo: 
commits@mariadb.orgSubject: 53730224efd: Improve histogram collection
performance by samplingDate: Sun, 10 Feb 2019 20:09:49 +0200 (EET)
revision-id: 53730224efd987f97a6cc968ff5214ee499d84e0 (mariadb-10.4.1-
163-g53730224efd)parent(s):
3c305d3f1951f1667f84e48ddd98674c6318c39dauthor: Vicențiu
Ciorbarucommitter: Vicențiu Ciorbarutimestamp: 2019-02-10 19:54:50
+0200message:
Improve histogram collection performance by sampling
Histogram collection is done by sampling a percentage of rows from the
table,not looking at all individual ones.
The default implementation, to keep the server's Engine
IndepenentStatistics component working uses Bernoulli sampling. It does
a simpletable scan, but only accepts rows that pass a dice roll.
Thisimplementation is done as a storage engine interface method, so as
toallow faster and/or more efficient implementations for storage
enginesinternally.
The number of rows collected is capped to a minimum of 5
andincreases logarithmically with a coffecient of 4096. The coffecient
ischosen so that we expect an error of less than 3% in our
estimationsaccording to the paper:"Random Sampling for Histogram
Construction: How much is enough?”– Surajit Chaudhuri, Rajeev Motwani,
Vivek Narasayya, ACM SIGMOD, 1998.
This interface is also a precursor to allowing SELECT ... FROM
tablewith sampling to work.
Performance wise, for a table of 10 million rows and 13 columns, 6
int,6 doubles, one string, the previous analyze table statistics took1
minute 20 seconds to collect all data. Post implementation, thetime is
less than 6 seconds. This was run on an InnoDB table, NVME SSD
withapproximately 2GB/s read speed and 500MB/s write speed.
--- mysql-test/main/selectivity.result|  8 +++ mysql-
test/main/selectivity_innodb.result |  8 +++--
-- sql/handler.cc| 14
+++ sql/handler.h | 40
++- sql/sql_statistics.cc  
   | 32 +++-- 5 files changed, 86 insertions(+), 16
deletions(-)
diff --git a/mysql-test/main/selectivity.result b/mysql-
test/main/selectivity.resultindex 00907235ecc..6d09c1c9b62 100644---
a/mysql-test/main/selectivity.result+++ b/mysql-
test/main/selectivity.result@@ -1290,9 +1290,9 @@ explain
extended select * from t1, t2, t1 as t3 where t1.b=t2.c and t2.d=t3.a
and t3.b<5 and t1.a < 2000; id  select_type table   typepossibl
e_keys  key key_len ref rowsfilteredExtra-1 SIMPLE  
t1  ALL NULLNULLNULLNULL262144  100.00  Using
where+1 SIMPLE  t1  ALL NULLNULLNULLNULL262117  
100.00  Using where 1   SIMPLE  t2  ref c,d c   5   
test.t1.b   5   100.00  -1  SIMPLE  t3  ALL NULL
NULLNULLNULL262144  100.00  Using where; Using join buffer
(flat, BNL join)+1  SIMPLE  t3  ALL NULLNULLNULL
NULL262117  100.00  Using where; Using join buffer (flat, BNL
join) Warnings: Note1003select `test`.`t1`.`a` AS
`a`,`test`.`t1`.`b` AS `b`,`test`.`t2`.`c` AS `c`,`test`.`t2`.`d` AS
`d`,`test`.`t3`.`a` AS `a`,`test`.`t3`.`b` AS `b` from `test`.`t1` join
`test`.`t2` join `test`.`t1` `t3` where `test`.`t2`.`c` =
`test`.`t1`.`b` and `test`.`t3`.`a` = `test`.`t2`.`d` and
`test`.`t3`.`b` < 5 and `test`.`t1`.`a` < 2000 select * from t1, t2, t1
as t3@@ -1307,9 +1307,9 @@ explain extended select * from t1, t2, t1 as
t3 where t1.b=t2.c and t2.d=t3.a and t3.b<5 and t1.a < 2000; id select_
typetable   typepossible_keys   key key_len ref rows
filteredExtra-1 SIMPLE  t3  ALL NULLNULLNULL
NULL262144  0.00Using where+1   SIMPLE  t3  ALL NULL
NULLNULLNULL262117  0.00Using where 1   SIMPLE  t2  
ref c,d d   5   test.t3.a   7   100.00  -1  
SIMPLE  t1  ALL NULLNULLNULLNULL262144  2.00
Using where; Using join buffer (flat, BNL join)+1   SIMPLE  t1  
ALL NULLNULLNULLNULL262117  2.00Using where;
Using join buffer (flat, BNL join) Warnings: Note   1003select
`test`.`t1`.`a` AS `a`,`test`.`t1`.`b` AS `b`,`test`.`t2`.`c` AS
`c`,`test`.`t2`.`d` AS `d`,`test`.`t3`.`a` AS `a`,`test`.`t3`.`b` AS
`b` from `test`.`t1` join `test`.`t2` join `test`.`t1` `t3` where
`test`.`t1`.`b` = `test`.`t2`.`c` and `test`.`t2`.`d` = `test`.`t3`.`a`
and `test`.`t3`.`b` < 5 and `test`.`t1`.`a` < 2000 select * from t1,
t2, t1 as t3diff --git a/mysql-test/main/selectivity_innodb.result
b/mysql-test/main/selectivity_innodb.resultindex
93917065722..0b20a40f69f 

[Maria-developers] Review for follow up fix of MDEV-17323: Backport INFORMATION_SCHEMA.CHECK_CONSTRAINTS to 10.2

[Vicențiu Ciorbaru]

Hi Anel!
This is not a terribly bad patch, however you have forgotten the following
things:

You removed the misspelled is_check_constraint.test file, but forgot to 
remove

the is_check_constraint.result file. Now we would be left with a dangling
result file. Please be more careful next time.

You modified the logic of is_check_constraints.test file from the one in 
10.3


I suggest you use git cherry-pick next time when backporting a patch to get
the correct changes. This also helps with merges later, as a change in 10.2
will be easily reconcileable in 10.3.

Moving onwards with the patch, you will find other comments inline.
>
>
> commit c94f53a34e0b6bf313ecb36a13d5d138150a8a8d
> Author: Anel Husakovic 
> Date:   Thu Jan 24 03:06:56 2019 -0800
>
> diff --git a/mysql-test/suite/funcs_1/t/is_check_constraint.test 
b/mysql-test/suite/funcs_1/t/is_check_constraint.test

> deleted file mode 100644
> index 30a72d02b34..000
> --- a/mysql-test/suite/funcs_1/t/is_check_constraint.test
> +++ /dev/null
> @@ -1,92 +0,0 @@
> ---source include/have_innodb.inc
> ---source include/not_embedded.inc
> ---echo #
> ---echo # MDEV-17323: Backport INFORMATION_SCHEMA.CHECK_CONSTRAINTS 
to 10.2

> ---echo #
> -CREATE user boo1;
> -GRANT select,create,alter,drop on foo.* to boo1;
> -SHOW GRANTS for boo1;
> -CREATE user boo2;
> -create database foo;
> -# Connect with user boo1
> -CONNECT(con1,localhost, boo1,, foo);
> -
> -SET check_constraint_checks=1;
> -CREATE TABLE t0
> -(
> - t int, check (t>32) # table constraint
> -) ENGINE=myisam;
> ---sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -ALTER TABLE t0
> -ADD CONSTRAINT CHK_t0_t CHECK(t<100);
> ---sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -ALTER TABLE t0
> -DROP CONSTRAINT CHK_t0_t;
> ---sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -ALTER TABLE t0
> -ADD CONSTRAINT CHECK(t<50);
> ---sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -CREATE TABLE t1
> -( t int CHECK(t>2), # field constraint
> - tt int,
> - CONSTRAINT CHECK (tt > 32), CONSTRAINT CHECK (tt <50),# 
autogenerated names table constraints

> - CONSTRAINT CHK_tt CHECK(tt<100) # named table constraint
> -) ENGINE=InnoDB;
> - --sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -ALTER TABLE t1
> -DROP CONSTRAINT CHK_tt;
> ---sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -CREATE TABLE t2
> -(
> -name VARCHAR(30) CHECK(CHAR_LENGTH(name)>2), #field constraint
> -start_date DATE,
> -end_date DATE,
> -CONSTRAINT CHK_dates CHECK(start_date IS NULL) #table constraint
> -)ENGINE=Innodb;
> - --sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -ALTER TABLE t1
> -ADD CONSTRAINT CHK_new_ CHECK(t>tt);
> ---sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -# Create table with same field and table check constraint name
> -CREATE TABLE t3
> -(
> -a int,
> -b int check (b>0), # field constraint named 'b'
> -CONSTRAINT b check (b>10) # table constraint
> -) ENGINE=InnoDB;
> - --sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -DISCONNECT con1;
> -CONNECT(con2, localhost, boo2,, test);
> - --sorted_result
> -SELECT * from information_schema.check_constraints;
> -
> -DISCONNECT con2;
> -CONNECT(con1, localhost, boo1,,foo);
> -DROP TABLE t0;
> -DROP TABLE t1;
> -DROP TABLE t2;
> -DROP TABLE t3;
> -DROP DATABASE foo;
> -
> -DISCONNECT con1;
> ---CONNECTION default
> -DROP USER boo1;
> -DROP USER boo2;
This updated version of a test covers less functionality than the previous
version, I don't like that. Please keep the old version as is, just 
rename it
properly. Serg has already added the include to skip embedded tests in 
the most

up-to-date 10.2. Please rebase on top of that version before you perform the
rename.
>
> diff --git a/sql/sql_show.cc b/sql/sql_show.cc
> index 39763451dfb..e951c69d009 100644
> --- a/sql/sql_show.cc
> +++ b/sql/sql_show.cc
> @@ -6474,33 +6474,35 @@ static int get_check_constraints_record(THD 
*thd, TABLE_LIST *tables,

>  LEX_STRING *table_name)
>  {
>    DBUG_ENTER("get_check_constraints_record");
> -  if(res)
> +  if (res)
Nice catch with the missing whitespace.
>    {
>  if (thd->is_error())
>    push_warning(thd, Sql_condition::WARN_LEVEL_WARN,
> thd->get_stmt_da()->sql_errno(),
> thd->get_stmt_da()->message());
> -  thd->clear_error();
> -  DBUG_RETURN(0);
> +    thd->clear_error();
> +    DBUG_RETURN(0);
Great change, making sure the code is indented appropriately.
>    }
> -  if(!tables->view)
> +  else if (!tables->view)
Again, nice catch with the missing whitespace, but there is no need for
the else statement here. The DBUG_RETURN from above covers it.
>    {
> -    StringBuffer str(system_charset_info);
> -    for (uint i= 0; i < 

Re: [Maria-developers] 02587f61f40: MDEV-13095 Implement User Account locking

[Vicențiu Ciorbaru]

Hi Sergei!

I agree with most review comments except one:


> > information.
> > 
> > diff --git a/mysql-test/include/switch_to_mysql_user.inc b/mysql-
> > test/include/switch_to_mysql_user.inc
> > index f5801db6114..eff1d98c2df 100644
> > --- a/mysql-test/include/switch_to_mysql_user.inc
> > +++ b/mysql-test/include/switch_to_mysql_user.inc
> > @@ -45,6 +45,9 @@ CREATE TABLE mysql.user (
> >plugin char(64) CHARACTER SET latin1 DEFAULT '' NOT NULL,
> >authentication_string TEXT NOT NULL,
> >password_expired ENUM('N','Y') COLLATE utf8_general_ci DEFAULT
> > 'N' NOT NULL,
> > +  password_last_changed datetime NULL,
> > +  password_lifetime int(20) unsigned NULL,
> > +  account_locked enum('N','Y') COLLATE utf8_general_ci DEFAULT 'N'
> > NOT NULL,
> 
> No-no. mysql.user structure was frozen in time the moment I pushed
> the
> commit with the new structure. It doesn't change anymore.
> 
> To test 5.7 support, just add ALTER TABLE in one specific test file.
> See main/system_mysql_db_507.test

If we are to freeze mysql.user and be as "compatible" as possible , I
would freeze it with the latest version of 5.7 included, it feels the
most complete.

Any other reason that this is a no-no? :)

As for password_xxx columns, they were added at the same time here so
as to have the  final format in and simplify logic slightly with
mysql.user tabular. Your suggestion of handling it via types could work
even with the password columns I think.

One could split it into 2 commits, one that just adds columns and
minimal changes to keep things working and one that adds account
locking logic. 

Vicențiu


___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Extending storage engine API for random-row extraction for histogram collection (and others)

[Vicențiu Ciorbaru]

Hi Sergei!

> For the first iteration, after considering your input, I'd go with
> > "init function", "get random row", "end function", without imposing an
> > index, but somehow passing a (COND or similar) arg to the init
> > function.
>
> For the first iteration I'd go without a condition. You, probably
> shouldn't add an API that you won't use, and in the first iteration you
> won't use it, right? It can be added later when needed.
>

That's my dilemma, do we want to fiddle with the API frequently, as more
development work happens? If that's the case, I'll use the simplest
possible format that covers my use case for the time being.

Vicențiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Extending storage engine API for random-row extraction for histogram collection (and others)

[Vicențiu Ciorbaru]

On Tue, 11 Dec 2018 at 14:33 Sergei Golubchik  wrote:

> Hi, Vicențiu!
>
> On Dec 11, Vicențiu Ciorbaru wrote:
> > Hi!
> >
> > Here is my proposal on extending the storage engine API to provide a
> > functionality for retrieving random rows from tables (those that have
> > indexes). The storage engines for which I plan to implement this are:
> > MyISAM, Aria, Innodb. Possibly RocksDB, TokuDB.
> ...
> > Maybe I am complicating it too much and a simple random_sample()
> > function is sufficient, kind of how ha_records_in_range does it.
>
> Yeah. My first thought was something like index_random(). That is,
> index_init(), index_random(), index_random(), ..., index_end().
>
> index_init() does whatever it always needs to do to prepare using an
> index. and index_random() gets one random row.
>
> Not like records_in_range, that takes the index number and does
> _init/_end internally, because records_in_range is typically done only
> once or just a few times, while index_random() is more like
> index_next(), will be repeated continuously.
>
> But then I was thinking, why do you need to specify an index at all?
> Shouldn't it be just "get me a random row"? Index or whatever - that's
> engine implementation detail. For example, MyISAM with a fixed-size rows
> can just read from lseek(floor((file_size/row_size)*rand())*row_size).
>

I agree that the need for an index seems a bit much. My reasoning was that
I wanted to allow random sampling on a particular range. This could help
for example when one wants to collect histograms for a multi-distribution
dataset, to get individual distributions (if the indexed column is able to
separate them).

A more generic idea would be if one could pass some conditions for random
row retrieval to the storage engine, but it feels like this would
complicate storage engine implementation by quite a bit.

For the first iteration, after considering your input, I'd go with "init
function", "get random row", "end function", without imposing an index, but
somehow passing a (COND or similar) arg to the init function.

Sounds reasonable, or too much?
Vicențiu





Sergey had a good point about being able to repeat exactly the same
> sampling by specifying the seed. This should be solved if the engine
> will get its random numbers from the thd_rnd service.
>
> Regards,
> Sergei
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Extending storage engine API for random-row extraction for histogram collection (and others)

[Vicențiu Ciorbaru]

On Tue, 11 Dec 2018 at 12:43 Sergey Petrunia  wrote:

>
> On Tue, Dec 11, 2018 at 12:34:12AM +0200, Vicențiu Ciorbaru wrote:
> > Hi!
> >
> > Here is my proposal on extending the storage engine API to provide a
> > functionality for retrieving random rows from tables (those that have
> > indexes). The storage engines for which I plan to implement this are:
> > MyISAM, Aria, Innodb. Possibly RocksDB, TokuDB.
>
> Observations:
>
> - as far as I understand, random skip scan is not possible with this API?
> (which is probably fine as we expect that sampling will only retrieve a
> small
> fraction of table rows, which means the difference between a
> forward-walking
> skip scan and genuinely random probing is negligible).
>

One could add a flag to the init function specify this, but it feels a bit
too much, hence I left it out.


> - Can the scan return the same row twice?
>
> The way I see it now, it is implementation defined within the storage
engine. I'd vote to keep it like that. I don't see a real use case for
ensuring no row duplication inside the storage engine. One can keep a
Unique Object if this is really problematic.


> - Do we want/need a concept of random "seed" which will cause the same
> rows to
> be returned on the same table?
>

I've thought about this. I think it would be useful and it could be passed
via the init function, but I couldn't find similar examples in the API and
I kept it out. Should I add it?

>
> >
> > --- a/sql/handler.h
> > +++ b/sql/handler.h
> > @@ -2927,7 +2927,7 @@ class handler :public Sql_alloc
> >/** Length of ref (1-8 or the clustered key length) */
> >uint ref_length;
> >FT_INFO *ft_handler;
> > -  enum init_stat { NONE=0, INDEX, RND };
> > +  enum init_stat { NONE=0, INDEX, RND, RANDOM };
> >init_stat inited, pre_inited;
> > 
> > +  virtual int ha_random_sample_init()
> __attribute__((warn_unused_result))
> > +  {
> > +DBUG_ENTER("ha_random_sample_init");
> > +inited= RANDOM;
> > +DBUG_RETURN(random_sample_init());
> > +  }
> > +  virtual int ha_random_sample(uint inx,
> > +   key_range *min_key,
> > +   key_range *max_key)
> > +__attribute__((warn_unused_result))
> > +  {
> > +DBUG_ENTER("ha_random_sample");
> > +DBUG_ASSERT(inited == RANDOM);
> > +DBUG_RETURN(random_sample(inx, min_key, max_key));
> > +  }
> > +  virtual int ha_random_sample_end() __attribute__((warn_unused_result))
> > +  {
> > +DBUG_ENTER("ha_random_sample_end");
> > +inited= NONE;
> > +DBUG_RETURN(random_sample_end());
> > +  }
> > +
> >
> > This is the default implementation for a storage engine which does not
> > support it:
> >
> > +  virtual int random_sample_init() { return 0; } ;
> > +  virtual int random_sample(uint idx, key_range *min_key, key_range
> > *max_key)
> > +  {
> > +return HA_ERR_WRONG_COMMAND;
> > +  }
> > +  virtual int random_sample_end() { return 0; };
> >
> > Alternative ideas: random_sample_init() takes the idx as a parameter and
> > random_sample just fetches a row from the range using the index
> previously
> > specified. The range can be left unspecified with nulls to provide a
> fetch
> > from the full table range.
> >  I don't know enough about storage engine internals to know if an index
> > declaration within the init function instead of within the "sample"
> > function is better. Maybe I am complicating it too much and a simple
> > random_sample() function is sufficient, kind of how ha_records_in_range
> > does it.
> >
> > Thoughts?
> > Vicențiu
>
> --
> BR
>  Sergei
> --
> Sergei Petrunia, Software Developer
> MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog
>
>
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Extending storage engine API for random-row extraction for histogram collection (and others)

[Vicențiu Ciorbaru]

Hi!

Here is my proposal on extending the storage engine API to provide a
functionality for retrieving random rows from tables (those that have
indexes). The storage engines for which I plan to implement this are:
MyISAM, Aria, Innodb. Possibly RocksDB, TokuDB.

--- a/sql/handler.h
+++ b/sql/handler.h
@@ -2927,7 +2927,7 @@ class handler :public Sql_alloc
   /** Length of ref (1-8 or the clustered key length) */
   uint ref_length;
   FT_INFO *ft_handler;
-  enum init_stat { NONE=0, INDEX, RND };
+  enum init_stat { NONE=0, INDEX, RND, RANDOM };
   init_stat inited, pre_inited;

+  virtual int ha_random_sample_init() __attribute__((warn_unused_result))
+  {
+DBUG_ENTER("ha_random_sample_init");
+inited= RANDOM;
+DBUG_RETURN(random_sample_init());
+  }
+  virtual int ha_random_sample(uint inx,
+   key_range *min_key,
+   key_range *max_key)
+__attribute__((warn_unused_result))
+  {
+DBUG_ENTER("ha_random_sample");
+DBUG_ASSERT(inited == RANDOM);
+DBUG_RETURN(random_sample(inx, min_key, max_key));
+  }
+  virtual int ha_random_sample_end() __attribute__((warn_unused_result))
+  {
+DBUG_ENTER("ha_random_sample_end");
+inited= NONE;
+DBUG_RETURN(random_sample_end());
+  }
+

This is the default implementation for a storage engine which does not
support it:

+  virtual int random_sample_init() { return 0; } ;
+  virtual int random_sample(uint idx, key_range *min_key, key_range
*max_key)
+  {
+return HA_ERR_WRONG_COMMAND;
+  }
+  virtual int random_sample_end() { return 0; };

Alternative ideas: random_sample_init() takes the idx as a parameter and
random_sample just fetches a row from the range using the index previously
specified. The range can be left unspecified with nulls to provide a fetch
from the full table range.
 I don't know enough about storage engine internals to know if an index
declaration within the init function instead of within the "sample"
function is better. Maybe I am complicating it too much and a simple
random_sample() function is sufficient, kind of how ha_records_in_range
does it.

Thoughts?
Vicențiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] MDEV-12321 authentication plugin: SET PASSWORD support

[Vicențiu Ciorbaru]

Hi Sergei!

About the syntax problem, inline:

On 17/10/2018 23:40, Sergei Golubchik wrote:

Hi,

Now, the syntax problem.

Old MySQL syntax (for the last ~20 years) was

(1)  GRANT ... TO user IDENTIFIED BY 'plain-text password'
(2)  GRANT ... TO user IDENTIFIED BY PASSWORD 'password hash'
(3)  SET PASSWORD = 'password hash'
(4)  SET PASSWORD = PASSWORD('plain-text password')
(5)  SET PASSWORD = OLD_PASSWORD('plain-text password')

here, syntax (1) and (4) were forcing mysql_native_password
authentication, (5) was forcing mysql_old_password, and (2) and (3) were
auto-detecting, based on the hash length.

Later MariaDB and MySQL added support for pluggable authentication with
the syntax

(6)  GRANT ... TO user IDENTIFIED VIA plugin AS 'password hash'

MySQL 5.7 added support for specifying plain-text password for plugins
using the syntax

(7) GRANT ... TO user IDENTIFIED WITH plugin BY 'plain-text password'

I don't quite like it because there's no logical reason why "BY" means a
plain-text password, while "AS" means a hash. Also we support "USING"
instead of "AS", which also means a hash. One can easily get lost in all
these USING/AS/BY and what special semantics each of them has.


I can also never remember what is the difference between BY/AS/USING.


The syntax I've implemented is based on SET PASSWORD:

(8) GRANT ... TO user IDENTIFIED VIA plugin AS PASSWORD('plain-text password')

This is quite intuitive and pretends that there's a sql function
PASSWORD() which returns a hash and it's used as an expression where a
hash is anyway expected. Same works in SET PASSWORD too, obviously.
A PASSWORD() function actually exists and returns the password hash.

The problem here is that PASSWORD() function becomes quite magical. It
returns a different password, depending on what plugin the user is
using. One can still do SELECT PASSWORD("foo"), OLD_PASSWORD("foo"),
but they'll return values for mysql_native_password and
mysql_old_password as before. In the context of SET PASSWORD or GRANT
(or CREATE/ALTER USER) it becomes context dependent, it's a bit
difficult to swallow.

Another approach would be not to pretend it's a function. Say

(9) GRANT ... TO user IDENTIFIED VIA plugin AS PASSWORD 'plain-text password'
 SET PASSWORD = PASSWORD 'plain-text password'

but, unfortunately, it is exactly backwards from the historical behaviour
of (1) and (2).

All in all I'm leaning towards (8), but I'm not quite happy with it :(
One way to solve it could be to extend PASSWORD() function to allow a
second argument, plugin name, like in

   SELECT PASSWORD("foo", "ed25519")

Yet another way could be to remove SQL-level functions PASSWORD() and
OLD_PASSWORD(). That would be my favorite, they always were nothing but
trouble. But I wouldn't risk doing it now :)


I think (8) is ok and intuitively that's what I tried to do when writing 
test cases for your patch.


I personally would add the extra parameters to PASSWORD. I don't think 
we should remove PASSWORD() function, but we should clearly 
mark it as not-recommended. One might need to figure out what hash a 
plugin generates for a certain plain-text string and I don't see a way 
one would do that currently without this extension.


Vicențiu


___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] MDEV-12321 authentication plugin: SET PASSWORD support

[Vicențiu Ciorbaru]

Hi Sergei!

Here are my review comments, inline. Read them from oldest to newest 
commit, that's how I wrote them. :)


On 17/10/2018 23:40, Sergei Golubchik wrote:

Hi,

Please review commits for

   MDEV-12321 "authentication plugin: SET PASSWORD support"

They're now in the bb-10.4-serg branch, the last seven commits:

=
commit 62e8340f513
Author: Sergei Golubchik 
Date:   Wed Oct 17 12:48:13 2018 +0200

 MDEV-12321 authentication plugin: SET PASSWORD support
 
 Support SET PASSWORD for authentication plugins.
 
 Authentication plugin API is extended with two optional methods:

 * hash_password() is used to compute a password hash (or digest)
   from the plain-text password. This digest will be stored in mysql.user
   table
 * preprocess_hash() is used to convert this digest into some memory
   representation that can be later used to authenticate a user.
   Build-in plugins convert the hash from hexadecimal or base64 to binary,
   to avoid doing it on every authentication attempt.
 
 Note a change in behavior: when loading privileges (on startup or on

 FLUSH PRIVILEGES) an account with an unknown plugin was loaded with a
 warning (e.g. "Plugin 'foo' is not loaded"). But such an account could
 not be used for authentication until the plugin is installed. Now an
 account like that will not be loaded at all (with a warning, still).
 Indeed, without plugin's preprocess_hash() method the server cannot know
 how to load an account. Thus, if a new authentication plugin is
 installed run-time, one might need FLUSH PRIVILEGES to activate all
 existing accounts that were using this new plugin.

First a note about the behavior change:

This potentially causes a difficulty in debugging failed user 
authentication. Before, if say unix socket auth plugin was not installed 
when trying to connect, you'd get ER_PLUGIN_IS_NOT_LOADED, now you get 
ER_ACCESS_DENIED_ERROR. This provided a hint as to what went wrong. Is 
there a way for an admin to track down the issue now, without resorting 
to blind "FLUSH PRIVILEGES;"? (I've never been in that position so I 
don't know if this is really an issue or not)


Also, warnings now on flush privileges are hard to associate with a 
particular user, if one has a lot of them. Maybe we add a "for user xxx" 
to the warnings?


Can one update privileges on a user which was not loaded? I saw that 
dropping works, but granting something leads to a weird side-effect of 
updating the plugin & authentication_string fields for the user:


 Before grant. (from grant5.test)

select user, host, plugin, authentication_string from mysql.user where 
user = 'u1';

user    host    plugin    authentication_string
u1    h    mysql_native_password    bad

 After grant

grant SELECT on *.* to u1@h;
select user, host, plugin, authentication_string from mysql.user where 
user = 'u1';

user    host    plugin    authentication_string
u1    h    mysql_native_password

If before grant, the user would have had something other than 
mysql_native_password, that one would have been changed too. It's a 
weird corner case, but do you think this is the correct way to handle 
it? I'd rather not mess with changing the plugin and 
authentication_string in this case. (I did see the code about the 
historical-hack and guess_auth_plugin)


I also did check this in 10.2 (just because it was already compiled, but 
nothing significant changed in 10.3), that grant in this case would not 
drop the user's password.



Here's an example:
CREATE USER foo@localhost identified by 'pwd';

select user, host, password, plugin, authentication_string from 
mysql.user where user='foo';

#This grant won't drop the password.
grant select on *.* to foo@localhost;
select user, host, password, plugin, authentication_string from 
mysql.user where user='foo';


update mysql.user set authentication_string = 'bad' where user='foo';
flush privileges;

select user, host, password, plugin, authentication_string from 
mysql.user where user='foo';

# This grant will drop the password.
grant select on *.* to foo@localhost;
select user, host, password, plugin, authentication_string from 
mysql.user where user='foo';


connect con1, localhost, foo,,;
# foo can select everything, with no password
select user, host from mysql.user;

I suggest we don't overwrite authentication string in this case within 
the grant statement in this case.



Why the change in mysql-test/suite/rpl/include/rpl_mixed_dml.inc? It 
feels a bit related to the paragraph above.



Now about the code itself:

hash_password function call could lead to buffer overflow, then all 
sorts of potential security hacks if the plugin doesn't play nice and 
writes more than MAX_SCRAMBLE_LENGTH to the buffer. Not sure if there's 
a way to guard against this or if we want to in the first place. It is 
the admin's job to load sane password plugins right?


I know this would 

Re: [Maria-developers] Please review MDEV-16584 SP with a cursor inside a loop wastes THD memory aggressively

[Vicențiu Ciorbaru]

Hi Alexander!

Can you double check that this same problem doesn't occur with custom
aggregate functions? There is a switch involved with cursors and custom
aggregate functions here:

  /* Only pop cursors when we're done with group aggregate running. */
  if (m_chistics.agg_type != GROUP_AGGREGATE ||
  (m_chistics.agg_type == GROUP_AGGREGATE && thd->spcont->quit_func))
thd->spcont->pop_all_cursors(thd); // To avoid memory leaks after an
error

That may require a different approach.

Vicențiu


On Tue, 26 Jun 2018 at 13:44 Alexander Barkov  wrote:

> Hello Sanja,
>
> Please review a patch for MDEV-16584.
>
> Thanks!
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Please review a patch for MDEV-15758 Split Item_bool_func::get_mm_leaf() into virtual methods in Field and Type_handler

[Vicențiu Ciorbaru]

Hi Alexander!

(I was doing some experimentation with the code and forgot to send the
email properly)

So the main things we discussed during the review:

We discussed alternative of not adding SEL_ARG_XXX classes and instead have
separate constructors within SEL_ARG. I tried to do this myself but I am
not happy with the results. So in this case, let's add extra comments for
SEL_ARG_XXX classes, explaining their purpose as Sergey Petrunia mentioned.

Check DBUG_ENTER strings. Some don't match the actual function name.

The patch in whole looks good, however it took me quite a bit of time to
understand where the extra logic is added and where it's just part of
refactoring. If feasible, with least amount of effort, split the patch up
into 2 pieces, one that only does refactoring and one that introduces the
extra optimization code (plus small refactoring to get it to work). The
specific function I'm looking at
is: Field::stored_field_make_mm_leaf_bounded_int

This will make it a lot easier when tracking history, in case new bugs
arise to understand the scope of the change much quicker. The first patch
should not require any test result changes at all, while the second one
should highlight the optimizations now happening. If for some reason the
type system gets smarter just through the refactoring part, then we update
test results there, but from what I understand from the patch, that should
not be the case. If it does change the result, maybe we should discuss this
more as it means I missed something.

Within stored_field_cmp_to_item, the long if statement logic about various
field types that has now been split within the type system, I like that
bit. I did step through the code for quite some time and it seems like
perfectly fine logic, but the mechanism takes a while to grasp and I am not
100% confident I remember or understood everything. For the scope of this
review, I was satisfied with what I could figure out.

Thanks for your patience in receiving this review.
Vicențiu

On Mon, 9 Jul 2018 at 11:03 Alexander Barkov  wrote:

> Hi Vicențiu,
>
> Sending the patch adjusted to the latest 10.4.
>
> Thanks.
>
>
> On 04/03/2018 12:57 PM, Alexander Barkov wrote:
> > Hello Vicentiu,
> >
> > Please review my patch for MDEV-15758.
> >
> > It also fixes this bug:
> >
> > MDEV-15759 Expect "Impossible WHERE" for
> > indexed_int_column=out_of_range_int_constant
> >
> > Thanks!
> >
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] unbundle RocksDB

[Vicențiu Ciorbaru]

Hi Michal,

Just so you know, we've been discussing the possibility of adding
compression services to the server, which would allow removing some static
libraries from the code. This however is probably a bit far on the horizon
as we haven't particularly scheduled it for 10.4 (yet).

Vicentiu

On Sat, 9 Jun 2018 at 17:49 Sergey Petrunia  wrote:

> Hi Michal,
>
> On Wed, Jun 06, 2018 at 11:21:34AM +0200, Michal Schorm wrote:
> > Since we have a rocksdb package ready in Fedora, I tried to unbundle it,
> > using the system package instead.
> >
> > I wasn't successful though.
> > I haven't found any CMake argument which seemed related and from the
> build
> > log it seems, the code tries to link against the static libraries in any
> > case.
> >
> > Before I'll start to dig further, I'm stopping by on this list, making
> sure
> > it is possible.
> >  - is it currently supported?
> >  - have I overlooked any related CMake argument?
>
> It is not currently supported.
>
> MariaDB gets MyRocks (and RocksDB) from the upstream:
> https://github.com/facebook/mysql-5.6/. Both upstream and MariaDB have
> RocksDB
> as a submodule and so are tied to a particular revision of RocksDB.
>
> Using a different revision of RocksDB will typically work, but
> - it is not a tested combination
> - it will not necessarily work, especially if you use an older version of
> RocksDB than the one that MyRocks used. RocksDB development continues, so
> there may be implicit dependencies that MyRocks at revision X requires
> RocksDB to be at least at revision Y.
>
> BR
>  Sergei
> --
> Sergei Petrunia, Software Developer
> MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog
>
>
>
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Proposal for Negative Grants Project

[Vicențiu Ciorbaru]

Hi Sergey!

On Tue, 29 May 2018 at 17:54 Sergei Golubchik  wrote:

> Hi, Vicențiu!



As you might've guessed from my comments above, I'd rather prefer to use
> REVOKE instead of DENY, if we can make it to work (and to explain how it
> works afterwards :).
>
> So here I suggest a semantics how REVOKE could be used for negative
> grants (note, I'm not talking about storage or any other technical
> details here).
>
> In the standard REVOKE removes an existing GRANT and must match the
> existing GRANT exactly. Extending this idea, we say that REVOKE *makes a
> hole* in the existing GRANT and must be *a subset* of an existing GRANT
> (a subset could be equal to the whole GRANT too). So REVOKE is a set
> difference operation.
>
> Note the difference with DENY approach - DENY exists as a separate
> entity, it may deny access to something that isn't granted, after
> revoking a grant DENY stays and needs to be revoked separately.
>
> While REVOKE is not a separate entity. It simply reduces the scope of
> the original GRANT. You cannot "revoke a REVOKE", to remove a hole you
> revoke that GRANT that has it.
>
> This interpretation doesn't have some features that DENY approach has.
> On the other hand it's surely much more intuitive. And it's an extention
> of the standard semantics, not a new totally feature to learn.
>
>
I've spent some time thinking about this today. I believe we can
potentially do something, however do you think it is ok to modify grant
tables such that the privilege columns now have the form enum(Y,N,D)? I
have not fully fleshed out the design of how revoke would work in all cases
but this is how I think we can implement grants to work efficiently.

The scenario I'm thinking of is:

Huge db with a million tables.

GRANT SELECT ON db.* TO foo;
REVOKE SELECT ON db.test FROM foo;

If we have a revoke which carves out holes, our current basic approach is
to just drop the db SELECT privilege and grant SELECT on all other tables.
I have a feeling that this imposes a big performance penalty, both that we
now need to write a million rows to mysql grant table table and the fact
that we need to load a whole bunch of stuff in memory now. I don't remember
the exact implementation details here, but I remember that it's not exactly
O(1) to check if a table is in the grant table list. (I will double check
this).

On the other hand it should be rather easy for revoke to set a "deny" bit
for more specific grants if there already is a more general grant (as in
this example: a whole database level grant is present and we introduce a
table level grant too)

When one GRANTS SELECT ON db.* TO foo; again, we would need to double check
all possible affected entries (all table and column level grants that match
database db) and clear any SELECT deny bits.

I believe this would be the most intuitive approach. It would keep working
the same for already existing applications, except for those that revoke
stuff expecting it to have no effect whatsoever, and maybe we can set an
SQL MODE for that.

Give me a couple of days to fully think about this, but please let me know
if you agree with changing the privilege columns table structure from
enum(Y,N) to enum(Y,N,D). This is the best solution I could come up with
that is "minimally invasive" and also should provide sufficient info to
recreate the internal structures on server restart. I don't see any
potential problems with this, but you've been through more GA releases than
me so you know the user pains better :)

Regards,
Vicențiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] With_sum_func_cache

[Vicențiu Ciorbaru]

Hi Alexander!

I was looking through this patch as I am rather familiar with this code. I
didn't take time to test this out, but maybe you can explain if this is a
possible concern or not:

index 4a95189..7d1532c 100644
--- a/sql/item_sum.cc
+++ b/sql/item_sum.cc
@@ -404,7 +404,7 @@ bool Item_sum::register_sum_func(THD *thd, Item **ref)
 for (sl= thd->lex->current_select;
  sl && sl != aggr_sel && sl->master_unit()->item;
  sl= sl->master_unit()->outer_select() )
-  sl->master_unit()->item->with_sum_func= 1;
+
sl->master_unit()->item->get_with_sum_func_cache()->set_with_sum_func();
   }
   thd->lex->current_select->mark_as_dependent(thd, aggr_sel, NULL);

@@ -484,7 +484,6 @@ void Item_sum::mark_as_sum_func()
   cur_select->n_sum_items++;
   cur_select->with_sum_func= 1;
   const_item_cache= false;
-  with_sum_func= 1;
   with_field= 0;
   window_func_sum_expr_flag= false;
 }
diff --git a/sql/item_sum.h b/sql/item_sum.h
index 96f1153..37f3fe0 100644
--- a/sql/item_sum.h
+++ b/sql/item_sum.h
@@ -582,6 +582,8 @@ class Item_sum :public Item_func_or_sum
   void mark_as_window_func_sum_expr() { window_func_sum_expr_flag= true; }
   bool is_window_func_sum_expr() { return window_func_sum_expr_flag; }
   virtual void setup_caches(THD *thd) {};
+
+  bool with_sum_func() const { return true; }
 };

For Item_sum::register_sum_func, if sl->master_unit()->item is an
Item_sum_sum for example, an Item_sum won't have get_with_sum_func_cache()
overwritten so it will be the base Item::get_with_sum_func_cache(), which
returns null and you will crash. Am I missing something?

Is it impossible for sl->master_unit()->item to be an Item_sum_... subclass?

I am not a very big fan of the get_with_sum_func_cache() indirection
required and would prefer, if possible to call set_with_sum_func()
directly. Perhaps behind the scenes the set function can do that and throw
an assert if the call is illegal? (Just an opinion, not something I have a
very very strong opinion on.

Also, I have a feeling that it's sufficient to keep just
join_with_sum_func. I can't really think of a place where that was not the
intent anyway, but those few cases where copy_with_sum_func is used need to
be analysed throughly to make sure.

Regards,
Vicențiu

PS: Yes, the review assigned to me is coming :)


On Tue, 29 May 2018 at 15:07 Alexander Barkov  wrote:

> Hello Sanja,
>
> I recently worked on MDEV-16309 and had hard time to find
> which Item classes in the hierarchy:
> - have always with_sum_func==true
> - have always with_sum_func==false
> - have variable with_sum_func
>
> To make it sure, before actually working on MDEV-16309,
> I had to create a separate working tree and did with
> Item::with_sum_func the same change that we previously
> did for Item::with_subselect in:
>
> MDEV-14517 Cleanup for Item::with_subselect and Item::has_subquery()
> (which you reviewed)
>
> - I find the code easier to read this way
>   (instead of searching for all possible assignments,
>one can search who overrides the method)
> - Also, some Item can become smaller in size.
>
> It's pity to throw the patch away. So perhaps we could just apply it.
>
> Can you please have a look?
>
> Thanks.
>
>
> I the meanwhile I'll create a new MDEV for it
> (with a similar description to MDEV-14517)
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 4861977ce76: MDEV-14695: Assertion `n < m_size' failed in Bounds_checked_array::operator

[Vicențiu Ciorbaru]

Hi Varun!

I will take a second look at this but a first minor comment is to not save
two characters by shortening fields to flds, it makes reading unnatural.

Vicentiu

On Sat, 12 May 2018 at 21:45 Varun  wrote:

> revision-id: 4861977ce76761fefb9779a774984b6afa818f2e
> (mariadb-10.2.5-636-g4861977ce76)
> parent(s): 8b26fea83572cf5c043721b7835c3828937f9c27
> author: Varun Gupta
> committer: Varun Gupta
> timestamp: 2018-05-13 00:09:15 +0530
> message:
>
> MDEV-14695: Assertion `n < m_size' failed in
> Bounds_checked_array::operator
>
> In this issue we hit the assert because we are adding addition fields to
> the field JOIN::all_fields list. This
> is done because  HEAP tables can't index BIT fields so  we need to use an
> additional hidden field for grouping because later it will be
> converted to a LONG field. Original field will remain of the BIT type and
> will be returned. This happens when we convert DISTINCT to
> GROUP BY.
>
> The solution is to take into account the number of such hidden fields that
> would be added to the field JOIN::all_fields list while calculating
> the size of the ref_pointer_array.
>
> ---
>  sql/sql_base.cc   | 9 ++---
>  sql/sql_base.h| 4 ++--
>  sql/sql_delete.cc | 3 ++-
>  sql/sql_lex.cc| 8 +++-
>  sql/sql_lex.h | 5 +
>  sql/sql_select.cc | 4 +++-
>  sql/sql_union.cc  | 1 +
>  7 files changed, 26 insertions(+), 8 deletions(-)
>
> diff --git a/sql/sql_base.cc b/sql/sql_base.cc
> index 95ff41cb6f4..81ccfe5e514 100644
> --- a/sql/sql_base.cc
> +++ b/sql/sql_base.cc
> @@ -6919,7 +6919,7 @@ static bool setup_natural_join_row_types(THD *thd,
>
>  int setup_wild(THD *thd, TABLE_LIST *tables, List ,
>List *sum_func_list,
> -  uint wild_num)
> +  uint wild_num, uint *hidden_bit_flds)
>  {
>if (!wild_num)
>  return(0);
> @@ -6960,7 +6960,7 @@ int setup_wild(THD *thd, TABLE_LIST *tables,
> List ,
>else if (insert_fields(thd, ((Item_field*) item)->context,
>   ((Item_field*) item)->db_name,
>   ((Item_field*) item)->table_name, ,
> - any_privileges))
> + any_privileges, hidden_bit_flds))
>{
> if (arena)
>   thd->restore_active_arena(arena, );
> @@ -7425,7 +7425,7 @@ bool get_key_map_from_key_list(key_map *map, TABLE
> *table,
>  bool
>  insert_fields(THD *thd, Name_resolution_context *context, const char
> *db_name,
>   const char *table_name, List_iterator *it,
> -  bool any_privileges)
> +  bool any_privileges, uint *hidden_bit_flds)
>  {
>Field_iterator_table_ref field_iterator;
>bool found;
> @@ -7545,6 +7545,9 @@ insert_fields(THD *thd, Name_resolution_context
> *context, const char *db_name,
>else
>  it->after(item);   /* Add 'item' to the SELECT list. */
>
> +  if (item->type() == Item::FIELD_ITEM && item->field_type() ==
> MYSQL_TYPE_BIT)
> +(*hidden_bit_flds)++;
> +
>  #ifndef NO_EMBEDDED_ACCESS_CHECKS
>/*
>  Set privilege information for the fields of newly created views.
> diff --git a/sql/sql_base.h b/sql/sql_base.h
> index d6063f1b771..8d0dcb2f197 100644
> --- a/sql/sql_base.h
> +++ b/sql/sql_base.h
> @@ -154,11 +154,11 @@ bool fill_record_n_invoke_before_triggers(THD *thd,
> TABLE *table,
>enum trg_event_type event);
>  bool insert_fields(THD *thd, Name_resolution_context *context,
>const char *db_name, const char *table_name,
> -   List_iterator *it, bool any_privileges);
> +   List_iterator *it, bool any_privileges, uint
> *hidden_bit_flds);
>  void make_leaves_list(THD *thd, List , TABLE_LIST
> *tables,
>bool full_table_list, TABLE_LIST *boundary);
>  int setup_wild(THD *thd, TABLE_LIST *tables, List ,
> -  List *sum_func_list, uint wild_num);
> +  List *sum_func_list, uint wild_num, uint *
> hidden_bit_flds);
>  bool setup_fields(THD *thd, Ref_ptr_array ref_pointer_array,
>List , enum_mark_columns mark_used_columns,
>List *sum_func_list, List *pre_fix,
> diff --git a/sql/sql_delete.cc b/sql/sql_delete.cc
> index 7b7a7e3f804..7bae7736cf3 100644
> --- a/sql/sql_delete.cc
> +++ b/sql/sql_delete.cc
> @@ -760,7 +760,8 @@ l
>  select_lex->leaf_tables, FALSE,
>  DELETE_ACL, SELECT_ACL, TRUE))
>  DBUG_RETURN(TRUE);
> -  if ((wild_num && setup_wild(thd, table_list, field_list, NULL,
> wild_num)) ||
> +  if ((wild_num && setup_wild(thd, table_list, field_list, NULL, wild_num,
> +  _lex->hidden_bit_flds)) ||
>setup_fields(thd, Ref_ptr_array(),
> field_list, MARK_COLUMNS_READ, NULL, NULL, 0) ||
>setup_conds(thd, table_list, 

Re: [Maria-developers] Proposal for Negative Grants Project

[Vicențiu Ciorbaru]

Hi Sergei!

On Fri, 11 May 2018 at 21:10 Sergei Golubchik  wrote:

> Hi, Rutuja!
>
> I think it would've be useful to put your main ideas into MDEV-14443, as
> a comment.
>
> For now I've just added a link to your email.
>
> On May 09, Rutuja Surve wrote:
> > Hello,
> > Please find the proposal for the Negative Grants project attached along
> > with this e-mail. It would be great to hear your opinions and suggestions
> > on the approach proposed here, before delving into its implementation.
> > Some context about the project:
> >
> > *The current privilege system allows access in this manner:GRANT SELECT
> ON
> > some_database.* TO a_user@%;Any revoke would revoke access from the
> entire
> > database. We want to support a revoke that would disable select on a
> > specific table, like:REVOKE SELECT ON some_database.secret_table FROM
> > a_user@%;Reference:[1] https://jira.mariadb.org/browse/MDEV-14443
> > https://mariadb.com/kb/en/library/google-summer-of-code-2018/
> > Thanks,Rutuja*
>
> Random thoughts:
>
> * It's good that SQL Server supports DENY statement, a precedent is
> a very strong argument for us to do it that way.
>
> * I wasn't able to find any other database that has this functionality.
> (but many questions about how to achieve that result)
>
> * DENY is not really an antonym of GRANT. If we'd have an ALLOW
> statement, DENY would've been a logical choice. But we don't.
>
> * REVOKE is a logical complement to GRANT, some of these questions
> that I've found were answered with, like
>   GRANT ... ON *.*
>   REVOKE ... ON somedb.*
> which, of course, is wrong, but it shows what an intuitive answer is
>
> * DENY encourages wrong thinking - I've seen questions like "how to deny
> access to all tables, but one". The correct answer is, of course, just
> grant access to one table, don't use DENY at all. If there's no DENY,
> there's no place for such a mistake.
>
> * DENY semantics is quite simple in SQL Server (if we disregard strange
> treatment of column level grants). It's very easy to explain, which is
> good. I'm not totally grasping how a REVOKE could work :(
>
> * How do you undo a DENY statement?
>

I proposed an extension of REVOKE:
REVOKE DENY xxx, similar to how a REVOKE ROLE would function.

REVOKE ALL DENIES FOR foo

This way there are 2 separate commands. GRANT and REVOKE for positive
grants and
DENY and REVOKE DENY for negative grants.

Perhaps REVOKE DENY is not the greatest idea, but that's why this was
brought up for discussion. Any better solution here would help.

* I'm not a great fan of duplicating all tables, I'd rather reuse
> existing ones
>

>From an implementation standpoint it's almost trivial to reuse as well as
duplicate. Reusing tables means that each mysql privilege table can
potentially have two entries for the same user@host combination, one for
denies, one for grants and have them distinguished through a boolean column
(say is_deny, very similar to is_role)

This requires changing the current primary key definition of these tables
as now we have host and user as uniques.

I think that 2 entries in the privilege table for the same user,host
combination might cause confusion for users, hence the suggestion for
duplicating tables (only the host, user && priv_xxx columns, not the other
ones).

SQL server has a bit of a strange approach to revoking denies, it feels
like each priv column internally for them is a tri-state version of our
solution. (Yes, No, Deny) and REVOKING moves it to No always, regardless of
previous state.

Thoughts?
Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 326db1a: Mdev-14853 Grant does not work correctly when table contains...

[Vicențiu Ciorbaru]

Hi Sachin!

Did you run this test on embedded server? Usually grant related tests
require some form of not_embedded.inc include. (check roles suite for
examples)

Vicentiu

On Thu, 26 Apr 2018 at 00:20 sachin  wrote:

> revision-id: 326db1a2aaa9b275a1a21a863e8cd2d9fa1b1d5f
> (mariadb-10.3.6-46-g326db1a)
> parent(s): 9477a2a9ba17c0db362e2bb39d5048e369096f39
> author: Sachin Setiya
> committer: Sachin Setiya
> timestamp: 2018-04-26 12:47:25 +0530
> message:
>
> Mdev-14853 Grant does not work correctly when table contains...
> SYSTEM_INVISIBLE or COMPLETELY_INVISIBLE
>
> This commit does multiple things to solve this mdev
> 1st add field into the parameter of check_column_grant_in_table_ref, so
> that
> we can find out field invisibility.
> 2nd If field->invisible >= INVISIBLE_SYSTEM skip access check and simple
> grant access.
>
> ---
>  mysql-test/main/invisible_field_grant.result | 111
> +++
>  mysql-test/main/invisible_field_grant.test   |  77 +++
>  sql/sp_rcontext.cc   |   9 ++-
>  sql/sql_acl.cc   |  11 ++-
>  sql/sql_acl.h|   2 +-
>  sql/sql_base.cc  |   4 +-
>  6 files changed, 206 insertions(+), 8 deletions(-)
>
> diff --git a/mysql-test/main/invisible_field_grant.result
> b/mysql-test/main/invisible_field_grant.result
> new file mode 100644
> index 000..c3ccbb1
> --- /dev/null
> +++ b/mysql-test/main/invisible_field_grant.result
> @@ -0,0 +1,111 @@
> +set @old_debug= @@debug_dbug;
> +create user user_1;
> +show grants for user_1;
> +Grants for user_1@%
> +GRANT USAGE ON *.* TO 'user_1'@'%'
> +# create user
> +create database d;
> +use d;
> +
> +#System_Invisible
> +set debug_dbug= "+d,test_pseudo_invisible";
> +create table t1(a int);
> +set debug_dbug=@old_debug;
> +insert into t1 values(1);
> +select a,invisible from t1;
> +a  invisible
> +1  9
> +grant insert(a) on t1 to user_1;
> +grant update(a) on t1 to user_1;
> +grant select(a) on t1 to user_1;
> +grant delete on t1 to user_1;
> +connect  con1, localhost, user_1,,test;
> +connection con1;
> +select user();
> +user()
> +user_1@localhost
> +use d;
> +select * from t1;
> +a
> +1
> +insert into t1 values(2);
> +select * from t1;
> +a
> +1
> +2
> +insert into t1(a) values(3);
> +select * from t1;
> +a
> +1
> +2
> +3
> +select invisible,a from t1;
> +invisible  a
> +9  1
> +9  2
> +9  3
> +delete from t1 where a =1;
> +update t1 set a=1 where a=3;
> +select * from t1;
> +a
> +2
> +1
> +disconnect con1;
> +
> +#Cleanup
> +connection default;
> +drop table t1;
> +REVOKE ALL PRIVILEGES, GRANT OPTION FROM user_1;
> +
> +#Completely Invisible
> +set debug_dbug= "+d,test_completely_invisible";
> +create table t1(a int);
> +insert into t1 values(1);
> +select a,invisible from t1;
> +a  invisible
> +1  9
> +set debug_dbug=@old_debug;
> +grant insert(a) on t1 to user_1;
> +grant update(a) on t1 to user_1;
> +grant select(a) on t1 to user_1;
> +grant delete on t1 to user_1;
> +connect  con1, localhost, user_1,,test;
> +connection con1;
> +select user();
> +user()
> +user_1@localhost
> +use d;
> +select * from t1;
> +a
> +1
> +insert into t1 values(2);
> +select * from t1;
> +a
> +1
> +2
> +insert into t1(a) values(3);
> +select * from t1;
> +a
> +1
> +2
> +3
> +select invisible,a from t1;
> +ERROR 42S22: Unknown column 'invisible' in 'field list'
> +delete from t1 where a =1;
> +update t1 set a=1 where a=3;
> +select * from t1;
> +a
> +2
> +1
> +disconnect con1;
> +
> +#Final Cleanup
> +connection default;
> +set debug_dbug= "+d,test_completely_invisible";
> +select a,invisible from t1;
> +a  invisible
> +2  9
> +1  9
> +drop user user_1;
> +drop database d;
> +set @old_debug= @@debug_dbug;
> diff --git a/mysql-test/main/invisible_field_grant.test
> b/mysql-test/main/invisible_field_grant.test
> new file mode 100644
> index 000..0d627e5
> --- /dev/null
> +++ b/mysql-test/main/invisible_field_grant.test
> @@ -0,0 +1,77 @@
> +--source include/have_debug.inc
> +##TEST for invisible coloumn level 2
> +set @old_debug= @@debug_dbug;
> +create user user_1;
> +show grants for user_1;
> +--echo # create user
> +create database d;
> +use d;
> +
> +--echo
> +--echo #System_Invisible
> +set debug_dbug= "+d,test_pseudo_invisible";
> +create table t1(a int);
> +set debug_dbug=@old_debug;
> +insert into t1 values(1);
> +select a,invisible from t1;
> +grant insert(a) on t1 to user_1;
> +grant update(a) on t1 to user_1;
> +grant select(a) on t1 to user_1;
> +grant delete on t1 to user_1;
> +connect (con1, localhost, user_1,,test);
> +connection con1;
> +select user();
> +use d;
> +select * from t1;
> +insert into t1 values(2);
> +select * from t1;
> +insert into t1(a) values(3);
> +select * from t1;
> +select invisible,a from t1;
> +delete from t1 where a =1;
> +update t1 set a=1 where a=3;
> +select * from t1;
> +disconnect con1;
> +--source 

Re: [Maria-developers] Regarding bugs (mysqldump)

[Vicențiu Ciorbaru]

Hi Rutuja!

Great analysis again and lets share this with the maria-developers mailing
list as well.

I've tested your proposed change and the change for dump_all_tables_in_db
seems to not be necessary. If you look at how dump_all_tables_in_db gets
called, it's followed by dump_all_views_in_db. The actual view creation
happens in dump_all_views_in_db, while dump_all_tables_in_db will create
some dummy tables for views. We create dummy tables first, so as to avoid
having to check dependencies within views. There's more details in the
comments of why we do that in the code of get_table_structure:

/*
  Create a table with the same name as the view and with columns of
  the same name in order to satisfy views that depend on this view.
  The table will be removed when the actual view is created.

  The properties of each column, are not preserved in this temporary
  table, because they are not necessary.

  This will not be necessary once we can determine dependencies
  between views and can simply dump them in the appropriate order.
*/

For dump_selected_tables, your proposed change should be sufficient and
according to my tests it works. Please submit a pull request for this.
Looking forward to having this fixed soon. :)

Unrelated to your analysis:

There are plenty of edge cases where views won't work:
For example:
If we have 2 databases and one view uses something from a different
database, if you're not careful about the order of things, it's going to
fail. To create a "truly" always working dump one would have to create a
full dependency graph and then do a topological sort for all objects in the
database that depend on the objects we're dumping. This is outside of the
scope of this bug, however it could prove to be a "better way to dump
stuff".

Regards,
Vicentiu

On Sun, 1 Apr 2018 at 06:28 Rutuja Surve  wrote:

> Hello,
> Could you review my following approach regarding the mysqldump bug (
> https://jira.mariadb.org/browse/MDEV-15021):
>
> I went through the code and observed the following:
> 1. dump_all_tables_in_db calls dump_table first (for the list of all
> tables), then dump_table calls get_view_structure.
> 2. get_view_structure dumps the view for the corresponding table to
> mysqldump
> 3. After this call, dump_all_tables_in_db calls dump_routines_for_db
>
> dump_selected_tables also behaves in the exact same way as 
> dump_all_tables_in_db,
> following steps 1,2,3, except that it does it for a list of given tables
> instead of doing it for all tables.
>
> We want the routines to get dumped before the views for the dump to become
> importable.
> There are two solutions for this:
> 1. Make dump_all_tables_in_db call dump_routines_for_db before it calls
> dump_table. This will dump routines first and views later.
> This can be done by shifting the call to dump_routines_for_db before the
> loop in which dump_table is called. I.E:
> Shift :
> if (opt_routines && mysql_get_server_version(mysql) >= 50009)
>   {
> DBUG_PRINT("info", ("Dumping routines for database %s", database));
> *dump_routines_for_db*(database);
>   }
> Before this loop:
>   while ((table= getTableName(0)))
>   {
> char *end= strmov(afterdot, table);
> if (include_table((uchar*) hash_key, end - hash_key))
> {
>   *dump_table*(table,database);
>   my_free(order_by);
>   
> }
>   }
>
> We should do this for both dump_all_tables_in_db and dump_selected_tables.
>
> 2. Approach 2:
> As suggested in the jira thread, we can use negative logic to call
> dump_table on tables we intend to skip.
> This appears to be a workaround. In the place where dump_all_tables_in_db
> is called, we'd rather call dump_selected_tables
> with a list of tables that don't use the routines we'll be listing after
> that. This would fix the issue but is not a good approach.
>
> If dump_table doesn't alter the values for opt_routines, the first
> approach should fix the issue.
>
> Thanks,
> Rutuja
>
>
>
> On Fri, Mar 23, 2018 at 10:36 PM, Rutuja Surve 
> wrote:
>
>> Hello,
>> With respect to https://jira.mariadb.org/browse/MDEV-15021 :
>> From what I understood from the thread, since the routines are placed
>> below the tables/views, they cannot be used in views while importing.
>> I went through the problematic_dump.sql file that's attached there. What
>> does the '@' in the following mean:
>> SET @saved_cs_client = @@character_set_client;
>> (I'm not familiar with this syntax)
>> Also, what does the following piece of code do: (what does create definer
>> mean and does this function simply return 1 as there are no other
>> statements after begin? This is a routine defined below test_view)
>> CREATE DEFINER=`root`@`localhost` FUNCTION `test_function`() RETURNS
>> int(11)
>> BEGIN
>>
>> RETURN 1;
>> END ;;
>> DELIMITER ;
>> From my understanding, test_view uses test_function and test_view2 would

Re: [Maria-developers] Observation for CHECK_CONSTRAINTS MDEV 14474

[Vicențiu Ciorbaru]

Hi Anel!

I've reviewed your patch and generally things look good. You do some good
analysis usually and I think we should highlight your work to the broader
developer scene. I've thus CC'ed this email to the maria-developers mailing
list. This gives us a chance to get feedback from outside, particularly as
this feature is of general interest to the community.

Please include the .result files in future versions as it makes reviewing a
lot easier.

Serg: coding style and implementation aside, I'll do the review on that
bit, please have a look at Anel's patch and suggest if you think the
current table header for I_S.check_constraints is good enough for our needs
or if you have more input on this task.

Relevant bit:

+ST_FIELD_INFO check_constraints_fields_info[]=
+{
+  {"CONSTRAINT_CATALOG", FN_REFLEN, MYSQL_TYPE_STRING, 0, 0, 0,
OPEN_FULL_TABLE},
+  {"CONSTRAINT_SCHEMA", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0,
+   OPEN_FULL_TABLE},
+  {"CONSTRAINT_NAME", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0,
+   OPEN_FULL_TABLE},
+  {"TABLE_SCHEMA", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0,
OPEN_FULL_TABLE},
+  {"TABLE_NAME", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0,
OPEN_FULL_TABLE},
+  {"CHECK_CLAUSE", NAME_CHAR_LEN, MYSQL_TYPE_STRING, 0, 0, 0,
+   OPEN_FULL_TABLE},
+  {0, 0, MYSQL_TYPE_STRING, 0, 0, 0, SKIP_OPEN_TABLE}
+};

Vicențiu

On Thu, 29 Mar 2018 at 12:01 Anel Husakovic  wrote:

> Hi Vin,
>
> I'm pleased to announce my new patch prototype for MDEV 14474 where are
> implemented field and table constraints:
>
> https://github.com/an3l/server/commit/f92fd77d2695be52faf280245faa014c44617aa7
>
> According to the standard there are only 4 column to represent.
> CONSTRAINT_CATALOG, CONSTRAINT_SCHEMA, CONSTRAINT_NAME,CHECK_CLAUSE
> But because Serg mentioned that alternative would be to change
> *information_schema.table_constraints* with adding only 1 column (
> *check_clause*) I also used 2 additional columns which are represented
> also in *table_constraints *and they are TABLE_SHEMA , TABLE_NAME.
>
> Function of interest is: get_check_constraints_record
>
> Also I added 2 test cases:
>
> One that works with only 1 table:  MDEV14474-one_table_play.test
> 
> Second that works with more tables:  MDEV14474-I_S_check_constraint.test
> 
>
> Last night I had a problem with field constraints.
> As a reference I used only way how currently all check constraints are
> shown and its obtained via* show create table t;*
>
> Bellow is shown that  field constraints were not visible as well as that
> name used for them is *NULL*, at least this is how *show create table*
> represented them ( there is no way to give a name to a field constraint*
> for example: f int constraint name check(f<0)*). According to my opinion,
> because this constraint belongs to the field, it should be also called as a
> name of a field and not NULL.
> As an argument to my opinion here is mariadb  kb
> https://mariadb.com/kb/en/library/constraint/
> Please take a look at *constraint **name*, which is field constraint and
> error message.
>
> INSERT INTO t2(name, start_date, end_date) VALUES('Io', '2003-12-15', 
> '2014-11-09');ERROR 4022 (23000): CONSTRAINT `name` failed for `test`.`t2`
>
> If the name of constraint = name of the field , than definitely it is not
> consistent with* show create table* output, and *show create table*
> output *needs to be changed*,  , but this is question that I would like
> to discuss with you ?!
>
> When you try to use show create table x
> [image: image.png]
>
>
> I'm waiting on your reply for global temporary tables as well as
> check_constraints and how to proceed further
>
> *Not understand yet a concept*:
>
> In *sql/table.h*
> There is a struct* st_schema_table *which has fill data and process data.
>
> How they work, and how to use that on other database for example PSI.
>
> Regards,
>
> Anel
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] c9ae1234814: MDEV-14592: Custom Aggregates Usage Status Variable

[Vicențiu Ciorbaru]

Hi Varun

This generally looks good, 2 comments inline.

Vicentiu

diff --git a/mysql-test/t/custom_aggregate_functions.test
b/mysql-test/t/custom_aggregate_functions.test
index 20fcc35f39f..a93ec545f01 100644
--- a/mysql-test/t/custom_aggregate_functions.test
+++ b/mysql-test/t/custom_aggregate_functions.test
@@ -16,7 +16,9 @@ create table t1 (sal int(10),id int(10));
 INSERT INTO t1 (sal,id) VALUES (5000,1);
 INSERT INTO t1 (sal,id) VALUES (2000,1);
 INSERT INTO t1 (sal,id) VALUES (1000,1);

# Please test that this works with subqueries, views, ctes and when calling
from
# other stored procedures.


+show global status like "Feature_custom_aggregate_functions";
 select f1(sal) from t1 where id>= 1;
+show global status like "Feature_custom_aggregate_functions";
 select * from t2;
 drop table t2;
 drop function f1;
diff --git a/sql/item_sum.cc b/sql/item_sum.cc
index d4abdfc614f..7c0390e543f 100644
--- a/sql/item_sum.cc
+++ b/sql/item_sum.cc
@@ -313,6 +313,8 @@ bool Item_sum::check_sum_func(THD *thd, Item **ref)
 }
   }
   aggr_sel->set_agg_func_used(true);
+  if (sum_func() == SP_AGGREGATE_FUNC)
+aggr_sel->set_custom_agg_func_used(true);
   update_used_tables();
   thd->lex->in_sum_func= in_sum_func;
   return FALSE;
diff --git a/sql/mysqld.cc b/sql/mysqld.cc
index c7fbf0a594c..75ccda408b1 100644
--- a/sql/mysqld.cc
+++ b/sql/mysqld.cc
@@ -8491,6 +8491,7 @@ SHOW_VAR status_vars[]= {
   {"Executed_triggers",(char*) offsetof(STATUS_VAR,
executed_triggers), SHOW_LONG_STATUS},
   {"Feature_check_constraint", (char*) _check_constraint,
SHOW_LONG },
   {"Feature_delay_key_write",  (char*)
_files_opened_with_delayed_keys, SHOW_LONG },

Please move this up so the variable names are ordered alphabetically.

+  {"Feature_custom_aggregate_functions", (char*) offsetof(STATUS_VAR,
feature_custom_aggregate_functions), SHOW_LONG_STATUS},
   {"Feature_dynamic_columns",  (char*) offsetof(STATUS_VAR,
feature_dynamic_columns), SHOW_LONG_STATUS},
   {"Feature_fulltext", (char*) offsetof(STATUS_VAR,
feature_fulltext), SHOW_LONG_STATUS},
   {"Feature_gis",  (char*) offsetof(STATUS_VAR, feature_gis),
SHOW_LONG_STATUS},

On Sat, 9 Dec 2017 at 08:53 Varun  wrote:

> revision-id: c9ae12348143f59e875a52e2eb285c5ed7d7af56
> (mariadb-10.3.0-285-gc9ae1234814)
> parent(s): c60095a818dce92838940525899a13a05633d148
> author: Varun Gupta
> committer: Varun Gupta
> timestamp: 2017-12-09 12:22:26 +0530
> message:
>
> MDEV-14592: Custom Aggregates Usage Status Variable
>
> ---
>  mysql-test/r/custom_aggregate_functions.result | 6 ++
>  mysql-test/t/custom_aggregate_functions.test   | 2 ++
>  sql/item_sum.cc| 2 ++
>  sql/mysqld.cc  | 1 +
>  sql/sql_lex.cc | 2 ++
>  sql/sql_lex.h  | 3 +++
>  sql/sql_select.cc  | 2 ++
>  7 files changed, 18 insertions(+)
>
> diff --git a/mysql-test/r/custom_aggregate_functions.result
> b/mysql-test/r/custom_aggregate_functions.result
> index ca8612ba652..f70675fa4db 100644
> --- a/mysql-test/r/custom_aggregate_functions.result
> +++ b/mysql-test/r/custom_aggregate_functions.result
> @@ -11,6 +11,9 @@ create table t1 (sal int(10),id int(10));
>  INSERT INTO t1 (sal,id) VALUES (5000,1);
>  INSERT INTO t1 (sal,id) VALUES (2000,1);
>  INSERT INTO t1 (sal,id) VALUES (1000,1);
> +show global status like "Feature_custom_aggregate_functions";
> +Variable_name  Value
> +Feature_custom_aggregate_functions 0
>  select f1(sal) from t1 where id>= 1;
>  f1(sal)
>  0
> @@ -18,6 +21,9 @@ Warnings:
>  Note   4093At line 5 in test.f1
>  Note   4093At line 5 in test.f1
>  Note   4093At line 5 in test.f1
> +show global status like "Feature_custom_aggregate_functions";
> +Variable_name  Value
> +Feature_custom_aggregate_functions 1
>  select * from t2;
>  sal
>  5000
> diff --git a/mysql-test/t/custom_aggregate_functions.test
> b/mysql-test/t/custom_aggregate_functions.test
> index 20fcc35f39f..a93ec545f01 100644
> --- a/mysql-test/t/custom_aggregate_functions.test
> +++ b/mysql-test/t/custom_aggregate_functions.test
> @@ -16,7 +16,9 @@ create table t1 (sal int(10),id int(10));
>  INSERT INTO t1 (sal,id) VALUES (5000,1);
>  INSERT INTO t1 (sal,id) VALUES (2000,1);
>  INSERT INTO t1 (sal,id) VALUES (1000,1);
> +show global status like "Feature_custom_aggregate_functions";
>  select f1(sal) from t1 where id>= 1;
> +show global status like "Feature_custom_aggregate_functions";
>  select * from t2;
>  drop table t2;
>  drop function f1;
> diff --git a/sql/item_sum.cc b/sql/item_sum.cc
> index d4abdfc614f..7c0390e543f 100644
> --- a/sql/item_sum.cc
> +++ b/sql/item_sum.cc
> @@ -313,6 +313,8 @@ bool Item_sum::check_sum_func(THD *thd, Item **ref)
>  }
>}
>aggr_sel->set_agg_func_used(true);
> +  if (sum_func() == SP_AGGREGATE_FUNC)
> +aggr_sel->set_custom_agg_func_used(true);
>

Re: [Maria-developers] GSoC 2018 - Histograms with equal-width bins

[Vicențiu Ciorbaru]

Hi Teodor,

In order to get started with the project you need to see how we currently
gather statistics from tables. You should first read up on:

https://mariadb.com/kb/en/library/engine-independent-table-statistics/

Note we are interested in *engine independent table statistics*, not engine
specific table statistics. InnoDB does perform statistics collection when
you call ANALYZE TABLE, however you also have to specify PERSISTENT FOR ...
to have the server collect the statistics separately.

I assume you have compiled the server already, I recommend you create a
testcase as outlined in this page:

https://mariadb.org/get-involved/getting-started-for-developers/writing-good-test-cases-mariadb-server/

The page is still a draft but should have sufficient information to get you
started. Afterwards start the test case under debugger. You can do that
with ./mtr --gdb 

Put a breakpoint in collect_statistics_for_table. You will find the
function in sql/sql_statistics.cc

When the server executes an ANALYZE TABLE t PERSISTENT FOR ALL, that
function should be called. See if you can figure out how we gather data and
store equal-height histograms. That should give you an idea how one would
proceed to create similar histograms as equal-width ones instead.

Feel free to reach out on IRC #maria on freenode or via this mailing list
if you have any more questions.

Vicentiu

On Tue, 20 Mar 2018 at 21:10 Teodor Niculescu 
wrote:

> Greetings,
>
> My name is Teodor Niculescu and I am a second year undergraduate student
> at the The Faculty of Automatic Control and Computers, University
> Politehnica of Bucharest, Romania. I am interested in contributing to
> MariaDB as part of this years Google Summer of Code.
>
> I am interested in the project: MDEV-12313
>  Histograms with equal-width
> bins.
>
> I would like to ask you about a good place to start reading the codebase, in 
> order to help me get an idea about what I need to do for this project.
>
>
> I look forward to hearing from you.
>
> Sincerely,
> Teodor.
>
>
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Review Custom Aggregate Functions

[Vicențiu Ciorbaru]

Hi Varun!

Here's a final review for Custom Aggregate Functions

* You haven't run the full test suite, there are a few failing tests.
Please update those.
  rpl.rpl_sp funcs_1.is_columns_mysql funcs_1.storedproc
* Push your code to a BB branch and rebase it on top of current 10.3.
* Fix commits messages while rebasing. Use the reword flag to
clean those up. The rule is this:
1. Have one line that doesn't exceed 80 Characters as a Title.
2. Leave one empty line
3. Add the rest of the commit message. Don't exceed 80 Characters per line.

> diff --git a/sql/sp_head.h b/sql/sp_head.h
> index bb516187a57..d447807bb3b 100644
> --- a/sql/sp_head.h
> +++ b/sql/sp_head.h
> @@ -334,6 +334,11 @@ class sp_head :private Query_arena,
>execute_function(THD *thd, Item **args, uint argcount, Field
*return_fld);
>
>bool
Rename this to execute_function please and delete the now dead-code from the
current execute_function. Update comments for this function appropriately.
Please run all tests again after this and make sure you update results for
all the failing ones.
> +  execute_aggregate_function(THD *thd, Item **args, uint argcount,
> + Field *return_fld, sp_rcontext **nctx,
> + MEM_ROOT *caller_mem_root);
> +
> +  bool
>execute_procedure(THD *thd, List *args);
>
>static void
> @@ -813,6 +818,9 @@ class sp_head :private Query_arena,
>bool
>execute(THD *thd, bool merge_da_on_success);
>
This is no longer used please remove!
> +  bool
> +  execute_agg(THD *thd, bool merge_da_on_success);
> +
>/**
>  Perform a forward flow analysis in the generated code.
>  Mark reachable instructions, for the optimizer.

> +++ b/mysql-test/t/sp_agg.test
A lot of empty lines at the end of this test. Please delete those.

Ping me once you have a green buildbot before pushing.

Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Review of Percentile Window Functions

[Vicențiu Ciorbaru]

Hi Varun,

Here's the final review for Percentile Functions. Just 2 minor coding style
comments. Rebase on top of current 10.3 and push to a bb-* branch to double
check nothing breaks. Make sure that no merge commit happens.

Regards,
Vicențiu

> diff --git a/sql/item_windowfunc.h b/sql/item_windowfunc.h
> index 77cbd556e60..589f7cf5d96 100644
> --- a/sql/item_windowfunc.h
> +++ b/sql/item_windowfunc.h
> @@ -678,6 +687,277 @@ class Item_sum_ntile : public
Item_sum_window_with_row_count
>ulong current_row_count_;
>  };
>
> +class Item_sum_percentile_disc : public Item_sum_cume_dist,
> + public Type_handler_hybrid_field_type
> +{
> +public:
> +  Item_sum_percentile_disc(THD *thd, Item* arg) :
Item_sum_cume_dist(thd, arg),
> +
Type_handler_hybrid_field_type(_handler_longlong),
> +   value(NULL), val_calculated(FALSE),
first_call(TRUE),
> +   prev_value(0), order_item(NULL){}
> +
> +  double val_real()
> +  {
> +if (get_row_count() == 0 || get_arg(0)->is_null())
> +{
> +  null_value= true;
> +  return 0;
> +}
> +null_value= false;
> +return value->val_real();
> +  }
> +
> +  longlong val_int()
> +  {
> +if (get_row_count() == 0 || get_arg(0)->is_null())
> +{
> +  null_value= true;
> +  return 0;
> +}
> +null_value= false;
> +return value->val_int();
> +  }
> +
> +  my_decimal* val_decimal(my_decimal* dec)
> +  {
> +if (get_row_count() == 0 || get_arg(0)->is_null())
> +{
> +  null_value= true;
> +  return 0;
> +}
> +null_value= false;
> +return value->val_decimal(dec);
> +  }
> +
> +  String* val_str(String *str)
> +  {
> +if (get_row_count() == 0 || get_arg(0)->is_null())
> +{
> +  null_value= true;
> +  return 0;
> +}
> +null_value= false;
> +return value->val_str(str);
> +  }
> +
> +  bool add()
> +  {
> +Item *arg= get_arg(0);
> +if (arg->is_null())
> +  return false;
> +
> +if (first_call)
> +{
> +  prev_value= arg->val_real();
> +  if (prev_value > 1 || prev_value < 0)
> +  {
> +my_error(ER_ARGUMENT_OUT_OF_RANGE, MYF(0), func_name());
> +return true;
> +  }
> +  first_call= false;
> +}
> +
> +double arg_val= arg->val_real();
> +
> +if (prev_value != arg_val)
> +{
> +  my_error(ER_ARGUMENT_NOT_CONSTANT, MYF(0), func_name());
> +  return true;
> +}
> +
> +if (val_calculated)
> +  return false;
> +
> +value->store(order_item);
> +value->cache_value();
> +if (value->null_value)
> +   return false;
> +
> +Item_sum_cume_dist::add();
> +double val= Item_sum_cume_dist::val_real();
> +
> +if (val >= prev_value && !val_calculated)
> +  val_calculated= true;
> +return false;
> +  }
> +
> +  enum Sumfunctype sum_func() const
> +  {
> +return PERCENTILE_DISC_FUNC;
> +  }
> +
> +  void clear()
> +  {
> +val_calculated= false;
> +first_call= true;
> +value->clear();
> +Item_sum_cume_dist::clear();
> +  }
> +
> +  const char*func_name() const
> +  {
> +return "percentile_disc";
> +  }
> +
> +  void update_field() {}
> +  void set_type_handler(Window_spec *window_spec);
> +  const Type_handler *type_handler() const
> +  {return Type_handler_hybrid_field_type::type_handler();}
> +
> +  void fix_length_and_dec()
> +  {
> +decimals = 10;  // TODO-cvicentiu find out how many decimals the
standard
> +// requires.
> +  }
> +
> +  Item *get_copy(THD *thd, MEM_ROOT *mem_root)
> +  { return get_item_copy(thd, mem_root, this);
}
> +  void setup_window_func(THD *thd, Window_spec *window_spec);
> +  void setup_hybrid(THD *thd, Item *item);
> +  bool fix_fields(THD *thd, Item **ref);
> +
> +private:
> +  Item_cache *value;
> +  bool val_calculated;
> +  bool first_call;
> +  double prev_value;
> +  Item *order_item;
> +};
> +
> +class Item_sum_percentile_cont : public Item_sum_cume_dist,
> + public Type_handler_hybrid_field_type
> +{
> +public:
> +  Item_sum_percentile_cont(THD *thd, Item* arg) :
Item_sum_cume_dist(thd, arg),
> +
Type_handler_hybrid_field_type(_handler_double),
> +   floor_value(NULL), ceil_value(NULL),
first_call(TRUE),prev_value(0),
> +   ceil_val_calculated(FALSE),
floor_val_calculated(FALSE), order_item(NULL){}
> +
> +  double val_real()
> +  {
> +if (get_row_count() == 0 || get_arg(0)->is_null())
> +{
> +  null_value= true;
> +  return 0;
> +}
> +null_value= false;
> +double val= 1 + prev_value * (get_row_count()-1);
> +
> +/*
> +  Applying the formula to get the value
> +  If (CRN = FRN = RN) then the result is (value of expression from
row at RN)
> +  Otherwise the result is
> +  (CRN - RN) * (value of expression for row at FRN) +
> +  (RN - FRN) * (value of expression for row at CRN)
> +*/
> +
> +

Re: [Maria-developers] 46b93fc: MDEV-13676: Field "create Procedure" is NULL, even if the the user has role which is the definer. (SHOW CREATE PROCEDURE)

[Vicențiu Ciorbaru]

On Mon, 9 Oct 2017 at 20:45 Sergei Golubchik <s...@mariadb.org> wrote:

> Hi, Vicentiu!
>
> Looks good! A couple of comments, see below.
> Ok to push after fixing, the second review is not needed.
>
> On Oct 09, Vicentiu Ciorbaru wrote:
> > revision-id: 46b93fc2a2fa198d721813d9b93bbe09d2e36eb3
> (mariadb-10.0.32-50-g46b93fc)
> > parent(s): dbeffabc83ed01112e09d7e782d44f044cfcb691
> > author: Vicențiu Ciorbaru
> > committer: Vicențiu Ciorbaru
> > timestamp: 2017-10-09 13:32:40 +0300
> > message:
> >
> > MDEV-13676: Field "create Procedure" is NULL, even if the the user has
> role which is the definer. (SHOW CREATE PROCEDURE)
> >
> > During show create procedure we ommited to check the current role, if it
> > is the actual definer of the procedure. In addition, we should support
> > indirectly granted roles to the current role. Implemented a recursive
> > lookup to search the tree of grants if the rolename is present.
>
> We both had to check the SQL standard to see what the behavior should
> be. Please add here (in the commit comment) a reference to the exact
> part of the standard that specifies this behavior. Like
>
>   SQL Standard 2016, Part ... Section ... View I_S.ROUTINES selects
>   ROUTINE_BODY and its WHERE clause says that the GRANTEE must be either
>   PUBLIC, or CURRENT_USER or in the ENABLED_ROLES.
>
> > diff --git a/sql/sp_head.cc b/sql/sp_head.cc
> > index ea9e1c1..3dd1a65 100644
> > --- a/sql/sp_head.cc
> > +++ b/sql/sp_head.cc
> > @@ -2588,10 +2588,18 @@ bool check_show_routine_access(THD *thd, sp_head
> *sp, bool *full_access)
> >*full_access= ((!check_table_access(thd, SELECT_ACL, , FALSE,
> >   1, TRUE) &&
> >(tables.grant.privilege & SELECT_ACL) != 0) ||
> > + /* Check if user owns the routine. */
> >   (!strcmp(sp->m_definer_user.str,
> >thd->security_ctx->priv_user) &&
> >!strcmp(sp->m_definer_host.str,
> > -  thd->security_ctx->priv_host)));
> > +  thd->security_ctx->priv_host)) ||
> > + /* Check if current role or any of the sub-granted
> roles
> > +own the routine. */
> > + (sp->m_definer_host.length == 0 &&
> > +  (!strcmp(sp->m_definer_user.str,
> > +   thd->security_ctx->priv_role) ||
> > +   check_role_is_granted(thd->security_ctx->priv_role,
> NULL,
> > + sp->m_definer_user.str;
>
> Perhaps you could simplify the condition above a little bit,
> by replacing it with
>
>  /* Check if current role or any of the sub-granted roles
> own the routine. */
>   (sp->m_definer_host.length == 0 &&
>check_role_is_granted(thd->security_ctx->priv_user,
>  thd->security_ctx->priv_host,
>  sp->m_definer_user.str)));
>
> I fixed the other change, but the logic you propose here is slightly
different. Your version will check all APPLICABLE_ROLES, not only
ENABLED_ROLES. We need to look at the subgraph of thd->priv_role only, not
those of priv_user@priv_host.

I've added an extra test case for this exact behaviour now too.

Regards,
> Sergei
> Chief Architect MariaDB
> and secur...@mariadb.org
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] dccebdf862d: Rdb_io_watchdog doesn't support osx so disabling it

[Vicențiu Ciorbaru]

Hi Varun!

Please change the CMakeLists.txt to be like so:

diff --git a/storage/rocksdb/CMakeLists.txt b/storage/rocksdb/CMakeLists.txt
index 5db6d88..6db8211 100644
--- a/storage/rocksdb/CMakeLists.txt
+++ b/storage/rocksdb/CMakeLists.txt
@@ -146,7 +146,10 @@ ADD_CONVENIENCE_LIBRARY(rocksdb_aux_lib
 ADD_DEPENDENCIES(rocksdb_aux_lib GenError)

 # MARIAROCKS-TODO: how to properly depend on -lrt ?
-TARGET_LINK_LIBRARIES(rocksdb_aux_lib rocksdblib ${ZLIB_LIBRARY} -lrt)
+TARGET_LINK_LIBRARIES(rocksdb_aux_lib rocksdblib ${ZLIB_LIBRARY})
+if (UNIX AND NOT APPLE)
+  TARGET_LINK_LIBRARIES(rocksdb_aux_lib -lrt)
+endif()

 TARGET_LINK_LIBRARIES(rocksdb rocksdb_aux_lib)

@@ -182,7 +185,9 @@ ENDIF()
 #  ADD_SUBDIRECTORY(unittest)
 #ENDIF()

-SET(rocksdb_static_libs ${rocksdb_static_libs} "-lrt")
+if (UNIX AND NOT APPLE)
+  SET(rocksdb_static_libs ${rocksdb_static_libs} "-lrt")
+endif()

 ADD_LIBRARY(rocksdb_tools STATIC
rocksdb/tools/ldb_tool.cc

Also, change how you define the __APPLE__ #ifdef like so:

- #ifndef _WIN32
+ #if not defined(_WIN32) && not defined(__APPLE__)

This way, you don't have to add an extra #endif. The point of #ifndef is to
avoid typing, but it doesn't work well with multiple conditions. We can
make use #if then.

Vicențiu


On Wed, 30 Aug 2017 at 15:37 Varun  wrote:

> revision-id: dccebdf862d66e65528e18562c4503f40b695eca
> (mariadb-10.3.0-107-gdccebdf862d)
> parent(s): e1051590b61b45532284071ac78c50e7a2f24c63
> author: Varun Gupta
> committer: Varun Gupta
> timestamp: 2017-08-30 18:05:52 +0530
> message:
>
> Rdb_io_watchdog doesn't support osx so disabling it
>
> ---
>  storage/rocksdb/CMakeLists.txt |  4 
>  storage/rocksdb/ha_rocksdb.cc  | 14 +-
>  storage/rocksdb/rdb_io_watchdog.cc |  2 ++
>  storage/rocksdb/rdb_io_watchdog.h  |  6 --
>  4 files changed, 23 insertions(+), 3 deletions(-)
>
> diff --git a/storage/rocksdb/CMakeLists.txt
> b/storage/rocksdb/CMakeLists.txt
> index 5db6d888bb6..435fe784d6c 100644
> --- a/storage/rocksdb/CMakeLists.txt
> +++ b/storage/rocksdb/CMakeLists.txt
> @@ -146,7 +146,11 @@ ADD_CONVENIENCE_LIBRARY(rocksdb_aux_lib
>  ADD_DEPENDENCIES(rocksdb_aux_lib GenError)
>
>  # MARIAROCKS-TODO: how to properly depend on -lrt ?
> +if (APPLE)
> +TARGET_LINK_LIBRARIES(rocksdb_aux_lib rocksdblib ${ZLIB_LIBRARY})
> +else()
>  TARGET_LINK_LIBRARIES(rocksdb_aux_lib rocksdblib ${ZLIB_LIBRARY} -lrt)
> +endif()
>
>  TARGET_LINK_LIBRARIES(rocksdb rocksdb_aux_lib)
>
> diff --git a/storage/rocksdb/ha_rocksdb.cc b/storage/rocksdb/ha_rocksdb.cc
> index e4d9ce02f10..4ffe6caf70b 100644
> --- a/storage/rocksdb/ha_rocksdb.cc
> +++ b/storage/rocksdb/ha_rocksdb.cc
> @@ -173,9 +173,11 @@ Rdb_ddl_manager ddl_manager;
>  const char *m_mysql_gtid;
>  Rdb_binlog_manager binlog_manager;
>
> -#ifndef _WIN32
> +#ifndef _WIN32
> +#ifndef __APPLE__
>  Rdb_io_watchdog *io_watchdog = nullptr;
>  #endif
> +#endif
>  /**
>MyRocks background thread control
>N.B. This is besides RocksDB's own background threads
> @@ -555,7 +557,9 @@ static void rocksdb_set_io_write_timeout(
>DBUG_ASSERT(save != nullptr);
>DBUG_ASSERT(rdb != nullptr);
>  #ifndef _WIN32
> +#ifndef __APPLE__
>DBUG_ASSERT(io_watchdog != nullptr);
> +#endif
>  #endif
>
>RDB_MUTEX_LOCK_CHECK(rdb_sysvars_mutex);
> @@ -564,7 +568,9 @@ static void rocksdb_set_io_write_timeout(
>
>rocksdb_io_write_timeout_secs = new_val;
>  #ifndef _WIN32
> +#ifndef __APPLE__
>io_watchdog->reset_timeout(rocksdb_io_write_timeout_secs);
> +#endif
>  #endif
>RDB_MUTEX_UNLOCK_CHECK(rdb_sysvars_mutex);
>  }
> @@ -3746,7 +3752,9 @@ static int rocksdb_init_func(void *const p) {
>  */
>  if (status.IsIOError()
>  #ifndef _WIN32
> +#ifndef __APPLE__
>&& errno == ENOENT
> +#endif
>  #endif
>) {
>sql_print_information("RocksDB: Got ENOENT when listing column
> families");
> @@ -3985,8 +3993,10 @@ static int rocksdb_init_func(void *const p) {
>}
>
>  #ifndef _WIN32
> +#ifndef __APPLE__
>io_watchdog = new Rdb_io_watchdog(directories);
>io_watchdog->reset_timeout(rocksdb_io_write_timeout_secs);
> +#endif
>  #endif
>
>// NO_LINT_DEBUG
> @@ -4077,9 +4087,11 @@ static int rocksdb_done_func(void *const p) {
>commit_latency_stats = nullptr;
>
>  #ifndef _WIN32
> +#ifndef __APPLE__
>delete io_watchdog;
>io_watchdog = nullptr;
>  #endif
> +#endif
>
>  // Disown the cache data since we're shutting down.
>  // This results in memory leaks but it improved the shutdown time.
> diff --git a/storage/rocksdb/rdb_io_watchdog.cc
> b/storage/rocksdb/rdb_io_watchdog.cc
> index a599ba58aec..a9e79283382 100644
> --- a/storage/rocksdb/rdb_io_watchdog.cc
> +++ b/storage/rocksdb/rdb_io_watchdog.cc
> @@ -23,6 +23,7 @@
>
>  /* Rdb_io_watchdog doesn't work on Windows [yet] */
>  #ifndef _WIN32
> +#ifndef __APPLE__
>
>  namespace myrocks {
>
> @@ -236,4 +237,5 @@ int Rdb_io_watchdog::reset_timeout(const uint32_t
> _timeout) {
>  }  // namespace myrocks

[Maria-developers] Review: MDEV-12985: Support percentile and media window functions

[Vicențiu Ciorbaru]

Hi Varun!


Here is the in-detail review for the percentile functions. Please add a
grammar definition for median too and test it. All in all, not many
changes, but please address my comments. If you disagree with some of them,
let's discuss them.

> diff --git a/mysql-test/r/win_percentile_cont.result
b/mysql-test/r/win_percentile_cont.result
> new file mode 100644
> index 000..61f70892887
> --- /dev/null
> +++ b/mysql-test/r/win_percentile_cont.result
> diff --git a/mysql-test/t/win_percentile_cont.test
b/mysql-test/t/win_percentile_cont.test
> new file mode 100644
> index 000..75fde963b2a
> --- /dev/null
> +++ b/mysql-test/t/win_percentile_cont.test
> @@ -0,0 +1,55 @@
Please put more tests here, as we discussed.
Also, to make test cases easier to follow, please add cume_dist as a
function
in the select to all these use cases.
> +CREATE TABLE student (name CHAR(10), test double, score DECIMAL(19,4));
> +INSERT INTO student VALUES
> +('Chun', 0, 3), ('Chun', 0, 7),
> +('Kaolin', 0.5, 3), ('Kaolin', 0.6, 7),
> +('Kaolin', 0.5, 4),
> +('Tatiana', 0.8, 4), ('Tata', 0.8, 4);
> +
Make this --echo # No partition clause
> +#no partition clause
> +select name, percentile_disc(0.5) within group(order by score) over ()
from student;
> +select name, percentile_cont(0.5) within group(order by score) over ()
from student;
> +
Same as before, use --echo # so we can see it in the result set too.
> +# argument set to null
> +select name, percentile_cont(null) within group(order by score) over
(partition by name) from student;
> +select name, percentile_disc(null) within group(order by score) over
(partition by name) from student;
> +
Use --echo #
> +# complete query with partition column
> +select name, percentile_cont(0.5) within group(order by score) over
(partition by name) as c from student;
> +select name, percentile_disc(0.5) within group(order by score) over
(partition by name) as c from student;
> +
Use --echo #
> +#subqueries having percentile functions
> +
> +select * from ( select name , percentile_cont(0.5) within group ( order
by score) over (partition by name ) from student ) as t;
> +select * from ( select name , percentile_disc(0.5) within group ( order
by score) over (partition by name ) from student ) as t;
> +select name from student a where (select percentile_disc(0.5) within
group (order by score) over (partition by name) from student b limit 1) >=
0.5;
> +
USe --echo
> +# WITH STORED PROCEDURES
> +
> +
USe --echo
> +#DISALLOWED FIELDS IN ORDER BY CLAUSE
> +--error ER_WRONG_TYPE_FOR_PERCENTILE_CONT
> +select score, percentile_cont(0.5) within group(order by name) over
(partition by score) from student;
> +select score, percentile_disc(0.5) within group(order by name) over
(partition by score) from student;
>
USe --echo
> +
> +# error with 2 order by elements
> +
> +--error ER_NOT_SINGLE_ELEMENT_ORDER_LIST
> +select percentile_disc(0.5) within group(order by score,test) over
(partition by name) from student;
> +--error ER_NOT_SINGLE_ELEMENT_ORDER_LIST
> +select percentile_cont(0.5) within group(order by score,test) over
(partition by name) from student;
> +
Use --echo and fix that comment. We thought of a way for this right?
> +#parameter value should be in the range of 0 to 1( NEED TO THINK A WAY
FOR THIS)
> +--error ER_ARGUMENT_OUT_OF_RANGE
> +select percentile_disc(1.5) within group(order by score) over (partition
by name) from student;
> +--error ER_ARGUMENT_OUT_OF_RANGE
> +select percentile_cont(1.5) within group(order by score) over (partition
by name) from student;
> +
> +--error ER_ARGUMENT_NOT_CONSTANT
> +select name,percentile_cont(test) within group(order by score) over
(partition by name) from student;
> +--error ER_ARGUMENT_NOT_CONSTANT
> +select name, percentile_disc(test) within group(order by score) over
(partition by name) from student;
> +
This is commented out. Not good, fix it please. And use --echo #
> +#CHECK TYPE OF THE ARGUMENT, SHOULD BE ONLY NUMERICAL
> +#select name, percentile_cont(name) within group(order by score) over
(partition by name) from student;
> +
>
> +drop table student;
> diff --git a/sql/item.h b/sql/item.h
> index 76ce4aa935f..108b0838dfc 100644
> --- a/sql/item.h
> +++ b/sql/item.h
> @@ -5245,6 +5246,7 @@ class Cached_item_int :public Cached_item_item
> Cached_item_int(Item *item_par) :Cached_item_item(item_par),value(0) {}
> bool cmp(void);
> int cmp_read_only();
Delete this, it's not used.
> + longlong get_value(){ return value;}
> };
>
>
> @@ -5255,6 +5257,7 @@ class Cached_item_decimal :public Cached_item_item
> Cached_item_decimal(Item *item_par);
> bool cmp(void);
> int cmp_read_only();
Delete this, it's not used.
> + my_decimal *get_value(){ return };
> };
>
> class Cached_item_field :public Cached_item
> diff --git a/sql/item_windowfunc.cc b/sql/item_windowfunc.cc
> index 6ab903a81bb..916f4113682 100644
> --- a/sql/item_windowfunc.cc
> +++ b/sql/item_windowfunc.cc
> diff --git a/sql/item_windowfunc.h b/sql/item_windowfunc.h
> index 

[Maria-developers] Review of PERCENTILE_CONT && PERCENTILE_DISC

[Vicențiu Ciorbaru]

Hi Varun,

Here's the review for your patch.

> diff --git a/mysql-test/t/percentile.test b/mysql-test/t/percentile.test
> new file mode 100644
> index 000..0958fc05e7d
> --- /dev/null
> +++ b/mysql-test/t/percentile.test
> @@ -0,0 +1,41 @@
Whenever you are adding a test case, please add the result file too.
It makes things easier to review.
> +CREATE TABLE student (name CHAR(10), test double, score TINYINT);
> +
> +INSERT INTO student VALUES
> +('Chun', 0, null), ('Chun', 0, 4),
> +('Esben', 1, null), ('Esben', 1, null),
> +('Kaolin', 0.5, 56), ('Kaolin', 0.5, 88),
> +('Tatiana', 0.8, 2), ('Tatiana', 0.8, 1);
> +
> +
> +
> +select name, percentile_disc(0.6)  within group(order by score) over
(partition by name) from student;
> +select name, percentile_disc(test)  within group(order by score) over
(partition by name) from student;
> +select name, percentile_disc(0.4)  within group(order by score) over
(partition by name) from student;
> +
> +
> +#select name, percentile_cont(null)  within group(order by score) over
(partition by name) from student;
> +#select name, cume_dist()  over (partition by name order by score) from
student;
> +
> +
> +#normal parsing
> +#select percentile_cont(0.5) within group(order by score)  over w1  from
student
> +#window w1 AS (partition by name);
> +
> +# no partition clause
> +#select percentile_cont(0.5) within group(order by score)  over ()  from
student;
> +
> +
> +# only one sort allowed
> +#select percentile_cont(0.5) within group(order by score) over
(partition by name);
> +
> +#parameter value should be in the range of 0 to 1
> +#select percentile_cont(1.5) within group(order by score) over
(partition by name);
> +
> +
> +#
> +#select rank() over (partition by name order by score  ROWS BETWEEN
UNBOUNDED PRECEDING AND CURRENT ROW) from student;
> +
> +
> +
> +drop table student;
> +
> diff --git a/sql/item.h b/sql/item.h
> index 76ce4aa935f..3df4ec4f358 100644
> --- a/sql/item.h
> +++ b/sql/item.h
> @@ -5215,6 +5217,14 @@ class Cached_item_item : public Cached_item
>  cmp();
>  item= save;
>}
This is not used in the patch. Is there a need for it any more?
> +  Item* get_item()
> +  {
> +return item;
> +  }
Add an empty line here here.
> +  void clear()
> +  {
> +null_value= false;
> +  }
>  };
>
>  class Cached_item_str :public Cached_item_item
> @@ -5236,6 +5250,12 @@ class Cached_item_real :public Cached_item_item
>Cached_item_real(Item *item_par)
:Cached_item_item(item_par),value(0.0) {}
>bool cmp(void);
>int  cmp_read_only();
Space after ). Mark this function as const
> +  double get_value(){ return value;}
I don't quite get what this clear is supposed to do. I'd expect clear to set
the null_value to true, not to false. Same for other functions in the diff.
Add a comment explaining this.
> +  void clear()
> +  {
Space after =
> +value=0.0;
> +null_value= false;
> +  }
>  };
>
>  class Cached_item_int :public Cached_item_item
> @@ -5245,6 +5265,12 @@ class Cached_item_int :public Cached_item_item
>Cached_item_int(Item *item_par) :Cached_item_item(item_par),value(0) {}
>bool cmp(void);
>int  cmp_read_only();
Space after ). Mark this function as const.
> +  longlong get_value(){ return value;}
> +  void clear()
> +  {
Space after =
> +value=0.0;
> +null_value= false;
> +  }
>  };
>
>
> @@ -5255,6 +5281,12 @@ class Cached_item_decimal :public Cached_item_item
>Cached_item_decimal(Item *item_par);
>bool cmp(void);
>int  cmp_read_only();
Space after ) Mark this function as const.
> +  my_decimal *get_value(){ return };
> +  void clear()
> +  {
> +null_value= false;
> +my_decimal_set_zero();
> +  }
>  };
>
>  class Cached_item_field :public Cached_item
> diff --git a/sql/item_windowfunc.cc b/sql/item_windowfunc.cc
> index 6ab903a81bb..b711642c52c 100644
> --- a/sql/item_windowfunc.cc
> +++ b/sql/item_windowfunc.cc
> @@ -93,6 +93,17 @@ Item_window_func::fix_fields(THD *thd, Item **ref)
>  my_error(ER_NO_ORDER_LIST_IN_WINDOW_SPEC, MYF(0),
window_func()->func_name());
>  return true;
>}
> +
> +  if (only_single_element_order_list())
> +  {
Mark such comments as TODO, like so:
TODO(varunt): 
> +// need to change the error, the error should say that we have more
than one element in the order list
> +if (window_spec->order_list->elements != 1)
> +{
> +  my_error(ER_NO_ORDER_LIST_IN_WINDOW_SPEC, MYF(0),
window_func()->func_name());
> +  return true;
> +}
> +  }
> +
>/*
>  TODO: why the last parameter is 'ref' in this call? What if
window_func
>  decides to substitute itself for something else and does *ref= ?
> @@ -179,6 +190,16 @@ void Item_sum_dense_rank::setup_window_func(THD
*thd, Window_spec *window_spec)
>clear();
>  }
>
>
I don't see any added value for this extra setup_percentile_func, when it's
only
I guess you can just put the code in directly.
> +void Item_sum_percentile_disc::setup_window_func(THD *thd, Window_spec

[Maria-developers] Fwd: ecbad0a07a8: MDEV-13189: Window functions crash when using INTERVAL

[Vicențiu Ciorbaru]

Hi Sergey,

Can you please review this change that makes window functions work with
INTERVAL function?

Thanks,
Vicentiu

-- Forwarded message -
From: vicentiu <vicen...@mariadb.org>
Date: Wed, 28 Jun 2017 at 21:08
Subject: ecbad0a07a8: MDEV-13189: Window functions crash when using INTERVAL
To: <comm...@mariadb.org>


revision-id: ecbad0a07a848c1f9f474c4c749c87b6bfe7986c
(mariadb-10.2.6-74-gecbad0a07a8)
parent(s): 31ba0fa48d27715e82258b1e74401093e0ee17a2
author: Vicențiu Ciorbaru
committer: Vicențiu Ciorbaru
timestamp: 2017-06-28 21:07:55 +0300
message:

MDEV-13189: Window functions crash when using INTERVAL

Interval function makes use of Item_row. Item_row did not correctly mark
with_window_func flag according to its arguments. Fix it by making
Item_row aware of this flag.

---
 mysql-test/r/win.result | 14 ++
 mysql-test/t/win.test   |  9 +
 sql/item_row.cc |  1 +
 3 files changed, 24 insertions(+)

diff --git a/mysql-test/r/win.result b/mysql-test/r/win.result
index 952bd82ac7f..7f8777ed491 100644
--- a/mysql-test/r/win.result
+++ b/mysql-test/r/win.result
@@ -3127,3 +3127,17 @@ NULL 1   0
 1  0   1
 2  0   1
 drop table t1;
+#
+# MDEV-13189: Window functions crash when using INTERVAL function
+#
+create table t1(i int);
+insert into t1 values (1),(2),(10),(20),(30);
+select sum(i) over (order by i), interval(sum(i) over (order by i), 10, 20)
+from t1;
+sum(i) over (order by i)   interval(sum(i) over (order by i), 10, 20)
+1  0
+3  0
+13 1
+33 2
+63 2
+drop table t1;
diff --git a/mysql-test/t/win.test b/mysql-test/t/win.test
index 45653b18682..aff717e3151 100644
--- a/mysql-test/t/win.test
+++ b/mysql-test/t/win.test
@@ -1914,3 +1914,12 @@ select max(i) over (order by i),
max(i) over (order by i) is not null
 from t1;
 drop table t1;
+
+--echo #
+--echo # MDEV-13189: Window functions crash when using INTERVAL function
+--echo #
+create table t1(i int);
+insert into t1 values (1),(2),(10),(20),(30);
+select sum(i) over (order by i), interval(sum(i) over (order by i), 10, 20)
+from t1;
+drop table t1;
diff --git a/sql/item_row.cc b/sql/item_row.cc
index ddbb0736d54..fc484f560ee 100644
--- a/sql/item_row.cc
+++ b/sql/item_row.cc
@@ -63,6 +63,7 @@ bool Item_row::fix_fields(THD *thd, Item **ref)
 }
 maybe_null|= item->maybe_null;
 with_sum_func= with_sum_func || item->with_sum_func;
+with_window_func = with_window_func || item->with_window_func;
 with_field= with_field || item->with_field;
 with_subselect|= item->with_subselect;
   }
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Cannot push branch to github

[Vicențiu Ciorbaru]

Hi Valentin!

I've just replied to your pull request. I can guarantee a time for when it
will be reviewed, but it will happen. :) Thanks again for the contribution!

Regards,
Vicențiu

On Fri, 26 May 2017 at 15:54 Valentin Rakush <valentin.rak...@gmail.com>
wrote:

> Hi Vicențiu, hi Sergei,
>
> thank you for explanations, yesterday I have submitted pull request
> https://github.com/MariaDB/server/pull/398 It does not have assignee or
> reviewers and I am not sure if somebody will be able to see it. Please have
> a look.
>
> Thank you,
> Valentin
>
>
> On Thu, May 25, 2017 at 12:44 PM, Vicențiu Ciorbaru <vicen...@mariadb.org>
> wrote:
>
>> Hi Valentin!
>>
>> The correct way to do a pull request is to create your own fork of the
>> server repository. You don't need any special permissions for that.
>> Afterwards, push your changes to your fork. Finally, click on New pull
>> request button in *your* repository. Select the appropriate branch for the
>> base fork and the appropriate branch from your head fork. Submit the pull
>> request and we'll take it from there.
>>
>> Regards,
>> Vicențiu
>>
>> On Thu, 25 May 2017 at 12:37 Valentin Rakush <valentin.rak...@gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I am trying to do first pull request to
>>> https://github.com/MariaDB/server but I cannot push my new branch to
>>> github. Git says
>>>
>>> "ERROR: Permission to MariaDB/server.git denied to alpes214.
>>> fatal: Could not read from remote repository.
>>>
>>> Please make sure you have the correct access rights
>>> and the repository exists."
>>>
>>>
>>>
>>> Should I talk to somebody to get access rights first?
>>>
>>> --
>>> Kind regards,
>>> Valentin Rakush
>>> ___
>>> Mailing list: https://launchpad.net/~maria-developers
>>> Post to : maria-developers@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~maria-developers
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>
>
> --
> Best Regards,
> Valentin Rakush.
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Cannot push branch to github

[Vicențiu Ciorbaru]

Hi Valentin!

The correct way to do a pull request is to create your own fork of the
server repository. You don't need any special permissions for that.
Afterwards, push your changes to your fork. Finally, click on New pull
request button in *your* repository. Select the appropriate branch for the
base fork and the appropriate branch from your head fork. Submit the pull
request and we'll take it from there.

Regards,
Vicențiu

On Thu, 25 May 2017 at 12:37 Valentin Rakush 
wrote:

> Hi all,
>
> I am trying to do first pull request to https://github.com/MariaDB/server
> but I cannot push my new branch to github. Git says
>
> "ERROR: Permission to MariaDB/server.git denied to alpes214.
> fatal: Could not read from remote repository.
>
> Please make sure you have the correct access rights
> and the repository exists."
>
>
>
> Should I talk to somebody to get access rights first?
>
> --
> Kind regards,
> Valentin Rakush
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] cte page

[Vicențiu Ciorbaru]

Hi Jocelyn!

There have been quite a few requests to benchmark this. Although CTE
merging conflicts with reusing, it would be interesting to get some results
in. The best person to do this is a third party, as any one from MySQL or
MariaDB may be biased in choosing workloads. Nonetheless it is something I
plan to do in the near future.

Vicențiu

On Fri, 28 Apr 2017 at 01:34 jocelyn fournier 
wrote:

> Hi!
>
>
> Le 27/04/2017 à 18:52, Galina Shalygina a écrit :
> > Hi!
> >
> > I'm so sorry for this, I was in a hurry and chose a wrong address to
> > mail:(
> >
> > All in all, thank you! It is still a raw version and I will improve it.
> >
> > Later I will send you a final version in a normal email:)
> >
> > Sorry for this again,
> > Best regards,
> > Galina
> Just curious, how MySQL 8 is performing compared to PostgreSQL and
> MariaDB? (I think it could be interesting since it uses both CTE merge &
> CTE reuse optimizations)
>
> Thanks!
>Jocelyn
>
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] cte page

[Vicențiu Ciorbaru]

Very good!  Congratulations on finishing the paper! Thank you for all the
great work. :)

Vicențiu

On Thu, 27 Apr 2017 at 09:11 Galina Shalygina 
wrote:

> мои артиклы
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Refactoring in LEX_STRING and LEX_CSTRING

[Vicențiu Ciorbaru]

Hi!

I would like to add a few more arguments in favour of using const
references and not const pointers. I do not endorse non-const references,
that makes the code silly and confusing a lot of the times. When we need
output or input-output parameters we should keep using pointers. For
strictly mandatory (must never be null) input parameters, const references
are the way to go, unless there is a strong reason not to.

The guarantee that references enforce is "never null". This is a very
important and useful feature of C++. We can completely skip any checks such
as:

func1(LEX_CSTRING *str)
{
   if (!str) { .. }
}

Or DBUG_ASSERTS such as:
func1(LEX_CSTRING *str)
{
DBUG_ASSERT(str);
...
}

The code with *const references*  is self documenting in this sense. It
makes it very clear that the function does not own the object it is being
passed (no questions wether the pointer should be freed or not) and that
the reference is valid.

Other arguments, not strictly related to this use case, but are in favor of
const references:

* By limiting our use of references to const references we avoid the
ambiguity between regular pass-by-value arguments and non-const reference
arguments. You work with them as if they are pass-by-value. Since they are
const you can't do anything to them anyway, you're not changing any of the
caller's data.
* Pointers are unclear wether they point to a buffer array or a single
item. References do not allow you to do: ptr[0], ptr[1], ptr[2], etc.
* If we chose to make use of STL algorithms or C++11 features down the
line, references tend to work easier than pointers. Can't come up with a
good example right this moment, but most stl algorithms expect
reference-based parameters, ex:
std::max(*int_ptr_a, *int_ptr_b) vs std::max(int_a, int_b)
* If some legacy code does need a pointer parameter, using the unary &
operator on the reference will produce the required pointer type.

If we are doing refactoring anyway, I strongly request we reconsider the
policy of avoiding references completely. I suggest we at least make use of
them as input parameters to functions and methods in new code.

Vicențiu



On Thu, 20 Apr 2017 at 00:10 Alexander Barkov  wrote:

> Hello all,
>
>
> Monty is now doing refactoring:
>
> 1. Replacing a lot of LEX_STRING to LEX_CSTRING.
>
> 2. Fixing functions and methods that have arguments of types:
> a. pointer "const LEX_STRING *name" and
> b. reference "const LEX_STRING ".
> to the same style. Monty chose passing by pointer (2a).
>
>
> I think that:
>
> - #1 is good.
> - #2 is good, in the sense of changing to the same style.
>
> But I think that the choice of "passing by pointer" in #2 is bad.
> Passing by reference for LEX_STRING/LEX_CSTRING related things would be
> better.
>
>
> With the "passing by pointer" approach whenever one needs to pass
> a LEX_STRING to a function accepting LEX_CSTRING, one will have to do:
>
>
> void func1(const LEX_STRING *str)
> {
>   LEX_CSTRING tmp= {str->str, str->length};
>   func2();
> }
>
>
>
>
> I'd propose two things instead:
>
> 1. Add a class:
>
>
> class Lex_cstring: public LEX_CSTRING
> {
> public:
>   Lex_cstring() {} /*Uninitialized */
>   Lex_cstring(const char *str, size_t length)
>   {
> LEX_STRING::str= str;
> LEX_STRING::length= length;
>   }
>   Lex_cstring(const LEX_STRING *str)
>   {
> LEX_STRING::str= str->str;
> LEX_STRING::length= str->length;
>   }
> };
>
>
> 2. Instead of "const LEX_CSTRING *str"
> we use "const Lex_cstring " all around C++ code.
>
> So the above example with passing a LEX_STRING to
> something that expect a LEX_CSTRING would be simplified to
>
> void func1(const LEX_STRING *str)
> {
>   func2(str);
> }
>
>
> 3. In some classes now we have duplicate methods and constructors like
> this:
>
> class XXX
> {
> public:
>   XXX(const char *str, size_t length) { ... }
>   XXX(const LEX_STRING *str) { ... }
>   void method(const char *str, size_t length) {...}
>   void method(const LEX_STRING *str) {...}
> };
>
> to make it possible to call like this:
>
>   new XXX(str, length);
>   new XXX(lex_str_ptr)
>   new XXX(_str)
>   method(str, length);
>   method(lex_str_ptr);
>   method(_str);
>
>
> We change this to "passing by reference" and remove duplicates:
>
> class XXX
> {
> public:
>   XXX(const Lex_cstring ) { ... }
>   void method(const Lex_cstring ) {...}
> };
>
> After this change we can call like this:
>
>   new XXX(Lex_cstring(str, length));
>   new XXX(*lex_str_ptr)
>   new XXX(lex_str);
>   method(Lex_cstring(str, length));
>   method(*lex_str_ptr);
>   method(lex_str);
>
> Notice, not needs for separate methods accepting
> two arguments "const char *str, size_t length".
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Please review patch for 12017 and 12018.

[Vicențiu Ciorbaru]

Hi Sachin!

I'll review the patch. If Monty wants to chip in he can do so.

1. Indentation is wrong for the patch, you have an extra step of
indentation.
2. I would use only one return true statement, after the if/else statement.
3. For singular subject sentence use does instead of do. So:
'Flashback does not support %s'. Same for the error message 'MariaDB Galera
does not support binlog format: %s'
4. A testcase would be really useful for this.

Vicențiu


On Fri, 7 Apr 2017 at 09:46 Sachin Setiya  wrote:

Hi Monty!,

Please review patch for 12017 and 12018 . this patch solves both the
problem.


--
Regards
Sachin Setiya
Software Engineer at  MariaDB
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Errors in building debs

[Vicențiu Ciorbaru]

Latest push ought to fix this. Waiting for buildbot to confirm.

On Fri, 31 Mar 2017 at 23:05, Sergey Petrunia  wrote:

Hi Vicențiu,


I'm looking at the last build in bb-10.2-mariarocks

http://buildbot.askmonty.org/buildbot/builders/kvm-deb-trusty-x86/builds/3558/steps/compile/logs/stdio
and I see this:

+ DH_BUILD_DDEBS=1
+ echo
+ debian/autobake-deb.sh
debian/autobake-deb.sh: line 80: [: i686: integer expression expected
debian/autobake-deb.sh: line 80: [i686: command not found

and then it eventually fails with this error:

cp: cannot stat ‘debian/tmp/etc/mysql/conf.d/rocksdb_se.cnf’: No such file
or

I assume this is caused by your recent changes?

BR
 Sergei
--
Sergei Petrunia, Software Developer
MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] GSoC 2016 Student Introduction. [ MDEV-4989 ]

[Vicențiu Ciorbaru]

Hi Mike!

For debugging on OS X I suggest you use lldb instead of gdb, if you're
having troubles with gdb. At the same time you can get gdb to work but
sometimes it doesn't work as you expect it to, so lldb might be your best
alternative. Using mysql-test-run with lldb will also be somewhat tricky.
You can run the commands manually against an already written binlog file.

There are quite a few articles on gtid that you have probably already seen,
linked in jira. In case you missed them, you can read up on them, perhaps
they will answer more of your questions. The knowledge base one will
probably answer the star position vs gtid position problem you're having.

https://mariadb.com/kb/en/mariadb/gtid/

There currently isn't one assigned mentor to the task so asking on the
mailing list is good. This allows everybody to follow the discussion.

Regards,
Vicentiu


On Mon, 13 Mar 2017 at 00:58, Mike Reiss  wrote:

> Hi,
>
> I am Mike Reiss a MS Computer Science student and I would love to work on
> enhancing the mysqlbinlog tool  during GSoC 2017 [MDEV-4989
> ]. I have studied the
> mysqlbinlog.cc file and experimented with the various options available in
> the my_options array of the tool. The proposed idea is straight forward to
> implement: We add the --start-gtid and --stop-gtid options to the array of
> options then parse the GTID passed to these options to get the GTID
> sequence number for the binlog. One quick question. Is there any
> relationship between a GTID sequence number and the argument passed to the
> --start-position option? that way we need to set the start position to the
> GTID sequence number. I am still doing more research into the source files
> and I will love to discuss this more with the mentor. I am having
> difficulties debugging mysqlbinlog on OS X sierra because it does not
> support GDB any ideas please?
>
> Thanks.
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Fwd: 41f885f64fe: MDEV-10859: Wrong result of aggregate window function in query with HAVING and no ORDER BY

[Vicențiu Ciorbaru]

Hi Igor, Sergey!

I need a review on this patch that fixes a problem with Window Functions
being computed over too many rows. The details are in the commit message.

Thank you!
Vicențiu

-- Forwarded message -
From: vicentiu <vicen...@mariadb.org>
Date: Tue, 14 Feb 2017 at 14:08
Subject: 41f885f64fe: MDEV-10859: Wrong result of aggregate window function
in query with HAVING and no ORDER BY
To: <comm...@mariadb.org>


revision-id: 41f885f64fe4c02e03ec655f225b1dc01e9d5376
(mariadb-10.2.3-252-g41f885f64fe)
parent(s): d731ce21a7bc25a49958789d24b3f0eec5845aea
author: Vicențiu Ciorbaru
committer: Vicențiu Ciorbaru
timestamp: 2017-02-14 14:02:29 +0200
message:

MDEV-10859: Wrong result of aggregate window function in query with HAVING
and no ORDER BY

Window functions need to be computed after applying the HAVING clause.
An optimization that we have for regular, non-window function, cases is
to apply having only during sending of the rows to the client. This
allows rows that should be filtered from the temporary table used to
store aggregation results to be stored there.

This behaviour is undesireable for window functions, as we have to
compute window functions on the result-set after HAVING is applied.
Storing extra rows in the table leads to wrong values as the frame
bounds might capture those -to be filtered afterwards- rows.

---
 mysql-test/r/win.result | 33 +
 mysql-test/t/win.test   | 23 +++
 sql/sql_select.cc   | 11 ---
 sql/sql_window.cc   |  6 ++
 4 files changed, 70 insertions(+), 3 deletions(-)

diff --git a/mysql-test/r/win.result b/mysql-test/r/win.result
index fd3aea80083..b3adf358be6 100644
--- a/mysql-test/r/win.result
+++ b/mysql-test/r/win.result
@@ -2925,3 +2925,36 @@ WHERE EXISTS ( SELECT * FROM t2 WHERE c IN ( SELECT
MAX(d) FROM t3 ) );
 a  MAX(a)  AVG(a) OVER (PARTITION BY b)
 NULL   NULLNULL
 DROP TABLE t1,t2,t3;
+#
+# MDEV-MDEV-10859: Wrong result of aggregate window function in query
+#  with HAVING and no ORDER BY
+#
+create table empsalary (depname varchar(32), empno smallint primary key,
salary int);
+insert into empsalary values
+('develop', 1, 5000), ('develop', 2, 4000),('sales', 3, '6000'),('sales',
4, 5000);
+SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname)
FROM empsalary;
+depnameempno   salary  avg(salary) OVER (PARTITION BY depname)
+develop1   50004500.
+develop2   40004500.
+sales  3   60005500.
+sales  4   50005500.
+SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname)
FROM empsalary ORDER BY depname;
+depnameempno   salary  avg(salary) OVER (PARTITION BY depname)
+develop1   50004500.
+develop2   40004500.
+sales  3   60005500.
+sales  4   50005500.
+#
+# These last 2 should have the same row results, ignoring order.
+#
+SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname)
FROM empsalary HAVING empno > 1;
+depnameempno   salary  avg(salary) OVER (PARTITION BY depname)
+develop2   40004000.
+sales  3   60005500.
+sales  4   50005500.
+SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname)
FROM empsalary HAVING empno > 1 ORDER BY depname;
+depnameempno   salary  avg(salary) OVER (PARTITION BY depname)
+develop2   40004000.
+sales  3   60005500.
+sales  4   50005500.
+drop table empsalary;
diff --git a/mysql-test/t/win.test b/mysql-test/t/win.test
index aa636f7a294..90bbf41688f 100644
--- a/mysql-test/t/win.test
+++ b/mysql-test/t/win.test
@@ -1711,3 +1711,26 @@ SELECT a, MAX(a), AVG(a) OVER (PARTITION BY b) FROM
t1
 WHERE EXISTS ( SELECT * FROM t2 WHERE c IN ( SELECT MAX(d) FROM t3 ) );

 DROP TABLE t1,t2,t3;
+
+--echo #
+--echo # MDEV-MDEV-10859: Wrong result of aggregate window function in
query
+--echo #  with HAVING and no ORDER BY
+--echo #
+
+create table empsalary (depname varchar(32), empno smallint primary key,
salary int);
+insert into empsalary values
+  ('develop', 1, 5000), ('develop', 2, 4000),('sales', 3,
'6000'),('sales', 4, 5000);
+
+--sorted_result
+SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname)
FROM empsalary;
+--sorted_result
+SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname)
FROM empsalary ORDER BY depname;
+--echo #
+--echo # These last 2 should have the same row results, ignoring order.
+--echo #
+--sorted_result
+SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname)
FROM empsalary HAVING empno > 1;
+--sorted_result
+SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname)
FROM empsalary HAVING empno > 1 ORDER BY depname;
+
+drop table empsalary;
diff --git a/sql/sql_select.cc b/sql/sql_select.cc
index 5c7ae1e

Re: [Maria-developers] Please review MDEV-11692 Comparison data type aggregation for pluggable data types

[Vicențiu Ciorbaru]

Hi Alexander!

Comments inline. OK to push otherwise.

> diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc
> index fc3f81a..6fcfec4 100644
> --- a/sql/item_cmpfunc.cc
> +++ b/sql/item_cmpfunc.cc
> @@ -118,8 +118,11 @@ static int cmp_row_type(Item* item1, Item* item2)
>  0  otherwise
>  */
>
> -bool Type_handler_hybrid_field_type::aggregate_for_comparison(Item
**items,
> -  uint
nitems)
> +bool
> +Type_handler_hybrid_field_type::aggregate_for_comparison(const char
*funcname,
> + Item **items,
> + uint nitems,
> +  bool
treat_int_to_uint_as_decimal)
The comment above this function is outdated. Please update. Also, weird
indenting here to try and stay within 80 characters. I'd say new line for
each
parameter in the function and 4 space indent for each line.
>  {
>uint unsigned_count= items[0]->unsigned_flag;
>/*
> @@ -471,8 +489,16 @@ int Arg_comparator::set_cmp_func(Item_func_or_sum
*owner_arg,
>set_null= set_null && owner_arg;
>a= a1;
>b= a2;
> -  m_compare_handler=
Type_handler::get_handler_by_cmp_type(item_cmp_type(*a1,
> -
*a2));
> +  Item *tmp_args[2];
> +  tmp_args[0]= *a1;
> +  tmp_args[1]= *a2;
>
This could simply be Item *tmp_args[2]= {*a1, *a2};
>
> +  Type_handler_hybrid_field_type tmp;
> +  if (tmp.aggregate_for_comparison(owner_arg->func_name(), tmp_args, 2,
false))
> +  {
> +DBUG_ASSERT(thd->is_error());
> +return 1;
> +  }
> +  m_compare_handler= tmp.type_handler();
>return m_compare_handler->set_comparator_func(this);
>  }
>
> @@ -3033,7 +3063,7 @@ bool
Item_func_case::prepare_predicant_and_values(THD *thd, uint *found_types)
>add_predicant(this, (uint) first_expr_num);
>for (uint i= 0 ; i < ncases / 2; i++)
>{
> -if (add_value_skip_null(this, i * 2, _null))
> +if (add_value_skip_null("CASE..WHEN", this, i * 2, _null))
If you look at the test results, the CASE..WHEN is the only operation that
ends up in CAPS, while the rest are lowercase. I'd say to have this in lower
case as well, for consistency.
>return true;
>}
>all_values_added(, _cnt, _found_types);

Vicentiu


On Fri, 30 Dec 2016 at 22:40 Alexander Barkov  wrote:

> Hello Vicențiu,
>
> can you please review a patch for MDEV-11692.
>
> The patch is for bb-10.2-ext.
>
> Thanks!
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Please review MDEV-9979 Keywords UNBOUNDED, PRECEDING, FOLLOWING, TIES, OTHERS should be non-reserved

[Vicențiu Ciorbaru]

Hi Alexander,

OK to push. Please add the version of the standard you used to check these
keywords as non-reserved both to the commit message, as well as the
keywords themselves. I used 2011 standard draft to check.

Vicențiu

On Mon, 23 Jan 2017 at 09:28 Alexander Barkov  wrote:

> Hello Vicențiu,
>
> Can you please review a patch for MDEV-9979.
>
> Thanks!
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Please review MDEV-9522 Split sql_select.cc:can_change_cond_ref_to_const into virtual methods in Type_handler

[Vicențiu Ciorbaru]

Hi Alexander!

Ok to push this one. I really like how this splits the function.

Please put a comment that collation does not make sense (and thus is no
concern) for numerical type handlers in the function where you compare them
by pointer address.

Also, Item_get_cache should be on a separate line. In this patch it ends up
being one long line:

+   Item *source_expr, Item *source_const)
const;  Item_cache *Item_get_cache(THD *thd, const Item *item) const;
Regards,
Vicentiu

On Sat, 10 Dec 2016 at 18:20 Alexander Barkov  wrote:

> Forgot to attach the patch in the previous letter.
>
>
> On 12/10/2016 08:17 PM, Alexander Barkov wrote:
> > Hello Vicențiu,
> >
> > can you please review a patch for MDEV-9522 ?
> >
> > Thanks!
> >
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Fwd: mixing of user-defined data types with other data types

[Vicențiu Ciorbaru]

Hi Alexander,

I've reviewed your second patch. Comments inline.

>> +}
> >>
> >>
> >>  /*
> >> diff --git a/sql/field.h b/sql/field.h
> >> index 541da5a..fa84e7d 100644
> >> --- a/sql/field.h
> >> +++ b/sql/field.h
> >> @@ -835,7 +835,6 @@ class Field: public Value_source
> >>virtual Item_result cmp_type () const { return result_type(); }
> >>static bool type_can_have_key_part(enum_field_types);
> >>
> > The field_type_merge function breaks all the other naming patterns. We
> > have result_type, cmp_type real_type and now field_type_merge. Wouldn't
> it be
> > better to name it field_merge_type, to be consistent? Or since this
> > is a lookup kind of operation, perhaps name it
> (get|lookup)_merge_field_type?
> > I know it's not _exactly_ like result_type and compare_type, but it
> generally
> > is used in a simillar context.
>
> Don't we make our life harder in respect of merging when renaming?
>
> Usually I try not to rename existing functions:
> - There is a chance that we'll merge something from earlier versions,
>   and renaming can cause conflicts.
> - Also, developers are used to this name and can do
>   "grep field_type_merge" or similar when searching the code.
>

I see your point. I am not 100% in favor of the idea of protecting
ourselves (making life easier) in future merges, as it tends to keep us
stuck with the same version of (often) difficult to read code. For this
case, I guess we can live with it, but I'll keep on pointing such things in
future reviews. :)


>
> >>
> >>static enum_field_types field_type_merge(enum_field_types,
> enum_field_types);
> >> -  static Item_result result_merge_type(enum_field_types);
> >>virtual bool eq(Field *field)
> >>{
> >>  return (ptr == field->ptr && null_ptr == field->null_ptr &&
> >> diff --git a/sql/item.cc b/sql/item.cc
> >> index c97f41f..c9b6155 100644
> >> --- a/sql/item.cc
> >> +++ b/sql/item.cc
> >> @@ -9657,20 +9657,8 @@ Item_type_holder::Item_type_holder(THD *thd,
> Item *item)
> >>maybe_null= item->maybe_null;
> >>collation.set(item->collation);
> >>get_full_info(item);
> >> -  /**
> >> -Field::result_merge_type(real_field_type()) should be equal to
> >> -result_type(), with one exception when "this" is a Item_field for
> >> -a BIT field:
> >> -- Field_bit::result_type() returns INT_RESULT, so does its
> Item_field.
> >> -- Field::result_merge_type(MYSQL_TYPE_BIT) returns STRING_RESULT.
> >> -Perhaps we need a new method in Type_handler to cover these type
> >> -merging rules for UNION.
> >> -  */
> >> -  DBUG_ASSERT(real_field_type() == MYSQL_TYPE_BIT ||
> >> -  Item_type_holder::result_type()  ==
> >> -
> Field::result_merge_type(Item_type_holder::real_field_type()));
> >>/* fix variable decimals which always is NOT_FIXED_DEC */
> >> -  if (Field::result_merge_type(real_field_type()) == INT_RESULT)
> >>
> > Alright so this seems to be fixed here, looking at Type_handler_bit,
> > inheriting from Type_handler_int_result. Do we test this somewhere
> though?
> > I couldn't find it in the test case, perhaps you can point it out to me.
>
>
> In theory, decimals is always 0 if result_type() is INT_RESULT.
> But I'm not fully sure that in reality non of the Items return
> non-zero decimals in combination with INT_RESULT.
> There's so many hacks in the code, so we combination can
> be used somewhere.
>
> I just tried to comment out these two lines:
> > -  if (Item_type_holder::result_type() == INT_RESULT)
> > -decimals= 0;
> > +//  if (Item_type_holder::result_type() == INT_RESULT)
> > +//decimals= 0;
> both in the constructor and in the method join_types()
> and run test. Nothing failed.
>
> So perhaps these two lines can be just replaced to:
>
> DBUG_ASSERT(decimals == 0 ||
> Item_type_holder::result_type() != INT_RESULT);
>
> push, and see.
>
> Any suggestions?
>

This is indeed ugly. I've tried to look into the calling places, then to
backtrack from there but I gave up after about 30 minutes of seeing a never
ending branching possibility of items. Please add the assert.


>
> > Let's discuss about cleaning this up later. To me it feels like this
> Item does
> > not really belong in the Item class and should be factored out. Probably
> > a whole project on its own :)
>
> I made attempts to move Item_type_holder out of the Item hierarchy in
> the past, but failed. It caused too much refactoring, because
> Item_type_holder is used with List all around the UNION
> and table creation code.
>
> Perhaps we should make another attempt eventually.
> I suggest to postpone this at least after the main Type_handler related
> changes are done.
>
>
I agree.


> >>
> >>item_decimals= 0;
> >>  decimals= MY_MAX(decimals, item_decimals);
> >>}
> >> diff --git a/sql/item_cmpfunc.cc b/sql/item_cmpfunc.cc
> >> index 98b179b..e5e366e 100644
> >> --- a/sql/item_cmpfunc.cc
> >> +++ b/sql/item_cmpfunc.cc
> >> @@ -180,32 +179,40 @@ static int 

[Maria-developers] Fwd: mixing of user-defined data types with other data types

[Vicențiu Ciorbaru]

Hi Alexander!

Comments inline. Most are stylistic comments. I think the patch is
great. The Item_type_holder is really bugging me though. It feels
poorly designed in the current place.

> diff --git a/sql/field.cc b/sql/field.cc
> index 6f273e6..7876a6f 100644
> --- a/sql/field.cc
> +++ b/sql/field.cc
> @@ -84,9 +84,13 @@ const char field_separator=',';
>  */
>  #define FIELDTYPE_TEAR_FROM (MYSQL_TYPE_BIT + 1)
>  #define FIELDTYPE_TEAR_TO   (MYSQL_TYPE_NEWDECIMAL - 1)
> -#define FIELDTYPE_NUM (FIELDTYPE_TEAR_FROM + (255 - FIELDTYPE_TEAR_TO))
> +#define FIELDTYPE_LAST  254
> +#define FIELDTYPE_NUM (FIELDTYPE_TEAR_FROM + (FIELDTYPE_LAST - 
> FIELDTYPE_TEAR_TO))
>
These defines here look sketchy. I know it's not from your patch but can we not
turn them to const ints instead?
Also, the naming is horibly inconsistent, one uses 2 and others use to.
I vote for _to_. We can have the full function definition as:

const int FIELDTYPE_TEAR_FROM= MYSQL_TYPE_BIT + 1;
const int FIELDTYPE_TEAR_TO=   MYSQL_TYPE_NEWDECIMAL - 1;
const int FIELDTYPE_LAST=  254;
const int FIELDTYPE_NUM=   FIELDTYPE_TEAR_FROM + (FIELDTYPE_LAST -
 FIELDTYPE_TEAR_TO);
static inline int field_type_to_index (enum_field_types field_type)
{
 DBUG_ASSERT(real_type_to_type(field_type) < FIELDTYPE_TEAR_FROM ||
 real_type_to_type(field_type) > FIELDTYPE_TEAR_TO);
 DBUG_ASSERT(field_type <= FIELDTYPE_LAST);
 field_type= real_type_to_type(field_type);
 if (field_type < FIELDTYPE_TEAR_FROM)
   return field_type;
 return FIELDTYPE_TEAR_FROM + (field_type - FIELDTYPE_TEAR_TO) - 1;
}

The ternary operator provides no benefit in this case, let's replace it with
regular if statement. Same number of lines but the code is cleaner I'd say.
>
>  static inline int field_type2index (enum_field_types field_type)
>  {
> +  DBUG_ASSERT(real_type_to_type(field_type) < FIELDTYPE_TEAR_FROM ||
> +  real_type_to_type(field_type) > FIELDTYPE_TEAR_TO);
> +  DBUG_ASSERT(field_type <= FIELDTYPE_LAST);
>field_type= real_type_to_type(field_type);
>return (field_type < FIELDTYPE_TEAR_FROM ?
>field_type :
> @@ -94,6 +98,12 @@ static inline int field_type2index (enum_field_types 
> field_type)
>  }
>
>
> +/**
> +  Implements data type merge rules for the built-in traditional data types.
> +  - UNION
> +  - CASE and its abbreviations COALESCE, IF, IFNULL
> +  - LEAST/GREATEST
> +*/
>
I don't like this comment here as it's not complete. I'd much rather have it
also explain how to interpret this table. I had to look up a calling spot
to figure out exactly how it's done. I'd also have it say:
Ex:
Given Fields A and B of real_types a and b, we find the result type of
COALESCE(A, B) by querying:
field_types_merge_rules[field_type_to_index(a)][field_type_to_index(b)].
>
>
>  static enum_field_types field_types_merge_rules 
> [FIELDTYPE_NUM][FIELDTYPE_NUM]=
>  {
>/* MYSQL_TYPE_DECIMAL -> */
> @@ -948,46 +927,18 @@ static enum_field_types field_types_merge_rules 
> [FIELDTYPE_NUM][FIELDTYPE_NUM]=
>  enum_field_types Field::field_type_merge(enum_field_types a,
>   enum_field_types b)
>  {
> -  DBUG_ASSERT(real_type_to_type(a) < FIELDTYPE_TEAR_FROM ||
> -  real_type_to_type(a) > FIELDTYPE_TEAR_TO);
> -  DBUG_ASSERT(real_type_to_type(b) < FIELDTYPE_TEAR_FROM ||
> -  real_type_to_type(b) > FIELDTYPE_TEAR_TO);
>return field_types_merge_rules[field_type2index(a)]
>  [field_type2index(b)];
>  }
>
> -
> -static Item_result field_types_result_type [FIELDTYPE_NUM]=
> +const Type_handler *
> +Type_handler::aggregate_for_result_traditional(const Type_handler *a,
> +   const Type_handler *b)
>  {
> -  //MYSQL_TYPE_DECIMAL  MYSQL_TYPE_TINY
> -  DECIMAL_RESULT,   INT_RESULT,
> -  //MYSQL_TYPE_SHORTMYSQL_TYPE_LONG
> -  INT_RESULT,   INT_RESULT,
> -  //MYSQL_TYPE_FLOATMYSQL_TYPE_DOUBLE
> -  REAL_RESULT,  REAL_RESULT,
> -  //MYSQL_TYPE_NULL MYSQL_TYPE_TIMESTAMP
> -  STRING_RESULT,STRING_RESULT,
> -  //MYSQL_TYPE_LONGLONG MYSQL_TYPE_INT24
> -  INT_RESULT,   INT_RESULT,
> -  //MYSQL_TYPE_DATE MYSQL_TYPE_TIME
> -  STRING_RESULT,STRING_RESULT,
> -  //MYSQL_TYPE_DATETIME MYSQL_TYPE_YEAR
> -  STRING_RESULT,INT_RESULT,
> -  //MYSQL_TYPE_NEWDATE  MYSQL_TYPE_VARCHAR
> -  STRING_RESULT,STRING_RESULT,
> -  //MYSQL_TYPE_BIT  <16>-<245>
> -  STRING_RESULT,
> -  //MYSQL_TYPE_NEWDECIMAL   MYSQL_TYPE_ENUM
> -  DECIMAL_RESULT,   STRING_RESULT,
> -  //MYSQL_TYPE_SET  MYSQL_TYPE_TINY_BLOB
> -  STRING_RESULT,STRING_RESULT,
> -  //MYSQL_TYPE_MEDIUM_BLOB  MYSQL_TYPE_LONG_BLOB
> -  STRING_RESULT,STRING_RESULT,
> -  //MYSQL_TYPE_BLOB MYSQL_TYPE_VAR_STRING
> -  STRING_RESULT,

Re: [Maria-developers] [Commits] 02e74a5: MDEV-11700: MariaDB 10.2 cannot start on MySQL 5.7 datadir:

[Vicențiu Ciorbaru]

Hi Daniel!

On Mon, 12 Dec 2016 at 04:08 Daniel Black <daniel.bl...@au1.ibm.com> wrote:

> nit - its really MDEV-11170
>

Argh, thanks. That's always annoying to get right :)


> On 12/12/16 12:57, Vicențiu Ciorbaru wrote:
> > Hi Sergei!
> >
> > Can you please review this patch for MDEV-11700? Details presented in
> > the commit message. I'm not sure which solution we want.
> > Possible solutions:
> > A) have the user fix it by first starting the server with
> > --skip-grant-tables to avoid "mysql.user table is damaged", then run
> > mysql_upgrade, to fix the problem. Without this patch, this solution
> > fails at mysql_upgrade.
> > B) Have the server understand MySQL 5.7.6 + table format where the
> > Password column is missing.
> >
> > For 10.2 I've chosen solution A. Although this should probably also be
> > targeted for 10.1?
> > For 10.3, since we're planning on doing some authentication changes, we
> > can implement a variant of B.
>
> Without B or a population of the password column the authentication
> isn't migrated.
>

 Actually that's not quite true. We still have the authentication_string
and the plugin columns from MySQL. They will contain the password and it
will work as MariaDB understands that. I've specifically tested this with a
MySQL data dir locally.

Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] 02e74a5: MDEV-11700: MariaDB 10.2 cannot start on MySQL 5.7 datadir:

[Vicențiu Ciorbaru]

Hi Sergei!

Can you please review this patch for MDEV-11700? Details presented in the
commit message. I'm not sure which solution we want.
Possible solutions:
A) have the user fix it by first starting the server with
--skip-grant-tables to avoid "mysql.user table is damaged", then run
mysql_upgrade, to fix the problem. Without this patch, this solution fails
at mysql_upgrade.
B) Have the server understand MySQL 5.7.6 + table format where the Password
column is missing.

For 10.2 I've chosen solution A. Although this should probably also be
targeted for 10.1?
For 10.3, since we're planning on doing some authentication changes, we can
implement a variant of B.

Thoughts?

On Mon, 12 Dec 2016 at 03:50 vicentiu <vicen...@mariadb.org> wrote:

> revision-id: 02e74a5614b08cfce505002abc26d4634648308e
> (mariadb-10.2.2-138-g02e74a5)
> parent(s): 9320d8ae30c18420bef659618175836221d363ea
> author: Vicențiu Ciorbaru
> committer: Vicențiu Ciorbaru
> timestamp: 2016-12-12 03:42:11 +0200
> message:
>
> MDEV-11700: MariaDB 10.2 cannot start on MySQL 5.7 datadir:
>
> Fatal error: mysql.user table is damaged or in unsupported 3.20 format
>
> The problem stems from MySQL 5.7.6. According to MySQL documentation:
> In MySQL 5.7.6, the Password column was removed and all credentials are
> stored in the authentication_string column.
>
> If opening a MySQL 5.7.6 (and up) datadir with MariaDB 10.2, the user table
> appears corrupted. In order to fix this, the server must be started with
> --skip-grant-tables and then a subsequent mysql_upgrade command must be
> issued.
>
> This patch updates the mysql_upgrade command to also add the removed
> Password column. The password column is necessary, otherwise
> the mysql_upgrade script fails due to the Event_scheduler not being able
> to start, as it can't find Event_priv in the table where it ought to be.
> MySQL's version has column position 28 (0 index) vs our datadir version
> expects position 29.
>
> ---
>  scripts/mysql_system_tables_fix.sql | 6 ++
>  1 file changed, 6 insertions(+)
>
> diff --git a/scripts/mysql_system_tables_fix.sql
> b/scripts/mysql_system_tables_fix.sql
> index ea1059c..80d0ef7 100644
> --- a/scripts/mysql_system_tables_fix.sql
> +++ b/scripts/mysql_system_tables_fix.sql
> @@ -164,6 +164,12 @@ ALTER TABLE user
>MODIFY Host char(60) NOT NULL default '',
>MODIFY User char(80) NOT NULL default '',
>ENGINE=MyISAM, CONVERT TO CHARACTER SET utf8 COLLATE utf8_bin;
> +
> +# In MySQL 5.7.6 the Password column is removed. Recreate it to preserve
> the number
> +# of columns MariaDB expects in the user table.
> +ALTER TABLE user
> +  ADD Password char(41) character set latin1 collate latin1_bin NOT NULL
> default '' AFTER User;
> +
>  ALTER TABLE user
>MODIFY Password char(41) character set latin1 collate latin1_bin NOT
> NULL default '',
>MODIFY Select_priv enum('N','Y') COLLATE utf8_general_ci DEFAULT 'N'
> NOT NULL,
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 27af988: MDEV-11479 Improved wsrep_dirty_reads

[Vicențiu Ciorbaru]

Hi Sachin!

Comments inline:

On Sun, 4 Dec 2016 at 08:36  wrote:

> revision-id: 27af98805c27c8e0c41dc8c19bf80aefff4c6d3d
> (mariadb-galera-5.5.53-3-g27af988)
> parent(s): 72fd15f7c31aa3e3705ae1b005a3247a985c5bb3
> author: SachinSetiya
> committer: SachinSetiya
>

NIT: To get these to display correctly you should have a space between
Sachin and Setiya.


> timestamp: 2016-12-04 12:05:05 +0530
> message:
>
> MDEV-11479 Improved wsrep_dirty_reads
>
> Tasks:-
> Changes in wsrep_dirty_reads variable
> 1.) Global + Session scope (Current: session-only)
> 2.) Allow all commands that do not change data (besides SELECT)
> 3.) Allow prepared Statements that do not change data
> 4.) Works with wsrep_sync_wait enabled
>

You also seemed to have made the variable available as a command line
option. Please specify that as well.


> ---
>  .../suite/galera/r/galera_var_dirty_reads.result   | 29
> +++---
>  .../suite/galera/t/galera_var_dirty_reads.test | 25
> ++-
>  .../sys_vars/r/wsrep_dirty_reads_basic.result  | 19 --
>  .../suite/sys_vars/t/wsrep_dirty_reads_basic.test  | 13 --
>  sql/sql_parse.cc   | 18 ++
>  sql/sys_vars.cc|  4 +--
>  sql/wsrep_mysqld.cc|  3 ++-
>  sql/wsrep_mysqld.h |  1 +
>  8 files changed, 96 insertions(+), 16 deletions(-)
>
> diff --git a/mysql-test/suite/galera/r/galera_var_dirty_reads.result
> b/mysql-test/suite/galera/r/galera_var_dirty_reads.result
> index 6a2aa1e..d009c1b 100644
> --- a/mysql-test/suite/galera/r/galera_var_dirty_reads.result
> +++ b/mysql-test/suite/galera/r/galera_var_dirty_reads.result
> @@ -28,9 +28,32 @@ SELECT @@max_allowed_packet;
>  SELECT 2+2 from DUAL;
>  2+2
>  4
> -SELECT sysdate() from DUAL;
> -sysdate()
> -2016-10-28 23:13:06
> +SET @@session.wsrep_dirty_reads=ON;
> +SELECT * FROM t1;
> +i
> +1
> +prepare stmt_show from 'select 1';
> +prepare stmt_select from 'select * from t1';
> +prepare stmt_insert from 'insert into t1 values(1)';
> +execute stmt_show;
> +1
> +1
> +execute stmt_select;
> +i
> +1
> +execute stmt_insert;
> +ERROR 08S01: WSREP has not yet prepared node for application use
> +select * from t1;
> +i
> +1
> +execute stmt_show;
> +1
> +1
> +execute stmt_select;
> +i
> +1
> +execute stmt_insert;
> +ERROR 08S01: WSREP has not yet prepared node for application use
>  SELECT * FROM t1;
>  i
>  1
>

You're not really testing the global aspect of the variable here. You're
only testing the sesssion behaviour of it. Also, test with it both ON and
OFF (when setting it globally). Here's a sample that tests both session and
global behaviour:

--enable_connect_log

SELECT @@expensive_subquery_limit;
SET @@session.expensive_subquery_limit=300;
SELECT @@expensive_subquery_limit;

create user user1;
create user user2;
--connect (con1, localhost, user1,,)
SELECT @@expensive_subquery_limit;
SET @@session.expensive_subquery_limit=300;
SELECT @@expensive_subquery_limit;

connection default;

SELECT @@expensive_subquery_limit;
SET @@global.expensive_subquery_limit=400;
SELECT @@expensive_subquery_limit;

--connect (con2, localhost, user1,,)
SELECT @@expensive_subquery_limit;


connection default;
drop user user1;
drop user user2;



SELECT @@expensive_subquery_limit;
@@expensive_subquery_limit
100
SET @@session.expensive_subquery_limit=300;
SELECT @@expensive_subquery_limit;
@@expensive_subquery_limit
300
create user user1;
create user user2;
connect  con1, localhost, user1,,;
SELECT @@expensive_subquery_limit;
@@expensive_subquery_limit
100
SET @@session.expensive_subquery_limit=300;
SELECT @@expensive_subquery_limit;
@@expensive_subquery_limit
300
connection default;
SELECT @@expensive_subquery_limit;
@@expensive_subquery_limit
300
SET @@global.expensive_subquery_limit=400;
SELECT @@expensive_subquery_limit;
@@expensive_subquery_limit
300
connect  con2, localhost, user1,,;
SELECT @@expensive_subquery_limit;
@@expensive_subquery_limit
400
connection default;
drop user user1;
drop user user2;

Notice how the last connection has the default value 400 as that's the
global value. For your use case, you'd also have to execute the prepared
statements to see if it works (execute select_stmt).



> diff --git a/mysql-test/suite/galera/t/galera_var_dirty_reads.test
> b/mysql-test/suite/galera/t/galera_var_dirty_reads.test
> index dd7bc88..44931aa 100644
> --- a/mysql-test/suite/galera/t/galera_var_dirty_reads.test
> +++ b/mysql-test/suite/galera/t/galera_var_dirty_reads.test
> @@ -44,7 +44,30 @@ SET @@session.wsrep_dirty_reads=OFF;
>  SELECT 2;
>  SELECT @@max_allowed_packet;
>  SELECT 2+2 from DUAL;
> -SELECT sysdate() from DUAL;
> +
> +#Query which does not change data should be allowed MDEV-11479
> +SET @@session.wsrep_dirty_reads=ON;
> +SELECT * FROM t1;
> +
> 

Re: [Maria-developers] MDEV-11330 Split Item_func_hybrid_field_type::val_xxx() into methods in Type_handler

[Vicențiu Ciorbaru]

Hi Alexander!

I find this version much cleaner! Looks good to me.
What I'd really like to see later is if we can write some unit tests for
these conversions. Not mtr linked tests but actual unit tests specifically
using these Items. I am up for chipping in and writing some myself.

One thing I'd like to discuss later in the context of reentrant functions
are these side effects such as "return (null_value= true);" when fetching
data from an Item. I think they should go from base items and only perhaps
happen in "cache" items. We can potentially then turn all these methods to
be const methods.

Vicențiu

On Thu, 1 Dec 2016 at 15:18 Alexander Barkov  wrote:

> Hello Vicențiu,
>
> Please review a modified version.
>
> As agreed, I'm not using the cast approach for the built-in types
> and define a set of data type conversion methods
> directly Type_handler_hybrid_field_type methods instead.
>
> Thanks!
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Deleting unused branches on github

[Vicențiu Ciorbaru]

Hi!

I'm wondering if it makes sense to have an automatic cleanup process for
branches older than X months?

We can potentially add some exceptions for work that needs to stay.

Regards,
Vicențiu

On Mon, 7 Nov 2016 at 18:54 Sergey Vojtovich  wrote:

> Hi!
>
> ALL BRANCH OWNERS: PLEASE REVIEW AND CLEANUP YOUR STALE BRANCHES AT
> https://github.com/MariaDB/server/branches/yours
>
> IT SHOULDN'T TAKE MORE THAN A FEW MINUTES!
>
> On Mon, Nov 07, 2016 at 04:59:44PM +0100, Kristian Nielsen wrote:
> ...skip...
>
> > Right, that was the intention of my mail, Cc:'ing the top committer in
> each
> > branch (sorry I seem to have forgotten to Cc: you).
> >
> > Or do you have a better way to determine who is the branch owner of a
> > particular branch?
> Top committer is fine.
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] aa9bd40: MDEV-10824 - Crash in CREATE OR REPLACE TABLE t1 AS SELECT spfunc()

[Vicențiu Ciorbaru]

Hi Sergey!

I think you commited the AAA.test file by mystake.

Regards,
Vicentiu

On Mon, 24 Oct 2016 at 13:47 Sergey Vojtovich  wrote:

> revision-id: aa9bd40f8067e1421ad71d2ada367544a6db78ca
> (mariadb-10.0.27-8-gaa9bd40)
> parent(s): 4dfb6a3f54cfb26535636197cc5fa70fe5bacc2e
> committer: Sergey Vojtovich
> timestamp: 2016-10-24 15:26:11 +0400
> message:
>
> MDEV-10824 - Crash in CREATE OR REPLACE TABLE t1 AS SELECT spfunc()
>
> Code flow hit incorrect branch while closing table instances before
> removal.
> This branch expects thread to hold open table instance, whereas CREATE OR
> REPLACE doesn't actually hold open table instance.
>
> Before CREATE OR REPLACE TABLE it was impossible to hit this condition in
> LTM_PRELOCKED mode, thus the problem didn't expose itself during DROP TABLE
> or DROP DATABASE.
>
> Fixed by adjusting condition to take into account LTM_PRELOCKED mode,
> which can
> be set during CREATE OR REPLACE TABLE.
>
> ---
>  mysql-test/r/create_or_replace.result | 11 +++
>  mysql-test/t/AAA.test | 24 
>  mysql-test/t/create_or_replace.test   | 12 
>  sql/sql_parse.cc  | 12 
>  sql/sql_table.cc  |  3 ++-
>  5 files changed, 49 insertions(+), 13 deletions(-)
>
> diff --git a/mysql-test/r/create_or_replace.result
> b/mysql-test/r/create_or_replace.result
> index 3a894e9..a43dc2e 100644
> --- a/mysql-test/r/create_or_replace.result
> +++ b/mysql-test/r/create_or_replace.result
> @@ -442,3 +442,14 @@ KILL QUERY con_id;
>  ERROR 70100: Query execution was interrupted
>  drop table t1;
>  DROP TABLE t2;
> +#
> +# MDEV-10824 - Crash in CREATE OR REPLACE TABLE t1 AS SELECT spfunc()
> +#
> +CREATE TABLE t1(a INT);
> +CREATE FUNCTION f1() RETURNS VARCHAR(16383) RETURN 'test';
> +CREATE OR REPLACE TABLE t1 AS SELECT f1();
> +LOCK TABLE t1 WRITE;
> +CREATE OR REPLACE TABLE t1 AS SELECT f1();
> +UNLOCK TABLES;
> +DROP FUNCTION f1;
> +DROP TABLE t1;
> diff --git a/mysql-test/t/AAA.test b/mysql-test/t/AAA.test
> new file mode 100644
> index 000..22e22dd
> --- /dev/null
> +++ b/mysql-test/t/AAA.test
> @@ -0,0 +1,24 @@
> +CREATE TABLE t1(a INT);
> +DELIMITER $$;
> +CREATE FUNCTION f2() RETURNS VARCHAR(16383) RETURN 'test';
> +CREATE FUNCTION f1() RETURNS VARCHAR(16383)
> +BEGIN
> +  INSERT INTO t1 VALUES(1);
> +  RETURN 'test';
> +END;
> +$$
> +CREATE PROCEDURE p1() CREATE OR REPLACE TABLE t1 AS SELECT f2();
> +$$
> +DELIMITER ;$$
> +
> +CALL p1;
> +
> +#CREATE OR REPLACE TABLE t1 AS SELECT f1();
> +LOCK TABLE t1 WRITE;
> +#CREATE OR REPLACE TABLE t1 AS SELECT f1();
> +UNLOCK TABLES;
> +
> +DROP PROCEDURE p1;
> +DROP FUNCTION f1;
> +DROP FUNCTION f2;
> +DROP TABLE t1;
> diff --git a/mysql-test/t/create_or_replace.test
> b/mysql-test/t/create_or_replace.test
> index 7bba2b3..b37417f 100644
> --- a/mysql-test/t/create_or_replace.test
> +++ b/mysql-test/t/create_or_replace.test
> @@ -386,3 +386,15 @@ drop table t1;
>  # Cleanup
>  #
>  DROP TABLE t2;
> +
> +--echo #
> +--echo # MDEV-10824 - Crash in CREATE OR REPLACE TABLE t1 AS SELECT
> spfunc()
> +--echo #
> +CREATE TABLE t1(a INT);
> +CREATE FUNCTION f1() RETURNS VARCHAR(16383) RETURN 'test';
> +CREATE OR REPLACE TABLE t1 AS SELECT f1();
> +LOCK TABLE t1 WRITE;
> +CREATE OR REPLACE TABLE t1 AS SELECT f1();
> +UNLOCK TABLES;
> +DROP FUNCTION f1;
> +DROP TABLE t1;
> diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc
> index cbf723c..70511fc 100644
> --- a/sql/sql_parse.cc
> +++ b/sql/sql_parse.cc
> @@ -2858,12 +2858,6 @@ case SQLCOM_PREPARE:
>  }
>
>  /*
> -  For CREATE TABLE we should not open the table even if it exists.
> -  If the table exists, we should either not create it or replace it
> -*/
> -lex->query_tables->open_strategy= TABLE_LIST::OPEN_STUB;
> -
> -/*
>If we are a slave, we should add OR REPLACE if we don't have
>IF EXISTS. This will help a slave to recover from
>CREATE TABLE OR EXISTS failures by dropping the table and
> @@ -8225,12 +8219,6 @@ bool create_table_precheck(THD *thd, TABLE_LIST
> *tables,
>if (check_fk_parent_table_access(thd, >create_info,
> >alter_info, create_table->db))
>  goto err;
>
> -  /*
> -For CREATE TABLE we should not open the table even if it exists.
> -If the table exists, we should either not create it or replace it
> -  */
> -  lex->query_tables->open_strategy= TABLE_LIST::OPEN_STUB;
> -
>error= FALSE;
>
>  err:
> diff --git a/sql/sql_table.cc b/sql/sql_table.cc
> index 7cf31ee..050a338 100644
> --- a/sql/sql_table.cc
> +++ b/sql/sql_table.cc
> @@ -2464,7 +2464,8 @@ int mysql_rm_table_no_locks(THD *thd, TABLE_LIST
> *tables, bool if_exists,
>if (table_type && table_type != view_pseudo_hton)
>  ha_lock_engine(thd, table_type);
>
> -  if (thd->locked_tables_mode)
> +  if (thd->locked_tables_mode == LTM_LOCK_TABLES ||
> +  thd->locked_tables_mode == LTM_PRELOCKED_UNDER_LOCK_TABLES)
> 

Re: [Maria-developers] Error building mariadb-10.0 with Clang 3.8: Unqualified lookup in templates

[Vicențiu Ciorbaru]

Hi Luke!

This is a know problem that will be fixed in the upcoming release.

https://jira.mariadb.org/browse/MDEV-5944

Regards,
Vicențiu

On Thu, 13 Oct 2016 at 09:16 Luke Benes  wrote:

> Building mariadb-10.0 with Clang results in:
>
> graph_concepts.hpp:93:17: error: call to function 'out_edges' that is
> neither visible in the template definition nor found by argument-dependent
> lookup
>
> Full log:
>
> http://clang.debian.net/logs/2016-08-30/mariadb-10.0_10.0.26-3_unstable_clang.log
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Sachin weekly report

[Vicențiu Ciorbaru]

Hi Sachim,

Sergei's suggestion with STRING_WITH_LEN macro or sizeof("string") should
fix the problem you're raising.

Regards,
Vicentiu

On Fri, 26 Aug 2016 at 18:11 Sachin Setia <sachinsetia1...@gmail.com> wrote:

> Hi Vicențiu
>
> Thanks Vicențiu for your comment. Although I agree with you but defining
> #define HA_HASH_STR_LEN 4
> or
> const int HA_HASH_STR_LEN = 4;
>
> make it independent of length of "hash".  Although we rarely gone
> change "hash", so I think it
> is a good idea. What do you think , Sergei?
> Regards
> sachin
>
> On Fri, Aug 26, 2016 at 7:26 PM, Vicențiu Ciorbaru <vicen...@mariadb.org>
> wrote:
> > Hi Sachin, Sergei!
> >
> > One quick thing I wanted to point out. I did not specifically look at how
> > things get called, but,
> > when defining constants, I don't agree with:
> >
> >> > +#define HA_HASH_STR_LEN strlen(HA_HASH_STR)
> >
> > Or:
> >>
> >> > +#define HA_HASH_STR_INDEX_LEN   strlen(HA_HASH_STR_INDEX)
> >
> >
> > This hides an underlying strlen. Better make it a real constant value.
> > Perhaps the compiler is smart enough to optimize it away, but why risk
> it?
> >
> > Another one is why not define them as const char * and const int? This
> also
> > helps during debugging, as you can do:
> >
> > (gdb) $ print HA_HAST_STR_INDEX_LEN
> >
> > I know that a lot of the code makes use of defines with #define, but why
> not
> > enforce a bit of type safety while we're at it?
> >
> > Just my 2 cents, feel free to disagree. :)
> > Vicentiu
> >
> >
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] c56c966: MDEV-10563 Crash during shutdown in Master_info_index::any_slave_sql_running

[Vicențiu Ciorbaru]

Hi Kristian,

Thanks for the quick and thorough reply.

As to the patch, you're basically moving the check for NULL into all the
> callers. I'm not sure if the check is even needed for all callers, but
> then,
> I also do not think it's worth it to try and determine where it is needed
> and where not.
>

Looking at the code, I am about 90% certain that we do need the checks.
Most of the code paths can execute with the pointer being null.


> But wouldn't it be nicer to instead change the
> give_error_if_slave_running()
> to be a normal function that takes the Master_info_index pointer as an
> argument (or a static method, if you want)? This way, you can keep the
> null-pointer check in the function. Also helps avoid a new bug if another
> caller is added.
>

The reason why I'm asking this about the return TRUE value is that it is
something that Sergei wrote not too long ago (if 2 years can be considered
not "too long" ago). I see how converting that to a function would be able
to make us implement the guards better, but I feel that the semantics of it
are best left as a member function. I would, however, wrap the null pointer
check.

Personally, I would've allocated this object as a global variable, adding
the pesky LOCK code inside it and hiding it. I would have its members be
pointers instead. That's a bit too much work for little gain however.

I am inclined to push it _as is_ for now, but I'll wait for Sergei's
opinion as well (he's on vacation right now, so it might take a bit).

Thanks,
Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 5486f34: MDEV-10341: InnoDB: Failing assertion: mutex_own(mutex) - mutex_exit_func

[Vicențiu Ciorbaru]

Hi Sergey,

Thanks for the review. I would push the fix into 5.5 if there are no
buildbot failures. There seems to be something wrong with solaris though.
Need to #ifdef the __sync_synchronize() there. The patch generally applies
to 5.5 as well.

Since we're having a 5.5 week next, I'll test it for 5.5 as well, then push.

Vicentiu

On Tue, 2 Aug 2016 at 13:47 Sergey Vojtovich <s...@mariadb.org> wrote:

> Hi Vicentiu,
>
> Ok to push. This patch is against 10.2, do you plan to fix previous
> versions
> separately?
>
> On Mon, Aug 01, 2016 at 02:46:03PM +0300, vicentiu wrote:
> > revision-id: 5486f3458859fa4bd161150b9ac0e9c0f633a0e3
> (mariadb-10.2.1-9-g5486f34)
> > parent(s): 08683a726773f8cdf16a4a3dfb3920e5f7842481
> > author: Vicențiu Ciorbaru
> > committer: Vicențiu Ciorbaru
> > timestamp: 2016-08-01 14:43:41 +0300
> > message:
> >
> > MDEV-10341: InnoDB: Failing assertion: mutex_own(mutex) - mutex_exit_func
> >
> > Fix memory barrier issues on releasing mutexes. We must have
> Looks like a cutted comment.
>
> >
> > ---
> >  storage/innobase/include/os0sync.h| 17 -
> >  storage/innobase/include/sync0sync.ic |  2 ++
> >  storage/xtradb/include/os0sync.h  | 16 
> >  storage/xtradb/include/sync0sync.ic   |  2 ++
> >  4 files changed, 20 insertions(+), 17 deletions(-)
>
> Thanks,
> Sergey
>
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Aggregate Stored Functions

[Vicențiu Ciorbaru]

t;>>>>>>| AGGREGSTE_SYM option
>>>>>>>>>> option:
>>>>>>>>>>| YES
>>>>>>>>>>| NO
>>>>>>>>>>
>>>>>>>>>> or option could be any string.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jun 3, 2016 at 1:22 PM, Varun Gupta <
>>>>>>>>>> varungupta1...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> The error is fixed , now show works for aggregate functions too
>>>>>>>>>>> :)
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jun 3, 2016 at 1:01 PM, Varun Gupta <
>>>>>>>>>>> varungupta1...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> For non aggregate function , show output is correct and when I
>>>>>>>>>>>> run the query SELECT * from mysql.proc table, aggregate field 
>>>>>>>>>>>> shows NO.
>>>>>>>>>>>> For aggregate function, when I run the query SELECT * from
>>>>>>>>>>>> mysql.proc table, aggregate field shows YES.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jun 3, 2016 at 12:56 PM, Varun Gupta <
>>>>>>>>>>>> varungupta1...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes it does, but I am not adding anything to the buffer in
>>>>>>>>>>>>> that case.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jun 3, 2016 at 12:53 PM, Sanja <
>>>>>>>>>>>>> sanja.byel...@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Could you be more specific? what test?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Does your code change SHOW for non-aggregate function?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jun 3, 2016 at 9:20 AM, Varun Gupta <
>>>>>>>>>>>>>> varungupta1...@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>> To solve the problem with the buffer ,can you tell me is
>>>>>>>>>>>>>>> there any tests that are run before I run my own tests .
>>>>>>>>>>>>>>> I meant if I change the value of
>>>>>>>>>>>>>>> the buf->append(STRING_WITH_LEN("FUNCTION "));
>>>>>>>>>>>>>>> to buf->append(STRING_WITH_LEN("pUNCTION ")); , even then i get 
>>>>>>>>>>>>>>> the same
>>>>>>>>>>>>>>> error.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jun 3, 2016 at 1:32 AM, Varun Gupta <
>>>>>>>>>>>>>>> varungupta1...@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Have you gone through the patch. Do you like that I have
>>>>>>>>>>>>>>>> added a field agg_res to the functions ? Or should I find some 
>>>>>>>>>>>>>>>> other to
>>>>>>>>>>>>>>>> figure it out .
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Jun 2, 2016 at 10:59 PM, Varun Gupta <
>>>>>>>>>>>>>>>> varungupta1...@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> patch commited .
>>>>>>>>>>>>>>>&

[Maria-developers] Please review MDEV-9497 oqgraph fails to build with boost 1.60

[Vicențiu Ciorbaru]

Hi Sergei!


Can you please review the following patch for MariaDB 10.0. It fixes a
compilation failure in Debian/Ubuntu when boost 1.60 or later is installed.
I've looked into the code and checked if the #ifdef'ed code is used
anywhere. There is absolutely no reference to it. I doubt it affect
anything.


Thanks!
Vicentiu

+--- mariadb-10.0-10.0.25.orig/storage/oqgraph/oqgraph_shim.h
 mariadb-10.0-10.0.25/storage/oqgraph/oqgraph_shim.h
+@@ -254,7 +254,7 @@ namespace boost
- #if BOOST_VERSION >= 104601
+ #if BOOST_VERSION >= 104601 && BOOST_VERSION < 106000
 template <>
 struct graph_bundle_type
 {
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Aggregate Stored Functions

[Vicențiu Ciorbaru]

Hi Varun,

I've reviewed your patch. Looks good from my side. Just stylistic comments.
Feel free to keep your own version if you don't agree with them.

I think that you could have used the m_flags field, but having a specific
member makes things a lot clearer in my opinion. Perhaps Sanja has a
different opinion.

Next step is to figure out how to use this new flag from sp_head in the
execution part. If you get completely stuck, let us know. :)

Vicentiu

On Tue, 24 May 2016 at 11:30 Sanja <sanja.byel...@gmail.com> wrote:

> Yes, the decision is right. I'll check later the code on github.
>
> On Tue, May 24, 2016 at 10:27 AM, Varun Gupta <varungupta1...@gmail.com>
> wrote:
>
>> Hi,
>> I had been going through the LEX struct and could not find any flag
>> member there which could be used to specify if a function is aggregate or
>> not. So i created the new flag inside sp_head, so as to make sure it could
>> be used for stored procedures too in the future.
>> I have committed the changes on GitHub :)
>>
>> On Mon, May 23, 2016 at 4:21 PM, Vicențiu Ciorbaru <cvicen...@gmail.com>
>> wrote:
>>
>>> Hi Varun,
>>>
>>> Getting the parser to accept the syntax is a good first step. Writing
>>> tests is the correct way to go also.
>>>
>>> Now we need to have a way to pass this extra information to the part of
>>> the code that stores / executes this procedure. When we encounter this
>>> AGGREGATE_SYM syntax we have to record it somewhere. We generally use the
>>> LEX structure for this. See if there is any flag member within it that you
>>> can use for this purpose. If you can't find any, you can potentially create
>>> one yourself.
>>>
>>> Now, it would be a good time to try and familiarize yourself with how we
>>> get from having a regular parsed function to storing it and afterwards
>>> executing it. This is the main logic that we have to deal with. I'm not
>>> going to suggest you any specific thing to do right now as there are
>>> multiple ways to do this. Try and come up with a simple plan on how to
>>> extend this functionality for our use case. You don't have to code it all,
>>> just yet :). We'll improve (or perhaps change it) afterwards. It doesn't
>>> have to be perfect the first time, but this way you'll get a try at
>>> designing an implementation idea.
>>>
>>> Great job so far!
>>> Vicentiu
>>>
>>> On Mon, 23 May 2016 at 09:04 Varun Gupta <varungupta1...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>> As in my previous mail I have added the FETCH statement to the parser
>>>> and have tested it, when the syntax is correct . Now I am writing test that
>>>> would also give an error for incorrect syntax. Also I would like how to
>>>> proceed further :).
>>>> ___
>>>> Mailing list: https://launchpad.net/~maria-developers
>>>> Post to : maria-developers@lists.launchpad.net
>>>> Unsubscribe : https://launchpad.net/~maria-developers
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>
>>
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Aggregate Stored Functions

[Vicențiu Ciorbaru]

Hi Varun,

Getting the parser to accept the syntax is a good first step. Writing tests
is the correct way to go also.

Now we need to have a way to pass this extra information to the part of the
code that stores / executes this procedure. When we encounter this
AGGREGATE_SYM syntax we have to record it somewhere. We generally use the
LEX structure for this. See if there is any flag member within it that you
can use for this purpose. If you can't find any, you can potentially create
one yourself.

Now, it would be a good time to try and familiarize yourself with how we
get from having a regular parsed function to storing it and afterwards
executing it. This is the main logic that we have to deal with. I'm not
going to suggest you any specific thing to do right now as there are
multiple ways to do this. Try and come up with a simple plan on how to
extend this functionality for our use case. You don't have to code it all,
just yet :). We'll improve (or perhaps change it) afterwards. It doesn't
have to be perfect the first time, but this way you'll get a try at
designing an implementation idea.

Great job so far!
Vicentiu

On Mon, 23 May 2016 at 09:04 Varun Gupta  wrote:

> Hi,
> As in my previous mail I have added the FETCH statement to the parser and
> have tested it, when the syntax is correct . Now I am writing test that
> would also give an error for incorrect syntax. Also I would like how to
> proceed further :).
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Aggregate Stored Functions

[Vicențiu Ciorbaru]

Hi Sanja,

Since we have not officially started the coding period, I figured that an
exercise in adding a parser syntax would be a good first step.

We can do automatic detection by looking at the cursor without to much
extra work.

Vicentiu

On Fri, 20 May 2016 at 08:29, Sanja  wrote:

> Hi!
>
> Have you read our discussion about automatic aggregate functions detection?
> Am 20.05.2016 07:15 schrieb "Varun Gupta" :
>
>> Hi,
>> I have added the AGGREGATE keyword to the parser . Here is the link to
>> the repository https://github.com/varunraiko/aggregate-functions .
>>
>>
>> On Wed, May 18, 2016 at 9:50 PM, Sanja  wrote:
>>
>>> Hi!
>>>
>>> If we get it automatically then of course it should be done, but I
>>> doubts... We will see.
>>>
>>> On Wed, May 18, 2016 at 6:18 PM, Sergei Golubchik 
>>> wrote:
>>>
 Hi, Sanja!

 On May 18, Sanja wrote:
 > >
 > > No, sorry, this doesn't work. It works for procedures, but not for
 > > functions. See:
 > >
 > > CREATE FUNCTION f1 (a INT) RETURNS INT
 > > BEGIN
 > >   RETURN SELECT f2(val) FROM t1 WHERE id=a;
 > > END;
 > >
 > > CREATE FUNCTION f2 (b INT) RETURNS INT
 > > BEGIN
 > >   ...
 > >   FETCH GROUP NEXT ROW;
 > >   ...
 > >   RETURN something;
 > > END;
 > >
 > > Here, depending on what function is declared aggregate you will have
 > > different results.
 >
 > I think in the first implementation we can have only one level
 > functions.  if we will have time we can then expand it for calls of
 > other functions.  But first the mechanism of temporary leaving then
 > entering functions should be created (then it can be reused for
 > recursive calls.

 First implementation - may be (althought I don't understand why - this
 requires no extra coding, nested function calls will just work
 automatically). But the first implementation should not choose to use
 the syntax that makes this extension impossible.

 For example, Varun's project does not include window function support.
 At all. But we will be able to add it later without redoing everything,
 exising syntax can accomodate this new feature.

 Regards,
 Sergei
 Chief Architect MariaDB
 and secur...@mariadb.org

>>>
>>>
>>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] b1ddc7d: MDEV-9487: Server crashes in Time_and_counter_tracker::incr_loops with UNION in ALL subquery

[Vicențiu Ciorbaru]

Hi Sanja, Sergey,

I think Sergey might have mean the same thing but it is not clear to me.
in_optimize never gets set to true. By mentioning that it is redundant that
means that it can be removed from the current code right?

Also, I would put the "return true" conditions after the "return false"
conditions. Makes the code slightly easier to follow.

Vicentiu

On Wed, 4 May 2016 at 18:59 Sergey Petrunia  wrote:

> Hi Sanja,
>
> I have one comment: the patch introduces JOIN::in_optimize which can have
> values of 1) not initialized and 2) false. This is clearly redundant.
>
> Ok to push after the above is addressed.
>
> On Sun, Feb 21, 2016 at 10:12:25PM +0100, OleksandrByelkin wrote:
> > revision-id: b1ddc7d546e6b147838af72dd03f86a8b272fdf0
> (mariadb-10.1.11-18-gb1ddc7d)
> > parent(s): fd8e846a3b049903706267d58e6d8e61eea97df8
> > committer: Oleksandr Byelkin
> > timestamp: 2016-02-21 22:12:25 +0100
> > message:
> >
> > MDEV-9487: Server crashes in Time_and_counter_tracker::incr_loops with
> UNION in ALL subquery
> >
> > Do not mark subquery as inexpensive when it is not optimized.
> >
> > ---
> >  mysql-test/r/derived_view.result  |  6 +++---
> >  mysql-test/r/subselect.result | 20 +---
> >  mysql-test/r/subselect_no_exists_to_in.result | 20 +---
> >  mysql-test/r/subselect_no_mat.result  | 20 +---
> >  mysql-test/r/subselect_no_opts.result | 20 +---
> >  mysql-test/r/subselect_no_scache.result   | 20 +---
> >  mysql-test/r/subselect_no_semijoin.result | 20 +---
> >  mysql-test/r/type_year.result |  1 +
> >  mysql-test/t/subselect.test   | 11 +++
> >  sql/item_subselect.cc | 20 
> >  sql/sql_select.h  |  2 ++
> >  11 files changed, 135 insertions(+), 25 deletions(-)
> >
> > diff --git a/mysql-test/r/derived_view.result
> b/mysql-test/r/derived_view.result
> > index 639942f..5783247 100644
> > --- a/mysql-test/r/derived_view.result
> > +++ b/mysql-test/r/derived_view.result
> > @@ -1101,7 +1101,7 @@ id  select_type table   type
> possible_keys   key key_len ref rowsfilteredExtra
> >  1PRIMARY t3  system  NULLNULLNULLNULL1
>  100.00
> >  2SUBQUERYNULLNULLNULLNULLNULLNULL
> NULLNULLNo tables used
> >  Warnings:
> > -Note 1003select 6 AS `a`,5 AS `b` from `test`.`t3` where
> (not(<6,5>(((6,5),(select 7,5 having
> (trigcond6) = 7) or isnull(7))) and trigcond5) = 5)
> or isnull(5))) and trigcond((7)) and
> trigcond((5
> > +Note 1003select 6 AS `a`,5 AS `b` from `test`.`t3` where 1
> >  SELECT t.a,t.b FROM t3 RIGHT JOIN ((SELECT * FROM t1) AS t, t2) ON t2.b
> != 0
> >  WHERE (t.a,t.b) NOT IN (SELECT 7, 5);
> >  ab
> > @@ -1115,7 +1115,7 @@ id  select_type table   type
> possible_keys   key key_len ref rowsfilteredExtra
> >  1PRIMARY t3  system  NULLNULLNULLNULL1
>  100.00
> >  3SUBQUERYNULLNULLNULLNULLNULLNULL
> NULLNULLNo tables used
> >  Warnings:
> > -Note 1003select 6 AS `a`,5 AS `b` from `test`.`t3` where
> (not(<6,5>(((6,5),(select 7,5 having
> (trigcond6) = 7) or isnull(7))) and trigcond5) = 5)
> or isnull(5))) and trigcond((7)) and
> trigcond((5
> > +Note 1003select 6 AS `a`,5 AS `b` from `test`.`t3` where 1
> >  SELECT t.a,t.b FROM t3 RIGHT JOIN (v1 AS t, t2) ON t2.b != 0
> >  WHERE (t.a,t.b) NOT IN (SELECT 7, 5);
> >  ab
> > @@ -1129,7 +1129,7 @@ id  select_type table   type
> possible_keys   key key_len ref rowsfilteredExtra
> >  1PRIMARY t3  system  NULLNULLNULLNULL1
>  100.00
> >  2SUBQUERYNULLNULLNULLNULLNULLNULL
> NULLNULLNo tables used
> >  Warnings:
> > -Note 1003select 6 AS `a`,5 AS `b` from `test`.`t3` where
> (not(<6,5>(((6,5),(select 7,5 having
> (trigcond6) = 7) or isnull(7))) and trigcond5) = 5)
> or isnull(5))) and trigcond((7)) and
> trigcond((5
> > +Note 1003select 6 AS `a`,5 AS `b` from `test`.`t3` where 1
> >  DROP VIEW v1;
> >  DROP TABLE t1,t2,t3;
> >  #
> > diff --git a/mysql-test/r/subselect.result
> b/mysql-test/r/subselect.result
> > index 75c8597..90d8f9f 100644
> > --- a/mysql-test/r/subselect.result
> > +++ b/mysql-test/r/subselect.result
> > @@ -6805,7 +6805,9 @@ FROM t1 AS alias1, t1 AS alias2, t1 AS alias3
> >  WHERE alias1.a = alias2.a OR alias1.a = 'y'
> >  HAVING field>'B' AND ( 'Moscow' ) IN ( SELECT a FROM t1 );
> >  id   select_type table   typepossible_keys   key key_len
> ref rowsExtra
> > -1PRIMARY NULLNULLNULLNULLNULLNULLNULL
> Impossible HAVING
> > +1PRIMARY alias1  index   a   a   

Re: [Maria-developers] [Commits] a0c06ba: Preliminary implementation for the aggregate sum function as a window function

[Vicențiu Ciorbaru]

On Mon, 14 Mar 2016 at 21:43 Sergey Petrunia <ser...@mariadb.com> wrote:

> Hi Vicentiu,
>
> I think it add_helper() is a really poor choice of name when the function
> can
> do removal.
> Maybe add_or_remove() name would be better?
>

Agreed.


> On Mon, Mar 14, 2016 at 03:44:48PM +0200, Vicentiu Ciorbaru wrote:
> > revision-id: a0c06ba1edb54c8c4705189c0455137a85658297
> (mariadb-10.1.8-154-ga0c06ba)
> > parent(s): ce8a0d8e19e7bf1486a2c80ff6b7a30ef35bf99f
> > author: Vicențiu Ciorbaru
> > committer: Vicențiu Ciorbaru
> > timestamp: 2016-03-14 15:42:00 +0200
> > message:
> >
> > Preliminary implementation for the aggregate sum function as a window
> function
> >
> > This implementation does not deal with the case where removal of
> > elements from the window frame causes the item to turn to a null value.
> >
> > ---
> >  mysql-test/r/win_sum.result | 42
> ++
> >  mysql-test/t/win_sum.test   | 32 
> >  sql/item_sum.cc | 29 +
> >  sql/item_sum.h  |  4 
> >  sql/sql_window.cc   |  1 +
> >  5 files changed, 104 insertions(+), 4 deletions(-)
> >
> > diff --git a/mysql-test/r/win_sum.result b/mysql-test/r/win_sum.result
> > new file mode 100644
> > index 000..1db6c6e
> > --- /dev/null
> > +++ b/mysql-test/r/win_sum.result
> > @@ -0,0 +1,42 @@
> > +create table t1 (
> > +pk int primary key,
> > +a int,
> > +b int,
> > +c real
> > +);
> > +insert into t1 values
> > +(101 , 0, 10, 1.1),
> > +(102 , 0, 10, 2.1),
> > +(103 , 1, 10, 3.1),
> > +(104 , 1, 10, 4.1),
> > +(108 , 2, 10, 5.1),
> > +(105 , 2, 20, 6.1),
> > +(106 , 2, 20, 7.1),
> > +(107 , 2, 20, 8.15),
> > +(109 , 4, 20, 9.15),
> > +(110 , 4, 20, 10.15),
> > +(111 , 5, NULL, 11.15),
> > +(112 , 5, 1, 12.25),
> > +(113 , 5, NULL, 13.35),
> > +(114 , 5, NULL, 14.50),
> > +(115 , 5, NULL, 15.65);
> > +select pk, a, b, sum(b) over (partition by a order by pk ROWS BETWEEN 1
> PRECEDING AND 1 FOLLOWING),
> > +sum(c) over (partition by a order by pk ROWS BETWEEN 1 PRECEDING AND 1
> FOLLOWING)
> > +from t1;
> > +pk   a   b   sum(b) over (partition by a order by pk ROWS
> BETWEEN 1 PRECEDING AND 1 FOLLOWING)   sum(c) over (partition by a
> order by pk ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING)
> > +101  0   10  20  3.2
> > +102  0   10  20  3.2
> > +103  1   10  20  7.199
> > +104  1   10  20  7.199
> > +105  2   20  40  13.2
> > +106  2   20  60  21.35
> > +107  2   20  50  20.35
> > +108  2   10  30  13.252
> > +109  4   20  40  19.3
> > +110  4   20  40  19.3
> > +111  5   NULL1   23.4
> > +112  5   1   1   36.75
> > +113  5   NULL1   40.1
> > +114  5   NULL0   43.5
> > +115  5   NULL0   30.15
> > +drop table t1;
> > diff --git a/mysql-test/t/win_sum.test b/mysql-test/t/win_sum.test
> > new file mode 100644
> > index 000..3c12b08
> > --- /dev/null
> > +++ b/mysql-test/t/win_sum.test
> > @@ -0,0 +1,32 @@
> > +create table t1 (
> > +  pk int primary key,
> > +  a int,
> > +  b int,
> > +  c real
> > +);
> > +
> > +
> > +insert into t1 values
> > +(101 , 0, 10, 1.1),
> > +(102 , 0, 10, 2.1),
> > +(103 , 1, 10, 3.1),
> > +(104 , 1, 10, 4.1),
> > +(108 , 2, 10, 5.1),
> > +(105 , 2, 20, 6.1),
> > +(106 , 2, 20, 7.1),
> > +(107 , 2, 20, 8.15),
> > +(109 , 4, 20, 9.15),
> > +(110 , 4, 20, 10.15),
> > +(111 , 5, NULL, 11.15),
> > +(112 , 5, 1, 12.25),
> > +(113 , 5, NULL, 13.35),
> > +(114 , 5, NULL, 14.50),
> > +(115 , 5, NULL, 15.65);
> > +
> > +--sorted_result
> > +select pk, a, b, sum(b) over (partition by a order by pk ROWS BETWEEN 1
> PRECEDING AND 1 FOLLOWING),
> > + sum(c) over (partition by a order by pk ROWS BETWEEN 1
> PRECEDING AND 1 FOLLOWING)
> > +
> > +from t1;
> > +
> > +drop table t1;
> > diff --git a/sql/item_sum.cc b/sql/item_sum.cc
> > index c78c206..3f4853a 100644
> > --- a/sql/item_sum.cc
> > +++ b/sql/item_sum.cc
> > @@ -1318,25 +1318,39 @@ void Item_sum_sum::fix_length_and_dec()
> >  bool Item_sum_sum::add()
> >  {
> >DBUG_ENTER("It

[Maria-developers] Invalid code in server_audit.c

[Vicențiu Ciorbaru]

Hi Alexei,

While investigating community contributions from Daniel Black (MDEV-9433),
I came across this comment:

[image: danblack]Daniel Black
 added
a comment - 2016-01-20 02:54

[plugin/server_audit/server_audit.c:402]: (error) Invalid number of
character '{' when these macros are defined:
'DBUG_VOID_RETURN=return;DBUG_RETURN(a)=return a;HAVE_PSI_INTERFACE'.
[plugin/server_audit/server_audit.c:402]: (error) Invalid number of
character '{' when these macros are defined:
'DBUG_VOID_RETURN=return;DBUG_RETURN(a)=return
a;HAVE_PSI_INTERFACE;HUGETLB_USE_PROC_MEMINFO'.
[plugin/server_audit/server_audit.c:402]: (error) Invalid number of
character '{' when these macros are defined:
'DBUG_VOID_RETURN=return;DBUG_RETURN(a)=return
a;HAVE_PSI_INTERFACE;USE_ALARM_THREAD'.

Exists in 5.5
https://github.com/MariaDB/server/blob/5.5/plugin/server_audit/server_audit.c#L541

Remove second {{

I've had a look at the code mentioned there and key_LOCK_bigbuffer does not
exist anywhere else. Since we're not getting any compilation issues, it
means we're not compiling that bit of code due to the defines. This indeed
is so, as MYSQL_DYNAMIC_PLUGIN is defined for that file on the default
cmake config which leads to FLOGGER being defined.

I could not figure out a way to actually compile the code with defines such
that that bit of code actually compiles. Either way it would not compile,
due to the missing symbol. Do you think that we should just remove the line
that mentions key_LOCK_bigbuffer? Also, since this is something we have not
caught during testing, can we test for this compilation configuration
somehow?

Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Please review MDEV-8092 Change Create_field::field from "const char *" to LEX_CSTRING

[Vicențiu Ciorbaru]

Hi Alexander,

Thanks for the explanation. I agree with your points regarding std::string.
Although it is possible to control allocation using an allocator, it does
indeed make sense that std::string is not easily applicable in this case.

I am personally for the introduction of an extra string class if it serves
our purpose better. However we should strive to make the interfaces for our
classes more uniform, as it quickly becomes confusing which operations are
supported and which are not. The classes at hand seem reasonable in this
regard. I think I would override the equality operator also.

Vicentiu


On Fri, 8 Jan 2016 at 09:36, Alexander Barkov <b...@mariadb.org> wrote:

> Hi Vicențiu,
>
>
> On 01/08/2016 02:48 AM, Vicențiu Ciorbaru wrote:
> > Hi Alexander, Sergei,
> >
> > On Fri, 8 Jan 2016 at 00:10 Alexander Barkov <b...@mariadb.org
> > <mailto:b...@mariadb.org>> wrote:
> >
> > Hi Sergei,
> >
> >
> > On 01/07/2016 08:53 PM, Sergei Golubchik wrote:
> >  > Hi, Alexander!
> >  >
> >  > On Nov 27, Alexander Barkov wrote:
> >  >> Hi Serg,
> >  >>
> >  >> Please review a patch for MDEV-8092.
> >  >
> >  > I didn't review a lot of it, sorry. Because my first two comments
> > - if
> >  > we agree on them - might cause quite a few changes
> > (search/replace kind
> >  > of changes, but still). So I'd better look at the whole patch
> after
> >  > that.
> >
> > Sure. Thanks.
> >
> > Please see below.
> >
> >  >
> >  >> diff --git a/sql/field.h b/sql/field.h
> >  >> index 0591914..a40e307 100644
> >  >> --- a/sql/field.h
> >  >> +++ b/sql/field.h
> >  >> @@ -537,6 +537,115 @@ inline bool
> > is_temporal_type_with_time(enum_field_types type)
> >  >>   }
> >  >>
> >  >>
> >  >> +/**
> >  >> +  @class Name
> >  >> +  An arbitrary SQL name, consisting of a NULL-terminated string
> >  >> +  and its length.
> >  >> +  Later it can be reused for table and schema names.
> >  >> +*/
> >  >> +class Name
> >  >> +{
> >  >> +  const char *m_str;
> >  >> +  uint m_length;
> >  >
> >  > Sorry, we have too many "string" classes already.
> >  > Either use LEX_STRING or create a Lex_string
> >  > which is inherited from LEX_STRING, binary compatible with it
> >  > and only adds C++ methods (and can be casted back and forth
> > automatically).
> >  > So that one could use Lex_string where a LEX_STRING is expected
> > and vice
> >  > versa
> >
> > Sorry for a long reply.
> >
> > This is a very convenient moment of time when
> > we can improve a few things. I disagree to go the easier way
> > with LEX_STRING.
> >
> >
> > LEX_STRING is a very unfortunately designed thing for the server
> > side code.
> >
> > It was a kind of Ok on 32 bit platforms.
> > But now on 64bit platforms it's just a waste of memory.
> > Length can never be large enough to use the entire "size_t".
> >
> > Btw, it can even never be long enough to use "uint". It should be
> enough
> > to use uint16 for length and use the available remainder (6 bytes) to
> > store extra useful things, like string metadata.
> > It will help to avoid some operations, like character set conversion
> > in the client-server protocol, and I planned to add some optimization
> > in the future.
> >
> >
> > I propose to do the other way around:
> > - gradually get rid of using LEX_STRING and LEX_CSTRING for names
> > in the server side and use more suitable Name-alike classes instead.
> > - keep LEX_XXX only for compatibility with the existing pure C API,
> > where it already presents now.
> >
> >
> > Please note, in the current patch version the Name class has
> > constructors that can accept LEX_STRING and LEX_CSTRING.
> > This means that you can pass LEX_STRING and LEX_CSTRING to any
> > function or method that is designed to accept Name,
> > and the patch already uses this. So this switch will go very
> smoothly.
> >
> >
> > As for "too many string classes", I have heard this a few times
&

Re: [Maria-developers] Please review MDEV-8092 Change Create_field::field from "const char *" to LEX_CSTRING

[Vicențiu Ciorbaru]

Hi Alexander, Sergei,

On Fri, 8 Jan 2016 at 00:10 Alexander Barkov  wrote:

> Hi Sergei,
>
>
> On 01/07/2016 08:53 PM, Sergei Golubchik wrote:
> > Hi, Alexander!
> >
> > On Nov 27, Alexander Barkov wrote:
> >> Hi Serg,
> >>
> >> Please review a patch for MDEV-8092.
> >
> > I didn't review a lot of it, sorry. Because my first two comments - if
> > we agree on them - might cause quite a few changes (search/replace kind
> > of changes, but still). So I'd better look at the whole patch after
> > that.
>
> Sure. Thanks.
>
> Please see below.
>
> >
> >> diff --git a/sql/field.h b/sql/field.h
> >> index 0591914..a40e307 100644
> >> --- a/sql/field.h
> >> +++ b/sql/field.h
> >> @@ -537,6 +537,115 @@ inline bool
> is_temporal_type_with_time(enum_field_types type)
> >>   }
> >>
> >>
> >> +/**
> >> +  @class Name
> >> +  An arbitrary SQL name, consisting of a NULL-terminated string
> >> +  and its length.
> >> +  Later it can be reused for table and schema names.
> >> +*/
> >> +class Name
> >> +{
> >> +  const char *m_str;
> >> +  uint m_length;
> >
> > Sorry, we have too many "string" classes already.
> > Either use LEX_STRING or create a Lex_string
> > which is inherited from LEX_STRING, binary compatible with it
> > and only adds C++ methods (and can be casted back and forth
> automatically).
> > So that one could use Lex_string where a LEX_STRING is expected and vice
> > versa
>
> Sorry for a long reply.
>
> This is a very convenient moment of time when
> we can improve a few things. I disagree to go the easier way
> with LEX_STRING.
>
>
> LEX_STRING is a very unfortunately designed thing for the server side code.
>
> It was a kind of Ok on 32 bit platforms.
> But now on 64bit platforms it's just a waste of memory.
> Length can never be large enough to use the entire "size_t".
>
> Btw, it can even never be long enough to use "uint". It should be enough
> to use uint16 for length and use the available remainder (6 bytes) to
> store extra useful things, like string metadata.
> It will help to avoid some operations, like character set conversion
> in the client-server protocol, and I planned to add some optimization
> in the future.
>
>
> I propose to do the other way around:
> - gradually get rid of using LEX_STRING and LEX_CSTRING for names
> in the server side and use more suitable Name-alike classes instead.
> - keep LEX_XXX only for compatibility with the existing pure C API,
>where it already presents now.
>
>
> Please note, in the current patch version the Name class has
> constructors that can accept LEX_STRING and LEX_CSTRING.
> This means that you can pass LEX_STRING and LEX_CSTRING to any
> function or method that is designed to accept Name,
> and the patch already uses this. So this switch will go very smoothly.
>
>
> As for "too many string classes", I have heard this a few times already,
> both in MariaDB and earlier in Oracle :)
> I don't agree with this statement.
>

Hi Alexander, Sergei!

Since this discussion might be somewhat similar to a difficulty I'm facing
(see recent email on dev list "Use of std::string"), I figured I'd join in.

In that email I suggested making use of std::string instead of regular raw
pointers and length fields. From looking at the patch (mostly just the name
classes, did not investigate everything), the std::string class would work
just fine for the Name class and be very clear cut parent class for the
Column_name. Implementing an equality operator perhaps, instead of eq_name
function would improve clarity also.

The advantages of using the standard library would be that compilers
provide specific optimizations for them. At the same time, we would not be
adding an extra string API.

Regards,
Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Use of std::string

[Vicențiu Ciorbaru]

On Wed, 6 Jan 2016 at 21:48 Sergei Golubchik <s...@mariadb.org> wrote:

> Hi, Vicențiu!
>

Hi Sergei!


> On Jan 05, Vicențiu Ciorbaru wrote:
> >
> > Sample code where we can avoid ifs:
> >
> > case SSL_TYPE_SPECIFIED:
> >   table->field[next_field]->store(STRING_WITH_LEN("SPECIFIED"),
> >   _charset_latin1);
> >   table->field[next_field+1]->store("", 0, _charset_latin1);
> >   table->field[next_field+2]->store("", 0, _charset_latin1);
> >   table->field[next_field+3]->store("", 0, _charset_latin1);
> >   if (lex->ssl_cipher)
> > table->field[next_field+1]->store(lex->ssl_cipher,
> >   strlen(lex->ssl_cipher),
> >   system_charset_info);
> >   if (lex->x509_issuer)
> > table->field[next_field+2]->store(lex->x509_issuer,
> >   strlen(lex->x509_issuer),
> >   system_charset_info);
> >   if (lex->x509_subject)
> > table->field[next_field+3]->store(lex->x509_subject,
> >   strlen(lex->x509_subject),
> >   system_charset_info);
> >
> > This just becomes:
> > case SSL_TYPE_SPECIFIED:
> >   table->field[next_field]->store(STRING_WITH_LEN("SPECIFIED"),
> _charset_latin1);
> >   table->field[next_field+1]->store(ssl_cipher.c_str(),
> ssl_cipher.size(), _charset_latin1);
> >   table->field[next_field+2]->store(x509_issuer.c_str(),
> x509_issuer.size(), _charset_latin1);
> >   table->field[next_field+3]->store(x509_subject.c_str(),
> x509_subject.size(), _charset_latin1);
> >
> > Thoughts?
>
> I can immediately think of two more approaches:
>
>  1. using String class (sql/sql_string.h)
>  2. using safe_str helper:
>
>table->field[next_field+1]->store(safe_str(lex->ssl_cipher),
>  safe_strlen(lex->ssl_cipher),
>  _charset_latin1);
>
>
> As for std:string - currently the server code uses these approaches to
> storing strings: char*, LEX_STRING, DYNAMIC_STRING, CSET_STRING, String.
> With variations like uchar*, LEX_CSTRING, LEX_CUSTRING, StringBuffer,
> may be more.
>
> I don't particularly like an idea of adding yes another alternative into
> this mess without a clear rule of what to use where.
>
That's what I feel would be an issue too.


> So, could you say when should one use std::string and when one should
> stick to one of the existing types?


The place where using the std::string would be preferable, I'd say is where
we don't make use of a memroot to automatically take care of memory
allocations. This is indeed rare however.

Another interesting alternative is when we copy the string a few times. We
talked about the move semantics in Germany. Using a std::string might allow
the compiler to optimise such copies as it has optimisations specifically
tailored for std::string.

Another problem with LEX_STRING and it's derivatives is that we do not
actually enforce const-ness during compile time.

f(const LEX_STRING* s) {
s->str[1] = 'b';
}

This function compiles perfectly but the const parameter does not help in
detecting this bug. I agree that this kind of code clearly "smells", but it
is another thing we need to watch out for.

When considering ownership of a string, it is sometimes unclear if we own
the string or not and you need to check the caller to understand where the
string parameter actually comes from.

For the use-case I mentioned, we can do just fine without std::string, but
I do think that std::string could potentially replace most of our in-house
strings, with the added benefit of handling memory allocations, where
memroot might not be an option and having the extra type safety that comes
with using it.

Regards,
Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 6414959: MDEV-7526: TokuDB doesn't build on OS X

[Vicențiu Ciorbaru]

On Sun, 20 Dec 2015 at 11:44 Sergei Golubchik <s...@mariadb.org> wrote:

> Hi, Vicentiu!
>
> On Dec 19, Vicentiu Ciorbaru wrote:
> > revision-id: 64149590c47d1cf6b1b227d8c90bdc23d20a8956
> (mariadb-5.5.47-8-g6414959)
> > parent(s): f89c9fc4b7b5d82c79775cb848225900b45a6b79
> > author: Vicențiu Ciorbaru
> > committer: Vicențiu Ciorbaru
> > timestamp: 2015-12-19 14:14:10 +0200
> > message:
> >
> > MDEV-7526: TokuDB doesn't build on OS X
> >
> > This patch fixes another compilation error caused by specifying
> > attribute nonnull for all the parameters of the copyout function. This
> > is incorrect as the function actually gets called with null parameters
> > indirectly and thus only the output parameter should be nonnull.
> >
> > diff --git a/storage/tokudb/ft-index/util/dmt.h
> b/storage/tokudb/ft-index/util/dmt.h
> > index d4b032f..43e44df 100644
> > --- a/storage/tokudb/ft-index/util/dmt.h
> > +++ b/storage/tokudb/ft-index/util/dmt.h
> > @@ -679,16 +679,16 @@ class dmt {
> >  __attribute__((nonnull))
> >  void rebalance(subtree *const subtree);
> >
> > -__attribute__((nonnull))
> > +__attribute__((nonnull(3)))
> >  static void copyout(uint32_t *const outlen, dmtdata_t *const out,
> const dmt_node *const n);
>
> Where is copyout called with the null parameter?
> In 10.0 they've fixed it differently, it seems E.g.
>

The call stack that leads to null parameters for copyout:
in ft-index/util/dmt.cc

line 196:
int dmt<dmtdata_t, dmtdataout_t, dmtwriter_t>::insert(const dmtwriter_t
, const dmtcmp_t , uint32_t *const idx) {
r = this->find_zero<dmtcmp_t, h>(v, nullptr, nullptr, _idx);

line 527:
int dmt<dmtdata_t, dmtdataout_t, dmtwriter_t>::find_zero(const dmtcmp_t
, uint32_t *const value_len, dmtdataout_t *const value, uint32_t
*const idxp) const {
r = this->find_internal_zero_array<dmtcmp_t, h>(extra, value_len,
value, child_idxp);

line 966:
int dmt<dmtdata_t, dmtdataout_t,
dmtwriter_t>::find_internal_zero_array(const dmtcmp_t , uint32_t
*const value_len, dmtdataout_t *const value, uint32_t *const idxp) const

This eventually calls copyout with nullptr arguments.

if (value != nullptr) {
> copyout(value, >d.a.values[best_zero]);
> }
>
> May be we'd better use
> the upstream fix?
>

I could not find the code snippet that you are referring to. I am looking
at the file:

https://github.com/Tokutek/ft-index/blob/master/util/dmt.cc

Here there seems to be no available fix. Did I look in the wrong place? The
copyout function that I'm referring to has 3 or 4 parameters, not 2, as in
your code snippet.

Regards,
> Sergei
> Chief Architect MariaDB
> and secur...@mariadb.org
> --
> Vote for my Percona Live 2016 talks:
>
> https://www.percona.com/live/data-performance-conference-2016/sessions/mariadb-connectors-fast-and-smart-new-protocol-optimizations#community-voting
>
> https://www.percona.com/live/data-performance-conference-2016/sessions/mariadb-101-security-validation-authentication-encryption#community-voting
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 9d8ad0d: Add --persistent option for mysqlcheck

[Vicențiu Ciorbaru]

Git log show his name, I think that the commit message generated by the
hook is at fault.


On Tue, 15 Dec 2015 at 16:20 Vicențiu Ciorbaru <cvicen...@gmail.com> wrote:

> Hi Sergey,
>
> The commit message still has his name on it.
>
> I've only added some code to it, but I've left his original author.
>
> Vicentiu
>
> On Tue, 15 Dec 2015 at 11:32 Sergey Vojtovich <s...@mariadb.org> wrote:
>
>> Hi Vicentu,
>>
>> this is contribution by Daniel. He should get some credit for this.
>> Assuming you
>> didn't change this code much, this should be committed on Daniel's
>> behalf, so
>> that he gets credit on github. Or at the very least mention in a comment
>> contribution author.
>>
>> Thanks,
>> Sergey
>>
>> On Tue, Dec 15, 2015 at 11:11:50AM +0200, Vicentiu Ciorbaru wrote:
>> > revision-id: 9d8ad0d79e3c3a1a9bf7c8cc6477ee4e69fba633
>> (mariadb-10.0.22-63-g9d8ad0d)
>> > parent(s): e9b4a041af5122dffd59012493c565e6e3db2664
>> > committer: Vicențiu Ciorbaru
>> > timestamp: 2015-12-15 11:11:28 +0200
>> > message:
>> >
>> > Add --persistent option for mysqlcheck
>> >
>> > 10.0 has an "analyze table .. persistent for all" syntax. This adds
>> > --persistent to mysqlcheck(mysqlanalyize) to perform this extended
>> > analyze table option.
>> >
>> > ---
>> >  client/mysqlcheck.c |  6 +-
>> >  man/mysqlcheck.1| 18 +-
>> >  2 files changed, 22 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/client/mysqlcheck.c b/client/mysqlcheck.c
>> > index ead4127..b9e5478 100644
>> > --- a/client/mysqlcheck.c
>> > +++ b/client/mysqlcheck.c
>> > @@ -43,7 +43,7 @@ static my_bool opt_alldbs = 0, opt_check_only_changed
>> = 0, opt_extended = 0,
>> > opt_silent = 0, opt_auto_repair = 0, ignore_errors = 0,
>> > tty_password= 0, opt_frm= 0, debug_info_flag= 0,
>> debug_check_flag= 0,
>> > opt_fix_table_names= 0, opt_fix_db_names= 0,
>> opt_upgrade= 0,
>> > -   opt_do_tables= 1;
>> > +   opt_persistent_all= 0, opt_do_tables= 1;
>> >  static my_bool opt_write_binlog= 1, opt_flush_tables= 0;
>> >  static uint verbose = 0, opt_mysql_port=0;
>> >  static int my_end_arg;
>> > @@ -160,6 +160,9 @@ static struct my_option my_long_options[] =
>> >{"password", 'p',
>> > "Password to use when connecting to server. If password is not
>> given, it's solicited on the tty.",
>> > 0, 0, 0, GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
>> > +  {"persistent", 'Z', "When using ANALYZE TABLE use the PERSISTENT FOR
>> ALL option.",
>> > +   _persistent_all, _persistent_all, 0, GET_BOOL, NO_ARG, 0, 0,
>> > +   0, 0, 0, 0},
>> >  #ifdef __WIN__
>> >{"pipe", 'W', "Use named pipes to connect to server.", 0, 0, 0,
>> GET_NO_ARG,
>> > NO_ARG, 0, 0, 0, 0, 0, 0},
>> > @@ -909,6 +912,7 @@ static int handle_request_for_tables(char *tables,
>> size_t length, my_bool view)
>> >case DO_ANALYZE:
>> >  DBUG_ASSERT(!view);
>> >  op= (opt_write_binlog) ? "ANALYZE" : "ANALYZE NO_WRITE_TO_BINLOG";
>> > +if (opt_persistent_all) end = strmov(end, " PERSISTENT FOR
>> ALL");
>> >  break;
>> >case DO_OPTIMIZE:
>> >  DBUG_ASSERT(!view);
>> > diff --git a/man/mysqlcheck.1 b/man/mysqlcheck.1
>> > index c175483..e05bbf8 100644
>> > --- a/man/mysqlcheck.1
>> > +++ b/man/mysqlcheck.1
>> > @@ -1,6 +1,6 @@
>> >  '\" t
>> >  .\"
>> > -.TH "\FBMYSQLCHECK\FR" "1" "04/08/2015" "MariaDB 10\&.0" "MariaDB
>> Database System"
>> > +.TH "\FBMYSQLCHECK\FR" "1" "27/12/2015" "MariaDB 10\&.0" "MariaDB
>> Database System"
>> >  .\" -
>> >  .\" * set default formatting
>> >  .\" -
>> > @@ -677,6 +677,22 @@ Specifying a password on the command line should
>> be considered insecure\&. You c
>> >  .sp -1
>> >  .IP \(bu 2.3
>> >  .\}
>> > +.\" mysqlcheck: persisent option
>> > +.\" persistent option: mysql
>> > +\fB\-\-persistent\fR,
>> > +\fB\-Z\fR
>> > +.sp
>> > +Used with ANALYZE TABLE to append the option PERSISENT FOR ALL.
>> > +.RE
>> > +.sp
>> > +.RS 4
>> > +.ie n \{\
>> > +\h'-04'\(bu\h'+03'\c
>> > +.\}
>> > +.el \{\
>> > +.sp -1
>> > +.IP \(bu 2.3
>> > +.\}
>> >  .\" mysqlcheck: pipe option
>> >  .\" pipe option: mysql
>> >  \fB\-\-pipe\fR,
>> > ___
>> > commits mailing list
>> > comm...@mariadb.org
>> > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits
>> ___
>> commits mailing list
>> comm...@mariadb.org
>> https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits
>
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 9d8ad0d: Add --persistent option for mysqlcheck

[Vicențiu Ciorbaru]

Hi Sergey,

The commit message still has his name on it.

I've only added some code to it, but I've left his original author.

Vicentiu

On Tue, 15 Dec 2015 at 11:32 Sergey Vojtovich <s...@mariadb.org> wrote:

> Hi Vicentu,
>
> this is contribution by Daniel. He should get some credit for this.
> Assuming you
> didn't change this code much, this should be committed on Daniel's behalf,
> so
> that he gets credit on github. Or at the very least mention in a comment
> contribution author.
>
> Thanks,
> Sergey
>
> On Tue, Dec 15, 2015 at 11:11:50AM +0200, Vicentiu Ciorbaru wrote:
> > revision-id: 9d8ad0d79e3c3a1a9bf7c8cc6477ee4e69fba633
> (mariadb-10.0.22-63-g9d8ad0d)
> > parent(s): e9b4a041af5122dffd59012493c565e6e3db2664
> > committer: Vicențiu Ciorbaru
> > timestamp: 2015-12-15 11:11:28 +0200
> > message:
> >
> > Add --persistent option for mysqlcheck
> >
> > 10.0 has an "analyze table .. persistent for all" syntax. This adds
> > --persistent to mysqlcheck(mysqlanalyize) to perform this extended
> > analyze table option.
> >
> > ---
> >  client/mysqlcheck.c |  6 +-
> >  man/mysqlcheck.1| 18 +-
> >  2 files changed, 22 insertions(+), 2 deletions(-)
> >
> > diff --git a/client/mysqlcheck.c b/client/mysqlcheck.c
> > index ead4127..b9e5478 100644
> > --- a/client/mysqlcheck.c
> > +++ b/client/mysqlcheck.c
> > @@ -43,7 +43,7 @@ static my_bool opt_alldbs = 0, opt_check_only_changed
> = 0, opt_extended = 0,
> > opt_silent = 0, opt_auto_repair = 0, ignore_errors = 0,
> > tty_password= 0, opt_frm= 0, debug_info_flag= 0,
> debug_check_flag= 0,
> > opt_fix_table_names= 0, opt_fix_db_names= 0,
> opt_upgrade= 0,
> > -   opt_do_tables= 1;
> > +   opt_persistent_all= 0, opt_do_tables= 1;
> >  static my_bool opt_write_binlog= 1, opt_flush_tables= 0;
> >  static uint verbose = 0, opt_mysql_port=0;
> >  static int my_end_arg;
> > @@ -160,6 +160,9 @@ static struct my_option my_long_options[] =
> >{"password", 'p',
> > "Password to use when connecting to server. If password is not
> given, it's solicited on the tty.",
> > 0, 0, 0, GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
> > +  {"persistent", 'Z', "When using ANALYZE TABLE use the PERSISTENT FOR
> ALL option.",
> > +   _persistent_all, _persistent_all, 0, GET_BOOL, NO_ARG, 0, 0,
> > +   0, 0, 0, 0},
> >  #ifdef __WIN__
> >{"pipe", 'W', "Use named pipes to connect to server.", 0, 0, 0,
> GET_NO_ARG,
> > NO_ARG, 0, 0, 0, 0, 0, 0},
> > @@ -909,6 +912,7 @@ static int handle_request_for_tables(char *tables,
> size_t length, my_bool view)
> >case DO_ANALYZE:
> >  DBUG_ASSERT(!view);
> >  op= (opt_write_binlog) ? "ANALYZE" : "ANALYZE NO_WRITE_TO_BINLOG";
> > +if (opt_persistent_all) end = strmov(end, " PERSISTENT FOR
> ALL");
> >  break;
> >case DO_OPTIMIZE:
> >  DBUG_ASSERT(!view);
> > diff --git a/man/mysqlcheck.1 b/man/mysqlcheck.1
> > index c175483..e05bbf8 100644
> > --- a/man/mysqlcheck.1
> > +++ b/man/mysqlcheck.1
> > @@ -1,6 +1,6 @@
> >  '\" t
> >  .\"
> > -.TH "\FBMYSQLCHECK\FR" "1" "04/08/2015" "MariaDB 10\&.0" "MariaDB
> Database System"
> > +.TH "\FBMYSQLCHECK\FR" "1" "27/12/2015" "MariaDB 10\&.0" "MariaDB
> Database System"
> >  .\" -
> >  .\" * set default formatting
> >  .\" -
> > @@ -677,6 +677,22 @@ Specifying a password on the command line should be
> considered insecure\&. You c
> >  .sp -1
> >  .IP \(bu 2.3
> >  .\}
> > +.\" mysqlcheck: persisent option
> > +.\" persistent option: mysql
> > +\fB\-\-persistent\fR,
> > +\fB\-Z\fR
> > +.sp
> > +Used with ANALYZE TABLE to append the option PERSISENT FOR ALL.
> > +.RE
> > +.sp
> > +.RS 4
> > +.ie n \{\
> > +\h'-04'\(bu\h'+03'\c
> > +.\}
> > +.el \{\
> > +.sp -1
> > +.IP \(bu 2.3
> > +.\}
> >  .\" mysqlcheck: pipe option
> >  .\" pipe option: mysql
> >  \fB\-\-pipe\fR,
> > ___
> > commits mailing list
> > comm...@mariadb.org
> > https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits
> ___
> commits mailing list
> comm...@mariadb.org
> https://lists.askmonty.org/cgi-bin/mailman/listinfo/commits
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] BUILD ERROR

[Vicențiu Ciorbaru]

Hi Laketra,
As far as I know, MariaDB tokudb engine does not work on Mac OS. To not
compile tokudb you can run during cmake
Cmake. -DPLUGIN_TOKUDB=NO

Then run make. Make sure to delete cmakecache.txt before running cmake
again.

Vicentiu
On Fri, 11 Dec 2015 at 11:19, Laketra Musa  wrote:

> Hi,
> I was Building mariadb on Mac OS X and got this error:
>
> [ 81%] Built target pfs_misc-t
> [ 81%] Built target pfs_timer-t
> [ 81%] Built target pfs_user-oom-t
> [ 81%] Built target sphinx
> [ 83%] Built target spider
> [ 83%] Built target test_sql_discovery
> [ 84%] Built target tokuportability_static_conv
> [ 84%] Built target tokuportability_static
> [ 84%] Built target logformat
> [ 84%] Built target generate_log_code
> [ 84%] Built target make_tdb
> [ 84%] Built target install_tdb_h
> [ 84%] Building CXX object
> storage/tokudb/ft-index/src/CMakeFiles/tokudb_static_conv.dir/ydb.cc.o
> In file included from
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/src/ydb.cc:97:
> /Users/osiris/Desktop/server/build/storage/tokudb/ft-index/buildheader/db.h:323:1:
> error:
>   empty struct has size 0 in C, size 1 in C++
> [-Werror,-Wextern-c-compat]
> struct __toku_db_lsn {
> ^
> In file included from
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/src/ydb.cc:107:
> In file included from
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/ft/ft-flusher.h:94:
> In file included from
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/ft/ft-internal.h:103:
> In file included from
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/ft/node.h:91:
> In file included from
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/ft/bndata.h:93:
> In file included from
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/util/dmt.h:731:
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/util/dmt.cc:925:9:
> error:
>   nonnull parameter 'outlen' will evaluate to 'true' on first encounter
>   [-Werror,-Wpointer-bool-conversion]
> if (outlen) {
> ~~  ^~
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/util/dmt.cc:935:9:
> error:
>   nonnull parameter 'outlen' will evaluate to 'true' on first encounter
>   [-Werror,-Wpointer-bool-conversion]
> if (outlen) {
> ~~  ^~
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/util/dmt.cc:945:9:
> error:
>   nonnull parameter 'outlen' will evaluate to 'true' on first encounter
>   [-Werror,-Wpointer-bool-conversion]
> if (outlen) {
> ~~  ^~
> /Users/osiris/Desktop/server/storage/tokudb/ft-index/util/dmt.cc:955:9:
> error:
>   nonnull parameter 'outlen' will evaluate to 'true' on first encounter
>   [-Werror,-Wpointer-bool-conversion]
> if (outlen) {
> ~~  ^~
> 5 errors generated.
> make[2]: ***
> [storage/tokudb/ft-index/src/CMakeFiles/tokudb_static_conv.dir/ydb.cc.o]
> Error 1
> make[1]: ***
> [storage/tokudb/ft-index/src/CMakeFiles/tokudb_static_conv.dir/all] Error 2
> make: *** [all] Error 2
>
>
> How can I fix this please?
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] [MDEV-7122] Questions regarding code history

[Vicențiu Ciorbaru]

Hi Sergey!

I've been investigating MDEV-7122. I'm still quite a ways to go before I
can understand all the underlying bits of logic. I need a little help
regarding a change that you've committed a while back.

Do you know (remember) why you've placed DBUG_ASSERT(0) in the commit:
a1b4eadf89a3940b80a6ed0eadd15ba456c2fa7a
BUG#868908: Crash in check_simple_equality() with semijoin +
materialization + prepared statement
- Part2: safety and code cleanup

To me, it feels like it's some leftover debugging code, but it would be
good to get your input on this.

@@ -4182,6 +4183,8 @@ bool subselect_hash_sj_engine::init(List
*tmp_columns, uint subquery_id)
   */
   if (tmp_table->s->keys == 0)
   {
+//fprintf(stderr, "Q: %s\n", current_thd->query());
+DBUG_ASSERT(0);

Obviously we should also get rid of the commented fprintf statement.

After trying to debug this by just looking at the adjacent function calls
and logic, I've come to the conclusion that I need to actually understand
the code methodically. This is what I am actually doing right now, starting
from the parser, all the way to how this particular query is executed.
Expect more questions from me on this topic.

Thanks,
Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Error Compiling MariaDB with clang on Mac Os X

[Vicențiu Ciorbaru]

Hi Sergei,

While working on features for 10.2 I've encountered a problem when
compiling MariaDB as a Debug build.

According to your change on
commit 29dd634a4c0b9c3579cf9d318ed64d748d848b1d
dbug: correct trace for DBUG_RETURN(func());

This causes a small problem, in two places within the codebase. One is in
the myisam storage engine and one in the maria storage engine.
Within thr_find_all_keys, there is a hack with goto statements and using
DBUG_ENTER after my_thread_init().

/Users/cvicentiu/Workspace/MariaDB/storage/myisam/sort.c:352:5: error:
cannot jump from this goto statement to its label
goto err;
^
/Users/cvicentiu/Workspace/MariaDB/storage/myisam/sort.c:355:5: note: jump
bypasses initialization of variable with __attribute__((cleanup))
DBUG_ENTER("thr_find_all_keys");
^
/Users/cvicentiu/Workspace/MariaDB/include/my_dbug.h:74:47: note: expanded
from macro 'DBUG_ENTER'
#define DBUG_ENTER(a) struct _db_stack_frame_ _db_stack_frame_
 __attribute__((cleanup(_db_return_))); \


In the current code state, the attribute_cleanup callback does not make
sense if my_thread_init() fails and forces the goto err; Technically, it is
undefined behaviour, as we're jumping within the block that has the
variables defined, without initialising it.

The sensible thing to do is to refactor the block that has DBUG_ENTER into
a separate function and decoupling the error cleanup from the
my_thread_init() call. I've looked at that code and have seen that much of
the logic is redundant during the cleanup part. I can refactor it to fix
the problem, but I'm looking for input on your side if you're ok with me
doing that.

I've also found a bug in that same code in the error checking that I can
fix.

Regards,
Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] MDEV-7937: Enforce SSL when --ssl client option is used

[Vicențiu Ciorbaru]

Hi Sergei!

I've done some work on this issue. I've read MySQL's implementation of this
and have looked at our implementation. They have done a bit of refactoring,
introducing an enforce_ssl flag, as well as changing the C interface a bit,
to allow setting this flag programatically.

I've created a patch here that changes the minimum amount possible, in
order to implement what MDEV-7937 requires. That being said, I agree with
(most of) MySQL's refactoring in this case. They've moved all the SSL
related connection code into its own separate function, before actually
calling send_client_reply_packet.

I can work towards implementing things the way MySQL does, but since I saw
that you've actually done most of the work in that area of the code, I
figured I'd ask for your input on it.

There are two more things that I'm not sure of:
1. Specifying --ssl as a command line parameter to the mysql client is not
enough to enforce ssl and the client's code in this case just ignores the
option. We need to provide at least one of the additional ones like
--ssl-key or --ssl-ca. My patch will not cause the client to report an
error in this case. Is this acceptable behaviour or not?

2. Do we want mysql's enforce_ssl feature?

Regards,
Vicențiu
commit e5dadaa289d17365198cf0571fa6093504a74536
Author: Vicențiu Ciorbaru cvicen...@gmail.com
Date:   Sun May 17 19:36:54 2015 +

MDEV-7937: Enforce SSL when --ssl client option is used

The mysql client will now print an error if ssl is requested,
but the server can not handle a ssl connection.

diff --git a/sql-common/client.c b/sql-common/client.c
index 006b173..068b9b3 100644
--- a/sql-common/client.c
+++ b/sql-common/client.c
@@ -2682,6 +2682,16 @@ static int send_client_reply_packet(MCPVIO_EXT *mpvio,
 end= buff+5;
   }
 #ifdef HAVE_OPENSSL
+  if (mysql-options.use_ssl  !(mysql-server_capabilities  CLIENT_SSL))
+  {
+  set_mysql_extended_error(mysql, CR_SSL_CONNECTION_ERROR, unknown_sqlstate,
+   ER(CR_SSL_CONNECTION_ERROR),
+   SSL is required, but the server does not 
+   support it
+   );
+  goto error;
+  }
+
   if (mysql-client_flag  CLIENT_SSL)
   {
 /* Do the SSL layering. */
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] [Commits] 13ecde8: Fix compiler warnings.

[Vicențiu Ciorbaru]

Hi Jan!

I have one concern with this patch. See below:
On Thu, 30 Apr 2015 at 11:45 Jan Lindström jan.lindst...@mariadb.com
wrote:

 revision-id: 13ecde851f08a7952f774707b19734f351bb6598
 parent(s): 5027e0b035f96e2583eff2d1ac71c7c44a093ac8
 committer: Jan Lindström
 branch nick: 10.0-git
 timestamp: 2015-04-30 11:44:34 +0300
 message:

 Fix compiler warnings.

 ---
  storage/innobase/sync/sync0sync.cc | 27 +--
  storage/xtradb/sync/sync0sync.cc   | 27 +--
  2 files changed, 26 insertions(+), 28 deletions(-)

 diff --git a/storage/innobase/sync/sync0sync.cc
 b/storage/innobase/sync/sync0sync.cc
 index 121f8e9..78d86f3 100644
 --- a/storage/innobase/sync/sync0sync.cc
 +++ b/storage/innobase/sync/sync0sync.cc
 @@ -862,10 +862,10 @@ sync_thread_levels_g(
  {
 ulint   i;

 -   for (i = 0; i  arr-n_elems; i++) {
 +   for (i = 0; i  arr-size(); i++) {
 const sync_level_t* slot;

 I think this here has to be arr-at(i), or (*arr)[i]. Doing arr[i] will
actually get the i'th std::vector which does not exist, since arr is a
pointer to only one std::vector.

 -   slot = arr-elems[i];
 +   slot = (const sync_level_t*)arr[i];

 if (slot-latch != NULL  slot-level = limit) {
 if (warn) {
 @@ -897,10 +897,10 @@ sync_thread_levels_contain(
  {
 ulint   i;

 -   for (i = 0; i  arr-n_elems; i++) {
 +   for (i = 0; i  arr-size(); i++) {
 const sync_level_t* slot;

 Same here.

 -   slot = arr-elems[i];

+   slot = (const sync_level_t*)arr[i];

 if (slot-latch != NULL  slot-level == level) {

 @@ -944,10 +944,10 @@ sync_thread_levels_contains(

 arr = thread_slot-levels;

 -   for (i = 0; i  arr-n_elems; i++) {
 +   for (i = 0; i  arr-size(); i++) {
 sync_level_t*   slot;

Same here.

 -   slot = arr-elems[i];
 +   slot = (sync_level_t*)arr[i];

 if (slot-latch != NULL  slot-level == level) {

 @@ -993,10 +993,10 @@ sync_thread_levels_nonempty_gen(

 arr = thread_slot-levels;

 -   for (i = 0; i  arr-n_elems; ++i) {
 +   for (i = 0; i  arr-size(); ++i) {
 const sync_level_t* slot;


Same here.

 -   slot = arr-elems[i];
 +   slot = (const sync_level_t*)arr[i];

 if (slot-latch != NULL
  (!dict_mutex_allowed
 @@ -1053,10 +1053,10 @@ sync_thread_levels_nonempty_trx(

 arr = thread_slot-levels;

 -   for (i = 0; i  arr-n_elems; ++i) {
 +   for (i = 0; i  arr-size(); ++i) {
 const sync_level_t* slot;


Same here.

 -   slot = arr-elems[i];
 +   slot = (const sync_level_t*)arr[i];

 if (slot-latch != NULL
  (!has_search_latch
 @@ -1331,7 +1331,7 @@ sync_thread_add_level(

 sync_level.latch = latch;
 sync_level.level = level;
 -   array-elems.push_back(sync_level);
 +   array-push_back(sync_level);

 mutex_exit(sync_thread_mutex);
  }
 @@ -1377,14 +1377,14 @@ sync_thread_reset_level(

 array = thread_slot-levels;

-   for (std::vectorsync_level_t::iterator it = array-elems.begin();
 it != array-elems.end(); ++it) {
 +   for (std::vectorsync_level_t::iterator it = array-begin(); it
 != array-end(); ++it) {
 sync_level_t level = *it;

 if (level.latch != latch) {
 continue;
 }

 -   array-elems.erase(it);
 +   array-erase(it);
 mutex_exit(sync_thread_mutex);
 return(TRUE);
 }
 @@ -1471,7 +1471,6 @@ sync_thread_level_arrays_free(void)

 /* If this slot was allocated then free the slot memory
 too. */
 if (slot-levels != NULL) {
 -
  
 slot-levels-elems.erase(slot-levels-elems.begin(),slot-levels-elems.end());
 delete slot-levels;
 }
 }
 diff --git a/storage/xtradb/sync/sync0sync.cc
 b/storage/xtradb/sync/sync0sync.cc
 index d0e3b71..9dcdd9c 100644
 --- a/storage/xtradb/sync/sync0sync.cc
 +++ b/storage/xtradb/sync/sync0sync.cc
 @@ -970,10 +970,10 @@ sync_thread_levels_g(
  {
 ulint   i;

 -   for (i = 0; i  arr-n_elems; i++) {
 +   for (i = 0; i  arr-size(); i++) {
 const sync_level_t* slot;

 Same here.

 -   slot = arr-elems[i];
 +   slot = (const sync_level_t*)arr[i];

 if (slot-latch != NULL  slot-level = limit) {
 if (warn) {
 @@ -1005,10 +1005,10 @@ sync_thread_levels_contain(
  {
 ulint   i;

 -   for (i = 0; i  arr-n_elems; i++) {
 +   for (i = 0; i  arr-size(); i++) {
 const sync_level_t* slot;

Same here.

 -   

[Maria-developers] MDEV-7912 review and related concerns

[Vicențiu Ciorbaru]

Hi,

I'd like to report my findings during the investigation of MDEV-7912, ask
for a review on my patch and see your opinion on the status of the related
code. The patch and all the issues that I am raising are for the 5.5 branch.

With the attached patch, the server crash is eliminated. The problem we had
was caused by casting a 64bit unsigned int to a 32bit unsigned int,
representing a number of bytes to allocate. When the ranges were too big,
it caused an overflow and we allocated too little space.

On a related note, while looking at the code in the uniques.cc file, I've
discovered a lot of potential issues, that are almost guaranteed to fail,
if we actually do use more than 4 GB of memory. This holds true especially
for Windows, because ulong is 32 bits for a 64 bit compilation, while on
Linux, it is 64 bit.
I've tried to convert most of the code that made use of uint or ulong or
ulonglong for address space indexing (arrays, indices in arrays, memory
sizes) to size_t, so that we escape the portability problem when compiling
for 32 bit or 64 bit. Unfortunately, the task branches out into various
other files that make use of those values. An example of a such a problem
is in Unique::Unique constructor, line 90, where init_tree will get passed
an overflown parameter (max_in_memory_size). It just so happens that there
is a guard within the init_tree function that sets a minimum value, in case
we do happen to pass an overflown value. This is however a lucky occurrence.

Unique::get_use_cost can output a bad cost value, because
get_merge_many_buffs_cost casts the same ulonglong memory size parameter to
a ulong.
There are many examples like this.

I'd like to get your input on this issue, is this something we plan to fix
in the near future? Should I embark on the journey of slowly fixing at
least parts of it? Also, this issue seems to plague much of the code base,
as I get a huge wall of similar size_t - uint32 warnings across most files.

Regards,
Vicențiu
commit 40d6d95824ffdd53e0dcdf03ce7ef57cb28e1574
Author: Vicentiu Ciorbaru vicen...@mariadb.org
Date:   Fri Apr 24 13:47:16 2015 +0300

MDEV-7912 multitable delete with wrongly set sort_buffer_size crashes in 
merge_buffers

Fixed overflow error that caused fewer bites to be allocated than
necessary on Windows 64 bit. This is due to ulong being 32 bit on
64 bit Windows and 64 bit on 64 bit Linux.

diff --git a/sql/uniques.cc b/sql/uniques.cc
index 72411be..b9436ea 100644
--- a/sql/uniques.cc
+++ b/sql/uniques.cc
@@ -97,7 +97,7 @@ int unique_intersect_write_to_ptrs(uchar* key, element_count 
count, Unique *uniq
   max_elements= (ulong) (max_in_memory_size /
  ALIGN_SIZE(sizeof(TREE_ELEMENT)+size));
   (void) open_cached_file(file, mysql_tmpdir,TEMP_PREFIX, DISK_BUFFER_SIZE,
-  MYF(MY_WME));
+  MYF(MY_WME));
 }
 
 
@@ -607,8 +607,8 @@ bool Unique::walk(TABLE *table, tree_walk_action action, 
void *walk_action_arg)
 return 1;
   if (flush_io_cache(file) || reinit_io_cache(file, READ_CACHE, 0L, 0, 0))
 return 1;
-  ulong buff_sz= (max_in_memory_size / full_size + 1) * full_size;
-  if (!(merge_buffer= (uchar *) my_malloc((ulong) buff_sz, MYF(0
+  size_t buff_sz= (max_in_memory_size / full_size + 1) * full_size;
+  if (!(merge_buffer= (uchar *) my_malloc(buff_sz, MYF(0
 return 1;
   if (buff_sz  (ulong) (full_size * (file_ptrs.elements + 1)))
 res= merge(table, merge_buffer, buff_sz = full_size * MERGEBUFF2) ;
@@ -738,8 +738,8 @@ bool Unique::get(TABLE *table)
   if (flush())
 return 1;
   
-  ulong buff_sz= (max_in_memory_size / full_size + 1) * full_size;
-  if (!(sort_buffer= (uchar*) my_malloc(buff_sz, MYF(0
+  size_t buff_sz= (max_in_memory_size / full_size + 1) * full_size;
+  if (!(sort_buffer= (uchar*) my_malloc(buff_sz, MYF(MY_WME
 return 1;
 
   if (merge(table, sort_buffer, FALSE))
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] TRUE vs true (was: [Commits] ac05f41: postreview fix)

[Vicențiu Ciorbaru]

Hi,

I would like to point out that it would be a good idea to start writing our
own coding style doc, so that we no longer have these sort of unknowns. It
also helps the community provide patches that are easier for us to review.
I can start working on it myself.

Regards,
Vicențiu

On Thu, 23 Apr 2015 at 10:33 Oleksandr Byelkin sa...@montyprogram.com
wrote:

 Hi, Jan!

 On 23.04.15 08:38, Jan Lindström wrote:
  Sanja,
 
  Why you use TRUE/FALSE instead of true/false for bool ?
 Historical reasons. I was told to use TRUE/FALSE long time ago and
 nobody have told something else. We have no our code style doc so I use
 what I remember from old MySQL one.

 [skip]

 ___
 Mailing list: https://launchpad.net/~maria-developers
 Post to : maria-developers@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~maria-developers
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] MDEV-6877 - binlog_row_image implementation advice

[Vicențiu Ciorbaru]

Hi Kristian,

I've gotten to a sort of stable state with binlog_row_image. It's not
100% complete as I did not add any tests for the new use cases.
Also there are a couple of changes that I am not 100% sure if they are
complete or correct.

I would like some input on these changes:
https://github.com/MariaDB/server/compare/10.1-MDEV-6877-binlog_row_image

The unpack_current_row being a very iffy change for me. I understand why it
is needed in the Update_rows_event::do_exec_row code, as the after image
has other columns that get written as opposed to the before image. Thing is
I've just made it in an attempt to fix a HA_ERR_SAME_KEY error that I kept
getting on the slave and it seems to have worked.

Do you think the implementation that MySQL chose is the correct one? The
changes are mostly the same as the MySQL variant, except they have V2 ROW
LOG EVENTS which seem to pass some extra info which we don't use (need?).

There are a lot of changes made and I've tried my best to not miss
anything. I agree that the git history could be better. I will rewrite it
when I get the final code variant.

Regards,
Vicentiu

On Mon, 30 Mar 2015 at 17:22 Vicențiu Ciorbaru vicen...@mariadb.org wrote:

 Hi Kristian,

  Another concern that I have is how to test the implementation. Right
 now, I
  run the test, get the binlog file and print it with mysqlbinlog and see
 if
  the output is what I expect. Do we have something within mysql-test-run
  that automates this?

 There are some MTR tests that run mysqlbinlog and checks the output, for
 example mysqlbinlog.test.

 But the usual way to check replication is all the MTR tests in
 mysql-test/suite/rpl/t/. They start up a master and a slave server (or
 more),
 make some transactions on the master, check that they replicate as
 expected to
 the slave.


 Thank you for the reply! I have already started doing the changes and I
 seem to be making some progress. The mysqlbinlog tests were the ones that I
 was looking for.
 I'll let you know if I run into any more issues. This actually turned out
 to be a very good learning experience for me as I previously had no
 experience with replication. I'll mark you as a reviewer when I have a
 patch ready.

 Vicențiu

___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] MDEV-6877 - binlog_row_image implementation advice

[Vicențiu Ciorbaru]

I've also added a bunch of inline comments to the github commits to further
explain my reasoning. Going through the commits in order chronologically
might make the most sense.

Regards,
Vicențiu

On Mon, 6 Apr 2015 at 17:39 Vicențiu Ciorbaru vicen...@mariadb.org wrote:

 Hi Kristian,

 I've gotten to a sort of stable state with binlog_row_image. It's not
 100% complete as I did not add any tests for the new use cases.
 Also there are a couple of changes that I am not 100% sure if they are
 complete or correct.

 I would like some input on these changes:
 https://github.com/MariaDB/server/compare/10.1-MDEV-6877-binlog_row_image

 The unpack_current_row being a very iffy change for me. I understand why
 it is needed in the Update_rows_event::do_exec_row code, as the after image
 has other columns that get written as opposed to the before image. Thing is
 I've just made it in an attempt to fix a HA_ERR_SAME_KEY error that I kept
 getting on the slave and it seems to have worked.

 Do you think the implementation that MySQL chose is the correct one? The
 changes are mostly the same as the MySQL variant, except they have V2 ROW
 LOG EVENTS which seem to pass some extra info which we don't use (need?).

 There are a lot of changes made and I've tried my best to not miss
 anything. I agree that the git history could be better. I will rewrite it
 when I get the final code variant.

 Regards,
 Vicentiu

 On Mon, 30 Mar 2015 at 17:22 Vicențiu Ciorbaru vicen...@mariadb.org
 wrote:

 Hi Kristian,

  Another concern that I have is how to test the implementation. Right
 now, I
  run the test, get the binlog file and print it with mysqlbinlog and
 see if
  the output is what I expect. Do we have something within mysql-test-run
  that automates this?

 There are some MTR tests that run mysqlbinlog and checks the output, for
 example mysqlbinlog.test.

 But the usual way to check replication is all the MTR tests in
 mysql-test/suite/rpl/t/. They start up a master and a slave server (or
 more),
 make some transactions on the master, check that they replicate as
 expected to
 the slave.


 Thank you for the reply! I have already started doing the changes and I
 seem to be making some progress. The mysqlbinlog tests were the ones that I
 was looking for.
 I'll let you know if I run into any more issues. This actually turned out
 to be a very good learning experience for me as I previously had no
 experience with replication. I'll mark you as a reviewer when I have a
 patch ready.

 Vicențiu


___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] MDEV-6877 - binlog_row_image implementation advice

[Vicențiu Ciorbaru]

Hi Kristian,

 Another concern that I have is how to test the implementation. Right now,
 I
  run the test, get the binlog file and print it with mysqlbinlog and see
 if
  the output is what I expect. Do we have something within mysql-test-run
  that automates this?

 There are some MTR tests that run mysqlbinlog and checks the output, for
 example mysqlbinlog.test.

 But the usual way to check replication is all the MTR tests in
 mysql-test/suite/rpl/t/. They start up a master and a slave server (or
 more),
 make some transactions on the master, check that they replicate as
 expected to
 the slave.


Thank you for the reply! I have already started doing the changes and I
seem to be making some progress. The mysqlbinlog tests were the ones that I
was looking for.
I'll let you know if I run into any more issues. This actually turned out
to be a very good learning experience for me as I previously had no
experience with replication. I'll mark you as a reviewer when I have a
patch ready.

Vicențiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] MDEV-6877 - binlog_row_image implementation advice

[Vicențiu Ciorbaru]

Hi Sergei!

This past week I've done quite a bit of reading regarding the replication
code and how the binlog is used by both mysqlbinlog and the slaves in order
to do replication.

I think I now have a good enough understanding of the flow of the code for
both MySQL and MariaDB to actually start coding the feature into MariaDB. I
do have a couple of unknowns still. I don't quite understand what's the use
for the TABLE_MAP event. My intuition tells me its some sort of id we
assign to a table name (test.t1 for example) that gets used afterwards in
the binlog row events.

We seem to be using a bitmap of columns that we want to log into the
binlog. MySQL does extend the TABLE class to flip the bits according to the
binlog_row_image variable. That seems like the first place to start with
the addition. I could not find any other hidden logic that we use to
choose how we log row operations in the binlog. What I would like to know
is if there are any other pitfalls that I am supposed to watch out for, or
if there is anything else I may be missing regarding the logging part.

When it comes to actually doing the replication, I concluded that
Log_event::do_apply_event
is the place where we begin translating the event from the binlog to an SQL
statement.

Another concern that I have is how to test the implementation. Right now, I
run the test, get the binlog file and print it with mysqlbinlog and see if
the output is what I expect. Do we have something within mysql-test-run
that automates this?

Regards,
Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] Question regarding MDEV-7682

[Vicențiu Ciorbaru]

Hi Holyfoot!

I've made some changes regarding key_length computation to fix MDEV-6838.
Changing this seems to have lead to us discovering another issue outlined
in MDEV-7682 (that I've just created).

It seems that the spatial key returns a key_length of 0. I've spoken to
Sergei Petrunia and we are not sure about this particular case. pack_length
returns a bigger number (12). According to the other comments in the code,
pack_length also returns the length field associated with the column, but
we are only interested in the length of the column field, without any
metadata information. The function TABLE:create_key_part_by_field adds the
metadata length itself afterwards.

Do you have any input on this issue, namely why the key_length is returned
as 0 for the LINESTRING column?

Regards,
Vicentiu
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

[Maria-developers] MDEV-6838: Using too big key for internal temp tables

[Vicențiu Ciorbaru]

Hi Sergei and Sergey!

I've implemented the fix for MDEV-6838. I'd like a review from both of you
regarding the patch. There are a couple of changes made. The main issues
are outlined in the commit message.

I've changed the way create_key_part_by_field gets called in order to
decouple the KEY_PART_INFO creation process from any KEYINFO modifications
and also documented what the method does. My other concern is that
create_key_part_by_field does not seem to make any sense within the table
class as it's basically a constructor for KEY_PART_INFO and makes no use of
table fields. I suggest we move the code as either a constructor for
KEY_PART_INFO, or an init() function. I think that would be better left as
a separate patch, but the method itself is very easy to factor out in case
you agree with the change.

I'm looking forward for your input.

Regards,
Vicențiu

-- Forwarded message --
From: vicen...@mariadb.org
Date: 28 February 2015 at 23:58
Subject: 45b6edb: MDEV-6838: Using too big key for internal temp tables
To: comm...@mariadb.org


revision-id: 45b6edb158f8101d641f550179ee15df363f686f
parent(s): fa87fc733d7855e0e5f9b959ca0bddf772ca57e5
committer: Vicențiu Ciorbaru
branch nick: server
timestamp: 2015-02-28 23:58:05 +0200
message:

MDEV-6838: Using too big key for internal temp tables

This bug manifests due to wrong computation and evaluation of
keyinfo-key_length. The issues were:
* Using table-file-max_key_length() as an absolute value that must not be
  reached for a key, while it represents the maximum number of bytes
  possible for a table key.
* Incorrectly computing the keyinfo-key_length size during
  KEY_PART_INFO creation. The metadata information regarding the key
  such the field length (for strings) was added twice.

---
 mysql-test/r/table_keyinfo-6838.result | 12 
 mysql-test/t/table_keyinfo-6838.test   | 18 ++
 sql/sql_select.cc  |  7 ---
 sql/structs.h  |  1 +
 sql/table.cc   | 34
++
 sql/table.h|  2 +-
 6 files changed, 62 insertions(+), 12 deletions(-)

diff --git a/mysql-test/r/table_keyinfo-6838.result
b/mysql-test/r/table_keyinfo-6838.result
new file mode 100644
index 000..55b0350
--- /dev/null
+++ b/mysql-test/r/table_keyinfo-6838.result
@@ -0,0 +1,12 @@
+CREATE TABLE t1 (i INT, state VARCHAR(997)) ENGINE=MyISAM;
+INSERT INTO t1 VALUES (2,'Louisiana'),(9,'Maine');
+CREATE TABLE t2 (state VARCHAR(997), j INT) ENGINE=MyISAM;
+INSERT INTO t2 VALUES ('Louisiana',9),('Alaska',5);
+INSERT INTO t2 SELECT t2.* FROM t2 JOIN t2 AS t3 JOIN t2 AS t4 JOIN t2 AS
t5 JOIN t2 AS t6;
+SET @@max_heap_table_size= 16384;
+set @@optimizer_switch='derived_merge=OFF';
+set @@optimizer_switch='extended_keys=ON';
+SELECT * FROM t1 AS t1_1 LEFT JOIN ( t1 AS t1_2 INNER JOIN (SELECT * FROM
t2) v2 ON t1_2.i = j ) ON t1_1.state = v2.state LIMIT 1;
+i  state   i   state   state   j
+2  Louisiana   9   Maine   Louisiana   9
+DROP TABLE t1, t2;
diff --git a/mysql-test/t/table_keyinfo-6838.test
b/mysql-test/t/table_keyinfo-6838.test
new file mode 100644
index 000..5f4c9f4
--- /dev/null
+++ b/mysql-test/t/table_keyinfo-6838.test
@@ -0,0 +1,18 @@
+# Test case for MDEV-6838.
+# Due to incorrect key_length computation, this usecase failed with the
error
+# Using too big key for internal temp table.
+
+CREATE TABLE t1 (i INT, state VARCHAR(997)) ENGINE=MyISAM;
+INSERT INTO t1 VALUES (2,'Louisiana'),(9,'Maine');
+
+CREATE TABLE t2 (state VARCHAR(997), j INT) ENGINE=MyISAM;
+INSERT INTO t2 VALUES ('Louisiana',9),('Alaska',5);
+INSERT INTO t2 SELECT t2.* FROM t2 JOIN t2 AS t3 JOIN t2 AS t4 JOIN t2 AS
t5 JOIN t2 AS t6;
+
+SET @@max_heap_table_size= 16384;
+set @@optimizer_switch='derived_merge=OFF';
+set @@optimizer_switch='extended_keys=ON';
+
+SELECT * FROM t1 AS t1_1 LEFT JOIN ( t1 AS t1_2 INNER JOIN (SELECT * FROM
t2) v2 ON t1_2.i = j ) ON t1_1.state = v2.state LIMIT 1;
+
+DROP TABLE t1, t2;
diff --git a/sql/sql_select.cc b/sql/sql_select.cc
index 387fdb3..dabe46e 100644
--- a/sql/sql_select.cc
+++ b/sql/sql_select.cc
@@ -7957,7 +7957,8 @@ static bool create_hj_key_for_table(JOIN *join,
JOIN_TAB *join_tab,
   {
 Field *field= table-field[keyuse-keypart];
 uint fieldnr= keyuse-keypart+1;
-table-create_key_part_by_field(keyinfo, key_part_info, field,
fieldnr);
+table-create_key_part_by_field(key_part_info, field, fieldnr);
+keyinfo-key_length += key_part_info-store_length;
 key_part_info++;
   }
 }
@@ -15921,7 +15922,7 @@ bool create_internal_tmp_table(TABLE *table, KEY
*keyinfo,
   goto err;

 bzero(seg, sizeof(*seg) * keyinfo-key_parts);
-if (keyinfo-key_length = table-file-max_key_length() ||
+if (keyinfo-key_length  table-file-max_key_length() ||
keyinfo-key_parts  table-file-max_key_parts() ||
share-uniques

[Maria-developers] [MDEV-7586] Merged derived tables/VIEWs increment created_tmp_tables

[Vicențiu Ciorbaru]

Hi Sergey!

As discussed on the mdev page, I've implemented the fix for
incrementing the created_tmp_tables variable.
I've attached the patch for review. The changes in the patch are
explained within the mdev page.

commit 53d78644b793f7b667e71863ccd0b1ba54c894f3
Author: Vicențiu Ciorbaru cvicen...@gmail.com
Date:   Sun Feb 22 03:40:17 2015 +0200

MDEV-7586: Merged derived tables/VIEWs increment created_tmp_tables

Temporary table count fix. The number of temporary tables was increased
when the table is not actually created. (when do_not_open was passed
as TRUE to create_tmp_table).

Another issue fixed is that create_tmp_table was called incorrectly
in create_result_table, having keep_row_order passed for the do_not_open
parameter and keep_row_order always set to false.

diff --git a/mysql-test/r/tmp_table_count-7586.result
b/mysql-test/r/tmp_table_count-7586.result
new file mode 100644
index 000..0c526e0
--- /dev/null
+++ b/mysql-test/r/tmp_table_count-7586.result
@@ -0,0 +1,83 @@
+create table t2 (a int);
+insert into t2 values (1),(2),(3);
+create view v2 as select a from t2;
+flush status;
+select * from v2;
+a
+1
+2
+3
+show status like '%Created_tmp%';
+Variable_name Value
+Created_tmp_disk_tables 0
+Created_tmp_files 0
+Created_tmp_tables 0
+explain select * from v2;
+id select_type table type possible_keys key key_len ref rows Extra
+1 SIMPLE t2 ALL NULL NULL NULL NULL 3
+select * from (select * from t2) T1;
+a
+1
+2
+3
+show status like '%Created_tmp%';
+Variable_name Value
+Created_tmp_disk_tables 0
+Created_tmp_files 0
+Created_tmp_tables 0
+explain select * from (select * from t2) T1;
+id select_type table type possible_keys key key_len ref rows Extra
+1 SIMPLE t2 ALL NULL NULL NULL NULL 3
+drop view v2;
+drop table t2;
+CREATE TABLE t1(a int);
+INSERT INTO t1 values(1),(2);
+CREATE TABLE t2(a int);
+INSERT INTO t2 values(1),(2);
+EXPLAIN SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a HAVING a  1);
+id select_type table type possible_keys key key_len ref rows Extra
+1 PRIMARY t1 ALL NULL NULL NULL NULL 2 Using where
+1 PRIMARY subquery2 eq_ref distinct_key distinct_key 4 test.t1.a 1
+2 MATERIALIZED t2 ALL NULL NULL NULL NULL 2 Using temporary
+truncate table performance_schema.events_statements_history_long;
+flush status;
+CREATE TABLE t3 SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP
BY a HAVING a  1);
+# Performance schema should be the same as Created_tmp_tables variable below
+select sum(created_tmp_tables) from
performance_schema.events_statements_history_long;
+sum(created_tmp_tables)
+2
+show status like '%Created_tmp%';
+Variable_name Value
+Created_tmp_disk_tables 0
+Created_tmp_files 0
+Created_tmp_tables 2
+drop table t3;
+EXPLAIN SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a);
+id select_type table type possible_keys key key_len ref rows Extra
+1 PRIMARY t1 ALL NULL NULL NULL NULL 2
+1 PRIMARY subquery2 eq_ref distinct_key distinct_key 4 func 1
+2 MATERIALIZED t2 ALL NULL NULL NULL NULL 2
+truncate table performance_schema.events_statements_history_long;
+flush status;
+CREATE TABLE t3 SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a);
+# Performance schema should be the same as Created_tmp_tables variable below
+select sum(created_tmp_tables) from
performance_schema.events_statements_history_long;
+sum(created_tmp_tables)
+1
+show status like '%Created_tmp%';
+Variable_name Value
+Created_tmp_disk_tables 0
+Created_tmp_files 0
+Created_tmp_tables 1
+drop table t1,t2,t3;
+truncate table performance_schema.events_statements_history_long;
+flush status;
+# Performance schema should be the same as Created_tmp_tables variable below
+select sum(created_tmp_tables) from
performance_schema.events_statements_history_long;
+sum(created_tmp_tables)
+0
+show status like '%Created_tmp%';
+Variable_name Value
+Created_tmp_disk_tables 0
+Created_tmp_files 0
+Created_tmp_tables 0
diff --git a/mysql-test/t/tmp_table_count-7586.test
b/mysql-test/t/tmp_table_count-7586.test
new file mode 100644
index 000..756913e
--- /dev/null
+++ b/mysql-test/t/tmp_table_count-7586.test
@@ -0,0 +1,53 @@
+# MDEV-7586 regression test.
+# Test if created_tmp_tables status variable is correctly incremented.
+--source include/have_perfschema.inc
+
+create table t2 (a int);
+insert into t2 values (1),(2),(3);
+create view v2 as select a from t2;
+
+flush status;
+select * from v2;
+show status like '%Created_tmp%';
+
+explain select * from v2;
+
+select * from (select * from t2) T1;
+show status like '%Created_tmp%';
+
+explain select * from (select * from t2) T1;
+
+
+drop view v2;
+drop table t2;
+
+
+CREATE TABLE t1(a int);
+INSERT INTO t1 values(1),(2);
+CREATE TABLE t2(a int);
+INSERT INTO t2 values(1),(2);
+
+EXPLAIN SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP BY a HAVING a  1);
+truncate table performance_schema.events_statements_history_long;
+flush status;
+CREATE TABLE t3 SELECT * FROM t1 WHERE a IN (SELECT * FROM t2 GROUP

Re: [Maria-developers] Fwd: Contributing in MariaDB Project

[Vicențiu Ciorbaru]

Hi Rohit,

Since we had trouble communicating on IRC, feel free to send your questions
via email. It's also good that the community also gets a chance to
contribute with answers.

Regards,
Vicentiu

On Thu Feb 12 2015 at 4:18:45 PM Vicențiu Ciorbaru cvicen...@gmail.com
wrote:

 Hi Rohit,

 Ping me on IRC, you can find me on #maria on freenode, username vicentiu.

 Regards,
 Vicentiu

 On Thu Feb 12 2015 at 4:14:08 PM Rohit Kashyap rkgud...@gmail.com wrote:

 Hi Vincent,
 I am working on issue
  https://mariadb.atlassian.net/browse/MDEV-5772?filter=-1
 And currently stalled at few points.
 Can We have a quick chat on freenode or wherever you prefer :)

 On Thu, Feb 12, 2015 at 8:56 AM, Rohit Kashyap rkgud...@gmail.com
 wrote:

 Hi Vicentiu
 Apologies for slight delay in response.
 The edits in sql_yacc.yy are taking some time.
 I am almost there.
 --
 From: Vicențiu Ciorbaru cvicen...@gmail.com
 Sent: ‎10-‎02-‎2015 20:51
 To: Rohit Kashyap rkgud...@gmail.com; maria-developers@lists.
 launchpad.net
 Subject: Re: [Maria-developers] Fwd: Contributing in MariaDB Project

 Hi Rohit!

 It's good to know you are interested in contributing to MariaDB. We
 don't have a lot of beginner friendly bugs running around right now.
 Fortunately however, I do have a task that might be simple enough to tackle
 [1].

 This is part of a former GSOC project called Roles. The place to look
 for is in the file sql/sql_acl.cc, the function mysql_grant_role(). [2]

 The way the function is currently designed and called does not allow for
 multiple roles to be granted in one step. It would have to be refactored
 somehow.
 The fix will probably take you to sql/sql_yacc.yy [3] for a bit of
 tinkering there as well.

 Feel free to submit a pull request with any work in progress code you
 have. I would be happy to review it. Any other questions you have are also
 welcomed.

 Regards,
 Vicentiu

 [1] https://mariadb.atlassian.net/browse/MDEV-5772?filter=-1
 [2] https://github.com/MariaDB/server/blob/10.1/sql/sql_acl.cc#L6174
 [3] https://github.com/MariaDB/server/blob/10.1/sql/sql_yacc.yy#L15296

 On Tue Feb 10 2015 at 1:53:18 PM Rohit Kashyap rkgud...@gmail.com
 wrote:


 Hi,
 I am Rohit from NIT Patna, India. A Engineering Student and contributor
 in MySQL Community, approved by Oracle.
 You guys are doing great things at Team Maria DB. Kudos for that.
 I would like to start and contribute in the Maria project, and would
 like to ask for
 guiding in the same, with code base and introductory Bugs to prove
 myself.

 --
 Kind Regards,
 Rohit
  Freenode: rkgudboy


 --
 Kind Regards,
 Rohit
  ___
 Mailing list: https://launchpad.net/~maria-developers
 Post to : maria-developers@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~maria-developers
 More help   : https://help.launchpad.net/ListHelp




 --
 Kind Regards,
 Rohit


___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] Fwd: MDEV-6838 Using too big key for internal temp tables

[Vicențiu Ciorbaru]

Hi Patryk,

I'm still investigating to find the root cause of the issue. I'll let you
know when I find it.

Regards,
Vicentiu

On Tue Feb 10 2015 at 5:34:31 PM Patryk Pomykalski pom...@gmail.com wrote:


 Now it fails if you increment the varchar length (and I increased amount
 of data in t2 to trigger this on 32bit too):

 CREATE TABLE t1 (i INT, state VARCHAR(996)) ENGINE=MyISAM;
 INSERT INTO t1 VALUES (2,'Louisiana'),(9,'Maine');

 CREATE TABLE t2 (state VARCHAR(996), j INT) ENGINE=MyISAM;
 INSERT INTO t2 VALUES ('Louisiana',9),('Alaska',5);
 INSERT INTO t2 SELECT t2.* FROM t2 JOIN t2 AS t3 JOIN t2 AS t4 JOIN t2 AS
 t5 JOIN t2 AS t6;

 SET @@max_heap_table_size= 16384;
 set @@optimizer_switch='derived_merge=OFF';
 set @@optimizer_switch='extended_keys=ON';

 SELECT * FROM t1 AS t1_1 LEFT JOIN ( t1 AS t1_2 INNER JOIN (SELECT * FROM
 t2) v2 ON t1_2.i = j ) ON t1_1.state = v2.state LIMIT 1;

 DROP TABLE t1, t2;


 On Tue, Feb 10, 2015 at 1:41 PM, Vicențiu Ciorbaru vicen...@mariadb.org
 wrote:

 + CC maria-developers
 Hi Sergei!

 I have attached the diff for MDEV-6838. Let me know your thoughts on it.
 The test case now fails without the patch and passes with the patch on both
 5.5 and 10.0.

 Sergey Petrunia is alright with the patch.

 Regards,
 Vicențiu

 ___
 Mailing list: https://launchpad.net/~maria-developers
 Post to : maria-developers@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~maria-developers
 More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp