Re: [Maria-developers] MariaDB + PBXT
Paul McCullagh writes: > I must say I find it strange that bzr is merging files with .OTHER > and .BASE endings, because these are temp files which only occur after > a merge conflict. My thinking is that these were accidentally added at some point in the history, and subsequently removed. And when merging this add+remove, we get the conflict because the parent directory does not exist in the merge destination. But I do not know for sure. - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
On 13 May 2009, at 01:52, Paul McCullagh wrote: On May 13, 2009, at 10:37 AM, Kristian Nielsen wrote: Kristian Nielsen writes: One thing I might try is to re-do the merge in smaller steps. There is a _lot_ of add/move/delete/patch the same file in the same changeset, and it is clear that it is very hard to get all the special cases of this right in the bzr code (I would personally lean towards not allowing a single file to be both added, deleted, moved, and/or patched in the same commit). So if I re-do the merge with seperate commits: 1. In pbxt repo, move needed files into pbxt-to-be-merged, delete rest. 2. In MariaDB repo, delete PBXT files from the old merge. 3. Pull renamed pbxt into mariadb without other changes. 4. Move files in pbxt-to-be-merged into place 5. Apply needed patches. Then each individual step will be in a separate commit, which should make the history hopefully more sane. Ok, I tried this, and it did not make any difference, same problem. After re-merging PBXT revision 608 (same as orginal merge), I tried to merge in just revision 609 from PBXT (from a `bzr branch -r609 pbxt` branch): $ bzr merge ../test-merge-source +N mysql-test/suite/pbxt/r/type_varchar.result.OTHER +N mysql-test/suite/pbxt/t/type_varchar.test.OTHER M storage/pbxt/src/ha_pbxt.cc Contents conflict in mysql-test/suite/pbxt/r/type_varchar.result Contents conflict in mysql-test/suite/pbxt/t/type_varchar.test /usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) /usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) bzr: ERROR: No final name for trans_id 'new-1' file-id: None root trans-id: 'new-0' So I think this means we will have to put applying Stewart's technique to the PBXT merge on hold :-(. Yes, unfortunately we cannot proceed with this problem. :( I encountered same issues with my maria-federated merged repository. I found that if I pull into the 'renamed' intermediate repository, I could then merge that into the maria repository but if I try to do it direct, I get the bzr crash as above. (no merge conflicts though). Regards, Antony ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
On May 13, 2009, at 10:37 AM, Kristian Nielsen wrote:
Kristian Nielsen writes:
One thing I might try is to re-do the merge in smaller steps. There
is a _lot_
of add/move/delete/patch the same file in the same changeset, and
it is clear
that it is very hard to get all the special cases of this right in
the bzr
code (I would personally lean towards not allowing a single file to
be both
added, deleted, moved, and/or patched in the same commit).
So if I re-do the merge with seperate commits:
1. In pbxt repo, move needed files into pbxt-to-be-merged, delete
rest.
2. In MariaDB repo, delete PBXT files from the old merge.
3. Pull renamed pbxt into mariadb without other changes.
4. Move files in pbxt-to-be-merged into place
5. Apply needed patches.
Then each individual step will be in a separate commit, which
should make the
history hopefully more sane.
Ok, I tried this, and it did not make any difference, same problem.
After re-merging PBXT revision 608 (same as orginal merge), I tried
to merge
in just revision 609 from PBXT (from a `bzr branch -r609 pbxt`
branch):
$ bzr merge ../test-merge-source
+N mysql-test/suite/pbxt/r/type_varchar.result.OTHER
+N mysql-test/suite/pbxt/t/type_varchar.test.OTHER
M storage/pbxt/src/ha_pbxt.cc
Contents conflict in mysql-test/suite/pbxt/r/type_varchar.result
Contents conflict in mysql-test/suite/pbxt/t/type_varchar.test
/usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91:
UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active
task ProgressTask(None/None, msg='')
% (task, self._task_stack[-1]))
/usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91:
UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active
task ProgressTask(None/None, msg='')
% (task, self._task_stack[-1]))
bzr: ERROR: No final name for trans_id 'new-1'
file-id: None
root trans-id: 'new-0'
So I think this means we will have to put applying Stewart's
technique to the
PBXT merge on hold :-(.
Yes, unfortunately we cannot proceed with this problem. :(
I must say I find it strange that bzr is merging files with .OTHER
and .BASE endings, because these are temp files which only occur after
a merge conflict.
I will report this bug to bzr along with steps to reproduce.
Also, I will try if some of our people with contacts to the bzr
developers can
get them to look into this.
Great!
I see https://bugs.launchpad.net/bzr/+bug/364305 has just been
assigned to Aaron.
In case of interest, below are the steps I used for the new merge
(which I
like much better than the previous even if it did not solve the
problem, as
there were no merge conflicts at all to be solved and it just seems
much
cleaner). Also pushed as
lp:~knielsen/maria/mariadb-5.1-pbxt-merge-take2
Looks good to me, but it may be that the bzr developers can just give
us a hint how to do this in order to avoid the error.
- Kristian.
mkdir pbxt-new-merge-take2
cd pbxt-new-merge-take2/
bzr branch -r'revid:paul.mccull...@primebase.org-20090402202852-wa2fbcmrdy7gda2f
' lp:pbxt pbxt-rename
cd pbxt-rename/
bzr co
bzr mkdir pbxt-to-be-renamed
bzr rm aclocal.m4 config config.h.in configure configure-dzl.in
configure.in COPYING Makefile-dzl.am Makefile.in plug-dzl.in bin
src/Makefile-dzl.am src/Makefile.in
(cd test/mysql-test/t && bzr rm --force )
(cd test/mysql-test/r && bzr rm --force )
bzr mv test/mysql-test/t pbxt-to-be-renamed/
bzr mv test/mysql-test/r pbxt-to-be-renamed/
bzr rm test
bzr mv AUTHORS INSTALL NEWS plug.in src ChangeLog Makefile.am
README TODO pbxt-to-be-renamed/
bzr commit -m$'Prepare for merging PBXT into MariaDB tree:\n -
Delete not needed files.\n - Move needed files into subdirectory to
avoid conflicts.'
cd ..
bzr branch -r'revid:paul.mccull...@primebase.org-20090407105746-tolv5dita1d3eavm
' lp:maria mariadb-pbxt-merge
cd mariadb-pbxt-merge/
bzr co
bzr rm storage/pbxt mysql-test/suite/pbxt/{t,r}
bzr commit -m"Delete old PBXT files in preparation for re-merging
PBXT with proper history and merge tracking."
bzr merge ../pbxt-rename -r0..-1
bzr ci -m"Re-merge PBXT with proper history and merge tracking,
into temporary directory for later renaming into place."
bzr mv pbxt-to-be-renamed/{t,r} mysql-test/suite/pbxt/
bzr mkdir storage/pbxt/
bzr mv pbxt-to-be-renamed/* storage/pbxt/
bzr rm pbxt-to-be-renamed/
bzr ci -m"Re-merge PBXT into MariaDB with proper history and merge
tracking: move files into correct place."
patch -p0 -s < /tmp/1
(cd mysql-test/suite/pbxt/ && patch -p0 -s < /tmp/2)
(cd mysql-test/suite/pbxt/ && patch -p0 -s < /tmp/3)
bzr ci -m"After-merge fixes for re-merge of PBXT into MariaDB with
proper history and merge tracking."
--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com
___
Mailing list: https://launchpad.net/~maria-d
Re: [Maria-developers] MariaDB + PBXT
Kristian Nielsen writes:
> One thing I might try is to re-do the merge in smaller steps. There is a _lot_
> of add/move/delete/patch the same file in the same changeset, and it is clear
> that it is very hard to get all the special cases of this right in the bzr
> code (I would personally lean towards not allowing a single file to be both
> added, deleted, moved, and/or patched in the same commit).
>
> So if I re-do the merge with seperate commits:
>
> 1. In pbxt repo, move needed files into pbxt-to-be-merged, delete rest.
> 2. In MariaDB repo, delete PBXT files from the old merge.
> 3. Pull renamed pbxt into mariadb without other changes.
> 4. Move files in pbxt-to-be-merged into place
> 5. Apply needed patches.
>
> Then each individual step will be in a separate commit, which should make the
> history hopefully more sane.
Ok, I tried this, and it did not make any difference, same problem.
After re-merging PBXT revision 608 (same as orginal merge), I tried to merge
in just revision 609 from PBXT (from a `bzr branch -r609 pbxt` branch):
$ bzr merge ../test-merge-source
+N mysql-test/suite/pbxt/r/type_varchar.result.OTHER
+N mysql-test/suite/pbxt/t/type_varchar.test.OTHER
M storage/pbxt/src/ha_pbxt.cc
Contents conflict in mysql-test/suite/pbxt/r/type_varchar.result
Contents conflict in mysql-test/suite/pbxt/t/type_varchar.test
/usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning:
ProgressTask(0/2, msg='Apply phase') is not the active task
ProgressTask(None/None, msg='')
% (task, self._task_stack[-1]))
/usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning:
ProgressTask(2/3, msg='Merge phase') is not the active task
ProgressTask(None/None, msg='')
% (task, self._task_stack[-1]))
bzr: ERROR: No final name for trans_id 'new-1'
file-id: None
root trans-id: 'new-0'
So I think this means we will have to put applying Stewart's technique to the
PBXT merge on hold :-(.
I will report this bug to bzr along with steps to reproduce.
Also, I will try if some of our people with contacts to the bzr developers can
get them to look into this.
In case of interest, below are the steps I used for the new merge (which I
like much better than the previous even if it did not solve the problem, as
there were no merge conflicts at all to be solved and it just seems much
cleaner). Also pushed as
lp:~knielsen/maria/mariadb-5.1-pbxt-merge-take2
- Kristian.
mkdir pbxt-new-merge-take2
cd pbxt-new-merge-take2/
bzr branch
-r'revid:paul.mccull...@primebase.org-20090402202852-wa2fbcmrdy7gda2f' lp:pbxt
pbxt-rename
cd pbxt-rename/
bzr co
bzr mkdir pbxt-to-be-renamed
bzr rm aclocal.m4 config config.h.in configure configure-dzl.in configure.in
COPYING Makefile-dzl.am Makefile.in plug-dzl.in bin src/Makefile-dzl.am
src/Makefile.in
(cd test/mysql-test/t && bzr rm --force )
(cd test/mysql-test/r && bzr rm --force )
bzr mv test/mysql-test/t pbxt-to-be-renamed/
bzr mv test/mysql-test/r pbxt-to-be-renamed/
bzr rm test
bzr mv AUTHORS INSTALL NEWS plug.in src ChangeLog Makefile.am README TODO
pbxt-to-be-renamed/
bzr commit -m$'Prepare for merging PBXT into MariaDB tree:\n - Delete not
needed files.\n - Move needed files into subdirectory to avoid conflicts.'
cd ..
bzr branch
-r'revid:paul.mccull...@primebase.org-20090407105746-tolv5dita1d3eavm' lp:maria
mariadb-pbxt-merge
cd mariadb-pbxt-merge/
bzr co
bzr rm storage/pbxt mysql-test/suite/pbxt/{t,r}
bzr commit -m"Delete old PBXT files in preparation for re-merging PBXT with
proper history and merge tracking."
bzr merge ../pbxt-rename -r0..-1
bzr ci -m"Re-merge PBXT with proper history and merge tracking, into
temporary directory for later renaming into place."
bzr mv pbxt-to-be-renamed/{t,r} mysql-test/suite/pbxt/
bzr mkdir storage/pbxt/
bzr mv pbxt-to-be-renamed/* storage/pbxt/
bzr rm pbxt-to-be-renamed/
bzr ci -m"Re-merge PBXT into MariaDB with proper history and merge tracking:
move files into correct place."
patch -p0 -s < /tmp/1
(cd mysql-test/suite/pbxt/ && patch -p0 -s < /tmp/2)
(cd mysql-test/suite/pbxt/ && patch -p0 -s < /tmp/3)
bzr ci -m"After-merge fixes for re-merge of PBXT into MariaDB with proper
history and merge tracking."
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Paul McCullagh writes: > Looks like this bug was reported already: > > https://bugs.launchpad.net/bzr/+bug/364305 Yes, this looks like the same problem. > The question is: is this a show stopper for us at the moment? Yes, I think so. I do not want to merge this and run the risk of a corrupted repository in some way (ie. I am fairly certain that this is a bzr bug, but I am not certain whether the bug is when making my original tree, or when the bzr error message occurs during testing the merge). One thing I might try is to re-do the merge in smaller steps. There is a _lot_ of add/move/delete/patch the same file in the same changeset, and it is clear that it is very hard to get all the special cases of this right in the bzr code (I would personally lean towards not allowing a single file to be both added, deleted, moved, and/or patched in the same commit). So if I re-do the merge with seperate commits: 1. In pbxt repo, move needed files into pbxt-to-be-merged, delete rest. 2. In MariaDB repo, delete PBXT files from the old merge. 3. Pull renamed pbxt into mariadb without other changes. 4. Move files in pbxt-to-be-merged into place 5. Apply needed patches. Then each individual step will be in a separate commit, which should make the history hopefully more sane. - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Hi Kristian, Looks like this bug was reported already: https://bugs.launchpad.net/bzr/+bug/364305 Maybe we can all go in and click the "This bug doesn't affect me" link to bump up the priority. The question is: is this a show stopper for us at the moment? On May 12, 2009, at 5:30 PM, Kristian Nielsen wrote: Paul McCullagh writes: I am just trying to merge for the second time into mariadb-5.1-pbxt- merge-new, and get the following: pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% bzr merge lp:pbxt M storage/pbxt/src/cache_xt.cc M storage/pbxt/src/ha_pbxt.cc M storage/pbxt/src/index_xt.h M storage/pbxt/src/lock_xt.cc M storage/pbxt/src/lock_xt.h M storage/pbxt/src/table_xt.h /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) bzr: ERROR: No final name for trans_id 'new-1' file-id: None root trans-id: 'new-0' Yes, this looks similar to what I have seen. If you can repeat it, we should report it to the bzr guys. Hopefully they can fix it. (let me know if you will report it, else I will). - Kristian. -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Paul McCullagh writes: > I am just trying to merge for the second time into mariadb-5.1-pbxt- > merge-new, and get the following: > > pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% bzr merge > lp:pbxt > M storage/pbxt/src/cache_xt.cc > M storage/pbxt/src/ha_pbxt.cc > M storage/pbxt/src/index_xt.h > M storage/pbxt/src/lock_xt.cc > M storage/pbxt/src/lock_xt.h > M storage/pbxt/src/table_xt.h > /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: > UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active > task ProgressTask(None/None, msg='') > % (task, self._task_stack[-1])) > /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: > UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active > task ProgressTask(None/None, msg='') > % (task, self._task_stack[-1])) > bzr: ERROR: No final name for trans_id 'new-1' > file-id: None > root trans-id: 'new-0' Yes, this looks similar to what I have seen. If you can repeat it, we should report it to the bzr guys. Hopefully they can fix it. (let me know if you will report it, else I will). - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Hi Kristian and Stewart, On May 11, 2009, at 4:24 PM, Kristian Nielsen wrote: I did a quick test merge of newest PBXT just to check things, and it seems to work well. Most changes automerged. For deleted/renamed/newly added files, a conflict is flagged that must be resolved manually. Changes to files in PBXT that are omitted from the MariaDB source (COPYING, README, etc) also are flagged as a conflict that needs to be resolved. This is to be expected. I did see some wierd stuff in some testing. Eg. the `bzr diff` of my merge changeset is not empty as expected, rather it shows one diff removing a file and another identical (but reverse) diff adding that file back. I also saw a strange bzr error message in one case of testing merge. I guess we are (ab)using some uncommon corners of bzr here, but hopefully any problems/bug in bzr can be fixed. I am just trying to merge for the second time into mariadb-5.1-pbxt- merge-new, and get the following: pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% bzr merge lp:pbxt M storage/pbxt/src/cache_xt.cc M storage/pbxt/src/ha_pbxt.cc M storage/pbxt/src/index_xt.h M storage/pbxt/src/lock_xt.cc M storage/pbxt/src/lock_xt.h M storage/pbxt/src/table_xt.h /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) bzr: ERROR: No final name for trans_id 'new-1' file-id: None root trans-id: 'new-0' The merge was aborted because, after this I have: [pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% bzr diff . [pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% No diff. Any ideas? -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Hi Kristian, On May 11, 2009, at 4:24 PM, Kristian Nielsen wrote: Stewart, thanks for providing this procedure! Nice to know that this method with renaming to common paths and merging can work. Paul McCullagh writes: So to correct this I will have to redo the integration of PBXT is into MariaDB. Yes, I think we should re-do the merge with this tracking of history and merging. I spent some time today playing with this technique, as I would like to be really sure that we have a good base for working with the PBXT code in the future, and also I would like to use the same technique for merging XtraDB. So I created a MariaDB tree with the PBXT merge re-done using Stewart's technique, taking some care to get everything done correctly to smooth the future work. If you want to use it as-is, I write the exact commands I used below if you want to do something similar for Drizzle. If you want to re-do it yourself to tweak it or control things fully, that is perfectly fine with me as well. Whatever works best for you. Wow, excellent! Thanks, you have saved me the work :) Just let me know if you want me to push this tree to MariaDB, or prepare another tree for me to merge. I put my merged tree here: lp:~knielsen/maria/mariadb-5.1-pbxt-merge-new I just built it and tested: bzr merge lp:pbxt And all looks good :) Note that this tree is identical source-wise with the existing lp:maria tree, just the history now includes all of the PBXT changesets so history and merges can be tracked as per Stewarts method. So this does not include newest PBXT yet, just PBXT up to revision 608 (revid:paul.mccull...@primebase.org-20090402202852-wa2fbcmrdy7gda2f). OK. Great. I did a quick test merge of newest PBXT just to check things, and it seems to work well. Most changes automerged. For deleted/renamed/newly added files, a conflict is flagged that must be resolved manually. Changes to files in PBXT that are omitted from the MariaDB source (COPYING, README, etc) also are flagged as a conflict that needs to be resolved. This is to be expected. Yup. OK. I did see some wierd stuff in some testing. Eg. the `bzr diff` of my merge changeset is not empty as expected, rather it shows one diff removing a file and another identical (but reverse) diff adding that file back. I also saw a strange bzr error message in one case of testing merge. I guess we are (ab)using some uncommon corners of bzr here, but hopefully any problems/bug in bzr can be fixed. One think I wonder is what to do with the PBXT test suite in mysql-test/suite/pbxt/. In my merge I merged it into place with `bzr mv` so that merge tracking and history is kept. So I am not entirely sure how I should handle changes in the future. Maybe you have a suggestion. However, I don't know how well this will work for .result files, as I guess they are going to differ between MariaDB, Drizzle, and MySQL? Yes, this is a problem. And some of the tests have also be changed for MariaDB. Not 100% sure how we will deal with this. Could be that .result files cannot be meaningfully merged automatically. In any case, I think this is a usable starting point, and we can refine the .result handling later as we understand better how to do it. Just let me know if you think otherwise. Yes, I think we should just leave it as is for the moment, until we have a better idea. My original idea with the PBXT test suite is that it will act as a basic verification for the engine within MariaDB. The main testing of PBXT will probably continue as a plug-in for MySQL. In the PBXT tree, and can do "make test", and PBXT is automatically tested using the MySQL server that it was compiled with. So I was thinking of PBXT test suite for MariaDB as a branch of the PBXT tests that would never be merged again. Instead any bugs that turn up in MariaDB with PBXT would be added to this test suite. And, the test suite will be continually updated for any changes in MariaDB itself. This is similar to how things are in Drizzle, where PBXT is also tested against a different set of tests. Here I was also not planning to merge tests between Drizzle and the PBXT tree. One good thing about the Drizzle tests though, is that the same tests run with both InnoDB and PBXT. Would be great to add something like that to MariaDB. Here are the exact commands I did to make the merge: bzr branch -r'revid:paul.mccull...@primebase.org-20090402202852-wa2fbcmrdy7gda2f ' pbxt tmp-pbxt-rename cd tmp-pbxt-rename/ bzr co bzr mkdir storage bzr mkdir storage/pbxt bzr mv `ls|grep -v storage` storage/pbxt/ bzr commit -m"Move all PBXT files into storage/pbxt/ in preparation for merging into MariaDB." cd .. bzr branch -r'revid:paul.mccull...@primebase.org-20090407105746-tolv5dita1d3eavm ' mariadb-5.1 tmp-mariadb-pbxt-merge cd tmp-mariadb-pbxt-merge/ bzr co bzr merge ../tmp-pbxt-rename -r0..-1 bzr
Re: [Maria-developers] MariaDB + PBXT
Stewart, thanks for providing this procedure! Nice to know that this method with renaming to common paths and merging can work. Paul McCullagh writes: > So to correct this I will have to redo the integration of PBXT is into > MariaDB. Yes, I think we should re-do the merge with this tracking of history and merging. I spent some time today playing with this technique, as I would like to be really sure that we have a good base for working with the PBXT code in the future, and also I would like to use the same technique for merging XtraDB. So I created a MariaDB tree with the PBXT merge re-done using Stewart's technique, taking some care to get everything done correctly to smooth the future work. If you want to use it as-is, I write the exact commands I used below if you want to do something similar for Drizzle. If you want to re-do it yourself to tweak it or control things fully, that is perfectly fine with me as well. Whatever works best for you. Just let me know if you want me to push this tree to MariaDB, or prepare another tree for me to merge. I put my merged tree here: lp:~knielsen/maria/mariadb-5.1-pbxt-merge-new Note that this tree is identical source-wise with the existing lp:maria tree, just the history now includes all of the PBXT changesets so history and merges can be tracked as per Stewarts method. So this does not include newest PBXT yet, just PBXT up to revision 608 (revid:paul.mccull...@primebase.org-20090402202852-wa2fbcmrdy7gda2f). I did a quick test merge of newest PBXT just to check things, and it seems to work well. Most changes automerged. For deleted/renamed/newly added files, a conflict is flagged that must be resolved manually. Changes to files in PBXT that are omitted from the MariaDB source (COPYING, README, etc) also are flagged as a conflict that needs to be resolved. This is to be expected. I did see some wierd stuff in some testing. Eg. the `bzr diff` of my merge changeset is not empty as expected, rather it shows one diff removing a file and another identical (but reverse) diff adding that file back. I also saw a strange bzr error message in one case of testing merge. I guess we are (ab)using some uncommon corners of bzr here, but hopefully any problems/bug in bzr can be fixed. One think I wonder is what to do with the PBXT test suite in mysql-test/suite/pbxt/. In my merge I merged it into place with `bzr mv` so that merge tracking and history is kept. However, I don't know how well this will work for .result files, as I guess they are going to differ between MariaDB, Drizzle, and MySQL? Not 100% sure how we will deal with this. Could be that .result files cannot be meaningfully merged automatically. In any case, I think this is a usable starting point, and we can refine the .result handling later as we understand better how to do it. Just let me know if you think otherwise. Here are the exact commands I did to make the merge: bzr branch -r'revid:paul.mccull...@primebase.org-20090402202852-wa2fbcmrdy7gda2f' pbxt tmp-pbxt-rename cd tmp-pbxt-rename/ bzr co bzr mkdir storage bzr mkdir storage/pbxt bzr mv `ls|grep -v storage` storage/pbxt/ bzr commit -m"Move all PBXT files into storage/pbxt/ in preparation for merging into MariaDB." cd .. bzr branch -r'revid:paul.mccull...@primebase.org-20090407105746-tolv5dita1d3eavm' mariadb-5.1 tmp-mariadb-pbxt-merge cd tmp-mariadb-pbxt-merge/ bzr co bzr merge ../tmp-pbxt-rename -r0..-1 bzr mv storage storage.new bzr mv storage.moved storage bzr rm storage/pbxt bzr rm mysql-test/suite/pbxt/t bzr rm mysql-test/suite/pbxt/r bzr mv storage.new/pbxt/test/mysql-test/t mysql-test/suite/pbxt/t bzr mv storage.new/pbxt/test/mysql-test/r mysql-test/suite/pbxt/r bzr mv storage.new/pbxt storage/pbxt bzr rm --force storage.new # bzr rm + manual patch. Applying same changes done manually in previous PBXT merge. bzr resolved storage bzr ci -m "Re-merge PBXT into MariaDB to get proper history and merge tracking." cd .. bzr branch mariadb-5.1 mariadb-5.1-pbxt-merge-new cd mariadb-5.1-pbxt-merge-new bzr co bzr merge ../tmp-mariadb-pbxt-merge bzr ci -m "Pull new/improved PBXT merge into latest MariaDB trunk." So I used the revisions of MariaDB and PBXT that were also used to do the original merge. And I manually applied the same changes in the merge that you apparently did in your original merge of PBXT into MariaDB, mainly deleting some files like COPYING, configure.in, removing a number of non-pbxt test cases, and fixing some .result files. So I hope this will be a good base for future merging, with this it should be possible for either PBXT or MariaDB people to pull latest from PBXT into MariaDB without either getting in the way of the other. And it should be quite useful to us to have PBXT history available if debugging etc. Any comments welcome. > On May 11, 2009, at 7:17 AM, Stewart Smith wrote: >> With BZR you can merge two trees without a common ancestor. This is
Re: [Maria-developers] MariaDB + PBXT
On Mon, May 11, 2009 at 10:38:19AM +0200, Paul McCullagh wrote: > I was not aware of this bit of magic! :) > > So to correct this I will have to redo the integration of PBXT is into > MariaDB. > > I think it would also be best if I redid the integration of PBXT into > the Drizzle/PBXT tree because some of the merges have been done > manually, so the history is missing. err... should be okay if done off the tree I originally did for Drizzle (i think this has been the case) > Or would I get the same effect if I just followed Stewart's procedure > and merged over the pbxt files that already exist (in the MariaDB and > Drizzle/PBXT trees)? That may not be the best... probably lots of conflicts on files and having to work out which is right (and conflicts on directory paths too). best to extract patches you want, commit upstream, then merge in. -- Stewart Smith ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Hi Stewart & Kristian, I was not aware of this bit of magic! :) So to correct this I will have to redo the integration of PBXT is into MariaDB. I think it would also be best if I redid the integration of PBXT into the Drizzle/PBXT tree because some of the merges have been done manually, so the history is missing. Or would I get the same effect if I just followed Stewart's procedure and merged over the pbxt files that already exist (in the MariaDB and Drizzle/PBXT trees)? Best regards, Paul On May 11, 2009, at 7:17 AM, Stewart Smith wrote: On Thu, May 07, 2009 at 02:18:02PM +0200, Kristian Nielsen wrote: I thought a bit more about the issues of merging PBXT (and later XtraDB) changes into MariaDB. I would like to understand better how to do it. The main bzr repositories of both PBXT and XtraDB contain just the storage engine, and have no common history. So as I understand it, I cannot just pull in the changes using bzr (I tried bzr merge, and got an error due to no common ancestor). You can :) With BZR you can merge two trees without a common ancestor. This is useful for things like storage engines, file systems, so when they're merged, you can bring the whole revision history in. This is how I did it for PBXT and Drizzle: bzr branch lp:drizzle drizzle-pbxt bzr branch lp:pbxt pbxt bzr branch pbxt pbxt-rename pushd pbxt-rename mkdir storage/pbxt Is this correct, or should it be bzr mkdir storage/pbxt? bzr mv * storage/pbxt/ bzr commit popd pushd drizzle-pbxt bzr merge ../pbxt-rename -r1..-1 (fix up a conflict on storage/ or something... pretty easy, this is just done so that auto merges happen on file ops in main pbxt) bzr commit and then, when mainline pbxt is updated pushd drizzle-pbxt bzr merge lp:pbxt and it magically works. You cannot, however then merge back from drizzle-pbxt into pbxt - so either someone has to extract patches and deal with the no-op merge when coming back or all dev moves to drizzle-pbxt derived branches. Since we have two sep trees (MariaDB and Drizzle) both merging in PBXT, it makes sense for PBXT development to continue in the PBXT tree and then we can both merge PBXT into our trees from it. If we get patches and fixes back into the main PBXT tree, then I think we all live in some magic world of harmony :) It should also be (relatively) easy for someone to whip up a script that automatically tests a PBXT commit against current Drizzle and MariaDB. Hope this helps -- Stewart Smith -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
On Thu, May 07, 2009 at 02:18:02PM +0200, Kristian Nielsen wrote: > I thought a bit more about the issues of merging PBXT (and later XtraDB) > changes into MariaDB. I would like to understand better how to do it. > > The main bzr repositories of both PBXT and XtraDB contain just the storage > engine, and have no common history. So as I understand it, I cannot just pull > in the changes using bzr (I tried bzr merge, and got an error due to no common > ancestor). You can :) With BZR you can merge two trees without a common ancestor. This is useful for things like storage engines, file systems, so when they're merged, you can bring the whole revision history in. This is how I did it for PBXT and Drizzle: bzr branch lp:drizzle drizzle-pbxt bzr branch lp:pbxt pbxt bzr branch pbxt pbxt-rename pushd pbxt-rename mkdir storage/pbxt bzr mv * storage/pbxt/ bzr commit popd pushd drizzle-pbxt bzr merge ../pbxt-rename -r1..-1 (fix up a conflict on storage/ or something... pretty easy, this is just done so that auto merges happen on file ops in main pbxt) bzr commit and then, when mainline pbxt is updated pushd drizzle-pbxt bzr merge lp:pbxt and it magically works. You cannot, however then merge back from drizzle-pbxt into pbxt - so either someone has to extract patches and deal with the no-op merge when coming back or all dev moves to drizzle-pbxt derived branches. Since we have two sep trees (MariaDB and Drizzle) both merging in PBXT, it makes sense for PBXT development to continue in the PBXT tree and then we can both merge PBXT into our trees from it. If we get patches and fixes back into the main PBXT tree, then I think we all live in some magic world of harmony :) It should also be (relatively) easy for someone to whip up a script that automatically tests a PBXT commit against current Drizzle and MariaDB. Hope this helps -- Stewart Smith ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Hi Kristian, No real plan here... :( My idea was basically to do this manually, which is admittedly a bit of a pain. It requires at about 3 steps: 1. Get a patch for the changes to the PBXT tree in MariaDB since the last time PBXT was merged. 2. Apply the patch to the current PBXT tree, compile, test and push. 3. Copy the PBXT over to MariaBD, compile, test and push. As you mentioned, change histories are lost in the MariaDB tree with this method. I would welcome any suggestions for a better method... Best regards, Paul On May 7, 2009, at 2:18 PM, Kristian Nielsen wrote: Paul McCullagh writes: On Apr 6, 2009, at 1:17 PM, Kristian Nielsen wrote: Do you have a plan for merging in subsequent PBXT changes to MariaDB? (I guess you do, just don't know how it works). Would you be in charge of regular merging, or should the MariaDB team pull/merge from PBXT? Plan is to merge regularly and, of course, just before we make a release. However, we have not worked out a schedule yet. Generally we will make sure all our tests run through with MySQL before we merge into MariaDB. However, you are free to pull the latest PBXT tree and merge it into MariaDB any time you like. If you make any changes or fixes I would be glad to accept a patch for PBXT. I thought a bit more about the issues of merging PBXT (and later XtraDB) changes into MariaDB. I would like to understand better how to do it. The main bzr repositories of both PBXT and XtraDB contain just the storage engine, and have no common history. So as I understand it, I cannot just pull in the changes using bzr (I tried bzr merge, and got an error due to no common ancestor). However, Arjen Lentz told me you had some technique you are using to control merging into mysql, drizzle, and mariadb. Care to share it? ;-) As I see it, the way we can do it now is to import PBXT (and XtraDB) snapshots into MariaDB. So keep a separate maria clone which is never updated, except that from time to time the storage/pbxt directory is replaced with a snapshot of PBXT, followed by commit. We can then bzr merge from this into main MariaDB. So essentially ignoring the PBXT bzr history. Is this how you do / plan to do it? Or do you have a smarter way? It would be great if we could do this some bzr-supported way, so that we could have the PBXT (and XtraDB) history available within the MariaDB repository (it is not yet). But as I understand, this is not possible at the moment (correct me if I'm wrong). There is some planned bzr feature of nested-trees, but it appears to not be ready yet. Anyway, I am in no way trying to change how you are doing or planning things, just trying to understand what the plan is :-) Thanks, - Kristian. -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Paul McCullagh writes: > On Apr 6, 2009, at 1:17 PM, Kristian Nielsen wrote: >> Do you have a plan for merging in subsequent PBXT changes to >> MariaDB? (I guess >> you do, just don't know how it works). Would you be in charge of >> regular >> merging, or should the MariaDB team pull/merge from PBXT? > > Plan is to merge regularly and, of course, just before we make a > release. However, we have not worked out a schedule yet. > > Generally we will make sure all our tests run through with MySQL > before we merge into MariaDB. > > However, you are free to pull the latest PBXT tree and merge it into > MariaDB any time you like. If you make any changes or fixes I would be > glad to accept a patch for PBXT. I thought a bit more about the issues of merging PBXT (and later XtraDB) changes into MariaDB. I would like to understand better how to do it. The main bzr repositories of both PBXT and XtraDB contain just the storage engine, and have no common history. So as I understand it, I cannot just pull in the changes using bzr (I tried bzr merge, and got an error due to no common ancestor). However, Arjen Lentz told me you had some technique you are using to control merging into mysql, drizzle, and mariadb. Care to share it? ;-) As I see it, the way we can do it now is to import PBXT (and XtraDB) snapshots into MariaDB. So keep a separate maria clone which is never updated, except that from time to time the storage/pbxt directory is replaced with a snapshot of PBXT, followed by commit. We can then bzr merge from this into main MariaDB. So essentially ignoring the PBXT bzr history. Is this how you do / plan to do it? Or do you have a smarter way? It would be great if we could do this some bzr-supported way, so that we could have the PBXT (and XtraDB) history available within the MariaDB repository (it is not yet). But as I understand, this is not possible at the moment (correct me if I'm wrong). There is some planned bzr feature of nested-trees, but it appears to not be ready yet. Anyway, I am in no way trying to change how you are doing or planning things, just trying to understand what the plan is :-) Thanks, - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
On Apr 7, 2009, at 2:21 PM, Kristian Nielsen wrote: Paul McCullagh writes: The problem is occurring when running the PBXT test suite from the Maria mysql-test directory. So maybe the problem is that udf_example.so was not built. That was my thought too, but I checked, and the udf_example.so was present. And beside, even if it was not, the testcase does have contain a statement to skip the test when the udf is not there. Hm. Maybe the path is wrong, perhaps due to running in an different suite from main.udf? Anyway, if you cannot repeat at your end I can take a closer look. OK, we will try this on Linux, and let you know if we cannot repeat. -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Paul McCullagh writes: > The problem is occurring when running the PBXT test suite from the > Maria mysql-test directory. > > So maybe the problem is that udf_example.so was not built. That was my thought too, but I checked, and the udf_example.so was present. And beside, even if it was not, the testcase does have contain a statement to skip the test when the udf is not there. Hm. Maybe the path is wrong, perhaps due to running in an different suite from main.udf? Anyway, if you cannot repeat at your end I can take a closer look. - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
On Apr 7, 2009, at 1:19 PM, Vladimir Kolesnikov wrote: pbxt.udf [ fail ] Test ended at 2009-04-07 12:11:59 CURRENT_TEST: pbxt.udf mysqltest: At line 18: query 'CREATE FUNCTION metaphon RETURNS STRING SONAME "$UDF_EXAMPLE_LIB"' failed: 1126: Can't open shared library 'udf_example.so' (errno: 22 (null)) The result from queries just before the failure was: drop table if exists t1; Hmmm, not sure what the problem is here. Will investigate this usually happens because udf_example.so is not in the plugin_dir - which might be because it's not built. Another reason, if you "make test" from pbxt root - I simply copy udf_example.so to pbxt .libs directory and set up --plugin_dir to it, but this relies on the proper --with-mysql pbxt config option. PBXT is part of the MariaDB tree, and does not have its own config. The problem is occurring when running the PBXT test suite from the Maria mysql-test directory. So maybe the problem is that udf_example.so was not built. -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
We will look into this. Vlad: would it be possible to give the Maria buildbot access to dev8? Or will our firewall be a problem...? We need to investigate this. pbxt.udf [ fail ] Test ended at 2009-04-07 12:11:59 CURRENT_TEST: pbxt.udf mysqltest: At line 18: query 'CREATE FUNCTION metaphon RETURNS STRING SONAME "$UDF_EXAMPLE_LIB"' failed: 1126: Can't open shared library 'udf_example.so' (errno: 22 (null)) The result from queries just before the failure was: drop table if exists t1; Hmmm, not sure what the problem is here. Will investigate this usually happens because udf_example.so is not in the plugin_dir - which might be because it's not built. Another reason, if you "make test" from pbxt root - I simply copy udf_example.so to pbxt .libs directory and set up --plugin_dir to it, but this relies on the proper --with-mysql pbxt config option. Best regards, Paul -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com -- -- Best Regards, Vladimir ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Hi Kristian, On Apr 7, 2009, at 12:48 PM, Kristian Nielsen wrote: Paul McCullagh writes: On Apr 6, 2009, at 1:17 PM, Kristian Nielsen wrote: Just let me know if you need more help/information on these failures. Yes, thanks. I would like to get these fixed. I have attached a log of running the test suite on my machine. This is an Ubuntu Hardy 64-bit OS running on an Intel Q6600 quad-core CPU. It was built with BUILD/compile-pentium64-debug-max from your maria-pbxt branch. Let me know if you need further information. Also, I should be able to help more once I get a buildbot slave up running the pbxt tests. We will look into this. Vlad: would it be possible to give the Maria buildbot access to dev8? Or will our firewall be a problem...? OK, I will push it. Great, thanks. Is now pushed! :) pbxt.lowercase_table_grant [ fail ] ... GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' -GRANT ALL PRIVILEGES ON `mysqltest`.* TO 'mysqltest_1'@'localhost' +GRANT ALL PRIVILEGES ON `MySQLtest`.* TO 'mysqltest_1'@'localhost' select * from db where user = 'mysqltest_1'; ... Strange diff with 'mysqltest' and 'MySQLtest'. This may have to do with the fact that I test in my Mac with case-insensitive file system. Will check on Linux. pbxt.pbxt_bugs [ fail ] Test ended at 2009-04-07 12:09:04 CURRENT_TEST: pbxt.pbxt_bugs mysqltest: At line 924: query 'LOAD DATA LOCAL INFILE 'suite/pbxt/t/ load_unique_error1.inc' REPLACE INTO TABLE t1 FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (@c1,c2) SET c1 = @c1 % 2' failed: 2: File 'suite/pbxt/t/load_unique_error1.inc' not found (Errcode: 2) The result from queries just before the failure was: < snip > Sorry, forgot: 'load_unique_error1.inc'. Now pushed. pbxt.udf [ fail ] Test ended at 2009-04-07 12:11:59 CURRENT_TEST: pbxt.udf mysqltest: At line 18: query 'CREATE FUNCTION metaphon RETURNS STRING SONAME "$UDF_EXAMPLE_LIB"' failed: 1126: Can't open shared library 'udf_example.so' (errno: 22 (null)) The result from queries just before the failure was: drop table if exists t1; Hmmm, not sure what the problem is here. Will investigate Best regards, Paul -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Paul McCullagh writes: > On Apr 6, 2009, at 1:17 PM, Kristian Nielsen wrote: >> Just let me know if you need more help/information on these failures. > > Yes, thanks. I would like to get these fixed. I have attached a log of running the test suite on my machine. This is an Ubuntu Hardy 64-bit OS running on an Intel Q6600 quad-core CPU. It was built with BUILD/compile-pentium64-debug-max from your maria-pbxt branch. Let me know if you need further information. Also, I should be able to help more once I get a buildbot slave up running the pbxt tests. > OK, I will push it. Great, thanks. - Kristian. Script started on 2009-04-07T11:56:10 CEST ]0;kniel...@loke: ~/devel/maria/my/maria-pbxtkniel...@loke:~/devel/maria/my/maria-pbxt$ (cd mysql-test && perl mysql-test-run .pl --mysqld=--default-storage-engine=pbxt --suite=pbxt --force)[C[C-force)[C[C-force)s--force)k--force)i--force)p--force)[C[C-force)[C[1Pforce)[1P--force)[1P--force)[1P--force)[1P--force)r--force)e--force)p--force)e--force)a--force)t--force)[1P--force)[1P--force)[1P--force)[1P--force)t--force)r--force)y--force)=--force)1--force) --force)[C[C-force)[C[C-force)m--force)a--force)x--force)[C[C-force) [A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[Ct--force) )[A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[Ce--force)[A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[Cs--force)[A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[Ct--force)[A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C-force)[A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[Cf--force)[A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[Ca--force)[A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[Ci--force) l--force) [C=--force) [C[C0--force) [C[C[C --force) [C[C[C[C Logging: mysql-test-run.pl --mysqld=--default-storage-engine=pbxt --suite=pbxt --retry=1 --max-test-fail=0 --force MySQL Version 5.1.32 Using default engine 'pbxt' Checking supported features... - skipping ndbcluster, mysqld not compiled with ndbcluster - skipping SSL - binaries are debug compiled Collecting tests... vardir: /home/knielsen/devel/maria/my/maria-pbxt/mysql-test/var Checking leftover processes... Removing old var directory... Creating var directory '/home/knielsen/devel/maria/my/maria-pbxt/mysql-test/var'... Installing system database... Using server port 53722 Removing slot 251 used by missing process 31016 Removing slot 250 used by missing process 31015 worker[1] Using MTR_BUILD_THREAD 250, with reserved ports 12500..12509 worker[1] - 'localhost:12500' was not free worker[1] mysql-test-run: *** ERROR: Some port(s) was not free == TEST RESULT TIME (ms) mysql-test-run: *** ERROR: Test suite aborted ]0;kniel...@loke: ~/devel/maria/my/maria-pbxtkniel...@loke:~/devel/maria/my/maria-pbxt$ (cd mysql-test && perl mysql-test-run. .pl --mysqld=--default-storage-engine=pbxt --suite=pbxt --retry=1 --max-test-fail l=0 --force)[A[A[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[C[CMperl mysql-test-run.pl --mysqld=--default-storage-engine=pbxt --suite=pbxt --retry
Re: [Maria-developers] MariaDB + PBXT
Hi Kristian, On Apr 6, 2009, at 1:17 PM, Kristian Nielsen wrote: It resulted in me, and all other captains, getting a HUGE diff in an e- mail - SORRY! Mac Mail really struggled, Yikes! Don't worry. Anyway, I found a setting on Launchpad for limiting diff sizes in emails. I changed it to max 5000 lines (from unlimited). Not 100% sure it was the right place I changed though, Launchpad can be a bit confusing sometimes. Ah, good to know. I'll give that a try as well. This branch includes the current MariaDB revision (2689) and the PBXT storage engine. The only changes are the addition of a storage/pbxt and a mysql-test/suite/pbxt directory. PBXT seems to be included in BUILD/compile-pentium64-debug-max, great! The mysql-test/suite/pbxt directory contains a test suite for PBXT. The test includes 167 test files. All tests run through without error. To run the tests use: ./mysql-test-run --mysqld=--default-storage-engine=pbxt --suite=pbxt In other words, you have to set the default storage engine to PBXT for this test to run. Testsuite did not pass without errors for me in the MariaDB tree. I saw these failures: pbxt.count_distinct2 (glibc detected double free error) pbxt.count_distinct3 (mysqld crash) pbxt.lowercase_table_grant pbxt.lowercase_table_qcache pbxt.lowercase_view pbxt.pbxt_bugs pbxt.udf The count* tests did not fail everytime I ran the tests. The *lowercase* test failures seems related to case differences. Test pbxt_bugs is missing file suite/pbxt/t/load_unique_error1.inc. The test pbxt.udf complains about missing udf_example.so (not sure why). Just let me know if you need more help/information on these failures. Yes, thanks. I would like to get these fixed. However, since the PBXT test suite is not run by default, I think we can merge it anyway now, and then work on getting these issues solved afterwards. OK I have set up Buildbot to automatically test the MariaDB code, and I need to find a build slave somewhere that could run tests of the PBXT code (feel free to offer to run one if you have a spare machine handy ;-). That would be helpful in getting a clean PBXT test run (and keeping it clean). Yes, OK. I think we can work something out. Please let me know if this is OK for merge. This is ok to push as it is. Since you are a member of maria-captains on Launchpad, you can go ahead and push the merge to lp:maria yourself. (But if you prefer, I can also pull from your tree and push the merge). OK, I will push it. [The general policy for pushing to MariaDB is that you can push yourself once you get an 'ok to push' from review. Small fixes to a broken build etc. are sometimes pushed immediately to avoid delaying other work, and then reviewed after the fact.] Yup, sounds good. Do you have a plan for merging in subsequent PBXT changes to MariaDB? (I guess you do, just don't know how it works). Would you be in charge of regular merging, or should the MariaDB team pull/merge from PBXT? Plan is to merge regularly and, of course, just before we make a release. However, we have not worked out a schedule yet. Generally we will make sure all our tests run through with MySQL before we merge into MariaDB. However, you are free to pull the latest PBXT tree and merge it into MariaDB any time you like. If you make any changes or fixes I would be glad to accept a patch for PBXT. Our goal is to keep one set of source code for all versions of MySQL supported. With PBXT both in MySQL, Drizzle, and MariaDB, the merge effort on your part could be non-trivial, so feel free to ask for help if needed. Yes, thanks. I will let you know if we need some help. Currently it is only Vlad and myself working on PBXT. Again, thanks a lot for your efforts on this! np :) Best regards, Paul - Kristian. -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
Paul McCullagh writes: > I have just proposed merge of the maria-pbxt tree into lp:maria. Thanks a lot for your effort! I branched your tree locally and tested it a little bit. > It resulted in me, and all other captains, getting a HUGE diff in an > e- > mail - SORRY! Mac Mail really struggled, Yikes! Don't worry. Anyway, I found a setting on Launchpad for limiting diff sizes in emails. I changed it to max 5000 lines (from unlimited). Not 100% sure it was the right place I changed though, Launchpad can be a bit confusing sometimes. > This branch includes the current MariaDB revision (2689) and the PBXT > storage engine. The only changes are the addition of a storage/pbxt > and a mysql-test/suite/pbxt directory. PBXT seems to be included in BUILD/compile-pentium64-debug-max, great! > The mysql-test/suite/pbxt directory contains a test suite for PBXT. > The test includes 167 test files. All tests run through without error. > To run the tests use: > > ./mysql-test-run --mysqld=--default-storage-engine=pbxt --suite=pbxt > > In other words, you have to set the default storage engine to PBXT for > this test to run. Testsuite did not pass without errors for me in the MariaDB tree. I saw these failures: pbxt.count_distinct2 (glibc detected double free error) pbxt.count_distinct3 (mysqld crash) pbxt.lowercase_table_grant pbxt.lowercase_table_qcache pbxt.lowercase_view pbxt.pbxt_bugs pbxt.udf The count* tests did not fail everytime I ran the tests. The *lowercase* test failures seems related to case differences. Test pbxt_bugs is missing file suite/pbxt/t/load_unique_error1.inc. The test pbxt.udf complains about missing udf_example.so (not sure why). Just let me know if you need more help/information on these failures. However, since the PBXT test suite is not run by default, I think we can merge it anyway now, and then work on getting these issues solved afterwards. I have set up Buildbot to automatically test the MariaDB code, and I need to find a build slave somewhere that could run tests of the PBXT code (feel free to offer to run one if you have a spare machine handy ;-). That would be helpful in getting a clean PBXT test run (and keeping it clean). > PBXT only creates files when the first PBXT table is created. > > So when you start MariaDB you will only know that PBXT is installed > because of the startup message it prints in the log. It does not > create any files in the database root directory. Ok. > Please let me know if this is OK for merge. This is ok to push as it is. Since you are a member of maria-captains on Launchpad, you can go ahead and push the merge to lp:maria yourself. (But if you prefer, I can also pull from your tree and push the merge). [The general policy for pushing to MariaDB is that you can push yourself once you get an 'ok to push' from review. Small fixes to a broken build etc. are sometimes pushed immediately to avoid delaying other work, and then reviewed after the fact.] Do you have a plan for merging in subsequent PBXT changes to MariaDB? (I guess you do, just don't know how it works). Would you be in charge of regular merging, or should the MariaDB team pull/merge from PBXT? With PBXT both in MySQL, Drizzle, and MariaDB, the merge effort on your part could be non-trivial, so feel free to ask for help if needed. > Best regards, > > Paul Again, thanks a lot for your efforts on this! - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
Re: [Maria-developers] MariaDB + PBXT
hi! > "Paul" == Paul McCullagh writes: Paul> Hi All, Paul> I am not sure of the procedure myself. Paul> I have just proposed merge of the maria-pbxt tree into lp:maria. Paul> It resulted in me, and all other captains, getting a HUGE diff in an e- Paul> mail - SORRY! Mac Mail really struggled, Yikes! ouch :( (but things like that happens and bandwidth is reasonable good nowadays) Paul> Anyway, here is the what the change includes: Paul> This branch includes the current MariaDB revision (2689) and the PBXT Paul> storage engine. The only changes are the addition of a storage/pbxt Paul> and a mysql-test/suite/pbxt directory. Paul> The mysql-test/suite/pbxt directory contains a test suite for PBXT. Paul> The test includes 167 test files. All tests run through without error. Paul> To run the tests use: Paul> ./mysql-test-run --mysqld=--default-storage-engine=pbxt --suite=pbxt Paul> In other words, you have to set the default storage engine to PBXT for Paul> this test to run. Paul> PBXT only creates files when the first PBXT table is created. Paul> So when you start MariaDB you will only know that PBXT is installed Paul> because of the startup message it prints in the log. It does not Paul> create any files in the database root directory. Sounds good! Paul> Please let me know if this is OK for merge. Paul> Best regards, Ok. Thanks! Regards, Monty ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
[Maria-developers] MariaDB + PBXT
Hi All, I am not sure of the procedure myself. I have just proposed merge of the maria-pbxt tree into lp:maria. It resulted in me, and all other captains, getting a HUGE diff in an e- mail - SORRY! Mac Mail really struggled, Yikes! Anyway, here is the what the change includes: This branch includes the current MariaDB revision (2689) and the PBXT storage engine. The only changes are the addition of a storage/pbxt and a mysql-test/suite/pbxt directory. The mysql-test/suite/pbxt directory contains a test suite for PBXT. The test includes 167 test files. All tests run through without error. To run the tests use: ./mysql-test-run --mysqld=--default-storage-engine=pbxt --suite=pbxt In other words, you have to set the default storage engine to PBXT for this test to run. PBXT only creates files when the first PBXT table is created. So when you start MariaDB you will only know that PBXT is installed because of the startup message it prints in the log. It does not create any files in the database root directory. Please let me know if this is OK for merge. Best regards, Paul -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com ___ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp
