Re: [Firebird-devel] [SPAM] Re: Start transaction from base transaction
Awesome! :) On 2017.04.21. 1:30, Adriano dos Santos Fernandes wrote: > Em 20/04/2017 17:53, Vlad Khorsun escreveu: >> Also it is necessary >> to teach engine to use that metadata (instead of current one) within >> attachment >> working "in the past". >> > I'm doing a prototype implementation of this for active transactions, > i.e., the things I mentioned in this thread start. > > I've it starting working, but implementation is very simple, weak and > almost non-tested at the moment. > > > Adriano > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] [SPAM] Re: Start transaction from base transaction
Em 20/04/2017 17:53, Vlad Khorsun escreveu: > Also it is necessary > to teach engine to use that metadata (instead of current one) within > attachment > working "in the past". > I'm doing a prototype implementation of this for active transactions, i.e., the things I mentioned in this thread start. I've it starting working, but implementation is very simple, weak and almost non-tested at the moment. Adriano -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [SPAM] Re: Start transaction from base transaction
20.04.2017 10:12, Molnár Attila wrote: > +1 for this feature. I would be very happy for this. Also it would be > awesmone if this consistent view were accessible later in time (this > woudl mean garbage collection blocking). Blocking of GC is the easiest part of this task. One need also to preserve metadata that was valid at interesting moment of time. Also it is necessary to teach engine to use that metadata (instead of current one) within attachment working "in the past". To do it it is necessary to make "snapshot" of that metadata and save it with an interesting moment. And this is still just tip of the iceberg, i suspect... Regards, Vlad -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parallel execution
20.04.2017 17:50, Dimitry Sibiryakov wrote: > 20.04.2017 16:20, Vlad Khorsun wrote: >>> even in the simplest cases like "select f from t1 union select f from t2"? >> This case nor simplest nor better for parallel execution than "ordinary" >> "select * from t". > > Union used to produce plan SORT (T1 NATURAL), SORT(T2 NATURAL). Both > record streams can > be fetched and sorted in parallel and then merged, right? And because of > selects from > different tables, they won't race on data pages. No. This is very naive view on parallel query execution. Real implementations works differently. There is a lot of info in the net about it. Vlad -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parallel execution
20.04.2017 16:20, Vlad Khorsun wrote: >> even in the simplest cases like "select f from t1 union select f from t2"? >This case nor simplest nor better for parallel execution than "ordinary" > "select * from t". Union used to produce plan SORT (T1 NATURAL), SORT(T2 NATURAL). Both record streams can be fetched and sorted in parallel and then merged, right? And because of selects from different tables, they won't race on data pages. -- WBR, SD. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parallel execution
20.04.2017 15:49, Leyne, Sean wrote: > "select f from t1 union select f from t2" is ONE statement with 2 parts! > > "Attachment" refers to the connection between a client and the server, it > does not imply anything about how the engine might decompose a statement or > how the parts could/would be executed! It doesn't matter. If both parts are executed simultaneously, from technical POV they have no difference from independent queries executed simultaneously in one attachment and transaction. -- WBR, SD. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Parallel execution
20.04.2017 13:04, Dimitry Sibiryakov wrote: > 19.04.2017 14:12, Vlad Khorsun wrote: >> Engine can't run more than one statement in attachment at same time. >> This will not be >> changed. > > Does it also mean that Firebird will never be able to execute parts of > single query in > parallel No, it doesn't > even in the simplest cases like "select f from t1 union select f from t2"? This case nor simplest nor better for parallel execution than "ordinary" "select * from t". Vlad -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Start transaction from base transaction
> Being able to start next transaction, > basically, from the point where the previous left would be so convenient and > remove a lot of hacks we have in place right now. To be clear, in my example/use case the "master transaction" would still be active and the related connection would be require to be maintained. Vlad (or was it Alex) suggested that a feature allowing for "snapshot points" to be maintained could be possible, but this would be a separate issue (which could use some of the same code/basics). Sean -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] Parallel execution (was: Start transaction from base transaction)
19.04.2017 14:12, Vlad Khorsun wrote: >Engine can't run more than one statement in attachment at same time. This > will not be > changed. Does it also mean that Firebird will never be able to execute parts of single query in parallel even in the simplest cases like "select f from t1 union select f from t2"? -- WBR, SD. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] [SPAM] Re: Start transaction from base transaction
>This statement i not understand. Do you speak about transaction ? Or >about Transaction. > statement\cursor ? If later, why do you can't remember id of last fetched > row > and start next fetch from this position ? That's what we do now, in a nutshell. The problem is of course to have these IDs and related data (to prevent FK violations). I can explain it to you in better detail, probably rather outside the list. -- Mgr. Jiří Činčura https://www.tabsoverspaces.com/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Start transaction from base transaction
>Did you consider export of the data to local files and then sending of >these files? We did. And discarded it because of the hassle of managing these files. > File transfer over Internet is much faster. I'm not uploading over FIrebird's protocol. It's pushed over custom HTTPs endpoint. -- Mgr. Jiří Činčura https://www.tabsoverspaces.com/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
[Firebird-devel] [SPAM] Re: Start transaction from base transaction
20.04.2017 9:36, Jiří Činčura wrote: >> Could you explain, please ? > > Sure. There's a part of our system that reads data and sends these over > the internet. Given the deployments on the customer side the internet is > often very very slow (<100kBps sometimes), so we decided to not keep > transaction open, but do it repeatedly in bacches (there are other > requirements, I don't want to go into too much details). Being able to > start next transaction, basically, from the point where the previous > left would be so convenient and remove a lot of hacks we have in place > right now. "start next transaction, basically, from the point where the previous left" This statement i not understand. Do you speak about transaction ? Or about statement\cursor ? If later, why do you can't remember id of last fetched row and start next fetch from this position ? Regards, Vlad -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Start transaction from base transaction
20.04.2017 8:36, Jiří Činčura wrote: > Sure. There's a part of our system that reads data and sends these over > the internet. Given the deployments on the customer side the internet is > often very very slow (<100kBps sometimes), so we decided to not keep > transaction open, but do it repeatedly in bacches (there are other > requirements, I don't want to go into too much details). Being able to > start next transaction, basically, from the point where the previous > left would be so convenient and remove a lot of hacks we have in place > right now. Did you consider export of the data to local files and then sending of these files? File transfer over Internet is much faster. -- WBR, SD. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Start transaction from base transaction
19.04.2017 23:27, Leyne, Sean wrote: > I never said anything about a single application! > > I mentioned in one of my posts that the Master connection would request a > "Transaction Clone" which would be a GUID that could be sent to another > *process* to allow that process' connection to link to the "Master > Transaction". Indeed, the one who talked about multi-threaded application was Adriano. -- WBR, SD. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Start transaction from base transaction
+1 for this feature. I would be very happy for this. Also it would be awesmone if this consistent view were accessible later in time (this woudl mean garbage collection blocking). On 2017.04.18. 20:56, Leyne, Sean wrote: > >> If you need simultaneous queries - make them possible, >> what the point of transaction hacking? > You want a single "view" of the database from multiple _connections_. > > There is nothing that provides this, today -- There is no way to ensure that > all connections have that same view. > > > Sean > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > Firebird-Devel mailing list, web interface at > https://lists.sourceforge.net/lists/listinfo/firebird-devel -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
Re: [Firebird-devel] Start transaction from base transaction
>Could you explain, please ? Sure. There's a part of our system that reads data and sends these over the internet. Given the deployments on the customer side the internet is often very very slow (<100kBps sometimes), so we decided to not keep transaction open, but do it repeatedly in bacches (there are other requirements, I don't want to go into too much details). Being able to start next transaction, basically, from the point where the previous left would be so convenient and remove a lot of hacks we have in place right now. -- Mgr. Jiří Činčura https://www.tabsoverspaces.com/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
