[go-nuts] Re: MatrixOne: a hyperconverged and one-size-fits-most database in Go

[Nan DENG]

Hi Brian,

Thanks for your feedback. Indeed we are aware of the doc problem. Many 
technical design docs are still under edition. As for the AOE technical 
design, sorry for the wrong link in the doc website, you may refer to this good 
RFC link 

.
For the bold claim problem, as Yingfeng specified, MatrixOne is still at a 
very early stage, but our objectives of suiting most use cases were 
carefully thought over, we are also moving towards that objective. But I 
agree with you that before convincing proof and testing comes out, we'd 
improve the claim about what are final objectives and what are already 
implemented and proven. 
As we believe in open source, we'd like to share ideas and works as early 
as possible with community. We appreciate for your thoughtful and 
constructive feedback.

Thanks,

在2022年4月16日星期六 UTC+8 21:29:07 写道：

> On Friday, 15 April 2022 at 17:47:48 UTC+1 nicolas...@gmail.com wrote:
>
>> any feedback is welcome. 
>>
>
> The only feedback I can give is from your reading your documentation, 
> which appears at this point to be very vague.
>
> I was looking for a proper architectural white paper that says exactly 
> what CAP and transaction serializability tradeoffs you're making. The 
> nearest I could find was this one-page "tech design":
>
> https://docs.matrixorigin.io/0.3.0/MatrixOne/Overview/MatrixOne-Tech-Design/matrixone-techdesign/
> It's full of general statements like *"consistency protocols are proposed 
> to ensure the consistency of replicate logs. Common consistency protocols 
> include Paxos and Raft"* - rather than saying exactly what MatrixOne has 
> chosen to do.
>
> There is a link that says "Please refer to AOE Technical Design 
>  for 
> more details", but that link gives a 404 error.
>
> Searching for the word "isolation", it appears in the glossary only, and 
> nowhere else.
>
> In comparison, CockroachDB 
>  and 
> FoundationDB 
>  have a ton 
> of detailed documentation saying *exactly* how they handle things like 
> overlapping transactions, consistency, node failures, shard splitting etc; 
> and these change over time as they revisit the design decisions and 
> trade-offs they made.  I'm not saying you are positioning yourself against 
> those two particular databases. I just give them as examples where the 
> level of documentation is good enough to evaluate the database design for a 
> particular application.
>
> Otherwise, I think it's a dangerous claim to make that MatrixOne suits 
> almost all database use cases.  For example, many use cases value data 
> consistency and durability far above performance.  If you're saying 
> MatrixOne can match the consistency and durability of the best SQL 
> databases, *and* give higher performance, then that is a bold claim to make 
> without detailed justification.  What level of stress and regression 
> testing are you doing? Are you running any Jepsen-like scenarios? The happy 
> path is easy to get right - dealing with all possible failure modes is hard.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8e99c647-29e2-4bf0-8ac1-f73f77d1e29en%40googlegroups.com.

[go-nuts] Re: MatrixOne: a hyperconverged and one-size-fits-most database in Go

[Yingfeng Zhang]

Currently we are working on the update features, and upsert has not been 
considered initially and will be done in future.

On Saturday, April 16, 2022 at 4:42:41 PM UTC+8 yan.z...@gmail.com wrote:

> Does it support sequence and upsert in PostgreSQL?
>
> 在2022年4月16日星期六 UTC+8 00:47:48 写道：
>
>> Hi,
>>
>> I’d like to introduce a hyperconverged and one-size-fits-most database 
>> named MatrixOne, written in Go. 
>>
>> Github: https://github.com/matrixorigin/matrixone
>>
>> It’s a redesigned database with a combined architecture of CockroachDB, 
>> Clickhouse and Apache Flink. Our motivation is to bring a simple and 
>> one-size-fits-most data processing framework and significantly reduce the 
>> complexity of building modern data platform.
>>
>>
>> Key Objectives:
>>
>>- Simple usage by standardized SQL with support of multiple dialects. 
>>- Supports transactional, analytical, streaming workloads in one 
>>database.  
>>- Blazing fast query performance with strong consistency and ACID 
>>transactional property. 
>>- Large scalability across different infrastructures, including 
>>public/private cloud, on-premise and edge devices.  
>>
>>
>> It’s still a very early stage project, the latest 0.3.0 release version 
>> is mainly a MPP OLAP engine, with distributed strong consistency based on 
>> Raft.
>>
>> A SSB benchmark test is run over MatrixOne and it outperforms Clickhouse 
>> at 50%, with some constraints on. 
>>
>>
>> Feel free to test MatrixOne and any feedback is welcome. 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d35119a3-2a13-41f3-bc40-12a2a8636ec6n%40googlegroups.com.

[go-nuts] Re: MatrixOne: a hyperconverged and one-size-fits-most database in Go

[Yingfeng Zhang]

Hi, Brian,
Thanks for your comments. 
Actually the MatrixOne is still in its infant stage, so it's far from being 
able to be compared with other alternatives in terms of ACID,...,etc. 
Current codebase is just a demo such that MPP sql execution engine could 
work together with both columnar storage as well as multi-raft 
architecture, while the columnar storage engine adopts a name with AOE 
which means Append Only Engine, that's why we call it a demo stage. 
Currently we are working on transactional support to deliver a TAE 
engine(Transactional Analytical Engine),  the initial version would be 
visible during June this year which is still a single node version, and 
distributed TAE would be initially ready during Q3 of this year. If you are 
interested in the draft design of TAE, you may take a look here 
 .
After these works have been ready, could we start with such works as 
Jepsen-like testing. 

Thank you ~

On Saturday, April 16, 2022 at 9:29:07 PM UTC+8 Brian Candler wrote:

> On Friday, 15 April 2022 at 17:47:48 UTC+1 nicolas...@gmail.com wrote:
>
>> any feedback is welcome. 
>>
>
> The only feedback I can give is from your reading your documentation, 
> which appears at this point to be very vague.
>
> I was looking for a proper architectural white paper that says exactly 
> what CAP and transaction serializability tradeoffs you're making. The 
> nearest I could find was this one-page "tech design":
>
> https://docs.matrixorigin.io/0.3.0/MatrixOne/Overview/MatrixOne-Tech-Design/matrixone-techdesign/
> It's full of general statements like *"consistency protocols are proposed 
> to ensure the consistency of replicate logs. Common consistency protocols 
> include Paxos and Raft"* - rather than saying exactly what MatrixOne has 
> chosen to do.
>
> There is a link that says "Please refer to AOE Technical Design 
>  for 
> more details", but that link gives a 404 error.
>
> Searching for the word "isolation", it appears in the glossary only, and 
> nowhere else.
>
> In comparison, CockroachDB 
>  and 
> FoundationDB 
>  have a ton 
> of detailed documentation saying *exactly* how they handle things like 
> overlapping transactions, consistency, node failures, shard splitting etc; 
> and these change over time as they revisit the design decisions and 
> trade-offs they made.  I'm not saying you are positioning yourself against 
> those two particular databases. I just give them as examples where the 
> level of documentation is good enough to evaluate the database design for a 
> particular application.
>
> Otherwise, I think it's a dangerous claim to make that MatrixOne suits 
> almost all database use cases.  For example, many use cases value data 
> consistency and durability far above performance.  If you're saying 
> MatrixOne can match the consistency and durability of the best SQL 
> databases, *and* give higher performance, then that is a bold claim to make 
> without detailed justification.  What level of stress and regression 
> testing are you doing? Are you running any Jepsen-like scenarios? The happy 
> path is easy to get right - dealing with all possible failure modes is hard.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/88524fa8-2142-4239-af00-97383636111fn%40googlegroups.com.

[go-nuts] Re: MatrixOne: a hyperconverged and one-size-fits-most database in Go

[Brian Candler]

On Friday, 15 April 2022 at 17:47:48 UTC+1 nicolas...@gmail.com wrote:

> any feedback is welcome. 
>

The only feedback I can give is from your reading your documentation, which 
appears at this point to be very vague.

I was looking for a proper architectural white paper that says exactly what 
CAP and transaction serializability tradeoffs you're making. The nearest I 
could find was this one-page "tech design":
https://docs.matrixorigin.io/0.3.0/MatrixOne/Overview/MatrixOne-Tech-Design/matrixone-techdesign/
It's full of general statements like *"consistency protocols are proposed 
to ensure the consistency of replicate logs. Common consistency protocols 
include Paxos and Raft"* - rather than saying exactly what MatrixOne has 
chosen to do.

There is a link that says "Please refer to AOE Technical Design 
 for 
more details", but that link gives a 404 error.

Searching for the word "isolation", it appears in the glossary only, and 
nowhere else.

In comparison, CockroachDB 
 and 
FoundationDB  
have a ton of detailed documentation saying *exactly* how they handle 
things like overlapping transactions, consistency, node failures, shard 
splitting etc; and these change over time as they revisit the design 
decisions and trade-offs they made.  I'm not saying you are positioning 
yourself against those two particular databases. I just give them as 
examples where the level of documentation is good enough to evaluate the 
database design for a particular application.

Otherwise, I think it's a dangerous claim to make that MatrixOne suits 
almost all database use cases.  For example, many use cases value data 
consistency and durability far above performance.  If you're saying 
MatrixOne can match the consistency and durability of the best SQL 
databases, *and* give higher performance, then that is a bold claim to make 
without detailed justification.  What level of stress and regression 
testing are you doing? Are you running any Jepsen-like scenarios? The happy 
path is easy to get right - dealing with all possible failure modes is hard.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c0972709-f2c9-4f08-bafc-35ff5cc4c2aan%40googlegroups.com.

[go-nuts] Re: MatrixOne: a hyperconverged and one-size-fits-most database in Go

[Zhaoxun Yan]

Does it support sequence and upsert in PostgreSQL?

在2022年4月16日星期六 UTC+8 00:47:48 写道：

> Hi,
>
> I’d like to introduce a hyperconverged and one-size-fits-most database 
> named MatrixOne, written in Go. 
>
> Github: https://github.com/matrixorigin/matrixone
>
> It’s a redesigned database with a combined architecture of CockroachDB, 
> Clickhouse and Apache Flink. Our motivation is to bring a simple and 
> one-size-fits-most data processing framework and significantly reduce the 
> complexity of building modern data platform.
>
>
> Key Objectives:
>
>- Simple usage by standardized SQL with support of multiple dialects. 
>- Supports transactional, analytical, streaming workloads in one 
>database.  
>- Blazing fast query performance with strong consistency and ACID 
>transactional property. 
>- Large scalability across different infrastructures, including 
>public/private cloud, on-premise and edge devices.  
>
>
> It’s still a very early stage project, the latest 0.3.0 release version is 
> mainly a MPP OLAP engine, with distributed strong consistency based on Raft.
>
> A SSB benchmark test is run over MatrixOne and it outperforms Clickhouse 
> at 50%, with some constraints on. 
>
>
> Feel free to test MatrixOne and any feedback is welcome. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fd2cbd01-26b6-4983-bc96-4986d0bf6b31n%40googlegroups.com.