Re: Cosmos DB Graph queries broken

[Greg Keogh]

>
> I don't have any help for your question, but I was wondering if you could
> send a link to the music collection database video please? Sounds
> interesting.
>

This is the bloke: https://www.youtube.com/watch?v=vJcxRjJ982k At about1:45
he talks about a movie database. It struck me personally because I had my
music/video/book collection in RDBs for 26 years in a beautifully
normalised way, but it was so heavyweight and tiresome. Now I realise its
natural format is a graph DB. I'm using my own data as a serious migration
exercise to prepare for possibly using Graph in an upcoming live product.

This video is good too , but a
bit slow. At 12:20 he makes a good point about "answering questions you
didn't expect" about relational data.

I'm just so over RDBs now!

*GK*

>

Re: Cosmos DB Graph queries broken

[David Burstin]

Hi Greg,

I don't have any help for your question, but I was wondering if you could
send a link to the music collection database video please? Sounds
interesting.

Regards
David

On Wed., 19 Sep. 2018, 09:58 Greg Keogh,  wrote:

> Folks, is anyone using the Cosmos DB Graph API? I just resurrected a demo
> I had working a few months ago, updated to the recent packages and ran it,
> every query run now produces:
>
> System.MissingMethodException : Method not found: 'Double
> Microsoft.Azure.Documents.Helpers.GetHeaderValueDouble(System.Collections.Specialized.NameValueCollection,
> System.String, Double)'.
>
> I've now spent over 3 solid hours trying everything I can think of to get
> it working again, but without hope. I've twiddled config redirects, changed
> dozens of package versions in a wild variety of ways (including back to the
> ones from months ago) and tried all sorts of Gremlin queries, but
> absolutely nothing changes the error. Web searches produce zero matches.
>
> I thought I'd ask here just in case someone has seen this.
>
> I should point out though, that I'm really impressed by graph databases in
> general and see them as a fabulous alternative to RDBs for many realistic
> scenarios. I watched a good video yesterday by a chap who used a music
> collection database as an example where he tried to normalize it and
> finished up with dozens of tables and queries crammed with joins, whereas
> in a graph DB it was simple and natural. He points out quite rightly that
> if your find yourself making RDB queries with joins everywhere then you may
> be using the wrong database!
>
> Yesterday I watched about 20 Microsoft videos from conferences and tech
> announcements (some from Conf 2018), and now my brain is imprinted with
> images of smiling evangelists and product managers in blue shirts telling
> us how happy we are and how wonderful the future of software development
> will be. I must be living a different dream, because on the ground here all
> I get is compile errors, deploy fails, incomprehensible crashes, files not
> found, missing resources, package conflicts, authentication failures,
> assembly load errors, beta releases, breaking upgrades, software that won't
> install or uninstall, binding errors, Blazor, JavaScript, HTML, Xamarin and
> the bathroom drain keeps clogging up.
>
> *Greg K*
>

RE: Cosmos DB

[Paul Glavich]

>> * Was your client on your desktop or up in the cloud in the same region as 
>> the database?

All in azure same region.

 

>> * Are you using a different Guid as the partition key for each document? 
>> You're supposed to be more "coarse" and group large groups of documents into 
>> partitions to improve large query performance. I'm not sure how this would 
>> affect your bulk insert tests.

Yes, diff Guid. We spoke with the CosmosDb team and they were all for Guids as 
the partition key for a lot of cases. We spent a lot of time talking with 
Microsoft on this within the bounds of the project. Guid’s are really good for 
distribution and make partitioning really effective. Yes there is a grouping 
effect but there is a massive risk of grouping too much which causes excessive 
RU usage on a single partition. With 20,000 RU’s for a collection, if you have 
10 partitions, the maximum you can get per partition is 2,000 RU’s so if you 
group a lot of documents together on a single partition, you get really small 
throughput because you are limited to your partition. As a general rule of 
thumb, the broader the range, the better for overall throughput. Enabling cross 
partition queries satisfies most requests in a well performing way.

 

In this instance, I didn’t want to limit by partition in anyway and simply 
wanted impose the same distribution offsets for every test so that it was at 
least consistent.

 

*   Glav

 

 

From: ozdotnet-boun...@ozdotnet.com [mailto:ozdotnet-boun...@ozdotnet.com] On 
Behalf Of Greg Keogh
Sent: Wednesday, 7 February 2018 12:25 PM
To: ozDotNet 
Subject: Re: Cosmos DB

 

Only just saw this. I have worked with a designed a solution using CosmosDb 
which is currently in production. I actually just released a blog post on 
client performance related to CosmosDb here: 
https://weblogs.asp.net/pglavich/cosmosdb-and-client-performance

 

I just read it all, well done. It does concentrate on bulk insert speed, which 
may not be a concern for many products. It's interesting to see which "walls" 
you hit along the way. Some notes:

 

* Was your client on your desktop or up in the cloud in the same region as the 
database?

 

* Are you using a different Guid as the partition key for each document? You're 
supposed to be more "coarse" and group large groups of documents into 
partitions to improve large query performance. I'm not sure how this would 
affect your bulk insert tests.

 

* (nitpicking) When discussing performance, don't say "significant impact" or 
"greater impact", use unambiguous expressions like "improve" or "degrade".

 

 As to your question, I haven’t used it personally, but I believe the Table API 
over CosmosDb supports bulk operations ( 
https://docs.microsoft.com/en-us/azure/cosmos-db/table-support ) as it is the 
same as the general Windows Azure storage API which supports bulk.

 

I'm not using Tables as the underlying storage, so I can't use the batch 
operations. For Cosmos DB SQL API the only mention of batch or transaction 
operations is related to JavaScript procs, but I haven't researched this yet. 
If you have to register server-side JavaScript for this purpose, then I'm quite 
irritated, not just because of my JS bias, but because it's a weird language 
mix.

 

I also have a (long) blog article on how moving an app suite from SQL Server to 
Cosmos DB produced miraculous results 

 

https://gfkeogh.blogspot.com.au/2018/01/collections-database-history.html

 

Greg

 

Re: Cosmos DB

[Greg Keogh]

>
> Only just saw this. I have worked with a designed a solution using
> CosmosDb which is currently in production. I actually just released a blog
> post on client performance related to CosmosDb here:
> https://weblogs.asp.net/pglavich/cosmosdb-and-client-performance
>

I just read it all, well done. It does concentrate on bulk insert speed,
which may not be a concern for many products. It's interesting to see which
"walls" you hit along the way. Some notes:

* Was your client on your desktop or up in the cloud in the same region as
the database?

* Are you using a different Guid as the partition key for each document?
You're supposed to be more "coarse" and group large groups of documents
into partitions to improve large query performance. I'm not sure how this
would affect your bulk insert tests.

* (nitpicking) When discussing performance, don't say "significant impact"
or "greater impact", use unambiguous expressions like "improve" or
"degrade".


>  As to your question, I haven’t used it personally, but I believe the
> Table API over CosmosDb supports bulk operations (
> https://docs.microsoft.com/en-us/azure/cosmos-db/table-support ) as it is
> the same as the general Windows Azure storage API which supports bulk.
>

I'm not using Tables as the underlying storage, so I can't use the batch
operations. For Cosmos DB SQL API the only mention of batch or transaction
operations is related to JavaScript procs, but I haven't researched this
yet. If you have to register server-side JavaScript for this purpose, then
I'm quite irritated, not just because of my JS bias, but because it's a
weird language mix.

I also have a (long) blog article on how moving an app suite from SQL
Server to Cosmos DB produced miraculous results

https://gfkeogh.blogspot.com.au/2018/01/collections-database-history.html

*Greg*

RE: Cosmos DB

[Paul Glavich]

Sorry for the delay,

 

Only just saw this. I have worked with a designed a solution using CosmosDb 
which is currently in production. I actually just released a blog post on 
client performance related to CosmosDb here: 
https://weblogs.asp.net/pglavich/cosmosdb-and-client-performance

 

As to your question, I haven’t used it personally, but I believe the Table API 
over CosmosDb supports bulk operations ( 
https://docs.microsoft.com/en-us/azure/cosmos-db/table-support ) as it is the 
same as the general Windows Azure storage API which supports bulk.

 

*   Glav

 

 

 

From: ozdotnet-boun...@ozdotnet.com [mailto:ozdotnet-boun...@ozdotnet.com] On 
Behalf Of Greg Keogh
Sent: Friday, 12 January 2018 12:06 PM
To: ozDotNet 
Subject: Cosmos DB

 

A quick Friday query ... Is anyone here using Cosmos DB in anger? I only ask 
because I just returned to experiment with it for the first time since it 
changed name to Cosmos. I actually quite like it! The .NET SDK wraps reasonably 
sensible classes over the API and I had a demo program up and running quickly.

 

I notice there are no "batch" operations for bulk inserts like we have in Table 
Storage, and I'm not writing JavaScript procs. It was unclear how to mix 
different document types in the same collection and process them separately 
(using a bool flag property in each type is to the way to go). There is little 
guidance about how to use the unique string Id. I haven't figured out how to 
add attachment links yet. But despite some oddities about the way it works, I'm 
really impressed with the performance, the magical indexing and ease-of-use.

 

I'm thinking of moving my "music database" out of SQL Server into Cosmos and 
writing a fresh UI. This is my hobby project that has been going since 1992 and 
has never been completed because a new technology or fad comes along annually 
and forces an Xmas holiday rewrite. I realise now that cataloguing items like 
music, videos and books has always produced rather strict and complicated 
normalised SQL database tables. These sorts of items feel much more natural 
when converted to de-normalised documents.

 

Greg K