Re: [SR-Users] mtree on db Berkeley DB?

2011-11-17 Thread Javier Gallart
Hi Daniel

On Thu, Nov 17, 2011 at 8:03 AM, Daniel-Constantin Mierla mico...@gmail.com
 wrote:

 Hello,

 On 11/16/11 12:45 PM, Javier Gallart wrote:

 [...]

 All the data in Redis stays in memory, you can dump periodic snapshots to
 disk. Some time ago they worked in a feature called disktore, with some
 cache mechanism between disk and memory, but as far as I know that line was
 abandoned.


 so if the redis goes down, then the records are lost or is doing dump of
 memory to disk at that time? I try to figure out if redis makes more sense
 than using one kamailio as a load balancer in front of two kamailios having
 the mtree in memory. The two with mtree will be restarted one after the
 other every 15 min. Load balancer can be configured to use only one, with
 quick fallback (100ms) to the second, or via kamctl the active kamailio
 with mtree is set to be used first.

 That's pretty much our current configuration; instead of having  a
kamailio as load-balancer we are using our own softswitch that sends the
calls to 2 kamailio with mtree in memory. If there is a problem with one of
them, the call is quickly rerouted to the backup kamailio. We're happy with
that configuration; but as the tree keeps growing the time needed for
populating the db + reloading the tree increases (about 4-5 minutes
currently, not really under pressure yet...). That's why we were thinking
about a full in-memory solution. At this point we're going to focus our
tests with this setup:
-1 redis Master server for writing, with frequent snapshots
-2 redis Slave servers
-2 kamailios, each of them reading from a slave redis stored in a shared
variable for quick update via rpc
-If redis_cmd returns an error, kamailio will respond with a 503 Status so
that the softswitch will send the invite to the backup kamailio

Of course this will make sense only if we're able to implement a mt_match
equivalent in redis that gives an acceptable performance...

Regards

Javi

 kamailio with mtree will send back a 302 which is handled by load balancer
 Cheers,
 Daniel


 --
 Daniel-Constantin Mierla -- http://www.asipto.com
 Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
 http://linkedin.com/in/miconda -- http://twitter.com/miconda


___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-16 Thread Javier Gallart
Hi

On Wed, Nov 16, 2011 at 8:13 AM, Daniel-Constantin Mierla mico...@gmail.com
 wrote:

  Hello,


 On 11/15/11 10:09 PM, Javier Gallart wrote:

  Hello

 coming back to the topic related to mtree, to be able to set values via
 mi/rpc -- it won't be that difficult to add such functionality. Usually
 with a tree is mainly reading, due to fast matching on tree indexing.
 The question is how many times/often do you need to change values and
 how many of them at the same time (more or less).

  I was also thinking about that: our application works in such a way
 that the full tree is recalculated every 15 minutes. Currently we have
 aound 60 branches with 15 entries each.

 I assume many times the changes will be somewhere down the tree, since
 the first part of the number is usually the same (e.g., country code and
 operator prefix). To update the tree at runtime, while there are reads
 on it, there must be used a lock to be safe an consistent. If you do lot
 of writes and very often, then you keep the tree structure locked a lot,
 slowing the search.


  That's the case, tname is rarely updated, it's tvalue the column that we
 normally update.


 I guess you meant actually tprefix is not updated -- tname is table name,
 tprefix is the prefix used to build the tree. What would be the percentage
 of tvalue updates, do you have add/remove of tprefix-es?

You're right, sorry for the wrong names. The tprefixes are only updated
once a day, and only a tiny percentage of them (less than 1%) actually
changes.




 Can you estimate the number of writes and how often they would be on a
 daily basis? There might be other solutions to look at, if writes are
 very often.


  From the numbers above, let's say that we need to update around 8M
 records every 15 minutes (we expect this number will keep steadily
 increasing...)


 But not all of them change the value, right?

I've been checking this, and  on average about 40% are updated in each
reload.




  As a side note, I've looked at how to implement the mt_match equivalent
 in redis and it does't look that hard using kamailio s.prefixed
 transformation (as you suggested) and redis sorted sets. I'll need to make
 more tests to check the performance, I'll share the results.


 I have used this solution with mysql (using a table structure in memory of
 mysql server), since the changes could have been done in mysql in normal
 way, in one update query, which is a sync operation. Does redis do any
 caching or is using always the disk file db?

All the data in Redis stays in memory, you can dump periodic snapshots to
disk. Some time ago they worked in a feature called disktore, with some
cache mechanism between disk and memory, but as far as I know that line was
abandoned.


 Cheers,
 Daniel


  Thanks!

  Javi


 Cheers,
 Daniel
 
  Regards
 
  Javi



 ___
 SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing 
 listsr-us...@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


 --
 Daniel-Constantin Mierla -- http://www.asipto.com
 Kamailio Advanced Training, Dec 5-8, Berlin: 
 http://asipto.com/u/kathttp://linkedin.com/in/miconda -- 
 http://twitter.com/miconda


___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-15 Thread Daniel-Constantin Mierla

Hello,

On 11/14/11 3:24 PM, Javier Gallart wrote:

Hello

very interesting issue actually...the mtree module fits perfectly well 
in a key-value model becaue basically is what the mtree table 
structure defines; that's why redis was the first thing that came to 
my mind when I saw the redis module. Two problems with redis:

-no native mt_match function, up to the user to find the best option
-replication. Until the cluster feature is ready, we need to change by 
hand the server ip address, which implies a kamailio restart. There is 
no mi command for changing the server in the fly, right..(not in the 
module documentation at least)?
you cannot change the redis server attributes on the fly, but you can 
define many redis servers and based on the name attribute query a 
specific one. So if you define two, you can do round robin queries to 
both of them, by using a shared variable $shv(...) to know which one was 
used last time. In the same way, since $sht(...) can be changed via MI, 
you can query either first redis or second one, based on the value of 
$sht(). In this way you can build some failover solution just in config 
file of kamailio.


Cheers,
Daniel



Daniel, I agree that your suggestion about the mi/rpc method would be 
nice. I will also take a look at Mongo as Douglas suggests, and 
especially CouchDB, because you can talk to Couch DB via http...


Regards

Javi

On Mon, Nov 14, 2011 at 1:32 PM, Douglas Hubler doug...@hubler.us 
mailto:doug...@hubler.us wrote:


On Mon, Nov 14, 2011 at 5:10 AM, Daniel-Constantin Mierla
mico...@gmail.com mailto:mico...@gmail.com wrote:
 are there any other no-sql database systems that have such
mechanism? Might
 not be hard to make a connector when the time will allow -- just
to know the
 best options here.

mongodb will auto promote.  Caveat, (like redis if i understand
correctly), is that all writes are directed to a single master (be it
chosen dynamically), but reads can happen anywhere to spread the load.
 Also, you need to accept the distaster scenario of a network
partition  where a minority set of servers find themselves w/o a
master.  Example: 5 servers in datacenter #1 and 4 servers in
datacenter #2.  If the link between datacenters is broken, then all
servers in datacenter #2 will not have a master and will be read-only
until link is restored.  Good part about single master is there's no
chance of inconsistent data.

Turns out local fail-over v.s. consistent data is a well explored
area.

http://blog.nahurst.com/visual-guide-to-nosql-systems

I've worked w/the C++ driver to mongodb is anyone has questions.




___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-15 Thread Daniel-Constantin Mierla

Hello,

On 11/14/11 3:24 PM, Javier Gallart wrote:

Hello

very interesting issue actually...the mtree module fits perfectly well 
in a key-value model becaue basically is what the mtree table 
structure defines; that's why redis was the first thing that came to 
my mind when I saw the redis module. Two problems with redis:

-no native mt_match function, up to the user to find the best option
-replication. Until the cluster feature is ready, we need to change by 
hand the server ip address, which implies a kamailio restart. There is 
no mi command for changing the server in the fly, right..(not in the 
module documentation at least)?


Daniel, I agree that your suggestion about the mi/rpc method would be 
nice. I will also take a look at Mongo as Douglas suggests, and 
especially CouchDB, because you can talk to Couch DB via http...


coming back to the topic related to mtree, to be able to set values via 
mi/rpc -- it won't be that difficult to add such functionality. Usually 
with a tree is mainly reading, due to fast matching on tree indexing. 
The question is how many times/often do you need to change values and 
how many of them at the same time (more or less).


I assume many times the changes will be somewhere down the tree, since 
the first part of the number is usually the same (e.g., country code and 
operator prefix). To update the tree at runtime, while there are reads 
on it, there must be used a lock to be safe an consistent. If you do lot 
of writes and very often, then you keep the tree structure locked a lot, 
slowing the search.


Can you estimate the number of writes and how often they would be on a 
daily basis? There might be other solutions to look at, if writes are 
very often.


Cheers,
Daniel


Regards

Javi

On Mon, Nov 14, 2011 at 1:32 PM, Douglas Hubler doug...@hubler.us 
mailto:doug...@hubler.us wrote:


On Mon, Nov 14, 2011 at 5:10 AM, Daniel-Constantin Mierla
mico...@gmail.com mailto:mico...@gmail.com wrote:
 are there any other no-sql database systems that have such
mechanism? Might
 not be hard to make a connector when the time will allow -- just
to know the
 best options here.

mongodb will auto promote.  Caveat, (like redis if i understand
correctly), is that all writes are directed to a single master (be it
chosen dynamically), but reads can happen anywhere to spread the load.
 Also, you need to accept the distaster scenario of a network
partition  where a minority set of servers find themselves w/o a
master.  Example: 5 servers in datacenter #1 and 4 servers in
datacenter #2.  If the link between datacenters is broken, then all
servers in datacenter #2 will not have a master and will be read-only
until link is restored.  Good part about single master is there's no
chance of inconsistent data.

Turns out local fail-over v.s. consistent data is a well explored
area.

http://blog.nahurst.com/visual-guide-to-nosql-systems

I've worked w/the C++ driver to mongodb is anyone has questions.




___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-15 Thread Juha Heinanen
Daniel-Constantin Mierla writes:

 I assume many times the changes will be somewhere down the tree, since 
 the first part of the number is usually the same (e.g., country code and 
 operator prefix). To update the tree at runtime, while there are reads 
 on it, there must be used a lock to be safe an consistent. If you do lot 
 of writes and very often, then you keep the tree structure locked a lot, 
 slowing the search.

daniel,

could the lock be from the node that requires modification downwards,
which would leave all other branches unlocked?

 Can you estimate the number of writes and how often they would be on a 
 daily basis? There might be other solutions to look at, if writes are 
 very often.

i was thinking that perhaps mtree would be used to store numbers that
are ported to other operators instead of the current carrier route pdb
server, where the whole mmap file needs to be reloaded.  in that
application i would imagine a few hundred updates per hour, but many of
them would only affect the value associated with the node, not the
structure of the three.

-- juha

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-15 Thread Javier Gallart

 Hello

 coming back to the topic related to mtree, to be able to set values via
 mi/rpc -- it won't be that difficult to add such functionality. Usually
 with a tree is mainly reading, due to fast matching on tree indexing.
 The question is how many times/often do you need to change values and
 how many of them at the same time (more or less).

 I was also thinking about that: our application works in such a way that
the full tree is recalculated every 15 minutes. Currently we have aound 60
branches with 15 entries each.

 I assume many times the changes will be somewhere down the tree, since
 the first part of the number is usually the same (e.g., country code and
 operator prefix). To update the tree at runtime, while there are reads
 on it, there must be used a lock to be safe an consistent. If you do lot
 of writes and very often, then you keep the tree structure locked a lot,
 slowing the search.


That's the case, tname is rarely updated, it's tvalue the column that we
normally update.


 Can you estimate the number of writes and how often they would be on a
 daily basis? There might be other solutions to look at, if writes are
 very often.


From the numbers above, let's say that we need to update around 8M records
every 15 minutes (we expect this number will keep steadily increasing...)

As a side note, I've looked at how to implement the mt_match equivalent in
redis and it does't look that hard using kamailio s.prefixed transformation
(as you suggested) and redis sorted sets. I'll need to make more tests to
check the performance, I'll share the results.

Thanks!

Javi


 Cheers,
 Daniel
 
  Regards
 
  Javi

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-15 Thread Daniel-Constantin Mierla

Hello,

On 11/15/11 12:59 PM, Juha Heinanen wrote:

Daniel-Constantin Mierla writes:


I assume many times the changes will be somewhere down the tree, since
the first part of the number is usually the same (e.g., country code and
operator prefix). To update the tree at runtime, while there are reads
on it, there must be used a lock to be safe an consistent. If you do lot
of writes and very often, then you keep the tree structure locked a lot,
slowing the search.

daniel,

could the lock be from the node that requires modification downwards,
which would leave all other branches unlocked?


for large trees it would be not possible to have a lock for each node, 
there will be too many and different operating systems have limits. On a 
hash table there is a lock per slot, no matter how many items are in the 
table. In a tree is hard to define a set of locks and just going with 
one per node is not that feasible. For eaxample one record with prefix 
0123456789 creates 10 nodes, then some nodes are reused, but still there 
are a greater or equal number of nodes than the number of records loaded.





Can you estimate the number of writes and how often they would be on a
daily basis? There might be other solutions to look at, if writes are
very often.

i was thinking that perhaps mtree would be used to store numbers that
are ported to other operators instead of the current carrier route pdb
server, where the whole mmap file needs to be reloaded.  in that
application i would imagine a few hundred updates per hour, but many of
them would only affect the value associated with the node, not the
structure of the three.
Adding a record (i.e., prefix/value) in a tree should be fast if 
building the missing sub-tree part of it and then just linking it to the 
main tree. Removing might need more time, a solution for it can be just 
setting the value to NULL and keep the tree structure..

Cheers,
Daniel

--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda


___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-15 Thread Daniel-Constantin Mierla

Hello,

On 11/15/11 10:09 PM, Javier Gallart wrote:


Hello

coming back to the topic related to mtree, to be able to set
values via
mi/rpc -- it won't be that difficult to add such functionality.
Usually
with a tree is mainly reading, due to fast matching on tree indexing.
The question is how many times/often do you need to change values and
how many of them at the same time (more or less).

I was also thinking about that: our application works in such a way 
that the full tree is recalculated every 15 minutes. Currently we have 
aound 60 branches with 15 entries each.


I assume many times the changes will be somewhere down the tree, since
the first part of the number is usually the same (e.g., country
code and
operator prefix). To update the tree at runtime, while there are reads
on it, there must be used a lock to be safe an consistent. If you
do lot
of writes and very often, then you keep the tree structure locked
a lot,
slowing the search.


That's the case, tname is rarely updated, it's tvalue the column that 
we normally update.


I guess you meant actually tprefix is not updated -- tname is table 
name, tprefix is the prefix used to build the tree. What would be the 
percentage of tvalue updates, do you have add/remove of tprefix-es?




Can you estimate the number of writes and how often they would be on a
daily basis? There might be other solutions to look at, if writes are
very often.


From the numbers above, let's say that we need to update around 8M 
records every 15 minutes (we expect this number will keep steadily 
increasing...)


But not all of them change the value, right?



As a side note, I've looked at how to implement the mt_match 
equivalent in redis and it does't look that hard using kamailio 
s.prefixed transformation (as you suggested) and redis sorted sets. 
I'll need to make more tests to check the performance, I'll share the 
results.


I have used this solution with mysql (using a table structure in memory 
of mysql server), since the changes could have been done in mysql in 
normal way, in one update query, which is a sync operation. Does redis 
do any caching or is using always the disk file db?


Cheers,
Daniel



Thanks!

Javi


Cheers,
Daniel

 Regards

 Javi



___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-14 Thread Daniel-Constantin Mierla

Hello,

On 11/12/11 1:28 PM, Javier Gallart wrote:

Thanks Daniel

yes, I thought about that too, I like redis a lot, and the redis 
module addition to kamailio is excellent news. However in this context 
it's not trivial to write a function that looks for  the best match in 
a redis tree structure as mt_match does...
ok, in the same idea of a remote caching system, we have memcache 
connector module, but probably it is the same situation as with redis.


Back to initial topic, I am not a user of db_berkeley, but afaik, the 
module loads the content in memory of kamailio, so if you use mtree, 
then it is practically a duplicate of content. Besides, I guess 
db_berkeley will have some internal structure overhead that will use a 
bit more memory.


What I can think of at this moment for a solution will be 
adding/removing prefixes from mtree using mi/rpc command, so in case of 
change of records, instead of loading the database table, the updates 
can be done from command line or so.


Another option might be using database directly. With mysql, a good 
solution is to define database table in memory, then add/updates records 
there as needed. From config file, use sqlops with sql_query, matching 
using 'IN' operator, against matching number exapnded with s.prefixes 
transformation:


http://www.kamailio.org/wiki/cookbooks/3.2.x/transformations#sprefixes_len

Also, afaik, if you want postgres, it has some sort of index plugin that 
can be used to match on longest prefix.


Cheers,
Daniel



Regards

Javi

On Sat, Nov 12, 2011 at 9:28 AM, Daniel-Constantin Mierla 
mico...@gmail.com mailto:mico...@gmail.com wrote:


Hello,

just mentioning ndb_redis module (in 3.2) - you may want to look
at, it is key based access memory system. Otherwise, I haven't
used personally berkeley db to comment on this particular subject.

Cheers,
Daniel




On 11/11/11 7:50 PM, Javier Gallart wrote:

Hi list

we've been happily using the mtree module for months now. Lately
the size of the tree has grown a lot. The mtree table needs to be
fully repopulated and reloaded several times a day, and we are
looking for a fastest mechanism (for populating the table, I
guess the reload time does not depend much on the db backend...).
Does anyone tried with Berkeley DB? Is this combination
mtree-berkeley actually feasible...?

Thanks


Javi


___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org  mailto:sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


-- 
Daniel-Constantin Mierla --http://www.asipto.com

Kamailio Advanced Training, Dec 5-8, Berlin:http://asipto.com/u/kat
http://linkedin.com/in/miconda  -- http://twitter.com/miconda




___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-14 Thread Daniel-Constantin Mierla



On 11/13/11 9:37 AM, Juha Heinanen wrote:

Daniel-Constantin Mierla writes:


just mentioning ndb_redis module (in 3.2) - you may want to look at, it
is key based access memory system.

sad that redis does not support master-master redundancy, but requires
manual promotion of slave to master in case master fails.
are there any other no-sql database systems that have such mechanism? 
Might not be hard to make a connector when the time will allow -- just 
to know the best options here.


Otherwise, I haven't searched there might be some tools/scripts out 
there (e.g., monit, ucarp) that may help to make the detection of 
failure and switch to standby more automatic.


Cheers,
Daniel

--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda


___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-14 Thread Juha Heinanen
Daniel-Constantin Mierla writes:

 are there any other no-sql database systems that have such mechanism? 
 Might not be hard to make a connector when the time will allow -- just 
 to know the best options here.

http://www.couchbase.org/ claims out-of-box clustering support, but i
don't have personal experience with it.

what comes to adding/removing mtree prefixes using mi/rpc instead of
reloading the whole thing from database, would be a good improvement.

-- juha

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-14 Thread Douglas Hubler
On Mon, Nov 14, 2011 at 5:10 AM, Daniel-Constantin Mierla
mico...@gmail.com wrote:
 are there any other no-sql database systems that have such mechanism? Might
 not be hard to make a connector when the time will allow -- just to know the
 best options here.

mongodb will auto promote.  Caveat, (like redis if i understand
correctly), is that all writes are directed to a single master (be it
chosen dynamically), but reads can happen anywhere to spread the load.
 Also, you need to accept the distaster scenario of a network
partition  where a minority set of servers find themselves w/o a
master.  Example: 5 servers in datacenter #1 and 4 servers in
datacenter #2.  If the link between datacenters is broken, then all
servers in datacenter #2 will not have a master and will be read-only
until link is restored.  Good part about single master is there's no
chance of inconsistent data.

Turns out local fail-over v.s. consistent data is a well explored area.

 http://blog.nahurst.com/visual-guide-to-nosql-systems

I've worked w/the C++ driver to mongodb is anyone has questions.

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-14 Thread Javier Gallart
Hello

very interesting issue actually...the mtree module fits perfectly well in a
key-value model becaue basically is what the mtree table structure defines;
that's why redis was the first thing that came to my mind when I saw the
redis module. Two problems with redis:
-no native mt_match function, up to the user to find the best option
-replication. Until the cluster feature is ready, we need to change by hand
the server ip address, which implies a kamailio restart. There is no mi
command for changing the server in the fly, right..(not in the module
documentation at least)?

Daniel, I agree that your suggestion about the mi/rpc method would be nice.
I will also take a look at Mongo as Douglas suggests, and especially
CouchDB, because you can talk to Couch DB via http...

Regards

Javi

On Mon, Nov 14, 2011 at 1:32 PM, Douglas Hubler doug...@hubler.us wrote:

 On Mon, Nov 14, 2011 at 5:10 AM, Daniel-Constantin Mierla
 mico...@gmail.com wrote:
  are there any other no-sql database systems that have such mechanism?
 Might
  not be hard to make a connector when the time will allow -- just to know
 the
  best options here.

 mongodb will auto promote.  Caveat, (like redis if i understand
 correctly), is that all writes are directed to a single master (be it
 chosen dynamically), but reads can happen anywhere to spread the load.
  Also, you need to accept the distaster scenario of a network
 partition  where a minority set of servers find themselves w/o a
 master.  Example: 5 servers in datacenter #1 and 4 servers in
 datacenter #2.  If the link between datacenters is broken, then all
 servers in datacenter #2 will not have a master and will be read-only
 until link is restored.  Good part about single master is there's no
 chance of inconsistent data.

 Turns out local fail-over v.s. consistent data is a well explored area.

  http://blog.nahurst.com/visual-guide-to-nosql-systems

 I've worked w/the C++ driver to mongodb is anyone has questions.

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-13 Thread Juha Heinanen
Daniel-Constantin Mierla writes:

 just mentioning ndb_redis module (in 3.2) - you may want to look at, it 
 is key based access memory system.

sad that redis does not support master-master redundancy, but requires
manual promotion of slave to master in case master fails.

-- juha

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-12 Thread Daniel-Constantin Mierla

Hello,

just mentioning ndb_redis module (in 3.2) - you may want to look at, it 
is key based access memory system. Otherwise, I haven't used personally 
berkeley db to comment on this particular subject.


Cheers,
Daniel



On 11/11/11 7:50 PM, Javier Gallart wrote:

Hi list

we've been happily using the mtree module for months now. Lately the 
size of the tree has grown a lot. The mtree table needs to be fully 
repopulated and reloaded several times a day, and we are looking for a 
fastest mechanism (for populating the table, I guess the reload time 
does not depend much on the db backend...). Does anyone tried with 
Berkeley DB? Is this combination mtree-berkeley actually feasible...?


Thanks


Javi


___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


--
Daniel-Constantin Mierla -- http://www.asipto.com
Kamailio Advanced Training, Dec 5-8, Berlin: http://asipto.com/u/kat
http://linkedin.com/in/miconda -- http://twitter.com/miconda

___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


Re: [SR-Users] mtree on db Berkeley DB?

2011-11-12 Thread Javier Gallart
Thanks Daniel

yes, I thought about that too, I like redis a lot, and the redis module
addition to kamailio is excellent news. However in this context it's not
trivial to write a function that looks for  the best match in a redis tree
structure as mt_match does...

Regards

Javi

On Sat, Nov 12, 2011 at 9:28 AM, Daniel-Constantin Mierla mico...@gmail.com
 wrote:

  Hello,

 just mentioning ndb_redis module (in 3.2) - you may want to look at, it is
 key based access memory system. Otherwise, I haven't used personally
 berkeley db to comment on this particular subject.

 Cheers,
 Daniel




 On 11/11/11 7:50 PM, Javier Gallart wrote:

 Hi list

  we've been happily using the mtree module for months now. Lately the
 size of the tree has grown a lot. The mtree table needs to be fully
 repopulated and reloaded several times a day, and we are looking for a
 fastest mechanism (for populating the table, I guess the reload time does
 not depend much on the db backend...). Does anyone tried with Berkeley DB?
 Is this combination mtree-berkeley actually feasible...?

  Thanks


  Javi


 ___
 SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing 
 listsr-us...@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users


 --
 Daniel-Constantin Mierla -- http://www.asipto.com
 Kamailio Advanced Training, Dec 5-8, Berlin: 
 http://asipto.com/u/kathttp://linkedin.com/in/miconda -- 
 http://twitter.com/miconda


___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users