yes thank you!
On Fri, Mar 27, 2015 at 7:02 AM, madhuka udantha <madhukaudan...@gmail.com> wrote: > Hi, > > I'm sharing a sample mxn Matrix Graph[1] so you all can get an idea of what > my proposed solution is. This can represent high number of nodes with out > any issue and at a glance you can get the Summarized View of the Cluster. > Accordingly my mxn Matrix would represent nodes in the Cluster. > (Mockup UIs are attached to JIRA) > > Does this clarify the issue Robert? > > Your feedback / ideas mostly welcomed. > > [1] http://bost.ocks.org/mike/miserables/ > > On Wed, Mar 25, 2015 at 11:46 PM, madhuka udantha <madhukaudan...@gmail.com> > wrote: >> >> Hi, >> >> Sharing mockups UIs again, Since some ML-Users do not have account in >> box.com. >> I am sharing them via jira. >> >> On Wed, Mar 25, 2015 at 12:20 AM, madhuka udantha >> <madhukaudan...@gmail.com> wrote: >>> >>> Hi, >>> >>> After communicating with couchdb community got to know that there can be >>> 100~ 10000 nodes in cluster. So there will 2 layouts in cluster link >>> depending on node count in cluster. >>> >>> My first UI is for simple count of node in cluster (4 nodes in cluster) >>> >>> Here is mock UIs where it hit more than 5 to 100000 nodes, I am using >>> matrix visual. >>> Here is initial look of the page. [Page1v1-initial-page] >>> https://app.box.com/s/sn84u6wuzya58trssw11ba6fwtmn862f >> >> >> https://issues.apache.org/jira/secure/attachment/12707205/Page1v1-initial-page.png >> >>> >>> Features: >>> 1. User can select the attribute eg: Disk usage, http status, data usage, >>> alerts. etc. >>> 2. Chart / matrix will be draw with legend >>> >>> [Page2v1-searching-node] >>> https://app.box.com/s/i5uomdhirn96co6acbbrvvviifyw95pb >> >> >> https://issues.apache.org/jira/secure/attachment/12707218/Page2v1-searching-node.png >> >>> >>> Who can we search node in there? >>> You will have search in top of it. (it allow to search the node), It will >>> show it with red color box. >>> Or every node have tool tip to verify node, what is you look for. >>> >>> >>> [Page3v1-selecting-node] >>> https://app.box.com/s/rsd5ysld597gvra2nkcn7027bpok37jc >> >> >> https://issues.apache.org/jira/secure/attachment/12707219/Page3v1-selecting-node.png >> >>> >>> When user click on node, node panel will be open RHS. >>> It have all the details of the node and operation that you can perform in >>> node leve >>> >>> [Page4v1-searching-db_in_node] >>> https://app.box.com/s/jk3ojb0khj192ldnyw0eqs1p5t7l485r >> >> >> https://issues.apache.org/jira/secure/attachment/12707220/Page4v1-searching-db_in_node.png >> >>> >>> User can search Databases in nodes even >>> This searching have advance search query support >>> >>> [Page5v1-db_view] >>> https://app.box.com/s/20hjgtcb7c9ikgfdn7xy8xli4uhorbni >> >> >> https://issues.apache.org/jira/secure/attachment/12707206/Page5v1-db_view.png >> >>> User can select the database and it will show database details. >>> >>> >>> feed backs for UIs are welcome. >>> >>> >>> On Tue, Mar 24, 2015 at 4:35 PM, madhuka udantha >>> <madhukaudan...@gmail.com> wrote: >>>> >>>> Hi James, >>>> >>>> Thanks for you feedback. It will help me to improve my proposal and to >>>> get better understanding about couchdb cluster >>>> >>>> Since I am new to couchdb and I did not know much user case that you all >>>> have come across. But when you share those ideas. It is help to understand >>>> couchdb cluster. All you feedback is important. >>>> >>>> On Tue, Mar 24, 2015 at 12:07 PM, Mutton, James <jmut...@akamai.com> >>>> wrote: >>>>> >>>>> I can share a few quick thoughts (which I admit is rare, but I happen >>>>> to have a few minutes and this hit the top of my stack): >>>>> >>>>> * How do you envision this overflowing when you have hundreds of >>>>> databases within a cluster, or maybe 64 nodes in a cluster. >>>> >>>> Attached UIs can only handle very low level nodes (4 to 8) according to >>>> layout (we can called it expanded layout). When I am drawing I did not >>>> thought that level of count of nodes and databases. Therefore I will >>>> revisit >>>> the mockup ui. Here is updated UI >>>> >>>> Here you can have 1 ~ 1000 node even and each node can contain 100 >>>> databases. >>>> Node are listed and user can pick one of it then it will show it details >>>> and databases of it in list. >>>> >>>> UI for Listing node when nodes count is above 8 in cluster (This is how >>>> it look as it have 64 nodes in cluster) >>>> https://app.box.com/s/kqg2kadf0xa1zoym9sme22q5m4wbg8i5 >>>> [Page10-list-nodes-more-than-8] >>>> >>>> Features of the UI: >>>> >>>> Table header support sorting (ascending and descending) by just click on >>>> it >>>> User have searching option to search (node/databases by name) >>>> It support advance search query (popup form ui) >>>> Notification listed with icon and warning/alert level is shown by color >>>> of icon >>>> User can perform node operation by clicking hyperlink in it's raw >>>> >>>> >>>> Node contains 100 databases (answer) >>>> https://app.box.com/s/ggbzr00orkwacsi4cflpvn9d60q7h1st >>>> [Page11-list-dbs-in-selected-node] >>>> >>>> >>>> >>>> Features of the UI >>>> >>>> In Database panel contain search that scope to databases of the selected >>>> node >>>> When user select database it will show database summary details and >>>> operations that can be perform for that database >>>> >>>> >>>>> >>>>> ** On a related note, drag and drop is cool, but level of difficulty >>>>> involved in adding a shard from node-1 increases in direct proportion to >>>>> the >>>>> number of nodes there are. >>>> >>>> Yes, we can add button ( in database/node level >>>> >>>>> I think the “Add Database” button on a new server should be labeled >>>>> “Add Shard” and should present the option of selecting what shards to add. >>>>> The add button is also not present on the nodes that have shards already >>>>> so >>>>> it would be good to keep it around somehow. >>>> >>>> Yes, It will be label as “Add Shard”. It is possible to show only that >>>> is not shard so far. >>>> >>>> >>>>> * Speaking then of shards… They don’t appear to be captured here. How >>>>> does one know which shard-set each server has? Each database has a number >>>>> of shards that all have multiple shard copies and it is not a requirement >>>>> that all servers have all shards. None of that is really represented in >>>>> these mockups. When looking at a shard, it would be good to know if it is >>>>> 00000000-15555554 or aaaaaaa8-bffffffc, or even both. Knowing this and >>>>> being able to use it to scale/move hotspots in a cluster is pretty >>>>> important >>>>> for operations, once databases hit a significant size. >>>> >>>> Thanks for pointing out. shards will add for visualization. >>>> >>>> Can you share resources/documentations which that help to understand the >>>> couchdb clustering? >>>> I like to know what are most important operations regard couch >>>> clustering? (that you think good to added in here. since I am new for >>>> couchdb and some time I have miss them) >>>> >>>> >>>>> >>>>> * I don’t think doc-count and deleted-count is useful information at >>>>> the overview page >>>> >>>> Thanks, We can remove doc count details from overview page. If user need >>>> to read such details he/she can click 'more' so there it will show them. >>>>> >>>>> >>>>> >>>>> and runs the risk of requiring a lot of requests to fulfill (see point >>>>> 1 on hundreds of databases). >>>> >>>> Sorry, In here I did not get your point >>>> >>>>> * What does compact: false mean? That the database is not small? or >>>>> that it’s not actively compacting? And if the latter, wouldn’t that be >>>>> on a >>>>> db-shard basis, not a whole server. >>>> >>>> compact running is mapping to 'compact_running'[2] >>>> >>>> [1] https://wiki.apache.org/couchdb/Compaction >>>> >>>>> >>>>> * It would be nice if cluster usage graphing page would show http-stats >>>>> as well. Would give people a snapshot indication of where their load is >>>>> hot. >>>> >>>> Sure, Different status codes will be notify on cluster usage graph. >>>> (Mockups UI will shared tonight) >>>> >>>> Thanks JamesM for your feed backs >>>>> >>>>> >>>>> </JamesM> >>>>> >>>>> >>>>> On Mar 23, 2015, at 20:32, madhuka udantha <madhukaudan...@gmail.com> >>>>> wrote: >>>>> >>>>> > Hi All, >>>>> > >>>>> > are there any ideas and feedback for above mockup UIs? >>>>> > >>>>> > Community feedback will be useful in here. :) >>>>> > >>>>> > On Sun, Mar 22, 2015 at 11:51 PM, madhuka udantha >>>>> > <madhukaudan...@gmail.com> >>>>> > wrote: >>>>> > >>>>> >> Hi, >>>>> >> >>>>> >> Here are the mockup UIs for Visualizing the CouchDB Clusters. It is >>>>> >> just >>>>> >> the initial UIs. >>>>> >> >>>>> >> * Page 1-cluster.png* [ >>>>> >> https://app.box.com/s/isjbfop69j2cmz0kg0qar7l5nmupjnfl >>>>> >> >>>>> >> This Page visualize nodes in the clusters with summary information >>>>> >> for >>>>> >> each node. (Name, Memory usage..etc..) with a nice web UI interface. >>>>> >> It is >>>>> >> interactive (drag and drop supporting over nodes) >>>>> >> >>>>> >> *Page 2-cluster-notification-tip.png [* >>>>> >> https://app.box.com/s/euc5megmx5lwseqib4511309ocoypryb >>>>> >> <https://app.box.com/s/euc5megmx5lwseqib4511309ocoypryb>] >>>>> >> >>>>> >> Top of each panel of the nodes it contains a notification icon if >>>>> >> there is >>>>> >> notification. >>>>> >> eg: Memory/disk space availability, Error in node etc... >>>>> >> Notification will have a simple drop down structure as it is >>>>> >> showing. >>>>> >> >>>>> >> *Page 3-cluster-warning.png* [ >>>>> >> https://app.box.com/s/f2ljwxvz2w01f8uepsj4zyarhwbpiz8b] >>>>> >> >>>>> >> Critical level alert is showen. (Alert level can be set in config >>>>> >> and >>>>> >> notification center) >>>>> >> >>>>> >> *Page 4-adding-new-node.png* [ >>>>> >> https://app.box.com/s/noygbeyze792bh4t93iaoxywdciau057] >>>>> >> >>>>> >> By clicking "add new node" you can add new nodes for clusters. UI >>>>> >> Consistency is maintained >>>>> >> >>>>> >> *Page 5-adding-new-node-by-drag-drop.png*[ >>>>> >> https://app.box.com/s/gbfsbwb6zt1jdkijve72mnullplecpj7] >>>>> >> >>>>> >> User interface is supported for drag-drop of databases in nodes. >>>>> >> (replication operation in the BE) >>>>> >> >>>>> >> *Page 6-selected-db-in-node.png*[ >>>>> >> https://app.box.com/s/6ojrvpxuvmo9k18dlnf8i4nmibzdx5s9] >>>>> >> Each node shows the databases in it. By Clicking on database you can >>>>> >> see >>>>> >> detailed view of a particular database. This allows users to >>>>> >> understand >>>>> >> memory/data/doc distribution over DBs in particular nodes. >>>>> >> >>>>> >> *Page 7-custer-mgt-center.png* [ >>>>> >> https://app.box.com/s/t9s42g3gwvv6bocbdsxrgjimdrh2ty0z] >>>>> >> Each node is shown with disk usage under icon and summary text of >>>>> >> the node >>>>> >> is also given. It is easy to identify the disk space - free >>>>> >> availability. (In here we need some BE services help) >>>>> >> >>>>> >> >>>>> >> *Page 8-custer-mgt-center-infor.png* [ >>>>> >> https://app.box.com/s/vvo4v9ji79l8japo3nm9f4lfbjhdamkc] >>>>> >> User can customize notifications for each nodes by level (warning, >>>>> >> infor, >>>>> >> etc..) >>>>> >> >>>>> >> *Page 9-charting-disk-usages.png*[ >>>>> >> https://app.box.com/s/fk7eyoyrtg8d3b29xoo0mdho9rr8cmyz] >>>>> >> Cluster usage of the system is shown as a bar chart and the >>>>> >> attributes to >>>>> >> be shown can be selected according to user's preferences. >>>>> >> >>>>> >> You ideas are welcome in here. >>>>> >> Thanks >>>>> >> >>>>> >> On Sun, Mar 22, 2015 at 7:34 PM, madhuka udantha >>>>> >> <madhukaudan...@gmail.com >>>>> >>> wrote: >>>>> >> >>>>> >>> Hi All, >>>>> >>> >>>>> >>> As I'm drafting my proposal for GSoC 2015 I went through some of >>>>> >>> the >>>>> >>> existing Cluster Management Systems with Visualization. I >>>>> >>> recognized some >>>>> >>> of the features regarding to Cluster Management HTTP API. (Some >>>>> >>> times not >>>>> >>> fully documented yet) >>>>> >>> Such us partitioning, re-balance..etc. >>>>> >>> I was unable to find them on CouchDB API Documentation(only at the >>>>> >>> implemented level but not documented yet? ) but I was able to find >>>>> >>> some of >>>>> >>> the features such as adding nodes to clusters through dev-mailing >>>>> >>> list. >>>>> >>> Are some of these features regarding cluster management present on >>>>> >>> the >>>>> >>> API. If so where can I find the relevant documentations/resources? >>>>> >>> Even >>>>> >>> the source code is fine to me. >>>>> >>> >>>>> >>> >>>>> >>> Since I was able to clearly identify those missing features can I >>>>> >>> start a >>>>> >>> discussion on a thread on the best feature list for cluster >>>>> >>> management on >>>>> >>> CouchDB to be added to it on a future release. >>>>> >>> My current project could also include these features that are >>>>> >>> supported >>>>> >>> by the API else I could help on including them. >>>>> >>> >>>>> >>> [1] http://guide.couchdb.org/draft/clustering.html >>>>> >>> Thanks. >>>>> >>> BR, >>>>> >>> >>>>> >>> On Sat, Mar 21, 2015 at 11:40 AM, madhuka udantha < >>>>> >>> madhukaudan...@gmail.com> wrote: >>>>> >>> >>>>> >>>> >>>>> >>>> >>>>> >>>> On Sat, Mar 21, 2015 at 12:17 AM, Alexander Shorin >>>>> >>>> <kxe...@gmail.com> >>>>> >>>> wrote: >>>>> >>>> >>>>> >>>>> On Fri, Mar 20, 2015 at 9:28 PM, madhuka udantha >>>>> >>>>> <madhukaudan...@gmail.com> wrote: >>>>> >>>>>> I like to know what is *different of '_nodes' and 'node'*? >>>>> >>>>>> >>>>> >>>>>> http://localhost:15986/nodes >>>>> >>>>>> -->{"db_name":"nodes","doc_count":3..} >>>>> >>>>>> http://localhost:25986/nodes >>>>> >>>>>> -->{"db_name":"nodes","doc_count":1..} >>>>> >>>>>> http://localhost:35986/nodes >>>>> >>>>> -->{"error":"not_found","reason":"no_db_file"} >>>>> >>>>> >>>>> >>>>> Where did you get these nodes? >>>>> >>>>> >>>>> >>>> By '~/test/1/couchdb $ dev/run -n 4' I can added 4 nodes. (default >>>>> >>>> it is >>>>> >>>> 3 nodes) >>>>> >>>> >>>>> >>>> Here is my console print, Yes now I have four node running. >>>>> >>>> >>>>> >>>> *max@max-VirtualBox ~/test/1/couchdb $ dev/run -n 4* >>>>> >>>> *Setup environment... ok* >>>>> >>>> *Ensure CouchDB is built... ok* >>>>> >>>> *Prepare configuration files... ok* >>>>> >>>> *Start node node1... ok* >>>>> >>>> *Start node node2... ok* >>>>> >>>> *Start node node3... ok* >>>>> >>>> *Start node node4... ok* >>>>> >>>> *Ensure all nodes are run... ok* >>>>> >>>> *Join nodes into cluster... ok* >>>>> >>>> *Developers cluster is set up at http://127.0.0.1:15984 >>>>> >>>> <http://127.0.0.1:15984>. Time to hack!... * >>>>> >>>> >>>>> >>>> Each above nodes are running in localhost in port 15986, 25986, >>>>> >>>> 35986 >>>>> >>>> and 45986 >>>>> >>>> >>>>> >>>> It seems docs are mainly in 15986 (node1) shown by 'doc_count' >>>>> >>>> value, >>>>> >>>> is there any reason for that or is it happening randomly? >>>>> >>>> >>>>> >>>> >>>>> >>>> >>>>> >>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> ,,,^..^,,, >>>>> >>>>> >>>>> >>>> >>>>> >>>> >>>>> >>>> >>>>> >>>> -- >>>>> >>>> Cheers, >>>>> >>>> Madhuka Udantha >>>>> >>>> http://madhukaudantha.blogspot.com >>>>> >>>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> -- >>>>> >>> Cheers, >>>>> >>> Madhuka Udantha >>>>> >>> http://madhukaudantha.blogspot.com >>>>> >>> >>>>> >> >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> Cheers, >>>>> >> Madhuka Udantha >>>>> >> http://madhukaudantha.blogspot.com >>>>> >> >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > Cheers, >>>>> > Madhuka Udantha >>>>> > http://madhukaudantha.blogspot.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Cheers, >>>> Madhuka Udantha >>>> http://madhukaudantha.blogspot.com >>> >>> >>> >>> >>> -- >>> Cheers, >>> Madhuka Udantha >>> http://madhukaudantha.blogspot.com >> >> >> >> >> -- >> Cheers, >> Madhuka Udantha >> http://madhukaudantha.blogspot.com > > > > > -- > Cheers, > Madhuka Udantha > http://madhukaudantha.blogspot.com
