fix csv-table warning
Project: http://git-wip-us.apache.org/repos/asf/incubator-s2graph/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-s2graph/commit/c131990e Tree: http://git-wip-us.apache.org/repos/asf/incubator-s2graph/tree/c131990e Diff: http://git-wip-us.apache.org/repos/asf/incubator-s2graph/diff/c131990e Branch: refs/heads/master Commit: c131990e6d5c0a4b342d0c46f7d17727600f0f36 Parents: 6414cb7 Author: daewon <dae...@apache.org> Authored: Thu Dec 27 16:18:48 2018 +0900 Committer: daewon <dae...@apache.org> Committed: Thu Dec 27 16:18:48 2018 +0900 ---------------------------------------------------------------------- doc/source/api/management/index.rst | 444 +++++++++++++++++++++++++++ doc/source/api/mutate/index.rst | 10 + doc/source/api/mutate/mutate_edge.rst | 314 +++++++++++++++++++ doc/source/api/mutate/mutate_vertex.rst | 139 +++++++++ 4 files changed, 907 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-s2graph/blob/c131990e/doc/source/api/management/index.rst ---------------------------------------------------------------------- diff --git a/doc/source/api/management/index.rst b/doc/source/api/management/index.rst new file mode 100644 index 0000000..17061b0 --- /dev/null +++ b/doc/source/api/management/index.rst @@ -0,0 +1,444 @@ +Management APIs +================== +Admin Apis for Management Service, Label, Index .. + +**************** +Create a Service +**************** + +``Service`` is the top level abstraction in S2Graph which could be considered as a database in MySQL. + +.. code:: bash + + POST /admin/createService + +Service Fields +--------------- + +In order to create a Service, the following fields should be specified in the request. + +.. csv-table:: Option + :header: "Field Name", "Definition", "Data Type", "Example", "Note" + :widths: 15, 30, 30, 30, 30 + + "serviceName", "User defined namespace", "String", "talk_friendship", "Required" + "cluster", "Zookeeper quorum address", "String", "abc.com:2181,abd.com:2181", "Optional" + "hTableName", "HBase table name", "String", "test", "Optional" + "hTableTTL", "Time to live setting for data","Integer", "86000", "Optional" + "preSplitSize", "Factor for the table pre-split size", "Integer", "1", "Optional" + +.. list + - By default, S2Graph looks for "hbase.zookeeper.quorum" in your application.conf. If "hbase.zookeeper.quorum" is undefined, this value is set as "localhost". + + +Basic Service Operations +-------------------------- + +You can create a service using the following API: + +.. code:: bash + + curl -XPOST localhost:9000/admin/createService -H 'Content-Type: Application/json' -d ' + { + "serviceName": "s2graph", + "cluster": "address for zookeeper", + "hTableName": "hbase table name", + "hTableTTL": 86000, + "preSplitSize": 2 + }' + + +**************** +Create a Label +**************** + +A ``Label`` represents a relation between two serviceColumns. Labels are to S2Graph what tables are to RDBMS since they contain the schema information, i.e. descriptive information of the data being managed or indices used for efficient retrieval. +In most scenarios, defining an edge schema (in other words, label) requires a little more care compared to a vertex schema (which is pretty straightforward). +First, think about the kind of queries you will be using, then, model user actions or relations into ``edges`` and design a label accordingly. + +.. code:: bash + + POST /admin/createLabel + +Label Fields +--------------- + +A Label creation request includes the following information. + +.. csv-table:: Option + :header: "Field Name", "Definition", "Data Type", "Example", "Note" + :widths: 15, 30, 30, 30, 30 + + "label", "Name of the relation", "String", "talk_friendship", "Required" + "srcServiceName", "Source column's service", "String", "kakaotalk", "Required" + "srcColumnName", "Source column's name", "String", "user_id", "Required" + "srcColumnType", "Source column's data type","Long/Integer/String", "string", "Required" + "tgtServiceName", "Target column's service", "String", "kakaotalk/kakaomusic", "Optional" + "tgtColumnName", "Target column's name", "String", "item_id", "Required" + "tgtColumnType", "Target column's data type", "Long/Integer/String", "string", "Required" + "isDirected", "Wether the label is directed or undirected", "True/False", "true/false", "Optional. default is true" + "serviceName", "Which service the label belongs to", "String", "kakaotalk", "Optional. tgtServiceName is used by default" + "hTableName", "A dedicated HBase table to your Label", "String", "s2graph-batch", "Optional. Service hTableName is used by default" + "hTableTTL", "Data time to live setting", "Integer", "86000", "Optional. Service hTableTTL is used by default" + "consistencyLevel", "If set to 'strong', only one edge is alowed between a pair of source/ target vertices. Set to 'weak', and multiple-edge is supported", "String", "strong/weak", "Optional. default is 'weak'" + + +Props & Indices +---------------- + +A couple of key elements of a Label are its Properties (props) and indices. +Supplementary information of a Vertex or Edge can be stored as props. A single property can be defined in a simple key-value JSON as follows: + +.. code:: json + + { + "name": "name of property", + "dataType": "data type of property value", + "defaultValue": "default value in string" + } + +In a scenario where user - video playback history is stored in a Label, a typical example for props would look like this: + +.. code:: json + + [ + {"name": "play_count", "defaultValue": 0, "dataType": "integer"}, + {"name": "is_hidden","defaultValue": false,"dataType": "boolean"}, + {"name": "category","defaultValue": "jazz","dataType": "string"}, + {"name": "score","defaultValue": 0,"dataType": "float"} + ] + +Props can have data types of ``numeric`` (byte/ short/ integer/ float/ double), ``boolean`` or ``string``. +In order to achieve efficient data retrieval, a Label can be indexed using the "indices" option. +Default value for indices is ``_timestamp``, a hidden label property. + +All labels have ``_timestamp`` in their props under the hood + +The first index in indices array will be the primary index ``(Think of PRIMARY INDEX idx_xxx(p1, p2) in MySQL)`` +S2Graph will automatically store edges according to the primary index. +Trailing indices are used for multiple ordering on edges. ``(Think of ALTER TABLE ADD INDEX idx_xxx(p2, p1) in MySQL)`` + +props define meta datas that will not be affect the order of edges. +Please avoid using S2Graph-reserved property names: + + +- ``_timestamp`` is reserved for system wise timestamp. this can be interpreted as last_modified_at +- ``_from`` is reserved for label's start vertex. +- ``_to`` is reserved for label's target vertex. + + +Basic Label Operations +-------------------------- + +Here is an sample request that creates a label ``user_article_liked`` between column ``user_id`` of service ``s2graph`` and column ``article_id`` of service ``s2graph_news``. +Note that the default indexed property ``_timestamp`` will be created since the ``indexedProps`` field is empty. + +.. code:: bash + + curl -XPOST localhost:9000/admin/createLabel -H 'Content-Type: Application/json' -d ' + { + "label": "user_article_liked", + "srcServiceName": "s2graph", + "srcColumnName": "user_id", + "srcColumnType": "long", + "tgtServiceName": "s2graph_news", + "tgtColumnName": "article_id", + "tgtColumnType": "string", + "indices": [], // _timestamp will be used as default + "props": [], + "serviceName": "s2graph_news" + }' + + +The created label ``user_article_liked`` will manage edges in a timestamp-descending order (which seems to be the common requirement for most services). +Here is another example that creates a label ``friends``, which represents the friend relation between ``users`` in service ``s2graph``. +This time, edges are managed by both affinity_score and ``_timestamp``. + +Friends with higher affinity_scores come first and if affinity_score is a tie, recently added friends comes first. + +.. code:: bash + + curl -XPOST localhost:9000/admin/createLabel -H 'Content-Type: Application/json' -d ' + { + "label": "friends", + "srcServiceName": "s2graph", + "srcColumnName": "user_id", + "srcColumnType": "long", + "tgtServiceName": "s2graph", + "tgtColumnName": "user_id", + "tgtColumnType": "long", + "indices": [ + {"name": "idx_affinity_timestamp", "propNames": ["affinity_score", "_timestamp"]} + ], + "props": [ + {"name": "affinity_score", "dataType": "float", "defaultValue": 0.0}, + {"name": "_timestamp", "dataType": "long", "defaultValue": 0}, + {"name": "is_hidden", "dataType": "boolean", "defaultValue": false}, + {"name": "is_blocked", "dataType": "boolean", "defaultValue": true}, + {"name": "error_code", "dataType": "integer", "defaultValue": 500} + ], + "serviceName": "s2graph", + "consistencyLevel": "strong" + }' + +S2Graph supports **multiple indices** on a label which means you can add separate ordering options for edges. + + +.. code:: bash + + curl -XPOST localhost:9000/admin/addIndex -H 'Content-Type: Application/json' -d ' + { + "label": "friends", + "indices": [ + {"name": "idx_3rd", "propNames": ["is_blocked", "_timestamp"]} + ] + }' + +In order to get general information on a label, make a GET request to ``/admin/getLabel/{label name}`` + +.. code:: bash + + curl -XGET localhost:9000/admin/getLabel/friends + + +Delete a label with a PUT request to ``/admin/deleteLabel/{label name}`` + +.. code:: bash + + curl -XPUT localhost:9000/admin/deleteLabel/friends + + +Label updates are not supported (except when you are adding an index). Instead, you can delete the label and re-create it. + +Adding Extra Properties to Labels +---------------------------------- + +To add a new property, use ``/admin/addProp/{label name}`` + +.. code:: bash + + curl -XPOST localhost:9000/admin/addProp/friend -H 'Content-Type: Application/json' -d ' + { + "name": "is_blocked", + "defaultValue": false, + "dataType": "boolean" + }' + + +Consistency Level +------------------ + +Simply put, the consistency level of your label will determine how the edges are stored at storage level. +First, note that S2Graph identifies a unique edge by combining its from, label, to values as a key. + +Now, let's consider inserting the following two edges that have same keys (1, graph_test, 101) and different timestamps (1418950524721 and 1418950524723). + +.. code:: bash + + 1418950524721 insert e 1 101 graph_test {"weight": 10} = (1, graph_test, 101) + 1418950524723 insert e 1 101 graph_test {"weight": 20} = (1, graph_test, 101) + + +**Each consistency levels handle the case differently.** + +- strong + + - The strong option makes sure that there is only one edge record stored in the HBase table for edge key (1, graph_test, 101). With strong consistency level, the later insertion will overwrite the previous one. + +- weak + + - The weak option will allow two different edges stored in the table with different timestamps and weight values. + + +For a better understanding, let's simplify the notation for an edge that connects two vertices u - v at time t as u -> (t, v), and assume that we are inserting these four edges into two different labels with each consistency configuration (both indexed by timestamp only). + +.. code:: bash + + u1 -> (t1, v1) + u1 -> (t2, v2) + u1 -> (t3, v2) + u1 -> (t4, v1) + +With a strong consistencyLevel, your Label contents will be: + +.. code:: bash + + u1 -> (t4, v1) + u1 -> (t3, v2) + +Note that edges with same vertices and earlier timestamp (u1 -> (t1, v1) and u1 -> (t2, v2)) were overwritten and do not exist. +On the other hand, with consistencyLevel weak. + +.. code:: bash + + u1 -> (t1, v1) + u1 -> (t2, v2) + u1 -> (t3, v2) + u1 -> (t4, v1) + +**It is recommended to set consistencyLevel to weak unless you are expecting concurrent updates on same edge.** + +In real world systems, it is not guaranteed that operation requests arrive at S2Graph in the order of their timestamp. Depending on the environment (network conditions, client making asynchronous calls, use of a message que, and so on) request that were made earlier can arrive later. Consistency level also determines how S2Graph handles these cases. +Strong consistencyLevel promises a final result consistent to the timestamp. +For example, consider a set of operation requests on edge (1, graph_test, 101) were made in the following order; + + +.. code:: bash + + 1418950524721 insert e 1 101 graph_test {"is_blocked": false} + 1418950524722 delete e 1 101 graph_test + 1418950524723 insert e 1 101 graph_test {"is_hidden": false, "weight": 10} + 1418950524724 update e 1 101 graph_test {"time": 1, "weight": -10} + 1418950524726 update e 1 101 graph_test {"is_blocked": true} + + +and actually arrived in a shuffled order due to complications + + +.. code:: bash + + 1418950524726 update e 1 101 graph_test {"is_blocked": true} + 1418950524723 insert e 1 101 graph_test {"is_hidden": false, "weight": 10} + 1418950524722 delete e 1 101 graph_test + 1418950524721 insert e 1 101 graph_test {"is_blocked": false} + 1418950524724 update e 1 101 graph_test {"time": 1, "weight": -10} + +Strong consistency still makes sure that you get the same eventual state on (1, graph_test, 101). +Here is pseudocode of what S2Graph does to provide a strong consistency level. + +.. code:: bash + + complexity = O(one read) + O(one delete) + O(2 put) + + fetchedEdge = fetch edge with (1, graph_test, 101) from lookup table. + + if fetchedEdge is not exist: + create new edge same as current insert operation + update lookup table as current insert operation + else: + valid = compare fetchedEdge vs current insert operation. + if valid: + delete fetchedEdge + create new edge after comparing fetchedEdge and current insert. + update lookup table + +Limitations Since S2Graph makes asynchronous writes to HBase via Asynchbase, there is no consistency guaranteed on same edge within its flushInterval (1 second). + +Adding Extra Indices (Optional) +--------------------------------- + +.. code:: bash + + POST /admin/addIndex + +A label can have multiple properties set as indexes. When edges are queried, the ordering will determined according to indexes, therefore, deciding which edges will be included in the top-K results. + +**Edge retrieval queries in S2Graph by default returns top-K edges. Clients must issue another query to fetch the next K edges, i.e., top-K ~ 2 x top-K** + +Edges sorted according to the indices in order to limit the number of edges being fetched by a query. If no ordering property is given, S2Graph will use the timestamp as an index, thus resulting in the most recent data. + +**It would be extremely difficult to fetch millions of edges and sort them at request time and return a top-K in a reasonable amount of time. Instead, S2Graph uses vertex-centric indexes to avoid this. +Using a vertex-centric index, having millions of edges is fine as long as size K of the top-K values is reasonable (under 1K) Note that indexes must be created prior to inserting any data on the label (which is the same case with the conventional RDBMS).** + +New indexes can be dynamically added, but will not be applied to pre-existing data (support for this is planned for future versions). Currently, a label can have up to eight indices. +The following is an example of adding index ``play_count`` to a label ``graph_test``. + +.. code:: bash + + // add prop first + curl -XPOST localhost:9000/admin/addProp/graph_test -H 'Content-Type: Application/json' -d ' + { "name": "play_count", "defaultValue": 0, "dataType": "integer" }' + + // then add index + curl -XPOST localhost:9000/admin/addIndex -H 'Content-Type: Application/json' -d ' + { + "label": "graph_test", + "indices": [ + { name: "idx_play_count", propNames: ["play-count"] } + ] + }' + + +********************************** +Create a ServiceColumn (Optional) +********************************** + +.. code:: bash + + POST /admin/createServiceColumn + +If your use case requires props assigned to vertices instead of edges, what you need is a Service Column + +**Remark: If it is only the vertex id that you need and not additional props, there's no need to create a Service Column explicitly. At label creation, by default, S2Graph creates column space with empty properties according to the label schema.** + + +Service Column Fields +---------------------- + + +.. csv-table:: Option + :header: "Field Name", "Definition", "Data Type", "Example", "Note" + :widths: 15, 30, 30, 30, 30 + + "Field Name", "Definition", "Data Type", "Example", "Remarks" + "serviceName", "Which service the Service Column belongs to", "String", "kakaotalk", "Required" + "columnName", "Service Column`s name", "String", "talk_user_id", "Required" + "props", "Optional properties of Service Column", "JSON (array dictionaries)", "Please refer to the examples", "Optional" + + +Basic Service Column Operations +------------------------------- + +Here are some sample requests for Service Column creation as well as vertex insertion and selection. + + +.. code:: bash + + curl -XPOST localhost:9000/admin/createServiceColumn -H 'Content-Type: Application/json' -d ' + { + "serviceName": "s2graph", + "columnName": "user_id", + "columnType": "long", + "props": [ + {"name": "is_active", "dataType": "boolean", "defaultValue": true}, + {"name": "phone_number", "dataType": "string", "defaultValue": "-"}, + {"name": "nickname", "dataType": "string", "defaultValue": ".."}, + {"name": "activity_score", "dataType": "float", "defaultValue": 0.0}, + {"name": "age", "dataType": "integer", "defaultValue": 0} + ] + }' + +General information on a vertex schema can be retrieved with ``/admin/getServiceColumn/{service name}/{column name}`` + +.. code:: bash + + curl -XGET localhost:9000/admin/getServiceColumn/s2graph/user_id + +This will give all properties on serviceName ``s2graph`` and columnName ``user_id`` serviceColumn. +Properties can be added to a Service Column with ``/admin/addServiceColumnProps/{service name}/{column name}`` + +.. code:: bash + + curl -XPOST localhost:9000/admin/addServiceColumnProps/s2graph/user_id -H 'Content-Type: Application/json' -d ' + [ + {"name": "home_address", "defaultValue": "korea", "dataType": "string"} + ]' + +Vertices can be inserted to a Service Column using ``/admin/vertices/insert/{service name}/{column name}`` + +.. code:: bash + + curl -XPOST localhost:9000/mutate/vertex/insert/s2graph/user_id -H 'Content-Type: Application/json' -d ' + [ + {"id":1,"props":{"is_active":true}, "timestamp":1417616431}, + {"id":2,"props":{},"timestamp":1417616431} + ]' + +Finally, query your vertex via ``/graphs/getVertices`` + +.. code:: bash + + curl -XPOST localhost:9000/graphs/getVertices -H 'Content-Type: Application/json' -d ' + [ + {"serviceName": "s2graph", "columnName": "user_id", "ids": [1, 2, 3]} + ]' http://git-wip-us.apache.org/repos/asf/incubator-s2graph/blob/c131990e/doc/source/api/mutate/index.rst ---------------------------------------------------------------------- diff --git a/doc/source/api/mutate/index.rst b/doc/source/api/mutate/index.rst new file mode 100644 index 0000000..c85c55d --- /dev/null +++ b/doc/source/api/mutate/index.rst @@ -0,0 +1,10 @@ +Mutate APIs +============================================== + +Contents: + +.. toctree:: + :maxdepth: 2 + + mutate_edge + mutate_vertex http://git-wip-us.apache.org/repos/asf/incubator-s2graph/blob/c131990e/doc/source/api/mutate/mutate_edge.rst ---------------------------------------------------------------------- diff --git a/doc/source/api/mutate/mutate_edge.rst b/doc/source/api/mutate/mutate_edge.rst new file mode 100644 index 0000000..1e21079 --- /dev/null +++ b/doc/source/api/mutate/mutate_edge.rst @@ -0,0 +1,314 @@ +**************** +Manage Edges +**************** + +An ``Edge`` represents a relation between two vertices, with properties according to the schema defined in its label. + +Edge Fields +--------------- + +The following fields need to be specified when inserting an edge, and are returned when queried on edges. + +.. csv-table:: Option + :header: "Field Name", "Definition", "Data Type", "Example", "Note" + :widths: 15, 30, 30, 30, 30 + + "timestamp", "Issue time of request", "Long", "1430116731156", "Required. Unix Epoch time in milliseconds. S2Graph TTL and timestamp unit is milliseconds." + "operation", "One of insert, delete, update, or increment", "String", "'i', 'insert'", "Required only for bulk operations. Aliases are also available: i (insert), d (delete), u (update), in (increment). Default is insert" + "from", "Id of source vertex", "Long/String", "1", "Required. Use long if possible. Maximum string byte-size is 249" + "to", "Id of target vertex", "Long/String", "101", "Required. Use long if possible. Maximum string byte-size is 249" + "label", "Label name", "String", "graph_test", "Required" + "direction", "Direction of the edge. Should be one of out/ in/ undirected", "String", "out", "Required. Alias are also available: o (out), i (in), u (undirected)" + "props", "Additional properties of the edge", "JSON (dictionary)", "{""timestamp"": 1417616431, ""affinity_score"":10, ""is_hidden"": false, ""is_valid"": true}", "Required. If in indexed properties isn't given, default values will be added" + + +Basic Edge Operations +-------------------------- + +In S2Graph, an Edge supports five different operations. +- ``insert``: Create new edge. +- ``delete``: Delete existing edge. +- ``update``: Update existing edge`s state. +- ``increment``: Increment existing edge`s state. +- ``deleteAll``: Delete all adjacent edges from certain source vertex. (Available for strong consistency only) + +Edge operations work differently depending on the target label`s consistency level. +For a better understanding, please take a look at the following test cases. + +Create 2 different labels, one of each consistencyLevels. + +- s2graph_label_test (strong) +- s2graph_label_test_weak (weak) + +Then insert a same set of edges to each labels and query them as follows. + +**strong consistency** + +.. code:: bash + + curl -XPOST localhosnt:9000/graphs/edges/insert -H 'Content-Type: Application/json' -d ' + [ + {"timestamp": 1, "from": 101, "to": 10, "label": "s2graph_label_test", "props": {"time": 0}}, + {"timestamp": 2, "from": 101, "to": 10, "label": "s2graph_label_test", "props": {"time": -10}}, + {"timestamp": 3, "from": 101, "to": 10, "label": "s2graph_label_test", "props": {"time": -30}} + ]' + + +Note that only one edge exist between (101, 10, s2graph_label_test, out). + + +.. code:: json + + { + "size": 1, + "degrees": [ + { + "from": 101, + "label": "s2graph_label_test", + "direction": "out", + "_degree": 1 + } + ], + "results": [ + { + "cacheRemain": -20, + "from": 101, + "to": 10, + "label": "s2graph_label_test", + "direction": "out", + "_timestamp": 3, + "timestamp": 3, + "score": 1, + "props": { + "_timestamp": 3, + "time": -30, + "weight": 0, + "is_hidden": false, + "is_blocked": false + } + } + ], + "impressionId": -1650835965 + } + +**weak consistency** + +.. code:: bash + + curl -XPOST localhost:9000/graphs/edges/insert -H 'Content-Type: Application/json' -d ' + [ + {"timestamp": 1, "from": 101, "to": 10, "label": "s2graph_label_test_weak", "props": {"time": 0}}, + {"timestamp": 2, "from": 101, "to": 10, "label": "s2graph_label_test_weak", "props": {"time": -10}}, + {"timestamp": 3, "from": 101, "to": 10, "label": "s2graph_label_test_weak", "props": {"time": -30}} + ]' + +This time there are ``three edges`` between (101, 10, s2graph_label_test_weak, out). + +.. code:: json + + { + "size": 3, + "degrees": [ + { + "from": 101, + "label": "s2graph_label_test_weak", + "direction": "out", + "_degree": 3 + } + ], + "results": [ + { + "cacheRemain": -148, + "from": 101, + "to": "10", + "label": "s2graph_label_test_weak", + "direction": "out", + "_timestamp": 3, + "timestamp": 3, + "score": 1, + "props": { + "_timestamp": 3, + "time": -30, + "weight": 0, + "is_hidden": false, + "is_blocked": false + } + }, + { + "cacheRemain": -148, + "from": 101, + "to": "10", + "label": "s2graph_label_test_weak", + "direction": "out", + "_timestamp": 2, + "timestamp": 2, + "score": 1, + "props": { + "_timestamp": 2, + "time": -10, + "weight": 0, + "is_hidden": false, + "is_blocked": false + } + }, + { + "cacheRemain": -148, + "from": 101, + "to": "10", + "label": "s2graph_label_test_weak", + "direction": "out", + "_timestamp": 1, + "timestamp": 1, + "score": 1, + "props": { + "_timestamp": 1, + "time": 0, + "weight": 0, + "is_hidden": false, + "is_blocked": false + } + } + ], + "impressionId": 1972178414 + } + + +Strong Consistency +--------------------- + +**Insert** - ``POST /mutate/edge/insert`` +A unique edge is identified by a combination of (from, to, label, direction). For insert operations, S2Graph first checks if an edge with same (from, to, label, direction) information exists. If there is an existing edge, then insert will work as ``update``. See above example. + +**Delete** - ``POST /mutate/edge/delete`` +For edge deletion, again, S2Graph looks for a unique edge with (from, to, label, direction). However, this time it checks the timestamp of the delete request and the existing edge. The timestamp on the delete request ``must be larger than that on the existing edge`` or else the request will be ignored. If everything is well, the edge will be deleted. Also note that no props information is necessary for a delete request on a strongly consistent label since there will be only one edge with edge`s unique id(from, to, label, direction). + +.. code:: bash + + curl -XPOST localhost:9000/mutate/edge/delete -H 'Content-Type: Application/json' -d ' + [ + {"timestamp": 10, "from": 101, "to": 10, "label": "s2graph_label_test"} + ]' + +**Update** - ``POST /mutate/edge/update`` + +What an update operation does to a strongly consistent label is identical to an insert. + + +.. code:: bash + + curl -XPOST localhost:9000/mutate/edge/update -H 'Content-Type: Application/json' -d ' + [ + {"timestamp": 10, "from": 101, "to": 10, "label": "s2graph_label_test", "props": {"time": 100, "weight": -10}} + ]' + + +**Increment** - ``POST /mutate/edge/increment`` + +Works like update, other than it returns the incremented value and not the old value. + +.. code:: bash + + curl -XPOST localhost:9000/mutate/edge/increment -H 'Content-Type: Application/json' -d ' + [ + {"timestamp": 10, "from": 101, "to": 10, "label": "s2graph_label_test", "props": {"time": 100, "weight": -10}} + ]' + +**Delete All** - ``POST /mutate/edge/deleteAll`` + +Delete all adjacent edges to the source vertex. **Please note that edges with both in and out directions will be deleted** + +.. code:: bash + + curl -XPOST localhost:9000/mutate/edge/deleteAll -H 'Content-Type: Application/json' -d ' + [ + {"ids" : [101], "label":"s2graph_label_test", "direction": "out", "timestamp":1417616441000} + ]' + + +Weak Consistency +----------------- + + +**Insert** ``POST /mutate/edge/insert`` + +S2Graph **does not look** for a unique edge defined by (from, to, label, direction). It simply stores a new edge according to the request. No read, no consistency check. Note that this difference allows multiple edges with same (from, to, label, direction) id. + +**Delete** - ``POST /graphs/edges/delete`` + +For deletion on weakly consistent edges, first, S2Graph fetches existing edges from storage. Then, on each resulting edges, fires the actual delete operations. + +.. code:: bash + + curl -XPOST localhost:9000/graphs/edges/delete -H 'Content-Type: Application/json' -d ' + [ + { + "cacheRemain": -148, + "from": 101, + "to": "10", + "label": "s2graph_label_test_weak", + "direction": "out", + "_timestamp": 3, + "timestamp": 3, + "score": 1, + "props": { + "_timestamp": 3, + "time": -30, + "weight": 0, + "is_hidden": false, + "is_blocked": false + } + }, + { + "cacheRemain": -148, + "from": 101, + "to": "10", + "label": "s2graph_label_test_weak", + "direction": "out", + "_timestamp": 2, + "timestamp": 2, + "score": 1, + "props": { + "_timestamp": 2, + "time": -10, + "weight": 0, + "is_hidden": false, + "is_blocked": false + } + }, + { + "cacheRemain": -148, + "from": 101, + "to": "10", + "label": "s2graph_label_test_weak", + "direction": "out", + "_timestamp": 1, + "timestamp": 1, + "score": 1, + "props": { + "_timestamp": 1, + "time": 0, + "weight": 0, + "is_hidden": false, + "is_blocked": false + } + } + ]' + +**Update** - ``POST /mutate/edge/update`` + +Like insert, S2Graph **does not check** for uniqueness. Update requires a pre-fetch of existing edges, similar to delete. Props of the resulting edges will be updated. + +**Increment** - ``POST /mutate/edge/increment`` + +For increment, S2Graph also **does not check** for uniqueness. Update requires a pre-fetch of existing edges, similar to delete. Props of the resulting edges will be incremented. + +**Delete All** - ``POST /mutate/edge/deleteAll`` + +Identical to strong consistency. + +.. code:: bash + + curl -XPOST localhost:9000/mutate/edge/deleteAll -H 'Content-Type: Application/json' -d ' + [ + {"ids" : [101], "label":"s2graph_label_test", "direction": "out", "timestamp":1417616441} + ]' http://git-wip-us.apache.org/repos/asf/incubator-s2graph/blob/c131990e/doc/source/api/mutate/mutate_vertex.rst ---------------------------------------------------------------------- diff --git a/doc/source/api/mutate/mutate_vertex.rst b/doc/source/api/mutate/mutate_vertex.rst new file mode 100644 index 0000000..06cc61e --- /dev/null +++ b/doc/source/api/mutate/mutate_vertex.rst @@ -0,0 +1,139 @@ +**************************** +Manage Vertices (Optional) +**************************** + +Vertices are the two end points of an edge, and logically stored in columns of a service. If your use case requires storing metadata corresponding to vertices rather than edges, there are operations available on vertices as well. + + +Vertex Fields +---------------- + +Unlike edges and their labels, properties of a vertex are not indexed nor require a predefined schema. The following fields are used when operating on vertices. + + +.. csv-table:: Option + :header: "Field Name", "Definition", "Data Type", "Example", "Note" + :widths: 15, 30, 30, 30, 30 + + "timestamp", "Issue time of request", "Long", "1430116731156", "Required. Unix Epoch time in **milliseconds**" + "operation", "One of insert, delete, update, increment", "String", "i, insert", "Required only for bulk operations. Alias are also available: i (insert), d (delete), u (update), in (increment). Default is insert." + "**serviceName**", "Corresponding service name", "String", "kakaotalk/kakaogroup", "Required" + "**columnName**", "Corresponding column name", "String", "user_id", "Required" + "id", "Unique identifier of vertex", "Long/String", "101", "Required. Use Long if possible" + "**props**", "Additional properties of vertex", "JSON (dictionary)", "{""is_active_user"": true, ""age"":10, ""gender"": ""F"", ""country_iso"": ""kr""}", "Required" + + + + +Basic Vertex Operations +-------------------------- + +Insert - ``POST /mutate/vertex/insert/:serviceName/:columnName`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +.. code:: bash + + curl -XPOST localhost:9000/mutate/vertex/insert/s2graph/account_id -H 'Content-Type: Application/json' -d ' + [ + {"id":1,"props":{"is_active":true, "talk_user_id":10},"timestamp":1417616431000}, + {"id":2,"props":{"is_active":true, "talk_user_id":12},"timestamp":1417616431000}, + {"id":3,"props":{"is_active":false, "talk_user_id":13},"timestamp":1417616431000}, + {"id":4,"props":{"is_active":true, "talk_user_id":14},"timestamp":1417616431000}, + {"id":5,"props":{"is_active":true, "talk_user_id":15},"timestamp":1417616431000} + ]' + + +Delete - ``POST /mutate/vertex/delete/:serviceName/:columnName`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code:: bash + + curl -XPOST localhost:9000/mutate/vertex/delete/s2graph/account_id -H 'Content-Type: Application/json' -d ' + [ + {"id":1,"timestamp":1417616431001}, + {"id":2,"timestamp":1417616431002} + ]' + + +This operation will delete only the vertex data of a specified column and will not delete any edges connected to those vertices. +Also important thing is timestamp. vertex will be deleted only if delete requests ``timestamp is larger than all of vertexs`` property`s timestamp. + +following example shows the difference. + +.. code:: bash + + curl -XPOST localhost:9000/mutate/vertex/insert/s2graph_test/account_id -H 'Content-Type: Application/json' -d ' + [ + {"id":1,"props":{"is_active":true, "talk_user_id":10},"timestamp":1417616431000}, + {"id":1,"props":{"talk_user_id":20},"timestamp":1417616431002} + ]' + +if user request delete(ts) on vertex like below then vertex would not be deleted, but only properties on this vertex that is updated before ts will be deleted. + +.. code:: bash + + curl -XPOST localhost:9000/mutate/vertex/delete/s2graph_test/account_id -H 'Content-Type: Application/json' -d ' + [ + {"id":1,"timestamp":1417616431001} + ]' + + +then result still have vertex with property that is updated with larger timestamp. + + +.. code:: bash + + curl -XPOST localhost:9000/graphs/getVertices -H 'Content-Type: Application/json' -d ' + [ + {"serviceName": "s2graph_test", "columnName": "account_id", "ids": [1]} + ]' + + + # result + { + "serviceName": "s2graph_test", + "columnName": "account_id", + "id": 1, + "props": { + "talk_user_id": 20 + }, + "timestamp": 1417616431002 + } + + +**Important notes** + +.. note:: + This means that edges returned by a query can contain deleted vertices. Clients are responsible for checking validity of the vertices. + +Delete All - ``POST /mutate/vertex/deleteAll/:serviceName/:columnName`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This is a **very expensive** operation. If you're interested in what goes on under the hood, please refer to the following pseudocode: + +.. code:: python + + vertices = vertex list to delete + for vertex in vertices + labels = fetch all labels that this vertex is included. + for label in labels + for index in label.indices + edges = G.read with limit 50K + for edge in edges + edge.delete + + + +The total complexity is O(L L.I) reads + O(L L.I 50K) writes, worst case. **If the vertex you're trying to delete has more than 50K edges, the deletion will not be consistent**. + + +Update - POST /mutate/vertex/insert/:serviceName/:columnName +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Basically update on Vertex is same with insert overwrite so use insert for update. + +Increment +~~~~~~~~~~~ + +Not yet implemented; stay tuned.
