Thanks Ryan, for such a thorough response! On Thu, Jan 11, 2018 at 5:32 PM, Ryan Anderson <rander...@fargeo.com> wrote:
> Hi Josh, > > No problem, thanks for reaching out on the forum. We're here to help. > > json import of business data is primarily intended for transferring > business data between arches instances. Because of this it's not as user > friend to create or interpret, but not impossible.That being said, I'm > happy to go through some of the basics of the json format below. > > First, there are at least two ways you can familiarize yourself with the > format. The system settings in an Arches package is stored in this json > format, you can open one of those up and take a look. Perhaps a better way > in your case is to create some business data via the ui in your instance of > arches and export it to the json format using the business data export > command defined here https://github.com/archesproject/arches/wiki/ > Command-Line-Reference#export-commands. This can act as a template json > for data creation. For the next section it would be nice to have one of > these json files open to make it easier to follow along. > > General structure of the json file: > { > “business_data”: { > “resources”: [ > { > “resourceinstance”: {. . .}, > “tiles”: [. . .], > } > ] > } > } > > The json format is primarily a representation of the tiles table in the > arches postgres database with some information about the resource > instance(s) included. Within the business_data object of the json are two > objects, the tiles object and the resourceinstance object. Let's start with > the resource instance object. > Resource Instance Object - the resource instance object contains three > pieces of data to describe the instance: > graph_id - the id of the resource model for which this data was created > resourceinstanceid - the unique identifier of this resource instance > within Arches (this will need to be unique for every resource in Arches) > legacyid - an identifier that was used for this resource before it"s > inclusion in Arches. This can be the same as the resourceinstanceid (this > is the case when you provide a UUID to the ResourceID column in a csv) or > it can be another id. Either way it has to be unique among every resource > in Arches. > “resourceinstance”: { > “graph_id”: uuid, > “resourceinstanceid”: uuid, > “legacyid”: uuid or text > } > > Tiles Object - a list of tiles that comprise a resource instance. The tile > object is a bit more complicated and the structure can vary a little > depending on the cardinality of your nodes. I'll cover six different > cardinality scenarios: > 1.) 1 card > 2.) n cards > 3.) 1 parent card with 1 child card > 4.) 1 parent card with n child cards > 5.) n parent cards with 1 child card > 6.) n parent cards with n child cards > > But first a little about the general structure of a tile. > > General Structure of a tile: > { > “tileid”: uuid, > “resourceinstance_id”: uuid, > “nodegroupid”: uuid, > “sortorder”: 0, > “parenttile_id": uuid or null, > “data” {. . .} > } > > tileid - unique identifier of the tile this is the primary key in the > tiles table and must be a unique uuid > resourceinstance_id - the uuid corresponding to the instance this tile > belongs to (this should be the same as the resourceinstance_id from the > resourceinstance object. > nodegroup_id - the node group for which the nodes within the data array > participate > sortorder - the sort order of this data in the form/report relative to > other tiles (only applicable if cardinality is n) > parenttile_id - unique identifier of the parenttile of this tile (will be > null if this is a parent tile or the tile has no parent) > data - json structure of a node group including the nodeid and data > populating that node > > The tile object is tied to a resource model in two ways: one is through > the nodegroup_id the other is in the data object where nodeids are used as > keys for the business data itself. > > *1*: There is one and only one instance of this nodegroup/card in a > resource. > > { > “tileid”: unique uuid, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” { > “nodeid”: "some data", > “nodeid”: "some other data" > } > } > This structure represents a tile for a nodegroup(consisting of two nodes) > with no parents collecting data with a cardinality of 1 > > *n*: There are multiple instances of this nodegroup/card in a resource. > > { > “tileid”: unique uuid, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” { > “nodeid”: "some data", > “nodeid”: "some other data" > } > }, > { > “tileid”: unique uuid, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” { > “nodeid”: "more data", > “nodeid”: "more other data" > } > } > > > Note: From here on out I will use simplified tileids for ease of > understanding. > > *1-1*: One and only one parent nodegroup/card contains one and only one > child nodegroup/card. > { > “tileid”: *A*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *A*, > “data” { > “nodeid”: "data", > “nodeid”: "other data" > } > } > > *1-n*: One and only one parent nodegroup/card containing multiple > instances of child nodegroups/cards. > > { > “tileid”:* A*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *A*, > “data” { > “nodeid”: "data", > “nodeid”: "other data" > } > }, > { > “tileid”: Y, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id":* A*, > “data” { > “nodeid”: "more data", > “nodeid”: "more other data" > } > } > > *n-1*: Many parent nodegroups/cards each with one child nodegroup/card. > > { > “tileid”: *A*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *A*, > “data” { > “nodeid”: "data", > “nodeid”: "other data" > } > }, > > { > “tileid”: *B*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *B*, > “data” { > “nodeid”: "more data", > “nodeid”: "more other data" > } > } > > *n-n*: Many parent nodegroups/cards containing many child nodegroups/cards > > { > “tileid”: *A*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: X, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *A*, > “data” { > “nodeid”: "data", > “nodeid”: "other data" > } > }, > { > “tileid”: *B*, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": null, > “data” {} > }, > { > “tileid”: Y, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *B*, > “data” { > “nodeid”: "more data", > “nodeid”: "more other data" > } > }, > { > “tileid”: Z, > “resourceinstance_id”: uuid from resourceinstance.resourceinstanceid, > “nodegroupid”: uuid from resource model, > “sortorder”: 0, > “parenttile_id": *B*, > “data” { > “nodeid”: "even more data", > “nodeid”: even "more other data" > } > } > > > > > > On Thursday, January 11, 2018 at 10:19:02 AM UTC-8, Joshua Gomez wrote: >> >> Thanks for your help Ryan. I was not aware we could import business data >> in a JSON format. I don't see that in the documentation. How can we learn >> the proper structure for the import document? >> >> -Josh >> >> On Wednesday, January 10, 2018 at 2:17:25 PM UTC-8, Ryan Anderson wrote: >>> >>> Hey Guys, >>> >>> I was able to fix your issue by setting the cardinality of the 'Person >>> Appellation' card and the ‘Part' card to n. This imports all the data you >>> are looking for but in different Person Appellation cards. >>> >>> It looks like you have run into one of the limitations to csv import. >>> When importing a card/node group with cardinality n and your data spans >>> multiple columns (Part Value and Part Type), you do need to separate your >>> data into separate rows (check) but if your card/node group is nested >>> within another card/node group (Part within Person Appellation in this >>> case), the csv importer does not know that you would only like to create >>> multiple Part cards and not multiple Person Appellation cards. >>> >>> It is possible to import all the Part cards/node groups to one Person >>> Appellation card/node group via json import. >>> >>> >>> Regards, >>> Ryan >>> >>> >>> On Jan 10, 2018, at 11:26 AM, Nathan Lomeli <nathan...@gmail.com> wrote: >>> >>> Here's a Dropbox link to the files mentioned below: >>> >>> https://www.dropbox.com/sh/1wx9x4kdk5b7a33/AACqzIROyoT2R-03l >>> AgXE-qta?dl=0 >>> >>> >>> We are performing a CSV resources import (for our "Person" model) using >>> the included mapping file. Everything seems to be importing properly >>> except for this pair of columns: >>> >>> *name_part_type name_part_value* >>> >>> In accordance with the CSV import formatting described in the Arches >>> documention (link below), each new concept type and value appears on a new >>> row. >>> >>> http://arches4.readthedocs.io/en/latest/import-export/#conce >>> pt-values-in-csvs-and-the-concepts-file >>> >>> The import process seems to run successfully, but when we inspect the >>> imported data, only the first concept-value pair has been loaded, not those >>> on subsequent CSV rows for a given resource. >>> >>> We are not sure if this failure to load a list of concept-values is >>> problem with the graph, our card settings, our import process, or the >>> formatting of our CSV or mapping files. >>> >>> Our Arches graph data is included in order for the problem to be >>> reproduced (again, see link to Dropbox above). >>> >>> >>> -- >>> -- To post, send email to arches...@googlegroups.com. To unsubscribe, >>> send email to archesprojec...@googlegroups.com. For more information, >>> visit https://groups.google.com/d/forum/archesproject?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Arches Project" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to archesprojec...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- > -- To post, send email to archesproject@googlegroups.com. To unsubscribe, > send email to archesproject+unsubscr...@googlegroups.com. For more > information, visit https://groups.google.com/d/forum/archesproject?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Arches Project" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/archesproject/c8akHyjMLX0/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > archesproject+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- -- To post, send email to archesproject@googlegroups.com. To unsubscribe, send email to archesproject+unsubscr...@googlegroups.com. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to archesproject+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
