GitHub user wangzhewwzz added a comment to the discussion: 关于电网拓扑数据导入
{
"vertices": [
{
"label": "Substation",
"input": {
"type": "file",
"path":
"/loader/power_data/vertex_substation-1.csv",
"format": "CSV",
"header": ["id", "name", "objectType",
"objectHandle", "deviceType", "internalEndpointNo", "terminalNo", "usage",
"powerFlowDirection", "belongSubstation", "belongFeeder"],
"charset": "UTF-8"
},
"null_values": ["NULL", "null", ""]
},
{
"label": "LineSegment",
"input": {
"type": "file",
"path":
"/loader/power_data/vertex_linesegment-1.csv",
"format": "CSV",
"header": ["id", "name", "objectType",
"objectHandle", "deviceType", "internalEndpointNo", "terminalNo", "usage",
"powerFlowDirection", "belongSubstation", "belongFeeder"],
"charset": "UTF-8"
},
"null_values": ["NULL", "null", ""]
},
{
"label": "LineSwitch",
"input": {
"type": "file",
"path":
"/loader/power_data/vertex_lineswitch-1.csv",
"format": "CSV",
"header": ["id", "name", "objectType",
"objectHandle", "deviceType", "internalEndpointNo", "terminalNo", "usage",
"powerFlowDirection", "belongSubstation", "belongFeeder"],
"charset": "UTF-8"
},
"null_values": ["NULL", "null", ""]
},
{
"label": "StationHouse",
"input": {
"type": "file",
"path":
"/loader/power_data/vertex_stationhouse-1.csv",
"format": "CSV",
"header": ["id", "name", "objectType",
"objectHandle", "deviceType", "internalEndpointNo", "terminalNo", "usage",
"powerFlowDirection", "belongSubstation", "belongFeeder"],
"charset": "UTF-8"
},
"null_values": ["NULL", "null", ""]
}
],
"edges": [
{
"label": "Substation2LineSegment",
"source": ["from"],
"target": ["to"],
"input": {
"type": "file",
"path": "/loader/power_data/edge_connected_to.csv",
"format": "CSV",
"header": ["id", "from", "to",
"label","source_label","target_label"]
},
"field_mapping": {
"from": "id",
"to": "id"
},
"selected": ["from", "to"],
"value_mapping": {
"label": { "CONNECTED_TO": "Substation2LineSegment"
}
}
},
{
"label": "LineSegment2Substation",
"source": ["from"],
"target": ["to"],
"input": {
"type": "file",
"path": "/loader/power_data/edge_connected_to.csv",
"format": "CSV",
"header": ["id", "from", "to",
"label","source_label","target_label"]
},
"field_mapping": {
"from": "id",
"to": "id"
},
"selected": ["from", "to"],
"value_mapping": {
"label": { "CONNECTED_TO": "LineSegment2Substation"
}
}
},
{
"label": "LineSegment2StationHouse",
"source": ["from"],
"target": ["to"],
"input": {
"type": "file",
"path": "/loader/power_data/edge_connected_to.csv",
"format": "CSV",
"header": ["id", "from", "to",
"label","source_label","target_label"]
},
"field_mapping": {
"from": "id",
"to": "id"
},
"selected": ["from", "to"],
"value_mapping": {
"label": { "CONNECTED_TO":
"LineSegment2StationHouse" }
}
},
{
"label": "StationHouse2LineSegment",
"source": ["from"],
"target": ["to"],
"input": {
"type": "file",
"path": "/loader/power_data/edge_connected_to.csv",
"format": "CSV",
"header": ["id", "from", "to",
"label","source_label","target_label"]
},
"field_mapping": {
"from": "id",
"to": "id"
},
"selected": ["from", "to"],
"value_mapping": {
"label": { "CONNECTED_TO":
"StationHouse2LineSegment" }
}
},
{
"label": "LineSegment2LineSegment",
"source": ["from"],
"target": ["to"],
"input": {
"type": "file",
"path": "/loader/power_data/edge_connected_to.csv",
"format": "CSV",
"header": ["id", "from", "to",
"label","source_label","target_label"]
},
"field_mapping": {
"from": "id",
"to": "id"
},
"selected": ["from", "to"],
"value_mapping": {
"label": { "CONNECTED_TO":
"LineSegment2LineSegment" }
}
},
{
"label": "LineSegment2LineSwitch",
"source": ["from"],
"target": ["to"],
"input": {
"type": "file",
"path": "/loader/power_data/edge_connected_to.csv",
"format": "CSV",
"header": ["id", "from", "to",
"label","source_label","target_label"]
},
"field_mapping": {
"from": "id",
"to": "id"
},
"selected": ["from", "to"],
"value_mapping": {
"label": { "CONNECTED_TO": "LineSegment2LineSwitch"
}
}
},
{
"label": "LineSwitch2LineSegment",
"source": ["from"],
"target": ["to"],
"input": {
"type": "file",
"path": "/loader/power_data/edge_connected_to.csv",
"format": "CSV",
"header": ["id", "from", "to",
"label","source_label","target_label"]
},
"field_mapping": {
"from": "id",
"to": "id"
},
"selected": ["from", "to"],
"value_mapping": {
"label": { "CONNECTED_TO": "LineSwitch2LineSegment"
}
}
}
]
}
// 属性定义
schema.propertyKey("id").asText().ifNotExist().create();
schema.propertyKey("name").asText().ifNotExist().create();
schema.propertyKey("objectType").asText().ifNotExist().create();
schema.propertyKey("objectHandle").asText().ifNotExist().create();
schema.propertyKey("deviceType").asText().ifNotExist().create();
schema.propertyKey("internalEndpointNo").asInt().ifNotExist().create();
schema.propertyKey("terminalNo").asInt().ifNotExist().create();
schema.propertyKey("usage").asText().ifNotExist().create();
schema.propertyKey("powerFlowDirection").asInt().ifNotExist().create();
schema.propertyKey("belongSubstation").asText().ifNotExist().create();
schema.propertyKey("belongFeeder").asText().ifNotExist().create();
schema.propertyKey("source_label").asText().ifNotExist().create();
schema.propertyKey("target_label").asText().ifNotExist().create();
// 顶点标签
schema.vertexLabel("Substation").properties("id", "name", "objectType",
"objectHandle", "deviceType", "internalEndpointNo", "terminalNo", "usage",
"powerFlowDirection", "belongSubstation",
"belongFeeder").primaryKeys("id").ifNotExist().create();
schema.vertexLabel("LineSegment").properties("id", "name", "objectType",
"objectHandle", "deviceType", "internalEndpointNo", "terminalNo", "usage",
"powerFlowDirection", "belongSubstation",
"belongFeeder").primaryKeys("id").ifNotExist().create();
schema.vertexLabel("LineSwitch").properties("id", "name", "objectType",
"objectHandle", "deviceType", "internalEndpointNo", "terminalNo", "usage",
"powerFlowDirection", "belongSubstation",
"belongFeeder").primaryKeys("id").ifNotExist().create();
schema.vertexLabel("StationHouse").properties("id", "name", "objectType",
"objectHandle", "deviceType", "internalEndpointNo", "terminalNo", "usage",
"powerFlowDirection", "belongSubstation",
"belongFeeder").primaryKeys("id").ifNotExist().create();
// 边标签（每种类型对每种类型都定义一条边，便于扩展和双向连接）
schema.edgeLabel("Substation2LineSegment").sourceLabel("Substation").targetLabel("LineSegment").properties().ifNotExist().create();
schema.edgeLabel("LineSegment2StationHouse").sourceLabel("LineSegment").targetLabel("StationHouse").properties().ifNotExist().create();
schema.edgeLabel("LineSegment2LineSegment").sourceLabel("LineSegment").targetLabel("LineSegment").properties().ifNotExist().create();
//schema.edgeLabel("StationHouse2LineSwitch").sourceLabel("StationHouse").targetLabel("LineSwitch").properties().ifNotExist().create();
schema.edgeLabel("LineSegment2LineSwitch").sourceLabel("LineSegment").targetLabel("LineSwitch").properties().ifNotExist().create();
// ...根据实际需要继续补充
// 反向边（如需双向，定义反向边标签）
schema.edgeLabel("LineSegment2Substation").sourceLabel("LineSegment").targetLabel("Substation").properties().ifNotExist().create();
schema.edgeLabel("StationHouse2LineSegment").sourceLabel("StationHouse").targetLabel("LineSegment").properties().ifNotExist().create();
//schema.edgeLabel("LineSegment2LineSegment").sourceLabel("LineSegment").targetLabel("LineSegment").properties().ifNotExist().create();
//schema.edgeLabel("LineSwitch2StationHouse").sourceLabel("LineSwitch").targetLabel("StationHouse").properties().ifNotExist().create();
schema.edgeLabel("LineSwitch2LineSegment").sourceLabel("LineSwitch").targetLabel("LineSegment").properties().ifNotExist().create();
// ...以此类推
(base) PS C:\Users\GH-76\hugegraph-1.7.0> docker exec -it hugegraph-loader
bin/hugegraph-loader.sh -g hugegraph -f /loader/power_data/struct_1.json -s
/loader/power_data/schema_1.groovy -h server -p 8080 --clear-all-data true
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/loader/lib/log4j-slf4j-impl-2.18.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/loader/lib/apache-hugegraph-loader-incubating-1.7.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/loader/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will
impact performance.
ERROR StatusLogger Reconfiguration failed: No configuration found for 'Default'
at 'null' in 'null'
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by
org.codehaus.groovy.reflection.CachedClass
(file:/loader/lib/apache-hugegraph-loader-incubating-1.7.0-shaded.jar) to
method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of
org.codehaus.groovy.reflection.CachedClass
WARNING: Use --illegal-access=warn to enable warnings of further illegal
reflective access operations
WARNING: All illegal access operations will be denied in a future release
>> HugeGraphLoader worked in NORMAL MODE
vertices/edges loaded this time : 193/1351
--------------------------------------------------
count metrics
input read success : 386
input read failure : 0
vertex parse success : 193
vertex parse failure : 0
vertex insert success : 193
vertex insert failure : 0
edge parse success : 1351
edge parse failure : 0
edge insert success : 1351
edge insert failure : 0
--------------------------------------------------
meter metrics
total time
: 0.331s
read time
: 0.111s
load time
: 0.22s
vertex load time
: 0.134s
vertex load rate(vertices/s) : 1440
edge load time
: 0.086s
edge load rate(edges/s) : 15709
1.为什么边是1300多个
2.为什么变电站没有连接关系
3.为什么有的线路就能建立连接关系 有点没有？
原始邮件
发件人：dosubot[bot] ***@***.***>
发件时间：2025年12月10日 15:09
收件人：apache/incubator-hugegraph ***@***.***>
抄送：wangzhewwzz ***@***.***>, Mention ***@***.***>
主题：Re: [apache/incubator-hugegraph] 关于电网拓扑数据导入 (Discussion #2917)
你的配置现在只差一个关键点就能导入成功，但你还遗漏了最核心的地方：
你的 struct.json 里所有 edges 的 field_mapping 必须写成：
"field_mapping": { "from": "from", "to": "to" }
而且 不能写成 { "from": "id", "to": "id" } 或 { "from": "id", "to": "id", ...
}，否则 loader 会把 CSV 的 id 列当作边属性导入，但 schema.groovy 的 edgeLabel 没有声明 id 属性，导致
“Invalid property 'id' for edge label” 或 “主键字段 [id] 为空” 的报错。官方文档说明
你现在遇到的 “Make sure the primary key fields [id] are not empty, or check whether
the headers or field_mapping are configured correctly” 报错，根本原因是：
你的顶点主键是 id（vertex_xxx.csv 里有 id 列，schema 里 primaryKeys("id")），loader 只会用
from/to 字段的值直接作为顶点主键 id，不会再去查 vertex_xxx.csv 的其他字段。
edge_connected_to.csv 的 id 列可以保留，但 loader 不会导入它为边属性，只要 field_mapping 不写 id
就不会报错。
edgeLabel 的 .properties() 只包含 "source_label", "target_label"，不要加 "id"。
value_mapping 必须是平铺结构（你现在的写法没问题）。
null_values 建议只保留 ["NULL", "null"]，不要加 ""，否则空字符串会被判定为 null。
请你务必检查：
struct.json 里所有 edges 的 field_mapping 都是 "field_mapping": { "from": "from",
"to": "to" }，没有任何 id。
edge_connected_to.csv 的 from/to 字段和所有顶点 id 完全一致（无多余空格、无大小写问题、无空行）。
struct.json 的 null_values 只保留 ["NULL", "null"]，不要加 ""。
edgeLabel 的 .properties() 只包含 "source_label", "target_label"，不要加 "id"。
你可以加 "selected": ["from", "to"] 显式指定用到的字段，避免 loader 解析其它无关列。
如果还有报错，请贴出 power_edges_to.csv 的 header 和前几行原始数据，以及 struct.json 某个 edges
配置的完整内容，我可以帮你定位具体是哪一行或哪一列有问题。
只要这几点全部做到，边就能正常导入且无报错，所有关系都能导入。官方FAQ说明
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose |
Hallucination | Report 🐛 | Other
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.
GitHub link:
https://github.com/apache/incubator-hugegraph/discussions/2917#discussioncomment-15217518
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
