[
https://issues.apache.org/jira/browse/S2GRAPH-197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423407#comment-16423407
]
ASF GitHub Bot commented on S2GRAPH-197:
----------------------------------------
Github user elric-k commented on a diff in the pull request:
https://github.com/apache/incubator-s2graph/pull/151#discussion_r178703067
--- Diff: s2jobs/src/main/scala/org/apache/s2graph/s2jobs/task/Sink.scala
---
@@ -182,14 +200,40 @@ class ESSink(queryName:String, conf:TaskConf) extends
Sink(queryName, conf) {
/**
* S2graphSink
+ *
* @param queryName
* @param conf
*/
-class S2graphSink(queryName:String, conf:TaskConf) extends Sink(queryName,
conf) {
+class S2graphSink(queryName: String, conf: TaskConf) extends
Sink(queryName, conf) {
override def mandatoryOptions: Set[String] = Set()
+
override val FORMAT: String =
"org.apache.s2graph.spark.sql.streaming.S2SinkProvider"
- override protected def writeBatch(writer: DataFrameWriter[Row]): Unit =
- throw new RuntimeException(s"unsupported source type for
${this.getClass.getSimpleName} : ${conf.name}")
+ private def bulkload(df: DataFrame): Unit = {
+ val options = S2GraphHelper.toGraphFileOptions(conf)
--- End diff --
*toGraphFileOptions* method could handle only GraphFileOptions's parameters.
If we pass other parameters it will throw exception follows:
```
failed to parse options...
java.lang.IllegalArgumentException: failed to parse options...
at
org.apache.s2graph.s2jobs.loader.GraphFileOptions$.toOption(GraphFileOptions.scala:82)
```
> Provide S2graphSink for non-streaming dataset
> ---------------------------------------------
>
> Key: S2GRAPH-197
> URL: https://issues.apache.org/jira/browse/S2GRAPH-197
> Project: S2Graph
> Issue Type: Sub-task
> Components: s2jobs
> Reporter: Chul Kang
> Assignee: Chul Kang
> Priority: Major
>
> Currently, S2graphSink supports sink operation for spark structured streaming
> that is only for StreamingQuery.
> If we provide the same operation for the DataframeWriter in S2graphSink, we
> could use it in batch mode.
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
