Should that not cause more problems?
Regards,
Gourav Sengupta
On Wed, Feb 27, 2019 at 7:36 PM SRK wrote:
>
> Hi,
>
> We are using something like the following to write data to files in
> Structured Streaming and we seem to get file names as part* as mentioned in
>
> https://stackoverflow.com/questions/51056764/how-to-define-a-spark-structured-streaming-file-sink-file-path-or-file-name.
>
>
> How to get file names of our choice for each row in the dataframe? Like say
> /day/month/id/log.txt?
>
>
> df.writeStream
> .format("parquet") // can be "orc", "json", "csv", etc.
> .option("path", "/path/to/save/")
> .partitionBy("year", "month", "day", "hour")
> .start()
>
> Thanks for the help!!!
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>
