Hi, I found this ( https://spark.apache.org/docs/latest/api/java/org/apache/spark/util/SizeEstimator.html) that may be helpful, i use Java:
> org.apache.spark.util.SizeEstimator.estimate(dataset)); Le sam. 18 juin 2022 à 22:33, mbreuer <msbre...@gmail.com> a écrit : > Hello Community, > > I am working on optimizations for file sizes and number of files. In the > data frame there is a function input_file_name which returns the file > name. I miss a counterpart to get the size of the file. Just the size, > like "ls -l" returns. Is there something like that? > > Kind regards, > Markus > > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
