Hi guys, We would like to use spark hadoop api to get the first couple hundred lines in design time to quickly show users the file-structure/meta data, and the values in those lines without launching the full spark job in cluster.
Since we're web-based application, there will be multiple users using the spark hadoop api, for exmaple, sc.textFile(filePath). I wonder if those APIs are thread-safe in local mode (each user will have its own SparkContext object). Secondly, it seems that even in local mode, the jetty UI tracker will be lunched. For this kind of cheap operation, having jetty UI tracker for each operation will be very expensive. Is there a way to disable this behavior? Thanks. Sincerely, DB Tsai ------------------------------------------------------- My Blog: https://www.dbtsai.com LinkedIn: https://www.linkedin.com/in/dbtsai
