[ https://issues.apache.org/jira/browse/IMPALA-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Abhishek Rawat resolved IMPALA-9697. ------------------------------------ Resolution: Fixed > Support priority based scratch directory selection > --------------------------------------------------- > > Key: IMPALA-9697 > URL: https://issues.apache.org/jira/browse/IMPALA-9697 > Project: IMPALA > Issue Type: Task > Reporter: Abhishek Rawat > Assignee: Abhishek Rawat > Priority: Major > Fix For: Impala 4.0 > > > The `‑‑scratch_dirs` startup flag uses the given scratch directories in a > round robin manner. This may not always be ideal since these directories > could come from different class of storage system volumes having different > performance characteristics (SSD vs HDD, local storage vs network attached > storage, etc.). Giving user an option to configure the priority of their > scratch directories could help them optimize their workload based on their > storage system configuration. > One possible way could be that the user pass the priority as part of the > `–scratch_dirs` startup flag using <directory>:<spill_priority>. The > directories will be selected for spilling based on their priorities and if > multiple directories have the same priority then they will be selected in a > round robin fashion. In the below example, dir1 will be used as a spill > victim until its full and then dir2, dir3, and dir4 will be used in a round > robin fashion. > {code:java} > ‑‑scratch_dirs="dir1:200GB:0, dir2:1024GB:1, dir3:1024GB:1, > dir4:1024GB:1"{code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org