[
https://issues.apache.org/jira/browse/DRILL-4737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-4737:
-------------------------------
Description:
See https://drill.apache.org/docs/starting-drill-in-distributed-mode/
Requires a number of changes to reflect Drill's support of a configuration
directory as specified by:
drillbit.sh --config /path/to/config/dir cmd
"The default memory for a Drillbit is 8G, but Drill prefers 16G" The default
*direct* memory for Drill is 8G. The default total memory for Drill is 12G.
(Included 4G heap.)
"Drillbit startup script located in <drill_installation_directory>/conf/drill-
env.sh." The location is $SITE_DIR/drill-env.sh. $SITE_DIR is either:
1. $DRILL_HOME/conf by default (as stated in the docs), or
2. Specified by the --config <site_dir> option to drillbit.sh
"edit the XX:MaxDirectMemorySize parameter". Please show how to do this. The
correct form (to work with YARN) is:
export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"8G"}
Where the new value replaces the "8G". (This is different than the pre-1.8
form.)
"If this parameter is not set, the limit depends on the amount of available
system memory." This has never turned out to be true as the script always
provides a default value.
Another point: Drill assumes all nodes have the same amount of memory. Relying
on system memory will not, in general, work as some Drillbits (with less system
memory) will die with OOM errors. I suspect this is why a default setting is
always provided.
"After you edit <drill_installation_directory>/conf/drill-env.sh" change to
"After you edit drill-env.sh" to avoid repeating the path.
Further, note that in 1.8, drill-env.sh will become self-documenting: it will
contain example settings and comments for each supported config option. (Thanks
to John O. for that suggestion!) We migth want to mention this information
somewhere...
was:
See https://drill.apache.org/docs/starting-drill-in-distributed-mode/
Requires a number of changes to reflect Drill's support of a configuration
directory as specified by:
drillbit.sh --config /path/to/config/dir cmd
"The default memory for a Drillbit is 8G, but Drill prefers 16G" The default
*direct* memory for Drill is 8G. The default total memory for Drill is 12G.
(Included 4G heap.)
"Drillbit startup script located in <drill_installation_directory>/conf/drill-
env.sh." The location is $SITE_DIR/drill-env.sh. $SITE_DIR is either:
1. $DRILL_HOME/conf by default (as stated in the docs), or
2. Specified by the --config <site_dir> option to drillbit.sh
"edit the XX:MaxDirectMemorySize parameter". Please show how to do this. The
correct form (to work with YARN) is:
export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"8G"}
Where the new value replaces the "8G". (This is different than the pre-1.8
form.)
"If this parameter is not set, the limit depends on the amount of available
system memory." This has never turned out to be true as the script always
provides a default value.
Another point: Drill assumes all nodes have the same amount of memory. Relying
on system memory will not, in general, work as some Drillbits (with less system
memory) will die with OOM errors. I suspect this is why a default setting is
always provided.
"After you edit <drill_installation_directory>/conf/drill-env.sh" change to
"After you edit drill-env.sh" to avoid repeating the path.
> Adjust drill-env.sh instructions to reflect config/site directories
> -------------------------------------------------------------------
>
> Key: DRILL-4737
> URL: https://issues.apache.org/jira/browse/DRILL-4737
> Project: Apache Drill
> Issue Type: Improvement
> Components: Documentation
> Affects Versions: 1.8.0
> Reporter: Paul Rogers
> Priority: Minor
>
> See https://drill.apache.org/docs/starting-drill-in-distributed-mode/
> Requires a number of changes to reflect Drill's support of a configuration
> directory as specified by:
> drillbit.sh --config /path/to/config/dir cmd
> "The default memory for a Drillbit is 8G, but Drill prefers 16G" The default
> *direct* memory for Drill is 8G. The default total memory for Drill is 12G.
> (Included 4G heap.)
> "Drillbit startup script located in <drill_installation_directory>/conf/drill-
> env.sh." The location is $SITE_DIR/drill-env.sh. $SITE_DIR is either:
> 1. $DRILL_HOME/conf by default (as stated in the docs), or
> 2. Specified by the --config <site_dir> option to drillbit.sh
> "edit the XX:MaxDirectMemorySize parameter". Please show how to do this. The
> correct form (to work with YARN) is:
> export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"8G"}
> Where the new value replaces the "8G". (This is different than the pre-1.8
> form.)
> "If this parameter is not set, the limit depends on the amount of available
> system memory." This has never turned out to be true as the script always
> provides a default value.
> Another point: Drill assumes all nodes have the same amount of memory.
> Relying on system memory will not, in general, work as some Drillbits (with
> less system memory) will die with OOM errors. I suspect this is why a default
> setting is always provided.
> "After you edit <drill_installation_directory>/conf/drill-env.sh" change to
> "After you edit drill-env.sh" to avoid repeating the path.
> Further, note that in 1.8, drill-env.sh will become self-documenting: it will
> contain example settings and comments for each supported config option.
> (Thanks to John O. for that suggestion!) We migth want to mention this
> information somewhere...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
