[
https://issues.apache.org/jira/browse/IMPALA-6983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480154#comment-16480154
]
ASF subversion and git services commented on IMPALA-6983:
---------------------------------------------------------
Commit 6daf9800c07e6d291409dcf1abf06f3f97b85e9e in impala's branch
refs/heads/2.x from [~mikesbrown]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=6daf980 ]
IMPALA-6983: stress: don't write a null runtime profile
This patch fixes a problem in which the writing of profiles always
assumed profiles would be collected during binary. That's not the case
when queries are too expensive to run. The simple fix is to just check
for None and not perform the write.
Change-Id: Ic8299a8a97ad1f2bd1f2927e3111db8df1d3a3e5
Reviewed-on: http://gerrit.cloudera.org:8080/10381
Reviewed-by: Michael Brown <mi...@cloudera.com>
Tested-by: Michael Brown <mi...@cloudera.com>
> stress test binary search exits if process mem_limit is too low
> ---------------------------------------------------------------
>
> Key: IMPALA-6983
> URL: https://issues.apache.org/jira/browse/IMPALA-6983
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Reporter: Dan Hecht
> Assignee: Michael Brown
> Priority: Major
> Fix For: Impala 3.1.0
>
>
> This was running stress test on tpch SF=20 and minicluster process
> mem_limit=7857355161.
> {code:java}
> 2018-05-04 18:25:03,800 18531 MainThread
> INFO:concurrent_select[1303]:Collecting runtime info for query q5:
> select
> n_name,
> sum(l_extendedprice * (1 - l_discount)) as revenue
> from
> customer,
> orders,
> lineitem,
> supplier,
> nation,
> region
> where
> c_custkey = o_custkey
> and l_orderkey = o_orderkey
> and l_suppkey = s_suppkey
> and c_nationkey = s_nationkey
> and s_nationkey = n_nationkey
> and n_regionkey = r_regionkey
> and r_name = 'ASIA'
> and o_orderdate >= '1994-01-01'
> and o_orderdate < '1995-01-01'
> group by
> n_name
> order by
> revenue desc
> 2018-05-04 18:25:07,790 18531 MainThread INFO:concurrent_select[1406]:Finding
> a starting point for binary search
> 2018-05-04 18:25:07,790 18531 MainThread INFO:concurrent_select[1409]:Next
> mem_limit: 7493
> 2018-05-04 18:28:06,380 18531 MainThread
> WARNING:concurrent_select[1416]:Query couldn't be run even when using all
> available memory
> select
> n_name,
> sum(l_extendedprice * (1 - l_discount)) as revenue
> from
> customer,
> orders,
> lineitem,
> supplier,
> nation,
> region
> where
> c_custkey = o_custkey
> and l_orderkey = o_orderkey
> and l_suppkey = s_suppkey
> and c_nationkey = s_nationkey
> and s_nationkey = n_nationkey
> and n_regionkey = r_regionkey
> and r_name = 'ASIA'
> and o_orderdate >= '1994-01-01'
> and o_orderdate < '1995-01-01'
> group by
> n_name
> order by
> revenue desc
> Traceback (most recent call last):
> File "./tests/stress/concurrent_select.py", line 2265, in <module>
> main()
> File "./tests/stress/concurrent_select.py", line 2162, in main
> queries, impala, converted_args,
> queries_with_runtime_info_by_db_sql_and_options)
> File "./tests/stress/concurrent_select.py", line 1879, in populate_all_queries
> os.path.join(converted_args.results_dir, PROFILES_DIR))
> File "./tests/stress/concurrent_select.py", line 964, in
> write_runtime_info_profiles
> fh.write(profile)
> TypeError: expected a string or other character buffer object{code}
> I don't understand the details of {{concurrent_select.py}} control flow, but
> it looks like in this case {{update_runtime_info()}} won't get called leading
> to this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
