Sandhya Sundaresan created TRAFODION-3268: ---------------------------------------------
Summary: UDF|TMUDF returns various errors regarding dlopen cached_libs file Key: TRAFODION-3268 URL: https://issues.apache.org/jira/browse/TRAFODION-3268 Project: Apache Trafodion Issue Type: Bug Components: sql-general Reporter: Sandhya Sundaresan Assignee: Sandhya Sundaresan Fix For: 2.4 In the latest version, we started to see various random errors returned from queries with UDF or TMUDF. The errors sometimes are 4316 error indicating failures to create the cached_libs directory, sometimes are 11245|11248 errors complaining about 'file too short' with the cached file. Whenever the errors occurred, the actual file appeared to exist in the cached_libs directory on every node. It is a regression introduced presumably because of the UDF|TMUDF cached_libs concept. This problem doesn't always happen. Sometimes the same sequence of statements failed previously would work fine the next time. It's also not clear if this only happens on a multi-node cluster, as the errors were seen on nap035, which happens to be a 4-node cluster. Here are a few examples of these failures seen from the SQL regression run: =========================================================================== SQL>select * from myview v, mytable t +>where (qa_udf_integer(v.a), qa_udf_integer(t.b)) > +>all (select qa_udf_integer(a), qa_udf_integer(b) from mytable2 +>where qa_udf_integer(a) = qa_udf_integer(b)); *** ERROR[4316] An error was detected while extracting a udr routine library to local cache : Error 0 creating directory :/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/MYTEST/ . =========================================================================== SQL>select * from UDF(MYTMFUNC(TABLE(select * from trafodion.zzz_single_stream_features_namespace.mytable), 'TEST_CMPTIME_RUNTIME_OUT_TABLE_DEFINED_USING_PASSTHRU', '3')) t1, UDF(MYTMFUNC(TABLE(select * from trafodion.zzz_single_stream_features_namespace_myns.mytable), 'TEST_CMPTIME_RUNTIME_OUT_TABLE_DEFINED_USING_PASSTHRU', '3')) t2, UDF(MYTMFUNC(TABLE(select * from trafodion.zzz_single_stream_features_namespace_myns.mytable_too), 'TEST_CMPTIME_RUNTIME_OUT_TABLE_DEFINED_USING_PASSTHRU', '3')) t3 where t1.a = t2.a and t2.a = t3.a order by 1; *** ERROR[11245] An error occurred loading library 'qaTmudfTest_212414198362332378.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/ZZZ_SINGLE_STREAM_FEATURES_NAMESPACE_MYNS'. [2019-01-14 00:50:22] *** ERROR[11248] A call to dlopen returned errors 0 and 0. Details: /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/ZZZ_SINGLE_STREAM_FEATURES_NAMESPACE_MYNS/qaTmudfTest_212414198362332378.so: file too short. [2019-01-14 00:50:22] =========================================================================== SQL>select * from myview v, mytable t where (qa_udf_integer(v.a), qa_udf_integer(t.b)) > all (select qa_udf_integer(a), qa_udf_integer(b) from mytable2 where qa_udf_integer(a) = qa_udf_integer(b)); *** ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07] *** ERROR[11248] A call to dlopen returned errors 0 and 0. Details: /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07] *** ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07] *** ERROR[11248] A call to dlopen returned errors 0 and 0. Details: /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07] *** ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07] *** ERROR[11248] A call to dlopen returned errors 0 and 0. Details: /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07] *** ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07] *** ERROR[11248] A call to dlopen returned errors 0 and 0. Details: /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07] *** ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07] *** ERROR[11248] A call to dlopen returned errors 0 and 0. Details: /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07] *** ERROR[11245] An error occurred loading library 'qaUdfTest_212414083371782103.so' from path '/opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL'. [2019-01-12 15:20:07] *** ERROR[11248] A call to dlopen returned errors 0 and 0. Details: /opt/trafodion/esgyndb/udr/DB__ROOT/cached_libs/UDF_UDF_GENERAL/qaUdfTest_212414083371782103.so: file too short. [2019-01-12 15:20:07] -- This message was sent by Atlassian JIRA (v7.6.3#76005)