[
https://issues.apache.org/jira/browse/CALCITE-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobo Liao updated CALCITE-2950:
---------------------------------
Description:
Kylin's JDBC
Driver([https://github.com/apache/kylin/tree/master/jdbc)|https://github.com/apache/kylin/tree/master/jdbc),]
is based on Avatica. While using Kylin JDBC Driver, if repeatedly invokes
DriverVersion.load, native memory leak can be observed, see attached memory
profile diagram(from jeprof). Though it is nonsense to calling
DriverVersion.load repeatedly, but the leak is there.
{{
package org.apache.calcite.avatica.test;
import org.apache.calcite.avatica.DriverVersion;
import org.apache.calcite.avatica.remote.Driver;
/**
* This class demonstrates the native memory leak caused by DriverVersion.load,
* which has not closed the opened InputStream.
*/
public class DriverVersionTest {
public static void main(String[] args) throws Exception {
while (true)
{ DriverVersion version = DriverVersion.load(Driver.class,
"org-apache-kylin-jdbc.properties", "Kylin JDBC Driver", "unknown version",
"Kylin", "unknown version"); System.out.println(version);
Thread.sleep(100l); }
}
}}
By using jemalloc's jeprof utility, the leak is caused by below stack trace
at java.util.zip.Inflater.inflate()
at java.util.zip.InflaterInputStream.read()
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at java.util.Properties$LineReader.readLine(Properties.java:435)
at java.util.Properties.load0(Properties.java:353)
at java.util.Properties.load(Properties.java:341)
at
[org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.DriverVersion.load(DriverVersion.java:104|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.driverversion.load%28driverversion.java:104/])
at org.apache.kylin.jdbc.Driver.createDriverVersion(Driver.java:88)
at
[org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.UnregisteredDriver|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.unregistereddriver/].<init>(UnregisteredDriver.java:56)
at org.apache.kylin.jdbc.Driver.<init>(Driver.java:70)
By inspecting the code, the InputStream was not closed after use.
{code:java}
final InputStream inStream =
driverClass.getClassLoader().getResourceAsStream(resourceName);
{code}
The code to reproduce the leak is at
[https://github.com/leonliao/calcite-avatica-driver-version-test]. Follow the
README to reproduce the problem.
was:
Kylin's JDBC
Driver([https://github.com/apache/kylin/tree/master/jdbc)|https://github.com/apache/kylin/tree/master/jdbc),]
is based on Avatica. While using Kylin JDBC Driver, if repeatedly invokes
DriverVersion.load, native memory leak can be observed, see attached memory
profile diagram(from jeprof). Though it is nonsense to calling
DriverVersion.load repeatedly, but the leak is there.
package org.apache.calcite.avatica.test;
import org.apache.calcite.avatica.DriverVersion;
import org.apache.calcite.avatica.remote.Driver;
/**
* This class demonstrates the native memory leak caused by DriverVersion.load,
* which has not closed the opened InputStream.
*/
public class DriverVersionTest {
public static void main(String[] args) throws Exception {
while (true)
{ DriverVersion version = DriverVersion.load(Driver.class,
"org-apache-kylin-jdbc.properties", "Kylin JDBC Driver", "unknown version",
"Kylin", "unknown version"); System.out.println(version);
Thread.sleep(100l); }
}
By using jemalloc's jeprof utility, the leak is caused by below stack trace
at java.util.zip.Inflater.inflate()
at java.util.zip.InflaterInputStream.read()
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at java.util.Properties$LineReader.readLine(Properties.java:435)
at java.util.Properties.load0(Properties.java:353)
at java.util.Properties.load(Properties.java:341)
at
[org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.DriverVersion.load(DriverVersion.java:104|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.driverversion.load%28driverversion.java:104/])
at org.apache.kylin.jdbc.Driver.createDriverVersion(Driver.java:88)
at
[org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.UnregisteredDriver|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.unregistereddriver/].<init>(UnregisteredDriver.java:56)
at org.apache.kylin.jdbc.Driver.<init>(Driver.java:70)
By inspecting the code, the InputStream was not closed after use.
{code:java}
final InputStream inStream =
driverClass.getClassLoader().getResourceAsStream(resourceName);
{code}
The code to reproduce the leak is at
[https://github.com/leonliao/calcite-avatica-driver-version-test]. Follow the
README to reproduce the problem.
> Avatica DriverVersion.load leaks InputStream
> --------------------------------------------
>
> Key: CALCITE-2950
> URL: https://issues.apache.org/jira/browse/CALCITE-2950
> Project: Calcite
> Issue Type: Bug
> Components: avatica
> Affects Versions: avatica-1.13.0
> Environment: CentOS release 6.9 (Final)
> Reporter: Xiaobo Liao
> Priority: Minor
> Labels: pull-request-available
> Fix For: next
>
> Attachments: stream_not_closed.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Kylin's JDBC
> Driver([https://github.com/apache/kylin/tree/master/jdbc)|https://github.com/apache/kylin/tree/master/jdbc),]
> is based on Avatica. While using Kylin JDBC Driver, if repeatedly invokes
> DriverVersion.load, native memory leak can be observed, see attached memory
> profile diagram(from jeprof). Though it is nonsense to calling
> DriverVersion.load repeatedly, but the leak is there.
> {{
> package org.apache.calcite.avatica.test;
> import org.apache.calcite.avatica.DriverVersion;
> import org.apache.calcite.avatica.remote.Driver;
>
> /**
> * This class demonstrates the native memory leak caused by
> DriverVersion.load,
> * which has not closed the opened InputStream.
> */
> public class DriverVersionTest {
>
> public static void main(String[] args) throws Exception {
> while (true)
> { DriverVersion version = DriverVersion.load(Driver.class,
> "org-apache-kylin-jdbc.properties", "Kylin JDBC Driver", "unknown
> version", "Kylin", "unknown version"); System.out.println(version);
> Thread.sleep(100l); }
>
> }
> }}
> By using jemalloc's jeprof utility, the leak is caused by below stack trace
> at java.util.zip.Inflater.inflate()
> at java.util.zip.InflaterInputStream.read()
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.FilterInputStream.read(FilterInputStream.java:107)
> at java.util.Properties$LineReader.readLine(Properties.java:435)
> at java.util.Properties.load0(Properties.java:353)
> at java.util.Properties.load(Properties.java:341)
> at
> [org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.DriverVersion.load(DriverVersion.java:104|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.driverversion.load%28driverversion.java:104/])
> at org.apache.kylin.jdbc.Driver.createDriverVersion(Driver.java:88)
> at
> [org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.UnregisteredDriver|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.unregistereddriver/].<init>(UnregisteredDriver.java:56)
> at org.apache.kylin.jdbc.Driver.<init>(Driver.java:70)
>
> By inspecting the code, the InputStream was not closed after use.
>
> {code:java}
> final InputStream inStream =
> driverClass.getClassLoader().getResourceAsStream(resourceName);
> {code}
>
> The code to reproduce the leak is at
> [https://github.com/leonliao/calcite-avatica-driver-version-test]. Follow the
> README to reproduce the problem.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
