[jira] [Commented] (CALCITE-1240) Avatica client written in Golang
[ https://issues.apache.org/jira/browse/CALCITE-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094799#comment-16094799 ] Josh Elser commented on CALCITE-1240: - bq. 2. Is it possible to get a separate git repo for this? It makes it easier to import using Go's dependency tools like dep and the package structure would be more idiomatic. Yup bq. What is the process for getting the code into the Calcite/Avatica project? Essentially, we'll have to follow the IP Clearance process. https://incubator.apache.org/ip-clearance/index.html This isn't anything more than some formality in documenting: license compatibility, copyright assignment, and such. Working through the Go dependencies to make sure we follow ASF policies will be the biggest issue :) > Avatica client written in Golang > > > Key: CALCITE-1240 > URL: https://issues.apache.org/jira/browse/CALCITE-1240 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde > > Add a client for Avatica written in the Go language (aka "Golang"). > There is one at https://github.com/Boostport/avatica and the author has > offered to contribute it. > The driver is currently somewhat specialized for Phoenix but our goal should > be to allow it to work against any Avatica provider (without diminishing its > value to Phoenix users). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1240) Avatica client written in Golang
[ https://issues.apache.org/jira/browse/CALCITE-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094934#comment-16094934 ] Julian Hyde commented on CALCITE-1240: -- Here are the steps to go through the [IP clearance process|http://incubator.apache.org/ip-clearance/]. The steps are: # Vote on dev about accepting the code into the project # Add new IP clearance proposal to http://incubator.apache.org/ip-clearance/ # Contributor(s) (i.e. [~francischuang]) execute a [software grant document|https://www.apache.org/licenses/software-grant.txt] and PMC sends to the ASF secretary # Check that contributor(s) has filed an ICLA with ASF # Once the Software Grant has been acknowledged by the ASF secretary and the ICLAs are on file, start a lazy consensus vote about the IP clearance on general@incubator A separate git repo is certainly possible. I can see the benefits. I have a small concern that it might make unified releases of Avatica-Java and Avatica-Go more difficult (if we ever decided to do that). [~julienledem] since you've been through this, do you have any regrets about splitting Parquet into several repositories? > Avatica client written in Golang > > > Key: CALCITE-1240 > URL: https://issues.apache.org/jira/browse/CALCITE-1240 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde > > Add a client for Avatica written in the Go language (aka "Golang"). > There is one at https://github.com/Boostport/avatica and the author has > offered to contribute it. > The driver is currently somewhat specialized for Phoenix but our goal should > be to allow it to work against any Avatica provider (without diminishing its > value to Phoenix users). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (CALCITE-1896) sqlsh: SQL access to shell commands, as a shell command
Julian Hyde created CALCITE-1896:
Summary: sqlsh: SQL access to shell commands, as a shell command
Key: CALCITE-1896
URL: https://issues.apache.org/jira/browse/CALCITE-1896
Project: Calcite
Issue Type: Bug
Reporter: Julian Hyde
Assignee: Julian Hyde
Bourne shell has a toolkit of commands that allow you to do relational
processing: grep = WHERE, sort = ORDER BY, uniq = SELECT DISTINCT, head =
OFFSET/LIMIT, and so forth. Shell hackers are accustomed to writing pipelines.
For example, to find the 3 largest files one would type
{noformat}
$ find . -type f -print0 |xargs -0 ls -l | sort -nr -k 5 | head -3
-rw-r--r-- 1 jhyde staff 416028 Jul 16 10:06
./core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java
-rw-r--r-- 1 jhyde staff 325727 Jun 28 11:48
./core/src/test/java/org/apache/calcite/test/JdbcTest.java
-rw-r--r-- 1 jhyde staff 325106 Jul 16 10:06
./core/src/test/resources/org/apache/calcite/test/RelOptRulesTest.xml
{noformat}
SQL is not quite as concise but nevertheless:
{noformat}
$ sqlsh select \* from files where not directory order by size desc limit 3
{noformat}
{{sqlsh}} is "SQL shell", a wrapper around Calcite JDBC. {{files}} is a view
backed by a table function that calls "find .". Other commands:
* {{du}} table function - e.g. {{select path, size_k from du}}
* {{git_ls_files}} table function calls git-ls-files - e.g. {{select * from
git_ls_files}}
* {{ps}} table function calls {{ps aux}} - e.g. {{select * from ps}}
* {{wc}} function calls {{wc}} - e.g. {{select path, lineCount from
git_ls_files cross apply wc(path)}}
SQL would run in a lexical mode which is case-sensitive, and identifiers are
not upper-cased if not quoted.
We could consider allowing shell-safe characters such as '-' in unquoted
identifiers. (It's difficult to quote identifiers in SQL if the SQL has already
passed through bash's quote handling, and had double-quotes and single-quotes
removed.)
It gets really interesting when commands accept arguments, and Calcite pushes
down filters to become those arguments. For example, in
{code}sqlsh select distinct author from git_ls_files join git_commit_files
using \(path\) join git_commits using \(commit\){code}
Calcite should rewrite to use {{git log}}.
When accessing files, some kinds of files have implicit names (and types) for
fields, and some don't. You should be able to access fields by name or
position; the following are equivalent:
{noformat}
sqlsh select gid from /etc/passwd where uid = 100
sqlsh select $4 from /etc/passwd where $3 = 100
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1884) DateTimeUtils produces incorrect results for days before Gregorian cutovers
[
https://issues.apache.org/jira/browse/CALCITE-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095142#comment-16095142
]
Julian Hyde commented on CALCITE-1884:
--
You keep talking about "old style". My proposal is that dateStringToUnixDate
assumes that the date is in New Style, and that unixDateToString generates a
date string in New Style. So, old style doesn't enter into it. The fact that
you continually reference old style, and Java's behavior, muddies the issue.
You ask about how to add unit tests. There are several test cases in this bug
alone. One thing to test is that dateStringToUnixDate / unixDateToString are
inverses of each other. For a wide range of dates.
It should be obvious that tests in another project are not "unit tests". The
folks from the Flink side didn't seem aware of the Gregorian shift when this
conversation started.
As you can tell, I'm pissed off because you are making it difficult for me to
review this patch. You've basically moved a few lines of code around, extremely
complex code at that. You need to prove to me that the code now does the right
thing, not place the onus on me to prove that it doesn't.
> DateTimeUtils produces incorrect results for days before Gregorian cutovers
> ---
>
> Key: CALCITE-1884
> URL: https://issues.apache.org/jira/browse/CALCITE-1884
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.13.0
>Reporter: Haohui Mai
>Assignee: Haohui Mai
>
> dateStringToUnixDate() / unixDateToString() do not return consistent result.
> The following test fails:
> {noformat}
> @Test public void testUnixDate() {
> int days = DateTimeUtils.dateStringToUnixDate("1500-04-30");
> assertEquals("1500-04-30", DateTimeUtils.unixDateToString(days));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1240) Avatica client written in Golang
[ https://issues.apache.org/jira/browse/CALCITE-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095179#comment-16095179 ] Julian Hyde commented on CALCITE-1240: -- Actually, step 0 would be a *discussion* on dev. Let me start that. > Avatica client written in Golang > > > Key: CALCITE-1240 > URL: https://issues.apache.org/jira/browse/CALCITE-1240 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde > > Add a client for Avatica written in the Go language (aka "Golang"). > There is one at https://github.com/Boostport/avatica and the author has > offered to contribute it. > The driver is currently somewhat specialized for Phoenix but our goal should > be to allow it to work against any Avatica provider (without diminishing its > value to Phoenix users). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1884) DateTimeUtils produces incorrect results for days before Gregorian cutovers
[
https://issues.apache.org/jira/browse/CALCITE-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095201#comment-16095201
]
Haohui Mai commented on CALCITE-1884:
-
bq. ... My proposal is that dateStringToUnixDate assumes that the date is in
New Style, and that unixDateToString generates a date string in New Style ...
old style doesn't enter into it. ...
That's what I have tried and unfortunately it did not work (on a broader
picture).
See
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/runtime/SqlFunctions.java#L1646
for more details.
> DateTimeUtils produces incorrect results for days before Gregorian cutovers
> ---
>
> Key: CALCITE-1884
> URL: https://issues.apache.org/jira/browse/CALCITE-1884
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.13.0
>Reporter: Haohui Mai
>Assignee: Haohui Mai
>
> dateStringToUnixDate() / unixDateToString() do not return consistent result.
> The following test fails:
> {noformat}
> @Test public void testUnixDate() {
> int days = DateTimeUtils.dateStringToUnixDate("1500-04-30");
> assertEquals("1500-04-30", DateTimeUtils.unixDateToString(days));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1240) Avatica client written in Golang
[ https://issues.apache.org/jira/browse/CALCITE-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095516#comment-16095516 ] Francis Chuang commented on CALCITE-1240: - For the dependencies: Other than Go's standard library, the project also imports the following dependencies directly: https://github.com/Boostport/avatica/blob/master/Gopkg.toml The imported dependencies also imports further dependencies of their own. The list of ALL dependencies used are in here: https://github.com/Boostport/avatica/blob/master/Gopkg.lock Dependencies under the golang.org/x/ namespace are part of the Go project but are considered to be experimental. > Avatica client written in Golang > > > Key: CALCITE-1240 > URL: https://issues.apache.org/jira/browse/CALCITE-1240 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde > > Add a client for Avatica written in the Go language (aka "Golang"). > There is one at https://github.com/Boostport/avatica and the author has > offered to contribute it. > The driver is currently somewhat specialized for Phoenix but our goal should > be to allow it to work against any Avatica provider (without diminishing its > value to Phoenix users). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (CALCITE-1240) Avatica client written in Golang
[ https://issues.apache.org/jira/browse/CALCITE-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095516#comment-16095516 ] Francis Chuang edited comment on CALCITE-1240 at 7/20/17 11:03 PM: --- For the dependencies: Other than Go's standard library, the project also imports the following dependencies directly: https://github.com/Boostport/avatica/blob/master/Gopkg.toml The imported dependencies also import further dependencies of their own. The list of ALL dependencies used are in here: https://github.com/Boostport/avatica/blob/master/Gopkg.lock Dependencies under the golang.org/x/ namespace are part of the Go project but are considered to be experimental. was (Author: francischuang): For the dependencies: Other than Go's standard library, the project also imports the following dependencies directly: https://github.com/Boostport/avatica/blob/master/Gopkg.toml The imported dependencies also imports further dependencies of their own. The list of ALL dependencies used are in here: https://github.com/Boostport/avatica/blob/master/Gopkg.lock Dependencies under the golang.org/x/ namespace are part of the Go project but are considered to be experimental. > Avatica client written in Golang > > > Key: CALCITE-1240 > URL: https://issues.apache.org/jira/browse/CALCITE-1240 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde > > Add a client for Avatica written in the Go language (aka "Golang"). > There is one at https://github.com/Boostport/avatica and the author has > offered to contribute it. > The driver is currently somewhat specialized for Phoenix but our goal should > be to allow it to work against any Avatica provider (without diminishing its > value to Phoenix users). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1240) Avatica client written in Golang
[ https://issues.apache.org/jira/browse/CALCITE-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095559#comment-16095559 ] Julian Hyde commented on CALCITE-1240: -- It seems that [go-cleanhttp|https://github.com/hashicorp/go-cleanhttp/blob/master/LICENSE] is MPL, which is [category b (weak copyleft)|https://www.apache.org/legal/resolved.html#category-b]. We'll have to be careful with that. The others are Apache, BSD or MIT, so they're OK. > Avatica client written in Golang > > > Key: CALCITE-1240 > URL: https://issues.apache.org/jira/browse/CALCITE-1240 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde > > Add a client for Avatica written in the Go language (aka "Golang"). > There is one at https://github.com/Boostport/avatica and the author has > offered to contribute it. > The driver is currently somewhat specialized for Phoenix but our goal should > be to allow it to work against any Avatica provider (without diminishing its > value to Phoenix users). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1884) DateTimeUtils produces incorrect results for days before Gregorian cutovers
[
https://issues.apache.org/jira/browse/CALCITE-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095574#comment-16095574
]
Julian Hyde commented on CALCITE-1884:
--
It would help a lot if you were to write some unit tests. Then we can discuss
what those tests should produce.
> DateTimeUtils produces incorrect results for days before Gregorian cutovers
> ---
>
> Key: CALCITE-1884
> URL: https://issues.apache.org/jira/browse/CALCITE-1884
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.13.0
>Reporter: Haohui Mai
>Assignee: Haohui Mai
>
> dateStringToUnixDate() / unixDateToString() do not return consistent result.
> The following test fails:
> {noformat}
> @Test public void testUnixDate() {
> int days = DateTimeUtils.dateStringToUnixDate("1500-04-30");
> assertEquals("1500-04-30", DateTimeUtils.unixDateToString(days));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1884) DateTimeUtils produces incorrect results for days before Gregorian cutovers
[
https://issues.apache.org/jira/browse/CALCITE-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095573#comment-16095573
]
Julian Hyde commented on CALCITE-1884:
--
What does Date.toString() print for the day before 1752-09-14? Are you seeing
it print 1752-09-02? If so, then it's shifted into old-style. But it's still
the day before 1752-09-14, its julian day number is one smaller, and frankly
it's just a problem with how java prints it.
SQL, which stays in new style, would print 1752-09-13.
> DateTimeUtils produces incorrect results for days before Gregorian cutovers
> ---
>
> Key: CALCITE-1884
> URL: https://issues.apache.org/jira/browse/CALCITE-1884
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.13.0
>Reporter: Haohui Mai
>Assignee: Haohui Mai
>
> dateStringToUnixDate() / unixDateToString() do not return consistent result.
> The following test fails:
> {noformat}
> @Test public void testUnixDate() {
> int days = DateTimeUtils.dateStringToUnixDate("1500-04-30");
> assertEquals("1500-04-30", DateTimeUtils.unixDateToString(days));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1240) Avatica client written in Golang
[ https://issues.apache.org/jira/browse/CALCITE-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095590#comment-16095590 ] Francis Chuang commented on CALCITE-1240: - go-cleanhttp is a really simple library. All it does is set up the http client with sane defaults: https://github.com/hashicorp/go-cleanhttp/blob/master/cleanhttp.go If required, I can remove the dependency on it and initialize the http client manually with those defaults. > Avatica client written in Golang > > > Key: CALCITE-1240 > URL: https://issues.apache.org/jira/browse/CALCITE-1240 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde > > Add a client for Avatica written in the Go language (aka "Golang"). > There is one at https://github.com/Boostport/avatica and the author has > offered to contribute it. > The driver is currently somewhat specialized for Phoenix but our goal should > be to allow it to work against any Avatica provider (without diminishing its > value to Phoenix users). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1240) Avatica client written in Golang
[ https://issues.apache.org/jira/browse/CALCITE-1240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095599#comment-16095599 ] Julian Hyde commented on CALCITE-1240: -- Shouldn't be necessary. Category b licensed dependencies are OK, as long as we're careful. We can deal with this the first release after the code is accepted. > Avatica client written in Golang > > > Key: CALCITE-1240 > URL: https://issues.apache.org/jira/browse/CALCITE-1240 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde > > Add a client for Avatica written in the Go language (aka "Golang"). > There is one at https://github.com/Boostport/avatica and the author has > offered to contribute it. > The driver is currently somewhat specialized for Phoenix but our goal should > be to allow it to work against any Avatica provider (without diminishing its > value to Phoenix users). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (CALCITE-1885) calcite code gen make many class in perm and it cant be gc
[
https://issues.apache.org/jira/browse/CALCITE-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095640#comment-16095640
]
yiming.xu commented on CALCITE-1885:
i try it , It still can not gc.
> calcite code gen make many class in perm and it cant be gc
> ---
>
> Key: CALCITE-1885
> URL: https://issues.apache.org/jira/browse/CALCITE-1885
> Project: Calcite
> Issue Type: Bug
>Reporter: yiming.xu
>Assignee: Julian Hyde
>
> calcite code gen make many class in perm and it cant be gc, how to unload
> that class, i need help ! thx!
> this is log for cluster env , i find it locked in classload and
> Class.forname01:
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1735)
> - waiting to lock <0x000340001cf8> (a
> io.kyligence.kap.ext.CustomizedWebappClassloader)
> at
> io.kyligence.kap.ext.CustomizedWebappClassloader.loadClass(CustomizedWebappClassloader.java:76)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
> - locked <0x0005f9805b18> (a
> org.codehaus.janino.ByteArrayClassLoader)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at Baz.bind(Unknown Source)
> at
> org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:331)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:294)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:553)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:544)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:193)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.codehaus.janino.ClassLoaderIClassLoader.findIClass(ClassLoaderIClassLoader.java:78)
> at org.codehaus.janino.IClassLoader.loadIClass(IClassLoader.java:254)
> - locked <0x0005e9802818> (a
> org.codehaus.janino.ClassLoaderIClassLoader)
> this test case can reappear memory leak ,use it with csvtest!
> BasicDataSource dataSource = (BasicDataSource)
> BasicDataSourceFactory.createDataSource(new Properties());
> private void checkSql(String sql, String model, Function
> fn)
> throws SQLException {
> Connection connection = null;
> Statement statement = null;
> try {
> Properties info = new Properties();
> info.put("model", jsonPath(model));
> dataSource.setDriverClassName(Driver.class.getName());
> dataSource.setUrl("jdbc:calcite:model=" + jsonPath(model));
> connection = dataSource.getConnection();
> statement = connection.createStatement();
> final ResultSet resultSet =
> statement.executeQuery(
> sql);
> fn.apply(resultSet);
> } catch (Exception e) {
> e.printStackTrace();
> } finally {
> close(connection, statement);
> }
> }
> the test case:
> @Test
> public void testConcurrentQuery() throws SQLException, InterruptedException
> {
> final String sql = "select * \n"
> + "from long_emps";
> final AtomicLong a = new AtomicLong(0);
> long start = System.currentTimeMillis();
> final Semaphore semaphore = new Semaphore(0);
> for (int i = 0; i < 8; i++) {
> new Thread(new Runnable() {
> int count = 0;
> @Override
> public void run() {
> while (true) {
> sql("bug", sql).checking(new Function() {
> public Void apply(ResultSet resultSet) {
> return null;
> }
> }).ok();
> semaphore.release();
> //System.out.println(Thread.currentThread().getName() + ":" +
> count);
> }
> }
> }).start();
> }
> semaphore.acquire(100);
> System.out.println(System.currentTimeMillis() - start);
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CALCITE-1885) calcite code gen make many class in perm and it cant be gc
[
https://issues.apache.org/jira/browse/CALCITE-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095640#comment-16095640
]
yiming.xu edited comment on CALCITE-1885 at 7/21/17 1:30 AM:
-
i try it ,.permsize dont increase , but jvm stil full gc. cpu very hight
was (Author: hn5092):
i try it , It still can not gc.
> calcite code gen make many class in perm and it cant be gc
> ---
>
> Key: CALCITE-1885
> URL: https://issues.apache.org/jira/browse/CALCITE-1885
> Project: Calcite
> Issue Type: Bug
>Reporter: yiming.xu
>Assignee: Julian Hyde
>
> calcite code gen make many class in perm and it cant be gc, how to unload
> that class, i need help ! thx!
> this is log for cluster env , i find it locked in classload and
> Class.forname01:
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1735)
> - waiting to lock <0x000340001cf8> (a
> io.kyligence.kap.ext.CustomizedWebappClassloader)
> at
> io.kyligence.kap.ext.CustomizedWebappClassloader.loadClass(CustomizedWebappClassloader.java:76)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
> - locked <0x0005f9805b18> (a
> org.codehaus.janino.ByteArrayClassLoader)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at Baz.bind(Unknown Source)
> at
> org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:331)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:294)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:553)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:544)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:193)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at
> org.codehaus.janino.ClassLoaderIClassLoader.findIClass(ClassLoaderIClassLoader.java:78)
> at org.codehaus.janino.IClassLoader.loadIClass(IClassLoader.java:254)
> - locked <0x0005e9802818> (a
> org.codehaus.janino.ClassLoaderIClassLoader)
> this test case can reappear memory leak ,use it with csvtest!
> BasicDataSource dataSource = (BasicDataSource)
> BasicDataSourceFactory.createDataSource(new Properties());
> private void checkSql(String sql, String model, Function
> fn)
> throws SQLException {
> Connection connection = null;
> Statement statement = null;
> try {
> Properties info = new Properties();
> info.put("model", jsonPath(model));
> dataSource.setDriverClassName(Driver.class.getName());
> dataSource.setUrl("jdbc:calcite:model=" + jsonPath(model));
> connection = dataSource.getConnection();
> statement = connection.createStatement();
> final ResultSet resultSet =
> statement.executeQuery(
> sql);
> fn.apply(resultSet);
> } catch (Exception e) {
> e.printStackTrace();
> } finally {
> close(connection, statement);
> }
> }
> the test case:
> @Test
> public void testConcurrentQuery() throws SQLException, InterruptedException
> {
> final String sql = "select * \n"
> + "from long_emps";
> final AtomicLong a = new AtomicLong(0);
> long start = System.currentTimeMillis();
> final Semaphore semaphore = new Semaphore(0);
> for (int i = 0; i < 8; i++) {
> new Thread(new Runnable() {
> int count = 0;
> @Override
> public void run() {
> while (true) {
> sql("bug", sql).checking(new Function() {
> public Void apply(ResultSet resultSet) {
> return null;
> }
> }).ok();
> semaphore.release();
> //System.out.println(Thread.currentThread().getName() + ":" +
> count);
> }
> }
> }).start();
> }
> semaphore.acquire(100);
> System.out.println(System.currentTimeMillis() - start);
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (CALCITE-1570) Add MATCH_RECOGNIZE operator, for event pattern-matching
[
https://issues.apache.org/jira/browse/CALCITE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-1570:
-
Labels: match (was: )
> Add MATCH_RECOGNIZE operator, for event pattern-matching
>
>
> Key: CALCITE-1570
> URL: https://issues.apache.org/jira/browse/CALCITE-1570
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Zhiqiang He
> Labels: match
> Fix For: 1.13.0
>
>
> Add the MATCH_RECOGNIZE operator, for event pattern-matching. Oracle
> [introduced this in
> 11i|https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1]
> (for tables) and Esper [implemented
> it|http://www.espertech.com/esper/release-5.1.0/esper-reference/html/match-recognize.html]
> (for streams of events).
> It would be most useful for streaming SQL but it makes sense in non-streaming
> SQL too (and of course it's good to be able to run streaming queries on
> historic data).
> Here is an example from
> [oracle-base|https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1]:
> {code}
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
> PARTITION BY product
> ORDER BY tstamp
> MEASURES STRT.tstamp AS start_tstamp,
>FINAL LAST(UP.tstamp) AS peak_tstamp,
>MATCH_NUMBER() AS mno,
>CLASSIFIER() AS cls
> ALL ROWS PER MATCH
> AFTER MATCH SKIP TO LAST DOWN
> PATTERN (STRT UP+ DOWN{1} UP+)
> DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.tstamp;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (CALCITE-1847) Queries fail in the Cassandra adapter
[ https://issues.apache.org/jira/browse/CALCITE-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Hyde closed CALCITE-1847. Resolved in release 1.13.0 (2017-06-26). > Queries fail in the Cassandra adapter > - > > Key: CALCITE-1847 > URL: https://issues.apache.org/jira/browse/CALCITE-1847 > Project: Calcite > Issue Type: Bug > Components: cassandra >Affects Versions: 1.13.0 >Reporter: Michael Mior >Assignee: Michael Mior > Fix For: 1.13.0 > > > It seems like there has been a change with when getRowType is called. Not > sure exactly what changed, but the Cassandra adapter relied on the old > behaviour. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (CALCITE-1570) Add MATCH_RECOGNIZE operator, for event pattern-matching
[
https://issues.apache.org/jira/browse/CALCITE-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde closed CALCITE-1570.
Resolved in release 1.13.0 (2017-06-26).
> Add MATCH_RECOGNIZE operator, for event pattern-matching
>
>
> Key: CALCITE-1570
> URL: https://issues.apache.org/jira/browse/CALCITE-1570
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Zhiqiang He
> Labels: match
> Fix For: 1.13.0
>
>
> Add the MATCH_RECOGNIZE operator, for event pattern-matching. Oracle
> [introduced this in
> 11i|https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1]
> (for tables) and Esper [implemented
> it|http://www.espertech.com/esper/release-5.1.0/esper-reference/html/match-recognize.html]
> (for streams of events).
> It would be most useful for streaming SQL but it makes sense in non-streaming
> SQL too (and of course it's good to be able to run streaming queries on
> historic data).
> Here is an example from
> [oracle-base|https://oracle-base.com/articles/12c/pattern-matching-in-oracle-database-12cr1]:
> {code}
> SELECT *
> FROM sales_history MATCH_RECOGNIZE (
> PARTITION BY product
> ORDER BY tstamp
> MEASURES STRT.tstamp AS start_tstamp,
>FINAL LAST(UP.tstamp) AS peak_tstamp,
>MATCH_NUMBER() AS mno,
>CLASSIFIER() AS cls
> ALL ROWS PER MATCH
> AFTER MATCH SKIP TO LAST DOWN
> PATTERN (STRT UP+ DOWN{1} UP+)
> DEFINE
>UP AS UP.units_sold > PREV(UP.units_sold),
>DOWN AS DOWN.units_sold < PREV(DOWN.units_sold)
>) MR
> ORDER BY MR.product, MR.tstamp;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (CALCITE-1641) Base functions support for MATCH_RECOGNIZE
[
https://issues.apache.org/jira/browse/CALCITE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde closed CALCITE-1641.
Resolved in release 1.12.0 (2017-03-24).
> Base functions support for MATCH_RECOGNIZE
> --
>
> Key: CALCITE-1641
> URL: https://issues.apache.org/jira/browse/CALCITE-1641
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Affects Versions: 1.11.0
>Reporter: Zhiqiang He
>Assignee: Zhiqiang He
> Labels: features
> Fix For: 1.12.0
>
>
> MATCH_RECOGNIZE syntax like
> this:https://docs.oracle.com/database/121/DWHSG/pattern.htm#DWHSG8980
> Only pattern and define is supported in first step.
> h1. PATTERN: Defining the Row Pattern to Be Matched
> The PATTERN keyword specifies the pattern to be recognized in the ordered
> sequence of rows in a partition. Each variable name in a pattern corresponds
> to a Boolean condition, which is specified later using the DEFINE component
> of the syntax.
> The PATTERN clause is used to specify a regular expression. It is outside the
> scope of this material to explain regular expression concepts and details. If
> you are not familiar with regular expressions, you are encouraged to
> familiarize yourself with the topic using other sources.
> The regular expression in a PATTERN clause is enclosed in parentheses.
> PATTERN may use the following operators:
> * Concatenation
> Concatenation is used to list two or more items in a pattern to be matched in
> that order. Items are concatenated when there is no operator sign between two
> successive items. For example: PATTERN (A B C).
> * Quantifiers
> Quantifiers define the number of iterations accepted for a match. Quantifiers
> are postfix operators with the following choices:
> ** \* — 0 or more iterations
> ** + — 1 or more iterations
> ** ? — 0 or 1 iterations
> ** {n} — n iterations (n > 0)
> ** {n,} — n or more iterations (n >= 0)
> ** {n,m} — between n and m (inclusive) iterations (0 <= n <= m, 0 < m)
> ** {,m} — between 0 and m (inclusive) iterations (m > 0)
> ** reluctant quantifiers — indicated by an additional question mark following
> a quantifier (*?, +?, ??, {n,}?, { n, m }?, {,m}?). See "Reluctant Versus
> Greedy Quantifier" for the difference between reluctant and non-reluctant
> quantifiers.
> The following are examples of using quantifier operators:
> ** A* matches 0 or more iterations of A
> ** A{3,6} matches 3 to 6 iterations of A
> ** A{,4} matches 0 to 4 iterations of A
> * Alternation
> Alternation matches a single regular expression from a list of several
> possible regular expressions. The alternation list is created by placing a
> vertical bar (|) between each regular expression. Alternatives are preferred
> in the order they are specified. As an example, PATTERN (A | B | C) attempts
> to match A first. If A is not matched, it attempts to match B. If B is not
> matched, it attempts to match C.
> * Grouping
> Grouping treats a portion of the regular expression as a single unit,
> enabling you to apply regular expression operators such as quantifiers to
> that group. Grouping is created with parentheses. As an example, PATTERN ((A
> B){3} C) attempts to match the group (A B) three times and then seeks one
> occurrence of C.
> * PERMUTE
> See "How to Express All Permutations" for more information.
> * Exclusion
> Parts of the pattern to be excluded from the output of ALL ROWS PER MATCH are
> enclosed between {- and -}. See "How to Exclude Portions of the Pattern from
> the Output".
> * Anchors
> Anchors work in terms of positions rather than rows. They match a position
> either at the start or end of a partition.
> ** ^ matches the position before the first row in the partition.
> ** $ matches the position after the last row in the partition.
> As an example, PATTERN (^A+$) will match only if all rows in a partition
> satisfy the condition for A. The resulting match spans the entire partition.
> * Empty pattern (), matches an empty set of rows
> This section contains the following topics:
> Reluctant Versus Greedy Quantifier
> Operator Precedence
> h2. Reluctant Versus Greedy Quantifier
> Pattern quantifiers are referred to as greedy; they will attempt to match as
> many instances of the regular expression on which they are applied as
> possible. The exception is pattern quantifiers that have a question mark ? as
> a suffix, and those are referred to as reluctant. They will attempt to match
> as few instances as possible of the regular expression on which they are
> applied.
> The difference between greedy and reluctant quantifiers appended to a single
> pattern variable is illustrated as follows: A* tries to map as many rows as
> possible to A, whereas A*? tries to map as few rows as possible to A. For
> example
[jira] [Created] (CALCITE-1897) Support operator "%" as an alternative to "mod"
sunjincheng created CALCITE-1897: Summary: Support operator "%" as an alternative to "mod" Key: CALCITE-1897 URL: https://issues.apache.org/jira/browse/CALCITE-1897 Project: Calcite Issue Type: Bug Components: core Reporter: sunjincheng Assignee: sunjincheng Currently the following sql is not supported. SELECT a%3 FROM T We get the exception: Caused by: org.apache.calcite.sql.parser.SqlParseException: Lexical error at line 1, column 9. Encountered: "%" (37), after : "" at org.apache.calcite.sql.parser.impl.SqlParserImpl.convertException(SqlParserImpl.java:396) at org.apache.calcite.sql.parser.impl.SqlParserImpl.normalizeException(SqlParserImpl.java:129) In this JIRA. I'll fix this by support operator "%" as an alternative to "mod". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
