[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2024-09-11 Thread halim kim (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17880945#comment-17880945
 ] 

halim kim commented on SPARK-41063:
---

This issue is related to the zinc incremental compiler bug. 

spark 3.4.x uses scala-maven-plugin 4.8.0 where zinc 1.8.0 has been integrated.

zinc had a bug that loops infinitely with transitive invalidation. zinc 1.8.1 
and upper version has fixed the bug.

please refer 
[https://github.com/sbt/zinc/commit/3866ac44c80ccab40624077236e51ef07c0d0069]

when you face the same issue with building hive-thriftserver module, add zinc 
1.8.1 dependency for scala-maven-plugin

 

```xml

...

          
                
                    org.scala-sbt
                    zinc_2.13
                    1.8.1
                
            

...

```

 

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-10 Thread Hyukjin Kwon (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632053#comment-17632053
 ] 

Hyukjin Kwon commented on SPARK-41063:
--

Reverted at 
https://github.com/apache/spark/commit/73bca6e5cace0c2c46938e82fa12ab518faa2248

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Assignee: Hyukjin Kwon
>Priority: Major
> Fix For: 3.4.0
>
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-10 Thread Apache Spark (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631555#comment-17631555
 ] 

Apache Spark commented on SPARK-41063:
--

User 'HyukjinKwon' has created a pull request for this issue:
https://github.com/apache/spark/pull/38599

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-10 Thread Apache Spark (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631556#comment-17631556
 ] 

Apache Spark commented on SPARK-41063:
--

User 'HyukjinKwon' has created a pull request for this issue:
https://github.com/apache/spark/pull/38599

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-10 Thread Hyukjin Kwon (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631548#comment-17631548
 ] 

Hyukjin Kwon commented on SPARK-41063:
--

I have the internal fix we use in Databriclks. let me push that to OSS

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-09 Thread Yang Jie (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631487#comment-17631487
 ] 

Yang Jie commented on SPARK-41063:
--

Yes, from Scala 2.13 daily build

 

https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-10T07%3A05%3A18.1724380Z&urlSigningMethod=HMACV1&urlSignature=g8ZPJ9Amk2WIxjBRRl2pYZG40leY7oNav3fFNjt0HVw%3D

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-08 Thread Hyukjin Kwon (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630744#comment-17630744
 ] 

Hyukjin Kwon commented on SPARK-41063:
--

Is this from our CI?

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-08 Thread Hyukjin Kwon (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630742#comment-17630742
 ] 

Hyukjin Kwon commented on SPARK-41063:
--

I am aware of this issue but I don't exactly know how to fix. As a workaround, 
you can try {{git clean -fxd}}  (see also 
https://github.com/sbt/sbt/issues/6183)

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-08 Thread Yang Jie (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630731#comment-17630731
 ] 

Yang Jie commented on SPARK-41063:
--

cc [~hyukjin.kwon] 

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-41063) `hive-thriftserver` module compilation deadlock

2022-11-08 Thread Yang Jie (Jira) 


[ 
https://issues.apache.org/jira/browse/SPARK-41063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630729#comment-17630729
 ] 

Yang Jie commented on SPARK-41063:
--

 

Always printing similar logs in a loop

 
{code:java}
2022-11-09T01:01:42.8632147Z [info] Note: 
/home/runner/work/spark/spark/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/thrift/ThriftHttpServlet.java
 uses or overrides a deprecated API.
2022-11-09T01:01:42.8632717Z [info] Note: Recompile 
with -Xlint:deprecation for details.
2022-11-09T01:01:42.8633082Z [info] done compiling
2022-11-09T01:01:42.8633618Z [info] compiling 22 Scala 
sources and 24 Java sources to 
/home/runner/work/spark/spark/sql/hive-thriftserver/target/scala-2.13/classes 
...
2022-11-09T01:01:42.8634133Z [info] Note: Some input 
files use or override a deprecated API.
2022-11-09T01:01:42.8634581Z [info] Note: Recompile 
with -Xlint:deprecation for details.
2022-11-09T01:01:42.8634941Z [info] done compiling
2022-11-09T01:01:42.8635478Z [info] compiling 22 Scala 
sources and 9 Java sources to 
/home/runner/work/spark/spark/sql/hive-thriftserver/target/scala-2.13/classes 
... {code}

> `hive-thriftserver` module compilation deadlock
> ---
>
> Key: SPARK-41063
> URL: https://issues.apache.org/jira/browse/SPARK-41063
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.4.0
>Reporter: Yang Jie
>Priority: Major
>
> [https://pipelines.actions.githubusercontent.com/serviceHosts/03398d36-4378-4d47-a936-fba0a5e8ccb9/_apis/pipelines/1/runs/194220/signedlogcontent/20?urlExpires=2022-11-09T03%3A31%3A51.5657953Z&urlSigningMethod=HMACV1&urlSignature=Jnn4uML8U79K6MF%2F%2BRUrrUbaOqxsqMA0DL0%2BQtrlBpM%3D]
>  
> I have seen it when compiling with Maven locally,  but I haven't investigated
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org