Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/17886#discussion_r115196548
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
---
@@ -71,6 +71,10 @@
import
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/9113#issuecomment-197969602
+1 to this feature, may need to add this doc
http://spark.apache.org/docs/latest/sql-programming-guide.html#running-the-thrift-jdbcodbc-server
---
If your project is
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/8909#issuecomment-198071715
anyone tried use --hivevar or --hiveconf with beeline after this change?
variables from JDBC client is no more available in the hive session.
refer to https
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/10044#issuecomment-160913033
it was caused by this commit about half years ago:
https://github.com/apache/spark/commit/3bee0f1466ddd69f26e95297b5e0d2398b6c6268#diff
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/10044#issuecomment-160909430
@sarutak it seems branch-1.4 was correct with
```EventLoggingListener.getLogPath```
https://github.com/apache/spark/blob/v1.4.1/core/src/main/scala/org/apache
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/10044#issuecomment-160908412
thanks for merging, i will check this for branch-1.4
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/10044#issuecomment-160729514
ha, yes, it happens, okey, i will add some parameter names
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/10044
[SPARK-12053][CORE] EventLoggingListener.getLogPath needs 4 parameters
```EventLoggingListener.getLogPath``` needs 4 input arguments:
https://github.com/apache/spark/blob/v1.6.0-preview2/core
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/7421#issuecomment-122318896
getPartitionsByFilter is really a great improvement, normally in a
production hive data warehouse, there are tables with huge amount of
partitions. looking forward to
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1612#issuecomment-55317048
thanks for the review, i will try to improve it soon, adding more external
datasources is always helpful, then we can use Spark SQL as a data integration
platform, and
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r17203025
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcResultSetRDD.scala ---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r17101449
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcResultSetRDD.scala ---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/2039#issuecomment-53408822
thanks @marmbrus , i change the property name and default codec
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r16705825
--- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala ---
@@ -81,8 +113,14 @@ class JdbcRDD[T: ClassTag](
logInfo("statement fetch
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r16705207
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/jdbc/JdbcResultSetRDDSuite.scala
---
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r16704875
--- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala ---
@@ -57,6 +61,8 @@ class JdbcRDD[T: ClassTag](
mapRow: (ResultSet) =>
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/2084#issuecomment-53243726
is this PR for SPARK-3065 or SPARK-3173 ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1959#discussion_r16589482
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala ---
@@ -373,9 +373,11 @@ private[parquet] object ParquetTypesConverter
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1959#discussion_r16530668
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala ---
@@ -373,9 +373,11 @@ private[parquet] object ParquetTypesConverter
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1959#discussion_r16527681
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala ---
@@ -373,9 +373,11 @@ private[parquet] object ParquetTypesConverter
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/2044#issuecomment-52748630
actually i totally agree with you @liancheng , this was reported in user
list
http://apache-spark-user-list.1001560.n3.nabble.com/sqlContext-parquetFile-path-fails-if
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/2044#issuecomment-52747630
actually i totally agree with you @liancheng ,
and it is not "try to handle", it is more like "try to ignore useless
check"
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/2036#issuecomment-52713289
hmm... jenkins is not working again?...
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18886/consoleFull
---
If your project is set up for it, you
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/2044
[SPARK-3138][SQL] sqlContext.parquetFile should be able to take a single
file as parameter
```if (!fs.getFileStatus(path).isDir) throw Exception``` make no sense
after this commit #1370
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/2039#issuecomment-52707530
it seems failed test cases are not related to this PR, all of them are in
sql/hive-thriftserver modul
---
If your project is set up for it, you can reply to this email
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/2039
[SPARK-3131][SQL] Allow user to set parquet compression codec for writing
ParquetFile in SQLContext
There are 4 different compression codec available for
```ParquetOutputFormat
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1855#discussion_r16345194
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala ---
@@ -403,7 +406,10 @@ private[parquet] object ParquetTypesConverter
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1855#discussion_r16294353
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala ---
@@ -403,7 +406,10 @@ private[parquet] object ParquetTypesConverter
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1861#issuecomment-51660191
maybe ```spark-shell.cmd``` should also be changed?
```cmd /V /E /C %SPARK_HOME%\bin\spark-submit.cmd spark-shell --class
org.apache.spark.repl.Main
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/1861
[SPARK-2678] Follow-up commit for #1715 #1801, spark-shell is broken
all options are not working...
just run spark-shell with any options, such as --master
```./bin/spark-shell --master
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/195#discussion_r15936076
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala ---
@@ -72,16 +71,56 @@ case class ParquetRelation(val tableName: String
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1346#discussion_r15862768
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala ---
@@ -89,6 +88,44 @@ class SQLContext(@transient val sparkContext:
SparkContext
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1346#discussion_r15799720
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala ---
@@ -89,6 +88,44 @@ class SQLContext(@transient val sparkContext:
SparkContext
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1346#discussion_r15766362
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala ---
@@ -89,6 +88,44 @@ class SQLContext(@transient val sparkContext:
SparkContext
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1691#issuecomment-50859555
done, thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/1691
[SPARK-2700] [SQL] Hidden files (such as .impala_insert_staging) should be
filtered out by sqlContext.parquetFile
You can merge this pull request into a Git repository by running:
$ git pull
Github user chutium closed the pull request at:
https://github.com/apache/spark/pull/1599
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1636#issuecomment-50733743
thanks a lot to you both, cast string to timestamp is really clever :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1636#issuecomment-50625472
if you have time, maybe take a look at #1612 ? it is about this ticket,
https://issues.apache.org/jira/browse/SPARK-2710 , test suite just added.
it will be a
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1612#issuecomment-50617738
Test Suite added
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1636#issuecomment-50593505
NullableColumnBuilderSuite:
...
...
- TIMESTAMP column builder: empty column
- TIMESTAMP column builder: buffer size auto growth
- TIMESTAMP column
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r15573659
--- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala ---
@@ -67,6 +69,28 @@ class JdbcRDD[T: ClassTag](
}).toArray
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r15573636
--- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala ---
@@ -81,8 +105,10 @@ class JdbcRDD[T: ClassTag](
logInfo("statement fetch
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1636#issuecomment-50592256
it seems should be this one?
NullableColumnBuilderSuite.scala
i can do it
---
If your project is set up for it, you can reply to this email and have your
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1636#issuecomment-50531836
jo, thanks, i will add [SQL] next time
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/1636
[SPARK-2729] Forgot to match Timestamp type in ColumnBuilder
just a match forgot, found after SPARK-2710 , TimestampType can be used by
a SchemaRDD generated from JDBC ResultSet
You can merge this
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r15528919
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcResultSetRDD.scala ---
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r15522818
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcResultSetRDD.scala ---
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r15522485
--- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala ---
@@ -67,6 +69,28 @@ class JdbcRDD[T: ClassTag](
}).toArray
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r15452368
--- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala ---
@@ -81,8 +105,10 @@ class JdbcRDD[T: ClassTag](
logInfo("statement fetch
Github user chutium commented on a diff in the pull request:
https://github.com/apache/spark/pull/1612#discussion_r15451376
--- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala ---
@@ -67,6 +69,28 @@ class JdbcRDD[T: ClassTag](
}).toArray
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/1612
SPARK-2710 Build SchemaRDD from a JdbcRDD with MetaData
SPARK-2710 Build SchemaRDD from a JdbcRDD with MetaData
and a small bug fix on JdbcRDD, line 109
it seems conn will never be
Github user chutium closed the pull request at:
https://github.com/apache/spark/pull/1599
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user chutium reopened a pull request:
https://github.com/apache/spark/pull/1599
[SPARK-2700] [SQL] Hidden files (such as .impala_insert_staging) should be
filtered out by sqlContext.parquetFile
check if the path name starts with '.'
You can merge this pull requ
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/1599
[SPARK-2700] [SQL] Hidden files (such as .impala_insert_staging) should be
filtered out by sqlContext.parquetFile
check if the path name starts with '.'
You can merge this pull request
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1442#issuecomment-49407883
@egraldlo @willb thanks guys, substring is also added, @marmbrus test also
done and passed.
---
If your project is set up for it, you can reply to this email and have
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1442#issuecomment-49381658
and... how can i run a single test class using maven? i tried:
mvn -Pyarn -Phive -Dhadoop.version=2.3.0-cdh5.0.3 -Dtest=SQLQuerySuite test
but it does not work
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1442#issuecomment-49379575
added a simple test in SQLQuerySuite
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1359#issuecomment-49159677
PR submitted #1442
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user chutium opened a pull request:
https://github.com/apache/spark/pull/1442
SPARK-2407: Added Parse of SQL SUBSTR()
follow-up of #1359
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chutium/spark master
Alternatively
Github user chutium commented on the pull request:
https://github.com/apache/spark/pull/1359#issuecomment-49154812
hi, it is really very useful for us, i tried this implementation from
@willb , in spark-shell, i still got java.lang.UnsupportedOperationException by
Query Plan, i made
61 matches
Mail list logo
