``
```
[u'a', u'b', u'', u'd', u'e', u'f', u'g', u'h', u'i', u'j', u'k', u'l', u'']
```
## How was this patch tested?
Unit test in `python/pyspark/tests.py`.
Author: hyukjinkwon <gurwls...@gmail.com>
Closes #17282 from HyukjinKwon/SPARK-19872.
(cherry picked from commit 7387126f83dc0489eb1
85be
Author: hyukjinkwon <gurwls...@gmail.com>
Authored: Wed Mar 15 10:17:18 2017 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Wed Mar 15 10:17:18 2017 -0700
--
python/pyspark/
ion in DataFrames near the epoch.
## How was this patch tested?
Added a new test that fails without the change.
dongjoon-hyun davies Mind taking a look?
The contribution is my original work and I license the work to the project
under the projectâs open source license.
Author: Jason Wh
amp creation in DataFrames near the epoch.
## How was this patch tested?
Added a new test that fails without the change.
dongjoon-hyun davies Mind taking a look?
The contribution is my original work and I license the work to the project
under the projectâs open source license.
Author: Jason Wh
2.2, or fail to insert into InMemorySorter in 2.1).
This PR fix the off-by-one bug in BytesToBytesMap.
This PR also fix a bug that the array will never grow if it fail to grow once
(stay as initial capacity), introduced by #15722 .
## How was this patch tested?
Added regression test.
Author: Dav
2.2, or fail to insert into InMemorySorter in 2.1).
This PR fix the off-by-one bug in BytesToBytesMap.
This PR also fix a bug that the array will never grow if it fail to grow once
(stay as initial capacity), introduced by #15722 .
## How was this patch tested?
Added regression test.
Author: Dav
ail to insert into InMemorySorter in 2.1).
This PR fix the off-by-one bug in BytesToBytesMap.
This PR also fix a bug that the array will never grow if it fail to grow once
(stay as initial capacity), introduced by #15722 .
## How was this patch tested?
Added regression test.
Author: Davies
er
Commit: 303f00a4bf6660dd83c8bd9e3a107bb3438a421b
Parents: 6287c94
Author: Shixiong Zhu <shixi...@databricks.com>
Authored: Thu Feb 9 11:16:51 2017 -0800
Committer: Davies Liu <davies@gmail.com>
Committed: Thu Feb 9 11:16:51 2017 -0800
--
asf/spark/diff/148a84b3
Branch: refs/heads/master
Commit: 148a84b37082697c7f61c6a621010abe4b12f2eb
Parents: 63d8390
Author: Kazuaki Ishizaki <ishiz...@jp.ibm.com>
Authored: Thu Jan 19 15:16:05 2017 -0800
Committer: Davies Liu <davies@gmail.com>
Committed: Thu Jan 1
inished test(python3.6): pyspark.sql.window (4s)
Finished test(python3.6): pyspark.sql.readwriter (35s)
Tests passed in 433 seconds
```
Author: hyukjinkwon <gurwls...@gmail.com>
Closes #16429 from HyukjinKwon/SPARK-19019.
(cherry picked from commit 20e6280626fe243b170a2e7c5e018c67f3dac1db)
git-wip-us.apache.org/repos/asf/spark/commit/20e62806
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/20e62806
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/20e62806
Branch: refs/heads/master
Commit: 20e6280626fe243b170a2e7c5e018c67f3dac1db
Parents: b79cc7c
Author: hyukjinkwon <
e 2.
## How was this patch tested?
unit test
Author: Yucai Yu <yucai...@intel.com>
Closes #16555 from yucai/offheap_short.
(cherry picked from commit ad0dadaa251b031a480fc2080f792a54ed7dfc5f)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/
e 2.
## How was this patch tested?
unit test
Author: Yucai Yu <yucai...@intel.com>
Closes #16555 from yucai/offheap_short.
(cherry picked from commit ad0dadaa251b031a480fc2080f792a54ed7dfc5f)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/
asf/spark/tree/ad0dadaa
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ad0dadaa
Branch: refs/heads/master
Commit: ad0dadaa251b031a480fc2080f792a54ed7dfc5f
Parents: b0e8eb6
Author: Yucai Yu <yucai...@intel.com>
Authored: Fri Jan 13 13:40:53 2017 -0800
Committer: Davies Liu <da
..@gmail.com>
Authored: Tue Dec 20 13:12:16 2016 -0800
Committer: Davies Liu <davies@gmail.com>
Committed: Tue Dec 20 13:12:16 2016 -0800
--
python/pyspark/rdd.py | 11 +--
python/pyspark/tests.py | 12
ded tests into PySpark.
Please review http://spark.apache.org/contributing.html before opening a pull
request.
Author: Liang-Chi Hsieh <vii...@gmail.com>
Closes #16263 from viirya/fix-pyspark-localiterator.
(cherry picked from commit 95c95b71ed31b2971475aec6d7776dc234845d0a)
Signed-off-b
ded tests into PySpark.
Please review http://spark.apache.org/contributing.html before opening a pull
request.
Author: Liang-Chi Hsieh <vii...@gmail.com>
Closes #16263 from viirya/fix-pyspark-localiterator.
(cherry picked from commit 95c95b71ed31b2971475aec6d7776dc234845d0a)
Signed-off-b
*de*serialization.
## How was this patch tested?
Additional unit tests (sourced from #14248) plus one for testing a cartesian
with zip.
Author: Andrew Ray <ray.and...@gmail.com>
Closes #16121 from aray/fix-cartesian.
(cherry picked from commit 3c68944b229aaaeeaee3efcbae3e3be9a2914855)
Signed-off-b
..@gmail.com>
Authored: Thu Dec 8 11:08:12 2016 -0800
Committer: Davies Liu <davies@gmail.com>
Committed: Thu Dec 8 11:08:12 2016 -0800
--
python/pyspark/serializers.py | 58 +++---
py
0:50 2016 -0800
Committer: Davies Liu <davies@gmail.com>
Committed: Mon Dec 5 14:40:50 2016 -0800
--
docs/configuration.md | 9 +
1 file
park/diff/1e35e969
Branch: refs/heads/master
Commit: 1e35e96930dda02cb0788c8143e5f2e1944b
Parents: cd106b0
Author: Liang-Chi Hsieh <vii...@gmail.com>
Authored: Tue Oct 18 14:25:10 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Tue Oct 1
ds/master
Commit: 37686539f546ac7a3657dbfc59b7ac982b4b9bce
Parents: e59df62
Author: hyukjinkwon <gurwls...@gmail.com>
Authored: Tue Oct 18 13:20:42 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Tue Oct 1
ld Xin <r...@databricks.com>
Authored: Wed Oct 12 16:45:10 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Wed Oct 12 16:45:10 2016 -0700
--
python/pyspark/sql/tests.py | 25 +
ber of columns and
different data types. Manually test the reported case and confirmed that this
PR fix the bug.
Author: Davies Liu <dav...@databricks.com>
Closes #15390 from davies/rewrite_key.
(cherry picked from commit 94b24b84a666517e31e9c9d693f92d9bbfd7f9ad)
Signed-off-by: Davies Li
ber of columns and
different data types. Manually test the reported case and confirmed that this
PR fix the bug.
Author: Davies Liu <dav...@databricks.com>
Closes #15390 from davies/rewrite_key.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/
d: Mon Oct 3 14:12:03 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Mon Oct 3 14:12:03 2016 -0700
--
python/pyspark/java_gateway.py | 9 -
python/pyspark/ml/common.py| 4 ++--
python/pyspark
ong.
## How was this patch tested?
The flaky test should be fixed.
Author: Davies Liu <dav...@databricks.com>
Closes #15305 from davies/fix_MAP.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f327e168
Tree: http:
ome expressions are not evaluable, we should check that before
evaluate it.
## How was this patch tested?
Added regression tests.
Author: Davies Liu <dav...@databricks.com>
Closes #15103 from davies/udf_join.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http:
ome expressions are not evaluable, we should check that before
evaluate it.
## How was this patch tested?
Added regression tests.
Author: Davies Liu <dav...@databricks.com>
Closes #15103 from davies/udf_join.
(cherry picked from commit d8104158a922d86dd4f00e50d5d7dddc7b777a21)
S
xisting tests.
Author: Davies Liu <dav...@databricks.com>
Closes #15106 from davies/metric_sep.
(cherry picked from commit e0632062635c37cbc77df7ebd2a1846655193e12)
Signed-off-by:
xisting tests.
Author: Davies Liu <dav...@databricks.com>
Closes #15106 from davies/metric_sep.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.
x: len(b.value)).count()
run()
```
Before:
```
SystemError: error return without exception set
```
After:
```
cPickle.PicklingError: Could not serialize broadcast: SystemError: error return
without exception set
```
## How was this patch tested?
Manually tried out these cases
cc davies
Author
sql/tests.py` which asserts that the expected
number of jobs, stages, and tasks are run for both queries.
Author: Josh Rosen <joshro...@databricks.com>
Closes #15068 from JoshRosen/pyspark-collect-limit.
(cherry picked from commit 6d06ff6f7e2dd72ba8fe96cd875e83eda6ebb2a9)
Signed-off-by: Davies Li
om>
Authored: Wed Sep 14 10:10:01 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Wed Sep 14 10:10:01 2016 -0700
--
python/pyspark/sql/dataframe.py | 5 +
python
, I changed the TakeOrderedAndProjectExec
to no use Option[Seq[Expression]] to workaround it. cc JoshRosen
## How was this patch tested?
Added regression test.
Author: Davies Liu <dav...@databricks.com>
Closes #15030 from davies/all_expr.
Project: http://git-wip-us.apache.org/repos/asf/s
, I changed the TakeOrderedAndProjectExec
to no use Option[Seq[Expression]] to workaround it. cc JoshRosen
## How was this patch tested?
Added regression test.
Author: Davies Liu <dav...@databricks.com>
Closes #15030 from davies/all_expr.
(cherry picked fr
sts in `SQLQuerySuite`.
Author: hyukjinkwon <gurwls...@gmail.com>
Closes #14919 from HyukjinKwon/SPARK-17354.
(cherry picked from commit f7d2143705c8c1baeed0bc62940f9dba636e705b)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Sep 9 14:23:05 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Fri Sep 9 14:23:05 2016 -0700
--
.../execution/vectorized/ColumnVectorUtils.java | 5 +-
.../sql/execution/vectorized/ColumnarBatch.java | 6
park/tree/53438048
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/53438048
Branch: refs/heads/branch-2.0
Commit: 534380484ac5f56bd3e14a8917a24ca6cccf198f
Parents: 95e44dc
Author: Sameer Agarwal <samee...@cs.berkeley.edu>
Authored: Tue Sep 6 10:48:53 2016 -0700
Committer: Dav
out
Platform.LONG_ARRAY_OFFSET).
## How was this patch tested?
Added a test case with random generated keys, to improve the coverage. But this
test is not a regression test, that could require a Spark cluster that have at
least 32G heap in driver or executor.
Author: Davies Liu <dav...@databricks.com>
Closes #1
SET).
## How was this patch tested?
Added a test case with random generated keys, to improve the coverage. But this
test is not a regression test, that could require a Spark cluster that have at
least 32G heap in driver or executor.
Author: Davies Liu <dav...@databricks.com>
Closes #1
ch-2.0
Commit: c0ea7707127c92ecb51794b96ea40d7cdb28b168
Parents: b8f65da
Author: Davies Liu <davies@gmail.com>
Authored: Fri Sep 2 16:05:37 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Fri Sep 2 16:05:37 2016 -0700
--
iff: http://git-wip-us.apache.org/repos/asf/spark/diff/b8f65dad
Branch: refs/heads/branch-2.0
Commit: b8f65dad7be22231e982aaec3bbd69dbeacc20da
Parents: a3930c3
Author: Davies Liu <davies@gmail.com>
Authored: Fri Sep 2 15:40:02 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Com
er
Commit: a2c9acb0e54b2e38cb8ee6431f1ea0e0b4cd959a
Parents: ed9c884
Author: Sameer Agarwal <samee...@cs.berkeley.edu>
Authored: Fri Sep 2 15:16:16 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Fri Sep
ain.
Right now, we did not know a logical plan is optimized or not, we could
introduce a flag for that, and make sure a optimized logical plan will not be
analyzed again.
Added regression tests.
Author: Davies Liu <dav...@databricks.com>
Closes #14797 from davies/fix_writer.
(cherry pi
so they will not be optimized and
analyzed again.
Right now, we did not know a logical plan is optimized or not, we could
introduce a flag for that, and make sure a optimized logical plan will not be
analyzed again.
## How was this patch tested?
Added regression tests.
Author: Davies Liu &
/asf/spark/tree/ea662286
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ea662286
Branch: refs/heads/master
Commit: ea662286561aa9fe321cb0a0e10cdeaf60440b90
Parents: 6bcbf9b
Author: Jeff Zhang <zjf...@apache.org>
Authored: Fri Sep 2 10:08:14 2016 -0700
Committer: Davies Liu <davies
ark.sql("show databases").show()
```
Author: Jeff Zhang <zjf...@apache.org>
Closes #14857 from zjffdu/SPARK-17261.
(cherry picked from commit ea662286561aa9fe321cb0a0e10cdeaf60440b90)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/repo
refs/heads/master
Commit: 03d77af9ec4ce9a42affd6ab4381ae5bd3c79a5a
Parents: 15539e5
Author: Qifan Pu <qifan...@gmail.com>
Authored: Thu Sep 1 16:56:35 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Com
repos/asf/spark/tree/473d7864
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/473d7864
Branch: refs/heads/master
Commit: 473d78649dec7583bcc4ec24b6f38303c38e81a2
Parents: edb4573
Author: Brian Cho <b...@fb.com>
Authored: Thu Sep 1 14:13:17 2016 -0700
Committer: Davies Liu <da
How was this patch tested?
Existing unit tests.
Author: Brian Cho <b...@fb.com>
Closes #14515 from dafrista/partition-columns-metadata.
(cherry picked from commit 473d78649dec7583bcc4ec24b6f38303c38e81a2)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/
nds
in total, 2.5X faster (with larger cluster, gathering will much faster).
Author: Davies Liu <dav...@databricks.com>
Closes #14607 from davies/repair_batch.
(cherry picked from commit 48caec2516ef35bfa1a3de2dc0a80d0dc819e6bd)
Signed-off-by: Davies Liu <davies@gmail.com>
Project:
ing these
partitions took 25 seconds (most of the time spent in object store), 59 seconds
in total, 2.5X faster (with larger cluster, gathering will much faster).
Author: Davies Liu <dav...@databricks.com>
Closes #14607 from davies/repair_batch.
Project: http://git-wip-us.apache.org/repos/asf/s
der in provided schema,
this PR fix that by ignore the order of fields in this case.
Created regression tests for them.
Author: Davies Liu <dav...@databricks.com>
Closes #14469 from davies/py_dict.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.
ver, so did not add a regression
test.
Author: Davies Liu <dav...@databricks.com>
Closes #14722 from davies/keep_cause.
(cherry picked from commit 9afdfc94f49395e69a7959e881c19d787ce00c3e)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/repos
use).
## How was this patch tested?
Can't reproduce this on the default JDBC driver, so did not add a regression
test.
Author: Davies Liu <dav...@databricks.com>
Closes #14722 from davies/keep_cause.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.a
italkedia/fix_offheap_oom.
(cherry picked from commit cf0cce90364d17afe780ff9a5426dfcefa298535)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ae89c8e1
Tree: http://git-wip-us.a
er
Commit: cf0cce90364d17afe780ff9a5426dfcefa298535
Parents: 071eaaf
Author: Sital Kedia <ske...@fb.com>
Authored: Fri Aug 19 11:27:30 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Fri Aug 1
efs/heads/master
Commit: 0b0c8b95e3594db36d87ef0e59a30eefe8508ac1
Parents: 56d8674
Author: Herman van Hovell <hvanhov...@databricks.com>
Authored: Wed Aug 17 07:03:24 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Wed Aug 1
ark/tree/12a89e55
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/12a89e55
Branch: refs/heads/master
Commit: 12a89e55cbd630fa2986da984e066cd07d3bf1f7
Parents: 6f0988b
Author: Dongjoon Hyun <dongj...@apache.org>
Authored: Tue Aug 1
-subquery](https://cloud.githubusercontent.com/assets/40902/17573229/e578d93c-5f0d-11e6-8a3c-0150d81d3aed.png)
## How was this patch tested?
Existing tests.
Author: Davies Liu <dav...@databricks.com>
Closes #14548 from davies/subq.
Project: http://git-wip-us.apache.org/repos/asf/spark/re
Pu <qifan...@gmail.com>
Authored: Wed Aug 10 14:45:13 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Wed Aug 10 14:45:13 2016 -0700
--
.../parquet/VectorizedColumnReader.java | 22 +
om viirya/vectorized-reader-push-down-filter2.
(cherry picked from commit 19af298bb6d264adcf02f6f84c8dc1542b408507)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/977fbbfc
Tree: http://git-wi
b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/19af298b
Branch: refs/heads/master
Commit: 19af298bb6d264adcf02f6f84c8dc1542b408507
Parents: 11a6844
Author: Liang-Chi Hsieh <sim...@tw.ibm.com>
Authored: Wed Aug 10 10:03:55 2016 -0700
Commit
ONS
The implementation in this PR will only list partitions (not the files with a
partition) in driver (in parallel if needed).
Added unit tests for it and Hive compatibility test suite.
Author: Davies Liu <dav...@databricks.com>
Closes #14500 from davies/repair_table.
Project: http:
tem.
Another syntax is: ALTER TABLE table RECOVER PARTITIONS
The implementation in this PR will only list partitions (not the files with a
partition) in driver (in parallel if needed).
## How was this patch tested?
Added unit tests for it and Hive compatibility test suite.
Author: Davies Liu &
hor: Mariusz Strzelecki <mariusz.strzele...@allegrogroup.com>
Closes #14540 from szczeles/kafka_pyspark.
(cherry picked from commit 29081b587f3423bf5a3e0066357884d0c26a04bf)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
iff: http://git-wip-us.apache.org/repos/asf/spark/diff/29081b58
Branch: refs/heads/master
Commit: 29081b587f3423bf5a3e0066357884d0c26a04bf
Parents: 182e119
Author: Mariusz Strzelecki <mariusz.strzele...@allegrogroup.com>
Authored: Tue Aug 9 09:44:43 2016 -0700
Committer: Davies Liu <da
n't necessarily depend on an existing RDD. cc davies
## How was this patch tested?
Existing tests.
Author: Eric Liang <e...@databricks.com>
Closes #14487 from ericl/split-scan.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit
uch smaller then minKey, for example, key is
Long.MinValue, minKey is > 0).
## How was this patch tested?
Added regression test (also for SPARK-16740)
Author: Davies Liu <dav...@databricks.com>
Closes #14464 from davies/fix_overflow.
Project: http://git-wip-us.apache.org/repos/as
uch much smaller then minKey, for example, key is
Long.MinValue, minKey is > 0).
## How was this patch tested?
Added regression test (also for SPARK-16740)
Author: Davies Liu <dav...@databricks.com>
Closes #14464 from davies/fix_overflow.
(cherry picked
repos/asf/spark/tree/e6f226c5
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e6f226c5
Branch: refs/heads/master
Commit: e6f226c5670d9f332b49ca40ff7b86b81a218d1b
Parents: b55f343
Author: Eric Liang <e...@databricks.com>
Authored: Wed Aug 3 11:19:55 2016 -0700
Committer: Dav
teDataFrame([(i % 3, [PythonOnlyPoint(float(i), float(i))])
for i in range(10)], schema=schema)
df.show()
## How was this patch tested?
PySpark's sql tests.
Author: Liang-Chi Hsieh <sim...@tw.ibm.com>
Closes #13778 from viirya/fix-pyudt.
(cherry picked from commit 146001a9ffefc7aa
d
Author: Liang-Chi Hsieh <sim...@tw.ibm.com>
Authored: Tue Aug 2 10:08:18 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Tue Aug 2 10:08:18 2016 -0700
--
python/pyspark/sql/tests.py
ort failing on large
buffer size.
Tested by running a job which was failing without this change due to TimSort
bug.
Author: Sital Kedia <ske...@fb.com>
Closes #13107 from sitalkedia/fix_TimSort.
(cherry picked from commit 07f46afc733b1718d528a6ea5c0d774f047024fa)
Signed-off-by: Davies Li
er
Commit: 07f46afc733b1718d528a6ea5c0d774f047024fa
Parents: 5344bad
Author: Sital Kedia <ske...@fb.com>
Authored: Thu Jun 30 10:53:18 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Thu Jun 30 10:53:18 2016 -0700
---
uld respect the case sensitivity
setting.
## How was this patch tested?
New tests in ResolveNaturalJoinSuite
Author: Yin Huai <yh...@databricks.com>
Closes #13977 from yhuai/SPARK-16301.
(cherry picked from commit 8b5a8b25b9d29b7d0949d5663c7394b26154a836)
Signed-off-by: Davies Li
: Wed Jun 29 14:42:58 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Wed Jun 29 14:42:58 2016 -0700
--
.../spark/sql/catalyst/analysis/Analyzer.scala | 26 -
.../analysis/ResolveN
nly in https://github.com/apache/spark/pull/13948.
## How was this patch tested?
N/A
Author: hyukjinkwon <gurwls...@gmail.com>
Closes #13963 from HyukjinKwon/minor-orc-compress.
(cherry picked from commit d8a87a3ed211dd08f06eeb9560661b8f11ce82fa)
Signed-off-by: Davies Liu <davies..
Jun 29 13:32:03 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Wed Jun 29 13:32:03 2016 -0700
--
python/pyspark/sql/readwriter.py | 3 +--
1 file changed, 1 inse
ate the Python UDT to do this as well.
## How was this patch tested?
Added tests.
Author: Davies Liu <dav...@databricks.com>
Closes #13878 from davies/udt_null.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/35438fb0
T
PR update the Python UDT to do this as well.
## How was this patch tested?
Added tests.
Author: Davies Liu <dav...@databricks.com>
Closes #13878 from davies/udt_null.
(cherry picked from commit 35438fb0ad3bcda5c5a3a0ccde1a620699d012db)
Signed-off-by: Davies Liu <davies@gmail.com>
om commit 0923c4f5676691e28e70ecb05890e123540b91f0)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4c5e16f5
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/4c5e16f5
Diff: http://git-wip-us.a
78
Author: Yin Huai <yh...@databricks.com>
Authored: Tue Jun 28 07:54:44 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Tue Jun 28 07:54:44 2016 -0700
--
python/pyspark/context.py | 2 +
park/diff/576265f8
Branch: refs/heads/branch-1.5
Commit: 576265f838fe74879b5f6b0367f759d1c37c9468
Parents: 6001138
Author: Dongjoon Hyun <dongj...@apache.org>
Authored: Fri Jun 24 22:30:52 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Fri Jun 2
9499
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a7d29499
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a7d29499
Branch: refs/heads/master
Commit: a7d29499dca5b86e776abc225ece84391f09353a
Parents: d2e44d7
Author: Dongjoon Hyun <dongj...@apache.org>
Authored: Fri Jun 24 17:13:13 2
park/diff/b7acc1b7
Branch: refs/heads/branch-1.6
Commit: b7acc1b71c5d4b163a7451e8c6430afe920a04e0
Parents: d7223bb
Author: Dongjoon Hyun <dongj...@apache.org>
Authored: Fri Jun 24 22:30:52 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Fri Jun 2
es #13900 from dongjoon-hyun/SPARK-16173.
(cherry picked from commit e5d0928e2473d1838ff5420c6a8964557c33135e)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9de09551
T
e
Branch: refs/heads/master
Commit: e5d0928e2473d1838ff5420c6a8964557c33135e
Parents: 20768da
Author: Dongjoon Hyun <dongj...@apache.org>
Authored: Fri Jun 24 17:26:39 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Fri Jun 2
bce6e
Parents: a65bcbc
Author: Davies Liu <davies@gmail.com>
Authored: Fri Jun 24 17:21:18 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Fri Jun 24 17:21:18 2016 -0700
--
.../columnar/In
cbc2
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a65bcbc2
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a65bcbc2
Branch: refs/heads/master
Commit: a65bcbc27dcd9b3053cb13c5d67251c8d48f4397
Parents: 4435de1
Author: Dongjoon Hyun <dongj...@apache.org>
Authored: Fri Jun 24 17:13:13 2
ind the module).
## How was this patch tested?
Manual tested. Can't have a unit test for this.
Author: Davies Liu <dav...@databricks.com>
Closes #13788 from davies/whichmodule.
(cherry picked from commit d48935400ca47275f677b527c636976af09332c8)
Signed-off-by: Davies Liu <davies..
ind the module).
## How was this patch tested?
Manual tested. Can't have a unit test for this.
Author: Davies Liu <dav...@databricks.com>
Closes #13788 from davies/whichmodule.
(cherry picked from commit d48935400ca47275f677b527c636976af09332c8)
Signed-off-by: Davies Liu <davies..
t's only useful when used in another query (before planning), because once we
finished the planning, the statistics will not be used anymore.
## How was this patch tested?
Testsed with TPC-DS Q64, it could be planned in a second after the patch.
Author: Davies Liu <dav...@databricks.com>
nly useful when used in another query (before planning), because once we
finished the planning, the statistics will not be used anymore.
## How was this patch tested?
Testsed with TPC-DS Q64, it could be planned in a second after the patch.
Author: Davies Liu <dav...@databricks.com>
Clos
cc davies cloud-fan
## How was this patch tested?
Unit tests for SerializationDebugger.
Author: Eric Liang <e...@databricks.com>
Closes #13814 from ericl/spark-16003.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit
ava
cc davies cloud-fan
## How was this patch tested?
Unit tests for SerializationDebugger.
Author: Eric Liang <e...@databricks.com>
Closes #13814 from ericl/spark-16003.
(cherry picked from commit 6f915c9ec24003877d1ef675a59145699780a2ff)
Signed-off-by: Davies Liu <davies@gmail.co
er
Commit: 7580f3041a1a3757a0b14b9d8afeb720f261fff6
Parents: d77c4e6
Author: hyukjinkwon <gurwls...@gmail.com>
Authored: Tue Jun 21 21:58:38 2016 -0700
Committer: Davies Liu <davies@gmail.com>
Committed: Tue Jun 2
est for empty row and udf without arguments.
Author: Davies Liu <dav...@databricks.com>
Closes #13812 from davies/no_argus.
(cherry picked from commit 2d6919bea9fc213b5af530afab7793b63c6c8b51)
Signed-off-by: Davies Liu <davies@gmail.com>
Project: http://git-wip-us.apache.org/
est for empty row and udf without arguments.
Author: Davies Liu <dav...@databricks.com>
Closes #13812 from davies/no_argus.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2d6919be
Tree: http://git-wip-us.apache.org/repos/
1 - 100 of 716 matches
Mail list logo
