[jira] [Comment Edited] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253044#comment-17253044 ] Nicholas Chammas edited comment on SPARK-12890 at 1/14/21, 5:41 PM

[jira] [Reopened] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas reopened SPARK-12890: -- Reopening because I think there is a valid potential improvement to be made here

[jira] [Updated] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-12890: - Labels: (was: bulk-closed) > Spark SQL query related to only partition fields sho

[jira] [Updated] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-12890: - Priority: Minor (was: Major) > Spark SQL query related to only partition fields sho

[jira] [Commented] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17253044#comment-17253044 ] Nicholas Chammas commented on SPARK-12890: -- I think this is still an open issue. On Spark 2.4.6

[jira] [Updated] (SPARK-33436) PySpark equivalent of SparkContext.hadoopConfiguration

[ https://issues.apache.org/jira/browse/SPARK-33436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-33436: - Description: PySpark should offer an API to {{hadoopConfiguration}} to [match Scala's

[jira] [Updated] (SPARK-33436) PySpark equivalent of SparkContext.hadoopConfiguration

[ https://issues.apache.org/jira/browse/SPARK-33436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-33436: - Description: PySpark should offer an API to {{hadoopConfiguration}} to [match Scala's

[jira] [Created] (SPARK-33436) PySpark equivalent of SparkContext.hadoopConfiguration

Nicholas Chammas created SPARK-33436: Summary: PySpark equivalent of SparkContext.hadoopConfiguration Key: SPARK-33436 URL: https://issues.apache.org/jira/browse/SPARK-33436 Project: Spark

[jira] [Updated] (SPARK-33434) Document spark.conf.isModifiable()

[ https://issues.apache.org/jira/browse/SPARK-33434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-33434: - Affects Version/s: (was: 2.4.7) (was: 3.0.1

[jira] [Updated] (SPARK-33434) Document spark.conf.isModifiable()

[ https://issues.apache.org/jira/browse/SPARK-33434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-33434: - Affects Version/s: 3.0.1 > Document spark.conf.isModifia

[jira] [Updated] (SPARK-33434) Document spark.conf.isModifiable()

[ https://issues.apache.org/jira/browse/SPARK-33434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-33434: - Description: PySpark's docs make no mention of {{conf.isModifiable()}}, though

[jira] [Created] (SPARK-33434) Document spark.conf.isModifiable()

Nicholas Chammas created SPARK-33434: Summary: Document spark.conf.isModifiable() Key: SPARK-33434 URL: https://issues.apache.org/jira/browse/SPARK-33434 Project: Spark Issue Type

Re: [DISCUSS][SPIP] Standardize Spark Exception Messages

Just want to call out that this SPIP should probably account somehow for PySpark and the work being done in SPARK-32082 to improve PySpark exceptions. On Sun, Oct 25, 2020 at 8:05 PM Xinyi Yu wrote: > Hi all, > > We like to post a SPIP of

[jira] [Commented] (SPARK-26764) [SPIP] Spark Relational Cache

[ https://issues.apache.org/jira/browse/SPARK-26764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218374#comment-17218374 ] Nicholas Chammas commented on SPARK-26764: -- The SPIP PDF references a design doc, but I'm

[jira] [Commented] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215577#comment-17215577 ] Nicholas Chammas commented on SPARK-33000: -- Ctrl-D gracefully shuts down the Python REPL, so

[jira] [Commented] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215469#comment-17215469 ] Nicholas Chammas commented on SPARK-33000: -- I've tested this out a bit more, and I think

[jira] [Commented] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215085#comment-17215085 ] Nicholas Chammas commented on SPARK-33000: -- Thanks for the explanation! I'm happy to leave

[jira] [Commented] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214904#comment-17214904 ] Nicholas Chammas commented on SPARK-33000: -- Thanks for the pointer! No need for a new ticket

[jira] [Created] (SPARK-33017) PySpark Context should have getCheckpointDir() method

Nicholas Chammas created SPARK-33017: Summary: PySpark Context should have getCheckpointDir() method Key: SPARK-33017 URL: https://issues.apache.org/jira/browse/SPARK-33017 Project: Spark

[jira] [Updated] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-33000: - Description: Maybe it's just that the documentation needs to be updated, but I found

Re: get method guid prefix for file parts for write

I think what George is looking for is a way to determine ahead of time the partition IDs that Spark will use when writing output. George, I believe this is an example of what you're looking for:

[jira] [Created] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

Nicholas Chammas created SPARK-33000: Summary: cleanCheckpoints config does not clean all checkpointed RDDs on shutdown Key: SPARK-33000 URL: https://issues.apache.org/jira/browse/SPARK-33000

[jira] [Commented] (SPARK-32084) Replace dictionary-based function definitions to proper functions in functions.py

[ https://issues.apache.org/jira/browse/SPARK-32084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187364#comment-17187364 ] Nicholas Chammas commented on SPARK-32084: -- Can you share a couple of examples of what you

[jira] [Updated] (SPARK-31167) Refactor how we track Python test/build dependencies

[ https://issues.apache.org/jira/browse/SPARK-31167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31167: - Description: Ideally, we should have a single place to track Python development

[jira] [Updated] (SPARK-31167) Refactor how we track Python test/build dependencies

[ https://issues.apache.org/jira/browse/SPARK-31167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31167: - Description: Ideally, we should have a single place to track Python development

PySpark: Un-deprecating inferring DataFrame schema from list of dictionaries

https://github.com/apache/spark/pull/29510 I don't think this is a big deal, but since we're removing a deprecation that has been around for ~6 years, I figured it would be good to bring everyone's attention to this change. Hopefully, we are not breaking any hidden assumptions about the

[jira] [Created] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

Nicholas Chammas created SPARK-32686: Summary: Un-deprecate inferring DataFrame schema from list of dictionaries Key: SPARK-32686 URL: https://issues.apache.org/jira/browse/SPARK-32686 Project

[jira] [Commented] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

[ https://issues.apache.org/jira/browse/SPARK-32686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182040#comment-17182040 ] Nicholas Chammas commented on SPARK-32686: -- Not sure if I have the "Affects Version"

Re: Use Hadoop-3.2 as a default Hadoop profile in 3.0.0?

To rephrase my earlier email, PyPI users would care about the bundled Hadoop version if they have a workflow that, in effect, looks something like this: ``` pip install pyspark pyspark --packages org.apache.hadoop:hadoop-aws:2.7.7 spark.read.parquet('s3a://...') ``` I agree that Hadoop 3 would

Re: Use Hadoop-3.2 as a default Hadoop profile in 3.0.0?

The team I'm on currently uses pip-installed PySpark for local development, and we regularly access S3 directly from our laptops/workstations. One of the benefits of having Spark built against Hadoop 3.2 vs. 2.7 is being able to use a recent version of hadoop-aws that has mature support for s3a.

Re: [VOTE] Release Spark 2.4.6 (RC8)

I believe that was fixed in 3.0 and there was a decision not to backport the fix: SPARK-31170 On Wed, Jun 3, 2020 at 1:04 PM Xiao Li wrote: > Just downloaded it in my local macbook. Trying to create a table using the > pre-built PySpark. It

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

wrote: > Maybe it's time to switch. Do you know if we can still link the JIRA > against Github? > The script used to change the status of JIRA too but it stopped working > for a long time - I suspect this isn't a big deal. > > 2020년 4월 25일 (토) 오전 10:31, Nicholas Chammas 님이 > 작

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Have we asked Infra recently about enabling the native Jira-GitHub integration ? Maybe we can deprecate the part of this script that updates Jira tickets with links to the PR and rely on the native

[jira] [Commented] (SPARK-27623) Provider org.apache.spark.sql.avro.AvroFileFormat could not be instantiated

[ https://issues.apache.org/jira/browse/SPARK-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089917#comment-17089917 ] Nicholas Chammas commented on SPARK-27623: -- Is this perhaps just a documentation issue? i.e

[jira] [Commented] (SPARK-31170) Spark Cli does not respect hive-site.xml and spark.sql.warehouse.dir

[ https://issues.apache.org/jira/browse/SPARK-31170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088997#comment-17088997 ] Nicholas Chammas commented on SPARK-31170: -- Isn't this also an issue in Spark 2.4.5? {code:java

[jira] [Commented] (SPARK-31330) Automatically label PRs based on the paths they touch

[ https://issues.apache.org/jira/browse/SPARK-31330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074217#comment-17074217 ] Nicholas Chammas commented on SPARK-31330: -- Hmm, I didn't see anything from you on the mailing

[jira] [Commented] (SPARK-31330) Automatically label PRs based on the paths they touch

[ https://issues.apache.org/jira/browse/SPARK-31330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074124#comment-17074124 ] Nicholas Chammas commented on SPARK-31330: -- Unfortunately, it seems I jumped the gun on sending

Beginner PR against the Catalog API

I recently submitted my first Scala PR. It's very simple, though I don't know if I've done things correctly since I'm not a regular Scala user. SPARK-31000 : Add ability to set table description in the catalog

Re: Automatic PR labeling

SPARK-31330 <https://issues.apache.org/jira/browse/SPARK-31330>: Automatically label PRs based on the paths they touch On Wed, Apr 1, 2020 at 11:34 PM Hyukjin Kwon wrote: > @Nicholas Chammas Would you be interested in > tacking a look? I would love this to be done. > > 2020

[jira] [Created] (SPARK-31330) Automatically label PRs based on the paths they touch

Nicholas Chammas created SPARK-31330: Summary: Automatically label PRs based on the paths they touch Key: SPARK-31330 URL: https://issues.apache.org/jira/browse/SPARK-31330 Project: Spark

Re: [DISCUSS] filling affected versions on JIRA issue

Probably the discussion here about Improvement Jira tickets and the "Affects Version" field: https://github.com/apache/spark/pull/27534#issuecomment-588416416 On Wed, Apr 1, 2020 at 9:59 PM Hyukjin Kwon wrote: > > 2) check with older versions to fill up affects version for bug > I don't agree

Re: Release Manager's official `branch-3.0` Assessment?

I don't have a dog in this race, but: Would it be OK to ship 3.0 with some release notes and/or prominent documentation calling out this issue, and then fixing it in 3.0.1? On Sat, Mar 28, 2020 at 8:45 PM Jungtaek Lim wrote: > I'd say SPARK-31257 as open blocker, because the change in upcoming

Automatic PR labeling

Public Service Announcement: There is a GitHub action that lets you automatically label PRs based on what paths they modify. https://github.com/actions/labeler If we set this up, perhaps down the line we can update the PR dashboard and PR merge script to use the tags. cc @Dongjoon Hyun , who

[jira] [Commented] (HADOOP-16930) Add com.amazonaws.auth.profile.ProfileCredentialsProvider to hadoop-aws docs

[ https://issues.apache.org/jira/browse/HADOOP-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063734#comment-17063734 ] Nicholas Chammas commented on HADOOP-16930: --- cc [~ste...@apache.org] - I'd be happy to work

[jira] [Created] (HADOOP-16930) Add com.amazonaws.auth.profile.ProfileCredentialsProvider to hadoop-aws docs

Nicholas Chammas created HADOOP-16930: - Summary: Add com.amazonaws.auth.profile.ProfileCredentialsProvider to hadoop-aws docs Key: HADOOP-16930 URL: https://issues.apache.org/jira/browse/HADOOP-16930

[jira] [Created] (HADOOP-16930) Add com.amazonaws.auth.profile.ProfileCredentialsProvider to hadoop-aws docs

Nicholas Chammas created HADOOP-16930: - Summary: Add com.amazonaws.auth.profile.ProfileCredentialsProvider to hadoop-aws docs Key: HADOOP-16930 URL: https://issues.apache.org/jira/browse/HADOOP-16930

Re: [DISCUSS] Resolve ambiguous parser rule between two "create table"s

On Thu, Mar 19, 2020 at 3:46 AM Wenchen Fan wrote: > 2. PARTITIONED BY colTypeList: I think we can support it in the unified > syntax. Just make sure it doesn't appear together with PARTITIONED BY > transformList. > Another side note: Perhaps as part of (or after) unifying the CREATE TABLE

Re: [DISCUSS] Resolve ambiguous parser rule between two "create table"s

Side comment: The current docs for CREATE TABLE add to the confusion by describing the Hive-compatible command as "CREATE TABLE USING HIVE FORMAT", but neither

Re-triggering failed GitHub workflows

Is there any way contributors can retrigger a failed GitHub workflow, like we do with Jenkins? There's supposed to be a "Re-run all checks" button, but I don't see it. Do we need INFRA to grant permissions for that, perhaps? Right now I'm doing it by adding empty commits: ``` git commit

[jira] [Updated] (SPARK-31167) Refactor how we track Python test/build dependencies

[ https://issues.apache.org/jira/browse/SPARK-31167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31167: - Summary: Refactor how we track Python test/build dependencies (was: Refactor how we

[jira] [Updated] (SPARK-31167) Refactor how we track Python test dependencies

[ https://issues.apache.org/jira/browse/SPARK-31167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31167: - Summary: Refactor how we track Python test dependencies (was: Specify missing test

[jira] [Created] (SPARK-31167) Specify missing test dependencies

Nicholas Chammas created SPARK-31167: Summary: Specify missing test dependencies Key: SPARK-31167 URL: https://issues.apache.org/jira/browse/SPARK-31167 Project: Spark Issue Type

[jira] [Updated] (SPARK-31155) Remove pydocstyle tests

[ https://issues.apache.org/jira/browse/SPARK-31155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31155: - Summary: Remove pydocstyle tests (was: Enable pydocstyle tests) > Remove pydocst

[jira] [Updated] (SPARK-31155) Remove pydocstyle tests

[ https://issues.apache.org/jira/browse/SPARK-31155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31155: - Description: pydocstyle tests have been running neither on Jenkins nor on Github. We

[jira] [Updated] (SPARK-29280) DataFrameReader should support a compression option

[ https://issues.apache.org/jira/browse/SPARK-29280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-29280: - Affects Version/s: (was: 3.0.0) 3.1.0 > DataFrameRea

[jira] [Created] (SPARK-31155) Enable pydocstyle tests

Nicholas Chammas created SPARK-31155: Summary: Enable pydocstyle tests Key: SPARK-31155 URL: https://issues.apache.org/jira/browse/SPARK-31155 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-31153) Cleanup several failures in lint-python

Nicholas Chammas created SPARK-31153: Summary: Cleanup several failures in lint-python Key: SPARK-31153 URL: https://issues.apache.org/jira/browse/SPARK-31153 Project: Spark Issue Type

Re: Running Spark through a debugger

What are you having trouble with, does it build? > > On Mon, Dec 16, 2019 at 11:27 PM Nicholas Chammas > wrote: > > > > I normally stick to the Python parts of Spark, but I am interested in > walking through the DSv2 code and understanding how it works. I tried > fol

[issue20039] Missing documentation for argparse.ArgumentTypeError

Nicholas Chammas added the comment: Just a note that I also went looking for the docs for argparse.ArgumentTypeError after coming across it in this (highly viewed) post: https://stackoverflow.com/a/14117511/877069 -- nosy: +nchammas ___ Python

Re: Auto-linking from PRs to Jira tickets

Could you point us to the ticket? I'd like to follow along. On Tue, Mar 10, 2020 at 9:13 AM Alex Ott wrote: > For Zeppelin I've created recently the ASF INFRA Jira for that feature... > Although maybe it should be done for all projects. > > Nicholas Chammas at "Mon, 9 Mar 20

Re: Auto-linking from PRs to Jira tickets

t; > On Mon, Mar 9, 2020 at 2:14 PM Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> This is a feature of GitHub itself and would auto-link directly from the >> PR back to Jira. >> >> I haven't looked at the PR dashboard in a while, but I believe

Re: Auto-linking from PRs to Jira tickets

that runs the PR dashboard, > is it no longer working? > > On Mon, Mar 9, 2020 at 12:28 PM Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> https://github.blog/2019-10-14-introducing-autolink-references/ >> >> GitHub has a feature for auto-li

Auto-linking from PRs to Jira tickets

https://github.blog/2019-10-14-introducing-autolink-references/ GitHub has a feature for auto-linking from PRs to external tickets. It's only available for their paid plans, but perhaps Apache has some arrangement with them where we can get that feature. Since we include Jira ticket numbers in

[jira] [Resolved] (SPARK-31075) Add documentation for ALTER TABLE ... ADD PARTITION

[ https://issues.apache.org/jira/browse/SPARK-31075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-31075. -- Resolution: Duplicate > Add documentation for ALTER TABLE ... ADD PARTIT

[jira] [Commented] (SPARK-31043) Spark 3.0 built against hadoop2.7 can't start standalone master

[ https://issues.apache.org/jira/browse/SPARK-31043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054676#comment-17054676 ] Nicholas Chammas commented on SPARK-31043: -- It's working for me now (per my comment), but when

[jira] [Commented] (SPARK-31065) Empty string values cause schema_of_json() to return a schema not usable by from_json()

[ https://issues.apache.org/jira/browse/SPARK-31065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054674#comment-17054674 ] Nicholas Chammas commented on SPARK-31065: -- Thanks for looking into it. I have a silly

[jira] [Created] (SPARK-31075) Add documentation for ALTER TABLE ... ADD PARTITION

Nicholas Chammas created SPARK-31075: Summary: Add documentation for ALTER TABLE ... ADD PARTITION Key: SPARK-31075 URL: https://issues.apache.org/jira/browse/SPARK-31075 Project: Spark

[jira] [Updated] (SPARK-31041) Show Maven errors from within make-distribution.sh

[ https://issues.apache.org/jira/browse/SPARK-31041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31041: - Description: This works: {code:java} ./dev/make-distribution.sh \ --pip \ -Phadoop

[jira] [Updated] (SPARK-31041) Show Maven errors from within make-distribution.sh

[ https://issues.apache.org/jira/browse/SPARK-31041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31041: - Summary: Show Maven errors from within make-distribution.sh (was: Make arguments

[jira] [Commented] (SPARK-31043) Spark 3.0 built against hadoop2.7 can't start standalone master

[ https://issues.apache.org/jira/browse/SPARK-31043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053080#comment-17053080 ] Nicholas Chammas commented on SPARK-31043: -- FWIW I was seeing the same

[jira] [Commented] (SPARK-31065) Empty string values cause schema_of_json() to return a schema not usable by from_json()

[ https://issues.apache.org/jira/browse/SPARK-31065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053079#comment-17053079 ] Nicholas Chammas commented on SPARK-31065: -- Confirmed this issue is also present on {{branch

[jira] [Updated] (SPARK-31065) Empty string values cause schema_of_json() to return a schema not usable by from_json()

[ https://issues.apache.org/jira/browse/SPARK-31065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31065: - Affects Version/s: 3.0.0 > Empty string values cause schema_of_json() to ret

[jira] [Commented] (SPARK-31065) Empty string values cause schema_of_json() to return a schema not usable by from_json()

[ https://issues.apache.org/jira/browse/SPARK-31065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17053052#comment-17053052 ] Nicholas Chammas commented on SPARK-31065: -- cc [~hyukjin.kwon] > Empty string values ca

[jira] [Created] (SPARK-31065) Empty string values cause schema_of_json() to return a schema not usable by from_json()

Nicholas Chammas created SPARK-31065: Summary: Empty string values cause schema_of_json() to return a schema not usable by from_json() Key: SPARK-31065 URL: https://issues.apache.org/jira/browse/SPARK-31065

[jira] [Updated] (SPARK-31041) Make arguments to make-distribution.sh position-independent

[ https://issues.apache.org/jira/browse/SPARK-31041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31041: - Description: This works: {code:java} ./dev/make-distribution.sh \ --pip \ -Phadoop

[jira] [Updated] (SPARK-31041) Make arguments to make-distribution.sh position-independent

[ https://issues.apache.org/jira/browse/SPARK-31041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31041: - Description: This works: ``` ./dev/make-distribution.sh \ --pip \ -Phadoop-2.7

[jira] [Updated] (SPARK-31041) Make arguments to make-distribution.sh position-independent

[ https://issues.apache.org/jira/browse/SPARK-31041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31041: - Description: This works:   ``` ./dev/make-distribution.sh \ --pip \ -Phadoop-2.7

[jira] [Updated] (SPARK-31041) Make arguments to make-distribution.sh position-independent

[ https://issues.apache.org/jira/browse/SPARK-31041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-31041: - Summary: Make arguments to make-distribution.sh position-independent (was: Make

[jira] [Created] (SPARK-31041) Make argument to make-distribution position-independent

Nicholas Chammas created SPARK-31041: Summary: Make argument to make-distribution position-independent Key: SPARK-31041 URL: https://issues.apache.org/jira/browse/SPARK-31041 Project: Spark

[jira] [Created] (SPARK-31001) Add ability to create a partitioned table via catalog.createTable()

Nicholas Chammas created SPARK-31001: Summary: Add ability to create a partitioned table via catalog.createTable() Key: SPARK-31001 URL: https://issues.apache.org/jira/browse/SPARK-31001 Project

[jira] [Created] (SPARK-31000) Add ability to set table description in the catalog

Nicholas Chammas created SPARK-31000: Summary: Add ability to set table description in the catalog Key: SPARK-31000 URL: https://issues.apache.org/jira/browse/SPARK-31000 Project: Spark

Re: [DISCUSSION] Avoiding duplicate work

+1 to what Sean said. On Fri, Feb 21, 2020 at 10:14 AM Sean Owen wrote: > We've avoided using Assignee because it implies that someone 'owns' > resolving the issue, when we want to keep it collaborative, and many > times in the past someone would ask to be assigned and then didn't > follow

[jira] [Resolved] (SPARK-30838) Add missing pages to documentation index

[ https://issues.apache.org/jira/browse/SPARK-30838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas resolved SPARK-30838. -- Resolution: Won't Fix > Add missing pages to documentation in

[jira] [Commented] (SPARK-30838) Add missing pages to documentation index

[ https://issues.apache.org/jira/browse/SPARK-30838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039351#comment-17039351 ] Nicholas Chammas commented on SPARK-30838: -- Actually, it looks like the pages I wanted to add

[jira] [Updated] (SPARK-30838) Add missing pages to documentation index

[ https://issues.apache.org/jira/browse/SPARK-30838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30838: - Summary: Add missing pages to documentation index (was: Add missing pages

[jira] [Updated] (SPARK-30838) Add missing pages to documentation top navigation menu

[ https://issues.apache.org/jira/browse/SPARK-30838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30838: - Description: There are a few pages tracked in `docs/` that are not linked to from

[jira] [Updated] (SPARK-30838) Add missing pages to documentation top navigation menu

[ https://issues.apache.org/jira/browse/SPARK-30838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30838: - Description: There are a few pages tracked in `docs/` that are not linked to from

[jira] [Updated] (SPARK-30838) Add missing pages to documentation top navigation menu

[ https://issues.apache.org/jira/browse/SPARK-30838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30838: - Description: There are a few pages tracked in `docs/` that are not linked to from

[jira] [Created] (SPARK-30838) Add missing pages to documentation top navigation menu

Nicholas Chammas created SPARK-30838: Summary: Add missing pages to documentation top navigation menu Key: SPARK-30838 URL: https://issues.apache.org/jira/browse/SPARK-30838 Project: Spark

[jira] [Created] (SPARK-30832) SQL function doc headers should link to anchors

Nicholas Chammas created SPARK-30832: Summary: SQL function doc headers should link to anchors Key: SPARK-30832 URL: https://issues.apache.org/jira/browse/SPARK-30832 Project: Spark

[jira] [Created] (SPARK-30731) Refine doc-building workflow

Nicholas Chammas created SPARK-30731: Summary: Refine doc-building workflow Key: SPARK-30731 URL: https://issues.apache.org/jira/browse/SPARK-30731 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-30510) Publicly document options under spark.sql.*

[ https://issues.apache.org/jira/browse/SPARK-30510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30510: - Description: SPARK-20236 added a new option, {{spark.sql.sources.partitionOverwriteMode

[jira] [Updated] (SPARK-30510) Publicly document options under spark.sql.*

[ https://issues.apache.org/jira/browse/SPARK-30510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30510: - Summary: Publicly document options under spark.sql.* (was: Document

[jira] [Updated] (SPARK-30665) Eliminate pypandoc dependency

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-30665: - Summary: Eliminate pypandoc dependency (was: Remove Pandoc dependency in PySpark

[jira] [Created] (SPARK-30672) numpy is a dependency for building PySpark API docs

Nicholas Chammas created SPARK-30672: Summary: numpy is a dependency for building PySpark API docs Key: SPARK-30672 URL: https://issues.apache.org/jira/browse/SPARK-30672 Project: Spark

[jira] [Commented] (SPARK-30665) Remove Pandoc dependency in PySpark setup.py

[ https://issues.apache.org/jira/browse/SPARK-30665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026399#comment-17026399 ] Nicholas Chammas commented on SPARK-30665: --  > Remove Pandoc dependency in PySpark setup

[jira] [Created] (SPARK-30665) Remove Pandoc dependency in PySpark setup.py

Nicholas Chammas created SPARK-30665: Summary: Remove Pandoc dependency in PySpark setup.py Key: SPARK-30665 URL: https://issues.apache.org/jira/browse/SPARK-30665 Project: Spark Issue

Re: More publicly documenting the options under spark.sql.*

; experimental option that may change, or legacy, or safety valve flag. >>>>> Certainly anything that's marked an internal conf. (That does raise >>>>> the question of who it's for, if you have to read source to find it.) >>>>> >>>>> I don't

Re: Closing stale PRs with a GitHub Action

Apr+label%3AStale+is%3Aclosed> is how many PRs are active with relatively recent activity. It's a testament to how active this project is. On Sun, Dec 15, 2019 at 11:16 AM Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > Just an FYI to everyone, we’ve merged in an Action to

[jira] [Commented] (SPARK-19248) Regex_replace works in 1.6 but not in 2.0

[ https://issues.apache.org/jira/browse/SPARK-19248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022576#comment-17022576 ] Nicholas Chammas commented on SPARK-19248: -- Thanks for getting to the bottom of the issue

<    1   2   3   4   5   6   7   8   9   10   >