SPARK-30098 was merged about 6 months ago. It's not a clean revert and we
may need to spend quite a bit of time to resolve conflicts and fix tests.
I don't see why it's still a problem if a feature is disabled and hidden
from end-users (it's undocumented, the config is internal). The related
code
Hi all,
Our quarterly project board report needs to be submitted on May 13th, and I
wanted to include anything notable going on that we want to appear in the board
archive. Here is my draft below — let me know if you have suggested changes.
===
I'm all for getting the unified syntax into master. The only issue appears
to be whether or not to pass the presence of the EXTERNAL keyword through
to a catalog in v2. Maybe it's time to start a discuss thread for that
issue so we're not stuck for another 6 weeks on it.
On Mon, May 11, 2020 at
unsubscribe
--
Chenguang He
Btw another wondering here is, is it good to retain the flag on master as
an intermediate step? Wouldn't it be better for us to start "unified create
table syntax" from scratch?
On Tue, May 12, 2020 at 6:50 AM Jungtaek Lim
wrote:
> I'm sorry, but I have to agree with Ryan and Russell. I chose
I'm sorry, but I have to agree with Ryan and Russell. I chose the option 1
because it's less worse than option 2, but it doesn't mean I fully agree
with option 1.
Let's make below things clear if we really go with option 1, otherwise
please consider reverting it.
* Do you fully indicate about
I think reverting 30098 is the right decision here if we want to unblock
3.0. We shouldn't ship with features which we know do not function in the
way we intend, regardless of how little exposure most users have to them.
Even if it's off my default, we should probably work to avoid switches that
Hi all:
thanks for your reply. the job is hang as 20+ hours, The history server has
deleted the log. I will monitor and l try to use thread dump to try to find
something.
Best Regards
Kelly Zhang
At 2020-05-11 15:41:29, "ZHANG Wei" wrote:
>Sometimes, the Thread dump
I'm all for fixing behavior in master by turning this off as an
intermediate step, but I don't think that Spark 3.0 can safely include
SPARK-30098.
The problem is that SPARK-30098 introduces strange behavior, as Jungtaek
pointed out. And that behavior is not fully understood. While working on a
Had a short sync with Tom. I am going to postpone this for now since this
case is very unlikely - I have seen this twice for the last 5 years.
We'll go for a vote when we happen to see this more, and make a decision
based on the feedback in the vote thread.
2020년 5월 11일 (월) 오후 11:08, Hyukjin
The guide is our official guide, see "Code Style Guide" in
http://spark.apache.org/contributing.html.
As I said this is a general guidance, instead of a hard strict policy. I
don't target to change existing APIs either.
I would like to not start the vote when I see the clear objection to
address,
So as I've already stated and it looks like 2 others have issues with number 4
as written as well, I'm against you posting this as is. I do not think we
should recommend 4 for public user facing Scala API.
Also note the page you linked is a Databricks page, while I know we reference
it as a
I will wait a couple of more days and if there's no objection I hear, I
will document this at
https://github.com/databricks/scala-style-guide#java-interoperability.
2020년 5월 7일 (목) 오후 9:18, Hyukjin Kwon 님이 작성:
> Hi all, I would like to proceed this. Are there more thoughts on this? If
> not, I
+1. Agree with Xiao Li and Jungtaek Lim.
This seems to be controversial, and can not be done in a short time. It is
necessary to choose option 1 to unblock Spark 3.0 and support it in 3.1.
--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
Hi all,
Can someone guide me in accessing temp tables created in spark with
hive/beeline??
Regards,
Vikas
>
> 1. Turn on spark.sql.legacy.createHiveTableByDefault.enabled by default,
> which effectively revert SPARK-30098. The CREATE TABLE syntax is still
> confusing but it's the same as 2.4
> 2. Do not support the v2 CreateTable command if STORE AS/BY or EXTERNAL is
> specified. This gives us more
16 matches
Mail list logo
