Re: merge script stopped working; Python 2/3 input() issue?

2019-02-15 Thread Sean Owen
That was it, thanks! https://github.com/apache/spark-website/pull/183 Well, in my defense: PEP or IntelliJ made me do it, and I continue to resent Python for this kind of nonsense. But my bad for not testing this on a dummy commit, to be sure. On Fri, Feb 15, 2019 at 6:09 PM Marcelo Vanzin

Re: merge script stopped working; Python 2/3 input() issue?

2019-02-15 Thread Marcelo Vanzin
BTW the main script has this that the website script does not: if sys.version < '3': input = raw_input # noqa On Fri, Feb 15, 2019 at 3:55 PM Sean Owen wrote: > > I'm seriously confused on this one. The spark-website merge script > just stopped working for me. It fails on the call to

SparkThriftServer Authorization design

2019-02-15 Thread t4
Goal is to provide ability to deny/allow access to given tables on a per user basis (this is the user connecting via jdbc to spark thrift server. ie with LDAP creds). ie user bob can see table A but not table B. user mary can see table B but not table A. What are folks thoughts on the approach?

Re: merge script stopped working; Python 2/3 input() issue?

2019-02-15 Thread shane knapp
On Fri, Feb 15, 2019 at 4:03 PM Reynold Xin wrote: > lol > > couldn't have said it better myself. ;) shane -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu

Re: merge script stopped working; Python 2/3 input() issue?

2019-02-15 Thread Sean Owen
Oh my. Thanks for that. Let me get to the bottom of the right thing to do here. I still don't get why the main spark script works doing the same thing! On Fri, Feb 15, 2019, 6:02 PM Marcelo Vanzin You're talking about the spark-website script, right? The main repo's > script has been working for

Re: merge script stopped working; Python 2/3 input() issue?

2019-02-15 Thread Reynold Xin
lol On Fri, Feb 15, 2019 at 4:02 PM, Marcelo Vanzin < van...@cloudera.com.invalid > wrote: > > > > You're talking about the spark-website script, right? The main repo's > script has been working for me, the website one is broken. > > > > I think it was caused by this dude changing

Re: merge script stopped working; Python 2/3 input() issue?

2019-02-15 Thread Marcelo Vanzin
You're talking about the spark-website script, right? The main repo's script has been working for me, the website one is broken. I think it was caused by this dude changing raw_input to input recently: commit 8b6e7dceaf5d73de3f92907ceeab8925a2586685 Author: Sean Owen Date: Sat Jan 19 19:02:30

merge script stopped working; Python 2/3 input() issue?

2019-02-15 Thread Sean Owen
I'm seriously confused on this one. The spark-website merge script just stopped working for me. It fails on the call to input() that expects a y/n response, saying 'y' isn't defined. Indeed, it seems like Python 2's input() tries to evaluate the input, rather than return a string. Python 3

Re: Static functions

2019-02-15 Thread Jean Georges Perrin
Hey Jacek, You mean: * @groupname udf_funcs UDF functions * @groupname agg_funcs Aggregate functions * @groupname datetime_funcs Date time functions * @groupname sort_funcs Sorting functions * @groupname normal_funcs Non-aggregate functions * @groupname math_funcs Math functions *

Re: Compatibility on build-in DateTime functions with Hive/Presto

2019-02-15 Thread Xiao Li
We normally do not follow MySQL. Check the commercial database [like Oracle]? or the open source PostgreSQL? Sean Owen 于2019年2月15日周五 上午5:34写道: > year("1912") == 1912 makes sense; month("1912") == 1 is odd but not > wrong. On the one hand, some answer might be better than none. But > then, we

Re: Compatibility on build-in DateTime functions with Hive/Presto

2019-02-15 Thread Sean Owen
year("1912") == 1912 makes sense; month("1912") == 1 is odd but not wrong. On the one hand, some answer might be better than none. But then, we are trying to match Hive semantics where the SQL standard is silent. Is this actually defined behavior in a SQL standard, or, what does MySQL do? On Fri,

Compatibility on build-in DateTime functions with Hive/Presto

2019-02-15 Thread Darcy Shen
See https://issues.apache.org/jira/browse/SPARK-26885 and  https://github.com/apache/spark/blob/71170e74df5c7ec657f61154212d1dc2ba7d0613/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala stringToTimestamp, stringToDate support , as a result: select