Spark SQL filter DataFrame by date?

I'm trying to filter a DataFrame by a date column, with no luck so far. Here's what I'm doing: When I run reqs_day.count() I get zero, apparently because my date parameter gets translated to 16509. Is this a bug, or am I doing it wrong? -- View this message in context:

Re: How to read a multipart s3 file?

Thanks Nicholas! I looked at those docs several times without noticing that critical part you highlighted. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-read-a-multipart-s3-file-tp5463p5494.html Sent from the Apache Spark User List mailing list

Spark 0.9.1 -- assembly fails?

After upgrading to Spark 0.9.1, sbt assembly is failing. I'm trying to fix it with merge strategy, etc., but is anyone else seeing this? For example, -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-0-9-1-assembly-fails-tp4979.html Sent from the

Re: Spark 0.9.1 -- assembly fails?

So, good news and bad news. I have a customized Build.scala http://apache-spark-user-list.1001560.n3.nabble.com/libraryDependencies-configuration-is-different-for-sbt-assembly-vs-sbt-run-tt565.html#a1542 that allows me to use the 'run' and 'assembly' commands in sbt without toggling the

Re: Spark 0.9.1 -- assembly fails?

Um. When I updated the spark dependency, I unintentially deleted the provided attribute. Oops. Nothing to see here . . . -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-0-9-1-assembly-fails-tp4979p4982.html Sent from the Apache Spark User List

Re: Anyone using value classes in RDDs?

.nabble.com wrote: isn't valueclasses for primitives (AnyVal) only? that doesn't apply to string, which is an object (AnyRef) On Fri, Apr 18, 2014 at 2:51 PM, kamatsuoka [hidden email]http://user/SendEmail.jtp?type=nodenode=4475i=0 wrote: I'm wondering if anyone has tried using value

Anyone using value classes in RDDs?

I'm wondering if anyone has tried using value classes in RDDs? My use case is that I have a number of RDDs containing strings, e.g. val r1: RDD[(String, (String, Int)] = ... val r2: RDD[(String, (String, Int)] = ... and it might be clearer if I wrote case class ID(val id: String) extends

Spark with SSL?

Can Spark be configured to use SSL for all its network communication? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-with-SSL-tp3916.html Sent from the Apache Spark User List mailing list archive at Nabble.com.