Another correctness issue: https://issues.apache.org/jira/browse/SPARK-32659
On Tue, Aug 25, 2020 at 11:25 PM Sean Owen wrote:
> That isn't a blocker (see comments - not a regression).
> That said I think we have a fix ready to merge now, if there are no
> objections.
>
> On Tue, Aug 25, 2020
Hi,
So just realized there were already multiple attempts on this issue in the
past. From the discussion it seems the preferred approach is to eliminate
the cast before they get pushed to data sources, at least for a few
common cases such as numeric types. However, a few PRs following this
That isn't a blocker (see comments - not a regression).
That said I think we have a fix ready to merge now, if there are no objections.
On Tue, Aug 25, 2020 at 10:24 AM Dongjoon Hyun wrote:
>
> For the correctness blocker, we have the following, Tom.
>
> -
For the correctness blocker, we have the following, Tom.
- https://issues.apache.org/jira/browse/SPARK-32614
- https://github.com/apache/spark/pull/29516
Bests,
Dongjoon.
On Tue, Aug 25, 2020 at 6:32 AM Tom Graves
wrote:
> Hey,
>
> I'm just curious what the status of the 3.0.1 release is? Do
Any other feedback here? The couple I've heard preferred in various
conversations are excludeList and blockList. If not I'll just make proposal on
jira and continue discussion there and anyone interested can watch this jira.
Thanks,Tom
On Tuesday, August 4, 2020, 09:19:01 AM CDT, Tom
Thanks for the replies so far, is there any other feedback here? Of the
replies so far I think Leader has been mentioned the most.
Tom
On Tuesday, August 4, 2020, 09:33:14 AM CDT, Russell Spitzer
wrote:
I think we should use Scheduler or Comptroller or Leader; something that
Hey,
I'm just curious what the status of the 3.0.1 release is? Do we have some
blockers we are waiting on?
Thanks,Tom
On Sunday, August 16, 2020, 09:07:44 PM CDT, ruifengz
wrote:
Thanks for letting us know this issue.
On 8/16/20 11:31 PM, Takeshi Yamamuro wrote:
I've
works perfectly! Thanks Herman.
Am Di., 25. Aug. 2020 um 12:03 Uhr schrieb Herman van Hovell <
her...@databricks.com>:
> Hi Robert,
>
> Your Spark 3.0 code is missing the encoder that converts the Row to an
> InternalRow. Your Spark 3.0 code should look like this:
>
> def rowToCaseClass[C <:
Hi everyone
Thanks Takeshi. I run into the same issue as Mark for my row to case class
converter:
def rowToCaseClass[C <: Product : TypeTag](r: Row)(implicit encs:
(ExpressionEncoder[Row], ExpressionEncoder[C])): C = {
val ir = encs._1.toRow(r)
encs._2.fromRow(ir)
}
So in Spark3.0 I would
Bump this again.
On Tue, Aug 18, 2020 at 12:11 PM Jungtaek Lim
wrote:
> Bump again.
>
> Unlike file stream sink which has lots of limitations and many of us have
> been suggesting alternatives, file stream source is the only way if end
> users want to read the data from files. No alternative
10 matches
Mail list logo
