srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-781469446
Thanks @HyukjinKwon , I fixed that. I do see the failure you mention, but I
see lots of them. Are those 'normal', the rest of the errors? I recall we've
long had this problem
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-779898947
Waiting just one more round for any comments. I'd like to go with this
change for now.
This is an automated
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-778251325
It's a good point - catalyst classes are not technically private, but by
convention are not considered an API. People do hook into them anyway, at their
own risk.
My
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-774488455
@jnh5y @viirya I got a good question from @marmbrus - can you support
user-defined types by just defining an Encoder for it? so that it can work in a
Dataset?
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-773360047
It does sound like ideally the API would be refactored. @viirya I didn't see
pushback on your redesign.
We can just open this up, which at least re-enables the current
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-773360047
It does sound like ideally the API would be refactored. @viirya I didn't see
pushback on your redesign.
We can just open this up, which at least re-enables the current
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-772886355
Ah, I get it. That's what I was looking for. `InternalRow` is not
technically private, though informally so, being part of the `.catalyst`
package. For those that have much more
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-772853518
Hm, where are those types in the two newly public classes? I don't see any.
This is an automated message from
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-772844416
Yeah I was primarily looking at #16478 but it seems like some of those API
changes are already made? the API doesn't seem to refer to internal Spark
classes anymore. And @viirya
srowen commented on pull request #31461:
URL: https://github.com/apache/spark/pull/31461#issuecomment-772616003
@viirya @cloud-fan you may be best placed to say whether there are any
concerns about this that I don't know of, or what else needs to be opened.
10 matches
Mail list logo
