Hi Wenchen,
thanks for your email. I agree adding doc for decimal type, but I am not
sure what you mean speaking of the behavior when writing: we are not
performing any automatic casting before writing; if we want to do that, we
need a design about it I think.
I am not sure if it makes sense to s
AFAIK parquet spec says decimal scale can't be negative. If we want to
officially support negative-scale decimal, we should clearly define the
behavior when writing negative-scale decimals to parquet and other data
sources. The most straightforward way is to fail for this case, but maybe
we can do
In general we can say that some datasources allow them, others fail. At the
moment, we are doing no casting before writing (so we can state so in the
doc). But since there is ongoing discussion for DSv2, we can maybe add a
flag/interface there for "negative scale intollerant" DS and try and cast
be
I'm OK with it, i.e. fail the write if there are negative-scale decimals
(we need to document it though). We can improve it later in data source v2.
On Mon, Jan 7, 2019 at 10:09 PM Marco Gaido wrote:
> In general we can say that some datasources allow them, others fail. At
> the moment, we are d
srowen opened a new pull request #164: Suggest new Apache repo for committers
URL: https://github.com/apache/spark-website/pull/164
This suggests to committers that they should use the new github remote to
push to Apache.
Thi
rxin commented on issue #164: Suggest new Apache repo for committers
URL: https://github.com/apache/spark-website/pull/164#issuecomment-452047166
LGTM
This is an automated message from the Apache Git Service.
To respond to the
srowen closed pull request #164: Suggest new Apache repo for committers
URL: https://github.com/apache/spark-website/pull/164
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pu
after spending the last three hours unraveling how
dev/create-release/release-build.sh works, i think i figured out what needs
to happen.
1) https://issues.apache.org/jira/browse/SPARK-26565
2) create world-wide campaign to stop bash scripting
my brain... it hurts.
On Sun, Jan 6, 2019 at 1:07 P
>
> 1) https://issues.apache.org/jira/browse/SPARK-26565
> 2) create world-wide campaign to stop bash scripting
>
3) re-create these builds w/o calling release-build.sh at all (preferred)
