On 2021/03/12 06:36:24, Fan Liya <liya.fa...@gmail.com> wrote:
> Hi Bob,
>
> Thanks for reporting the issues.
> I remember encountering the same problems with the JDBC tests (over one
> year ago).
>
> Maybe it is not just related to the time zone, it is also related to the
> machine locale.
> I think we can open an issue to track it.
>
OK, opened an issue: https://issues.apache.org/jira/browse/ARROW-11957
I'll create the pull request today, probably.
I noticed that I can't add watchers or even assign to myself; saw on the doc
that someone needs to make me a Contributor.
Here's the bug for various Windows build nits:
https://issues.apache.org/jira/browse/ARROW-12006
@Liya, I don't think it has anything to do with locale; it's the offset
associated with the time zone which is showing up.
@Micah, I guess I'm used to doing local builds and having the JVM pick up my
timezone; now that we're on daylight savings, everything will be off by 7 hours
;-)
>
>
> On Fri, Mar 12, 2021 at 12:09 PM Micah Kornfield <emkornfi...@gmail.com>
> wrote:
>
> > Hi Bob,
> > Thanks for some feedback, I don't think a lot of people are developing on
> > windows. Some answers in line:
> >
> > * Build does require Java 8, not "8 or later" as stated in java/README.md
> > > There's a reference to sun.misc.Unsafe
> > > in
> > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java
> > > which of course went away in JDK 9.
> >
> > Is this the case even using "-Dio.netty.tryReflectionSetAccessible=true" as
> > mentioned in the README?
> >
> >
> > * The build won't work on Windows because the java/format POM downloads a
> > > binary flatc executable, but there's no artifact for Windows, just Linux
> > > and OSX. I wound up downloading Visual Studio and building the
> > flatbuffers
> > > project.
> >
> > This unfortunately sounds familiar, I think this indicate the popularity on
> > windows. I also think the hosting of the mac and linux are not exactly
> > official (they are hosted by a former contributor). Updating the Readme
> > might be a good first step with instructions on how to do this.
> >
> > I see in the pom.xml that user.timezone is set to UTC. I have seen these
> > > types of errors in tests before; I know there are ways to insulate the
> > test
> > > from the user's current timezone but maybe someone knows what's going on
> >
> > This is somewhat surprising. I would thought we had the user.timezone set
> > for exactly this reason. There might have been a regression, this might
> > make a good second contribution if you wanted to look into fixing it.
> >
> > * I bumped into what looks like a spurious checkstyle error: it reports
> > > memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java having no
> > > linefeed at the end when it definitely does. I set up Git not to do
> > Windows
> > > conversions, and I checked with various editors and binary dump
> > utilities.
> > > One source says that this because I'm running on Windows, checkstyle
> > > actually expects a CR-LF and throws an error if it doesn't find it! I've
> > > worked around this by disabling the check.
> >
> > It looks like we can force the checker to assume linux line feeds:
> >
> > https://stackoverflow.com/questions/997021/how-to-get-rid-of-checkstyle-message-file-does-not-end-with-a-newline
> > (second answer). This would also make a good contribution for someone new.
> >
> > Cheers,
> > Micah
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Thu, Mar 11, 2021 at 7:14 PM bobtins <bobti...@gmail.com> wrote:
> >
> > > My mail client took out all the linefeeds, so let me reformat; sorry
> > about
> > > that!
> > >
> > >
> > > In the process of slogging through the build, I've bumped into various
> > > issues. I'm happy to document them in java/README.md or make any other
> > > changes that might be helpful to others.
> > >
> > > I'm pretty experienced with Java and Maven, so I think these are not
> > > beginner's mistakes, but let me know if I'm missing something.
> > >
> > > A lot of these may be Windows-specific. I normally prefer Linux but just
> > > got a new laptop and haven't set it up, but this experience is giving me
> > a
> > > lot of incentive to run screaming back to Linux ;-)
> > >
> > > Environment details:
> > > * Windows 10
> > > * Java 8
> > > here's the output of java -version:
> > > openjdk version "1.8.0_282"
> > > OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_282-b08)
> > > OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.282-b08, mixed mode)
> > >
> > > * Cygwin environment
> > > * Maven 3.6.2
> > >
> > >
> > > Issues encountered thus far:
> > >
> > > * Build does require Java 8, not "8 or later" as stated in java/README.md
> > > There's a reference to sun.misc.Unsafe
> > > in
> > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java
> > > which of course went away in JDK 9.
> > > * The build won't work on Windows because the java/format POM downloads a
> > > binary flatc executable, but there's no artifact for Windows, just Linux
> > > and OSX. I wound up downloading Visual Studio and building the
> > flatbuffers
> > > project.
> > >
> > > * I bumped into what looks like a spurious checkstyle error: it reports
> > > memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java having no
> > > linefeed at the end when it definitely does. I set up Git not to do
> > Windows
> > > conversions, and I checked with various editors and binary dump
> > utilities.
> > > One source says that this because I'm running on Windows, checkstyle
> > > actually expects a CR-LF and throws an error if it doesn't find it! I've
> > > worked around this by disabling the check.
> > >
> > > * The one thing that I'm stuck on now is failures on the jdbc module:
> > > [INFO]
> > > [INFO] Results:
> > > [INFO]
> > > [ERROR] Failures:
> > > [ERROR]
> > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:209
> > > expected:<45935000> but was:<74735000>
> > > [ERROR]
> > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:213
> > > expected:<1518439535000> but was:<1518468335000>
> > > [ERROR]
> > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:205
> > > expected:<-365> but was:<-364>
> > > [ERROR]
> > >
> > JdbcToArrowNullTest.testJdbcToArrowValues:123->testDataSets:165->testAllVectorValues:209
> > > expected:<17574> but was:<17573>
> > > [ERROR] JdbcToArrowTest.testJdbcToArrowValues:138->testDataSets:206
> > > expected:<17574> but was:<17573>
> > > [ERROR]
> > >
> > JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit:107->validate:199->assertDateDayVectorValues:277
> > > expected:<17574> but was:<17573>
> > > [ERROR]
> > >
> > JdbcToArrowVectorIteratorTest.testJdbcToArrowValues:95->validate:199->assertDateDayVectorValues:277
> > > expected:<17574> but was:<17573>
> > > [INFO]
> > > [ERROR] Tests run: 93, Failures: 7, Errors: 0, Skipped: 0
> > >
> > > I attached the full build output.
> > > Looking more closely at these errors, they seem to be due to the timezone
> > > difference; for example, the difference between 74735000 (actual value)
> > and
> > > 45935000 (expected) is 2880000, or 8 hours in milliseconds, which is the
> > > PST timezone offset.
> > >
> > > I see in the pom.xml that user.timezone is set to UTC. I have seen these
> > > types of errors in tests before; I know there are ways to insulate the
> > test
> > > from the user's current timezone but maybe someone knows what's going on.
> > >
> > > Thanks for any input!
> > >
> > >
> > > On 2021/03/11 23:49:37, Bob Tinsman <bobt...@pacbell.net> wrote:
> > > > I've been mostly lurking for awhile, but I would like to start picking
> > > off some bugs in the Java implementation.In the process of slogging
> > through
> > > the build, I've bumped into various issues. I'm happy to document them in
> > > java/README.md or make any other changes that might be helpful to others.
> > > I'm pretty experienced with Java and Maven, so I think these are not
> > > super-obvious, but let me know if I'm missing something.A lot of these
> > may
> > > be Windows-specific. I normally prefer Linux but just got a new laptop
> > and
> > > haven't set it up, but this experience is giving me a lot of incentive to
> > > run screaming back to Linux ;-)
> > > > Environment details:- Windows 10- Java 8:openjdk version
> > > "1.8.0_282"OpenJDK Runtime Environment (AdoptOpenJDK)(build
> > > 1.8.0_282-b08)OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.282-b08,
> > > mixed mode)- Cygwin environment- Maven 3.6.2
> > > > Issues encountered thus far:- Build does require Java 8, not "8 or
> > > later" as stated in java/README.md There's a reference to
> > > sun.misc.Unsafe
> > > in
> > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java
> > > which of course went away in JDK 9. I can update the build
> > > instructions.- The build won't work on Windows because the java/format
> > POM
> > > downloads a binary flatc executables; when I looked, there was no version
> > > for Windows, just Linux and OSX. I wound up downloading Visual Studio and
> > > building the flatbuffers project.- I bumped into what looks like a
> > spurious
> > > checkstyle error: it
> > > reports memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java
> > > having no linefeed at the end when it definitely does. I set up Git not
> > to
> > > do Windows conversions, and I checked with various editors and binary
> > dump
> > > utilities. One source says that this because I'm running on Windows,
> > > checkstyle actually expects a CR-LF and throws an error if it doesn't
> > find
> > > it! I've worked
> > > around this by disabling the check.- The one thing that I'm stuck on
> > now
> > > is failures on jdbc:
> > > > [INFO] Results:[INFO][ERROR] Failures:[ERROR]
> > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:209
> > > expected:<45935000> but was:<74735000>[ERROR]
> > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:213
> > > expected:<1518439535000> but was:<1518468335000>[ERROR]
> > > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:205
> > > expected:<-365> but was:<-364>[ERROR]
> > >
> > JdbcToArrowNullTest.testJdbcToArrowValues:123->testDataSets:165->testAllVectorValues:209
> > > expected:<17574> but was:<17573>[ERROR]
> > > JdbcToArrowTest.testJdbcToArrowValues:138->testDataSets:206
> > > expected:<17574> but was:<17573>[ERROR]
> > >
> > JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit:107->validate:199->assertDateDayVectorValues:277
> > > expected:<17574> but was:<17573>[ERROR]
> > >
> > JdbcToArrowVectorIteratorTest.testJdbcToArrowValues:95->validate:199->assertDateDayVectorValues:277
> > > expected:<17574> but was:<17573>[INFO][ERROR] Tests run: 93, Failures: 7,
> > > Errors: 0, Skipped: 0
> > > > I attached the full build output.Looking more closely at these errors,
> > > they seem to be due to the timezone difference; for example, the
> > difference
> > > between 74735000 (actual value) and 45935000 (expected) is 2880000, or 8
> > > hours in milliseconds, which is the PST timezone offset. I see in the
> > > pom.xml that user.timezone is set to UTC. I have seen these types of
> > errors
> > > in tests before; I know there are ways to insulate the test from the
> > user's
> > > current timezone but maybe someone knows what's going on.
> > > > Thanks for any input!
> > > >
> > >
> >
>
