Charles, fixed your testComplexXSD in your xsd_reader branch.
https://github.com/cgivre/drill/pull/5 Need to add attribute support, but this is quite close. On Fri, Jul 14, 2023 at 5:59 PM Charles Givre <cgi...@gmail.com> wrote: > Hi Mike, > One more thing... I've been working on an XSD Reader for Drill for some > time. (This is still very buggy) > https://github.com/cgivre/drill/tree/xsd_reader > > What this does is attempt to convert a XML XSD file into a Drill Schema. > Best, > -- C > > > > On Jul 14, 2023, at 2:20 PM, Charles Givre <cgi...@gmail.com> wrote: > > Mike, > Are you able to build Drill w/o the tests? If so, my suggestion is really > just to start working on the DFDL extensions. I've been doing Drill stuff > for far too long and really haven't needed to run the full battery of unit > tests locally. As long as you can build it and can execute individual unit > tests, you should be ok. Others may disagree, but for what you're doing, > I'd think it would be fine. > Best, > -- C > > > > On Jul 14, 2023, at 2:04 PM, Mike Beckerle <mbecke...@apache.org> wrote: > > Update: I did a clean and install -DskipTests=true. > > Then I tried the mvn test using the non-UTC timezone stuff, as suggested. > > But alas, it still fails, this time the failure unique and is only in > "Java Execution Engine" > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-dependency-plugin:3.4.0:unpack > (unpack-vector-types) on project drill-java-exec: Artifact has not been > packaged yet. When used on reactor artifact, unpack should be executed > after packaging: see MDEP-98. -> [Help 1] > > The command and complete trace output are below. > > I need assistance on how to proceed. > > Complete trace from the mvn test is attached. > > > On Thu, Jul 13, 2023 at 1:13 PM Mike Beckerle <mbecke...@apache.org> > wrote: > >> To answer questions: >> >> 1. Paul: This is a 100% stock build. All I have done is clone the repo >> (master branch). Make a new git branch (in case I make future changes). Try >> to build (success) and test (failed so far). >> >> 2. James: The /opt/drill directory I created is owned by my userid and >> has full read/write access for all the development activities. I just put >> it there so it would have a shorter path to fix the first Hive-related >> glitch I encountered with the Linux 255 limit on file pathname length. >> >> I will try the suggested maven command line for non-UTC and see if things >> improve. >> >> The challenge for me as a newby is how do I know if I have everything >> properly configured? >> >> Can I just turn off building and testing of the Hive-related stuff in >> some supported/well-known way? >> >> If so, I would suggest I'd like to turn off not just Hive, but *as much >> as possible*. I really just need the embedded drill to work. >> >> I would agree with @Charles Givre <cgi...@gmail.com> that a contrib >> package addition is the ideal approach and that's what I'll be attempting. >> >> -mikeb >> >> On Thu, Jul 13, 2023 at 10:59 AM Charles Givre <cgi...@gmail.com> wrote: >> >>> I'll add some heresy here... IMHO, for the purposes of developing a DFDL >>> extension, you probably don't need all the Drill tests to run. For your >>> project, my suggestion would be to add a module to the contrib package and >>> that way your changes are relatively self contained. >>> Best, >>> -- C >>> >>> >>> >>> > On Jul 13, 2023, at 10:27 AM, James Turton <dz...@apache.org> wrote: >>> > >>> > Hi Mike >>> > >>> > Here's the command line I use to run tests on a machine that's not in >>> the UTC time zone (plus some unrelated memory size arguments). >>> > >>> > mvn test -Djunit.args="-Duser.timezone=UTC -Duser.language=en >>> -Duser.region=US" -DmemoryMb=2560 -DdirectMemoryMb=2560 >>> > >>> > I have one other question to add to Paul's comments - does the OS user >>> that you're running Maven under have write access to all of the source tree >>> that you put at /opt/drill? >>> > >>> > On 2023/07/11 22:12, Paul Rogers wrote: >>> >> Hi Mike, >>> >> >>> >> A quick glance at the log suggests a failure in the tests for the JSON >>> >> reader, in the Mongo extended types. Drill's date/time support has >>> >> historically been fragile. Some tests only work if your machine is >>> set to >>> >> use the UTC time zone (or Java is told to pretend that the time is >>> UTC.) >>> >> The Mongo types test failure seems to be around a date/time test so >>> maybe >>> >> this is the issue? >>> >> >>> >> There are also failures indicating that the Drillbit (Drill server) >>> died. >>> >> Not sure how this can happen, as tests run Drill embedded (or used >>> to.) >>> >> Looking earlier in the logs, it seems that the Drillbit didn't start >>> due to >>> >> UDF (user-defined function) failures: >>> >> >>> >> Found duplicated function in drill-custom-lower.jar: >>> >> custom_lower(VARCHAR-REQUIRED) >>> >> Found duplicated function in built-in: lower(VARCHAR-REQUIRED) >>> >> >>> >> Not sure how this could occur: it should have failed in all builds. >>> >> >>> >> Also: >>> >> >>> >> File >>> >> >>> /opt/drill/exec/java-exec/target/org.apache.drill.exec.udf.dynamic.TestDynamicUDFSupport/home/drill/happy/udf/staging/drill-custom-lower-sources.jar >>> >> does not exist on file system file:/// >>> >> >>> >> This is complaining that Drill needs the source code (not just class >>> file) >>> >> for its built-in functions. Again, this should not fail in a standard >>> >> build, because if it did, it would fail in all builds. >>> >> >>> >> There are other odd errors as well. >>> >> >>> >> Perhaps we should ask: is this a "stock" build? Check out Drill and >>> run >>> >> tests? Or, have you already started making changes for your project? >>> >> >>> >> - Paul >>> >> >>> >> >>> >> On Tue, Jul 11, 2023 at 9:07 AM Mike Beckerle <mbecke...@apache.org> >>> wrote: >>> >> >>> >>> I have drill building and running its tests. Some tests fail: [ERROR] >>> >>> Tests run: 4366, Failures: 2, Errors: 1, Skipped: 133 >>> >>> >>> >>> I am wondering if there is perhaps some setup step that I missed in >>> the >>> >>> instructions. >>> >>> >>> >>> I have attached the output from the 'mvn clean install >>> -DskipTests=false' >>> >>> execution. (zipped) >>> >>> I am running on Ubuntu 20.04, definitely have Java 8 setup. >>> >>> >>> >>> I'm hoping someone can skim it and spot the issue(s). >>> >>> >>> >>> Thanks for any help >>> >>> >>> >>> Mike Beckerle >>> >>> Apache Daffodil PMC | daffodil.apache.org >>> >>> OGF DFDL Workgroup Co-Chair | >>> www.ogf.org/ogf/doku.php/standards/dfdl/dfdl >>> >>> Owl Cyber Defense | www.owlcyberdefense.com >>> >>> >>> >>> >>> >>> >>> > >>> >>> <repos-ping-from-cat3.zip> > > > >
