Re: Graduation

2021-01-27 Thread Beckerle, Mike
So reviewing the apache project maturity model, I think we are good w.r.t. those things. Do we need to prepare anything specific? I see that some projects have documents that respond to the maturity model point by point. Given that we've been sitting on status "community building" for a while,

Re: [apache/incubator-daffodil] Embedded Schematron (#463)

2021-01-21 Thread Beckerle, Mike
d to support more complicated validation like co-constraints. On 1/21/21 12:13 PM, Beckerle, Mike wrote: > > Question on schematron or really the new validator system generally. > > Can I use both Daffodil's built in "limited" validation AND also use > schematron or other

Re: [apache/incubator-daffodil] Embedded Schematron (#463)

2021-01-21 Thread Beckerle, Mike
a validation I'd like to also​ use schematron rules. That's the motivation anyway. From: John Wass Sent: Wednesday, January 20, 2021 7:20 AM To: apache/incubator-daffodil Cc: Beckerle, Mike ; Push Subject: Re: [apache/incubator-daffodil] Embedded Schematron (#

Re: NewVariableInstance Suspension Issue

2021-01-11 Thread Beckerle, Mike
Tricky! The newVariableInstance needs to go out of scope for the ongoing unparse, but the existing variable instances need to be captured when the UStateForSuspension (I think it's callled that) is created. Probably the latter is what isn't happening. From:

debugger thoughts - parsing oriented

2021-01-11 Thread Beckerle, Mike
Here's some thoughts about the debugger for parsing. * The debugger doesn't distinguish the big parsers that correspond to user-specified possibly named entities (elements, model groups) from the finer-grained parsers that are the details. It needs to distinguish these. * There's

Re: The future of the daffodil DFDL schema debugger?

2021-01-11 Thread Beckerle, Mike
il -> pick from list of dfdl.xsd ... On Fri, Jan 8, 2021 at 2:47 PM Beckerle, Mike wrote: > Use cases or quasi-requirements. This is my summary so far. > > 1) capture a human-readable trace of parse/unparse information to a single > text file (might be same as 2 if machine-readab

Re: rfc: sbt-tdml

2021-01-11 Thread Beckerle, Mike
Can we split this into 1) test code boilerplate - I have big issues with removing this 2) capturing tests that are expected to fail due to issues in TDML vs. in the scala test driver code. - I'm on board with this. With respect to (1) the test code boilerplate The ability to start from a

Re: The future of the daffodil DFDL schema debugger?

2021-01-08 Thread Beckerle, Mike
looked at them. Would even allow for a TUI, which Java has little/no support for. Also enables things like remote deubgging if an socket IPC was used. Though I'm not sure all of that is necessary. Just thinking what would be ideal, and it can always be pared back. On 1/6/21 12:44 PM, Beckerl

Re: Timezones in DFDL

2021-01-08 Thread Beckerle, Mike
tween when Los Angeles is GMT-7 and GMT-8. > > Sent from my iPhone > >> On Jan 8, 2021, at 6:48 AM, Beckerle, Mike >> wrote: >> >> I believe SL is correct. This is as expected. This is data >> canonicalization, which is very typically what happens when a p

Re: Timezones in DFDL

2021-01-08 Thread Beckerle, Mike
I believe SL is correct. This is as expected. This is data canonicalization, which is very typically what happens when a parser tolerates many diverse formats, but the data format doesn't capture which of those specifically. They're considered, by the DFDL schema, to be 100% equivalent. The

Re: The future of the daffodil DFDL schema debugger?

2021-01-06 Thread Beckerle, Mike
I don't think of it as a daffodil debug protocol, but just a separation of concerns between display of information and the behaviors of parse/unparse that need to be points where users can pause, and data structures available to display. E.g., it is 100% a display issue that the infoset (shown

Re: How to add warnings that are not lost due to backtracking

2021-01-04 Thread Beckerle, Mike
the regex limit message? -Original Message- From: Beckerle, Mike [mailto:mbecke...@owlcyberdefense.com] Sent: Monday, December 28, 2020 12:15 PM To: dev@daffodil.apache.org Subject: Re: How to add warnings that are not lost due to backtracking So, I wanted to clarify a few things. Then I think I

Re: How to add warnings that are not lost due to backtracking

2020-12-28 Thread Beckerle, Mike
So, I wanted to clarify a few things. Then I think I agree we want runtime-issued SDW to not be lost when backtracking. An SDE, or schema definition error, is most commonly the Daffodil schema compiler telling you your schema isn't meaningful, so parsing/unparsing cannot even be started. We

Re: Implementing variable "direction" property

2020-12-21 Thread Beckerle, Mike
Are you missing the use of a SuspendableExpression for NewVariableInstance? I see SetVariableSuspendableExpression used in the Daffodil code base for SetVariable. This suggests that the mechanism to suspend on a variable-reference that hits a variable the assignment of which is suspended will

Re: Daffodil schema file extension

2020-12-21 Thread Beckerle, Mike
art: recognizes a .dfdl extension as DFDL _and_ knows the DFDL language, actions, etc) On Fri, Dec 18, 2020 at 12:45 PM Beckerle, Mike < mbecke...@owlcyberdefense.com> wrote: > Well, admittedly some of my observations may be dated to using Eclipse. I > agree IntelliJ does seem to just general

Re: Daffodil schema file extension

2020-12-18 Thread Beckerle, Mike
Dec 18, 2020 at 10:14 AM Beckerle, Mike < > mbecke...@owlcyberdefense.com> wrote: > >> If you have edited tdml in an XML aware editor, you know that the support >> for embedded dfdl schemas is better than it is in xsd editors for a dfdl >> schema file. >> >> F

Re: Daffodil schema file extension

2020-12-18 Thread Beckerle, Mike
If you have edited tdml in an XML aware editor, you know that the support for embedded dfdl schemas is better than it is in xsd editors for a dfdl schema file. For that reason I thought maybe we should use the ".dfdl" extension for a daffodil feature which replaces the xs:schema element of a

Re: Correct behavior when unparse does not ending on a byte boundary

2020-12-17 Thread Beckerle, Mike
What you call option 2 is definitely the right behavior, so this is a significant bug in the unparser streaming API. There should never be aligning to a byte boundary automatically except when the output stream is closed. The fillByte to use for a close is certainly the last fillbyte of the

thoughts on runtime2-2202

2020-11-30 Thread Beckerle, Mike
I'd like to split the C code into (a) essential runtime files (b) optional/test runtime files. Of the list of current files: daffodil_argp.c daffodil_argp.h daffodil_main.c infoset.c infoset.h stack.c stack.h xml_reader.c xml_reader.h xml_writer.c xml_writer.h I believe the infoset.c/h are

Re: [DRAFT] Daffodil Podling Report - December 2020

2020-11-30 Thread Beckerle, Mike
+1 from me. Looks good. From: Steve Lawrence Sent: Monday, November 30, 2020 11:25 AM To: dev@daffodil.apache.org Subject: [DRAFT] Daffodil Podling Report - December 2020 ## Daffodil Apache Daffodil is an implementation of the Data Format Description Language

Re: Rebasing runtime2-2202 on asf/master

2020-11-23 Thread Beckerle, Mike
I tried this, checked out runtime2-2202 then git pull asf master. There are like 5 conflicting files, but looks pretty manageable. I am looking forward to working on runtime2 more soon. From: Interrante, John A (GE Research, US) Sent: Monday, November 23,

Wanted: public comments on DFDL specification

2020-11-12 Thread Beckerle, Mike
The DFDL Spec is in public review until early January. It would be helpful if people made public comments by creating an "issue" here: https://github.com/OpenGridForum/Editor/issues The Open Grid Forum says "Brief affirmative comments are welcome - not just suggestions for changes. Public

Re: Daffodil Flow Diagram

2020-11-10 Thread Beckerle, Mike
Such design doc as we have is rooted here: https://cwiki.apache.org/confluence/display/DAFFODIL/Design+Notes Daffodil starts with a DFDL Schema Compiler. That breaks the DFDL schema into a number of objects called DSOM objects. DSOM = DFDL Schema Object Model. There is a wiki page describing

Re: IntelliJ IDEA is flagging “errors”

2020-11-09 Thread Beckerle, Mike
I believe one can adjust the various suggestions it makes, but IntelliJ often seems to suggest types where it thinks they would add value. The colorization it uses indicates that this is it's suggestions, not things actually in the text, and my eye has learned to read past these. There are

Re: Updated DFDL Spec Draft - Public Comment Starting

2020-11-02 Thread Beckerle, Mike
. From: Beckerle, Mike Sent: Thursday, October 29, 2020 4:27 PM To: dev@daffodil.apache.org Subject: Updated DFDL Spec Draft - Public Comment Starting The DFDL spec has been revised and will enter official public comment period shortly. I'll post the links for that as soon

Updated DFDL Spec Draft - Public Comment Starting

2020-10-29 Thread Beckerle, Mike
The DFDL spec has been revised and will enter official public comment period shortly. I'll post the links for that as soon as I get them. Right now the HTML version of the public-comment version is available here:

Re: Problems with expressions and variables

2020-10-29 Thread Beckerle, Mike
The default value expression should be evaluated not on demand, but when the sequence is entered. That will give it the correct context. The evaluation of the default value expression should work the same way as if it was in a setVariable annotation, with the exception that it sets the default

New SAX component for Daffodil JIRA tickets

2020-10-27 Thread Beckerle, Mike
Added this as a component of Daffodil that JIRA tickets can select when a ticket is related to the SAX API, or implementation thereof. [cid:3bb3208d-43c0-41a8-bc8d-cae701fbc910] Mike Beckerle | Principal Engineer [OWL Cyber Defense] P +1-781-330-0412 W

Re: 2399 - Changing error code for missing separators

2020-10-26 Thread Beckerle, Mike
transmission in error, please notify the sender immediately From: Beckerle, Mike<mailto:mbecke...@owlcyberdefense.com> Sent: Monday, October 26, 2020 11:27 AM To: dev@daffodil.apache.org<mailto:dev@daffodil.apache.org> Subject: Re: 2399 - Changing error code for missing separators

Fw: [SonarCloud] incubator-daffodil: 1 new issue (new debt: 5min)

2020-10-26 Thread Beckerle, Mike
need solarcloud to find this for us. I'd expect every developer to see these on every compile. From: SonarCloud Sent: Monday, October 26, 2020 2:45 PM To: Beckerle, Mike Subject: [SonarCloud] incubator-daffodil: 1 new issue (new debt: 5min) Project: incubator

Re: 2399 - Changing error code for missing separators

2020-10-26 Thread Beckerle, Mike
There's some discussion in the PR, but for this mailing list I wanted to comment that this is indeed a deep area of algorithmic complexity in the Daffodil runtime. Whether the code is actually factored adequately such that it enables implementing the right DFDL behavior is not entirely clear,

Re: TestDaffodilXMLLoader non-thread safe tests

2020-10-26 Thread Beckerle, Mike
Is there any way to have a separate test module where tests are all run with fork = true? From: Steve Lawrence Sent: Monday, October 26, 2020 9:54 AM To: dev@daffodil.apache.org Subject: TestDaffodilXMLLoader non-thread safe tests We have three tests in

Re: DAFFODIL-2216 PR

2020-10-19 Thread Beckerle, Mike
Chia-Hung Lin, You are added as a Daffodil contributor. I think that should enable you to assign that JIRA ticket to yourself. Try it ! ...mike beckerle From: Chia-Hung Lin Sent: Sunday, October 18, 2020 10:50 AM To: dev@daffodil.apache.org Subject:

Re: DAFFODIL-2216 PR

2020-10-19 Thread Beckerle, Mike
I approved the PR. Once we get two approvals we'll be able to merge this. I will look into adding you as a contributor so you can assign tickets to yourself. I just have to remember how to do that  Thanks for this contribution. From: Chia-Hung Lin Sent:

Re: issue with calendarPattern with SSSSSS not unparsing to 000000

2020-10-14 Thread Beckerle, Mike
specification don't agree. On 10/14/20 3:15 PM, Beckerle, Mike wrote: > I did not find JIRA tickets for this issue. Just want to see if anyone has > insight/workaround for this. > > This problem occurs for Daffodil 2.4.0 and 2.7.0 > > I have a schema with > >name="Timesta

issue with calendarPattern with SSSSSS not unparsing to 000000

2020-10-14 Thread Beckerle, Mike
I did not find JIRA tickets for this issue. Just want to see if anyone has insight/workaround for this. This problem occurs for Daffodil 2.4.0 and 2.7.0 I have a schema with Notice the 6 S in SS. This parses this fine: 2003-08-24T05:14:15.03-07:00 producing this XML Infoset:

Re: Prospective Daffodil Contributor

2020-10-13 Thread Beckerle, Mike
There is eclipse support for scala via "Scala IDE", which is a bundled thing, but is just a version of Eclipse with scala features properly installed. Note that I had no luck just taking Eclipse and installing scala features myself. I found I had to dowload "Scala IDE" and use it. However, I

Re: Daffodil-1300

2020-10-09 Thread Beckerle, Mike
sure many projects follow. I think using implemtation="" would be a good way to define which tests work and which don't. Then you don't even need to comment out tests. Those tests just get skipped when run. On 10/8/20 2:02 PM, Beckerle, Mike wrote: > I've depended, since the earlie

Re: Daffodil-1300

2020-10-08 Thread Beckerle, Mike
of dynamic test suites (it does have dynamic tests), and how nicely that plays with IDEs. I feel like we also shouldn't necessarily be tied to JUnit. While maybe preferred, I would be willing to switch to something else if it has this sort of capability and works with IDEs. On 10/8/20 10:5

Re: Daffodil-1300

2020-10-08 Thread Beckerle, Mike
by the GC when the test finishes? On 10/7/20 6:11 PM, Beckerle, Mike wrote: > The Runner objects are normally members of a scala singleton object. That way > they are constructed once and shared by all the tests. > > My understanding is once a singleton object is required, it is

Re: Daffodil-1300

2020-10-07 Thread Beckerle, Mike
ictly prohibited. If you have received this transmission in error, please notify the sender immediately From: Beckerle, Mike<mailto:mbecke...@owlcyberdefense.com> Sent: Thursday, October 1, 2020 4:03 PM To: dev@daffodil.apache.org<mailto:dev@daffodil.apache.org> Subject: Re: Daffodil-130

Re: Keep compiled C code or throw it away?

2020-10-05 Thread Beckerle, Mike
mplementation works. This TDMLDFDLProcesor just happens to use the same Daffodil frontend but with a different Daffodil backend. On 10/5/20 5:02 PM, Beckerle, Mike wrote: > There are 3 different kinds of code in Daffodil: > > 1) static code humans write - compiler code and runtime code

Re: Subproject names proposed for discussion

2020-10-05 Thread Beckerle, Mike
e: Subproject names proposed for discussion I'm fine with just removing it completely. As I said, I don't think the reasons I listed are strong reasons for keeping a prefix. On 10/5/20 11:36 AM, Beckerle, Mike wrote: > A shorter prefix like just "daf-" would bother me less. >

Re: Keep compiled C code or throw it away?

2020-10-05 Thread Beckerle, Mike
untime1" and "runtime2". Disadvantage is there's less discoverability, but a README could be added with short descriptions about what the backends try to accomplish. Not sure I like this, but thought I'd throw it out there. On 10/5/20 10:23 AM, Beckerle, Mike wrote: > +1 fr

Re: How to use and trigger different Daffodil backends

2020-10-05 Thread Beckerle, Mike
If 'sbt test' will run TDML tests on either or both backends, then I see no need for the new C-backend to directly/natively support command-line invocation of TDML tests at all. Moreover, if the existing daffodil scala CLI can run TDML tests (which it can), then if it can run TDML tests that

Re: Subproject names proposed for discussion

2020-10-05 Thread Beckerle, Mike
removing the prefix could help with that... On 10/5/20 11:11 AM, Beckerle, Mike wrote: > Some reponses inline below > > > From: Steve Lawrence > Sent: Monday, October 5, 2020 10:48 AM > To: dev@daffodil.apache.org > Subject: Re: Subproject na

Re: Subproject names proposed for discussion

2020-10-05 Thread Beckerle, Mike
Compilers are commonly divided into "front-end" - concerned with the language itself and its syntax, and "back end" - concerned with code-generation. Sometimes back-ends are language independent as in the same compiler back-end can be shared using front-ends for different languages (C and

Re: Subproject names proposed for discussion

2020-10-05 Thread Beckerle, Mike
nd-scala-parser-lib backend-scala-unparser-lib backend-c-lib tdml-lib tdml-processor . On 10/5/20 10:23 AM, Beckerle, Mike wrote: > +1 from me. > > > From: Interrante, John A (GE Research, US) > Sent: Monday, October 5, 2020 9:28 A

Re: Subproject names proposed for discussion

2020-10-05 Thread Beckerle, Mike
+1 from me. From: Interrante, John A (GE Research, US) Sent: Monday, October 5, 2020 9:28 AM To: dev@daffodil.apache.org Subject: Subproject names proposed for discussion Steve Lawrence and I would like to bring a topic to the dev list for discussion since not

Re: Validator SPI proposal

2020-10-01 Thread Beckerle, Mike
in the daffodil source tree. From: Beckerle, Mike Sent: Thursday, October 1, 2020 5:52 PM To: dev@daffodil.apache.org Subject: Re: Validator SPI proposal A few thoughts on top of John Interrante's review. The validator code/clases being found via SPI seems good. Sharing

Re: Daffodil-1300

2020-10-01 Thread Beckerle, Mike
I think the DFDLTestSuite constructor can't stay deprecated. We still need to construct them. It can become package private I think, so not usable outside the package. I think the scala syntax for that is: class Foo private[pkgName] (arg1: Type1, arg2: Type2) { ... } The deprecation was just

Re: Can Daffodil drop support for Scala 2.11?

2020-09-30 Thread Beckerle, Mike
There's actually some code-conditionalizations for 2.11 vs. 2.12 that would be able to go away if we get rid of 2.11, which would be an added benefit. I would support getting rid of 2.11. I did use an apache zeppelin notebook with Daffodil that needed 2.11. It was required to use 2.11 because

Turn back on emails from coverage.io

2020-09-28 Thread Beckerle, Mike
I'd like to turn back on emails coming from the automated coverage analysis. Since I had no idea this was even still running automatically, I think more noise from this it is not a distraction. At least that tells us if the automated checking still works, and a quick check of the email rules

Re: Handling Nested Suspensions

2020-09-24 Thread Beckerle, Mike
include MTA during unparse, and that instead is handled by the various separator unparsers and the suspensions they create? If if we come across other issues of nested suspensions the general solution is likely to combine the separate suspensions into a single suspension? On 9/24/20 2:12 PM, Beckerle

Re: Handling Nested Suspensions

2020-09-24 Thread Beckerle, Mike
I think there is an invariant that inserting these alignment suspensions is violating. The chain of connected DOSs, well... information propagates across them. Any time a later one in the chain wants to know information derived from prior ones, a request ripples back the chain. So there is an

Re: Removing Length Check for Complex Types

2020-09-23 Thread Beckerle, Mike
I think there is an issue here. isBitLengthOk is defined as val isLimitOk: Boolean = dis.withBitLengthLimit(nBits) { eParser.parse1(pState) } Assert.invariant(isLimitOk) You turned this invariant check into a PE if !isLimitOk. I'm going to suggest try not doing that. Just remove the

Re: Coroutines - was Re: Daffodil SAX API Proposal

2020-09-16 Thread Beckerle, Mike
I checked and the one on the gist I wrote is right. It also has a SAX producer consumer example worked out that wasn't in the original stuff that was part of daffodil. From: Beckerle, Mike Sent: Wednesday, September 16, 2020 4:07 PM To: dev@daffodil.apache.org

Re: Coroutines - was Re: Daffodil SAX API Proposal

2020-09-16 Thread Beckerle, Mike
guess if one thread is much faster than the other then there's really not much benefit and one thread might just spin waiting for the other to read/write and event? Does your coroutine library do something to prevent this from happening? On 9/16/20 10:00 AM, Beckerle, Mike wrote: > The point of

Re: Coroutines - was Re: Daffodil SAX API Proposal

2020-09-16 Thread Beckerle, Mike
e suspendable annotation). I wanted to check in to see if > there was a preferred one that I could focus my effort on? > > On 4/24/20 9:28 AM, Beckerle, Mike wrote: >> A further thought on this. The overhead difference between >> continuations and threads was 1 to 4 (roughly). >&

Re: Large dfdl:length values on complex types

2020-09-14 Thread Beckerle, Mike
of the complex type just happening to turn out to be larger than the explicit length, and also to someday implement the lengthKind 'endOfParent' for the final child. From: Beckerle, Mike Sent: Monday, September 14, 2020 11:58 AM To: dev@daffodil.apache.org Subject: Re: Large

Re: Large dfdl:length values on complex types

2020-09-14 Thread Beckerle, Mike
anonymous or named pipe are commonly used types of files which > do provide a file-length. > ____ > From: Beckerle, Mike > Sent: Monday, September 14, 2020 10:40 AM > To: dev@daffodil.apache.org > Subject: Re: Large dfdl:length values on complex t

Re: Large dfdl:length values on complex types

2020-09-14 Thread Beckerle, Mike
on backtracking >> capability, it just requires us to document them. > > The issue isn't so much backtracking, as fortracking. At the moment, we need > to read in all of dfdl:length before we begin parsing the first time, so we > get a problem even if we never attempt to backt

Re: Large dfdl:length values on complex types

2020-09-11 Thread Beckerle, Mike
e would want to use DFDL on such a format. ____ From: Beckerle, Mike Sent: Friday, September 11, 2020 1:24 PM To: dev@daffodil.apache.org Subject: Re: Large dfdl:length values on complex types Maybe a silly question, but why don't we just hit a tunable size limit

Re: Large dfdl:length values on complex types

2020-09-11 Thread Beckerle, Mike
Maybe a silly question, but why don't we just hit a tunable size limit immediately before we "try to read" that data? 256MB is very big. Is this a real format, or a test case designed to push the boundaries? From: Steve Lawrence Sent: Friday, September 11,

Re: [DRAFT] Daffodil Podling Report - September 2020

2020-09-01 Thread Beckerle, Mike
+1 Looks good From: Steve Lawrence Sent: Friday, August 28, 2020 9:30 AM To: dev@daffodil.apache.org Subject: [DRAFT] Daffodil Podling Report - September 2020 ## Daffodil Apache Daffodil is an implementation of the Data Format Description Language (DFDL)

Re: Points of Uncertainty and Unordered Sequences

2020-08-12 Thread Beckerle, Mike
There are two PoU here, one for each recurring element. The initiator once found should discriminate the element so that the assert for X, since it will fail, should cause the X array to fail entirely, and the whole schema should fail. I think you should get the message "the expected message"

Re: [VOTE] Release Apache Daffodil (incubating) 2.7.0-rc2

2020-06-29 Thread Beckerle, Mike
Urg. I updated the release instructions to remind people to edit the actual links, not just the text that HTML email editors display for the link, or use plaintext editing. I made some minor changes to the release instructions - e.g., you have to run podman in a terminal window since it

Re: [DISCUSS] Release Daffodil 2.7.0

2020-06-23 Thread Beckerle, Mike
n't mind doing it if no one else has the time right now. - Steve On 6/22/20 8:33 PM, Beckerle, Mike wrote: > I agree we can/should create a 2.7.0 for smooks and others. > > If the newVariableInstance bugs are easy, then fixing them is fine, but > otherwise I would suggest just releasing withou

Re: [DISCUSS] Release Daffodil 2.7.0

2020-06-22 Thread Beckerle, Mike
I agree we can/should create a 2.7.0 for smooks and others. If the newVariableInstance bugs are easy, then fixing them is fine, but otherwise I would suggest just releasing without those fixed. People have been living without newVariableInstance for a long time.

Re: Proposal: Infoset Streaming and Memory Reduction

2020-06-03 Thread Beckerle, Mike
minator) and discard it to return it back to the MarkPool. I hadn't thought too in detail about Unparse aspect, but what you mention was similar to how I thought these it would be handled. I'll figure out some details and update the proposal for unparse. - Steve On 6/2/20 3:26 PM, Beckerle, Mike wrote: &g

Re: Proposal: Infoset Streaming and Memory Reduction

2020-06-02 Thread Beckerle, Mike
I'm a little unclear on who owns the mark objects. You have the discriminator stack of marks, and also infoset nodes with referenced counters to marks. The discriminator stack of marks is used to determine when to call the walk() as well as to keep track of discrimination, and the infoset nodes

Re: [DRAFT] Daffodil Podling Report - June 2020

2020-05-28 Thread Beckerle, Mike
+1 Looks right to me. Under community - if you want, you can add that Nteligen has two student summer interns starting Daffodil-related projects next week. Or we can wait until the next report. Either way. From: Steve Lawrence Sent: Thursday, May 28, 2020

Re: Code generation from Daffodil runtime2

2020-05-23 Thread Beckerle, Mike
I'm not available until Tuesday, but would be happy to participate then or after. But you also don't have to wait for me. If we do something next week, then given timezones I assume this would be early in the day. My bookings Tuesday are only at 11am US.ET and 2pm, Wednesday I have a 10am

Re: Incorrect delimiter scanning when mixed encodings?

2020-04-30 Thread Beckerle, Mike
crazy. I wonder if some of the DFA complexity goes away? On 4/30/20 2:47 PM, Beckerle, Mike wrote: > > The encoding for the delimiter is the encoding in effect on the schema > component carrying the property. Making them take on contextual encodings > makes things much too complicated.

Re: Incorrect delimiter scanning when mixed encodings?

2020-04-30 Thread Beckerle, Mike
The encoding for the delimiter is the encoding in effect on the schema component carrying the property. Making them take on contextual encodings makes things much too complicated. So yeah, I think in your case, if we're scanning for that "§" but we're using a decoder for ASCII, that's

Re: Coroutines - was Re: Daffodil SAX API Proposal

2020-04-24 Thread Beckerle, Mike
: Beckerle, Mike Sent: Friday, April 24, 2020 8:53 AM To: dev@daffodil.apache.org Subject: Re: Coroutines - was Re: Daffodil SAX API Proposal That's really informative and confirms intuition that using threads really hurts performance when all you need is a stack switch. In this case reducing

Re: Coroutines - was Re: Daffodil SAX API Proposal

2020-04-24 Thread Beckerle, Mike
lly slower there would be less thread contention and actually allow for more parallel work? On 4/23/20 5:41 PM, Beckerle, Mike wrote: > I am pretty worried about the @suspendable annotation. The way this > shift/reset stuff works is it modifies the scala compiler to do something >

Re: Coroutines - was Re: Daffodil SAX API Proposal

2020-04-23 Thread Beckerle, Mike
a bit, but it's not clear to me how. On 4/22/20 7:18 PM, Beckerle, Mike wrote: > scala continuations is supported on 2.11 and 2.12, but work in progress for > 2.13. The main web page for it says it is looking for a lead developer and > without that typesafe/lightbeam is doing bar

Coroutines - was Re: Daffodil SAX API Proposal

2020-04-22 Thread Beckerle, Mike
_____ From: Beckerle, Mike Sent: Wednesday, April 22, 2020 5:01 PM To: dev@daffodil.apache.org Subject: Re: Daffodil SAX API Proposal Another possibility is scala-asynch which I think can do what we want. ____ From: Beckerle, Mike Sent: Wednesday, April 22, 2020 4:34

Re: Daffodil SAX API Proposal

2020-04-22 Thread Beckerle, Mike
Another possibility is scala-asynch which I think can do what we want. From: Beckerle, Mike Sent: Wednesday, April 22, 2020 4:34 PM To: dev@daffodil.apache.org Subject: Re: Daffodil SAX API Proposal The alternative is probably scala.util.continuations aka "

Re: Daffodil SAX API Proposal

2020-04-22 Thread Beckerle, Mike
st 3 years. There is a 2.12.x branch in their repo, but it too hasn't been updated in a long time. We might have to see how much effort it would take to update that library, or perhaps find another library. On 4/22/20 3:28 PM, Beckerle, Mike wrote: > I reviewed this and added a comment about the onl

Re: Daffodil SAX API Proposal

2020-04-22 Thread Beckerle, Mike
I reviewed this and added a comment about the only significant issue, which I think just boils down to trying to keep the coroutining back and forth as simple as possible. Another thought: Is the scala coroutines library supported in 2.11 and 2.12 (and 2.13 for being future-safe?)

Re: [VOTE] Release Apache Daffodil (incubating) 2.6.0-rc2

2020-04-10 Thread Beckerle, Mike
+1 from me. I checked that the checked out code builds and runs all built in tests. I inspected the checked out tree and found nothing surprising in it. Rather than re-inspect to double-check things other people have checked I decided to re-verify the essential functionality of the release,

Re: Using Scala 2.12.11

2020-04-06 Thread Beckerle, Mike
iar with how Scala version compatibility works, but I get > the sense it is pretty strict. Would we risk breaking things for people > running 2.12.11? > ____ > From: Beckerle, Mike > Sent: Monday, April 6, 2020 10:10 AM > To: dev@daffodil.apache.org &

Using Scala 2.12.11

2020-04-06 Thread Beckerle, Mike
I recently heard of a developer trying to use VSCode for Scala on Daffodil. Looks like there is a requirement for Scala version 2.12.11, which is just a minor point release from what we've been using which is 2.12.10. Can we move to 2.12.11 for the 2.6.0 release? Mike Beckerle | Principal

Recovery from parse errors - design note

2020-04-03 Thread Beckerle, Mike
I created a wiki design note about this topic: https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Recovery+from+Parse+Errors

Re: Compiler.setExternalDFDLVariable(s) considered challenged

2020-03-26 Thread Beckerle, Mike
d. And if someone adds their own locking around everything, then our internal locking isn't necessary. It feels to me like setExternalVariable is just fundamentally broken, and so trying to fix it is just going to add complication that still doesn't solve the fundamental problems. On 3/25/20 3:50 PM

sbt testOnly not working for me

2020-03-25 Thread Beckerle, Mike
So due to bug DAFFODIL-2302, I have a schema where I can't run all the tests at once via 'sbt test'. I wanted to run the tests separately via 'sbt testOnly **.Link16Test' and 'sbt testOnly **.Link16DERGTest' which are two different TDML test suites. But when I do that, in every case it runs

Re: Compiler.setExternalDFDLVariable(s) considered challenged

2020-03-25 Thread Beckerle, Mike
it". And rather than locking and asserting, we could just create a warning and say "You're using this in a potentially dangerous manner, you should use the new API"? - Steve On 3/25/20 3:35 PM, Beckerle, Mike wrote: > A fully functional API is probably good to have, and I'd li

Re: Compiler.setExternalDFDLVariable(s) considered challenged

2020-03-25 Thread Beckerle, Mike
locked" error because they tried to change some state. On 3/25/20 11:50 AM, Beckerle, Mike wrote: > Here's my concrete proposal to fix DataProcesor's API. > > We add a clone() method. It copies the object sharing the big/expensive stuff > like the compiled schema but gi

Re: [DISCUSS] Daffodil 2.6.0 Release

2020-03-25 Thread Beckerle, Mike
I have two issues I'd like fixed for 2.6.0 https://issues.apache.org/jira/browse/DAFFODIL-2302 - this is making it difficult to update a Link16 schema that is in use by multiple parties. There's a second issue, which is that we still have this "NadaParsers are supposed to optimize out"

Re: Compiler.setExternalDFDLVariable(s) considered challenged

2020-03-25 Thread Beckerle, Mike
arguments to the parse/unparse calls. ____ From: Beckerle, Mike Sent: Wednesday, March 25, 2020 11:29 AM To: dev@daffodil.apache.org Subject: Re: Compiler.setExternalDFDLVariable(s) considered challenged To respond to my own thread here, I think given that we allo

Re: Compiler.setExternalDFDLVariable(s) considered challenged

2020-03-25 Thread Beckerle, Mike
. From: Beckerle, Mike Sent: Wednesday, March 25, 2020 10:55 AM To: dev@daffodil.apache.org Subject: Compiler.setExternalDFDLVariable(s) considered challenged Why does the API for Daffodil have Compiler.setExternalDFDLVariable(...) and Compiler.setExternalDFDLVariables

Compiler.setExternalDFDLVariable(s) considered challenged

2020-03-25 Thread Beckerle, Mike
Why does the API for Daffodil have Compiler.setExternalDFDLVariable(...) and Compiler.setExternalDFDLVariables(...) on it. I believe we should deprecate this. Compilers are parameterized by some of the tunables I understand. But the external DFDL variables? These cannot affect compilation.

sonarcloud scan on push failing

2020-02-28 Thread Beckerle, Mike
I forced pushed a branch for a PR, it was failing to checkout before probably due to a transient github issue. Now it is failing the sonarcloud scan... except I thought those scans were only being done on merges to master?

Re: [DRAFT] Daffodil Podling Report - March 2020

2020-02-28 Thread Beckerle, Mike
. https://cwiki.apache.org/confluence/display/INCUBATOR/March2020#Daffodil We still have have a few more days to make updates if there is anything else of interest to add. Just wanted to get this up before the weekend so I don't forget. On 2/28/20 11:25 AM, Beckerle, Mike wrote: > Julian has not b

Re: [DRAFT] Daffodil Podling Report - March 2020

2020-02-28 Thread Beckerle, Mike
Julian has not been actively contributing, but just this week we got interest from another developer who has not previously contributed, but who wants to work on Runtime 2. That's why I actually have been rebasing that branch forward this week. From: Sloane,

Re: progress on schema compiler space/speed issue

2020-02-18 Thread Beckerle, Mike
addressed. ...mikeb ____ From: Beckerle, Mike Sent: Sunday, February 16, 2020 12:09 AM To: dev@daffodil.apache.org Cc: Sood, Harinder Subject: Re: progress on schema compiler space/speed issue Compiler space/speed issue is *solved* to my satisfaction. All tests pass. I a

Re: progress on schema compiler space/speed issue

2020-02-15 Thread Beckerle, Mike
terize what happens if we do these unimplemented things. ________ From: Beckerle, Mike Sent: Thursday, February 13, 2020 10:29 PM To: dev@daffodil.apache.org Subject: progress on schema compiler space/speed issue See https://github.com/apache/incubator-daffodil/pull/324 I

progress on schema compiler space/speed issue

2020-02-13 Thread Beckerle, Mike
See https://github.com/apache/incubator-daffodil/pull/324 I still have debugging to do, and I'm not happy with the non-type-safety, and maintainability headaches this creates, but I was able to get rid of the combinatorial explosion with a modest number of lines of code changed. But most

<    1   2   3   4   >