Re: Help implementing schema for SISO-J

or this type of packing > so that LINK-16 schemas can be reused in SISO-J messages. When does > the workgroup meet next? What do you think the likelihood is for a > change like this to be accepted? > > > Marshall Miller > > On Thu, Jan 11, 2018 at 3:48 PM, Mike Beckerle <

Code review request

I would very much like to get the changes described by these two pull requests integrated before the big apache package and file change. https://github.com/apache/incubator-daffodil/pull/23 https://github.com/mbeckerle/incubator-daffodil/pull/4 These two PRs are effectively part of the same

Re: Roadmap for next Daffodil releases

s to really do query-style expressions. From: Steve Lawrence <slawre...@apache.org> Sent: Thursday, January 4, 2018 11:00:33 AM To: dev@daffodil.apache.org; Mike Beckerle Subject: Re: Roadmap for next Daffodil releases I think this seems like a reasona

Please review & discuss - draft proposal for how to do base64, foldedLines, etc.

Attached is a note about a feature for DFDL/Daffodil to support base64, folded lines, etc. Formats using these are in high demand in the network-security community.

Re: need more code review on pull request #12

and address it from there, and get out profiling tools. At this point, due to the Xmas break, I'm assuming I won't get a final review/approval cycle until after the holidays. Oh well. Happy holidays everyone! ...mikeb From: Mike Beckerle Sent: Thursday, December 21

latest pushes to PR 12

So I have fixed all the issues with PR 12 save 1. I added 4 commits. Granular is better with this review system I've determined. Any insight into what changed that caused the need for the changes to InteractiveDebugger.scala in the last (6th) commit... are welcome. I didn't like putting this

need more code review on pull request #12

I have pushed updated commits to my pull request here: https://github.com/apache/incubator-daffodil/pull/12 I have done this in a way that I thought doesn't trash prior comments. But alas, they seem to be gone. I added a 2nd commit to the pull request for review purposes. I didn't

Re: No more write access?

7 3:13:27 PM To: dev@daffodil.apache.org Subject: Re: No more write access? Mike, Can you share your github username? As well as your apache username. John On Mon, Dec 18, 2017 at 2:45 PM Mike Beckerle <mbecke...@tresys.com> wrote: > I can't find any options for merging/rebasing

Re: Removal of ByteBuffer backing in the I/O layer

he.org; Mike Beckerle Subject: Re: Removal of ByteBuffer backing in the I/O layer Kindof. I was thinking that part of the resetForContinuedParse() method could delete all buckets from the front of the ArrayBuffer of the previous parse and move the last remaining buckets for the next parse to the

Re: Removal of ByteBuffer backing in the I/O layer

Additional operation is to support backtracking. The current single buffer design makes this easy - you just save/restore the position and limit, and restore after. With a bucketing algorithm the backtracking is trickier, because it is resolution of a point of uncertainty (PoU) that allows one

Re: Unordered Sequence and Choice Branch Violations

Lawrence; dev@daffodil.apache.org; Mike Beckerle Subject: Re: Unordered Sequence and Choice Branch Violations So having implemented this approach, Steve, I've run into a bit of a snafu. What I've done is that I modified the UnorderedSequenceCombinator to use the AltComp and RepUnbounded parsers

Re: When are Required Evaluations executed?

requiredEvaluations for all objects that have been created, are run when isError is called on the ProcessorFactory object. From: Taylor Wise Sent: Wednesday, November 29, 2017 12:43:17 PM To: Mike Beckerle Cc: dev@daffodil.apache.org Subject: When are Required

unable to merge into support/v2.0.x

I am unable to merge to close out my pull request. https://github.com/apache/incubator-daffodil/pull/6 Is there a permissions issue? This is not going to master, it's going to support/v2.0.x.

Re: Issue with separatorSuppressionPolicy and empty elements in IBM4690-TLOG scheams

imize out - the guard should be false if N is zero. That would fix the assertion failure. From: Joshua Adams Sent: Tuesday, November 28, 2017 12:37:46 PM To: Mike Beckerle; Steve Lawrence; dev@daffodil.apache.org Subject: Re: Issue with separatorSuppressionPoli

Re: Issue with separatorSuppressionPolicy and empty elements in IBM4690-TLOG scheams

g at the same schema. This matters in that for the optional items, separators can be present or absent. From: Steve Lawrence <slawre...@apache.org> Sent: Tuesday, November 28, 2017 8:08:25 AM To: Mike Beckerle; Joshua Adams; dev@daffodil.apa

Re: Issue with separatorSuppressionPolicy and empty elements in IBM4690-TLOG scheams

separators aren't processed,...". Probably best for us to talk this through on phone tomorrow (Tuesday). Look for me on the instant messenger. From: Joshua Adams Sent: Monday, November 27, 2017 3:59:14 PM To: Mike Beckerle; dev@daffodil.apache.org Cc: Stephen

Fw: LWAP project needs a fix to v2.0.0

Forgot to send to the dev list. From: Mike Beckerle Sent: Wednesday, November 22, 2017 1:30 PM To: Stephen Lawrence; Taylor Wise; Joshua Adams; Dave Thompson Subject: LWAP project needs a fix to v2.0.0 So in building some DFDL shcemas for LWAP I found a bug

Re: TIFF format - base+offset proposed features

rmat might be described in this way. From: Steve Lawrence <slawre...@apache.org> Sent: Friday, November 17, 2017 8:34:30 AM To: dev@daffodil.apache.org; Mike Beckerle Subject: Re: TIFF format - base+offset proposed features On 11/17/2017 08:05 AM, Mik

Proposal for Implementing Base64, folded lines, quoted-printable, compress/decompress, etc.

Perhaps a URL-style thing like dfdl:transferEncoding='compress?method=bz2' ...mike beckerle

final review of pull request 5

I have made a final set of review-driven changes to fix the typos and such found in the last review. If this patch is now acceptable, it should be squashed from 3 to 1 commit before merging/rebase. Who does that? The developer, or the person doing the merge/integration.

Re: streaming API design note for review

Comments inline below. From: Steve Lawrence <slawre...@apache.org> Sent: Thursday, November 16, 2017 10:04 AM To: Mike Beckerle; dev@daffodil.apache.org Subject: Re: streaming API design note for review On 11/09/2017 03:23 PM, Mike Beckerle wrote: >

Re: Extensibility for Daffodil - Concept/Idea for commentary

Yeah, so I'm thinking of turning this idea into a tutorial overview of how to add a feature to Daffodil. From: Steve Lawrence <slawre...@apache.org> Sent: Monday, November 13, 2017 8:34:59 AM To: dev@daffodil.apache.org; Mike Beckerle Subject: Re: Extensi

Extensibility for Daffodil - Concept/Idea for commentary

We need additional features in Daffodil. The number of variations on data formats is likely endless, and while DFDL is quite comprehensive there are still things it cannot express. Here's a way to extend daffodil quite fundamentally, but from outside. First we declare a DFDL extension like

Stack overflows, UnorderedSequences, DSOM

So I've had some stack-overflows while debugging of late. This has lead to thinking about the reasons why. There are some principles of operation for the code-base of Daffodil that are not articulated, but that need to be followed to stay out of trouble. I am planning to add package.scala

Re: streaming API design note for review

The low-level changes to support this API I believe require us to implement the bucket algorithm for the DataInputStream layer, so that the stream isn't being read into an ever-growing buffer. The key need/idea here is that the state, including our DaffodilInputStream state, and it's current

UML diagram to help with review

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74686682 Is a page with a UML diagram for the primary relationships among the various traits and classes.

streaming API design note for review

I created a design note on Streaming API features we need for message-by-message processing style. https://cwiki.apache.org/confluence/display/DAFFODIL/Message+Streaming+API+-+Eliminating+the+Single+Root+Element+Problem Please edit the wiki page directly, inserting comments in sub-paragraphs.

Re: [GitHub] incubator-daffodil pull request #4: Ensure marks are always returned to the ...

There are options in the review system for viewing the latest commit, or all commits. That may work well with the workflow you suggest here. We definitely want the squash done before the final rebase and merge. From: Steve Lawrence Sent:

Re: [GitHub] incubator-daffodil issue #4: Ensure marks are always returned to the markpoo...

To use the git log -p -1 hash suggestion, I have to pull your branch from your forked repo... which is where exactly? github calls it this but that's not the URL I need. stevedlawrence:daffodil-1862-markpool From: stevedlawrence Sent:

Re: Daffodil Website

Why wouldn't we just put an automatic redirect to a home landing page on the wiki? From: Steve Lawrence Sent: Friday, November 3, 2017 10:49:08 AM To: dev@daffodil.apache.org Subject: Daffodil Website I think our infrastructure move is

Re: [GitHub] incubator-daffodil pull request #1: Property Resolution - part of fixing sch...

One thing I'm hoping there is a way to do. I'd like to see all the files in a review in a tree view. Using fisheye on NCSA the review window has a tree-view of the complete change set on the left, so you can jump around the change in a logical order based on where and what all the changes

Hipchat account?

So our HipChat is now available: https://www.hipchat.com/gJt9EQs5l But when I try to actually login to this (as opposed to using it as guest), it asks for email, and it won't accept mbecke...@apache.org. Shouldn't that be my credentials on hipchat?

Re: Unordered Sequences Thoughts

A few more comments inline in blue From: Taylor Wise Sent: Tuesday, October 24, 2017 11:21 AM To: dev@daffodil.apache.org Subject: Fw: Unordered Sequences Thoughts Forwarding discussion on this. From: Stephen

Re: Import of Code to ASF Infrastructure

o: dev@daffodil.apache.org, Mike Beckerle <mbecke...@tresys.com> Subject: Re: Import of Code to ASF Infrastructure The problem is you really can't delete tags in git, so now is really the best time to do it since we're moving repos. Is the disruption having to move your local review branches f

Re: Remove concept of slots from InfosetImpl Design Discussion

773> opensource.ncsa.illinois.edu Daffodil; DFDL-1773; Choice ambiguous element name results in failed expression From: Taylor Wise Sent: Monday, October 23, 2017 12:43:36 PM To: Mike Beckerle; Steve Lawrence; dev@daffodil.apache.org Subject: Re: Remove concept of slot

Re: Remove concept of slots from InfosetImpl Design Discussion

Yes. The one giant ticket about the space problem compiling (DFDL-1444) is not specific enough. From: Taylor Wise Sent: Monday, October 23, 2017 12:28:04 PM To: Steve Lawrence; dev@daffodil.apache.org; Mike Beckerle Subject: Re: Remove concept of slots from

Notes on Daffodil Schema Compiler redesign - improve footprint and speed

ill needed - because it behaves like alignment unparser, but see below - we're getting rid of alignment parsers/unparsers too. * Mike Beckerle has code for this working except for bitOrderChangeUnparser and alignment unparsers are still there. * Alignment * SDE on outputValueCalc (OVC)

Packed and Zoned Design details

Our initial goal for the packed/zoned number support is to be able to run the IBM-created schemas on github for ISO8583 format, and for IBM4690_TLOG format, as well as run all the scala-debug tests we have that use the packed/zoned and other features that we've not yet implemented. What we

Interoperability goals for Daffodil to support DFDL in OGF Standardization

This is an introduction to our "interoperability" goals, to be followed by a number of design topic threads about implementing the features and tools we need. A good interoperability demonstration is the primary thing the Daffodil project needs to achieve in order to advance the DFDL

Re: Remove concept of slots from InfosetImpl Design Discussion

@daffodil.apache.org; Mike Beckerle Subject: Re: Remove concept of slots from InfosetImpl Design Discussion I'm down to a single error: test_dfdlPosition3 java.lang.Exception: Comparison failed. Expected 212212 Actual 212122 Differences were (path, expected, actual): (f,2,1

Re: When to discuss in email thread, when to use a document, or wiki page, or other?

need to register separately. > > > > John > > > > On Thu, Oct 12, 2017 at 7:59 AM Steve Lawrence <slawre...@apache.org> > wrote: > > > >> On 10/11/2017 01:58 PM, Mike Beckerle wrote: > >>> Lots of things are just too complicated to discu

When to discuss in email thread, when to use a document, or wiki page, or other?

Lots of things are just too complicated to discuss and comment on in an email thread. E.g., code. E.g., design documents. We have a big backlog of design notes/specs to create. My immediate task is to start writing these and getting them into review. How should I be creating these? Where

Re: Infrastructure Changes for ASF

Very good. I forgot that one can setup a git remote to several other repositories. That solves the visibility issue nicely. From: Steve Lawrence <slawre...@apache.org> Sent: Wednesday, October 11, 2017 12:39:40 PM To: dev@daffodil.apache.org; Mike Be

Re: Infrastructure Changes for ASF

Review branches have a lot of value. 1) Work Visibility I like being able to see OTHER developers review branches as a means of avoiding duplicate work, seeing if they are progressing, or the branches go quiet, are their changes going to create an integration nightmare with mine, or my

<    1   2   3   4   5   6