OK, I'll summarize the discussion to see if we have a consensus now. Right now
we have the following subprojects:
- containers
- daffodil-cli
- daffodil-core
- daffodil-io
- daffodil-lib
- daffodil-macro-lib
- daffodil-propgen
- daffodil-runtime1
- daffodil-runtime1-unparser
- daffodil-runtime2
- daffodil-sapi
- daffodil-tdml-lib
- daffodil-tdml-processor
- daffodil-test
- daffodil-test-ibm
- daffodil-udf
- project
- test-stdLayout
- tutorials
Now we might have the following new names:
- containers
- daf-backend-c-generator
- daf-backend-scala-parser (to be merged as daf-backend-scala)
- daf-backend-scala -unparser (to be merged as daf-backend-scala)
- daf-cli
- daf-io
- daf-lib
- daf-macro-lib
- daf-propgen
- daf-sapi
- daf-schema-compiler
- daf-tdml-lib
- daf-tdml-processor
- daf-test
- daf-test-ibm
- daf-udf
- project
- test-stdLayout
- tutorials
If we dropped the daf- prefix, we might need another name for test-stdLayout
(it would not be clear to me what the difference between test and
test-stdLayout is). I see Steve just commented that the sbt commands for
subprojects also would need to drop their "daffodil-" prefix and this would
introduce a confusion with the "sbt test" command. I'll let devs decide how to
make the names of test subprojects and sbt commands clearer.
However, Mike just pointed out we should build another set of "c-cli" C source
files (the C source files used for CLI parsing and conversion to/from XML to
support TDML tests) separately from the "common runtime" C source files (the
primitive parsers/unparsers called from the generated code) and the "generated
code" C source files (the result of compiling the DFDL schema). He also said
in a review comment:
Eventually, this "static code" is going to become a rather large
library. As big as both runtime1 parser and unparser together. Bigger given
that this is C code, in terms of Lines of code of course. Think in terms of 50K
lines of code kind of library. So I think it needs to be built as a static
library.
These comments motivate me to say that we eventually might need several
subprojects for the C runtime2 backend, not just one, since one subproject
should build one library, correct? I'm torn a bit, though. We can put C
source files in subdirectories like packages and generate multiple libraries
from these multiple directories using the sbt C compiler plugin without any
difficulty too. (I know the topic of whether to compile C source files in the
sbt build and distribute C-code libraries with daffodil warrants its own
discussion on the dev list too.)
If we keep going without any further changes, then after two more pull
requests, we would end up with:
- containers
- daf-backend-c-generator
- daf-backend-scala
- daf-cli
- daf-io
- daf-lib
- daf-macro-lib
- daf-propgen
- daf-sapi
- daf-schema-compiler
- daf-tdml-lib
- daf-tdml-processor
- daf-test
- daf-test-ibm
- daf-udf
- project
- test-stdLayout
- tutorials
-----Original Message-----
From: Steve Lawrence <slawre...@apache.org>
Sent: Monday, October 5, 2020 12:00 PM
To: dev@daffodil.apache.org
Subject: EXT: Re: Subproject names proposed for discussion
I'm fine with just removing it completely. As I said, I don't think the reasons
I listed are strong reasons for keeping a prefix.
On 10/5/20 11:36 AM, Beckerle, Mike wrote:
> A shorter prefix like just "daf-" would bother me less.
>
> It literally becomes screen-real-estate using IDEs because the project trees
> extend from the left of the screen, and they just go too far right before you
> get to the actual files contained.
> ________________________________
> From: Steve Lawrence <slawre...@apache.org>
> Sent: Monday, October 5, 2020 11:25 AM
> To: dev@daffodil.apache.org <dev@daffodil.apache.org>
> Subject: Re: Subproject names proposed for discussion
>
> Your suggestion seems reasonable to me. Adds a few more top level
> directories by splitting lib and schema-compiler which I'm not a huge
> fan of, but it is logical and reduces the size the lib jars.
>
> The one thing I like about the "daffodil-" prefix is it makes the code
> directories stand out a bit more. For example, the containers,
> project, target, tutorials, etc. directories or more about
> infrastructure. It makes it eaiser to ignore those when scanning
> through directories (of which this change would add quite a few more).
> I think this is also pretty common among java projects, but I don't
> think they all do it. But I'm not sure if either of those are strong
> reasons to keep it. We have had problems with long paths in the past
> on windows, so removing the prefix could help with that...
>
>
> On 10/5/20 11:11 AM, Beckerle, Mike wrote:
>> Some reponses inline below
>>
>> ________________________________
>> From: Steve Lawrence <slawre...@apache.org>
>> Sent: Monday, October 5, 2020 10:48 AM
>> To: dev@daffodil.apache.org <dev@daffodil.apache.org>
>> Subject: Re: Subproject names proposed for discussion
>>
>> A handful of unrelated thoughts, maybe overthinking things and I
>> don't feel strongly about anything below, but renaming is always pain
>> so it'd be nice to ensure we have something future proof.
>>
>> 1) Is there any benefit organizationally to having all backends being
>> in the same directory?
>>
>> I think the opposite. I think they should be separate and top level in fact,
>> so that we're not tied to say, sbt build system as we are now.
>>
>> 2) From a sorting perspective, it'd be nice if the scala projects
>> were together, so having it be scala-parser and scala-unparser rather
>> than parser-scala and unparser-scala has advantages.
>> Agree.
>>
>> 3) Maybe the scala parser/unparser should be considered the same "scala"
>> runtime, and so parser/unparser should be subdirectories of a
>> "daffodil-backend-scala" subdirectory?
>>
>> There is no reason to keep the scala parser and unparser separate.
>>
>> That was done at one time because compilation time was so long that we
>> wanted to be able to close the unparser project (on eclipse) and not be
>> recompiling it if just doing work involving the parser. This is less an
>> issue now.
>>
>> 4) Is there even a benefit to separating parser/unparser into
>> separatejars? There's so much shared logic between the two, and
>> there's even a bunch of unparsing stuff in the parser jar. Should we just
>> combine them under the same backend?
>>
>> Yes, but one step at a time. First I would do the renaming. I'd do the
>> merging of parser/unparser into one lib separately.
>>
>> Taking all of the above into account, perhaps something like this:
>>
>> ...
>> |-- daffodil-backends
>>
>> I would not add the backends parent.
>>
>> daffodil-backend-scala
>>
>> | |-- daffodil-scala
>> | | `-- src
>> | `-- daffodil-generator-c
>>
>> I would call the above daffodil-c-generator to be consistent. It is
>> important that this is identified as a generator.
>>
>> While we're at it, can't we drop the "daffodil-" prefix on all these?
>>
>> | `-- src
>> |-- daffodil-lib
>> | `-- src
>> |-- daffodil-schema-compiler
>>
>> This is problematic because each back-end is going to have its part of the
>> schema compiler that converts grammar objects into back-end primitives code
>> by generation of code that calls libraries.
>>
>> This should be distinguished from the 100% static code part of the back-end
>> which is statically written and testable independently.
>>
>> | `-- src
>> ...
>>
>> 5) Is there something better than "backend" for describing these. I
>> can't think of anything. Does the DFDL spec have a concept of this?
>>
>> There is no concept of this in the DFDL spec to borrow from.
>>
>> 6) Are there any benefits to using "codenames". My thinking is maybe
>> someday there could be multiple "scala" backends with different
>> goals/extensions, and so "daffodil-scala" is too generic. Codenames
>> would be more like what we have today, except real code names might
>> be easier to remember than "runtime1" and "runtime2". Disadvantage is
>> there's less discoverability, but a README could be added with short
>> descriptions about what the backends try to accomplish. Not sure I
>> like this, but thought I'd throw it out there.
>>
>> Naming backends after "famous runtimes" as in marathons and other road
>> races, .... except those just end up being named after places or people.
>> Might as well use the name or home-town of the person who wrote the first
>> line of code in whatever backend it is.
>>
>> Personally, I find separating them into backend scala, vs. backend
>> c-generator works fine, and if it splits below that, numbering will be
>> sufficient.
>>
>> My summary:
>>
>> daffodil
>> backend-scala-schema-compiler
>> core-schema-compiler
>> backend-c-generator-schema-compiler
>> lib
>> backend-scala-parser-lib
>> backend-scala-unparser-lib
>> backend-c-lib
>> tdml-lib
>> tdml-processor
>> .....
>>
>>
>>
>>
>>
>>
>> On 10/5/20 10:23 AM, Beckerle, Mike wrote:
>>> +1 from me.
>>>
>>> ________________________________
>>> From: Interrante, John A (GE Research, US)
>>> <inter...@research.ge.com>
>>> Sent: Monday, October 5, 2020 9:28 AM
>>> To: dev@daffodil.apache.org <dev@daffodil.apache.org>
>>> Subject: Subproject names proposed for discussion
>>>
>>> Steve Lawrence and I would like to bring a topic to the dev list for
>>> discussion since not everyone is paying attention to the review of my
>>> runtime2 push request. Steve suggested, and I agree, that renaming some of
>>> the Daffodil subprojects might make their meanings more obvious to newcomer
>>> devs. If we do rename some subprojects after discussing it on this list,
>>> we will do it immediately in its own pull request since mixing changes with
>>> renames makes it difficult to see which changes are just renames instead of
>>> actual changes.
>>>
>>> What do devs think about us renaming some subprojects like this?
>>>
>>> rename daffodil-core to daffodil-schema-compiler
>>> leave daffodil-lib alone
>>> rename daffodil-runtime1 to daffodil-backend-parser-scala
>>> rename daffodil-runtime1-unparser to daffodil-backend-unparser-scala
>>> rename daffodil-runtime2 to daffodil-backend-generator-c
>>>
>>>
>>
>>
>>
>
>
