This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/plc4x-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 521b501 Site checkin for project PLC4X: Jenkins Tools
521b501 is described below
commit 521b5013bedab14426cc29a04f5d98522aaaaa7c
Author: jenkins <[email protected]>
AuthorDate: Sun Nov 28 14:08:08 2021 +0000
Site checkin for project PLC4X: Jenkins Tools
---
developers/code-gen/index.html | 56 +++++++----
developers/code-gen/language/freemarker.html | 138 +++++++++++++++++++++++----
2 files changed, 157 insertions(+), 37 deletions(-)
diff --git a/developers/code-gen/index.html b/developers/code-gen/index.html
index bc21f45..c572c02 100644
--- a/developers/code-gen/index.html
+++ b/developers/code-gen/index.html
@@ -269,10 +269,10 @@
<p>Concrete protocol spec parsers and templates that actually generate code
are implemented in derived modules.</p>
</div>
<div class="paragraph">
-<p>Also didn’t we want to tie ourselves to only one way to specify
protocols and to generate code.</p>
+<p>We didn’t want to tie ourselves to only one way to specify protocols
and to generate code. Generally multiple types of formats for specifying
drivers are thinkable and the same way multiple ways of generating code are
possible. Currently however we only have one parser: <code>MSpec</code> and one
generator: <code>Freemarker</code>.</p>
</div>
<div class="paragraph">
-<p>This adds another layer to the hierarchy.</p>
+<p>These add more layers to the hierarchy.</p>
</div>
<div class="paragraph">
<p>So for example in case of generating a Siemens S7 Driver for Java this
would look like this:</p>
@@ -300,7 +300,7 @@
<p>Usually using them required a large amount of workarounds, which made the
solution quite complicated.</p>
</div>
<div class="paragraph">
-<p>In the end only DFDL and the corresponding Apache project <a
href="https://daffodil.apache.org";>Apache Daffodil (incubating)</a> seemed to
provide what we were looking for.</p>
+<p>In the end only DFDL and the corresponding Apache project <a
href="https://daffodil.apache.org";>Apache Daffodil</a> seemed to provide what
we were looking for.</p>
</div>
<div class="paragraph">
<p>With this we were able to provide first driver versions fully specified in
XML.</p>
@@ -321,7 +321,20 @@
<p>In general all you need to specify, is the <code>protocolName</code> and
the <code>languageName</code>.</p>
</div>
<div class="paragraph">
-<p>There is also an additional parameter: <code>outputDir</code>, which
defaults to <code>${project.build.directory}/generated-sources/plc4x/</code>
and usually shouldn’t require being changed.</p>
+<p>An additional option <code>outputFlavor</code> allows generating multiple
versions of a driver for a given language. This can come in handy if we want to
be able to generate <code>read-only</code> or <code>passive mode</code> driver
variants.</p>
+</div>
+<div class="paragraph">
+<p>Last, not least, we have a pretty generic <code>options</code> config
option, which is a Map type.</p>
+</div>
+<div class="paragraph">
+<p>With options is it possible to pass generic options to the code-generation.
So if a driver or language requires further customization, these options can be
used.</p>
+</div>
+<div class="paragraph">
+<p>Currently, the <code>Java</code> module makes use of such an option for
specifying the Java <code>package</code> the generated code uses.
+If no <code>package</code> option is provided, the default package
<code>org.apache.plc4x.{language-name}.{protocol-name}.{output-flavor}</code>
is used, but especially when generating custom drivers, which are not part of
the Apache PLC4X project, different package names are better suited. So in
these cases, the user can simply override the default package name.</p>
+</div>
+<div class="paragraph">
+<p>There is also an additional parameter: <code>outputDir</code>, which
defaults to <code>${project.build.directory}/generated-sources/plc4x/</code>
and usually shouldn’t require being changed in case of a
<code>Java</code> project, but usually requires tweaking when generating code
for other languages.</p>
</div>
<div class="paragraph">
<p>Here’s an example of a driver pom for building a <code>S7</code>
driver for <code>java</code>:</p>
@@ -375,6 +388,7 @@
<configuration>
<protocolName>s7</protocolName>
<languageName>java</languageName>
+ <outputFlavor>read-write</outputFlavor>
</configuration>
</execution>
</executions>
@@ -410,7 +424,7 @@
</div>
</div>
<div class="paragraph">
-<p>So the plugin configuration is pretty straight forward, all that is
specified, is the <code>protocolName</code> and the
<code>languageName</code>.</p>
+<p>So the plugin configuration is pretty straight forward, all that is
specified, is the <code>protocolName</code>, <code>languageName</code> and the
<code>output-flavor</code>.</p>
</div>
<div class="paragraph">
<p>The dependency:</p>
@@ -428,7 +442,7 @@
<p>For example contains all classes the generated code relies on.</p>
</div>
<div class="paragraph">
-<p>The definitions of both the <code>s7</code> protocol as well as the
<code>java</code> language are provided by the two dependencies:</p>
+<p>The definitions of both the <code>s7</code> protocol and <code>java</code>
language are provided by the two dependencies:</p>
</div>
<div class="literalblock">
<div class="content">
@@ -456,7 +470,7 @@
</div>
</div>
<div class="paragraph">
-<p>The reason for why the dependencies are added as code-dependencies and why
the scope is set the way it is is described in the <a
href="#why_are_the_protocol_and_language_dependencies_done_so_strangely">Why
are the protocol and language dependencies done so strangely?</a> section.</p>
+<p>The reason for why the dependencies are added as code-dependencies and why
the scope is set the way it is, is described in the <a
href="#why_are_the_protocol_and_language_dependencies_done_so_strangely">Why
are the protocol and language dependencies done so strangely?</a> section.</p>
</div>
</div>
<div class="sect2">
@@ -496,7 +510,7 @@ public interface Protocol {
* @return the Map of types that need to be generated.
* @throws GenerationException if anything goes wrong parsing.
*/
- Map<String, ComplexTypeDefinition> getTypeDefinitions() throws
GenerationException;
+ Map<String, TypeDefinition> getTypeDefinitions() throws
GenerationException;
}</pre>
</div>
@@ -506,7 +520,7 @@ public interface Protocol {
They could even be hard coded.</p>
</div>
<div class="paragraph">
-<p>However we have currently implemented utilities for universally providing
input:</p>
+<p>However, we have currently implemented utilities for universally providing
input:</p>
</div>
<div class="ulist">
<ul>
@@ -543,8 +557,18 @@ public interface LanguageOutput {
*/
String getName();
- void generate(File outputDir, String packageName, Map<String,
ComplexTypeDefinition> types)
- throws GenerationException;
+ List<String> supportedOutputFlavors();
+
+ /**
+ * An additional method which allows generator to have a hint which
options are supported by it.
+ * This method might be used to improve user experience and warn, if set
options are ones generator does not support.
+ *
+ * @return Set containing names of options this language output can accept.
+ */
+ Set<String> supportedOptions();
+
+ void generate(File outputDir, String languageName, String protocolName,
String outputFlavor,
+ Map<String, TypeDefinition> types, Map<String, String>
options) throws GenerationException;
}</pre>
</div>
@@ -553,7 +577,7 @@ public interface LanguageOutput {
<p>The file for registering Language modules is located at:
<code>META-INF/services/org.apache.plc4x.plugins.codegenerator.language.LanguageOutput</code></p>
</div>
<div class="paragraph">
-<p>Same as with the protocol modules, the language modules could too be
implemented in any thinkable way, however we have already implemented some
helpers for using:</p>
+<p>Same as with the protocol modules, the language modules could also be
implemented in any thinkable way, however we have already implemented some
helpers for using:</p>
</div>
<div class="ulist">
<ul>
@@ -575,7 +599,7 @@ public interface LanguageOutput {
<p>This is due to some restrictions in Maven, which result from the way Maven
generally works.</p>
</div>
<div class="paragraph">
-<p>The main problem is that when starting a build, in the
<code>validate</code> phase, Maven goes through the configuration, downloads
the plugins and configures these.
+<p>The main problem is that when starting a build, in the
<code>validate</code>-phase, Maven goes through the configuration, downloads
the plugins and configures these.
This means that Maven also tries to download the dependencies of the plugins
too.</p>
</div>
<div class="paragraph">
@@ -589,7 +613,7 @@ In this case the build will fail because there is no Plugin
with that version to
In this case the only option would be to manually build and install the plugin
in the release version and to re-start the release (Which is not a nice thing
for the release manager).</p>
</div>
<div class="paragraph">
-<p>For this reason we have stripped down the plugin and it’s
dependencies to an absolute minimum and have released (or will release) that
separately from the rest, hoping due to the minimality of the dependencies that
we will not have to do it very often.</p>
+<p>For this reason we have stripped down the plugin and its dependencies to an
absolute minimum and have released (or will release) that separately from the
rest, hoping due to the minimality of the dependencies that we will not have to
do it very often.</p>
</div>
<div class="paragraph">
<p>As soon as the tooling is released, the version is updated in the PLC4X
build and the release version is used without any complications.</p>
@@ -608,7 +632,7 @@ So during a release the new versions of the modules
wouldn’t exist, this w
<p>We could release the protocol- and the language modules separately too, but
we want the language and protocol modules to be part of the project, to not
over-complicate things - especially during a release.</p>
</div>
<div class="paragraph">
-<p>So the Maven plugin is built in a way, that it uses the modules
dependencies and creates it’s own Classloader to contain all of these
modules at runtime.</p>
+<p>So the Maven plugin is built in a way, that it uses the modules
dependencies and creates its own Classloader to contain all of these modules at
runtime.</p>
</div>
<div class="paragraph">
<p>This brings the benefit of being able to utilize Maven’s capability
of determining the build order and dynamically creating the modules build
classpath.</p>
@@ -625,7 +649,7 @@ Here the vendor of a Servlet engine is expected to provide
an implementation of
It is forbidden for an application to bring this along, but it is required to
build the application.</p>
</div>
<div class="paragraph">
-<p>For this the Maven scope <code>provided</code>, which tells Maven to
provide it during the build, but to exclude it from any applications it builds
because it will be provided by the system running the application.</p>
+<p>For this the Maven scope <code>provided</code>, which tells Maven to
provide it during the build, but to exclude it from any applications it builds,
because it will be provided by the system running the application.</p>
</div>
<div class="paragraph">
<p>This is not quite true, but it does the trick.</p>
diff --git a/developers/code-gen/language/freemarker.html
b/developers/code-gen/language/freemarker.html
index 9531d33..872be88 100644
--- a/developers/code-gen/language/freemarker.html
+++ b/developers/code-gen/language/freemarker.html
@@ -252,10 +252,32 @@
</ul>
</div>
<div class="paragraph">
-<p>A Freemarker-based output module, has to provide a list of
<code>Template</code> instances as well as provide a
<code>FreemarkerLanguageTemplateHelper</code> instance.</p>
+<p>A Freemarker-based output module, has to provide a set of
<code>Template</code> instances as well as provide a
<code>FreemarkerLanguageTemplateHelper</code> instance.</p>
</div>
<div class="paragraph">
-<p>What the <code>FreemarkerLanguageOutput</code> then does, is iterate over
all complex types provided by the protocol module, and then iterates over all
templates the current language defines.</p>
+<p>In general, we distinguish between these types of templates:</p>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p><code>Spec Templates</code> (Global output generated once per driver in
total)</p>
+</li>
+<li>
+<p><code>Complex Type Templates</code> (Generates output for a complex
type)</p>
+</li>
+<li>
+<p><code>Enum Templates</code> (Generates output for enum types)</p>
+</li>
+<li>
+<p><code>DataIO Templates</code> (Generates output for reading and writing
PlcValues, which are our PLC4X form of presenting input and output data to our
users)</p>
+</li>
+</ul>
+</div>
+<div class="paragraph">
+<p>For each of these, the developer can provide a list of templates, which
then can generate multiple files per type (Which is important for languages
such as <code>C</code> where for every type we need to generate a <code>Header
file (.h)</code> and an <code>Implementation (.c)</code>)</p>
+</div>
+<div class="paragraph">
+<p>What the <code>FreemarkerLanguageOutput</code> then does, is iterate over
all types provided by the protocol module, and then iterate over all templates
the current language defines.</p>
</div>
<div class="paragraph">
<p>The only convention used in this utility, is that the first line of output
a template generates will be treated as the path relative to the base output
directory.</p>
@@ -264,7 +286,7 @@
<p>It will automatically create all needed intermediate directories and
generate the rest of the input to the file specified by the first line.</p>
</div>
<div class="paragraph">
-<p>If this line is empty, the output is skipped for this instance.</p>
+<p>If this line is empty, the output is skipped for this type.</p>
</div>
<div class="sect2">
<h3 id="example_java_output">Example <code>Java</code> output</h3>
@@ -272,50 +294,120 @@
<div class="content">
<pre>package org.apache.plc4x.language.java;
-import freemarker.template.*;
+import com.google.googlejavaformat.java.Formatter;
+import com.google.googlejavaformat.java.FormatterException;
+import freemarker.template.Configuration;
+import freemarker.template.Template;
+import org.apache.commons.io.FileUtils;
import
org.apache.plc4x.plugins.codegenerator.protocol.freemarker.FreemarkerLanguageOutput;
import
org.apache.plc4x.plugins.codegenerator.protocol.freemarker.FreemarkerLanguageTemplateHelper;
-import
org.apache.plc4x.plugins.codegenerator.types.definitions.ComplexTypeDefinition;
+import org.apache.plc4x.plugins.codegenerator.types.definitions.TypeDefinition;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
-import java.io.*;
+import java.io.File;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
import java.util.*;
public class JavaLanguageOutput extends FreemarkerLanguageOutput {
+ private static final Logger LOGGER =
LoggerFactory.getLogger(JavaLanguageOutput.class);
+
+ private final Formatter formatter = new Formatter();
+
@Override
public String getName() {
return "Java";
}
- protected List<Template> getTemplates(Configuration
freemarkerConfiguration) throws IOException {
+ @Override
+ public Set<String> supportedOptions() {
+ return Collections.singleton("package");
+ }
+
+ @Override
+ public List<String> supportedOutputFlavors() {
+ return Arrays.asList("read-write", "read-only", "passive");
+ }
+
+ @Override
+ protected List<Template> getSpecTemplates(Configuration
freemarkerConfiguration) {
+ return Collections.emptyList();
+ }
+
+ @Override
+ protected List<Template> getComplexTypeTemplates(Configuration
freemarkerConfiguration) throws IOException {
return Arrays.asList(
-
freemarkerConfiguration.getTemplate("templates/java/pojo-template.ftlh"),
-
freemarkerConfiguration.getTemplate("templates/java/io-template.ftlh"));
+
freemarkerConfiguration.getTemplate("templates/java/pojo-template.java.ftlh"),
+
freemarkerConfiguration.getTemplate("templates/java/io-template.java.ftlh"));
}
- protected FreemarkerLanguageTemplateHelper getHelper(Map<String,
ComplexTypeDefinition> types) {
- return new JavaLanguageTemplateHelper(types);
+ @Override
+ protected List<Template> getEnumTypeTemplates(Configuration
freemarkerConfiguration) throws IOException {
+ return Collections.singletonList(
+
freemarkerConfiguration.getTemplate("templates/java/enum-template.java.ftlh"));
}
+ @Override
+ protected List<Template> getDataIoTemplates(Configuration
freemarkerConfiguration) throws IOException {
+ return Collections.singletonList(
+
freemarkerConfiguration.getTemplate("templates/java/data-io-template.java.ftlh"));
+ }
+
+ @Override
+ protected FreemarkerLanguageTemplateHelper getHelper(TypeDefinition
thisType, String protocolName, String flavorName, Map<String,
TypeDefinition> types,
+ Map<String,
String> options) {
+ return new JavaLanguageTemplateHelper(thisType, protocolName,
flavorName, types, options);
+ }
+
+ @Override
+ protected void postProcessTemplateOutput(File outputFile) {
+ try {
+ FileUtils.writeStringToFile(
+ outputFile,
+ formatter.formatSourceAndFixImports(
+ FileUtils.readFileToString(outputFile,
StandardCharsets.UTF_8)
+ ),
+ StandardCharsets.UTF_8
+ );
+ } catch (IOException | FormatterException e) {
+ LOGGER.error("Error formatting {}", outputFile, e);
+ }
+ }
}</pre>
</div>
</div>
<div class="paragraph">
-<p>As you can see, this output generates up to two files for every complex
type.</p>
+<p>The <code>getName</code> method returns <code>Java</code>, this is what
needs to be defined in the <code>plc4x-maven-plugin</code> configuration in the
<code>language</code> option in order to select this output format.</p>
</div>
<div class="paragraph">
-<p>This is one file, providing the code for the POJO itself.
-The second one generates the IO-component, which contains all the code for
parsing and serializing the corresponding POJO.</p>
+<p><code>supportedOptions</code> tells the plugin which <code>option</code>
tags this code-generation output supports. In case of the <code>Java</code>
output, this is only the <code>package</code> option, which defines the package
name of the generated output.</p>
</div>
<div class="paragraph">
-<p>In other languages, for example <code>C++</code> it’s possible to use
a third one to generate Header files or for <code>Python</code> perhaps only
one in total.</p>
+<p>With <code>supportedOutputFlavors</code> we tell the user, that in general
we support the three options: <code>read-write</code>, <code>read-only</code>
and <code>passive</code> as valid inputs for the <code>outputFlavor</code>
config option of the code-generation plugin.</p>
+</div>
+<div class="paragraph">
+<p>In this case Java doesn’t require any global files being generated
for java, so we simply return an empty collection.</p>
+</div>
+<div class="paragraph">
+<p>For complex types, we currently use two templates (however this will soon
be reduced to one). So for every complex type in a protocol definition, the
templates: <code>templates/java/pojo-template.java.ftlh</code> and
<code>templates/java/io-template.java.ftlh</code> will be executed.</p>
+</div>
+<div class="paragraph">
+<p>In case of enum types, only one template is being used.</p>
+</div>
+<div class="paragraph">
+<p>Same as for data-io.</p>
+</div>
+<div class="paragraph">
+<p>The next important method is the <code>getHelper</code> method, which
returns an object, that is passed to the templates with the name
<code>helper</code>. As mentioned before, a lot of operations would be too
complex to implement in pure Freemarker code, so with these helpers every
language can provide a helper utility for handling the complex operations.</p>
</div>
<div class="paragraph">
<p>Here an example for a part of a template for generating Java POJOs:</p>
</div>
<div class="literalblock">
<div class="content">
-<pre>${packageName?replace(".", "/")}/${typeName}.java
+<pre>${helper.packageName(protocolName, languageName,
outputFlavor)?replace(".", "/")}/${type.name}.java
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
@@ -334,11 +426,13 @@ The second one generates the IO-component, which contains
all the code for parsi
* specific language governing permissions and limitations
* under the License.
*/
-package ${packageName};
+package ${helper.packageName(protocolName, languageName, outputFlavor)};
-import org.apache.plc4x.java.utils.SizeAware;
+... imports ...
-public<#if type.abstract> abstract</#if> class ${typeName}<#if
type.parentType??> extends ${type.parentType.name}</#if> implements
SizeAware {
+// Code generated by code-generation. DO NOT EDIT.
+
+public<#if type.isDiscriminatedParentTypeDefinition()>
abstract</#if> class ${type.name}<#if type.parentType??> extends
${type.parentType.name}</#if> implements Message {
... SNIP ...
@@ -346,8 +440,10 @@ public<#if type.abstract> abstract</#if> class
${typeName}<#if ty
</div>
</div>
<div class="paragraph">
-<p>So as you can see, the first line will generate the file-path of the to be
generated output.
-Which is followed</p>
+<p>So as you can see, the first line will generate the file-path of the to be
generated output.</p>
+</div>
+<div class="paragraph">
+<p>As when creating more and more outputs for different languages, we have
realized, that a lot of the code needed in the <code>Helper</code> utility
repeats, we therefore introduced a so-called
<code>BaseFreemarkerLanguageTemplateHelper</code> which contains a lot of
stuff, that is important when generating new language output.</p>
</div>
</div>
</div>
