Well I had sufficient problems with it that I should re-verify if it works with Java 17. It may be that the java version thing was a non-issue that other things were interacting with it.
On Mon, Jun 5, 2023 at 4:45 PM Shane Dell <shaned...@apache.org> wrote: > The extension does not depend on Java 11 it should work fine with Java 17. > I remember back > porting to make sure we supported Java 8 also. Other than relying on the > system Java, I am not > sure how on else we can really on Java. I know Davin mentioned packaging a > JVM or JDK or > something like that but is that very common practice to do? The only time > I ever heard of doing > that was for a language server so was wondering if its necessary or might > be overkill. I agree > with the needing to fix the issue with the environment when it comes to > shell and think what you > posted will help us do that. However, when it comes to Java I think if > there is an issue with Java > 17 we should figure that out first. Then maybe put it out for a vote or > something about bundling > a Java version with the extension and then letting the user specify a > different path to Java or > something like that? > > > On 2023/05/31 23:02:03 Mike Beckerle wrote: > > So, I have the 1.3.0 RC2 vscode extension debugger working now for PCAP > w/ > > ethernetIP and I can set a breakpoint in ethernetIP.dfdl.xsd and I no > > longer get an NPE. > > > > Configuring this is really far too difficult, and I have to have the > schema > > working first with sbt in order to even hope to configure it. > > > > But I'll get back to that later. > > > > More basic issue is that I am unable to debug even dead-simple errors > with > > it. I clobbered the magic number in the first bytes of an icmp1.cap file. > > This fails an assert at line 97 of pcap.dfdl.xsd, which is the first > > element of simple type. > > > > I step through with the debugger, and nothing displays *anything* about > the > > assert failure. Execution ends. A little pop-up dialog says it wrote my > > infoset out, which it didn't. There's nothing showing that anything went > > wrong. All panels on the left are blank. > > > > If I modify the schema and make any mistake, then I get a blocking dialog > > showing the SDEs. This is basically intolerable, and is why I have to go > > back to sbt to debug this. You can't even go through the SDEs as you move > > the editor. It's a blocking dialog. > > > > So both for schema compilation, and runtime errors there simply has to > be a > > non-modal display of errors, probably on the left with variables, watch, > > call-stack, and breakpoints. But a quasi-edit-buffer containing them > would > > also be fine, or even just output them to the terminal window. > > > > The rest of this email is about the hoops I had to jump through to get > this > > to work at all. Arguably, these issues are about working with a > > multi-component schema, and we could define our objectives to simpler > > schemas than this, but that puts it in the realm of toy-only schemas. The > > example is just PCAP + ethernetIP, which is not an extreme example. > > > > The existing shell environment of the terminal definitely causes > problems. > > But just exiting that so that the debugger starts a new terminal doesn't > > fix it, as that inherits the environment from where vscode was started. > It > > starts a new login shell unless you take steps to prevent that. > > > > A big trip-up in usability is that the wizard that creates the > launch.json, > > doesn't actually let you populate the daffodilDebugClaspath. You have to > > edit that by hand, and nothing verifies it, so any typo and you get > tedious > > failures that are hard to figure out. You just get an SDE - if you are > > lucky - via a blocking dialog. > > > > The wizard should help you put directories and corresponding jars on the > > daffodilDebugClasspath, and it should not let you put a non-existing > > directory or jar file on it. Ideally, at launch time the > > daffodilDebugClasspath should be checked, just in case something on it no > > longer exists. > > > > If the debugger happens to find a file on the daffodilDebugClasspath in a > > jar, vs. in a regular directory (maybe you have a typo in the regular > > directory path), so it passes that and hits the jar, then setting a > > breakpoint causes a NPE. Something requires a "hierarchical URL" which a > > jar-file URL is not. This is the bug that really has to be fixed so that > > you can set breakpoints regardless of whether the source DFDL schema file > > was in a jar or not. > > > > Note: at Owl we have DFDL schemas where the starting root element of the > > schema isn't even part of the schema, it's part of a component schema > that > > is in a jar on the classpath. This happens when you have an > > envelope-payload idiom. The schema that combines the envelope and payload > > together contains only the glue part that connects the two together. The > > root element is the envelope's root element, and that schema is in a jar > > file. > > > > I did not see any feature in launch.json for specifying the > name/namespace > > of the root element. There is just the "program" element, which must be > to > > a file in the current schema. That needs to be able to be specified just > as > > element name + namespace. > > > > The jars have to be on the daffodilDebugClasspath because of the plugins > > that daffodil must find in them. That's understandable, the problem is > > having to also have a regular file location for any schema you want to be > > able to put a breakpoint in and it must be before the corresponding jar. > > > > That all said, here are the scripts and settings I used to get it to > work: > > > > First, I have this script I use to launch vscode in a minimal/empty > > environment. This technique is useful to > > determine if you are depending on the environment in any way: > > > > #! /bin/bash > > if [ "$1" = "emptyEnv" ] # is it the special first arg > > then > > shift # discard special first arg, and continue with the rest of the > > script > > else > > CMD=$0 > > exec -c $CMD "emptyEnv" $* # exec with an empty environment, special > > first arg > > fi > > # the environment is empty when we get here > > # just add env vars we need to get vscode to work > > export LANG=en_US.UTF-8 > > export HOME=/home/mbeckerle > > export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # java 11 required > > export PATH=$JAVA_HOME/bin:$PATH > > export DISPLAY=:0 > > export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus # vscode > spits > > lots of errors without this > > exec /usr/bin/code > > # end > > > > Then, within ~/.config/Code/User/settings.json, I had to create these > > settings for terminal: > > > > "terminal.integrated.inheritEnv": false, > > "terminal.integrated.profiles.linux": { > > "Custom Init": { > > "path": "/bin/bash", > > "args": [ > > "--norc" > > ] > > } > > }, > > "terminal.integrated.defaultProfile.linux": "Custom Init" > > > > That instructs vscode not to create a login shell for the terminal. The > > --norc option means it does not run the .bashrc file so that the java > > version specified in the launch script sticks. > > > > I am not sure the inheritEnv setting actually does anything. That alone > > sounds like the right thing, but that alone didn't solve the problem. > > You still have everything in the login shell environment where vscode was > > launched. Hence the need for the environment-emptying launch script. > > > > With those two things, a terminal created by vscode does not inherit > > everything my typical shell has. For example there is no daffodil on the > > path, no DAFFODIL_CLASSPATH env var, etc. > > > > Given the above if I just create a terminal with Terminal > new Terminal: > > The environment is relatively simple: > > > > bash-5.0$ env > > COLORTERM=truecolor > > TERM_PROGRAM_VERSION=1.78.2 > > JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 > > NO_AT_BRIDGE=1 > > PWD=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP > > VSCODE_GIT_ASKPASS_NODE=/usr/share/code/code > > HOME=/home/mbeckerle > > LANG=en_US.UTF-8 > > GIT_ASKPASS=/usr/share/code/resources/app/extensions/git/dist/askpass.sh > > CHROME_DESKTOP=code-url-handler.desktop > > VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node > > TERM=xterm-256color > > VSCODE_GIT_IPC_HANDLE=/tmp/vscode-git-56a269c0f5.sock > > DISPLAY=:0 > > SHLVL=1 > > > VSCODE_GIT_ASKPASS_MAIN=/usr/share/code/resources/app/extensions/git/dist/askpass-main.js > > GDK_BACKEND=x11 > > > PATH=/usr/lib/jvm/java-11-openjdk-amd64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > > ORIGINAL_XDG_CURRENT_DESKTOP=undefined > > DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus > > TERM_PROGRAM=vscode > > _=/usr/bin/env > > > > If I run daffodil, then c-C in the terminal it creates, then the > > environment has this added environment variable only. > > > > > DAFFODIL_DEBUG_CLASSPATH=/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/../ethernetIP/src/main/resources:/home/mbeckerle/dataiti/opensource/DFDLSchemas/PCAP/lib_managed/jars/com.owlcyberdefense/dfdl-ethernetip/dfdl-ethernetip-1.2.0.jar > > > > This works. My launch.json is attached. > > > > > > > > > > On Fri, May 26, 2023 at 2:06 PM Shane Dell <shaned...@apache.org> wrote: > > > > > Mike, > > > > > > The VSCode Extension should be using the daffodil command that is > bundled > > > with the > > > extension. The issue might still be with that shell env variable you > have > > > for > > > DAFFODIL_CLASSPATH, but this might only be an issue with a terminal > that > > > is already opened > > > with the name "daffodil-debugger". If you close that terminal, the next > > > you time you run it should > > > create a new terminal with the env set properly. If you could try this > and > > > see if it works how you > > > expect, that would help us single out this issue out to an already > active > > > terminal. > > > > > > Thank you, > > > > > > Shane Dell > > > > > > > > > On 2023/05/25 23:59:53 Mike Beckerle wrote: > > > > There are so many environment interactions.... > > > > > > > > So I have discovered that the VSCode extension just seems to run > whatever > > > > daffodil command is on the path. It's not isolated at all from the > shell > > > > environment. > > > > > > > > I found this out because I have a customized daffodil shell script > that > > > > setups a bunch of DAFFODIL_CLASSPATH contents, and I couldn't figure > out > > > > why those were part of the VSCode environment. > > > > > > > > Shouldn't vscode run exactly the daffodil bundled with it, unless you > > > > explicitly configure it not to? > > > > > > > > That's why it is finding the ethernetIP.dfdl.xsd file in the jar, > not in > > > > the file system. Because my daffodil command sets up lots of jars on > the > > > > DAFFODIL_CLASSPATH. > > > > > > > > It also depends on Java 11, breaks with Java 17. I also don't think > this > > > > should be inherited from the environment. > > > > > > > > I really have a hard time using, or even testing the VSCode > extension if > > > I > > > > have to scrub my shell environment of everything in it in order to > make > > > the > > > > vscode extension work. > > > > > > > > But it is this kind of problem that caused people to invent docker > > > > containers. > > > > > > > > I am going to make a launcher script for VSCode that empties the > > > > environment, then supplies only what is absolutely necessary, with no > > > > outside environment assumptions. > > > > > > > > I will try to do this Friday if possible. > > > > > > > > > > > > > > > > On Wed, May 24, 2023 at 2:50 PM Shane Dell <shaned...@apache.org> > wrote: > > > > > > > > > Mike, > > > > > > > > > > I am not sure why you would be seeing this. I am able to set the > > > > > breakpoint where you > > > > > mentioned without any issues. Can you try removing the daffodil-dap > > > > > folder? This should be > > > > > somewhere like: > > > > > /home/$USER/.local/share/daffodil-dap > > > > > The issue I am thinking is possibly a bug with the rc1 debugger > that > > > may > > > > > not be present on rc2. > > > > > But if the folder daffodil-dap/daffodil-debugger-3.4.0-1.3.0/ > already > > > > > exists from rc1 then rc2 will > > > > > not remake it. So in order to remake it you have to manually remove > > > this > > > > > folder. Let me know if > > > > > this helps. > > > > > > > > > > Thank you, > > > > > > > > > > Shane Dell > > > > > > > > > > On 2023/05/24 15:15:33 Mike Beckerle wrote: > > > > > > I have been able to continue evaluating RC2. > > > > > > > > > > > > My vote is -1 > > > > > > > > > > > > My testing steps listed below. I am trying to debug PCAP. I have > a > > > broken > > > > > > icmp packet (bad magic number), and also just step through a > correct > > > icmp > > > > > > packet. > > > > > > > > > > > > I am trying to debug icmp.badMagicNum.cap data file. > > > > > > > > > > > > I should get a failure of an assert because the magic number is > not > > > one > > > > > of > > > > > > the expected values. > > > > > > > > > > > > I'm just trying to single-step through it until it hits the > error. > > > > > > > > > > > > Once we get to the MagicNumber element, it doesn't position on > the > > > assert > > > > > > while evaluating that expression or even on the element carrying > the > > > > > assert. > > > > > > It jumps back to the top-level xs:schema position. > > > > > > > > > > > > I continue to click single step. Watching for the error to be > > > displayed > > > > > in > > > > > > the upper left under Variables > Parse > Diagnostics > Errors > which > > > is > > > > > > where I think it ought to go. > > > > > > > > > > > > At some point the failure is somehow detected as the debug > session > > > ends. > > > > > A > > > > > > dialog offers to open /tmp/infoset.xml for me, which I open and > it is > > > > > empty > > > > > > of course. > > > > > > > > > > > > Nothing displayed the error message from the assert, or even > > > indicated > > > > > what > > > > > > went wrong at all. Not the state display in the upper left, not > the > > > > > > terminal, nor debug console nor output. > > > > > > > > > > > > The upper left box with "Variables" is cleared at the end. It > should > > > > > > certainly not be cleared at the end of a run when final state > may be > > > of > > > > > > interest. At the start of a run, or manually, but not at the end > of a > > > > > run. > > > > > > > > > > > > Second attempt to use it: > > > > > > > > > > > > I create a second launch config this time for what should be a > > > successful > > > > > > parse of icmp1.cap > > > > > > > > > > > > As I single step, the position of the cursor on the schema is > jumping > > > > > > around all over the place back and forth between a containing > > > element and > > > > > > the child elements it contains. > > > > > > The infoset display does not get updated until long after the > > > display has > > > > > > moved on and is positioned on the next element. > > > > > > > > > > > > This jumping around gets completely out of hand once you step > into > > > the > > > > > > complexType "Ethernet" which is in the ethernetIP jar file. > > > > > > > > > > > > At that point, after each child of Ethernet, it jumps back to the > > > > > Ethernet > > > > > > element in the pcap.dfdl.xsd file, and then back into the > > > > > > ethernetIP.dfdl.xsd for the next child, then back to the > > > pcap.dfdl.xsd > > > > > > file, etc. > > > > > > > > > > > > All this jumping around basically makes stepping useless. So I > > > decide to > > > > > > put a breakpoint on the Version element inside the IPv4Header > element > > > > > > inside ethernetIP.dfdl.xsd. > > > > > > > > > > > > Just adding the breakpoint causes the debug session to end with > an > > > NPE in > > > > > > the dap setBreakpoint. > > > > > > > > > > > > ERROR o.a.d.d.d.DAPodil - unhandled error > > > > > > java.lang.NullPointerException: null > > > > > > at > > > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75) > > > > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69) > > > > > > at > > > > > > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279) > > > > > > at java.base/java.nio.file.Path.of(Path.java:147) > > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69) > > > > > > at > > > > > > > > > > > > > > > org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267) > > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281) > > > > > > at flatMap @ > > > > > > > > > > > > > > > org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at handleErrorWith @ > > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057) > > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ > > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929) > > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > 2023-05-24 11:09:10,717 [io-compute-1] DEBUG o.a.d.d.d.DAPodil - > > > > > whenDone: > > > > > > completed > > > > > > java.lang.NullPointerException > > > > > > at > > > > > java.base/sun.nio.fs.UnixPath.normalizeAndCheck(UnixPath.java:75) > > > > > > at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:69) > > > > > > at > > > > > > > java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279) > > > > > > at java.base/java.nio.file.Path.of(Path.java:147) > > > > > > at java.base/java.nio.file.Paths.get(Paths.java:69) > > > > > > at > > > > > > > > > > > > > > > org.apache.daffodil.debugger.dap.DAPodil.$anonfun$setBreakpoints$1(DAPodil.scala:267) > > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281) > > > > > > at flatMap @ > > > > > > > > > > > > > > > org.apache.daffodil.debugger.dap.DAPodil.setBreakpoints(DAPodil.scala:263) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at handleErrorWith @ > > > > > > fs2.Compiler$Target.handleErrorWith(Compiler.scala:161) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ fs2.Pull$.goEval$1(Pull.scala:1057) > > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > at flatMap @ > > > fs2.Pull$.fs2$Pull$$interruptGuard$1(Pull.scala:929) > > > > > > at get @ fs2.internal.Scope.openScope(Scope.scala:281) > > > > > > at flatMap @ > fs2.Compiler$Target.flatMap(Compiler.scala:163) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, May 23, 2023 at 5:19 PM Mike Beckerle < > mbecke...@apache.org> > > > > > wrote: > > > > > > > > > > > > > I really am trying to eval the RC2. > > > > > > > > > > > > > > Right now my VMWare Linux system has been broken by recent > updates > > > to > > > > > the > > > > > > > MS Windows host environment. > > > > > > > > > > > > > > I don't exactly understand this, but the VSCode front end can > no > > > longer > > > > > > > connect to the server at localhost:4711. I get ECONNREFUSED. > This > > > > > started > > > > > > > when a bunch of MS-Windows updates were done on the host OS. I > run > > > > > VSCode > > > > > > > on the linux client machine, which lives behind a NAT firewall > > > created > > > > > by > > > > > > > VMWare workstation - so I really don't get how these could be > > > > > interacting > > > > > > > this way, but... there it is. > > > > > > > > > > > > > > I will switch back to my standalone linux PC if this isn't > resolved > > > > > today. > > > > > > > > > > > > > > > > > > > > > On Tue, May 16, 2023 at 1:24 PM Shane Dell < > shaned...@apache.org> > > > > > wrote: > > > > > > > > > > > > > >> Hello all,I'd like to call a vote to release Apache Daffodil > VS > > > Code > > > > > > >> 1.3.0-rc2. > > > > > > >> > > > > > > >> All distribution packages, including signatures, digests, etc. > > > can be > > > > > > >> found at: > > > > > > >> > > > > > > > > > https://dist.apache.org/repos/dist/dev/daffodil/daffodil-vscode/1.3.0-rc2 > > > > > > >> > > > > > > >> This release has been signed with PGP key > > > > > > >> 86DDE7B41291E380237934F007570D3ADC76D51B, corresponding > > > > > > >> to shaned...@apache.org, which is included in the KEYS file > here: > > > > > > >> https://downloads.apache.org/daffodil/KEYS > > > > > > >> > > > > > > >> The release candidate has been tagged in git with 1.3.0-rc2. > The > > > > > binaries > > > > > > >> > > > > > > >> For reference, here is a list of all closed GitHub issues > tagged > > > with > > > > > > >> 1.3.0: > > > > > > >> > https://github.com/apache/daffodil-vscode/milestone/4?closed=1 > > > > > > >> > > > > > > >> Please review and vote. The vote will be open for at least 72 > > > hours > > > > > > >> (Friday, 19 May 2023, 1:30pm EST). > > > > > > >> > > > > > > >> [ ] +1 approve > > > > > > >> [ ] +0 no opinion > > > > > > >> [ ] -1 disapprove (and reason why) > > > > > > >> > > > > > > >> Documentation for 1.3.0 can be found here > > > > > > >> Apache Daffodil Extension for Visual Studio Code v1.3.0 Wiki > > > > > > >> < > > > > > > >> > > > > > > > > > https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0 > > > > > > >> >. > > > > > > >> > > > > > > >> Please note one of the large features that was focused on in > > > 1.3.0 was > > > > > > >> improving the data editor. To open the data editor you will > need > > > to > > > > > open > > > > > > >> the > > > > > > >> command palette using: > > > > > > >> > > > > > > >> - Ctrl + Shift + P (windows/linux) > > > > > > >> - Command + Shift + P (mac) > > > > > > >> > > > > > > >> Then typing "data.edit", make sure "Daffodil Debug: Data > Editor" > > > is > > > > > > >> selected > > > > > > >> and then hitting enter. By default the data editor will run on > > > port > > > > > 9000 > > > > > > >> of > > > > > > >> your machine. However, if you want to use a different port > follow > > > > > steps > > > > > > >> here > > > > > > >> Data Editor Launch Settings > > > > > > >> < > > > > > > >> > > > > > > > > > https://github.com/apache/daffodil-vscode/wiki/Apache-Daffodil%E2%84%A2-Extension-for-Visual-Studio-Code:-v1.3.0#data-editor-launch-settings > > > > > > >> > > > > > > > >> > > > > > > >> Thank you, > > > > > > >> > > > > > > >> - Shane Dell > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > >