Sorry https://issues.apache.org/jira/browse/TINKERPOP-2883 may not be an issue for tinkerpop. We are using the neo4j-gremlin plugin with it. We push a repo to Ironbank with our Dockerfile and some other files required.
Their process pulls the zip files from https://dlcdn.apache.org/tinkerpop/3.6.2/apache-tinkerpop-gremlin-server-3.6.2-bin.zip https://dlcdn.apache.org/tinkerpop/3.6.2/apache-tinkerpop-gremlin-console-3.6.2-bin.zip and then runs this command RUN bin/gremlin-server.sh install org.apache.tinkerpop neo4j-gremlin 3.6.2 I assume it pulls in the neo4j-gremlin plugin which is where the netty is referenced. I'm going to modify the Dockerfile to not run that line and see what happens. Thanks for your help. Jim On Wed, Mar 1, 2023 at 1:59 PM Stephen Mallette <spmalle...@gmail.com> wrote: > I saw the JIRAs. Most of those seem like transient dependencies that are > being problematic. Like, this one: > > https://issues.apache.org/jira/browse/TINKERPOP-2883 > > refers to netty 3.x being a problem. But we specifically use netty 4.x: > > https://github.com/apache/tinkerpop/blob/3.5.5/pom.xml#L177 > > The only reference to 3.x that I can think of would come from maybe > hadoop-gremlin. we could probably make some effort to sort out these > transitive dependency issues, but i'm curious as to why direct dependencies > like hadoop, neo4j, etc. are a problem for IronBank. we don't package them > in Docker images (which i'd come to understand IronBank was interested in) > nor are they packaged in our binary zip distributions of Gremlin Server or > Console which you alluded to here: > > https://lists.apache.org/thread/vsmrms1sy62sd1z3b7mcmzc9kh4gq5tk > > So, why is the IronBank scanner picking up these issues? I guess something > still isn't connecting for me in what IronBank is doing. > > > > On Mon, Feb 27, 2023 at 5:10 PM Jim Foscue <jim.fos...@jdmsolutions.com> > wrote: > > > I've added some JIRA tickets with the Label Ironbank. > > > > They all involve updating various libraries as part of the build. > > > > Jim > > > > On Wed, Feb 22, 2023 at 11:59 AM Jim Foscue <jim.fos...@jdmsolutions.com > > > > wrote: > > > > > I can definitely add JIRA issues and share the findings. > > > > > > I’ll start with version 3.6.2. Could take a couple of days to get this > > > going in Ironbank but I’ll keep you updated. > > > > > > Jim > > > > > > > On Feb 22, 2023, at 11:23 AM, Stephen Mallette <spmalle...@gmail.com > > > > > wrote: > > > > > > > > ok - thanks for clarifying. i hope you'll feel free to issue > PRs/JIRAs > > > and > > > > just generally interact with the community to help us get TinkerPop > > > > IronBank-ready. it would be nice if you kept us apprised of releases > > that > > > > were accepted there. i assume we could add a link somewhere to the > > place > > > > TinkerPop is hosted in ironbank if there is such a direct link > publicly > > > > available and you could share it? > > > > > > > >> On Wed, Feb 22, 2023 at 11:59 AM Jim Foscue < > > > jim.fos...@jdmsolutions.com> > > > >> wrote: > > > >> > > > >> What you're saying makes sense to me. We can be the custodians of > the > > > >> Ironbank process/image since we have access. > > > >> > > > >> I'm not sure I understand your release process. The ironbank build > > > pulls > > > >> in the files from https://dlcdn.apache.org/tinkerpop/. > > > >> > > > >> For example > > > >> # List of resources to make available to the offline build context > > > >> resources: > > > >> - url: " > > > >> > > > >> > > > > > > https://dlcdn.apache.org/tinkerpop/3.6.1/apache-tinkerpop-gremlin-server-3.6.1-bin.zip > > > >> " > > > >> filename: "apache-tinkerpop-gremlin-server-3.6.1-bin.zip" > > > >> validation: > > > >> type: sha512 > > > >> value: > > > >> > > > >> > > > > > > 4930e79caeed0ea019bf52da9d78d3440a0e6805b3232763b774116c10e7ee38dbe5ac9bb3873acf318c65349d00b204d7ec89c60324eda9a4aaaa2c2998053b > > > >> - url: " > > > >> > > > >> > > > > > > https://dlcdn.apache.org/tinkerpop/3.6.1/apache-tinkerpop-gremlin-console-3.6.1-bin.zip > > > >> " > > > >> filename: "apache-tinkerpop-gremlin-console-3.6.1-bin.zip" > > > >> validation: > > > >> type: sha512 > > > >> value: > > > >> > > > >> > > > > > > 3516b601298427e78580665016f6d29bf5ad39bfeeafe9e4cd762bb7a9454d844e131df4935b4fa61fb2f9d749eb5f2fa4b56df2908a8067825a203423b958c5 > > > >> > > > >> So when do these get updated? Is it when there is a new release or > bug > > > >> fixes? Whenever these are updated is when we would get the fixes. > > > >> > > > >> Thoughts? > > > >> > > > >> Jim > > > >> > > > >> On Wed, Feb 22, 2023 at 9:15 AM Stephen Mallette < > > spmalle...@gmail.com> > > > >> wrote: > > > >> > > > >>> Unless I'm not understanding something there isn't much change to > > > process > > > >>> for TinkerPop. You spot a problem in IronBank and either submit a > PR > > to > > > >> fix > > > >>> or create a JIRA for someone else to fix if you don't have the > > ability > > > to > > > >>> do it yourself. It gets merged and presumably you can get your > image > > > into > > > >>> IronBank. > > > >>> > > > >>> What I don't understand is how you consume that fix. Let's say > > IronBank > > > >> had > > > >>> a problem in 3.6.2. TinkerPop fixes it. Now what? Do you have to > wait > > > >> until > > > >>> we release 3.6.3 to get that into IronBank and hope you don't hit > > more > > > >>> problems? Is there a way to test an image ahead of a TinkerPop > > > release? I > > > >>> guess I'm trying to confirm that there really isn't any change to > > > >> anything > > > >>> TinkerPop does in the normal process of reviewing pull request and > > > fixing > > > >>> security problems because it seems pretty normal/simple otherwise. > No > > > one > > > >>> at TinkerPop can release anything to IronBank so this is purely a > > > >>> third-party maintained convenience which you are handling for the > > wider > > > >>> community. Is that a fair depiction of what were looking at here? > > > >>> > > > >>> So, if there's nothing out of the ordinary to do from TinkerPop's > > side > > > >> then > > > >>> I guess you should feel free to let the PRs/JIRAs fly (or maybe > just > > > call > > > >>> attention to existing ones if dependabot is already on it). Perhaps > > you > > > >>> should ensure you comment on these that they are "IronBank" related > > so > > > >> that > > > >>> they perhaps get some visibility that way. > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> On Wed, Feb 22, 2023 at 9:48 AM Jim Foscue < > > > jim.fos...@jdmsolutions.com> > > > >>> wrote: > > > >>> > > > >>>> I would like to bring this discussion back up. > > > >>>> > > > >>>> Has anyone given this any more thought? My team is willing to > help > > > but > > > >>> we > > > >>>> have not participated in the development of any of tinkerpop so it > > > >> would > > > >>> be > > > >>>> a huge effort on our part to get involved in the development. We > > are > > > >>>> developers but not our main language is C#. > > > >>>> > > > >>>> Thanks > > > >>>> Jim Foscue > > > >>>> > > > >>>> On Wed, Dec 21, 2022 at 5:40 AM Stephen Mallette < > > > spmalle...@gmail.com > > > >>> > > > >>>> wrote: > > > >>>> > > > >>>>> From what I'm gathering, this doesn't sound as if it changes much > > of > > > >>> how > > > >>>>> things operate here or the release process. It sounds as though > Jim > > > >>>> simply > > > >>>>> needs to submit dependency related PRs to TinkerPop. They would > be > > > >>>> reviewed > > > >>>>> and merged in similar fashion to any PR that arrived. As these > > would > > > >> be > > > >>>>> security related PRs, they probably wouldn't generate > > > >>> objections/concerns > > > >>>>> and would merge quickly. > > > >>>>> > > > >>>>> There is still something I don't quite understand. 3.6.1 scanned > by > > > >>>>> IronBank shows problems. Jim submits PRs to TinkerPop to fix > those > > > >>>> issues. > > > >>>>> What version ultimately lands in IronBank? IronBank has to wait > > for a > > > >>>> 3.6.2 > > > >>>>> as 3.6.1 can't pass their scanners? Can IronBank scanners work > from > > > >>>>> SNAPSHOT? Ideally, IronBank would be working on 3.6.2-SNAPSHOT > > right > > > >>> now > > > >>>> as > > > >>>>> release is arriving within the next few weeks. Then you could > have > > a > > > >>>> clean, > > > >>>>> pre-tested artifact in 3.6.2 to push into IronBank when it > > releases. > > > >>> This > > > >>>>> way IronBank would align with DockerHub and the rest of the > > release. > > > >>>>> Putting some kind of patched version of 3.6.1 in IronBank seems > > like > > > >> it > > > >>>>> could cause confusion imo. > > > >>>>> > > > >>>>> On Tue, Dec 20, 2022 at 12:51 PM Jim Foscue < > > > >>> jim.fos...@jdmsolutions.com > > > >>>>> > > > >>>>> wrote: > > > >>>>> > > > >>>>>> I believe most of the critical/high vulnerabilities are a result > > of > > > >>>>>> outdated dependencies. > > > >>>>>> > > > >>>>>> Another thing to consider is that to participate as an active > > > >>> developer > > > >>>>> on > > > >>>>>> Ironbank the user must have a DoD CAC which allows them access > to > > > >> the > > > >>>>>> Ironbank repository and the vulnerabilities. I'm not sure if > > > >> anyone > > > >>>> here > > > >>>>>> has one but I do. I was able to get the process started with > the > > > >>> 3.6.1 > > > >>>>>> version and there are a number of critical and highs. Some of > the > > > >>>>> examples > > > >>>>>> are: > > > >>>>>> Apache Shiro before 1.5.3, when using Apache Shiro with Spring > > > >>> dynamic > > > >>>>>> controllers, a specially crafted request may cause an > > > >> authentication > > > >>>>>> bypass. > > > >>>>>> Neo4j through 3.4.18 (with the shell server enabled) exposes an > > RMI > > > >>>>> service > > > >>>>>> that arbitrarily deserializes Java objects, e.g., through > > > >>>>>> setSessionVariable. An attacker can abuse this for remote code > > > >>>> execution > > > >>>>>> because there are dependencies with exploitable gadget chains. > > > >>>>>> HttpObjectDecoder.java in Netty before 4.1.44 allows a > > > >> Content-Length > > > >>>>>> header to be accompanied by a second Content-Length header, or > by > > a > > > >>>>>> Transfer-Encoding header. > > > >>>>>> > > > >>>>>> Looks like the Apache Shiro library is one of the problem spots. > > > >>>>>> > > > >>>>>> I do agree with your assessment that including a scanner in the > > > >>> normal > > > >>>>>> build process would help a lot. > > > >>>>>> > > > >>>>>> Jim > > > >>>>>> > > > >>>>>> > > > >>>>>> On Tue, Dec 20, 2022 at 3:08 AM Florian Hockmann < > > > >>>> f...@florian-hockmann.de > > > >>>>>> > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>>> It would be good to know what kind of vulnerabilities Ironbank > > > >> can > > > >>>>> find. > > > >>>>>>> Most of such scanners I know simply use a database of known > > > >>>>>> vulnerabilities > > > >>>>>>> that include the libraries and their versions that are > affected. > > > >>> The > > > >>>>>>> Log4Shell vulnerability from last year is a good example that > > > >> could > > > >>>> be > > > >>>>>>> found by such a scanner. > > > >>>>>>> > > > >>>>>>> In general I think that we should ideally try to include such a > > > >>>> scanner > > > >>>>>>> into our usual dev process so we can find those vulnerabilities > > > >>> early > > > >>>>> and > > > >>>>>>> then fix them. Fixing them will probably most of the time just > > > >> mean > > > >>>>>>> updating a dependency. > > > >>>>>>> If we just add Ironbank as an additional release channel behind > > > >> our > > > >>>>> usual > > > >>>>>>> release process, then we just get notified about > vulnerabilities > > > >>>> after > > > >>>>> we > > > >>>>>>> just have released a version with these vulnerabilities. I > think > > > >>> that > > > >>>>>> would > > > >>>>>>> only lead to additional pain for us as we would have to fix > them > > > >>> and > > > >>>>> then > > > >>>>>>> start another release process to perform a release without the > > > >>>>>>> vulnerability. > > > >>>>>>> > > > >>>>>>> GitHub already offers such a vulnerability scanner that works > > > >>>> together > > > >>>>>>> with Dependabot. This means that Dependabot will create a PR to > > > >>>> update > > > >>>>> a > > > >>>>>>> dependency if that contains a known vulnerability. These > security > > > >>>>> related > > > >>>>>>> PRs will be marked as such, but in a way that is only visible > to > > > >>>>>>> maintainers of the repo. So, we can prioritize these security > > > >>> related > > > >>>>> PRs > > > >>>>>>> over the usual Dependabot PRs that sometimes overwhelm us due > to > > > >>> the > > > >>>>> high > > > >>>>>>> number of dependencies we have. > > > >>>>>>> This improves the security of TinkerPop in general and I think > > > >> that > > > >>>> it > > > >>>>>>> also improves our dev process as we currently get tickets for > > > >> such > > > >>>>>> security > > > >>>>>>> problems manually created by users who probably already execute > > > >>> such > > > >>>>>>> scanners on our JARs / Docker images (two recent examples: > > > >>>>>>> TINKERPOP-2843[1] and TINKERPOP-2826[2]) and then they are > > > >> usually > > > >>>> also > > > >>>>>>> addressed by someone who manually updates the dependency. > > > >>>>>>> If the Ironbank scanner works in a similar way, then this would > > > >>> also > > > >>>>> make > > > >>>>>>> it a lot easier to publish our images to the Ironbank registry > as > > > >>>> that > > > >>>>>>> should ideally not uncover any new vulnerabilities since they > > > >> have > > > >>>>>> already > > > >>>>>>> been found and fixed by our own process (like GitHub security > > > >>>> scanning > > > >>>>> + > > > >>>>>>> Dependabot). > > > >>>>>>> > > > >>>>>>> By the way, I think this is the documentation of Ironbank: > > > >>>>>>> https://p1.dso.mil/products/iron-bank > > > >>>>>>> And here is a list of available Docker images: > > > >>>>>> https://repo1.dso.mil/dsop > > > >>>>>>> Unfortunately, the repos seem to be private so we can't see > much, > > > >>> but > > > >>>>> the > > > >>>>>>> list at least gives an idea of what kind of projects are > already > > > >>>>>> available > > > >>>>>>> there. > > > >>>>>>> > > > >>>>>>> [1]: https://issues.apache.org/jira/browse/TINKERPOP-2843 > > > >>>>>>> [2]: https://issues.apache.org/jira/browse/TINKERPOP-2826 > > > >>>>>>> > > > >>>>>>> -----Ursprüngliche Nachricht----- > > > >>>>>>> Von: Jim Foscue <jim.fos...@jdmsolutions.com> > > > >>>>>>> Gesendet: Montag, 19. Dezember 2022 17:10 > > > >>>>>>> An: dev@tinkerpop.apache.org > > > >>>>>>> Betreff: Re: [DISCUSS] > > > >>>>>>> > > > >>>>>>> Yes, Ironbank is an approved DoD dockerhub. There is no > > > >>> requirement > > > >>>>> for > > > >>>>>>> the releases on Ironbank to be in sync with the releases in > > > >>>>>>> Apache/Tinkerpop. The major requirement for Ironbank is for > > > >> there > > > >>>> not > > > >>>>> to > > > >>>>>>> be any critical/high vulnerabilities in the docker image. If > > > >> there > > > >>>> are > > > >>>>>>> vulnerabilities that "pop" up they should be addressed quickly. > > > >>>>>>> > > > >>>>>>> My team can help with that process but it would be hard for us > to > > > >>> fix > > > >>>>> the > > > >>>>>>> vulnerabilities without knowing the code. > > > >>>>>>> > > > >>>>>>> Jim > > > >>>>>>> > > > >>>>>>> On Mon, Dec 19, 2022 at 5:36 AM Stephen Mallette < > > > >>>> spmalle...@gmail.com > > > >>>>>> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> Hi Jim, thanks for bringing this discussion here. > > > >>>>>>>> > > > >>>>>>>> Generally speaking, it sounds like a reasonable idea to open > > > >>>>> TinkerPop > > > >>>>>>>> opportunities to the DoD environment. Would it be fair to call > > > >>>>>>>> IronBank a DoD approved DockerHub? Would the idea be that > > > >>> TinkerPop > > > >>>>>>>> would release to IronBank in the same cadence as is done with > > > >> the > > > >>>>>>>> standard release schedule that publishes to DockerHub? > > > >>>>>>>> > > > >>>>>>>> If so, I think a concern would be that supporting IronBank > > > >>> creates > > > >>>> a > > > >>>>>>>> hurdle to releasing that may be high and thus delay releases > > > >> that > > > >>>>>>>> might otherwise be acceptable outside of that space. I don't > > > >> know > > > >>>> if > > > >>>>>>>> it makes sense, but perhaps releasing to IronBank should be > > > >>>> decoupled > > > >>>>>>> from standard releases? > > > >>>>>>>> Like 3.5.5 would go out the door and then a separate process > > > >>> would > > > >>>> go > > > >>>>>>>> under way to harden 3.5.5 to meet IronBank needs? Of course, > > > >> then > > > >>>> the > > > >>>>>>>> 3.5.5 in IronBank isn't necessarily the 3.5.5 in DockerHub so > > > >>> that > > > >>>>>>>> could be problematic. Do you happen to know how other major > > > >> open > > > >>>>>>>> source projects do this? > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> On Wed, Dec 14, 2022 at 4:51 PM Jim Foscue > > > >>>>>>>> <jim.fos...@jdmsolutions.com> > > > >>>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> Ironbank discussion > > > >>>>>>>>> > > > >>>>>>>>> I would like to suggest making tinkerpop/gremlin available on > > > >>> the > > > >>>>>>>>> DoD Ironbank docker registry (https://ironbank.dso.mil). > > > >>>>>>>>> > > > >>>>>>>>> Ironbank is a DoD docker registry that maintains publicly > > > >>>> available > > > >>>>>>>> docker > > > >>>>>>>>> container images that have been "hardened' and given the ok > > > >> to > > > >>>> use > > > >>>>>>>>> on DoD contracts. The process of hardening an image involves > > > >>>>>>>>> running the docker container image through a vulnerability > > > >>>> scanning > > > >>>>>>>>> process. Any critical > > > >>>>>>>> and > > > >>>>>>>>> high vulnerabilities have to be addressed before getting the > > > >>>> docker > > > >>>>>>>>> container image approved for DoD use. > > > >>>>>>>>> > > > >>>>>>>>> My team has been using the publicly available tinkerpop/graph > > > >>> on > > > >>>> a > > > >>>>>>>>> DoD contract that is a prototype. But to continue using it > > > >> in > > > >>> a > > > >>>>>>>>> more > > > >>>>>>>> official > > > >>>>>>>>> process we have to have it approved by Ironbank. > > > >>>>>>>>> > > > >>>>>>>>> Therefore, I am trying to get the tinkerpop developer > > > >>> community > > > >>>> to > > > >>>>>>>>> get onboard to push this to Ironbank so that we and other DoD > > > >>>>>>>>> contracts can > > > >>>>>>>> use > > > >>>>>>>>> it. > > > >>>>>>>>> > > > >>>>>>>>> We have access to the Ironbank system (which requires a US > > > >>>>>>>>> government > > > >>>>>>>> CAC) > > > >>>>>>>>> and are willing to help guide the process. > > > >>>>>>>>> > > > >>>>>>>>> Please consider this request and provide any feedback to me > > > >> at > > > >>>> your > > > >>>>>>>>> convience. > > > >>>>>>>>> > > > >>>>>>>>> Regards, > > > >>>>>>>>> > > > >>>>>>>>> Jim Foscue > > > >>>>>>>>> JDM Solutions > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> -- > > > >>>>>>>>> *Jim Foscue*, Software Engineer > > > >>>>>>>>> *JDM Solutions* > > > >>>>>>>>> 256.694.9387 > > > >>>>>>>>> jim.fos...@jdmsolutions.com < > > > >> michele.woodr...@jdmsolutions.com > > > >>>> > > > >>>>>>>>> > > > >>>>>>>>> -- > > > >>>>>>>>> *This e-mail and any attachements are for the intended > > > >>>> recipient(s) > > > >>>>>>>>> only and may contain information proprietary or private to > > > >> JDM > > > >>>>>>> Solutions, LLC. > > > >>>>>>>>> If you are not the intended recipient **please delete without > > > >>>>>>>>> copying and kindly advise the sender by e-mail of the mistake > > > >>> in > > > >>>>>>>>> delivery. This > > > >>>>>>>> email > > > >>>>>>>>> may be personal in communication from the sender and as such > > > >>> does > > > >>>>>>>>> not represent the views of the company.* > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>>> -- > > > >>>>>>> *This e-mail and any attachements are for the intended > > > >> recipient(s) > > > >>>>> only > > > >>>>>>> and may contain information proprietary or private to JDM > > > >>> Solutions, > > > >>>>> LLC. > > > >>>>>>> If you are not the intended recipient **please delete without > > > >>> copying > > > >>>>> and > > > >>>>>>> kindly advise the sender by e-mail of the mistake in delivery. > > > >>> This > > > >>>>>> email > > > >>>>>>> may be personal in communication from the sender and as such > does > > > >>> not > > > >>>>>>> represent the views of the company.* > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>>>> -- > > > >>>>>> *This e-mail and any attachements are for the intended > > recipient(s) > > > >>>> only > > > >>>>>> and may contain information proprietary or private to JDM > > > >> Solutions, > > > >>>> LLC. > > > >>>>>> If you are not the intended recipient **please delete without > > > >> copying > > > >>>> and > > > >>>>>> kindly advise the sender by e-mail of the mistake in delivery. > > > >> This > > > >>>>> email > > > >>>>>> may be personal in communication from the sender and as such > does > > > >> not > > > >>>>>> represent the views of the company.* > > > >>>>>> > > > >>>>> > > > >>>> > > > >>>> -- > > > >>>> *This e-mail and any attachments are for the intended recipient(s) > > > only > > > >>>> and > > > >>>> may contain information proprietary or private to JDM Solutions, > > LLC. > > > >> If > > > >>>> you are not the intended recipient **please delete without copying > > and > > > >>>> kindly advise the sender by e-mail of the mistake in delivery. > This > > > >>> email > > > >>>> may be personal in communication from the sender and as such does > > not > > > >>>> represent the views of the company.* > > > >>>> > > > >>> > > > >> > > > >> -- > > > >> *This e-mail and any attachments are for the intended recipient(s) > > only > > > >> and > > > >> may contain information proprietary or private to JDM Solutions, > LLC. > > > If > > > >> you are not the intended recipient **please delete without copying > and > > > >> kindly advise the sender by e-mail of the mistake in delivery. This > > > email > > > >> may be personal in communication from the sender and as such does > not > > > >> represent the views of the company.* > > > >> > > > > > > > -- > > *This e-mail and any attachments are for the intended recipient(s) only > > and > > may contain information proprietary or private to JDM Solutions, LLC. If > > you are not the intended recipient **please delete without copying and > > kindly advise the sender by e-mail of the mistake in delivery. This > email > > may be personal in communication from the sender and as such does not > > represent the views of the company.* > > > -- *This e-mail and any attachments are for the intended recipient(s) only and may contain information proprietary or private to JDM Solutions, LLC. If you are not the intended recipient **please delete without copying and kindly advise the sender by e-mail of the mistake in delivery. This email may be personal in communication from the sender and as such does not represent the views of the company.*
